Source-linked AI summary
Coarse-to-Fine Decoding for Neural Semantic Parsing
Li Dong, Mirella Lapata
TL;DR
Semantic parsing must map natural-language expressions to structured meaning representations. The paper decomposes decoding into sketch generation followed by detail filling, using the sketch to guide and constrain output. Across four datasets, coarse-to-fine decoding improves performance and remains competitive despite relatively simple decoders.
Problem
The paper addresses learning semantic parsers that map natural-language expressions to structured meaning representations.
Method
The model first generates an abstract sketch omitting low-level details, then fills them in using both the natural-language input and sketch.
Results
Coarse-to-fine decoding improves performance across tasks and achieves competitive results compared with previous systems despite relatively simple decoders.
Takeaways & Limitations
A shared sketch-based framework can adapt to different domains and meaning representations, including logical forms, source code, and SQL queries.
Takeaways & Limitations
For DJANGO, the method relies on a copying mechanism to handle out-of-vocabulary tokens such as variable names and numbers.
Abstract
from arXiv · showhide
Semantic parsing aims at mapping natural language utterances into structured meaning representations. In this work, we propose a structure-aware neural architecture which decomposes the semantic parsing process into two stages. Given an input utterance, we first generate a rough sketch of its meaning, where low-level information (such as variable names and arguments) is glossed over. Then, we fill in missing details by taking into account the natural language input and the sketch itself. Experimental results on four datasets characteristic of different domains and meaning representations show that our approach consistently improves performance, achieving competitive results despite the use of relatively simple decoders.
1 Introduction
The paper proposes a two-stage neural semantic parser that first generates a meaning sketch and then fills in its omitted details. The framework separates semantic information by granularity, shares coarse structures, and achieves competitive results across diverse datasets despite simple decoders.
- Semantic parsing maps natural-language utterances to machine-interpretable meaning representations, motivating neural sequence-to-sequence approaches that account for structure.
- The proposed architecture first predicts a rough sketch omitting arguments and variable names, then generates the complete meaning representation conditioned on the input and sketch.
- The decomposition separates high-level from low-level semantics, makes compact sketches easier to generate, and enables knowledge sharing across examples with the same sketch.
- The framework applies across logical-form parsing, code generation, and SQL query generation using adaptations for four representative datasets.
- The framework achieves competitive performance compared with previous systems despite employing relatively simple sequence decoders.
2 Related Work
Related work spans symbolic and neural semantic parsers, structure-aware decoding, coarse-to-fine inference, and sketch-based program synthesis. The paper positions its sketches as a flexible intermediate representation applicable across structured meaning representations.
- Earlier semantic parsers learned lexicalized mapping rules and scoring models to construct meaning representations from natural-language expressions.
- Neural sequence-to-sequence semantic parsers reduced reliance on feature engineering, while later work explored augmentation, transfer, parameter sharing, and user feedback.
- Structure-aware neural decoders have used transition systems, grammars, and dynamically composed submodels to generate variable-free queries or abstract syntax trees.
- Coarse-to-fine semantic parsing has used coarse lexical entries or macro grammars to reduce search, whereas this work targets flexible sketches for trees and other structures.
- Program-synthesis research has used sketches as intermediate representations, including SQL sketches completed and iteratively repaired with synthesis techniques.
3 Problem Formulation
The model estimates meaning representations through a coarse-to-fine factorization: it predicts an abstract sketch from the input, then generates details conditioned on both input and sketch. Encoders, attention, constraints, and greedy inference implement this two-stage process.
- 3 Problem Formulation: The task is to learn semantic parsers from natural-language expressions paired with structured meaning representations.
- 3 Problem Formulation: The model estimates p(y|x) by decomposing generation into sketch prediction p(a|x) followed by meaning-representation generation p(y|x,a).
- 3 Problem Formulation: The sketch a abstracts y by stripping arguments and variable names in logical forms, schema-specific information in SQL, or replacing source-code tokens with types.
- 3.1 Sketch Generation: The input encoder maps words to vectors and a decoder generates the sketch conditioned on the encoded natural-language input.
- 3.1 Sketch Generation: The sketch decoder uses recurrent hidden states, attention over encoder states, and token-level probabilities that terminate at an end-of-sequence token.
- 3.2 Meaning Representation Generation: The fine decoder generates the complete representation from x and a, while a bidirectional sketch encoder supplies vectors aligned with sketch positions.
- 3.2 Meaning Representation Generation: Sketch alignment feeds corresponding sketch vectors into later decoding steps, while sketch tokens constrain output tokens, missing details, and type-compatible generation.
- 3.3 Training and Inference: Training maximizes log p(y|x,a) + log p(a|x), and test-time predictions use approximate greedy token generation for the two factorized stages.
4 Semantic Parsing Tasks
The paper adapts coarse-to-fine semantic parsing across logical forms, source code, and SQL, with task-specific sketch extraction and decoding mechanisms. For WIKISQL, the architecture incorporates table schemas, sketches, copying, and table-aware encoding.
- Tasks and datasets: The experiments cover logical-form parsing, Python source-code generation, and SQL query generation across multiple datasets.The logical-form task uses GEO and ATIS; source-code generation uses DJANGO; SQL generation uses WIKISQL.
- Logical-form parsing: Logical-form sketches strip arguments and variable names while retaining predicates, operators, and composition information.The extraction uses “@” to indicate missing predicate arguments and “?” when only partial argument information is omitted.
- Logical-form parsing: The logical-form decoder uses parent hidden states as inputs to softmax classifiers in both coarse and fine decoding stages.This parent-feeding design is motivated by the hierarchical structure of λ-calculus representations.
- Source-code generation: DJANGO sketches replace most code tokens with token types, while the fine decoder copies out-of-vocabulary tokens directly from the natural-language input.The copying mechanism addresses unseen variable names, numbers, and other tokens.
- SQL query generation: For WIKISQL, sketches are sorted WHERE-clause condition-operator sequences, while the decoder fills condition columns and values.The model generates the WHERE clause with sketch guidance and handles the fixed SELECT slots separately.
- SQL query generation: The WIKISQL encoder represents table columns and attends to relevant columns while encoding the question.Column names are encoded separately from the question, and attention links question states to column representations.
5 Experiments
Experiments evaluate COARSE2FINE across GEO, ATIS, DJANGO, and WIKISQL using accuracy-based comparisons against one-stage, prior, and ablated systems. The model generally outperforms ONESTAGE and prior systems, while sketch encoding and task-specific table awareness contribute to performance.
- Experimental Setup: The experiments use accuracy, with WIKISQL additionally evaluated by execution accuracy, and compare COARSE2FINE against ONESTAGE, prior systems, and ablations.The ablations remove the sketch encoder or table-aware input encoder.
- GEO and ATIS: COARSE2FINE outperforms ONESTAGE on GEO and ATIS, supporting the benefit of disentangling high-level from low-level information during decoding.Removing the sketch encoder harms performance, while oracle sketches provide an upper bound for the fine decoder.
- DJANGO: COARSE2FINE outperforms ONESTAGE by a wide margin on DJANGO and exceeds the best reported literature result, while the oracle is 8.9 points higher.The results also indicate that the sketch encoder is beneficial.
- WIKISQL: 90.2% and 92.0% are COARSE2FINE’s accuracies for WIKISQL aggregation operator and aggregation column prediction, respectively.These accuracies are comparable to SQLNET, while the largest gain comes from the improved WHERE-clause decoder.
- WIKISQL: A table-aware input encoder is critical for WIKISQL because identical questions can require different queries under different table schemas.The model predicts fixed SELECT slots with classifiers and uses the sketch-guided decoder for the WHERE clause.
- Sketch Analysis: COARSE2FINE produces more accurate sketches across datasets, but WIKISQL improvements mainly come from the fine decoder rather than marginally better sketches.The sketch supplies a canonical order for condition operators and may reduce interference when matching columns and extracting condition values.
6 Conclusions
The framework generates meaning sketches that abstract away low-level details, then fills them in to produce full meaning representations. Coarse-to-fine decoding improves performance across tasks and adapts to different domains and meaning representations.
- The framework first generates meaning sketches that abstract away arguments and variable names.
- It then predicts missing details to obtain full meaning representations.
- The framework can be adapted to different domains and meaning representations.
- Coarse-to-fine decoding improves performance across tasks.