Grammar-based Neural Text-to-SQL Generation

• Grammar-based decoding produces production-rule sequences rather than SQL token sequences, following evidence from related semantic parsing tasks.Source: §1 · p. 1
• SQL grammar construction is difficult because typed identifiers, schema-specific columns, and column-value relationships make generation non-context-free and prone to over-generation.Source: §1 · p. 1
• The proposed grammar covers more than 98% of instances in ATIS and SPIDER with minimal over-generation.Source: §1 · p. 1
• The model combines a base grammar with schema- and utterance-specific rules, while context-sensitive constraints can be imposed during training and inference.Source: §1 · pp. 1–2
• 73.7% denotation accuracy on contextual ATIS is a 4.5% absolute improvement over prior work, while SPIDER reaches 33.8% accuracy, 14.1% above comparable supervision.Source: §1 · p. 2

SQL Grammar

• Compiler grammars over-generate valid-looking SQL because programming languages are not fully context-free, and their deep derivations are difficult for semantic parsers to learn.Source: §2 · p. 2
• The grammar uses shallow parsing expression grammars covering the SQL needed by each dataset, improving compactness and learnability at the cost of dataset specificity.Source: §2 · p. 2
• Context sensitivity is handled with added non-terminals for reference consistency and runtime constraints for complex conditions such as foreign-key-compatible joins.Source: §2 · pp. 2–3

Adding schema non-terminals

• Schema-dependent rules add table and column productions and restrict comparisons to values occurring in the corresponding database column.Source: §2 · p. 3

SQL Grammar

• Additional rules are generated from utterance-detected numbers and times to handle cases where strict column-value constraints reject expressions such as departure time < 1701.Source: §2 · p. 3, Figure 2 · p. 3
• The decoder distinguishes global base-grammar rules from dynamically linked rules so rules unseen during training can be parameterized and used at test time.Source: §2 · p. 3

Other Considerations

• Table aliases are removed before training and restored during post-processing because they deepen the grammar, while complex aliases remain unsupported because they are rare.Source: §2 · pp. 3–4

Linearizing a syntax tree

• Training linearizes parsed SQL ASTs by depth-first, left-to-right traversal, and decoding applies grammar and runtime constraints to each production-rule choice.Source: §2 · p. 4, Figure 3 · p. 4

3 Model

• The model takes an utterance, database, and utterance-specific grammar, then outputs production rules that sequentially build a SQL AST.Rules are either global base-grammar productions or utterance-specific linked productions.Source: §3 · p. 4
• Identifier linking uses string-matching heuristics to connect utterance words with database identifiers such as table names, columns, and values.The resulting links support identifiers that may be unseen during testing.Source: §3 · p. 4
• The bidirectional LSTM encoder concatenates learned word vectors with link embeddings for each utterance token.For ATIS, previous utterances are appended to represent interaction history, but previous queries are unavailable.Source: §3 · p. 4
• The attention-based LSTM decoder predicts grammar productions while repeatedly expanding the leftmost non-terminal in the SQL AST.Global rules use learned embeddings, whereas linked rules use decoder attention and linking scores.Source: §3 · p. 5
• Identifier anonymization and identifier linking are used together, with anonymized variables treated as linked production rules.Anonymization is based only on linking heuristics rather than SQL queries, preserving a non-anonymized evaluation setting.Source: §3 · p. 5
• The model is trained by maximizing the log-likelihood of parsed SQL derivations, using the shortest derivation when multiple programs are available.The grammar therefore supplies structured supervision rather than token-sequence supervision.Source: §3 · p. 5

4.1 Datasets

• ATIS uses 1148/380/130 train/dev/test interactions with an average of 7 utterances per interaction, while SPIDER's database split has 146/20/40 train/dev/test databases.SPIDER evaluates composition of SQL operators and generalization to schemas unseen during training.Source: §4.1 · pp. 5–6

4.2 Implementation Details

• The ATIS model includes identifier anonymization and excludes runtime constraints, whereas the SPIDER model uses runtime constraints without identifier anonymization or database-value generation.These choices reflect the datasets' different emphases on linking values, joins, and table aliases.Source: §4.2 · p. 6

4.3 Results

• On SPIDER, the model achieves a 14.1% increase in exact component matching over the best previously published result.In the database split, all databases are unseen at test time, and a query is correct only when all components match exactly.Source: §4.3 · p. 6, §4.1 · p. 6
• On context-dependent ATIS, the model using identifier linking, link embeddings, and type-constraints improves denotation accuracy by 4.5% over the best prior published result.The evaluation uses Suhr et al. (2018)’s data re-split. Denotation accuracy credits queries that execute to the same table as the gold query, while non-executing queries receive no credit.Source: §4.1 · pp. 5–6, §4.3 · p. 6
• Ablations on ATIS show that link embeddings improve denotation accuracy by 3.3% without identifier anonymization, helping generate query structure and values for low-frequency identifiers.The model uses identifier information both for generating values and for selecting the correct query structure.Source: Table 3 · p. 7
• Adding the constraint that columns appear with their table does not significantly improve ATIS performance, possibly because training and test use the same tables.This ablation concerns schema-dependent grammar constraints on ATIS.Source: Table 4 · p. 7

5 Discussion

• Link embeddings improve denotation accuracy by 3.3% without identifier anonymization and help select the correct table and columns for rare identifiers.Source: §5 · p. 7, Figure 5
• The column-with-table constraint does not significantly improve ATIS performance, while the reported table also varies runtime-constraint and alias settings.Source: §5 · p. 7
• 9.6% denotation-accuracy decrease after removing value constraints shows that generating correct WHERE-clause values is central on ATIS.Source: §5 · p. 7
• Table aliases and runtime constraints both improve exact component accuracy on SPIDER, according to the reported ablation comparison.Source: §5 · p. 7, Table 5 · p. 7
• A production-rule copy mechanism shortens derivations and aids interpretability but produces no significant accuracy change.Source: §5 · p. 7

5.1 Error Analysis

• Grammar construction is sensitive to recursive-rule ordering, and ATIS errors frequently arise from linking or missing constraints, especially ambiguous airport–city references.Source: §5 · p. 7, §5.1 · p. 8
• Datetime linking limits parsing to queries whose identifiers are detected in the utterance; UWTime incorrectly identified dates in 27.6% of unparseable ATIS queries.Source: §5.1 · p. 8

6 Related Work

• SPIDER tests complex queries and generalization to unseen databases, while ATIS tests context-dependent utterances and many tables per database.Source: §6 · p. 8
• Prior grammar-based parsers generate production rules for formal languages, but WikiSQL’s relatively simple queries require only four grammar rules.Source: §6 · p. 8
• Compared with Yu et al. (2018b), this method outputs production rules rather than tokens and considerably outperforms that SQL-specific modular approach.Source: §6 · p. 8
• For SPIDER’s zero-shot setting, the grammar-based model separates global and linked rules to decouple program structure from identifiers without added complexity.Source: §6 · p. 8

7 Conclusion

• The model combines a dynamic schema-dependent SQL grammar with deterministic entity linking for natural-language database interfaces.Source: §7 · p. 8
• 4.5% higher ATIS denotation accuracy and 14.1% higher SPIDER exact component matching are reported over the best prior work.Source: §7 · p. 8
• The results suggest that link embeddings or identifier anonymization, type information, and context sensitivity are important for text-to-SQL performance.Source: §7 · p. 8