Source-linked AI summary
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL
Haoyang Li, Jing Zhang, Cuiping Li, Hong Chen
TL;DR
Text-to-SQL requires jointly linking schema items and parsing SQL structure, which becomes difficult for queries with many items and operators. RESDSQL decouples these tasks through ranked schema encoding and skeleton-aware decoding, and reports state-of-the-art performance on Spider with robustness across three variants.
Problem
Text-to-SQL couples schema linking with skeleton parsing, complicating SQL generation for queries involving many schema items and complex operators.
Method
RESDSQL ranks and filters relevant tables and columns for the encoder, then generates an SQL skeleton before the full query to guide decoding.
Results
RESDSQL achieves new state-of-the-art performance on Spider and strong robustness across Spider-DK, Spider-Syn, and Spider-Realistic.
Takeaways & Limitations
Decoupling schema linking and skeleton parsing can alleviate Text-to-SQL difficulty while supporting performance and robustness across benchmark settings.
Takeaways & Limitations
The experiments exclude Spider’s additional 1,659 training samples collected from single-domain datasets.
Abstract
from arXiv · showhide
One of the recent best attempts at Text-to-SQL is the pre-trained language model. Due to the structural property of the SQL queries, the seq2seq model takes the responsibility of parsing both the schema items (i.e., tables and columns) and the skeleton (i.e., SQL keywords). Such coupled targets increase the difficulty of parsing the correct SQL queries especially when they involve many schema items and logic operators. This paper proposes a ranking-enhanced encoding and skeleton-aware decoding framework to decouple the schema linking and the skeleton parsing. Specifically, for a seq2seq encoder-decode model, its encoder is injected by the most relevant schema items instead of the whole unordered ones, which could alleviate the schema linking effort during SQL parsing, and its decoder first generates the skeleton and then the actual SQL query, which could implicitly constrain the SQL parsing. We evaluate our proposed framework on Spider and its three robustness variants: Spider-DK, Spider-Syn, and Spider-Realistic. The experimental results show that our framework delivers promising performance and robustness. Our code is available at https://github.com/RUCKBReasoning/RESDSQL.
Introduction
Text-to-SQL models must jointly link question entities to schema items and parse SQL structure, making complex queries difficult. RESDSQL decouples these objectives by ranking relevant schema items for encoding and generating the SQL skeleton before the query.
- Text-to-SQL translates natural-language questions into executable SQL over database schemas.
- Complex queries require both schema linking and skeleton parsing, while operators such as GROUP BY, HAVING, and JOIN ON complicate their interaction.
- About 80% skeleton exact match versus about 70% full parsing accuracy suggests decoupling may reduce Text-to-SQL difficulty.
- RESDSQL ranks and filters relevant tables and columns before injecting them into the seq2seq encoder.
- Its decoder generates the skeleton first, allowing masked self-attention to guide subsequent SQL generation.
- The framework is evaluated on Spider and three robustness variants, with the authors reporting new Spider SOTA performance and strong robustness.
Problem Definition
A relational database schema consists of tables, their columns, and foreign-key relations. The Text-to-SQL task maps a natural-language question and database schema to an executable SQL query.
- A database schema contains N tables, a set of columns associated with those tables, and foreign-key relations.
- The total number of columns is represented as the sum of the columns across all tables.
- Schema items uniformly refer to tables and columns, each represented by an original name and a semantic name.
- Semantic names can express schema-item meaning more clearly than original names, such as “airline id” versus “uid”.
- Given a natural-language question and database schema, Text-to-SQL produces a SQL query executable on the database.
Methodology
RESDSQL decouples schema linking from skeleton parsing by ranking relevant schema items for encoding and generating the SQL skeleton before the full query.
- Ranking-Enhanced Encoder: The ranking input concatenates the question with a flattened schema sequence represented using semantic names rather than only original names.Semantic names are intended to express schema-item meaning more clearly in natural language.
- Ranking-Enhanced Encoder: A column-enhanced layer injects column information into table embeddings so tables can be identified when questions mention columns but omit table names.The layer applies multi-head attention from table representations to their column representations, then fuses the resulting embeddings.
- Ranking-Enhanced Encoder: A cross-encoder jointly classifies tables and columns with focal loss to address the severe imbalance between referenced and unreferenced schema items.Its multi-task loss combines table-classification and column-classification losses.
- Ranking-Enhanced Encoder: RESDSQL ranks schema items before encoding, retaining top-k1 tables and top-k2 columns per retained table instead of injecting the entire schema.The ranked sequence also supplies positional information for schema linking, while the hyperparameters trade omitted required items against irrelevant noise.
- Skeleton-Aware Decoder: The decoder first generates an SQL skeleton and then continues with the actual query, allowing later tokens to use the previously generated skeleton.The skeleton contains SQL keywords and slots, while required tables, columns, and values fill those slots; JOIN ON is excluded from extracted skeletons.
- Skeleton-Aware Decoder: SQL queries are normalized before training by standardizing keyword and schema-item case, spacing around parentheses, and quotation style.Normalization reduces variation in manually annotated Spider queries without changing execution results.
Experiments
RESDSQL is evaluated on Spider and three robustness variants, with results showing strong benchmark performance and robustness. Ablations indicate that ranked schema encoding contributes more than skeleton parsing, while focal loss and column enhancement also matter.
- Datasets: Spider contains 7,000 training samples, 1,034 development samples, and 2,147 hidden-test samples, with no database overlap across splits.Robustness evaluation uses Spider-DK, Spider-Syn, and Spider-Realistic with 535, 1,034, and 508 samples respectively.
- Results on Spider: RESDSQL-3B + NatSQL improves Spider development-set EM by 4.2% and EX by 3.6% absolute.On the hidden test set, it increases EX from 75.5% to 79.9% (+4.4%).
- Results on Robustness Settings: RESDSQL-3B + NatSQL outperforms all strong competitors by a large margin on Spider-DK, Spider-Syn, and Spider-Realistic.The authors attribute this robustness to the cross-encoder alleviating schema-linking difficulty under question perturbations.
- Ablation Studies: Removing the column-enhanced layer decreases total AUC, while replacing focal loss with cross-entropy causes a performance drop under label imbalance.The column-enhanced layer injects column information into table embeddings, and focal loss addresses classification imbalance.
- Ablation Studies: Replacing ranked schema sequences with unordered ones decreases EM by 4.5% and EX by 7.8%, identifying ranking-enhanced encoding as crucial.This comparison uses the ablation results reported in Table 5.
Related Work
Prior Text-to-SQL work primarily develops encoder-decoder architectures, schema item classification, and intermediate representations to encode schemas, constrain SQL generation, or bridge language and SQL.
- Sequence Encoder: Sequence encoders concatenate the question with a serialized database schema before applying a pretrained language-model encoder.
- Graph Encoder: Graph encoders represent question tokens, tables, and columns as heterogeneous graphs connected by relational edges.
- Decoders: Grammar-based and execution-guided decoders constrain SQL generation by pruning invalid actions, partial queries, or non-executable candidates.
- Schema Item Classification: Schema item classification is used as pretraining or multitask supervision to improve schema linking in Text-to-SQL systems.
- Intermediate Representation: Intermediate-representation methods generate a representation that a non-trainable transpiler converts into SQL, rather than producing full SQL directly.
Conclusion
RESDSQL decouples schema linking from skeleton parsing by ranking and filtering schema items for encoding and generating the SQL skeleton before the query. Experiments on Spider and three variants demonstrate its performance and robustness.
- Conclusion: RESDSQL ranks and filters schema items with a cross-encoder before injecting them into the seq2seq encoder.
- Conclusion: Its decoder generates the SQL skeleton first to guide subsequent SQL generation.
- Conclusion: Experiments on Spider and three variants demonstrate RESDSQL’s performance and robustness.