Source-linked AI summary
SPOC-SQL: Stage-wise Preference Optimization for Controllable Text-to-SQL
Yingnan Chen, Chun Ding, Tianshi Xu, Xu Yang, Si Wu
TL;DR
Text-to-SQL systems commonly optimize complete SQL sequences without targeted feedback or interaction over intermediate decisions. SPOC-SQL decomposes generation into four SQL-aligned stages, applies preference optimization at key decisions, and supports stage-wise intervention and correction. Experiments report consistent gains from human knowledge intervention across datasets, models, difficulty levels, and evaluation settings.
Problem
Existing Text-to-SQL methods optimize complete SQL sequences holistically, diluting signals for key decisions and limiting interpretability, interaction, and intermediate correction.
Method
SPOC-SQL decomposes Text-to-SQL into four SQL-aligned stages and combines PMDO preference optimization with RDIM for stage-wise verification and correction.
Results
Human knowledge intervention consistently improves performance across models, datasets, and difficulty levels, reaching 95.6% EX on Spider-Dev and 93.1% on Spider-Realistic with full-component integration.
Takeaways & Limitations
SPOC-SQL makes SQL generation more structured, interpretable, controllable, and collaborative by exposing intermediate decisions for verification and correction.
Abstract
from arXiv · showhide
Text-to-SQL aims to translate natural language questions into executable SQL queries over relational databases, requiring multi-stage structured reasoning over database schemas and query constraints. However, existing methods treat this task as single-step generation, where models optimize entire SQL sequences without targeted feedback at key decision points and lack support for interacting with and controlling the intermediate generation process. To address this issue, we propose SPOC-SQL, which decomposes Text-to-SQL into four sequential subtasks following standard SQL execution logic and designs stage-specific optimization strategies for the model to learn key decisions. Specifically, we propose the implementation of fine-grained preference optimisation at key decision points across SQL stages, with the objective of enhancing structured decision-making during query construction. Furthermore, a structured decomposition strategy is designed, facilitating stage-wise intervention and correction through explicit intermediate representations. This results in more controllable and reliable SQL generation. Experiments demonstrate that incorporating stage-wise human knowledge consistently improves performance, validating the effectiveness of stage perception controllable generation.
1 Introduction
SPOC-SQL reframes Text-to-SQL as a stage-wise structured decision process rather than single-step SQL generation. It combines fine-grained preference optimization with structured decomposition to support more targeted decisions, verification, and correction.
- Text-to-SQL requires multi-step decisions because questions may involve joins, nested subqueries, complex conditions, and semantic relationships in database schemas.
- Existing methods optimize entire SQL sequences holistically, diluting feedback for key interaction steps and limiting learned interaction strategies.
- SPOC-SQL decomposes Text-to-SQL into four sequential subtasks aligned with standard SQL execution logic for progressive decision-making.
- The framework constructs multi-turn question-answering samples by decomposing single-turn instances into sequential substages with stage-specific requirements.
- PMDO extends preference supervision from whole sequences to key decision points across SQL stages, using preferred intermediate decisions derived from gold SQL execution.
- RDIM exposes intermediate results for verification and correction, enabling more interpretable and controllable SQL generation with user-driven intervention.
2 Methodology
SPOC-SQL constructs stage-wise, multi-turn Text-to-SQL interactions and uses intermediate results and user feedback to support controllable SQL generation. It then optimizes fine-grained decisions within each SQL stage using stage-conditioned preference signals and parameter-efficient tuning.
- Multi-turn data construction: Single-turn queries are decomposed into SQL-aligned stages, and their subtasks are converted into sequential multi-turn interaction sequences.The stages include SELECT-FROM, WHERE, GROUP-HAVING, and ORDER-LIMIT; absent subtasks may be skipped.
- Multi-turn data construction: The constructed interactions represent explicit, vague, and revised requests, including cases where users supplement requirements or correct erroneous conditions.This preserves the original query semantics while modeling incremental user expression, supplementation, and correction.
- Evaluation: The evaluation framework measures multi-turn completion, independent single-turn performance, and responses to explicit user corrections.These settings assess dialogue consistency, local context comprehension, stage-specific execution, and revision of previously generated erroneous conditions.
- Requirement decomposition and interaction: During inference, stage responses are exposed for user confirmation or correction, and updated stage information is integrated with the complete query to generate the final SQL statement.The interaction path is designed to maintain syntactic correctness, logical consistency, and adherence to stage-specific requirements.
- Model optimization: PMDO optimizes stage-specific decision elements with preference signals conditioned on each subtask, while LoRA updates parameters without changing the original model weights.Decision elements cover table and column selection, conditions, aggregation or grouping, and ordering or limiting; later stages use earlier outputs as context.
- Model optimization: Stage-specific preference losses are accumulated sequentially so the model learns decision boundaries while preserving coherence across multi-turn SQL generation.The stage context includes the user query and outputs from previous SQL-generation stages.
3 Experiments
SPOC-SQL is evaluated across multiple Text-to-SQL benchmarks and a multi-turn dataset, covering single-turn and multi-turn scenarios with comprehensive comparisons and analyses.
- Evaluation scope: The experiments evaluate SPOC-SQL on multiple Text-to-SQL benchmarks and a multi-turn dataset.The evaluation includes comparisons with state-of-the-art methods, model and difficulty analyses, ablations, case studies, and user studies.
3.1 Settings
The experimental settings combine Spider benchmarks with a purpose-built multi-turn dataset and specify parameter-efficient fine-tuning configurations.
- Datasets: Spider-Dev contains 8,659 training instances across 200 databases, while Spider-Realistic removes explicit column references to test semantic understanding.
- Datasets: T2S-MTD contains 71,772 instances covering explicit, vague, and revised request interactions.These categories simulate direct generation, progressive supplementation, and modification of previous requirements.
- Implementation: LoRA fine-tuning uses rank r = 8, scaling factor α = 16, learning rate η = 2 × 10−4, weight decay 0.01, and effective batch size 32.The original model is frozen while only LoRA parameters are updated.
3.2 Comparison with State-of-the-Arts
SPOC-SQL improves Spider benchmark performance through progressive human-knowledge intervention, including on increasingly difficult query subsets.
- Overall comparison: 95.6% EX on Spider-Dev and 93.1% on Spider-Realistic are reached with full-component human-knowledge integration.Performance improves consistently as intervention is added across stages.
- Overall comparison: 89.7% EX after schema-focused intervention exceeds MCS-SQL’s 89.5% on the reported comparison.
- Difficulty analysis: 74.1% EX on Spider-Dev Extra Hard without human intervention surpasses MCS-SQL’s 72.9%.The reported decomposition is associated with improved performance on complex queries even before intervention.
- Difficulty analysis: 83.5% on Spider-Realistic Extra Hard follows an increase from 62.9% with human-knowledge intervention.The passage reports gains across all difficulty levels under intervention.
3.3 Performance of SPOC-SQL Across Models and Datasets
Across models and datasets, interactive stage interventions consistently improve SPOC-SQL, with larger gains on harder subsets and stronger multi-turn results from the proposed training strategy.
- Models and datasets: Interactive components improve performance across all DeepSeek-V3 and Qwen3-72B combinations on Spider-Dev and Spider-Realistic.The comparison includes the base configuration and four progressively enhanced intervention configurations.
- Models and datasets: Performance improvements are larger on Hard and Extra Hard subsets than across all difficulty levels.
- Multi-turn evaluation: Qwen-T2S outperforms Qwen-Base and Qwen-LoRA on multi-turn QA, single-turn QA, and error-correction QA.The evaluation uses the T2S-MTD dataset and compares base models with single-turn and proposed multi-turn training.
- Multi-turn evaluation: DS-T2S further surpasses Qwen-T2S under the same training strategy across the evaluated tasks.
3.4 Ablation Study
The ablation study evaluates PMDO and RDIM across Spider benchmarks and T2S-MTD, finding that both modules contribute to SPOC-SQL’s performance.
- The study uses DeepSeek-V3 on Spider-Dev, Spider-Realistic, and the proposed T2S-MTD dataset.
- Removing either PMDO or RDIM degrades performance on Spider benchmarks and T2S-MTD.Full SPOC-SQL achieves the best overall results, while LoRA-only improves over the base model but remains inferior to SPOC-SQL.
- PMDO introduces stage-wise preference optimization during training, while RDIM decomposes SQL generation into stage-wise subtasks during inference.The ablation isolates each module by removing it while retaining the other.
- The results support the effectiveness of combining stage-wise decomposition with preference-driven optimization.
3.5 Case Study
The case study shows how SPOC-SQL decomposes complex SQL generation into structured components, sub-questions, and corresponding parameters.
- Nested subqueries and set operations are decomposed into structured components with detailed sub-questions and parameters.The decomposition targets complex query structures that are difficult to generate directly.
- Multi-component queries are modularly parsed into joins, filtering, and grouping components.
- Splitting queries into sub-questions reduces generation complexity and helps users better understand the construction process.
3.6 User Study
The user study compares LLM, SPOC, and Human SQL information across subjective and objective metrics. SPOC scores close to Human and consistently exceeds LLM, with subjective gains significantly correlated with SQL accuracy gains.
- Over 100 participants evaluated LLM, SPOC, and Human SQL information versions on ten-point subjective metrics.Samples came from Spider-Dev and Spider-Realistic.
- SPOC achieved scores close to Human and consistently outperformed LLM across all metrics.
- ρ = 0.6699, p < 0.01 for the correlation between subjective improvement and SQL accuracy gain.
- Jaccard Similarity vs Human showed that SPOC outputs closely resemble human-authored content.
4 Conclusion
SPOC-SQL integrates multi-turn alignment with preference optimization for Text-to-SQL. It decomposes queries into structured stages and supports interactive verification, with experiments demonstrating effectiveness across models and datasets.
- SPOC-SQL integrates multi-turn alignment procedures with preference optimization strategies for Text-to-SQL.
- The method decomposes single-turn queries into structured multi-turn supervision and applies PMDO at key decision points across SQL stages.
- RDIM enables stage-wise decomposition and interactive verification, making SQL generation transparent, controllable, and correctable.
- Experiments across multiple models and datasets demonstrate effectiveness, robustness, and generalizability while improving accuracy and interpretability.