PaperlayerView exact revision on arXiv

Source-linked AI summary

SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning

Peixian Ma, Xialie Zhuang, Chengjin Xu, Xuhui Jiang, Ran Chen, Jian Guo

arXiv:2504.08600v5cs.DB

TL;DR

Complex multi-table joins and nested queries remain difficult for NL2SQL models, whose SFT-based training can limit adaptation and interpretability. SQL-R1 uses RL with specialized rewards, cold-start strategies, and synthetic data to train NL2SQL reasoning. It reports 88.6% Spider-Test accuracy and 66.6% BIRD accuracy while generating interpretable reasoning traces.

  • Problem

    NL2SQL models struggle with reasoning in complex database scenarios, while SFT may limit adaptability and interpretability in environments such as finance and healthcare.

  • Method

    SQL-R1 combines RL-based NL2SQL training with dynamic rewards, SFT cold start, and synthetic data engineering.

  • Results

    88.6% Spider-Test accuracy and 66.6% BIRD accuracy are reported, with SQL-R1 also generating interpretable reasoning traces.

  • Takeaways & Limitations

    The study supports RL, cold-start strategies, and synthetic data as components of training NL2SQL reasoning models for complex database scenarios.

  • Takeaways & Limitations

    SQL-R1 is primarily trained and evaluated on SQLite datasets, so generalization across database dialects such as Snowflake and DuckDB remains future work.

Abstract

from arXiv · show

Natural Language to SQL (NL2SQL) enables intuitive interactions with databases by transforming natural language queries into structured SQL statements. Despite recent advancements in enhancing human-computer interaction within database applications, significant challenges persist, particularly regarding the reasoning performance in complex scenarios involving multi-table joins and nested queries. Current methodologies primarily utilize supervised fine-tuning~(SFT) to train the NL2SQL model, which may limit adaptability and interpretability in new environments~(e.g., finance and healthcare). In order to enhance the reasoning performance of the NL2SQL model in the above complex situations, we introduce SQL-R1, a novel NL2SQL reasoning model trained by the reinforcement learning~(RL) algorithms. We design a specialized RL-based reward function tailored for NL2SQL tasks and discussed the impact of cold start and synthetic data on the effectiveness of intensive training. In addition, we achieve competitive accuracy using only a tiny amount of synthetic NL2SQL data for augmented training and further explore data engineering for RL. In existing experiments, SQL-R1 achieves execution accuracy of 88.6\% and 67.1\% on the benchmark Spider and BIRD, respectively. The code is available at https://github.com/IDEA-FinAI/SQL-R1 .

1 Introduction

NL2SQL systems still struggle with reasoning over complex schemas and ambiguous semantics, while SFT can be unstable for domain adaptation. SQL-R1 addresses these challenges with RL, specialized training strategies, and synthetic data, achieving strong benchmark accuracy.

  • 1 Introduction: NL2SQL research commonly optimizes schema linking, content retrieval, and generation correction, but complex database reasoning remains challenging.The difficulty is especially apparent for multi-table joins, nested queries, and ambiguous semantics.
  • 1 Introduction: SFT-based NL2SQL models depend on schema structure and training-data scale, which can limit stability in domain adaptation and high-risk applications.The paper identifies finance and healthcare as examples of high-risk fields affected by these limitations.
  • 1 Introduction: SQL-R1 trains an NL2SQL reasoning model with RL that directly optimizes SQL matching user intent through task-specific feedback and rewards.The paper frames reward design as central to successful RL training for NL2SQL reasoning.
  • 1 Introduction: 88.6% accuracy on Spider-Test and 66.6% on BIRD are reported for SQL-R1 trained with a few NL2SQL data, alongside explicit reasoning output.The contribution statement specifies that the model can be trained on a small dataset such as 5K samples.
  • 1 Introduction: The training strategy combines SFT cold start, RL, and synthetic data to strengthen instruction following, exploration, robustness, and generalization.Cold start first trains the base model with SFT, while synthetic data compensates for limited real NL2SQL data.

2 SQL-R1

SQL-R1 trains NL2SQL models with reinforcement learning, combining cold-start SFT, synthetic data, and a specialized progressive reward design. Its pipeline uses challenging synthetic examples, group-based policy optimization, and multiple checks to favor correctly formatted, executable, and accurate SQL with explicit reasoning.

  • Training strategy: SQL-R1 supports direct RL training and RL after an SFT cold start, with synthetic data used because real NL2SQL data is limited.Cold start trains the base model by SFT to improve instruction following and reasoning before RL.
  • Data engineering: Over 2.5 million samples across more than 16,000 synthetic databases form the primary SynSQL-2.5M data source.Samples pair databases, natural-language questions, SQL queries, and chain-of-thought solutions across simple through highly complex SQL.
  • Data engineering: 200,000 SynSQL-2.5M samples are used for SFT, while 5K complex NL-SQL pairs are sampled for RL without the source dataset’s chain-of-thought data.The SFT subset is balanced across four difficulty levels, whereas SynSQL-Complex-5K targets sophisticated SQL generation.
  • Reinforcement training: GRPO generates groups of SQL candidates and updates the policy using their relative composite-reward performance, without requiring a value model.The objective includes group-relative advantages, clipping, and divergence regularization against a reference policy.
  • Reward design: The progressive reward combines format, execution, result, and length rewards to encourage structured reasoning and correct user-intent-aligned SQL.Format checks require tagged reasoning and SQL; execution checks gate later rewards and limit runtime; result reward uses execution accuracy, while length reward balances answer and SQL lengths.
  • Inference: SQL-R1 generates multiple SQL candidates, selects the highest-scoring candidate by self-consistency voting, and exposes its reasoning process to users.This selection procedure evaluates all generated candidates before producing the final answer.

3 Experiments

SQL-R1 is evaluated on Spider, BIRD, additional Spider variants, complexity levels, model scales, and training strategies using Execution Accuracy. Results show strong benchmark performance, improved handling of challenging queries, and sensitivity to model scale, cold-start data, and reward design.

  • 3.1 Setup: Execution Accuracy (EX) is used to evaluate SQL-R1 and related models on Spider and BIRD.Spider contains 10,181 questions across 200 databases and 138 domains, while BIRD contains 12,751 pairs across 95 databases and 37 specialized domains.
  • 3.2 Main Results: SQL-R1 reaches 78.1% Spider-Dev, 78.9% Spider-Test, and 54.6% BIRD-Dev with Qwen2.5-Coder-3B.With Qwen2.5-Coder-7B, it reaches 87.6%, 88.7%, and 63.1%; with Qwen2.5-Coder-14B, it reaches 86.7%, 88.1%, and 67.1% on the same datasets.
  • 3.2 Main Results: SQL-R1 outperforms other NL2SQL solutions on Spider-DK, Spider-Syn, and Spider-Realistic, with further results reported for Spider2.0.These evaluations are described as demonstrating generalization and robustness across different database and query requirements.
  • 3.2 Main Results: 5.5 percentage points: scaling SQL-R1 from 7B to 14B raises Challenging BIRD-Dev accuracy from 51.0% to 56.5%.Simple and Moderate accuracy changes by only 0.3% and -1.1%, respectively, suggesting model capacity benefits the most complex queries.
  • 3.2 Main Results: On BIRD-Dev, SQL-R1 with a 7B base model surpasses larger-scale models in the performance-and-model-scale comparison.The analysis frames this result as evidence of performance and cost-efficiency trade-offs.
  • 3.2 Main Results: RL training improves reasoning on challenging BIRD-Dev examples, where the model exhibits a top-down strategy for generating SQL queries.The case study attributes the observed improvement in NL2SQL reasoning to reinforcement learning.
  • 3.3 Analysis of SFT Cold Start: Synthetic-data volume and provenance affect SFT performance and subsequent RL training, while SFT cold starts are not universally essential.SQL-R1 initialized with SynSQL-2.5M outperforms the version trained with SynSQL-200K, and effectiveness depends on data origin and volume.
  • 3.3 Analysis of SFT Cold Start: Removing any reward component harms reasoning performance, underscoring the importance of execution feedback and result reward.The reward ablation is conducted on BIRD-Dev while retaining the parameter settings from the full reward function.

4 Related Works

Related work spans NL2SQL workflow optimization and reinforcement learning for complex reasoning. SQL-R1 builds on both directions by targeting reasoning-oriented NL2SQL training.

  • NL2SQL Methods: NL2SQL research optimizes schema linking, translation strategies, SQL correction, and self-consistency-based SQL selection.
  • Reinforcement Learning for LLM Reasoning: Recent LLM reinforcement-learning research targets complex multi-step reasoning and interaction with external environments, including through GRPO.

5 Limitations

The study identifies scope limitations in database dialect coverage and model coverage. SQL-R1 is mainly evaluated with SQLite datasets and Qwen2.5-Coder models, leaving broader generalization for future work.

  • Supported Database Dialect: SQL-R1 is primarily trained and evaluated on SQLite, while generalization across dialects such as Snowflake and DuckDB remains future work.
  • Future Scope: The paper frames broader database-dialect and base-LLM coverage as significant future research directions.
  • Experiment: Experiments use Qwen2.5-Coder models and do not cover the broader range of newer LLMs, such as Llama4.

6 Conclusion

The conclusion presents SQL-R1 as an RL-trained NL2SQL reasoning model for complex database scenarios, supported by dynamic rewards, cold-start strategies, and synthetic data. Examples illustrate reasoning across challenge, moderate, and simple queries.

  • 6 Conclusion: SQL-R1 uses reinforcement learning to address semantic understanding, reasoning, and generalization challenges in complex database scenarios.
  • 6 Conclusion: 88.6% accuracy on Spider-Test and 66.6% on BIRD are reported alongside interpretable reasoning traces.
  • 6 Conclusion: The approach integrates dynamic reward mechanisms, cold-start strategies, and sustainable data engineering for training.
  • Reasoning Examples: Challenge examples address carcinogenic molecules and fourth-atom toxicology elements during NL2SQL reasoning.
  • Reasoning Examples: Other examples cover post-editing lookup and female-patient filtering across moderate and simple query settings.

B.1 Analysis and Comparison of Base LLMs

The base-LLM analysis compares execution accuracy across Spider and BIRD and examines how reinforcement-learning gains vary with model scale. Smaller models benefit more from RL than the 14B model.

  • Base-LLM Comparison: Table 7 reports execution accuracy for different base LLMs on the Spider and BIRD benchmarks.
  • RL Gains by Scale: Smaller Qwen2.5-Coder-3B and 7B models show substantial accuracy improvements from RL on Spider and BIRD.
  • RL Gains by Scale: Qwen2.5-Coder-14B shows a relatively minor improvement, indicating stronger RL gains for smaller models in these experiments.

B.2 Analysis and Comparison of Efficiency

SQL-R1 improves BIRD-Dev efficiency through self-consistent SQL generation, while outperforming XiYan-SQL under matched base-model conditions.

  • 0.7 seconds is the average per-query latency added by 8-candidate self-consistency over greedy search.The comparison uses the same inference settings and evaluation metric on BIRD-Dev.
  • 2.9% is the execution-accuracy improvement from 8-candidate self-consistency over greedy search.
  • SQL-R1 achieves higher accuracy than XiYan-SQL under the same conditions and shared Qwen2.5-Coder-7B base model.

B.3 Ablation Study of SQL Candidate Rollouts in Reasoning

The SQL candidate rollout ablation identifies 8 candidates as the preferred balance between diversity and computational efficiency across evaluated model scales.

  • 8 SQL candidates strike the ideal balance between rollout diversity and computational efficiency.The ablation evaluates Qwen2.5-Coder-3B and Qwen2.5-Coder-7B under varied temperature settings.
  • The study varies candidate-rollout counts and temperature settings on the BIRD-Dev dataset.

B.4 Ablation Study of Database Value Retrieval for RL Training

Adding database values during RL training does not significantly improve overall performance and reduces training efficiency through longer input sequences.

  • Database-value integration does not yield a statistically significant improvement in overall performance.
  • Longer input sequences from database values diminish training efficiency.
  • The authors therefore omit representative value annotations during training and prioritize schema linking.They connect this choice to improved exploratory capabilities during RL and improved reasoning and generalization skills.

B.5 Ablation Study of Reward Sensitivity

Reward sensitivity analysis shows that NL2SQL RL training depends on balanced reward calibration: weak execution or result rewards enable exploitation, while excessive early rewards destabilize learning.

  • Minimal Execution and Result reward weights degrade performance because the model may exploit simpler patterns instead of developing robust reasoning.
  • Excessively high reward weights early in training can destabilize learning before solid reasoning is established.
  • The model is less sensitive to Result reward adjustments, indicating difficulty bridging semantic gaps despite successful SQL-query exploration.
  • Fixed Length rewards are ineffective for fostering deep reasoning, whereas smooth sensitivity preserves differentiation without training oscillations.Excessive Length rewards may instead produce verbose solutions.
  • Balanced reward calibration promotes stable learning and minimizes exploitation in reinforcement learning.
Loading 2504.08600v5…