PaperlayerView exact revision on arXiv

Source-linked AI summary

TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas

Ai Jian, Xiaoyun Zhang, Wanrou Du, Jingqing Ruan, Jiangbo Pei, Weipeng Zhang, Ke Zeng, Xunliang Cai

arXiv:2603.16448v2cs.AI

TL;DR

Text-to-SQL systems often assume complete schemas are pre-loaded, an assumption that breaks down for large, noisy, evolving enterprise databases. TRUST-SQL treats the task as partially observable, using a four-phase tool interaction protocol and Dual-Track GRPO to ground schema exploration and SQL generation. Across five benchmarks, it improves substantially over base models and matches or surpasses schema-prefilled baselines without pre-loaded metadata.

  • Problem

    The Full Schema Assumption is impractical for large, noisy, evolving enterprise databases, motivating Text-to-SQL under the Unknown Schema setting.

  • Method

    TRUST-SQL formulates Unknown Schema Text-to-SQL as a POMDP and uses a four-phase protocol with Dual-Track GRPO and token-level masked advantages.

  • Results

    30.6% and 16.6% average absolute improvements are achieved by the 4B and 8B variants respectively across five benchmarks, while TRUST-SQL matches or surpasses schema-prefilled baselines without pre-loaded metadata.

  • Takeaways & Limitations

    TRUST-SQL establishes autonomous database exploration as an effective approach for reliable Text-to-SQL in unobservable environments.

  • Takeaways & Limitations

    Training and evaluation use SQLite-based benchmarks, and the framework incurs higher inference cost from live database calls.

Abstract

from arXiv · show

Text-to-SQL parsing has achieved remarkable progress under the Full Schema Assumption. However, this premise fails in real-world enterprise environments where databases contain hundreds of tables with massive noisy metadata. Rather than injecting the full schema upfront, an agent must actively identify and verify only the relevant subset, giving rise to the Unknown Schema scenario we study in this work. To address this, we propose TRUST-SQL (Truthful Reasoning with Unknown Schema via Tools). We formulate the task as a Partially Observable Markov Decision Process where our autonomous agent employs a structured four-phase protocol to ground reasoning in verified metadata. Crucially, this protocol provides a structural boundary for our novel Dual-Track GRPO strategy. By applying token-level masked advantages, this strategy isolates exploration rewards from execution outcomes to resolve credit assignment, yielding a 9.9% relative improvement over standard GRPO. Extensive experiments across five benchmarks demonstrate that TRUST-SQL achieves an average absolute improvement of 30.6% and 16.6% for the 4B and 8B variants respectively over their base models. Remarkably, despite operating entirely without pre-loaded metadata, our framework consistently matches or surpasses strong baselines that rely on schema prefilling.

1 Introduction

TRUST-SQL addresses Text-to-SQL when complete schemas are unavailable by combining active metadata exploration with a structured interaction protocol and Dual-Track GRPO. Across five benchmarks, it improves over base models and matches or surpasses schema-prefilled baselines without pre-loaded metadata.

  • Motivation: The Full Schema Assumption pre-loads complete database schemas, but enterprise databases often contain hundreds of evolving, noisy tables.Injecting all metadata can exceed finite context windows and distract models with irrelevant or stale tables.
  • Motivation: Unknown Schema Text-to-SQL requires agents to actively retrieve and verify relevant metadata instead of receiving the complete schema upfront.The setting is motivated by environments where schema observability cannot be assumed.
  • Challenges: Single-turn methods lack interactive capabilities, while multi-turn agents can lose intermediate observations, hallucinate schema elements, and face difficult trajectory-level credit assignment.Existing reward designs may conflate schema exploration quality with SQL generation quality.
  • Approach: TRUST-SQL formulates the task as a POMDP and uses Explore, Propose, Generate, and Confirm phases to ground SQL generation in verified metadata.The Propose phase is a mandatory checkpoint that commits the agent to verified metadata and provides a boundary for training.
  • Results: 9.9% relative improvement in execution accuracy over standard GRPO is achieved on BIRD-Dev by Dual-Track GRPO with token-level masked advantages and execution-coupled schema rewards.The strategy separately optimizes exploration and execution signals.
  • Results: 30.6% and 16.6% average absolute improvements are obtained by the 4B and 8B variants respectively across five benchmarks, while operating without pre-loaded metadata.The models consistently match or surpass baselines relying on schema injection.

2 Related Work

Prior Text-to-SQL methods largely assume complete schemas are available in static context, while multi-turn reinforcement learning introduces unresolved credit-assignment challenges. TRUST-SQL combines an explicit interaction protocol with Dual-Track GRPO to separate schema-retrieval and SQL-generation signals.

  • Full Schema Assumption: Supervised fine-tuning and single-turn reinforcement-learning methods generally assume the complete database structure is provided upfront.These approaches rely on static context or terminal execution rewards under full schema observability.
  • Multi-Turn RL: Multi-turn reinforcement-learning research explores trajectory-level optimization, process rewards, tree search, and intrinsic motivation for assigning credit across long interactions.The cited approaches address credit assignment through different trajectory or intermediate-signal designs.
  • TRUST-SQL: TRUST-SQL uses an explicit four-phase protocol and Dual-Track GRPO to disentangle credit assignment between schema retrieval and SQL generation.The framework decomposes trajectories into Schema and Full tracks with independently optimized rewards and masked advantages.

3 Methodology

TRUST-SQL addresses Unknown Schema Text-to-SQL by combining a four-phase tool interaction protocol with Dual-Track GRPO. The protocol grounds SQL generation in verified metadata, while separate masked learning signals optimize schema exploration and generation.

  • Four-phase protocol: The Propose phase forces commitment to verified metadata before SQL generation, reducing hallucinated schema elements.The pilot protocol compares increasingly constrained variants and identifies schema verification as central to suppressing hallucination.
  • Four-phase protocol: 26.4% of EC failures are hallucinations, compared with 2.8% under EPGC, a 9.4× reduction after adding the Propose checkpoint.The pilot also finds schema linking remains a persistent bottleneck, while semantic errors increase from 268 to 330 as hallucinations decline.
  • Problem formulation: TRUST-SQL models Unknown Schema Text-to-SQL as a POMDP with Explore, Propose, Generate, and Confirm phases.The agent receives tool-mediated observations and maintains verified schema knowledge while interacting with the database.
  • Reward design: The method uses execution, format, and schema rewards to evaluate SQL correctness, protocol adherence, and exploration quality.Execution reward distinguishes correct, executable-but-incorrect, and non-executable SQL; format reward requires full protocol adherence; schema reward measures overlap with the ground-truth schema.
  • Dual-Track GRPO: Dual-Track GRPO separates schema exploration from full-trajectory optimization at the Propose boundary.The Schema Track ends at t_propose, while the Full Track spans the complete interaction; each track receives a dedicated reward.
  • Dual-Track GRPO: Token-level masking assigns each track’s advantage only to its active tokens, preventing exploration rewards from crediting generation tokens.Tokens generated after the Propose checkpoint receive zero schema advantage, and the two track losses are combined into the total objective.

4 Experiments

TRUST-SQL is evaluated across five benchmarks and parameter scales under the Unknown Schema setting. It achieves competitive or leading execution accuracy while matching or surpassing schema-prefilled baselines.

  • Experimental Setup: TRUST-SQL uses Qwen3-4B and Qwen3-8B base models, trained with SFT warm-up followed by Dual-Track GRPO.The experiments compare TRUST-SQL with recent baselines across the 3B–8B parameter range.
  • Compact Models: 64.9% greedy and 67.2% majority-voting execution accuracy are achieved by TRUST-SQL-4B on BIRD-Dev, outperforming MTIR-SQL-4B.TRUST-SQL-4B also ranks first on Spider-DK and Spider-Realistic among robustness benchmarks.
  • Mid-Scale Models: 65.8% greedy and 67.7% majority-voting execution accuracy are achieved by TRUST-SQL-8B on BIRD-Dev.The 8B model outperforms all baselines on Spider-Syn and Spider-Realistic.
  • Comparison with Base Models: 30.6% and 16.6% average absolute improvements are obtained by the 4B and 8B variants over their respective base models across five benchmarks.On BIRD, TRUST-SQL-4B improves 35.6% over Qwen3-4B under Unknown Schema.
  • Schema Prefill: Full schema prefilling changes TRUST-SQL performance negligibly on BIRD and Spider and degrades it by 2.4% on Spider-DK for 4B and 1.6% on Spider-Realistic for 8B.The framework achieves these results without pre-loaded schemas, while baselines rely on full schema prefilling.

5 Analysis

The analysis studies reward balancing, schema-reward formulation, scaling, and SFT warm-up. Results favor separated execution and schema tracks, a moderate interaction horizon, and warm-start training.

  • How to Balance Exploration and Generation?: 64.5% is achieved at λ = 0.25, exceeding naive reward aggregation at 58.7% and the pure execution baseline at 60.9%.An excessive λ = 0.375 reduces performance to 54.2% by encouraging prolonged exploration.
  • What Makes a Good Schema Reward?: 64.5% execution accuracy with 5.64 turns is achieved by Sparse + Coupled, versus 52.7% with 6.71 turns for Sparse + Uncoupled and 64.0% with 5.03 turns for Dense + Coupled.The results indicate that coupling is more critical than reward density.
  • Test-Time Scaling: A 10-turn training budget provides the optimal balance between accuracy and exploration efficiency, whereas 12 turns causes instability and declining execution accuracy.Increasing the budget from 8 to 10 turns yields substantial gains on BIRD-Dev.
  • Cold-Start SFT: 59.9% on BIRD and 79.6% on Spider result when Dual-Track GRPO is applied without SFT warm-up, both below the full pipeline.Without SFT initialization, the model learns to query all tables and columns immediately, bypassing genuine active exploration.

6 Conclusion

TRUST-SQL reformulates Text-to-SQL under Unknown Schema as a POMDP with verified exploration and structured training. Across five benchmarks, it improves substantially over base models and matches or surpasses schema-prefilled baselines.

  • Conclusion: TRUST-SQL uses a four-phase protocol and Dual-Track GRPO to ground reasoning in verified metadata and address credit assignment.The protocol’s mandatory checkpoint provides a structural boundary for the training strategy.

Limitations

TRUST-SQL has limitations involving inference cost, database dialect coverage, and a fixed interaction horizon. The authors identify efficiency optimization, broader dialect support, and adaptive turn budgets as future directions.

  • Inference Overhead: Inference requires higher cost than single-turn methods because each interaction step involves a live database call.The authors report that this overhead remains modest in practice but remains relevant for latency-critical deployments.
  • SQLite Dialect Only: Training and evaluation use only SQLite-based benchmarks, leaving PostgreSQL and MySQL extension as future work.
  • Fixed Turn Budget: A fixed maximum turn budget may limit exploration for databases with exceptionally complex schemas.The authors propose adapting the turn budget dynamically to database complexity.

Reproducibility Statement

The paper emphasizes reproducibility through released code, documented dataset construction, and reported training details. Its training recipe is also described as highly constrained and curated relative to evaluated baselines.

  • Reproducibility: The complete source code is released to support full reproducibility.Dataset construction pipelines are documented in Appendix A.
  • Reproducibility: Training hyperparameters and hardware specifications are summarized in Appendix B.The experiments use NVIDIA A100 GPUs.
  • Training Data: TRUST-SQL uses a highly constrained and curated data recipe compared with baselines relying on millions of synthetic samples or large benchmark portions.

A.2 SFT Training Data Construction

The SFT and RL data pipelines construct execution-verified, structurally valid exploration trajectories and selectively retain difficult training questions. Schema supervision is derived through multi-model consensus over referenced tables and columns.

  • SFT Training Data: 9,217 unique source questions are retained from SynSQL-2.5M after filtering for Moderate, Complex, and Highly Complex difficulty.Simpler questions are excluded because they provide insufficient training signal for multi-turn exploration.
  • Annotation Pipeline: The annotation pipeline uses GPT-4.1-mini, GPT-4o-mini, and DeepSeek-R1 to generate complete four-phase interaction trajectories.
  • Annotation Pipeline: Trajectories are retained only when final SQL matches the ground-truth answer and every turn passes the required format check.This execution-verified filtering targets correct and structurally well-formed trajectories.
  • Dataset Statistics: GPT-4.1-mini contributes 43,803 retained samples, representing 61.7% of the dataset.DeepSeek-R1 contributes 9,972 samples across 1,442 unique questions.
  • RL Training Data: RL questions are drawn from BIRD and Spider training sets and filtered using repeated rollouts to retain cases with pass rates below 6/8.
  • Schema Supervision: Schema annotations are accepted only when at least two of three models agree on the tables and columns referenced by the ground-truth SQL.The consensus signal supervises evaluation of the agent’s Propose action.

B Implementation Details

TRUST-SQL trains Qwen3-4B and Qwen3-8B with SFT warm-up followed by Dual-Track GRPO, using a strict four-action protocol for metadata exploration and SQL generation. Its cost analysis shows targeted exploration can achieve strong accuracy with low latency and token use, while Pass@K improves across benchmarks.

  • Training Setup: Qwen3-4B and Qwen3-8B are trained through sequential SFT warm-up and Dual-Track GRPO stages.The 4B model uses synchronous training, whereas the 8B model uses asynchronous training.
  • Output Validation: Every turn must contain exactly one think block, one valid action block, and the content tag required by that action.Violations receive a format score of 0.0 and terminate format reward for the trajectory.
  • Action Protocol: The action space contains four phases: explore_schema, propose_schema, generate_sql, and confirm_answer.These phases query metadata, document verified schema, create SQL, and output the final query.
  • Grounded Generation: The protocol restricts schema knowledge to metadata queries and requires SQL generation to use only verified schema information.Agents can return to exploration or schema proposal when required tables or columns are missing.
  • Inference Cost: 64.9% accuracy is achieved by TRUST-SQL-4B with 0.6 seconds latency and 2.83K tokens per query on BIRD-Dev.This represents a 500× latency reduction and a 113× token reduction compared with CHESS.
  • Pass@K Scaling: All additional benchmarks show monotonic accuracy improvements as Pass@K increases.The persistent Pass@K–greedy gap indicates capability for correct solutions without a fully converged consistent policy.

D.3 Performance on Complex Benchmark (Spider 2.0)

TRUST-SQL is evaluated on Spider 2.0’s challenging SQLite subset, where complex enterprise schemas make full-schema prefilling impractical. Without pre-loaded metadata, it achieves competitive results and illustrates how verified schema exploration supports evidence-driven reasoning.

  • Benchmark Setting: 135 questions in Spider 2.0’s SQLite subset test Unknown Schema under enterprise-grade databases with more complex schemas and larger table counts than standard Spider.The setting is designed to assess whether active schema exploration remains effective when full schema prefilling becomes impractical.
  • Baseline Difficulty: 15.6% execution accuracy is achieved by GPT-4o and DeepSeek-V3, while OmniSQL-7B reaches 10.4%, underscoring the benchmark’s difficulty.These baselines use full schema prefilling.
  • TRUST-SQL Results: 14.8% greedy accuracy and 24.9% Pass@8 are achieved by TRUST-SQL-8B without pre-loaded metadata, surpassing OpenSearchSQL paired with Arctic-7B.The non-saturating Pass@8 curve suggests additional sampling could improve performance further.
  • Case Study: In a BIRD-Dev case study, answering a charter-school question requires grounding the funding predicate in database column values absent from the question and external knowledge.The interaction trace examines how schema availability shapes model behavior.
  • Case Study: The case study suggests Unknown Schema encourages more thorough, evidence-driven reasoning beyond merely discovering the schema.The observed benefit is framed as a qualitative interpretation of the interaction trace.
Loading 2603.16448v2…