PaperlayerView exact revision on arXiv

Source-linked AI summary

DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning

Shijie Chen, Yu Gan, Yeounoh Chung, Jiani Zhang, Quannan Li, Sravan Babu Bodapati, Cody J. Greer, Yu Su, Fatma Ozcan

arXiv:2609.18135v1cs.CLcs.AIcs.DB

TL;DR

Text-to-SQL systems have commonly trained separate models for schema linking and SQL generation, leaving their joint optimization unaddressed. DualSQL uses two agents sharing one model backbone, database tools, and a robust multi-agent RL framework; on BIRD-Dev, its 4B and 8B models achieve 68.0% and 71.1% EX, respectively, with the 8B model surpassing methods built on 32B models.

  • Problem

    Existing Text-to-SQL systems optimize schema linking and SQL generation separately, leaving the synergy between these interrelated stages unused.

    Source: §1 · p. 1

  • Method

    DualSQL jointly trains two agents sharing one model backbone and database tools with multi-agent reinforcement learning supported by rollout guardrails and REX.

    Source: §1 · p. 2

  • Results

    71.1% EX on BIRD-Dev makes DualSQL-8B a single-model state-of-the-art surpassing methods built on 32B models, while DualSQL-4B reaches 68.0% EX, matching prior 7B baselines.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The results demonstrate that joint multi-agent optimization is an effective path toward stronger Text-to-SQL systems.

    Source: §6 · p. 8

    REX remains a proxy for SQL equivalence checking that can produce false signals, and the reward design lacks fine-grained process-level reward signals.

    Source: Limitations · p. 9

Abstract

from arXiv · show

State-of-the-art Text-to-SQL systems are typically multi-agent pipelines centered around two fundamental tasks: schema linking and SQL generation. However, existing work trains separate models for each task, failing to leverage the synergy between these interrelated tasks. In this work, we propose DualSQL, a new Text-to-SQL system consisting of two agents powered by a single model backbone. The agents share the same model weights and agentic scaffold, enabling joint optimization through a robust multi-agent reinforcement learning (RL) framework. We design three database access tools to facilitate effective multi-step reasoning grounded to interactions with the databases. To improve training and avoid model collapse, we introduce a set of rollout guardrail mechanisms that stabilizes multi-agent RL training, supporting DualSQL to keep improving during training. We also introduce a new SQL correctness metric, robust execution match (REX), to more accurately judge SQL correctness and assign reward signals. Being trained on only 3755 examples, DualSQL-4B achieves an impressive 68.0% execution accuracy on the BIRD development set, matching previous 7B models. DualSQL-8B further improves to 71.1%, outperforming previous state-of-the-art single-model solutions with 32B parameters. These results demonstrate the strength of joint multi-agent reinforcement learning for building high performance Text-to-SQL pipelines.

1 Introduction

Text-to-SQL is difficult because large database context, schema-grounding demands, and semantic ambiguity complicate accurate SQL generation. DualSQL jointly trains schema-linking and SQL-generation agents over database tools, with rollout guardrails and REX supporting its multi-agent reinforcement learning.

1 Introduction

  • Existing Text-to-SQL systems optimize schema linking and SQL generation in isolation, leaving their end-to-end joint optimization unaddressed.Text-to-SQL also requires handling long database contexts, precise schema linking, and ambiguity between natural language and SQL.Source: §1 · p. 1
  • Schema linking and SQL generation share semantic retrieval over relational schemas and reasoning grounded in database context, motivating their treatment as dual facets of one agent.The proposed shared formulation is intended to support shared representations and transfer of reasoning skills.Source: §1 · p. 1
  • DualSQL combines schema linking and SQL generation in a single parameter-shared agentic pipeline that interacts with databases through metadata profiling, full-text search, and SQL execution.Agents dynamically verify and correct reasoning through tool-use feedback rather than following rigid predefined workflows.Source: §1 · p. 2
  • Rollout guardrails—including strict format checking with rollout cutoff and error-focused loss masking—stabilize multi-agent RL, while REX provides deterministic exact-execution-equivalence assessment for cleaner rewards.REX addresses EX's sensitivity to benign formatting variations and insensitivity to duplicates.Source: §1 · p. 2
  • On BIRD, DualSQL-8B establishes a new state-of-the-art for single-model solutions at 71.1% EX, rivaling systems using 32B parameters, while DualSQL-4B achieves a competitive 68.0% EX.Source: §1 · p. 2

2 Related Work

Prior Text-to-SQL work developed context engineering, supervised fine-tuning, multi-agent decomposition, and execution-feedback RL, but continued to optimize pipeline stages separately. DualSQL distinguishes itself through joint multi-agent optimization, database tools, REX, and rollout guardrails for multi-agent RL.

2.1 LLM-based Text-to-SQL

  • Recent Text-to-SQL approaches combine agentic decomposition and RL but optimize different stages separately, neglecting their synergy.The field progressed from in-context learning and context engineering through supervised fine-tuning and execution-feedback RL.Source: §2.1 · p. 2
  • DualSQL's distinguishing contributions are REX for SQL correctness evaluation, a database manipulation toolset, and a multi-agent RL framework for joint pipeline optimization.These contributions address the evaluation, interaction, and optimization components of Text-to-SQL.Source: §2.1 · p. 2

2.2 Reinforcement Learning with Verifiable Rewards

  • Database environments create stability challenges for agentic RL; DualSQL introduces efficient rollouts and guardrails that enhance training efficiency and stability, leading to better performance via prolonged exploration.Prior RLVR stability work included optimization techniques and multi-turn methods for mitigating distribution shifts.Source: §2.2 · p. 3

3 DualSQL Agentic System

DualSQL uses a shared-backbone agentic formulation in which schema linking and SQL generation interact with databases over multiple turns. Its tools support database inspection and query feedback, while the agents jointly model the linked schema and resulting SQL.

3.1 Database Access Tools

  • The SQL executor provides correctness feedback, literal grounding, and schema exploration; full-text search fuzzy-matches schema names and cell values, while the profiler supplies descriptions, inferred foreign keys, and column statistics.Full-text search uses SQLite FTS5 and returns up to five matching values per column.Source: §3.1 · p. 3

3 DualSQL Agentic System

  • Both agents use the same backbone LLM and can interact with the database over multiple turns using the system's tools.The unified agentic formulation is designed for the schema linker and SQL generator.Source: §3.2 · p. 3, §3 · p. 3

3.2 DualSQL Agents

  • The schema linker selects a relevant subset of the database schema to focus the SQL generator's context, and the SQL generator produces a query from the question and linked schema.The two stages are modeled as a joint stochastic process to encourage grounding in correct schema elements and reduce hallucination from irrelevant schemas.Source: §3.2 · p. 3
  • The unified design allows the full Text-to-SQL pipeline to be jointly optimized with multi-agent RL.The system equips its sub-agents with database access tools for multi-turn interactions to understand database context and reason about questions.Source: §3.2 · p. 3, §3 · p. 3

4 Multi-Agent RL Framework for Optimizing Text-to-SQL Pipelines

DualSQL jointly trains schema-linking and SQL-generation agents through one LLM, using asynchronous rollouts, guardrails, task-specific rewards, and GRPO-based policy updates.

4.1 Asynchronous Rollout System

  • Asynchronous rollout treats schema linking and SQL generation as one stochastic process, avoiding inter-stage synchronization and improving rollout throughput.A controller coordinates prompt-level pipeline actors that can run either task alone or chain both tasks.Source: §4.1 · pp. 3–4

4.2 Agentic Rollout Guardrails

  • Rollout guardrails address format errors, malformed actions, and noisy learning signals through strict checks, immediate cutoff, and turn-level loss masking.When a trajectory ends with a format error, loss on prior turns is masked so learning focuses on the erroneous turns.Source: §4.2 · pp. 4–5

4.3 Reward Design

  • Schema-linking rewards prioritize recall, while SQL-generation rewards use query correctness and add a bonus for correct tool-using trajectories.Schema-linking partial credit includes schema- and table-level recall, with a penalty for low precision and additional F1-based credit.Source: §4.3 · p. 5
  • REX evaluates SQL-result equivalence by maximum matching, addressing EX's false positives from duplicates and row ordering and false negatives from column permutations.For questions without required row ordering, REX sorts results by high-cardinality columns before searching for a column permutation that yields a perfect match.Source: §4.3 · p. 5
  • A soft overlong penalty favors concise, error-free trajectories and applies only when the rollout format is correct.The length penalty is weighted by λ_l = 0.2; the reward definition also clips negative rewards to zero.Source: §4.3 · p. 5

4.4 Training Objectives

  • DualSQL uses GRPO with token-mean loss, cliphigher, and sequence-level masked importance sampling; each prompt yields task-specific groups of trajectories for policy updates.The sampling scheme draws G trajectories for each of the schema-linking and SQL-generation tasks.Source: §4.4 · pp. 5–6

5 Experiments

Experiments report DualSQL’s BIRD-Dev results and Spider generalization, then examine joint agent training, REX rewards, rollout guardrails, and reasoning strategies.

5.2.1 In-Domain Evaluation

  • 68.0% EX for DualSQL-4B matched prior 7B baselines, while 71.1% EX for DualSQL-8B surpassed prior methods built on 14B or 32B models.The models were trained on 3,755 examples after filtering and annotation correction.Source: Table 1 · p. 6, §5.2.1 · p. 6, §5.1 · p. 6
  • DualSQL-8B exceeded 90% F1 in schema linking; multi-agent RL raised precision by 1.4% and C-Recall by 1.1%.C-Recall measures the percentage of examples where all ground-truth schema entities are retrieved.Source: Table 2 · p. 6, §5.2.1 · p. 6
  • Only 15.3% of DualSQL-8B errors were attributed to schema linking, and oracle linkers exposed 5.4 points of performance headroom while preserving a 0.8-point EX advantage over DualSQLsql.The authors describe C-Recall as strict and over-pessimistic because SQL-generation-stage tool use can recover missing schema context.Source: §5.2.1 · p. 7, Table 3 · p. 7

5.2.2 Out-of-domain Evaluation

  • Joint multi-agent RL further improved Text-to-SQL performance on both BIRD and Spider, where tool-use and reasoning capabilities learned during training generalized.The models were trained on BIRD training data and evaluated on Spider test data.Source: Table 4 · p. 7, §5.2.2 · p. 7

5.2.3 Ablation Studies

  • On BIRD-Dev, moving from single-turn to multi-turn agentic Text-to-SQL gained 2.3% EX and 5.4% REX; multi-agent training added 1.5% EX and 1.9% REX.The comparisons used the same training data and reward signal.Source: §5.2.3 · p. 7
  • REX as the reward signal led to better RL outcomes on both EX and REX; EX-reward training struggled on questions requiring DISTINCT and ORDER BY.Those are scenarios where EX fails to distinguish SQL correctness.Source: Table 5 · p. 7, §5.2.3 · p. 7
  • Rollout guardrails prevented mode collapse before convergence by guiding the model to unlearn erroneous behavior at error turns and maintain format correctness.Without guardrails, malformed actions could be retained in high-reward trajectories, reinforced, and lead to behavioral collapse.Source: Figure 2 · p. 8, §5.2.3 · p. 8

5.2.4 DualSQL Reasoning Strategies

  • DualSQL’s strategies combine interactive schema linking through metadata inspection and row sampling with execution-feedback-driven query refinement.Agents resolve lexical mismatches and verify schema elements before SQL generation; execution feedback supports adjustments such as type casting or string manipulation.Source: §5.2.4 · p. 8

6 Conclusion

DualSQL unifies schema linking and SQL generation in two agents powered by one LLM backbone, jointly optimized with reinforcement learning. Its BIRD results are competitive with or surpass larger single-model systems, and the authors identify reward design, training data, and longer trajectories as future-work directions.

6 Conclusion

  • DualSQL-4B reaches 68.0% EX on BIRD-Dev, while DualSQL-8B achieves 71.1% EX, surpassing methods built on 32B models.The authors conclude that joint multi-agent optimization is an effective path toward stronger Text-to-SQL systems.Source: §6 · p. 8
  • DualSQL unifies schema linking and SQL generation as two agents sharing one LLM backbone and uses database tools for joint RL optimization.The system is optimized with a robust reinforcement learning framework.Source: §6 · p. 8
  • Future work includes finer-grained reward signals and inter-agent credit assignment, scaling training data, and improving multi-agent RL for larger models and longer trajectories.The authors identify these as three promising directions.Source: §6 · pp. 8–9

Limitations

The authors identify limitations in reward design and restrict database access to read-only; they also describe the backbone choice and the capabilities their training recipe requires.

Limitations

  • The experiments use Qwen3 because it supports extended reasoning and tool calling and is supported by VeRL; the authors believe the recipe applies to other backbones with those capabilities.Source: Limitations · p. 9
  • REX remains a proxy for SQL equivalence and can produce false reward signals.The outcome-based reward also lacks fine-grained process-level signals that could matter for stronger agents.Source: Limitations · p. 9
  • DualSQL agents have read-only database access during training and evaluation, and production deployment warrants precautions against resource exhaustion and prompt injection.Source: Limitations · p. 9

Artifact and Data Usage

The work uses datasets, code, and models from prior research under permissive licenses for academic research.

Artifact and Data Usage

  • The work uses BIRD and Spider datasets, VeRL code, and Qwen3 models, each identified as CC BY-SA 4.0 or Apache 2.0 licensed.Source: Artifact and Data Usage · p. 9

AI Assistant Usage

The authors report using AI assistants for language editing, with all content reviewed by the authors. Tool implementation details describe asynchronous access, result formatting, and strategies for profiling, searching, and limiting query outputs.

AI Assistant Usage

  • AI assistants were used to edit and improve the paper’s language, and the authors reviewed all presented content.Source: AI Assistant Usage · p. 9

A More Tool Implementation Details

  • Tool results are serialized as JSON, and tools run as an asynchronous web service to isolate database latency from GPU inference.This design aims to prevent database access from bottlenecking GPU utilization during rollout.Source: A More Tool Implementation Details · p. 12
  • The SQL executor limits regular query output to 10 rows and truncates cells over 150 characters, while preserving full schema-query results.Source: A More Tool Implementation Details · p. 12
  • Full-text search uses SQLite FTS5 inverted indices for low-latency fuzzy matching and returns matched schema elements plus up to five distinct values per column.Source: A More Tool Implementation Details · p. 12
  • The database profiler precomputes table and column descriptions, inferred foreign-key relationships, and column statistics for runtime use.This offloads schema analysis so the agent can focus on query logic.Source: A More Tool Implementation Details · p. 12

B More Training Details

Multi-agent training improves performance over a SQL-generation-only baseline across question difficulties, with the largest reported gain on challenging questions. During training, the multi-agent model catches up with and then outperforms its single-agent counterpart.

B More Training Details

  • DualSQL-4B trains for 100 steps, while DualSQL-8B trains for 300 steps over about 4.5 days on a four-node cluster with eight H100 80GB GPUs per node.Source: B More Training Details · p. 12
  • After starting slightly worse because of limited initial schema-linking accuracy, DualSQL-8B catches up and consistently outperforms DualSQLsql during the rest of training.The passage identifies this trend as evidence of the multi-agent framework's effectiveness.Source: B More Training Details · p. 12

C Performance Breakdown by Difficulty

  • 3.0% EX improvement on challenging questions for the 8B model, with multi-agent training improving performance at every difficulty level over DualSQLsql.The comparison uses DualSQLsql trained only on SQL generation with the same recipe.Source: C Performance Breakdown by Difficulty · p. 12

D Ablation on Tools

Tool ablations show that removing any database tool reduces performance, especially SQL execution; the supplied material also describes REX, its matching procedures, and examples of agent behavior.

D Ablation on Tools

  • Removing SQL execution causes a 2.5% drop in EX and a 6.8% drop in REX during SQL generation; removing any tool reduces overall performance.Table 8 reports average tool use and overall performance.Source: D Ablation on Tools · p. 12, Table 8 · p. 15

E Robust Execution Match

  • REX scores similarity between ground-truth and sample dataframes, using configurable row ordering, content-based column matching, size penalties, and flexible handling of type differences.A score of 1.0 denotes equivalence; unmatched columns are paired using an LSAP solver.Source: E Robust Execution Match · p. 13

Strict Row Ordering

F.1 Schema Linking Agent Prompt / F.2 SQL Generation Agent Prompt

  • The appendix provides system and user prompt materials for both the schema-linking and SQL-generation agents.Source: §F.1 · p. 15, §F.2 · p. 15

G.1 Schema Linking Agent

  • Schema linking grounds table and column choices in profiled samples and descriptions, then chains search, profiling, and exploratory SQL to prune schemas and check joins.The examples include checking date formats and overlapping entity IDs.Source: §G.1 · p. 15, Figure 8 · p. 19, Figure 9 · p. 20

G.2 SQL Generation Agent

  • SQL generation favors minimal queries aligned with the question, such as using ORDER BY and LIMIT 1 to return only the requested entity.Source: §G.2 · p. 15, Figure 10 · p. 21
  • When execution returns NULL, the agent considers database-specific causes and uses CAST(... AS REAL) to avoid SQLite integer-division truncation.Source: §G.2 · p. 15, Figure 11 · p. 22

Related papers

Loading 2609.18135v1…