MTSQL-R1: Towards Long-Horizon Multi-Turn Text-to-SQL via Agentic Training

1 Introduction

• Existing systems use short-horizon reasoning, generating SQL from current utterances with little context and lacking database, history, and schema checks or feedback-driven correction.The paper associates these gaps with semantically invalid or inconsistent outputs and difficulty correcting earlier incorrect queries.Source: §1 · p. 1
• The comparison frames MTSQL-R1 as long-horizon and multi-turn-trained, unlike prior approaches described as short-horizon and based on prompting or single-turn training.Source: Table 1 · p. 2
• MTSQL-R1 verifies intermediate SQL through database execution feedback and long-term dialogue memory, then iteratively refines queries for executable, cross-turn-consistent outputs.The memory is used for explicit coherence checking.Source: §1 · pp. 1–2
• Training defines the task as an MDP, initializes with self-taught long-horizon trajectories generated using rejection sampling, then applies end-to-end RL with execution-success and memory-coherence rewards.The trajectories are used for warm-start supervised fine-tuning before reinforcement learning.Source: §1 · p. 2
• 1.7B- and 4B-parameter MTSQL-R1 models achieve state-of-the-art results on CoSQL and SParC.The introduction also reports consistent gains in coherence, executability, and generalization.Source: §1 · p. 2

Multi-turn Text-to-SQL

• Pre-LLM methods modeled dialogue and schema context with specialized neural architectures, including prior-SQL, graph-based, and dynamic schema-linking approaches.Source: §2 · p. 2
• Prompt-based ACT-SQL (Zhang et al., 2023) rewrites queries for single-turn input; CoE-SQL (Zhang et al., 2024) edits prior SQL, and both lack database verification and self-correction.Both methods depend on closed-source GPT models.Source: §2 · p. 2

Reasoning Models for Single-Turn Text-to-SQL

• Single-turn reasoning methods use rationale-based SFT or reinforcement learning for logical and execution consistency, but omit dialogue coherence and interactive verification.Source: §2 · p. 2

Long-Horizon Reasoning with RL

• Long-horizon RL systems use environment interaction, but the cited approaches do not operate in multi-turn Text-to-SQL.Source: §2 · p. 2

3.1 Problem Formulation

• The MDP combines database execution and dialogue-memory checks in propose, execute, verify, and refine cycles until checks pass.The state includes dialogue history, schema, current utterance, intermediate SQL, and accumulated execution observations.Source: §3.1 · pp. 3–4

3.2.1 Data Formats

• SFT trajectories follow the MDP, with instructions, tool outputs, and memory prompts masked so training supervises actions and SQL.The format routes execution and memory verification through environment tools and repeats self-correction when either verification fails.Source: §3.2.1 · p. 4

Self-Taught Warm-Start SFT

• Self-taught warm-start SFT iteratively collects correct rollouts, selects trajectories by difficulty, fine-tunes the model, and updates the training subset.It keeps up to two short trajectories for easy cases and samples three representatives from clustered longer trajectories for hard cases.Source: Self-Taught Warm-Start SFT · p. 4

3.3.1 Curriculum RL Training

• RL uses an easy-to-hard curriculum, excluding examples solved in all 20 sampled trajectories and binning remaining examples by success count.The policy interacts with database and memory tools during RL to produce trajectories.Source: §3.3.1 · pp. 4–5

3.3.2 Reward

• The reward combines execution-match and exact-match outcomes with process rewards, including clause-level SQL F1 for proposals and self-corrections.The trajectory reward is a weighted sum of outcome-level and process-level rewards.Source: §3.3.2 · p. 5

3.3.3 GRPO Training with Loss Masking

• GRPO normalizes sampled trajectory rewards into advantages shared across tokens, while loss masking excludes execution outputs and instruction tokens.The token mask retains reasoning tokens for optimization.Source: §3.3.3 · pp. 5–6

4.1 Datasets, Implementation and Baselines

• Evaluation uses CoSQL and SParC, reporting Execution Accuracy (EX) and Exact Match (EM), alongside frontier-model, prompting, agent, and pretrained baselines.CoSQL contains 3,000 multi-turn dialogues; SParC contains 4,298 coherent question sequences.Source: §4.1 · p. 6

Effectiveness: The long-horizon loop drives substantial improvements, while process rewards

• Self-taught Warm-Start SFT increases high-quality long-horizon trajectory coverage, while RL improves EX and EM in both in-domain and out-of-domain settings.Removing either EXECUTE or MEMORY-VERIFY degrades performance; process rewards further help on hard examples compared with sparse outcome-only rewards.Source: §4.2 · p. 6, Effectiveness: The long-horizon loop… · p. 7, Table 3 · p. 8

4.2 RQ1: Effectiveness, Generalization and Efficiency

• MTSQL-R1 achieves the best performance among prior baselines at the same model size across datasets, including in-domain and out-of-domain settings.Table 2 reports in-domain and cross-dataset out-of-domain results.Source: §4.2 · p. 6, Table 2 · p. 7

Effectiveness: The long-horizon loop drives substantial improvements, while process rewards

• With model-generated prior SQL, MTSQL-R1 still outperforms the baseline and widens the performance gap to +5.3 versus +4.7.The predicted-history evaluation tests robustness when the system relies on its own earlier predictions.Source: Effectiveness: The long-horizon loop… · p. 8, Table 6 · p. 8
• MTSQL-R1 lies on the Pareto frontier between execution accuracy and latency, with a maximum latency of about 28 seconds.The paper frames this trade-off as relevant to high-risk enterprise data access and offline processing.Source: Effectiveness: The long-horizon loop… · p. 8, Figure 4 · p. 9

4.3 RQ2: Quantifying Long-Horizon Abilities

• All five measured long-horizon abilities improve after Warm-Start and further after RL; RL especially boosts memory-related abilities, which correlate with better SQL performance.The five abilities are EXECUTE and MEMORY-VERIFY function calling, execution verification, memory-based verification, and generation/self-correction.Source: §4.3 · pp. 8–9, Figure 5 · p. 9

4.4 RQ3: SQL Generation Quality

• Execution and context-coherence errors decrease after training, while Aggregation Drift changes little; case studies show resolution of execution failures and coherence errors.Six remaining execution failures stem from truncation at the 8,000-token cap; aggregation drift is mostly associated with extra-hard queries.Source: §4.4 · p. 9, Figure 6 · p. 9

5 Discussion

• The discussion identifies industrial-scale schema handling and memory-design flexibility as considerations outside the core experimental scope.The proposed extensions include schema retrieval and alternative memory backends.Source: §5 · p. 9
• Schema_Retrieval is proposed before PROPOSE to fetch top-K relevant tables from a vector database instead of loading the full schema.The paper presents this as a way to address industrial-scale schemas, beyond its core experimental scope.Source: §5 · p. 9
• The memory interface is backend-agnostic: the agent uses function calls such as memory_retrieve, while the current implementation stores parsed constraints as text.A vector database with similarity search is proposed for long dialogues or unstructured guidelines.Source: §5 · p. 9

• MTSQL-R1 is the first multi-turn Text-to-SQL agent trained with explicit long-horizon reasoning.Source: §6 · p. 9
• Experiments on CoSQL and SParC show that MTSQL-R1 outperforms all baselines.Source: §6 · p. 9
• The authors identify long-horizon reasoning as valuable for conversational semantic parsing and note its potential for future research.Source: §6 · p. 9

Limitations

• Residual errors include Aggregation Drift and unresolved extra-hard cases.Source: Limitations · p. 10
• Higher latency and token usage than baselines may limit applicability in real-time settings.Source: Limitations · p. 10

F Prompt for LLM-as-Judge Used to Classify SQL Errors 34

• Self-Taught Warm-Start SFT collects 20 rollouts per item, uses difficulty-aware rejection sampling, and fine-tunes on the sampled trajectories.Easy-SQL or consistently correct items retain fewer-interaction trajectories; other items retain longer-interaction trajectories after clustering.Source: F Prompt for LLM-as-Judge Used to… · p. 13

B Dataset Statistics: The SQL Hardness Criteria and Statistics of Two Multiturn Text-to-SQL Datasets

• Queries are divided into easy, medium, hard, and extra-hard levels by SQL complexity, with thresholds for hard and extra-hard queries.The criteria count constructs such as selections, predicates, GROUP BY, set operations, and nesting; dataset comparison and difficulty-distribution tables are referenced.Source: B Dataset Statistics: The SQL Hardness… · pp. 13–14, Table 7 · p. 14, Table 8 · p. 14

C Implementation Details

• The implementation uses Qwen3-1.7B and Qwen3-4B backbones and trains on one node with eight NVIDIA A100 GPUs.Warm-start SFT uses a 5e-6 learning rate; end-to-end GRPO uses a 1e-6 learning rate, batch size 256, and maximum prompt and response lengths of 4000 and 8000.Source: C Implementation Details · p. 14

C.1 Long-Horizon Reasoning as Tools Settings

• The exec_sql tool checks execution results for errors, unexpected NULLs or empty sets, and relevant schema columns, then permits SQL refinement and re-execution.Source: §C.1 · p. 14

TOOL CONFIGURATION

• The memory verifier checks proposed SQL against the current question and dialogue history, including whether columns, conditions, ordering, and joins remain coherent.On a failed check, the instructions call for refining and re-executing the SQL until verification passes.Source: TOOL CONFIGURATION · p. 15

HYPERPARAMETERS FOR RL

• The RL configuration sets a training batch size of 256, maximum prompt and response lengths of 4000 and 8000, and runs on eight GPUs across one node.KL loss is disabled.Source: HYPERPARAMETERS FOR RL · p. 15

C.3 Discussion about the training pipelines

• Computational constraints have prevented exploration of RL-post-training a very large model to generate SFT or distillation data for smaller models.Source: §C.3 · p. 15
• The self-taught SFT-then-RL pipeline trains stably even when the initial model lacks frontier-level reasoning.Source: §C.3 · p. 16

D.1 Warm-Start SFT: Verified Long-Horizon Trajectories Coverage

• As self-taught rounds increase, the model covers a larger share of training samples, yielding more high-quality, natural trajectories for warm-start training.Source: §D.1 · p. 16, Table 9 · p. 16

Difficulty-wise results

• Figure 8 reports CoSQL execution accuracy and token length or interactions by difficulty level.The supplied caption does not state the direction or size of any result.Source: §D.2 · p. 16

Turn-wise results

• Figure 9 reports CoSQL execution accuracy and token length or interactions by dialogue turn.The supplied caption does not state the direction or size of any result.Source: Figure 9 · p. 16, §D.2 · p. 16

D.3 Effectiveness: RL Reward Ablation Studies

• Both reward modules clearly contribute to performance gains in the CoSQL Qwen3-4B ablation.The passage identifies Table 10 as the results source.Source: §D.3 · p. 16

D.4 Generalization: Full Performance Across Different Base Models

• Table 11 presents results across two base models, supporting the framework’s reported robustness and generalizability across architectures and datasets.The caption says the framework consistently outperforms the base models and short-horizon RL baselines.Source: §D.4 · p. 16

Single-turn performance within multi-turn benchmarks

• CoSQL and SParC include first-turn single-turn evaluation, where the method has already shown superior performance.The discussion presents the trained-model evaluation on external single-turn datasets as an additional test.Source: §D.5 · pp. 16–17

OOD single-turn datasets

• A model trained on CoSQL, without training on Spider or BIRD training sets, achieves strong single-turn results on those datasets.The paper describes this as a proof of concept and says the results indicate strong generalization; Table 12’s caption characterizes performance as competitive without domain-specific training.Source: §D.5 · p. 17

D.6 Efficiency-Performance Tradeoff

• With a 2,000-token output cap, performance drops because the agent may not complete its long-horizon reasoning and a final query.The passage says intermediate SQL may have to be extracted before a final query is produced.Source: §D.6 · p. 17
• Reported latency uses basic vLLM without KV-cache optimization, quantization, or speculative decoding, so it is presented as a reference value.The paper expects those techniques could substantially reduce latency.Source: §D.6 · p. 17

D.7 Quantifying Long-Horizon Abilities

• The method achieves competitive performance on single-turn Text-to-SQL datasets without domain-specific training.Table 12 is the performance comparison source.Source: Table 12 · p. 18

D.6 Efficiency-Performance Tradeoff

• The method is reported to achieve a Pareto frontier between inference tokens and performance.Figure 10 and Table 13 are identified as the tradeoff visuals.Source: Figure 10 · p. 17, Table 13 · p. 18

D.7 Quantifying Long-Horizon Abilities

• Figure 11 tracks Qwen3-1.7B long-horizon abilities and related Execution Match performance from the base model to the RL model.The passage identifies the figure’s scope but does not state the direction or magnitude of the changes.Source: §D.7 · p. 17, Figure 11 · p. 18

D.8 RQ4: Training Dynamics

• Reward rises rapidly for easy and medium samples but more gradually for hard samples, while combined outcome and process reward trends upward smoothly.The reward dynamics are shown in Figures 12 and 13.Source: §D.8 · p. 18, Figure 12 · p. 19, Figure 13 · p. 19
• Response length follows a similar pattern to reward, while entropy drops sharply early in training and then stabilizes at a lower level.Response length and entropy dynamics are shown in Figures 14 and 16, respectively.Source: §D.8 · p. 18, Figure 14 · p. 19, Figure 16 · p. 19
• Test-set scores improve substantially early with curriculum levels 1 and 2, then plateau as harder samples yield sparser outcome rewards; dense and process rewards help sustain improvement.The test-score trajectory is shown in Figure 15.Source: §D.8 · p. 18, Figure 15 · p. 19

RESPONSE FROM OURS

• Execution returned [(4,)], and the verifier judged the SQL coherent with the question and history.It checked that the query’s country-specific filter appropriately changed the historical continent grouping and that the result was valid.Source: RESPONSE FROM OURS · p. 27

E.2 Case 2: Base Model Fails but Long-Horizon Reasoning Model Succeeds with the help of “Memory-Verification”-related Action

• The case is labeled medium difficulty and occurs at Turn 2.Source: §E.2 · p. 27

PROMPT FOR QWEN3-4B

• The follow-up asks for the population of “those government forms,” leaving unclear whether to retain the prior life-expectancy condition.The prior turn selected government forms with average life expectancy above 72; the reasoning considers both filtered and unfiltered interpretations.Source: PROMPT FOR QWEN3-4B · pp. 28–29
• The interpretation resolves “those” as the government forms from the previous question whose average life expectancy exceeds 72, then retrieves the population for each of those forms.The SQL filters GovernmentForm through a subquery with HAVING AVG(LifeExpectancy) > 72.Source: PROMPT FOR QWEN3-4B · p. 30

PROMPT FOR OURS

• The OURS prompt requires drafting SQL, executing it at least once, verifying results, and refining the query until execution checks pass.After execution passes, it also requires retrieving memory to check coherence with prior turns.Source: PROMPT FOR OURS · p. 30, RESPONSE FROM OURS · p. 32

RESPONSE FROM OURS

• The OURS response initially sums population across all government forms, but memory verification identifies the missing prior-turn filter and adds HAVING avg(LifeExpectancy) > 72.The correction preserves the earlier condition that the forms have average life expectancy above 72.Source: RESPONSE FROM OURS · p. 34

F Prompt for LLM-as-Judge Used to Classify SQL Errors

• The LLM-as-judge prompt classifies coherence failures as SchemaLinking, JoinPath, AggregationDrift, or ConstraintCoherence, reserving Others for errors not attributable to multi-turn incoherence.The categories cover wrong tables or columns, changed joins, lost aggregation context, and dropped or over-carried constraints or scope.Source: F Prompt for LLM-as-Judge Used to… · pp. 34–35
• The prompt identifies over-carry, result-set scope mismatch, and coreference or ellipsis resolved to incorrect values as cross-turn error patterns.It also assigns “Others” when the prediction is correct, the error is not plausibly due to cross-turn incoherence, or evidence is insufficient to attribute it to the listed categories.Source: F Prompt for LLM-as-Judge Used to… · p. 35
• Tie-breaks prioritize SchemaLinking, JoinPath, AggregationDrift, and ConstraintCoherence, in that order, before “Others.”The rules distinguish wrong table or column sets, divergent relationship edges or keys, lost or mutated aggregation or ranking context, and broken constraint, value, or scope coherence.Source: F Prompt for LLM-as-Judge Used to… · p. 35
• The judge must return valid JSON with a category, a concise evidence-driven rationale, cross-turn signals, and a confidence score.The rationale is specified as 2 to 4 sentences, and the output must contain no extra text.Source: F Prompt for LLM-as-Judge Used to… · p. 35