Source-linked AI summary
ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models
Xinmei Huang, Jie Song, Peng Li, Fuxin Jiang, Jing Zhang, Tieying Zhang, Jianjun Chen, Chenming Liu, Tao Yang, Maoyin Liu, Wenda Li, Hong Chen, Cuiping Li
TL;DR
Correcting SQL reliably remains difficult because real-world queries involve complex semantics and existing approaches can propagate errors through linear refinement. ACTS-SQL uses training-free, plan-guided tree-structured debugging with alternative repair strategies, backtracking, and diagnostic tools, achieving state-of-the-art benchmark performance and higher production execution accuracy.
Problem
Real-world SQL correction requires handling complex semantics, costly supervision, multiple dialects, and error propagation in predominantly linear refinement workflows.
Method
ACTS-SQL formulates training-free SQL debugging as plan-guided tree search with alternative repair branches, backtracking, and lightweight diagnostic tools.
Results
The framework achieves state-of-the-art BIRD-Critic performance, surpassing the previous best system by 9.42%, and raises industrial execution accuracy from 36.77% to 53.61%.
Takeaways & Limitations
Structured, plan-based SQL correction is effective across dialects and LLM backbones and suitable for production Text-to-SQL systems under practical latency and cost constraints.
Takeaways & Limitations
Existing linear correction processes can accumulate errors and cause semantic drift when clause-level modifications are repeatedly layered onto SQL queries.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have been increasingly adopted in Text-to-SQL systems, yet SQL errors remain a major obstacle in real-world Text-to-SQL inference pipelines. Existing SQL correction approaches either rely on large-scale, high-quality training data with substantial overhead, or adopt single-path agentic workflows that are brittle to early mistakes and prone to error propagation. To develop a practical SQL correctness system for industrial scenarios, we present a training-free framework that formulates SQL correction as a plan-guided, tree-structured debugging process. By maintaining multiple correction strategies and enabling backtracking, the framework mitigates error accumulation during iterative refinement. We further integrate execution-based verification and clause-level diagnostic tools to support strategy pruning and precise error localization. We evaluate the system on the BIRD-Critic benchmark and observe consistent accuracy gains over strong LLM backbones and representative agent-based baselines, achieving a 9.42% improvement over the previous state-of-the-art method. The framework is also deployed in the Torch Log Service (TLS) of Volcano Engine to support an online Text-to-TLS API. In production, it improves execution accuracy from 36.77% to 53.61% on real user queries with a representative strong LLM backbone (GPT-5). These results demonstrate the effectiveness and stability of our approach in real-world deployments.
1 INTRODUCTION
ACTS-SQL addresses the fragility and limited practicality of single-path, training-dependent SQL correction with a training-free, tree-structured debugging framework that supports branching and backtracking. It achieves state-of-the-art benchmark performance and improves execution accuracy in industrial Text-to-TLS deployment.
- Motivation: Single-pass Text-to-SQL generation remains error-prone, producing syntax, join-path, aggregation, and semantic errors, especially in heterogeneous databases with customized SQL dialects.Claude-4 achieves only 25.78% accuracy on Spider2.0.
- Limitations of Existing Methods: Linear agentic correction is vulnerable to early implicit assumptions, error accumulation, and irreversible clause changes that can cause semantic drift from user intent.Repeated local modifications can have global effects on SQL result sets.
- Our Proposal: ACTS-SQL formulates SQL correction as a tree-structured debugging process with multiple critic strategies, semantic hypotheses, and backtracking to earlier steps.The approach replaces single irreversible refinement with alternative correction paths.
- Evaluation & Application: 9.42% improvement over the previous best system demonstrates state-of-the-art performance on the BIRD-Critic benchmark.In industrial Text-to-TLS deployment with GPT-5, execution accuracy improves by 16.84 percentage points on real user queries under practical latency and cost constraints.
- Contributions: The framework uses a revision-plan–based debugging paradigm and specialized operators for ambiguity detection, sub-query decomposition, and schema inspection.These operators target both syntactic and semantic challenges in real-world SQL debugging.
2 RELATED WORK
Prior Text-to-SQL work improves generation and debugging through compositional reasoning, static analysis, AI refinement, and decomposition-based tools, but most refinement remains linear. These single-path approaches limit exploration and propagate errors, motivating tree-structured, multi-branch debugging for SQL’s dependent clauses.
- Agentic and compositional reasoning: Compositional Text-to-SQL methods decompose generation into hierarchical CTE or subquery subgoals, improving interpretability and controllability while typically refining SQL linearly.LearNAT, MAG-SQL, MAC-SQL, and CoE-SQL use this paradigm, but their refinement processes generally follow one-pass or linear trajectories.
- Agentic and compositional reasoning: Alpha-SQL organizes schema linking, generation, revision, and other subtasks hierarchically, but its tree primarily represents stages rather than alternative correction branches.Its revision component still refines SQL along a single trajectory.
- SQL debugging approaches: Static SQL debugging tools use AST-based rules to identify anti-patterns, type inconsistencies, and inefficient structures, but lack user-intent awareness and adaptive query repair.Examples include UPM, Visual Expert, SQLFluff, and SQLCheck.
- SQL debugging approaches: AI-based refinement methods iteratively inspect, execute, and modify SQL, using perturbed variants, execution-guided candidate selection, and decomposition-based feedback for more precise correction.Representative approaches include SQLFixAgent, CSC-SQL, Tool-Assisted Agent, CoE-SQL, Qr-Hint, SQLCritic, and DAC.
- Limitations and motivation: Most existing frameworks rely on linear refinement loops, suffering from error propagation and limited exploration when debugging complex queries.This limitation also affects cyclic code-refinement systems, whose repair trajectories remain single-threaded.
- Limitations and motivation: SQL’s symbolic compositionality and dependencies among SELECT, JOIN, and WHERE clauses make single-path refinement unstable, motivating explicit revision plans with tree-structured multi-branch exploration.The proposed direction integrates dynamic tool feedback and interpretable refinement trajectories.
3 SYSTEM OVERVIEW
ACTS-SQL structures SQL correction as a revision-plan-guided, tree-structured debugging workflow coordinated by a Central Agent and specialized tools. The system dynamically explores and evaluates branches using ambiguity analysis, schema inspection, execution feedback, syntax repair, and final SQL generation.
- Plan-driven workflow: The workflow begins with an LLM-generated revision plan represented as a tree of tool invocations, interpretable steps, and potential branching paths.The plan is produced from the user query and faulty SQL and serves as a high-level debugging blueprint.
- Plan-driven workflow: The Central Agent generates the initial plan, selects branches when decisions arise, and evaluates candidate SQL using the query, schema, faulty SQL, and execution history.It coordinates the entire correction process and decides how the workflow proceeds after tool calls.
- System components: The framework combines five tools: Detect Ambiguous, Inspect Columns, Run SQL, Split and Fix Syntax Error SQL, and Generate SQL.Together, these tools provide language-model reasoning, execution feedback, and hybrid reasoning-execution operators for iterative SQL refinement.
- Specialized tools: Detect Ambiguous identifies unclear query phrases and plausible interpretations, such as alternative meanings of “active users” in a StackOverflow database.The tool takes the raw user query as input and produces ambiguity hypotheses for subsequent branch selection.
- Specialized tools: Inspect Columns retrieves sample column values, Run SQL executes arbitrary queries for validation, and Split and Fix Syntax Error SQL decomposes faulty SQL into independently executable subqueries.The syntax-repair tool feeds subquery results and the subqueries back to the LLM to produce a revised query, while Generate SQL serves as the tree leaf that outputs corrected SQL and its execution result.
4 ACTS-SQL
ACTS-SQL formulates SQL correction as a plan-guided, tree-structured debugging process in which tool invocations create alternative reasoning paths. A Central Agent selects, validates, and backtracks across these paths using semantic diagnostics, database execution, and clause-level syntax analysis.
- Tree-Structured SQL Debugging: The debugging workflow represents each tool invocation as a tree node and each possible follow-up action as an edge.The system constructs an initial revision plan from the user issue, SQL dialect, schema, problematic query, and execution result, then starts at the root node.
- Tree-Structured SQL Debugging: Central Agent selects promising branches from intermediate results, validates generated SQL candidates, and backtracks when validation fails.Execution terminates when a leaf candidate passes validation; otherwise, the agent may return to any plan node and continue revision.
- Diagnostic Tools: Split and Fix Syntax Error SQL decomposes invalid queries into independently executable flat SELECT units for parallel, clause-level syntax diagnosis.The resulting diagnostic feedback enables revised SQL that resolves syntax errors while preserving intended semantics.
- Diagnostic Tools: Detect Ambiguities identifies unclear natural-language phrases and produces 2–3 schema-grounded interpretations that become semantic branches.These branches expose implicit ambiguity and support semantic-error resolution beyond SQL execution alone.
- Diagnostic Tools: Run SQL and Inspect Column Format provide database feedback for testing semantic assumptions, validating subqueries, and probing column values.Run SQL returns resulting tuples, while Inspect Column Format samples values through a fixed query template.
- Diagnostic Tools: The Generate SQL tool instructs the LLM to minimally edit the original query, avoid speculation, and confirm the candidate by execution.This evidence-grounded design aims to produce a faithful correction rather than an overly rewritten alternative.
5 EXPERIMENTAL EVALUATION
The evaluation examines the system’s effectiveness, robustness, and practicality through benchmark results, ablation studies, real-world TLS testing, and representative case studies.
- Experimental Evaluation: The experiments assess overall performance gains, robustness, and practical applicability.The evaluation is designed as a comprehensive assessment of the proposed system.
- Experimental Evaluation: Benchmark results quantify overall performance gains, followed by ablations analyzing key component contributions.
- Experimental Evaluation: A real-world Text-to-SQL benchmark evaluates performance under practical system constraints.
- Experimental Evaluation: Representative case studies illustrate the system’s behavior in practice.
5.1 Evaluation on Benchmarks
On BIRD-CRITIC, a challenging benchmark for systematic SQL debugging across PostgreSQL and multiple dialects, ACTS-SQL outperforms raw LLMs, agent-based correction methods, and a fine-tuned Text-to-SQL model. The results support structured, execution-guided correction over one-shot generation, linear refinement, or dialect-specific fine-tuning.
- Benchmark: BIRD-CRITIC evaluates SQL issue debugging on real-world user-reported errors across a PostgreSQL subset and an open multi-dialect subset spanning four database dialects.The benchmark targets systematic SQL debugging rather than SQL generation.
- Results: ACTS-SQL consistently outperforms strong raw and reasoning-based LLMs on both BIRD-CRITIC-PG and the multi-dialect setting.The paper attributes this advantage to explicit error localization and iterative verification for structured program repair.
- Results: ACTS-SQL achieves consistently better performance than existing agent-based SQL correction methods that rely on linear refinement and execution feedback.Its structured process supports backtracking and avoids accumulating errors after an incorrect early semantic assumption.
- Results: A Text-to-SQL model fine-tuned on public SQLite-based datasets shows limited effectiveness on BIRD-CRITIC, particularly in the multi-dialect setting.The comparison indicates limited transfer from single-dialect, generation-oriented fine-tuning to diverse SQL debugging scenarios.
- Overall findings: Overall, ACTS-SQL outperforms strong base models, representative agent-based approaches, and a fine-tuned Text-to-SQL model across diverse SQL dialects.The results favor structured, execution-guided correction over linear refinement or dialect-specific fine-tuning.
5.2 Ablation Study
The ablation study shows that tree-structured planning with branching and rollback, execution-based feedback, and ambiguity handling are important for reliable SQL correction on BIRD-CRITIC Open and PG. Removing these components degrades performance, with SQL execution contributing the largest observed drop.
- Structured planning: A linear, single-path refinement variant substantially reduces performance on both BIRD-CRITIC Open and PG compared with tree-structured planning.The full method supports branching and rollback, whereas the linear variant generates SQL and calls tools sequentially along one path.
- Structured planning: Tree-structured plans improve robustness by deferring commitment, recovering from invalid intermediate decisions, and avoiding error accumulation from early mistakes.Branching enables alternative refinement paths instead of committing to a single correction hypothesis at each step.
- Critic tools: Removing either the SQL execution tool or ambiguity detection tool consistently degrades performance on both BIRD-CRITIC Open and PG.The execution ablation removes run SQL and inspect columns, while the other ablation removes user-query ambiguity detection.
- Critic tools: Removing the SQL execution tool causes a substantially larger performance drop than removing ambiguity detection, underscoring the critical role of execution feedback.Execution feedback guides refinement by providing intermediate evidence for reliable SQL correction.
5.3 Evaluation on Real-world Data
On the industrial TLS dataset, ACTS-SQL improves real-world Text-to-TLS correctness through execution-aware refinement, while maintaining moderate latency and robustness across backbones and task settings. These results support its use as a practical, training-free correction layer in industrial pipelines.
- Dataset: TLS is an industrial, SQL-like log-analysis dataset spanning 288 tables with an average of 21.33 columns per table.It contains real user natural-language queries paired with ground-truth TLS queries executed in production, and differs substantially from mainstream SQL dialects.
- Effectiveness: Execution-aware critic refinement substantially improves end-to-end Text-to-TLS performance and outperforms existing LLM-based critics and agent-based baselines.ACTS-SQL achieves the highest accuracy across evaluated TLS settings and identifies and corrects semantic and logical issues in generated queries.
- Efficiency: ACTS-SQL achieves its performance gains with moderate, comparable latency rather than prohibitive execution overhead.Its runtime remains comparable to other agent-based baselines, supporting practical usability.
- Robustness: ACTS-SQL remains effective across multiple LLM backbones and in pure Text-to-TLS generation from scratch.The refinement mechanism is therefore not tied to a specific model architecture, training condition, or pre-existing erroneous query.
- Deployment implications: As a training-free post-generation module, ACTS-SQL can improve query correctness for automated data cleaning and online inference without annotation or fine-tuning costs.Its accuracy gains, moderate latency overhead, and stable cross-backbone performance support deployment as a general-purpose correction layer in industrial systems.
5.4 Case Study
The case study shows that underspecified “monthly sales” semantics trap linear SQL refinement in an incorrect annual-aggregation interpretation. ACTS-SQL resolves this ambiguity through parallel hypothesis exploration, execution validation, pruning, and backtracking to generate correct SQL.
- Semantic ambiguity: The query’s underspecified “monthly sales” phrase admits multiple plausible interpretations and creates a common SQL-correction failure mode.The task asks for the top-3 products with the highest monthly sales in 2022 for each product category.
- Linear refinement failure: The base model aggregates sales across all of 2022 and ranks products by annual totals, omitting explicit month-level granularity.This reflects the assumption that monthly sales can be derived from annual aggregation.
- Linear refinement failure: Linear agents preserve the flawed annual-aggregation assumption, repeatedly applying local SQL changes while producing syntactically valid but semantically incorrect queries.They modify aggregation functions, grouping keys, or temporal filters without revisiting the interpretation of “monthly sales”.
- Branching and backtracking: ACTS-SQL branches at the ambiguous semantic decision, independently executes alternative hypotheses, and backtracks when the annual branch yields inconsistent results.An alternative branch introduces explicit month-level aggregation before computing product-level statistics, enabling convergence to the correct interpretation and SQL.
5.5 Cost Analysis
ACTS-SQL maintains a compact workflow with moderate interaction rounds while incurring model-dependent latency, measured through API calls, visible output tokens, and end-to-end runtime. Reasoning models take longer primarily because of model-side inference time, not additional framework interactions.
- API and token cost: 4.97 LLM API calls per instance and 8,984.5 visible tokens on average characterize ACTS-SQL’s moderate interaction cost, measured with GPT-o3.Calls mainly support agent decision-making, ambiguity detection, and iterative SQL refinement.
- Latency: Around 3 minutes of average end-to-end runtime occurs with non-reasoning GPT-5, versus approximately 4–5 minutes with reasoning GPT-o3.The difference is primarily attributed to reasoning-model inference time rather than additional API calls or tool interactions.
5.6 Application
ACTS-SQL is deployed in Volcengine Log Service through the Text2TLS interface to support real-world log analysis queries. Its TLS repair capability uses the proposed tree-structured debugging framework to automatically correct erroneous TLS queries during execution.
- Production deployment: ACTS-SQL is deployed in Volcengine Log Service’s Text2TLS interface, where its TLS repair capability automatically corrects erroneous TLS queries during execution.The repair capability incorporates the proposed tree-structured debugging framework and supports real-world log analysis queries.
6 CONCLUSION
The paper presents a training-free, plan-guided tree-structured SQL correction framework that explores alternative repairs and recovers from incorrect decisions. Evaluations and industrial deployment show reliable, robust, and production-suitable performance without additional supervision or fine-tuning.
- Framework: The training-free framework formulates SQL debugging as a tree-structured, plan-guided process for exploring alternative repair strategies and recovering from incorrect decisions.It avoids additional supervision and model fine-tuning.
- Framework: Lightweight tools for ambiguity analysis, execution feedback, and clause-level validation improve SQL correction reliability without additional supervision or model fine-tuning.
- Evaluation and deployment: The method achieves state-of-the-art performance across multiple SQL dialects on the public BIRD-Critic benchmark.
- Evaluation and deployment: Industrial deployment significantly improves end-to-end execution accuracy with practical latency and cost, supporting production Text-to-SQL use.The framework is reported as robust across different LLM backbones and suitable for online SQL generation.