Source-linked AI summary
AgentSM: Semantic Memory for Agentic Text-to-SQL
Asim Biswal, Chuan Lei, Xiao Qin, Aodong Li, Balakrishnan Narayanaswamy, Tim Kraska
TL;DR
Enterprise Text-to-SQL systems struggle with complex schemas, dialect diversity, and costly, unstable agentic reasoning. AgentSM addresses this gap by storing and retrieving structured semantic trajectories and using designed tools to guide future queries. It reports 44.8% execution accuracy on Spider 2.0 Lite alongside shorter trajectories and improved accuracy, while benefits are limited for some complex reasoning tasks.
Problem
Text-to-SQL systems remain difficult to scale in enterprise settings, while agentic methods incur redundant exploration, planning variance, and high computational cost.
Method
AgentSM stores and retrieves structured semantic trajectories and uses specialized agents and composite tools to guide reasoning on related database queries.
Results
44.8% execution accuracy on Spider 2.0 Lite is reported, with 25% shorter average trajectories and 35% improved execution accuracy in ablation studies.
Takeaways & Limitations
Structured trajectory reuse improves efficiency and scalability for related Text-to-SQL tasks within the same database.
Takeaways & Limitations
Benefits are limited for queries requiring complex mathematical operations or intricate CTE reasoning, which cannot be easily inferred from prior trajectories.
Abstract
from arXiv · showhide
Recent advances in LLM-based Text-to-SQL have achieved remarkable gains on public benchmarks such as BIRD and Spider. Yet, these systems struggle to scale in realistic enterprise settings with large, complex schemas, diverse SQL dialects, and expensive multi-step reasoning. Emerging agentic approaches show potential for adaptive reasoning but often suffer from inefficiency and instability-repeating interactions with databases, producing inconsistent outputs, and occasionally failing to generate valid answers. To address these challenges, we introduce Agent Semantic Memory (AgentSM), an agentic framework for Text-to-SQL that builds and leverages interpretable semantic memory. Instead of relying on raw scratchpads or vector retrieval, AgentSM captures prior execution traces-or synthesizes curated ones-as structured programs that directly guide future reasoning. This design enables systematic reuse of reasoning paths, which allows agents to scale to larger schemas, more complex questions, and longer trajectories efficiently and reliably. Compared to state-of-the-art systems, AgentSM achieves higher efficiency by reducing average token usage and trajectory length by 25% and 35%, respectively, on the Spider 2.0 benchmark. It also improves execution accuracy, reaching a state-of-the-art accuracy of 44.8% on the Spider 2.0 Lite benchmark.
1 INTRODUCTION
AgentSM addresses the difficulty of scaling Text-to-SQL agents in complex enterprise environments by reusing structured semantic memory from prior trajectories. It combines trajectory reuse and composite tools to improve efficiency, consistency, and execution accuracy.
- Enterprise Text-to-SQL remains difficult to scale because complex schemas, diverse SQL dialects, and domain-specific business logic degrade accuracy and efficiency.
- Agentic systems add adaptability but incur redundant exploration, planning variance, and high token, time, and latency costs.
- AgentSM captures and reuses semantically annotated trajectories from prior executions to eliminate redundant exploration and support more consistent reasoning.
- Composite tools couple frequently co-occurring tool sequences, shortening trajectories and improving execution efficiency.
- 44.8% execution accuracy on Spider 2.0 Lite establishes AgentSM's reported state-of-the-art benchmark result.
- 25% shorter average trajectories and 35% improved execution accuracy are reported in ablation studies.
2 BACKGROUND AND PROBLEM FORMULATION
Agentic Text-to-SQL alternates exploration, query generation, and validation, but repeated exploration, rigid strategies, and run-to-run variance limit scalable performance. AgentSM's formulation seeks an accurate executable reasoning trajectory while adapting tool use and trajectory structure to each task.
- 2.1 Agentic Text-to-SQL: Agentic Text-to-SQL workflows alternate among data exploration, SQL generation and execution, and response validation.
- 2.1.1 Repeated exploration.: Fewer than 10–20% of BIRD agent trajectories are distinct, motivating reuse of prior trajectories for new questions on the same database.
- 2.1.1 Repeated exploration.: Initial agent actions repeatedly inspect schema files, issue PRAGMA queries, and read available external knowledge files.
- 2.1.2 Strategy selection.: On the firebase database, vector search was used in only 30% of questions, illustrating that fixed workflows may add irrelevant context and wasted steps.
- 2.1.3 Reducing variance.: Agent trajectories vary across runs because intermediate deviations, including syntax errors, can derail reasoning and change final results.
- 2.1.3 Reducing variance.: Zero-temperature settings cannot reliably enforce determinism, while controlled variance can help discover solutions for unseen or complex tasks.
- 2.2 Problem Statement: The problem formulation selects an executable reasoning trajectory τ from T(q, D, U) that maximizes SQL accuracy Acc(τ).
3 METHODOLOGY
AgentSM combines trajectory retrieval, specialized agents, structured trajectory synthesis, and composite tools to support efficient and consistent Text-to-SQL reasoning. Its methodology reuses exploration traces and packages recurring tool sequences into higher-level operations.
- AgentSM Architecture: AgentSM uses planner and schema linking agents, with the planner integrating SQL generation, execution, reasoning, and refinement.The schema linking agent performs deeper inspection of candidate tables when requested by the planner.
- Trajectory Synthesis and Retrieval: The planner retrieves trajectories from semantically similar questions so future queries can reuse prior exploration instead of repeating it.Trajectory retrieval is designed to reduce redundant exploration, while validation and saving tools support final outputs.
- Trajectory Synthesis and Retrieval: Synthetic questions generate dense exploration traces by prompting an LLM over schemas, external knowledge, and existing questions, then storing agent-produced trajectories for reuse.Question allocation covers every database and distributes additional questions according to the query distribution.
- Trajectory Synthesis and Retrieval: Structured Markdown or JSON trajectories reduce average steps and improve execution accuracy, whereas raw unstructured trajectories provide no improvement over using no trajectory.The comparison is reported on a sample of Spider 2.0 Lite questions.
- Composite Tools: Composite tools combine frequently co-occurring tool sequences into single higher-level operations to reduce unnecessary tool usage and simplify planning.AgentSM constrains composition to semantically coherent reasoning phases and limits composite size to preserve modularity.
4 EVALUATION
AgentSM is evaluated on Spider 2.0, an enterprise-scale benchmark with long contexts, nested schemas, and multiple SQL dialects. It improves execution accuracy and efficiency, while ablations and error analysis identify trajectory reuse, composite tools, schema linking, and domain variation as important factors.
- Main Results: AgentSM achieves 44.8% overall execution accuracy on Spider 2.0 Lite and outperforms SpiderAgent across SQL dialects.It reports a 14.1% improvement over SpiderAgent and over 20% improvement over a standard coding agent.
- Efficiency: AgentSM transitions from exploration to execution earlier and completes tasks with fewer overall steps than a standard coding agent.Structured semantic memory reduces redundant schema exploration and supports more consistent reasoning efficiency.
- Ablations: Trajectory reading reduces average trajectory length by 25% while improving accuracy by 35% on 75 sampled Spider 2.0 Lite questions.The ablation compares examples with and without trajectory reading.
- Ablations: Removing composite tools produces the largest step count and the lowest overall execution accuracy across the sampled instances.The ablation evaluates synthesized trajectories and composite tools as separate components affecting trajectory length and accuracy.
- Error Analysis: With gold tables and columns, AgentSM reaches 55.3% execution accuracy, indicating that schema linking remains a major hurdle in complex databases.Agents often struggle to identify the correct tables and columns when the question and data are ambiguous.
- Error Analysis: Performance varies by domain: common databases reach 60–78% accuracy, whereas heterogeneous databases reach 14–40%.The results indicate stronger performance where schema patterns and reasoning structures recur.
5 RELATED WORK
Related Text-to-SQL work uses supervised pipelines, prompting, and reinforcement learning, while coding-agent research emphasizes iterative tool use and feedback. Agent-memory research distinguishes state-oriented memory from knowledge-oriented memory, motivating structured reuse for agentic tasks.
- Text-to-SQL: Supervised, prompting-based, and reinforcement-learning Text-to-SQL systems improve benchmark performance but often struggle with ambiguous, nested, and relationally complex enterprise schemas.These approaches commonly use schema linking, decomposition, candidate generation, voting, and execution feedback.
- Coding Agents: Coding-agent approaches solve complex tasks through iterative reasoning with external tools, planning, execution, and feedback.Agentic Text-to-SQL applies these patterns through interleaved tool executions and reasoning for schema inspection and query validation.
- Agent Memory: Agent-memory research separates action-oriented memory, which persists agent state, from knowledge-oriented memory, which persists knowledge gained through interaction.Basic systems use the context window as scratchpad-like working memory, while other systems address context limits with memory hierarchies.
6 CONCLUSION
AgentSM reuses structured reasoning trajectories across related Text-to-SQL tasks within the same database. On Spider 2.0 Lite, it combines reduced redundant exploration and improved efficiency with 44.8% execution accuracy.
- Conclusion: AgentSM synthesizes, stores, and retrieves structured trajectories to reuse reasoning steps across related Text-to-SQL tasks within one database.The framework targets redundant exploration and supports larger schemas and more complex queries.
- Conclusion: 44.8% execution accuracy is achieved by AgentSM on the Spider 2.0 Lite benchmark.The conclusion also reports efficiency gains across agent turns, token usage, and latency.