Source-linked AI summary
APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL
Bowen Cao, Weibin Liao, Yushi Sun, Dong Fang, Haitao Li, Wai Lam
TL;DR
Text-to-SQL systems perform well on academic benchmarks but struggle in enterprise environments, where ambiguous schemas and static context leave data meaning difficult to resolve. APEX-SQL uses a hypothesis-verification loop to ground schema linking and SQL generation in real data. It outperforms competitive baselines on BIRD and Spider 2.0-Snow, while answer selection remains a bottleneck.
Problem
Text-to-SQL models that rely on static schema representations struggle with semantic ambiguity in complex enterprise databases.
Source: §1 · p. 1Method
APEX-SQL uses a hypothesis-verification loop, combining hypothesis-driven schema linking with deterministic retrieval of exploration directives for SQL generation.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
Agentic exploration acts as a performance multiplier, particularly for stronger models, while deterministic guidance enhances SQL-generation effectiveness and efficiency.
Source: §5 · p. 9Answer selection remains a bottleneck: DeepSeek-V3.2 achieves 79.17 Pass@8, yet voting captures only 57.50.
Source: §4.4.2 · p. 8
Abstract
from arXiv · showhide
Text-to-SQL systems powered by Large Language Models have excelled on academic benchmarks but struggle in complex enterprise environments. The primary limitation lies in their reliance on static schema representations, which fails to resolve semantic ambiguity and scale effectively to large, complex databases. To address this, we propose APEX-SQL, an Agentic Text-to-SQL Framework that shifts the paradigm from passive translation to agentic exploration. Our framework employs a hypothesis-verification loop to ground model reasoning in real data. In the schema linking phase, we use logical planning to verbalize hypotheses, dual-pathway pruning to reduce the search space, and parallel data profiling to validate column roles against real data, followed by global synthesis to ensure topological connectivity. For SQL generation, we introduce a deterministic mechanism to retrieve exploration directives, allowing the agent to effectively explore data distributions, refine hypotheses, and generate semantically accurate SQLs. Experiments on BIRD (70.65% execution accuracy) and Spider 2.0-Snow (51.01% execution accuracy) demonstrate that APEX-SQL outperforms competitive baselines with reduced token consumption. Further analysis reveals that agentic exploration acts as a performance multiplier, unlocking the latent reasoning potential of foundation models in enterprise settings. Ablation studies confirm the critical contributions of each component in ensuring robust and accurate data analysis. Our code is released at https://github.com/Tencent/APEX-SQL-Project.
1 Introduction
APEX-SQL addresses the gap between strong academic-benchmark performance and weaker performance in enterprise databases, where ambiguous metadata and static context can leave models guessing. It uses agentic exploration to ground schema linking and SQL generation in real data.
1 Introduction
- Enterprise databases often hide column meaning in data values, while static summaries can miss question-specific evidence and add noise and token overhead.These passive approaches can produce syntactically correct but semantically flawed queries.Source: §1 · p. 1
- A hypothesis-verification loop has the agent formulate assumptions from the question and validate them by executing exploratory SQL against actual data.This shifts Text-to-SQL from static translation toward interactive reasoning grounded in database reality.Source: §1 · p. 1
- 70.7% execution accuracy on BIRD-Dev and 53.0% on Spider 2.0-Snow; APEX-SQL surpasses competitive baselines on BIRD and existing agentic systems on Spider 2.0-Snow.On BIRD-Dev, it exceeds OpenSearch-SQL (69.3%) [31] and RSL-SQL (67.2%) [3].Source: §1 · p. 2
- For schema linking, APEX-SQL verbalizes hypotheses, prunes the search space, profiles data in parallel, and synthesizes verified elements to preserve topological connectivity.The pipeline aims to ground column selection in empirical evidence while navigating large databases.Source: §1 · p. 2
- For SQL generation, deterministic retrieval supplies exploration directives so the agent can verify hypotheses, consolidate findings, and resolve implementation ambiguities before query synthesis.The framework applies agentic exploration to both schema linking and SQL generation.Source: §1 · pp. 1–2
2 Related Work
Prior Text-to-SQL work enriches schema prompts or uses agentic database interaction, but these approaches can remain static, untargeted, or inefficient. APEX-SQL’s related-work discussion identifies the need for targeted, coherent exploration.
Static Schema Prompting
- Static schema prompting leaves models dependent on surface metadata, so ambiguous names and hidden data distributions can lead to semantically flawed queries.Candidate generation and consensus or ranking still require models to infer implementation details from ambiguous schemas [8] [18] [27].Source: §2 · p. 2
Pre-processed Schema Augmentation
- Pre-processed schema augmentation adds profiles, query logs, field descriptions, or external knowledge, but its static context may misalign with a novel query and incur token overhead and noise.The cited approaches include AskData [22], TA-SQL [20], and knowledge-enhanced retrieval methods [23] [25].Source: §2 · pp. 2–3
3 Methodology
APEX-SQL uses a hypothesis-verification loop to ground schema linking and SQL generation in database evidence. Its methodology combines logical planning, dual-pathway pruning, profiling, and guided query exploration.
3.1 Problem Formulation
- The pipeline first selects a sufficient set of question-relevant columns, then synthesizes a correct query against that set; real data is accessed during agentic exploration.The schema-linking objective balances recall of relevant columns against the number selected.Source: §3.1 · p. 3, §3.2 · p. 3
3.2.1 Hypothesis Generation via Logical Planning.
- APEX-SQL creates a schema-agnostic logical plan to separate high-level reasoning from concrete schema names, then synthesizes multiple reasoning paths into a master plan.The plan hypothesizes computational steps and latent constraints.Source: §3.2.1 · p. 3
3.2.2 Search Space Reduction via Dual-Pathway Pruning.
- Dual-pathway pruning removes obvious noise while preserving task-relevant columns, retaining ambiguous elements unless confidently rejected as noise and not recognized as critical.It applies deletion and preservation decisions to manageable schema batches.Source: §3.2.2 · p. 3
3.2.3 Hypothesis Verification via Agentic Exploration.
- For schema linking, APEX-SQL verifies hypothesized table and column roles through parallel data profiling, then synthesizes observations into a sufficient schema subgraph.The global synthesis step integrates the pruned schema with empirical observations.Source: §3.2.3 · pp. 3–4, Figure 2 · p. 4
3.3 Agentic Exploration for SQL Generation
- For SQL generation, APEX-SQL interleaves reasoning and exploration, using question-relevant guidance to constrain the search space.Exploration can profile distributions, consolidate findings, synthesize queries, and confirm executable SQL.Source: §3.3 · p. 4, Figure 2 · p. 4
3.3.1 Guidance via Deterministic Retrieval.
- Deterministic retrieval matches operational keywords to a library of SQL best practices, turning the logical plan into focused exploration guidance.For example, detecting RANK activates directives such as checking NULL values in the corresponding column.Source: §3.3.1 · pp. 4–5
3.3.2 Grounding via Agentic Exploration.
- During agentic exploration, APEX-SQL profiles data with exploratory queries and compresses large outputs into statistical summaries, then periodically consolidates observations against the logical plan.These steps support schema-semantic refinement while limiting context saturation.Source: §3.3.2 · p. 5
- After grounding implementation details in evidence, APEX-SQL synthesizes and executes a candidate query, using execution errors for self-correction and confirming semantic fidelity before finalizing.Confirmation checks the SQL against user requirements, observations, and the latest logical plan.Source: §3.3.2 · p. 5
4 Experiments
Experiments evaluate schema linking and SQL execution across BIRD and Spider 2.0-Snow, then isolate exploration, test-time scaling, and component contributions. APEX-SQL reports strong benchmark results, while ablations examine retrieval quality and exploration efficiency.
4.1 Datasets; 4.2.1 Metrics.; 4.2.2 Experimental Setup.; 4.3.1 Metrics.
- The evaluation covers BIRD-Dev and Spider 2.0-Snow, with schema-linking metrics including SRR, non-strict recall, precision, and F1, and SQL-generation metrics including EX and Pass@8.BIRD-Dev includes 1,534 queries across 11 databases; the Spider 2.0-Snow task uses Snowflake SQL and databases averaging over 800 columns. The Spider 2.0-Snow subset uses 120 cases with golden SQLs, while the full execution ground truth covers 547 cases.Source: §4.1 · p. 5, §4.2.1 · p. 5, §4.2.2 · p. 5
4.2.4 Results on Schema Linking.
- APEX-SQL records the highest Strict Recall Rate (SRR) across the three schema-linking settings: 97.28% on the BIRD subset, 87.68% on BIRD full set, and 88.33% on Spider 2.0-Snow.On Spider 2.0, ReFoRCE’s SRR is 35.00%; AutoLink’s SRR is close on some settings but its precision is lower, and its SRR drops on the BIRD full set.Source: §4.2.4 · pp. 5–6, Table 1 · p. 6
4.3.3 Result Analysis.
- 70.7% execution accuracy on BIRD-Dev and 53.03% EX on Spider 2.0-Snow are reported for APEX-SQL; Spider 2.0-Snow Pass@8 is 68.44%.On BIRD-Dev, the paper reports 70.7% versus DAIL-SQL’s 54.3% and RSL-SQL’s 67.2%; Spider 2.0 results use an updated evaluation script, complicating direct comparison with earlier results.Source: Table 2 · p. 6, §4.3.3 · pp. 6–7, Table 3 · p. 7
4.4.1 Effect of Exploration.
- DeepSeek-V3.2’s EX rises from 39.17% without exploration to 57.50% with exploration, while reported exploration gains are larger for stronger models than for GPT-4o.Oracle guidance improves all metrics for GPT-4o, but for stronger models it can improve EX while reducing Pass@8; self-exploration yields higher EX@8 than Oracle for DeepSeek-V3.2 and Gemini-3-Pro.Source: §4.4.1 · p. 7
4.4.2 Test-Time Compute Scaling.
- At k=8, GPT-4o reaches 55.8% Pass@8, exceeding DeepSeek-V3.2’s 52.7% average single-pass EX, while answer selection remains a bottleneck.DeepSeek-V3.2 reaches 79.17 Pass@8 but voting captures 57.50; the authors leave answer-selection optimization to future research. Figure 3 depicts performance scaling for GPT-4o and DeepSeek-V3.2 on the 120-case oracle-schema subset.Source: §4.4.2 · pp. 7–8, Figure 3 · p. 8
4.4.3 Ablation Study on Schema Linking Components.
- Schema-linking ablations show that removing Logical Planning reduces final SRR by 5.0%, while using only selection or only deletion reduces pruning-stage SRR by 4.2% or 16.7%.Combining both pruning pathways yields 97.5% SRR; the method retains an average of 383 columns, about 7k tokens, for verification. The full method trades precision for recall to avoid pruning columns needed for valid SQL.Source: §4.4.3 · pp. 8–9, Table 5 · p. 8
- Removing Agentic Verification reduces SRR by 21.7%; Semantic Linking, Table Exploration, and Global Synthesis contribute drops of 14.2%, 9.2%, and 4.2%, respectively.The reported roles are hypothesizing column roles, resolving ambiguities beyond metadata, and ensuring selected columns form a connected graph.Source: §4.4.3 · p. 9
4.4.4 Ablation Study on SQL Generation Components.
- Guidance improves EX@8 across models, including a 3.75% increase for GPT-4o, while reducing exploration rounds; total query counts remain comparable or increase.The ablation compares guidance against a no-directives baseline and reports EX@8, average rounds, and query count.Source: §4.4.4 · p. 9, Table 6 · p. 9
5 Conclusion
This section summarizes APEX-SQL’s agentic exploration approach, including templates and guidance for schema linking, SQL generation, and iterative data exploration. The directives support evidence-based interpretation and selection of tables, columns, conditions, and joins using verified schema and data relationships.
5 Conclusion
- APEX-SQL combines logical planning and dual-pathway pruning for schema linking, then uses empirical evidence and deterministic profiling guidance to improve SQL generation.The conclusion reports that the framework outperforms competitive baselines on BIRD and Spider 2.0-Snow, and characterizes agentic exploration as a performance multiplier, particularly for stronger models.Source: §5 · p. 9
A Instruction Templates
- The templates ask the model to build a master logical plan and apply negative pruning and positive selection while retaining potentially relevant columns and join identifiers.Selection guidance covers columns useful for joins, filters, grouping, aggregation, or results, and recommends keeping alternatives when their purpose is uncertain.Source: A Instruction Templates · p. 11
- Semantic-linking templates classify tables by query-specific function, while exploration prompts test anticipated roles using data semantics, granularity, connectivity, and data quality.Prompts distinguish target, bridge, and filtering roles and call for checking whether foreign keys are populated and critical columns are usable.Source: A Instruction Templates · p. 12
- Final synthesis instructions require selected columns to connect the tables and cover the query’s functional requirements, including identification, linking, filtering, aggregation, grouping, sorting, and output.When multiple columns might serve the same purpose, the instructions say to keep all of them.Source: A Instruction Templates · pp. 13–14
- The instruction templates guide logical-plan refinement and an EXPLORE–REFINE–SQL–CONFIRM workflow, requiring reasons when previously relevant columns are rejected.Plan steps identify information needs, possible retrieval paths, and keywords; exploration gathers data evidence, refinement updates the plan, and confirmation follows satisfactory SQL execution.Source: A Instruction Templates · pp. 14–15
B Tip Library Details
- The tip library contains 49 SQL-generation tips in 14 categories, extracted by DeepSeek-V3.2 from a randomly selected 10% of BIRD-train cases.The tips are formatted as they appear in case prompts, with an ID, title, and description.Source: B Tip Library Details · p. 17
B.1 Categories Overview
- The 14 categories cover evidence, string matching, output columns, table selection, schema grounding, joins, filters, aggregation, sorting, multi-step logic, syntax, pitfalls, and advanced patterns.The overview also includes column interpretation as a category.Source: §B.1 · p. 17
B.2.1 Category: Evidence Enforcement.
- Evidence-enforcement tips require using evidence-specified columns, formulas, and values exactly, and following evidence when it resolves ambiguous schema terms.They also warn against rephrasing or simplifying evidence-provided formulas.Source: §B.2.1 · p. 17
B.2.2 Category: String Matching.
- String-matching tips use exact equality for known values and reserve LIKE for partial matching, choosing the pattern to match the question’s intent.After exploration finds a value, the guidance favors matching that exact value rather than a pattern.Source: §B.2.2 · p. 17
B.2.3 Category: Output Columns.
- Output-column tips return only requested columns in question order, excluding columns used solely for filtering or sorting unless requested.For “how many” questions, the guidance calls for returning only COUNT.Source: §B.2.3 · p. 17
B.2.4 Category: Table Selection.
- Before using a JOIN, check whether one table contains all required information and prefer a single-table solution when it does.Source: §B.2.4 · p. 17
- Use JOINs only when information spans multiple tables, and avoid unnecessary joins, subqueries, or conditions.Choose the simplest solution that answers the question.Source: §B.2.4 · p. 17
B.2.5 Category: Column Interpretation.
- When exploring data, treat parenthetical text in column names as part of the column name that provides context, not as an additional filter condition to apply.Source: §B.2.5 · p. 17
B.2.6 Category: Schema Grounding.
- Match logical-plan concepts to similarly named columns first, use semantic matching if needed, and verify column existence before use.Direct matching is case-insensitive and may account for variations.Source: §B.2.6 · pp. 17–18
B.2.7 Category: Join Strategy.
- For joins, identify keys from schema relationships, verify matching values and NULLs, choose the join type based on required records, and prefix columns with table aliases.Common key patterns include ID, Code, and Name columns.Source: §B.2.7 · p. 18
B.2.8 Category: Filter Implementation.
The appendix’s exploration tips map question and plan cues to SQL choices, covering filters, aggregation, sorting, multi-step logic, syntax, and common pitfalls. Rule-based retrieval selects relevant tips from evidence, question, plan, and schema patterns, alongside universal tips.
B.2.8 Category: Filter Implementation.
- Exact-value filters use equality, and numeric filters must preserve the question’s inclusive or exclusive comparison semantics.BETWEEN is inclusive on both ends.Source: §B.2.8 · p. 18
B.2.9 Category: Aggregation.
- Aggregation tips distinguish function choice, unique-entity listing from occurrence counting, per-category grouping, and post-aggregation HAVING filters.Ratio and percentage calculations should guard against division by zero and use floating-point division; avoid arbitrary rounding unless requested.Source: §B.2.9 · p. 18
B.2.10 Category: Sorting and Limiting.
- Sorting tips translate ranking keywords into ORDER BY direction, use LIMIT for top-N or extreme-value requests, and exclude NULLs when finding extremes.Source: §B.2.10 · p. 18
B.2.11 Category: Multi-Step Logic.
- Dependent logical-plan steps can be implemented with subqueries or CTEs, with CTEs suggested for clarity in complex multi-step queries.Source: §B.2.11 · p. 18
B.2.12 Category: SQL Syntax.
- Syntax guidance calls for quoting identifiers with spaces, special characters, or reserved words, and using aliases for calculated columns.A SELECT alias cannot be used in the WHERE clause at the same query level; use a subquery or repeat the expression.Source: §B.2.12 · p. 18
B.2.13 Category: Common Pitfalls.
- Common-pitfall tips warn against hard-coding explored values, mishandling NULLs, integer division, and aggregating mixed-entity tables without isolating entity type.Source: §B.2.13 · p. 18
B.2.14 Category: Advanced Patterns.
- Advanced patterns use conditional aggregation for different subsets and window functions for ranking or top-N records per group.Source: §B.2.14 · p. 19
B.3 Tip Retrieval Relationships
- Rule-based tip retrieval combines keyword matching with evidence, question, logical-plan, and schema patterns, while four specified tips apply to every query.Evidence patterns select tips such as TIP002, TIP027, and TIP040 for arithmetic; question and plan patterns trigger other tip groups.Source: §B.3 · p. 19, §B.3.1 · p. 19, §B.3.2 · p. 19
C Case Study
The case study presents Spider 2.0-Snow example sf_bq028 as illustrating how agentic exploration can reveal data quality issues and adapt the query strategy. The passage says the baseline failed while the explored approach succeeded.
C Case Study
- In Spider 2.0-Snow case sf_bq028, agentic exploration discovers data quality issues and adapts the strategy, succeeding where the baseline fails.Source: C Case Study · p. 19
C.1.1 Task Description.
The task asks for the eight most popular NPM packages among latest release versions, ranked by GitHub star count, with their versions.
C.1.1 Task Description.
- Identify the top eight most popular NPM packages among latest releases by GitHub stars and return their versions.Source: §C.1.1 · p. 19
C.1.2 Database Schema.
The schema passage lists three database tables and describes columns for package versions and their linked projects; it also gives the project table’s fields.
C.1.2 Database Schema.
- The project table describes Name as project identifier, Type as project type, StarsCount as star count, and SnapshotAt as snapshot timestamp.Source: §C.1.2 · p. 19
- PACKAGEVERSIONS includes package name, version, system, release and ordinal data in VersionInfo, and publication and snapshot timestamps.Source: §C.1.2 · p. 19
- PACKAGEVERSIONTOPROJECT records package and version, system, linked project type and name, and relationship type.Source: §C.1.2 · p. 20
C.1.3 Direct SQL Generation Result & Analysis.
The direct SQL baseline treated UpstreamPublishedAt as the timestamp for selecting each NPM package’s latest release. Because the field is NULL for most NPM releases, its query returned an incorrect result and scored zero.
C.1.3 Direct SQL Generation Result & Analysis.
- The baseline assumed UpstreamPublishedAt identified each NPM package’s latest release.The assumption was made without verifying whether the field was populated for NPM release versions.Source: §C.1.3 · p. 20
- The SQL filtered out rows where UpstreamPublishedAt was NULL.It used ROW_NUMBER() ordered by UpstreamPublishedAt DESC and included an IS NOT NULL condition.Source: §C.1.3 · p. 20
- Because UpstreamPublishedAt is NULL for the majority of NPM releases, the query missed most packages and returned an incorrect result.The field’s null prevalence was not verified by the baseline.Source: §C.1.3 · p. 20
- The baseline result was Failed (Score: 0).Source: §C.1.3 · p. 20
C.1.4 APEX-SQL Result & Analysis.
APEX-SQL explored the NPM release data before choosing a latest-version strategy, finding that UpstreamPublishedAt was unavailable and verifying VersionInfo:Ordinal as an alternative. It then generated a query using ordinal ordering and GitHub project joins, which succeeded.
C.1.4 APEX-SQL Result & Analysis.
- The exploration checked UpstreamPublishedAt availability and VersionInfo structure before adopting ordinal-based ordering.The release rows sampled showed UpstreamPublishedAt was None, and VersionInfo had no timestamp.Source: §C.1.4 · p. 20
- 43.7% of NPM releases have NULL UpstreamPublishedAt, making the field unsuitable for ordering releases.The count was 3,935,243 out of approximately 9 million NPM releases.Source: §C.1.4 · p. 20
- Exploration confirmed VersionInfo contains an Ordinal field that can order releases, while SnapshotAt is a system snapshot time rather than a release time.The VersionInfo structure was confirmed to contain IsRelease and Ordinal but no timestamp.Source: §C.1.4 · p. 20
- The model verified that PACKAGEVERSIONTOPROJECT project names match PROJECTS.Name before joining them.The verified example format was ’steven-tey/dub’.Source: §C.1.4 · p. 20
- The final APEX-SQL query succeeded (Score: 1).It selected each package’s latest release using VersionInfo:Ordinal, then joined package versions to GitHub projects and ordered by star count.Source: §C.1.4 · pp. 20–21
C.1.5 Key Insights.
The case illustrates exploration as systematic verification of data quality and query assumptions: the model identified a NULL-heavy field, found an alternative ordering signal, and checked it before generating SQL.
C.1.5 Key Insights.
- Exploration revealed that UpstreamPublishedAt is NULL for 43.7% of NPM releases, a data quality issue the baseline could not detect.Source: §C.1.5 · p. 21
- The model identified VersionInfo:Ordinal as an alternative for ordering releases after discovering the NULL issue.Source: §C.1.5 · p. 21
- The model verified its alternative approach before generating final SQL, adapting its strategy to actual database characteristics.The case characterizes exploration as systematic data quality verification rather than guessing.Source: §C.1.5 · p. 21