Source-linked AI summary
Adaptive Test-Time Inference for Text2Cypher with Trace Budgeting and Selective Refinement
Makbule Gulcin Ozsoy
TL;DR
Natural-language database interfaces still produce unreliable queries, while existing test-time methods often spend equal computation on questions of different complexity. The paper introduces difficulty-aware trace budgeting and selective execution-guided refinement for Text2Cypher. Across Gemma-2-9B and Qwen-2.5-7B, the approach cuts inference cost while preserving comparable generation quality and nearly all refinement gains.
Problem
Existing test-time inference strategies for query generation commonly use fixed computation and uniform refinement despite substantial variation in question complexity.
Method
The framework dynamically budgets candidate traces by estimated question difficulty and selectively applies execution-guided correction when additional inference is expected to help.
Results
30.7% lower average generated traces and 21–25% lower wall-clock inference time were achieved with comparable generation quality, while selective refinement reduced execution success by only 0.2–0.5% versus full refinement.
Takeaways & Limitations
Adaptive budgeting and selective refinement reduce unnecessary test-time computation while retaining most of the reliability benefits of broader inference.
Takeaways & Limitations
The approach relies on heuristically estimated difficulty, thresholds, and budgets, and its broader applicability remains limited by evaluation on two generator models and one refinement model.
Abstract
from arXiv · showhide
Large language models have enabled natural language interfaces for structured databases, but generated queries may still contain syntactic errors, violate database schemas, or fail during execution. Test-time inference strategies improve generation reliability without additional training, but existing approaches often use fixed inference budgets and uniform refinement strategies, leading to unnecessary computation across questions with different complexity levels. In this work, we investigate adaptive test-time inference for Text2Cypher and introduce two strategies: adaptive trace budgeting, which dynamically adjusts the candidate generation budget based on question difficulty, and selective execution-guided refinement, which applies correction only when additional inference is expected to be beneficial. Experiments on Gemma-2-9B and Qwen-2.5-7B show that adaptive trace budgeting reduces the average generation budget by 30.7% and wall-clock inference time by 21-25% while maintaining comparable generation quality. Selective refinement preserves nearly all execution success gains of full refinement, reducing execution success by only 0.2-0.5% while avoiding unnecessary refinement for simpler questions. Experiments show that a single correction model (Gemma-4) effectively refines outputs from a different model family, suggesting refinement transfers across families.
1 Introduction
The paper introduces adaptive test-time inference for Text2Cypher to reduce unnecessary computation while preserving query-generation reliability. It combines difficulty-aware trace budgeting with selective execution-guided refinement and evaluates the framework across two LLM families.
- Generated queries can contain syntax errors, violate database schemas, or fail during execution, limiting the reliability of natural-language database interfaces.
- Existing test-time strategies improve robustness without retraining but commonly allocate the same computation to every question despite differing complexity.
- Adaptive trace budgeting estimates question difficulty before generation and dynamically allocates the candidate-generation budget.
- Selective execution-guided refinement applies iterative correction only to predictions likely to benefit from additional inference, avoiding unnecessary refinement for low-complexity queries.
- The framework was evaluated on Text2Cypher using Gemma-2-9B and Qwen2.5-7B, and a shared refinement model generalized across the evaluated generator families.
- 30.7% lower average generation budget and 21-25% lower wall-clock inference time were achieved while maintaining comparable generation quality.
2 Related Work
Related work improves query generation through better inputs, training, test-time candidate selection, adaptive computation, and iterative correction. The paper addresses the limited application of adaptive computation and selective refinement specifically to Text2Cypher.
- Text2Query research has emphasized prompt engineering, schema serialization, context augmentation, grounding, and fine-tuning to improve model inputs or training.
- Test-time inference improves generation quality without retraining through sampling-based candidate generation and selection or aggregation.
- Adaptive computation dynamically allocates sampling budgets by input difficulty or stops when a reliability criterion is satisfied, but has mainly been studied outside query generation.
- Iterative refinement uses feedback and correction to improve outputs, including execution-guided repair methods for Text2SQL and verification loops for Text2Cypher.
- Selective execution-guided refinement applies additional processing only when expected to improve the generated query, combining with adaptive budgeting to reduce inference cost while maintaining generation quality.
3 Methodology
The methodology extends confidence- and structure-aware Text2Cypher inference with adaptive trace budgeting and selective execution-guided refinement. It estimates question difficulty before generation, adjusts candidate budgets across difficulty tiers, and refines only predictions expected to benefit.
- Motivation: Existing Text2Cypher frameworks use fixed inference budgets and may produce empty outputs when structural filtering rejects every candidate.These limitations motivate adaptive budgeting and additional refinement strategies.
- Adaptive Trace Budgeting: Adaptive trace budgeting assigns candidate-generation budgets according to estimated question difficulty.Easy questions receive fewer traces, medium questions an intermediate budget, and hard questions the baseline framework’s full budget.
- Adaptive Trace Budgeting: Difficulty estimation uses a lightweight rule-based feature extractor covering aggregation, superlatives, comparisons, temporal reasoning, negation, traversal, filtering, length, and named entities.The estimator is a proxy rather than a precise predictor and requires no additional model or training.
- Adaptive Trace Budgeting: The estimator maps binary complexity cues to Easy, Medium, or Hard tiers using two score thresholds.Higher tiers are expected to require more inference computation because they correspond to more complex query structures.
- Selective Refinement: Selective execution-guided refinement verifies an initial query and sends verifier feedback to a correction model only when execution fails and refinement is warranted.The policy targets non-easy questions and empty filtered outputs, terminating when verification succeeds or the refinement budget is exhausted.
4 Experiments
Experiments evaluate adaptive trace budgeting and selective execution-guided refinement on a 789-question Text2Cypher benchmark using Gemma-2-9B and Qwen-2.5-7B. Adaptive budgeting lowers inference cost while preserving comparable quality, and selective refinement retains most execution-success gains while reducing unnecessary corrections.
- Experimental setup: The evaluation uses 789 questions across three graph databases and tests Gemma-2-9B and Qwen-2.5-7B from different model families.The setup follows prior Text2Cypher inference studies and reports translation, execution, and computational metrics.
- Difficulty estimation: Difficulty tiers guide adaptive trace budgets and selective refinement, with easy, medium, and hard questions receiving 21, 30, and 45 traces.The rule-based estimator assigns tiers from input difficulty scores, using thresholds τ1 = 0 and τ2 = 2.
- Adaptive trace budgeting: 30.7% fewer generated traces reduces average inference time from 23.4 s to 17.5 s while maintaining comparable performance to fixed-budget inference.Average trace budget falls from 45.0 to 31.2 traces per question; execution success changes from 84.0% to 83.1%, and the difference is not statistically significant.
- Execution-guided refinement: Full execution-guided refinement raises execution success from 84.0% to 92.6% for fixed budgeting and from 83.1% to 92.4% for adaptive budgeting.Paired bootstrap resampling shows significant improvements over corresponding non-refined baselines, with approximately 1.1 s added per question.
- Selective refinement: Selective refinement reaches 91.9% execution success versus 92.4% for refining all predictions while reducing refinement calls.Refining only empty predictions, or empty plus hard questions, performs worse at 87.8% and 88.3%, respectively, because medium-difficulty questions account for about 76% of successful corrections.
- Generalization across models: Across Qwen-2.5-7B, adaptive inference reduces average time from 17.7 s to 14.0 s, while Gemma-4 remains effective as a corrector across both generator families.Selective refinement reaches 91.4% execution success, only 0.2% below refining all predictions.
5 Conclusion
The paper concludes that difficulty-aware trace budgeting and selective execution-guided refinement reduce test-time computation while preserving generation and execution performance. Results across two generator families also suggest that correction models need not share the generator’s model family.
- Main findings: Adaptive trace budgeting reduces generated traces by 30.7% and wall-clock inference time by 21–25% while maintaining comparable generation quality.The strategy dynamically allocates computation according to estimated question difficulty.
- Main findings: Selective refinement reduces execution success by only 0.2–0.5% relative to refining all queries while avoiding unnecessary refinement for low-complexity questions.The approach applies correction when additional inference is expected to provide benefit.
- Cross-family refinement: Using Gemma-4 across Gemma-2-9B and Qwen-2.5-7B suggests effective refinement does not require matching generator and corrector model families.The cross-family observation is supported by improvements in both generator settings.
- Future work: Future work should replace the heuristic difficulty estimator with confidence signals derived directly from generation and extend adaptive inference to other query-generation tasks.These directions define the paper’s stated scope for improving the estimator and broadening evaluation.
Limitations
The study’s limitations concern heuristic budgeting, restricted evaluation scope, and deployment-specific latency measurements. Broader validation and stronger baseline comparisons are needed to assess general applicability.
- Adaptive trace budgeting relies on heuristic linguistic difficulty estimates and heuristically selected thresholds and budgets.These choices were effective on the evaluated benchmark but may require adaptation or calibration elsewhere.
- Comparisons with additional fixed-budget baselines using the same average budget would better isolate difficulty-aware allocation effects.
- The evaluation uses a single Text2Cypher benchmark and only two generator models, limiting evidence for broader task and model applicability.Additional Text2SQL, Text2SPARQL, open-weight, and proprietary-model experiments are proposed.
- The study uses one refinement model, so cross-family refinement transfer is suggestive rather than comprehensively established.Experiments with additional models would provide stronger evidence.
- Absolute inference latency may differ across deployment environments because measurements used the authors’ experimental hardware configuration.The authors expect relative computational savings to remain similar.
A Declaration of AI assistance
The authors used AI-based writing and coding assistants for manuscript wording, auxiliary analyses, evaluation scripts, and implementation discussions. They reviewed and verified the assisted text and code and did not use AI to generate results or references.
- AI assistants supported manuscript wording, auxiliary analysis, evaluation scripts, and implementation discussions.
- The authors reviewed, edited, and verified all AI-assisted text and code.
- AI assistants were not used to generate experimental results or references.