PaperlayerView exact revision on arXiv

Source-linked AI summary

Conversational Task Disambiguation over Tabular Data: Leakage-Aware Formulation, Benchmark Suite, and Training

Nafiseh Ghoroghchian, Luis Scoccola, Tina Sedaghat, Omid Vaheb, Hannah Chen, Dino D'Agostino, Keyvan Golestan

arXiv:2610.10740v1cs.LGcs.AIcs.CL

TL;DR

Task disambiguation over tabular data is difficult to evaluate because success mixes asking and solving, simulators may leak information, and datasets lack shared access representations. The paper introduces a leakage-aware framework and AmbiTab, and reports improved asking metrics across six corpora while substantial gaps persist.

  • Problem

    Evaluation and training lack a leakage-aware foundation because success mixes asking with solving, simulated oracles may reveal information beyond what real users would, and datasets lack shared representations of ambiguities and access boundaries.

    Source: §1.1 · pp. 1–2

  • Method

    The paper formalizes ambiguous verifiable tasks, separates asking from solving and oracle from verifier, and applies the framework in AmbiTab across six text-to-SQL datasets.

    Source: §1.2 · pp. 2–3

  • Results

    Question-F1 training improves precision and recall across all six corpora, but substantial gaps in question quality and task success remain.

    Source: §5 · p. 9

  • Takeaways & Limitations

    Separating asking, solving, oracle, and verifier while measuring precision and recall alongside success distinguishes failures in acquiring information, using it, and respecting disclosure boundaries.

    Source: §5 · p. 9

    Because interactions are simulated, the results do not establish effectiveness with real users.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Conversational task disambiguation over tabular data uses dialogue to resolve missing information about a user's intended task before producing a solution over tables or databases. Existing evaluation and training lack a leakage-aware foundation. Task success mixes the agent's disambiguation and solution-generation capabilities and can also reflect oracle leakage, that is, information that a user simulator reveals beyond what a real user would. Existing datasets also lack a shared representation of ambiguities and access boundaries. We introduce the notion of an ambiguous verifiable task, which formalizes ambiguities and resolutions, decomposing the agent into an asking policy and a solution policy, and the environment into an oracle and verifier. This framework provides baselines and metrics for evaluating task disambiguation separately from solution generation, formal definitions of oracle leakage, judge-free leakage diagnostics, and a training objective for the asking policy. We instantiate the framework in text-to-SQL with AmbiTab, a benchmark suite that unifies six ambiguous datasets under a common representation specifying what the agent, oracle, and verifier may access. We evaluate clarification strategies and oracle leakage, and train an asking policy with reinforcement learning. The trained asker improves our disambiguation metrics on all six datasets and task success on five, and our leakage diagnostics measure how training affects oracle leakage.

1 INTRODUCTION

The paper targets task underspecification in text-to-SQL, where success-rate evaluations conflate asking with solving, simulators may leak information, and datasets lack a shared representation of access boundaries. It responds with a leakage-aware formalism, AmbiTab, and experiments on clarification, leakage, and asker training.

1.1 CONTEXT AND PROBLEM STATEMENT

  • Success rate conflates disambiguation with solution generation, while simulated oracles may leak information and existing datasets lack a shared representation of ambiguities and access boundaries.A success may require no clarification, and a failure may stem from either a missing question or a wrong solution.Source: §1.1 · pp. 1–2

1.2 CONTRIBUTIONS

  • The paper formalizes AVTs, separates asking from solving, defines leakage diagnostics and disambiguation metrics, and unifies six ambiguous text-to-SQL datasets in AmbiTab.Experiments compare no-disambiguation, evaluated-asker, and full-disambiguation conditions, test leakage diagnostics, and train askers with a fixed solver.Source: §1.2 · pp. 2–3

1.3 RELATED WORK

  • Prior work studies ambiguous-task benchmarks, simulated-user clarification, information-gain and coverage methods, and SQL reinforcement learning; this paper makes oracle access and leakage explicit.Its question-F1 reward balances ambiguity coverage and question efficiency without additional shaping coefficients.Source: §1.3 · p. 3

2 THEORY

The theory formalizes underspecified tasks and an ask-then-solve interaction that separates clarification from solution generation. It defines success and leakage measures, then gives diagnostics that approximate leakage under stated conditions.

LEAKAGE

  • Oracle leakage is exploitation of deviations from a perfect oracle; the paper distinguishes success-rate leakage from recall leakage.Exact leakage generally cannot be computed directly; estimating success-rate leakage may require costly sampling with human annotators and a perfect oracle.Source: §2 · p. 4, LEAKAGE · p. 5

2 THEORY

  • An ambiguous verifiable task (AVT) formalizes a specification, hidden interpretations, ambiguities and their resolutions, and a verifier; the ask-then-solve protocol separates asking from solving.The environment supplies an oracle and verifier, while the agent’s observable trajectory consists of the specification and dialogue.Source: §2 · pp. 4–5

SUCCESS METRICS AND BASELINES

  • Success rate evaluates the solution, while asking precision and recall measure the asking policy against full-disambiguation and no-disambiguation baselines.The full-disambiguation policy acquires every ambiguity without unproductive questions; the no-disambiguation policy stops without asking.Source: SUCCESS METRICS AND BASELINES · p. 5

LEAKAGE DIAGNOSTICS

  • Three diagnostics probe responder leakage, specification randomization, and adversarial asking policies to estimate leakage without directly computing its definitions.The responder diagnostic swaps in the perfect responder while holding the asking policy and actual matcher fixed; the other diagnostics randomize the specification or use policies covering all or none of the ambiguities.Source: LEAKAGE DIAGNOSTICS · p. 5
  • Theorem 1 shows the responder diagnostic approximates success-rate leakage when matcher replacement changes trajectory distributions little, and specification randomization approximates recall leakage under stated recall conditions.For the latter result, a mixture model relates the randomized policy’s recall leakage to that of the original policy.Source: LEAKAGE DIAGNOSTICS · p. 5

3 AMBITAB BENCHMARK SUITE

AmbiTab organizes ambiguous text-to-SQL tasks as records with explicit ambiguity annotations, resolutions, and verification assets. The suite adapts existing datasets and standardizes the information represented in each record.

3 AMBITAB BENCHMARK SUITE

  • AmbiTab comprises six datasets of ambiguous text-to-SQL tasks, each with explicit ambiguity annotations, fixed resolutions, and verification assets.The task record is a fixed-interpretation sample of an ambiguous verifiable task.Source: §3 · p. 6
  • Records include a task specification and context, ambiguity-resolution pairs, and verification assets such as an executable database and reference query.Optional context can include schema, column meanings, or external knowledge; optional verification assets include test cases and result-set comparison policies.Source: §3 · p. 6
  • The suite adapts BIRD, BIRD-Interact, Ambrosia, AmbiQT, and TACO; AmbiTab-BIRD-Interact uses PostgreSQL, while the other datasets use SQLite.Table 1 lists the six datasets; AmbiTab-Bird and AmbiTab-EW-Bird counts combine their train and development splits.Source: §3 · p. 6, Table 1 · p. 6

4 EXPERIMENTS

The experiments evaluate clarification quality, task success, and oracle leakage across six datasets. The trained asker improves mean precision, recall, and success rate, while oracle comparisons show that higher success can coincide with higher recall-leakage diagnostics.

4.3 SUCCESS-LEAKAGE TRADEOFF ACROSS ORACLES

  • Gemma-3-27B yields higher asker SR than Qwen3-14B on all six datasets and higher values for both recall-leakage diagnostics; ∆Jresp has no consistent direction and ∆Jfull differs by less than one point.On AmbiTab-AmbiQT, SR rises from 57.7% to 69.3%, while ∆RecQspec rises from 9.9 to 70.9 percentage points.Source: Figure 3 · p. 7, §4.3 · p. 7

4.1 EXPERIMENTAL SETUP

  • The evaluation uses an ask-then-solve protocol, reporting SR against no- and full-disambiguation baselines alongside macro asking precision and recall.Qwen3-8B asks through dialogue, a fixed Qwen3-Coder-30B-A3B-Instruct solver produces SQL, and the default Checklist prompt tracks ambiguous terms and resolutions.Source: §4.1 · pp. 6–7, §4 · p. 6

4.2 AMBITAB DISAMBIGUATION CAPACITY

  • On AmbiTab-Bird, SR is 26.3% without disambiguation, 66.7% with asking, and 86.2% with full disambiguation; asker gaps are 19.5 points there and 7.1–17.4 points on four other datasets.Full disambiguation substantially improves SR over no disambiguation on five datasets.Source: §4.2 · p. 7, Figure 3 · p. 7
  • Recall remains below complete coverage on every dataset, while low precision indicates many questions receive no new ambiguity credit.Together, these metrics show that the asker leaves ambiguities uncovered and asks unproductive questions.Source: §4.2 · p. 7, Figure 3 · p. 7

4.3 SUCCESS-LEAKAGE TRADEOFF ACROSS ORACLES

  • Showing the LLM responder the full reference SQL raises ∆Jfull by up to 10.4 points and ∆Jresp by up to 7.1 points, with intervals excluding zero in six of ten dataset-oracle pairs.AmbiTab-TACO shows no detectable change.Source: Figure 4 · p. 8, §4.3 · p. 8

4.4 TRAINING IMPROVES THE ASKING POLICY

  • 31.8% vs 28.2% precision, 77.4% vs 68.4% recall, and 54.6% vs 50.4% SR are the equally weighted six-dataset means for trained and untrained askers.The trained policy improves macro precision and recall on all six datasets.Source: Table 2 · p. 8, §4.4 · p. 8, Figure 5 · p. 9
  • Question-F1 training uses a multi-turn GRPO variant with DAPO-style asymmetric clipping and zero-advantage group filtering, rewarding productive ambiguity coverage without solver output.Requests with ambiguities receive zero when no question is productive; requests without ambiguities receive one for abstaining and zero for asking.Source: §4.4 · p. 8
  • Both outcome-reward variants achieve lower mean precision, recall, and SR; Qwen3-14B outcome reward has higher ∆Jresp in 9 of 10 dataset-oracle pairs, but only one interval excludes zero.Question-F1 training keeps ∆Jresp below the untrained asker's on all five evaluated Qwen3-14B datasets; one training seed per configuration and unequal budgets prevent a general ranking of rewards or training oracles.Source: §4.4 · p. 9, Figure 5 · p. 9

5 CONCLUSIONS

The paper concludes that its leakage-aware framework separates asking from solving and supports evaluation and training of conversational task disambiguation. Three-seed question-F1 training improves precision and recall across six corpora, though substantial gaps remain.

5 CONCLUSIONS

  • The framework formalizes ambiguous verifiable tasks and an ask-then-solve protocol, separating the asker, solver, oracle, and verifier.It treats precision and recall alongside success rate and uses no-disambiguation and full-disambiguation evaluations as reference points for a fixed solver and oracle.Source: §5 · p. 9
  • Leakage diagnostics help distinguish failures to acquire information, use it, and respect disclosure boundaries.Source: §5 · p. 9
  • Question-F1 training with the deterministic oracle yields higher precision and recall across all six corpora, while substantial gaps in question quality and task success persist.The conclusion also reports that clarification and asker training improve performance.Source: §5 · p. 9
  • Future work will develop better asker training methods and extend the benchmark to broader tasks, ambiguity types, and interaction settings.Source: §5 · p. 9

ETHICS STATEMENT

  • Because the study uses existing text-to-SQL benchmarks and simulated user responses, it does not establish effectiveness with real users.The source benchmarks may occur in evaluated models’ pretraining data, which can inflate absolute success rates; comparisons hold models fixed across conditions.Source: ETHICS STATEMENT · p. 10

A.5 PROOF OF THE DIAGNOSTIC THEOREM

The proof couples recall under a specification-randomized policy with recall under the original policy. It bounds their difference using the probability that the coupled dialogues diverge.

A.5 PROOF OF THE DIAGNOSTIC THEOREM

  • The proof defines the success event using the verifier and final solution, then invokes total variation distance to complete the claim.Source: §A.5 · p. 18
  • The proof couples episodes using the same hidden interpretation and independent Bernoulli(p) coins to choose between the invariant and replacement policies.At the question budget, both runs stop.Source: §A.5 · p. 18
  • When the invariant policy is selected at a shared history, replacing the task specification leaves its action distribution unchanged, allowing both runs to share the question, credits, and response.Source: §A.5 · p. 19
  • Expected recalls differ by at most 1-p^B, because the coupled runs agree on recall when all B coins select the specification-invariant policy.On that event, the dialogues and credited ambiguities match; per-episode recall lies in [0, 1].Source: §A.5 · p. 19

B.1 DATASET CONSTRUCTION

The dataset-construction appendix describes evaluated pools, their source-to-record conversions, and limits on what some construction choices establish.

B.1 DATASET CONSTRUCTION

  • Table 4 reports evaluated pools and registered ambiguities, including 49 zero-ambiguity AmbiTab-Bird controls and 7 AmbiTab-BIRD-Interact records without a registered ambiguity.Source: Table 4 · p. 19
  • Table 5 presents source-to-record conversions and gives each evaluated pool’s principal selection rule and limit.Source: Table 5 · p. 20
  • AmbiTab-EW-Bird’s filter measures benefit at the row level, not whether each hint is necessary.Source: §B.1 · p. 20
  • AmbiTab-Bird’s admission and judgment are model dependent, while AmbiTab-TACO’s table choices can retain year-based clues.Source: §B.1 · p. 20
  • The AmbiTab-BIRD-Interact response catalog is audited, but the audit does not establish agreement with the perfect oracle.Source: §B.1 · p. 20

B.2 ORACLE IMPLEMENTATIONS

The appendix distinguishes frozen-response and evaluated LLM oracle variants by their runtime access. Its full-disambiguation baseline uses the evaluated oracle through the same interaction as the evaluated asker.

B.2 ORACLE IMPLEMENTATIONS

  • The evaluated LLM oracles may have access beyond the intended boundary, unlike the frozen-response oracles distinguished in the table.Source: §B.2 · p. 20
  • Table 6 lays out evaluated oracle variants and their runtime access.Source: Table 6 · p. 21
  • The full-disambiguation baseline asks one canonical question per registered ambiguity, then passes the resulting dialogue to the solver.Source: §B.2 · p. 21
  • The baseline uses the evaluated oracle’s responses through the same interaction as the evaluated asker; those responses are not perfect-oracle samples by definition.Source: §B.2 · p. 21

C.1 EVALUATION CONDITIONS AND POPULATIONS

The evaluation compares clarification and solution success under three conditions, using registered ambiguities and a shared solver to distinguish the asking policy’s contribution. Oracle comparisons, credited-coverage metrics, and prompt variants clarify what the reported success and asking scores capture.

C.1 EVALUATION CONDITIONS AND POPULATIONS

  • The benchmark comparison evaluates no-disambiguation, evaluated-asker, and full-asker conditions over 8,380 records, with corpus-specific populations and success determined by task tests or execution agreement.AmbiTab-Bird and AmbiTab-EW-Bird pool train and development splits; other corpora use the specified full, ambiguous, or synonym-subset populations.Source: §C.1 · p. 21

C.2 THE FULL-DISAMBIGUATION REFERENCE

  • The scripted full asker asks once about each registered ambiguity, but an imperfect oracle can leave ambiguities unresolved, so its measured success is not a universal upper bound.On the 2,375 EW-Bird records completed by every oracle, evaluated-asker success is 72.9%, 69.5%, and 69.9% for deterministic, Qwen3-14B, and Gemma-3-27B oracles; full-asker success is 81.4%, 78.9%, and 75.0%.Source: §C.2 · pp. 21–22

C.3 METRIC DEFINITIONS AND AGGREGATION

  • Asking precision credits questions that receive a previously unseen ambiguity credit, while recall measures credited registered terms; their eligible populations can differ.Scores are macro-averaged over defined episodes, and oracle wording can affect credited coverage, including by volunteering a resolution the question did not name.Source: §C.3 · p. 22

C.4 COMPLETE ORACLE COMPARISON

  • The oracle comparison reports inference results over five seeds and leakage diagnostics over three; deterministic-oracle leakage differences are zero by definition.For EW-Bird, the LLM-oracle rows count 162 training records that exceed the oracle’s context as failures.Source: §C.3 · p. 22, Table 8 · p. 23

C.5 PROMPT-DRIVEN PRECISION–RECALL TRADEOFF

  • Both checklist prompts raise recall and lower precision relative to Direct on every dataset; Checklist (ledger) is the default because it has the highest mean recall.Higher recall is associated with greater asking gain across prompts, though not for every prompt pair.Source: §C.5 · pp. 23–24, Figure 6 · p. 24

C.6 CLARIFICATION PROMPTS AND COMPLETE PROMPT COMPARISON

  • The prompt comparison holds question budget constant, but Checklist (chat) versus Checklist (ledger) changes both dialogue representation and stopping instructions, preventing attribution to either change alone.The mean rows equally average the six corpora; Checklist (ledger) uses five seeds, while Direct and Checklist (chat) use three.Source: §C.6 · p. 24

D TRAINING THE ASKING POLICY: IMPLEMENTATION AND ADDITIONAL

The appendix details the question-F1 training objective, optimization choices, and evaluation design for the asking policy and its outcome-reward ablations. It distinguishes native validation from common-solver evaluation because solver changes can affect success independently of clarification.

D TRAINING THE ASKING POLICY: IMPLEMENTATION AND ADDITIONAL

  • Native validation uses the adapted Qwen3-8B SQL solver, whereas common inference uses a fixed Coder-30B solver, so success changes need not reflect clarification alone.Question-F1 training ends after clarification; outcome-reward training also generates and verifies SQL.Source: D TRAINING THE ASKING POLICY:… · p. 24, §D.1 · p. 26

D.1 QUESTION-F1 REWARD AND TRAJECTORY-LEVEL CREDIT

  • The main policy trains Qwen3-8B on 537 AmbiTab-Bird requests using a multi-turn GRPO variant with asymmetric clipping, zero-advantage filtering, and episode-level question-F1 reward.Question-F1 is the harmonic mean of episode asking precision and recall; ambiguity-free requests receive reward for abstaining, while requests with ambiguities receive zero if no question is productive.Source: §D.1 · p. 25
  • Every retained clarification output, including stopping, receives its trajectory’s advantage; oracle responses and SQL outputs carry no loss, and constant-reward groups are filtered.If all groups have zero advantage, one is retained to avoid an empty batch; only regularization terms then contribute to the loss gradient.Source: §D.1 · p. 26

D.2 HYPERPARAMETERS AND SHARED MODEL PARAMETERS

  • The main configuration selects a checkpoint after 40 rollout batches, sampling eight requests and 16 trajectories per request, with one optimizer update per rollout batch.The asker and native SQL solver share a trainable LoRA adapter, so clarification-token updates also change the adapter used for SQL generation.Source: §D.2 · pp. 26–27

D.3 ABLATION DESIGN AND CHECKPOINT SELECTION

  • The comparison uses one training seed per configuration and designated checkpoints at iterations 40, 16, and 40 for question-F1, deterministic-oracle outcome reward, and LLM-oracle outcome reward.The policies are evaluated on AmbiTab-Bird development data and have complete common-inference evaluations across all six corpora; the development scores are not heldout test estimates.Source: §D.3 · p. 28

D.4 NATIVE LEARNING CURVES AND OBJECTIVE MISMATCH

Native validation shows gains for the question-F1 policy, but native success combines clarification with SQL solving, and a separate run illustrates unstable learning. Common-inference results use a fixed solver and specified checkpoints to assess the trained askers.

D.4 NATIVE LEARNING CURVES AND OBJECTIVE MISMATCH

  • At iteration 40, question-F1 training reaches 83.3% native recall and 60.6% native SR, versus 70.4% and 54.3% at initialization.Native validation uses the adapted Qwen3-8B solver, while the untrained reference uses the fixed Coder-30B solver and common-inference protocol; comparisons therefore include protocol and solver differences.Source: §D.4 · pp. 28–29, Figure 9 · p. 29
  • On BIRD-Interact, native recall falls from 66.8% at iteration 8 to 59.4% at iteration 48, while native SR remains 6.7%; few request groups provide differing binary rewards.The run crashed before completing training, and its curves do not identify the cause; different response-budget and schedule settings mean it does not isolate dataset effects.Source: §D.4 · pp. 29–30, Figure 10 · p. 30
  • Logged reward-contrast fractions measure how often a request’s sampled trajectories have nonconstant rewards, not question quality.With 16 trajectories sampled per request, such groups supply within-request reward contrast for policy updates; constant-reward groups have zero policy-gradient advantage.Source: §D.4 · p. 30

D.5 COMMON INFERENCE PROTOCOL AND AGGREGATION

  • Common inference evaluates trained askers with a fixed Coder-30B solver and deterministic oracle, while trained-policy means average three inference seeds that do not measure training-run variation.The six-family mean weights each family equally, and metric denominators can differ because precision and recall omit ineligible episodes.Source: §D.5 · pp. 30–31

D.6 COMPLETE INFERENCE RESULTS

Across six corpora, both outcome-reward ablations fall below the question-F1 policy on mean precision, recall, and success. Differences between the two oracle-training conditions are small and vary across corpora.

D.6 COMPLETE INFERENCE RESULTS

  • Both outcome-reward ablations fall below the question-F1 policy on six-corpus means for precision, recall, and SR.The outcome-reward policies trained with deterministic and LLM oracles have means of 29.2% precision, 70.7% recall, and 51.7% SR, versus 29.2%, 70.9%, and 51.9%, respectively.Source: §D.6 · p. 31
  • The deterministic-oracle outcome-reward policy has six-corpus means of 29.2% precision, 70.7% recall, and 51.7% SR.These are complete three-seed evaluations across all six corpora.Source: §D.6 · p. 31
  • Differences between deterministic- and LLM-oracle outcome-reward policies are small on the six-corpus means and vary across corpora.The LLM-oracle policy’s corresponding means are 29.2% precision, 70.9% recall, and 51.9% SR.Source: §D.6 · p. 31

E LEAKAGE EXPERIMENTS

The leakage experiments test how access to reference SQL affects leakage diagnostics, with effects varying across datasets. They use paired comparisons of the same policies under different responder-access conditions.

E LEAKAGE EXPERIMENTS

  • The experiments cover all six datasets over three seeds, reporting means ± sample standard deviations and paired differences with 95% AVT-cluster bootstrap intervals.The setup comprises 234 runs and 141,024 episodes.Source: E LEAKAGE EXPERIMENTS · p. 31
  • The leakage diagnostics distinguish success-rate differences from recall measures over records with at least one ambiguity.∆Jresp and ∆Jfull are success-rate differences; ∆RecQspec and ∆Recnone are a recall and a recall difference, respectively.Source: E LEAKAGE EXPERIMENTS · p. 31

E.1 RESPONDER ACCESS TO THE REFERENCE SQL

  • The access comparison changes whether the responder reads full reference SQL, keeps the matcher unchanged, and pairs the same policy by task across conditions.The compared conditions were run in different inference-server sessions.Source: §E.1 · pp. 31–32
  • Removing SQL lowers ∆Jresp on AmbiTab-EW-Bird and AmbiTab-Bird; hidden SQL can still reach responders through annotations, schemas, and matcher actions quoting SQL.The ∆Jresp intervals exclude zero in three of the four dataset–oracle pairs.Source: §E.1 · p. 32
  • Showing SQL raises ∆Jfull by 1.24 to 10.36 points on AmbiTab-Ambrosia and AmbiTab-AmbiQT; on AmbiTab-TACO, no effect is detectable.The ∆Jfull changes are reported under both oracles, and all AmbiTab-TACO intervals include zero.Source: §E.1 · p. 32, Table 15 · p. 33

E.2 TRAINED ASKING POLICIES

This section compares question-F1- and outcome-reward-trained asking policies with an untrained asker, using a prompt that requires reasoning followed by final SQL and prohibits clarification questions. The preregistered primary comparison is null, although the outcome-reward policy has higher ∆Jresp in most dataset–oracle pairs.

E.2 TRAINED ASKING POLICIES

  • The three-seed development-split runs covered AmbiTab-Bird, AmbiTab-EW-Bird, AmbiTab-Ambrosia, AmbiTab-AmbiQT, and AmbiTab-TACO; BIRD-Interact and the deterministic outcome policy were excluded.The two iteration-40 policies used question-F1 and outcome rewards, respectively; ∆Jfull and ∆Recnone do not depend on the asking policy.Source: §E.2 · p. 33
  • The primary comparison found a 1.72-point ∆Jresp difference on AmbiTab-EW-Bird dev, with a 95% interval of [−1.87, 5.23].This was the prespecified comparison of outcome reward minus question-F1 reward under the Qwen3-14B oracle.Source: §E.2 · p. 33, Table 17 · p. 34
  • The outcome-reward policy had higher ∆Jresp in 9 of 10 dataset–oracle pairs, but only AmbiTab-AmbiQT under Gemma-3-27B had an interval excluding zero.∆RecQspec intervals are weakly informative because the shared donor map leaves only 2 to 8 independent components for resampling.Source: §E.2 · p. 34

G PROMPTS

  • The response must use the specified JSON format with reasoning, final_sql type, and SQL output fields.The format is marked strictly required, and deviations are prohibited.Source: G PROMPTS · p. 39
  • The model must reason through schema linking, ambiguity resolution, and its join/filter plan before writing SQL.Source: G PROMPTS · p. 39
  • The prompt says not to ask clarification questions.Source: G PROMPTS · p. 39
Loading 2610.10740v1…