PaperlayerView exact revision on arXiv

Source-linked AI summary

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification

Jiale Zhao, Ke Fang, Lu Cheng

arXiv:2602.11199v2cs.CLcs.LG

TL;DR

LLM evaluations often judge one-shot answers despite prompts that omit critical details or contain misleading claims. AskBench turns QA pairs into checkpoint-based interactions, and rubric-guided RLVR trains models to ask targeted questions and correct false premises. Across the reported results, the approach improves answer correctness and clarification quality; its evaluation remains bounded by offline QA-derived checkpoints and a single-judge setup.

  • Problem

    One-shot QA benchmarks judge answers to fully specified queries and do not assess whether clarification would improve outcomes, despite real prompts omitting key details or containing misleading claims.

    Source: §1 · p. 1

  • Method

    AskBench converts QA pairs into multi-turn AskMind and AskOverconfidence tasks with explicit rubrics, while a judge loop evaluates interactions and rubric-guided RLVR rewards targeted information gathering and correct answers.

    Source: §1 · pp. 1–2, §4.3 · p. 6

  • Results

    Rubric-guided RLVR improves answer correctness and clarification/correction quality; on AskMind, OursI improves Acc. from 0.332 to 0.615 over Qwen.

    Source: §6.1 · p. 7, §7 · p. 9

  • Takeaways & Limitations

    Accuracy gains track checkpoint coverage rather than more aggressive questioning, with asking and answering functioning as complementary behaviors.

    Source: §6.1 · p. 8

    AskBench is an offline benchmark limited to QA-derived rubric checkpoints, uses an LLM user simulator, and evaluates interactions under a fixed turn budget with a single judge model.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or reinforced misconceptions. We study how to evaluate and improve LLMs' ability to decide when and what to ask for clarification without sacrificing task performance. We introduce AskBench, an interactive benchmark that converts standard QA pairs into multi-turn interactions with explicit checkpoints. A unified judge loop evaluates final answers and simulates user responses as needed. AskBench covers two settings: AskMind, with intent-deficient queries requiring clarification, and AskOverconfidence, with queries containing false premises that must be identified and corrected. We further propose rubric-guided reinforcement learning with verifier-based rewards (RLVR), which uses structured rubrics to encourage targeted clarification. Experiments show consistent improvements in accuracy, rubric adherence, and interaction efficiency, with strong generalization to unseen domains.

1 Introduction

One-shot QA benchmarks overlook whether a model should ask before answering, despite the risks of underspecified queries and false premises. AskBench evaluates clarification and correction through explicit checkpoints, and rubric-guided RLVR trains models to gather information before answering.

1 Introduction

  • Underspecified prompts can lead models to infer incorrect intent, fabricate details, or reinforce users’ misconceptions, compromising safety and trust.The problem also includes confidently stated incorrect intermediate claims that may be accepted uncritically.Source: §1 · p. 1
  • One-shot benchmarks judge final-answer correctness on fully specified queries, while prior clarification frameworks rely on task-specific heuristics or scripted interactions.This leaves a gap between evaluation and iterative real-world conversations.Source: §1 · p. 1
  • AskBench converts standard QA pairs into multi-turn interactions with checkpoints for missing information (AskMind) and misleading claims (AskOverconfidence).The rubrics specify what must be resolved before a final answer.Source: §1 · p. 1
  • A unified judge identifies clarification requests, grades final answers against hidden queries and checkpoints, and reveals only information the model explicitly asks for.Rubric checkpoints also support fine-grained failure attribution.Source: §1 · p. 2
  • Rubric-guided RLVR rewards correct answers and targeted information gathering while penalizing premature final answers.The paper presents this training recipe alongside AskBench to improve answer correctness and clarification quality.Source: §1 · p. 2

2 Related Work

Prior work studies interactive QA, clarification, and judge-based evaluation, but often relies on scripted interactions or focuses on narrower task settings. AskBench combines automatic construction from QA pairs, explicit checkpoints, and an interactive judge loop.

2 Related Work

  • AskBench is automatically constructed from existing QA datasets and couples explicit checkpoints with a unified judge loop for scalable, cross-domain evaluation.The related-work discussion contrasts this with prior benchmarks that rely on scripted user simulators or handcrafted templates.Source: §2 · p. 2
  • AskBench evaluates open-ended multi-turn clarification with potentially multiple missing points, unlike benchmarks focused on selecting one question or tool-agent execution.It also preserves standard QA answer scoring while supporting both missing-information and misleading-claim rubrics.Source: §2 · p. 2, Table 1 · p. 2
  • The paper uses explicit rubric criteria to define rewards balancing correctness, checkpoint coverage, and interaction cost, extending rubric-based supervision to clarification behavior.The criteria also support attributing failures to specific missing information rather than relying only on scalar scores.Source: §2 · p. 3
  • Unlike single-turn LLM-as-a-judge grading, AskBench’s judge classifies clarification requests, scores final answers, and simulates rubric-consistent user responses.Its itemized rubrics make missing information interpretable as testable requirements.Source: §2 · p. 3

3 AskBench Design

AskBench defines two checkpoint-based interaction settings, constructs balanced datasets from existing QA benchmarks, and evaluates models through a judge-mediated dialogue loop. It reports answer accuracy alongside measures of checkpoint resolution and redundant questioning.

3.1 Task Setting and Checkpoints

  • AskBench creates AskMind queries by removing or blurring intent-critical details and AskOverconfidence queries by injecting confidently stated false intermediate claims, with rubrics specifying what must be resolved.AskMind targets missing or ambiguous intent; AskOverconfidence requires claims to be identified and corrected before answering.Source: §3.1 · p. 3

3.2 Datasets and Statistics

  • AskBench samples 100 generated items per domain from Math500, MedQA, BBH, and GPQA-d, yielding 400 dialogues each for AskMind and AskOverconfidence.The construction pipeline generates query variants with rubrics, validates their schema, and exports aligned fields for evaluation.Source: §3.2 · pp. 3–4

3.3 Multi-Turn Evaluation Loop

  • In the multi-turn loop, the judge distinguishes clarification requests from final answers, scores final answers against the hidden query and rubric, and supplies rubric-consistent information until the turn limit.The last allowed turn must be a final answer, enabling joint assessment of answer accuracy and clarification quality.Source: §3.3 · p. 4

3.4 Outputs and Metrics

  • AskBench reports final-answer accuracy, checkpoint coverage before answering, and redundant questioning after all rubric criteria are resolved.For IN3, it reports vague-task Ask rate and clear-task direct-response rate; Table 2 illustrates an instance with an original query, variant, and shortened checkpoint list.Source: §3.4 · p. 4, Table 2 · p. 5

4 Rubric-Guided RLVR Training

Training dialogues are generated and scored against rubric checkpoints, then used in verifier-based RLVR to improve clarification behavior. The reward design favors targeted information gathering while requiring a correct final answer.

4.1 Training Data Collection

  • Training rollouts on DAPO and MedMCQA yield rubric-annotated dialogues, with high-coverage conversations as positive examples and failures exposing overlooked criteria.The same degradation-and-rubric pipeline also creates AskOverconfidence dialogues by injecting misleading claims and rolling out correction-oriented interactions.Source: §4.1 · p. 5

4.2 Reward Design

  • The judge rewards intermediate turns −2.0 for premature answers, −0.8 for untargeted questions, 0.8 for partial rubric coverage, and 1.0 for covering all m items.The hand-set weights are shared across tasks and keep rewards bounded in [−2, 1].Source: §4.2 · p. 5
  • At the final turn, the judge assigns 1.0 for a correct answer, −1.0 for a wrong answer, and −2.0 if the model is still asking.The judge compares the answer with the hidden ground truth and rubric.Source: §4.2 · p. 5
  • The judge loop classifies each reply as a clarification request or final answer, simulating a user follow-up for clarification requests and scoring final answers.This loop supplies interaction feedback while the reward function evaluates rubric coverage and final correctness.Source: §4.2 · p. 5, Figure 2 · p. 6
  • The shared reward balances answer correctness with rubric-targeted information gathering, discouraging premature answers and failure to provide a final answer.Across turns, rewards run from −2.0 to 1.0.Source: §4.2 · pp. 5–6

4.3 Policy Optimization with GRPO

  • Verifier-based RLVR uses the judge as a reward model because asking is sequential and no single gold clarification trajectory is available.The verifier evaluates partial rubric coverage across turns alongside the final ask-or-answer decision.Source: §4.3 · p. 6
  • GRPO updates the policy from rubric-derived rollout rewards, while sampling shifts from diverse clarification styles early to higher-coverage behaviors later.The training setup uses GRPO to stabilize optimization.Source: §4.3 · p. 6
  • Preliminary SFT results suggest limited generalizability, motivating RLVR to optimize turn-level rubric coverage and answer correctness while preserving broad task performance.Source: §4.3 · p. 6

5 Experimental Setup

The experiments use an instruction-tuned Qwen2.5 policy and a frozen Qwen3 judge. Evaluation combines fixed-budget multi-turn dialogue with single-turn QA and additional robustness checks.

5.1 Models

  • The asking policy starts from a 7B-parameter Qwen2.5 instruct model; the frozen judge is Qwen3-30B-A3B-Instruct-2507.The judge reads the dialogue, hidden original query, and checkpoints, then outputs scores and simulated user replies; OursI and OursO are trained on AskMind and AskOverconfidence, respectively.Source: §5.1 · p. 6

5.2 Evaluation Protocol

  • Multi-turn evaluation rolls out dialogues for three turns unless noted, requires a final answer on the last turn, and uses a frozen policy with the judge.The evaluation judge classifies replies, simulates user responses, and scores final answers.Source: §5.2 · p. 6
  • Table 3 reports single-turn accuracy on standard QA benchmarks and a HealthBench score for 500 sampled conversations scored by the A3B judge.These HealthBench numbers are not directly comparable to the original HealthBench protocol.Source: §5.2 · p. 6, Table 3 · p. 7
  • Additional evaluation includes a stricter two-turn Hard protocol, cross-judge robustness and simulator-sensitivity checks, and a Self-Alert baseline on AskOverconfidence only.Source: §5.2 · p. 7

6 Results and Analysis

Rubric-guided RLVR improves multi-turn asking and correction performance across AskBench and related benchmarks, while retaining strong single-turn QA performance. The results also show cross-dimension transfer and a trade-off between checkpoint coverage and interaction efficiency.

6.1 Main Results

  • Single-turn results show rubric-guided RLVR preserves strong QA performance, with OursI improving in-domain and out-of-domain benchmarks and HealthBench over Qwen.OursO performs strongly, including MedQA accuracy of 0.992, while trading off some Math500 accuracy.Source: §6.1 · p. 7
  • On AskMind, OursI raises Acc. from 0.332 to 0.615 and Cov. from 0.214 to 0.679 over Qwen; OursO reaches 0.617/0.807, with Unq. 0.141.The reported gains extend to AskOverconfidence and QuestBench-Math; on IN3, both policies ask more on vague tasks but have lower clear-task direct rates than Qwen.Source: §6.1 · pp. 7–8, Table 4 · p. 8
  • Accuracy gains consistently track improved checkpoint coverage rather than more aggressive questioning, making asking and answering complementary behaviors.The authors attribute the shift to rewarding rubric-targeted questions and penalizing premature answers.Source: §6.1 · pp. 7–8

6.2 In-depth Analysis

  • OursI improves AskMind accuracy and coverage on representative in-domain Math500 and out-of-domain GPQA-d splits, with the same pattern on remaining splits.For AskOverconfidence, gains persist in- and out-of-domain; OursO raises coverage substantially but retains higher Unq.Source: §6.2 · p. 8

6.3 Strict-mode (Hard) Evaluation

  • Under the strict two-turn Hard protocol, OursI performs best on AskMind and OursO on AskOverconfidence, while both remain competitive across dimensions.The protocol permits one clarification opportunity, then requires a final answer; it is a stress test rather than a model of natural conversation.Source: §6.3 · p. 8, Table 5 · p. 8
  • On AskOverconfidence (Hard), Gemini answers on the first turn in 387/400 cases and achieves 0.010 accuracy despite 0.735 coverage.The results illustrate that checkpoint coverage alone does not ensure adherence to the clarification-first protocol.Source: §6.3 · pp. 8–9

7 Conclusion

AskBench evaluates ask-before-answer behavior through interactive dialogues with explicit checkpoints and a unified judge loop. The paper also presents rubric-guided RLVR as a way to improve answer correctness and clarification or correction quality.

7 Conclusion

  • AskBench operationalizes ask-before-answer evaluation with explicit checkpoints and a unified judge loop that scores final answers and simulates user replies when needed.Rubric-guided RLVR uses checkpoint-derived signals to train policies that improve answer correctness and clarification/correction quality.Source: §7 · p. 9

Limitations

The paper identifies limits in AskBench’s offline, checkpoint-bounded interactions, its LLM-based judging, and the breadth of its evaluation. The supplied passages also describe its construction, dialogue pipeline, quality audit, and an SFT baseline.

Limitations

  • AskBench is built offline from existing QA pairs, so its checkpoints do not fully capture open-ended dialogues with evolving goals.The study also leaves value disagreements, long-horizon planning, and stylistic effects such as perceived empathy outside its analysis.Source: Limitations · p. 9
  • Evaluation uses a fixed turn budget, forced final answers, no personalization, and a single judge for reward and evaluation signals.Cross-judge checks support stable final-turn correctness decisions, but the paper lacks a comprehensive bias audit and broad evidence beyond domains concentrated in mathematics and medicine.Source: Limitations · p. 9

C.1 Intent Degradation and Rubric Creation

  • The data-construction prompts create AskMind variants by removing or blurring critical information, and AskOverconfidence variants by injecting confidently stated wrong intermediate claims.The corresponding rubrics list missing criteria or misleading points.Source: §C.1 · p. 15

C.2 Multi-Turn Clarification and Answering

  • The dialogue pipeline targets unresolved rubric criteria with clarification turns, simulates user replies from hidden rubric information, and checks coverage before final answering.For false-premise cases, the prompts require explicit correction of misleading points before the final answer.Source: §C.2 · pp. 15–16

C.3 Direct Answer and Self-Correction

  • The pipeline also supports direct answering and self-correction: a judge compares the initial answer with the reference, and an incorrect answer can be reconstructed to match the standard answer.The reconstructed answer is used in the final conversation trace.Source: §C.3 · p. 16

D Training Data Details

  • Training dialogues use QA pairs from DAPO and MedMCQA, with difficulty balancing, schema validation, and rubric-annotated query variants.The pool is built by rejection sampling and pass-rate bucketing.Source: D Training Data Details · p. 16

D.1 Quality Audit of Generated Query Variants and Rubrics

  • A stratified audit of 50 instances per dimension found strong overall quality and rare severe failures; AskMind completeness and objectivity were 88–90%, versus 98% for AskOverconfidence.The audit covered four source domains and five quality dimensions.Source: §D.1 · p. 17, Table 7 · p. 17

E SFT Baseline

  • AskMind SFT improves over the base model on AskMind but can hurt out-of-domain performance, notably the HealthBench score.Source: E SFT Baseline · p. 17

F Implementation Notes

The implementation uses rubric-based verifier rewards to train policies and evaluates them with judge-driven multi-turn protocols. The supplied passages specify training settings, evaluation variants, and protocol boundaries.

F Implementation Notes

  • GRPO-style RLVR training scores final-answer correctness and rubric coverage, combines intermediate coverage signals with terminal correctness, and penalizes premature or missing final answers.AskMind and AskOverconfidence share optimization hyperparameters, while training data and rewards differ.Source: F Implementation Notes · p. 17
  • Training uses batch size 64, prompt/response limits of 2048/8192 tokens, eight rollouts per prompt, and 500 steps over 10 epochs on eight H200 GPUs.Source: F Implementation Notes · p. 17

G Figurative Illustration of the Training Data Collection Process

  • Figure 3 depicts difficulty-balanced data pooling followed by query-variant construction, judge-driven dialogue rollouts, and rubric annotation.Source: Figure 3 · p. 18

F Implementation Notes

  • For AskOverconfidence, a checkpoint counts as resolved only when the assistant explicitly identifies and corrects a misleading claim before answering.This rubric signal is described as analogous to the AskMind coverage signal.Source: F Implementation Notes · p. 18

H.1 Full Evaluation Protocol

  • Standard evaluation allows up to three turns and forces a final answer on the last turn; the judge classifies replies, scores answers, and simulates checkpoint-based user responses.QuestBench-Math is converted to the checkpointed format, and HealthBench uses a 500-conversation subset scored by an A3B judge, so those results are not directly comparable to published HealthBench numbers.Source: §H.1 · p. 18
  • The Hard protocol caps interaction at two turns, requires a clarification-only first message for checkpoint tasks, and enforces a unique final answer on the second turn.Source: §H.1 · p. 19

H.4 Multi-Turn Protocol and IN3 Metrics

  • IN3 lacks ground-truth final answers in this evaluation, so the paper reports vague-task asking and clear-task direct-answer rates instead of answer accuracy.This differs from IN3’s original focus on tool-using agents and downstream execution.Source: §H.4 · p. 19

H.5 Cross-Judge Robustness and Simulator Sensitivity

Sampled AskMind evaluations preserve the main ranking across judges, while final-turn correctness agreement is high. Split results report gains on representative in-domain and out-of-domain AskMind and AskOverconfidence sets.

H.5 Cross-Judge Robustness and Simulator Sensitivity

  • Final-turn correctness agreement across judges is 0.96–0.98, while non-final checklist agreement is 0.60–0.80.The analysis re-scores 50 sampled AskMind trajectories from an extended 4-turn rollout.Source: Table 9 · p. 19, §H.5 · p. 19
  • With external judge gptoss-120b, OursI scores 0.645 versus 0.424 for Qwen and 0.365 for AskToAct; simulator statistics remain close across alternatives.The checks use sampled subsets and are not a comprehensive bias audit.Source: §H.5 · p. 19

I Split Breakdown Tables

  • AskMind-Math500 Acc. rose 0.422→0.730 and Cov. 0.340→0.830; on GPQA-d, Acc. rose 0.090→0.463 and Cov. 0.150→0.836.The reported pattern also holds on remaining AskMind splits.Source: Table 10 · p. 20, I Split Breakdown Tables · p. 20
  • On AskOverconfidence-Math500, OursO Acc. rose 0.655→0.667 and Cov. 0.358→0.906; on GPQA-d, Acc. rose 0.145→0.323 and Cov. 0.059→0.783.OursO has elevated Unq. across splits, indicating a caution-efficiency trade-off.Source: Table 11 · p. 20, I Split Breakdown Tables · p. 20

J Additional AskOverconfidence Split Results

Additional AskOverconfidence results on MedQA and BBH show high coverage gains alongside improved, but domain-dependent, accuracy.

J Additional AskOverconfidence Split Results

K Additional AskMind Split Results

Additional AskMind results report accuracy and coverage gains on both the in-domain MedQA split and out-of-domain BBH split.

K Additional AskMind Split Results

Loading 2602.11199v2…