PaperlayerView exact revision on arXiv

Source-linked AI summary

Asking for What Was Never Requested: Horizontal and Vertical Proactivity in Agents

Ido Levy, Asaf Yehudai, Segev Shlomov, Asaf Adi, Leshem Choshen

arXiv:2609.37236v1cs.AIcs.CL

TL;DR

The paper studies what information an agent should pursue when completing a task requires details the user did not state. It introduces need-graph evaluation and Q&D, which trains question selection from retrieval consequences; the trained questioner improves evidence recovery across held-out benchmarks and transfers to customer service.

  • Problem

    Research on proactive agents largely asks whether and when they should act, leaving less attention to what information they should pursue unasked.

    Source: §1 · p. 1

  • Method

    Q&D uses benchmark-derived need graphs to score transcript evidence and trains a questioner to prefer questions whose continuations retrieve more required evidence, without a reward model or judge.

    Source: §1 · pp. 2–3, §3 · p. 4

  • Results

    At equal spend, the trained questioner recovers more required evidence than same-model prompting across three held-out benchmarks and exceeds a prompted 15× larger model on two of three.

    Source: §1 · p. 2, §8 · p. 9

  • Takeaways & Limitations

    Without further training, the questioner transfers to customer service, completing more tasks with fewer questions and outperforming the 15× larger model in retail with fewer customer follow-up turns.

    Source: §1 · p. 2, §8 · p. 9

    Released retrieval pools are small enough that most prerequisite edges do not gate retrieval, so the vertical result concerns depth in benchmark decompositions.

    Source: §7 · p. 9

Abstract

from arXiv · show

An agent that uses tools typically responds to what the user explicitly asks, yet completing the task may require information the user never requested. Work on proactive agents mainly studies whether and when an agent should act on its own, not what information it should pursue. We study a distinct axis of proactivity: its content. Horizontal proactivity pursues unstated information that the current context already identifies, and vertical proactivity pursues needs that only earlier evidence reveals. A need graph, recovered from a benchmark's own decomposition, records which needs depend on which, so both forms, and whether the agent stops at the right time, can be scored from a transcript without a model judge. To learn this behavior, we propose Q&D (questioner and drafter), which trains a questioner to prefer the question whose continuation retrieves more of the required evidence, with no reward model or judge. On held-out splits of three multi-hop question-answering benchmarks, at equal retrieval spend, the trained questioner improves both forms of proactivity over the same model, prompted, and outperforms a prompted model $15\times$ larger in the same role on two of the three, and the gain persists after controlling for question volume and length. Without further training, we place the questioner in an interactive customer-service agent with a simulated customer, where it completes more tasks while asking fewer questions, and in retail it outperforms the $15\times$ larger model with fewer follow-up turns from the customer. These results show that proactivity depends not only on whether an agent acts without being asked, but also on what it chooses to pursue and when it stops.

1 INTRODUCTION

The paper frames proactivity as a choice of what information to pursue, then introduces need-graph measurement and Q&D, which learns from the consequences of questions. On held-out benchmarks and simulated customer-service tasks, the trained questioner improves evidence seeking and task completion.

1 INTRODUCTION

  • Prior work on proactive agents emphasizes whether and when to act, while information-seeking methods often follow explicit chains or optimize final-answer correctness.The paper notes that asking more questions can be mistaken for asking better ones.Source: §1 · p. 1
  • A need graph represents required evidence and dependencies, enabling transcript-based scoring of proactivity and stopping without a model judge.Agents are compared after the same number of questions, so asking more cannot count as asking better.Source: §1 · p. 2
  • Q&D trains a questioner to prefer questions whose continuations retrieve more required evidence, while a frozen drafter incorporates retrieved evidence into a draft.The training procedure uses no reward model or judge.Source: §1 · p. 2
  • On three held-out QA benchmarks, Q&D improves both forms of evidence recovery over same-model prompting at equal spend and beats a prompted 15× larger model on two.The gain persists after controlling for question volume and length.Source: §1 · p. 2
  • Without further training, Q&D completes more simulated retail tasks than a 15× larger prompted model, with fewer customer follow-up turns.It more than doubles retail task success in the simulated customer-service setting.Source: §1 · p. 2

2 A NEED GRAPH MAKES THE CONTENT OF PROACTIVITY MEASURABLE

A need graph represents required evidence and prerequisite relationships, making it possible to assess what a run pursues from its transcript. Its metrics distinguish horizontal breadth, vertical depth, overall coverage, and stopping behavior.

2 A NEED GRAPH MAKES THE CONTENT OF PROACTIVITY MEASURABLE

  • A need graph lists task-required evidence as needs and links a need to its prerequisite when it becomes nameable only after that prerequisite is resolved.Depth is the longest prerequisite path above a need; connected prerequisite needs form an independent line of inquiry.Source: §2 · p. 3
  • Breadth measures horizontal proactivity, while depth-weighted recall and deepest need resolved measure vertical proactivity.Required-evidence coverage captures both forms, including branches within a line of inquiry.Source: §2 · p. 3
  • The evaluation also measures whether an agent stops after finding everything and whether it keeps asking while evidence remains missing.In user-facing settings, follow-up turns are read together with task success because giving up also reduces user effort.Source: §2 · p. 3

3 Q&D TRAINS A QUESTIONER ON WHAT ITS QUESTIONS RETRIEVE

Q&D separates question selection from drafting so each question can be trained on what its continuation retrieves. Candidate questions are compared by downstream consequences, with training stages that also teach when to stop.

3 Q&D TRAINS A QUESTIONER ON WHAT ITS QUESTIONS RETRIEVE

  • At each step, the questioner sees the task, evidence, draft, and past questions, then asks one question or stops; a frozen drafter rewrites the draft from evidence.A frozen answerer produces the final answer and is shared across policy arms.Source: §3 · p. 3
  • The draft exposes which needs are nameable, letting the questioner choose between pursuing a newly opened chain and another need already on the frontier.This is the choice between going deeper and going wider.Source: §3 · p. 4
  • Q&D forks a recorded run, continues sampled alternative questions under the originating policy, and ranks them by answer completion, earlier complete evidence, then evidence gained per turn.At a graph-marked unfinished state, asking ranks above stopping; the questioner itself never sees the graph.Source: §3 · p. 4
  • Training proceeds from imitation of good decisions to direct preference optimization on question pairs, then preference optimization with question-versus-stop contrasts.Stop contrasts address unfinished states where imitation supplies no target because no sampled question clears its consequence-score floor.Source: §3 · p. 4

4 EXPERIMENTAL SETUP

The evaluation uses held-out tasks from three multi-hop QA benchmarks, with benchmark-specific need graphs and a separate customer-service transfer setting. Policies are compared through equal-spend and self-stopping readings, with paired task-level uncertainty estimates.

4 EXPERIMENTAL SETUP

  • Held-out test suites contain 200 tasks per benchmark, drawn from MuSiQue, StrategyQA, and 2WikiMultiHopQA, whose decompositions supply the need graphs.FRAMES has no need graph and is scored by answer matching; retail and airline transfer use τ 2-bench domains not used for training.Source: §4 · p. 4
  • The questioner is Qwen3-8B, compared mainly with the same prompted model; other comparators include prompted GPT-OSS-120B and structured retrieval systems.Retrieval uses BM25 over released task pools, with a fixed number of returned passages per benchmark.Source: §4 · p. 4
  • Equal-spend comparisons use the lower question count for each task, while own-stop comparisons let each policy stop itself within an eight-call limit.Contrasts are paired by task and assessed with task-level bootstrap intervals.Source: §4 · p. 4
  • No question-answering contrast was registered in advance as a confirmatory test, and the reported intervals are not a multiplicity correction.Each comparator is read at its own lower question count, so contrasts do not subtract.Source: §4 · p. 5

5 RESULTS

Q&D improves evidence recovery and reaches deeper needs at equal question counts, while its strongest benchmark advantage appears on the deepest chains. The results also show transfer to customer service, though the training signals teach stopping less readily than asking.

5 RESULTS

  • 11.2 points on MuSiQue, 7.0 on StrategyQA, and 5.0 on 2WikiMultiHopQA: Q&D improves required-evidence coverage over the same prompted model at equal question counts.The gains were stable across training seeds.Source: §5 · p. 5
  • Depth-weighted recall rises by 12.5, 5.4, and 5.5 points, while breadth also rises, indicating gains in deeper evidence without narrowing the search.The deepest resolved need advances by 0.22 levels on MuSiQue and 0.10 on StrategyQA.Source: §5 · p. 5
  • Removing evidence from the questioner’s state costs 8.4 and 8.7 MuSiQue coverage points, while planning from the request alone costs 4.3 and 6.4.On StrategyQA, neither ablation makes a detectable difference because the ablated questioners ask little more than one question.Source: §5 · p. 6
  • The trained questioner’s coverage advantage persists when the prompted model may spend as many question tokens, despite longer questions of 17.5 versus 13.1 words on MuSiQue.At matched question-token spend, the prompted model recovers 14.9 and 4.7 points less on MuSiQue and StrategyQA development tasks.Source: §5 · p. 6
  • At equal spend, the 8B questioner beats GPT-OSS-120B by 7.0 points on MuSiQue and 4.3 on StrategyQA, but trails by 3.5 on 2WikiMultiHopQA.Its lead is largest on MuSiQue’s deeper chains, and reverses on 2WikiMultiHopQA, where needs are at most one step deep.Source: §5 · p. 6
  • At its own stop, Q&D asks fewer questions than either prompted model on every suite and keeps its lead on MuSiQue by 9.6 and 10.1 points.It uses its full MuSiQue call budget in 34% of runs under a four-call budget and 8% under a 24-call budget.Source: §5 · p. 6
  • Retail success exceeds GPT-OSS-120B by 17.0 and 12.7 points, with 1.6 and 1.8 fewer customer follow-up turns.The customer-service agent was used without further training; Figure 3 reports task success and follow-up-turn differences across retail and airline tasks.Source: Figure 3 · p. 8, §5 · p. 8
  • Question-pair training raises coverage but stops too early; stop contrasts alone almost never stop, while the final stage combines coverage gains with asking when work remains.The final stage stops half as often as imitation once the work is done.Source: §5 · pp. 7–8, Table 3 · p. 7

6 RELATED WORK

Prior work studies agent initiative, search strategies, and supervision for information-seeking, but leaves the content of pursued information less directly measured. This paper distinguishes its approach through evidence-based need graphs, consequence-ranked questions, and controls for length and model judging.

Learning what to ask

Proactivity

  • Prior initiative work studies whether and when agents act, while content-side benchmarks assess gap recognition or intervention timing; this work measures what an agent pursues.The nearest content-side work grades implied needs under a model judge, without checking whether the need could have been named from the task [Harfi et al., 2026].Source: §6 · p. 9

Agentic search and retrieval

  • Decomposition and adaptive retrieval methods structure search or decide when to retrieve, whereas need graphs represent required evidence and score runs without being shown to the policy.The comparison with PAR2-RAG holds the retriever, pool, and drafter fixed.Source: §6 · p. 9

Length and model judges

7 LIMITATIONS

The reported intervals cover task variation but not variation from the shared supervised checkpoint and pair export.

7 LIMITATIONS

  • Both reported seeds share one supervised checkpoint and pair export, so task-only intervals do not estimate variation from those shared training inputs.The paper also reports that no checkpoint met the pre-committed selection rule as written.Source: §7 · p. 9

8 CONCLUSION

The conclusion frames proactivity as both whether or when an agent acts and what it pursues, measured through a need graph. Q&D improves evidence recovery and transfers to customer service, while stopping remains an area for further work.

8 CONCLUSION

  • A need graph makes horizontal and vertical proactivity measurable as the content of what an agent pursues unasked.Q&D trains the questioner on the consequences of its questions without a reward model or judge.Source: §8 · p. 9
  • At equal spend, the trained 8B questioner reaches deeper into task dependencies and beats a prompted model 15× larger in the same role on two of three benchmarks.The gain lies in what it asks, not how much, and is largest when evidence must be followed step by step.Source: §8 · p. 9
  • In customer service, Q&D completes more tasks with fewer questions, and in retail it beats the larger model with fewer customer follow-up turns.The conclusion identifies stop supervision and retrieval pools with gated prerequisites as next steps.Source: §8 · p. 9

A METRIC DEFINITIONS AND ESTIMATOR

The section defines transcript-based measurements of proactivity and stopping on need graphs recovered from benchmark decompositions. It also examines whether breadth and depth can be distinguished in constructed tasks and illustrates vertical proactivity on one held-out task.

A METRIC DEFINITIONS AND ESTIMATOR

  • A need graph represents required evidence as nodes and prerequisite edges, with depth from the longest prerequisite path and facets as independent lines of inquiry.Breadth counts lines advanced past their first need; depth-weighted recall weights coverage by depth, while the stopping pair is read at decision points.Source: A METRIC DEFINITIONS AND ESTIMATOR · p. 17, Table 5 · p. 19, Table 6 · p. 19
  • Need-graph edges are recovered mechanically from benchmark structure, but StrategyQA leaves some steps unconnected, making its measured depth a lower bound.The random-edge-deletion test could hide a depth result but not manufacture it; systematic edge error is not covered.Source: A METRIC DEFINITIONS AND ESTIMATOR · pp. 17–18
  • On the benchmark suites used for training and selection, breadth is measurable but not an independent axis because branching is absent or limited in their annotations.The section reports no annotated need at the second level in the branching suite and no separating suite could be assembled from the annotations.Source: A METRIC DEFINITIONS AND ESTIMATOR · p. 18
  • In constructed corpora, the trained questioner gains breadth on the two shorter chains, ties at the longest, and improves depth-weighted recall on all three shapes.The figure’s policy positions are not matched on spend, so their placement relative to the breadth-first/depth-first line does not rank them against those policies.Source: A METRIC DEFINITIONS AND ESTIMATOR · p. 18, Figure 4 · p. 20
  • On composed natural tasks, the trained questioner improves breadth, required-evidence coverage, and depth-weighted recall over the same model, prompted, at equal realized spend.The gains are +0.276, +0.152, and +0.159, respectively; the pairing creates composed tasks rather than a natural benchmark.Source: A METRIC DEFINITIONS AND ESTIMATOR · p. 19

B ONE TASK, THREE POLICIES

  • On the illustrated held-out task, both trained-questioner seeds reach a deeper need before its prerequisites, while the prompted model stops one need short after repeated questions.All four runs have zero strict answer match, so the example reports resolved needs rather than answer correctness.Source: B ONE TASK, THREE POLICIES · p. 21

C COMPARISON AT EQUAL RETRIEVAL SPEND

The section compares policies at equal retrieval spend and examines whether gains reflect better questions rather than more questions or longer wording. It also reports comparisons with structured retrieval and on a suite without need graphs.

C COMPARISON AT EQUAL RETRIEVAL SPEND

  • Equal-spend readings rely on the questioner not being told its cap, so a longer-run prefix represents a run it would have produced under a shorter cap.The policy state contains no spend ledger or remaining allowance; a budget-aware policy would not preserve this property.Source: C COMPARISON AT EQUAL RETRIEVAL SPEND · p. 23
  • The trained questioner reduces tokens per required need found on all three held-out suites.On MuSiQue, the reduction is −7,744 tokens, from 19,354 to 11,610 tokens per need found; Figure 6 shows the paired efficiency readings.Source: C COMPARISON AT EQUAL RETRIEVAL SPEND · p. 23, Figure 6 · p. 24
  • The trained questioner’s questions are longer than the same model’s prompted questions on development, so equal-call comparisons alone do not control for question length.It fails the declared not-longer rule on both suites in words and tokens.Source: C COMPARISON AT EQUAL RETRIEVAL SPEND · p. 23
  • At equal retrieval calls, the trained questioner’s evidence-coverage lead over the comparator does not disappear when the comparator receives a question-token budget.The differences are +0.1556 on MuSiQue and +0.0799 on StrategyQA; all four intervals exclude zero.Source: C COMPARISON AT EQUAL RETRIEVAL SPEND · p. 24
  • At equal retrieval spend, the trained questioner leads the PAR2-RAG reimplementation on most reported comparisons, while Claude Opus 5 prompted plainly leads it on MuSiQue.The exception is a −0.029 trained-minus-Claude difference on MuSiQue, where Claude asks 3.5 questions against 4.2.Source: C COMPARISON AT EQUAL RETRIEVAL SPEND · p. 24
  • On FRAMES, which has no need graph, no difference from prompted comparators is decided, while Claude Opus 5 running PAR2-RAG leads the trained questioner in answer correctness.The trained questioner uses 3.6 retrieval calls against 4.5 for that comparator; this is not an equal-spend comparison.Source: C COMPARISON AT EQUAL RETRIEVAL SPEND · p. 25

D CORPUS, EXPORT RULES AND TRAINING RECIPES

The section describes how recorded runs are turned into decision examples and preference pairs, and how the arms are compared and selected. The training export uses need graphs to evaluate decision consequences, while the trainer itself does not access them.

D CORPUS, EXPORT RULES AND TRAINING RECIPES

  • Recorded runs are forked at a decision state into eight alternative questions, whose completed continuations are labeled by answer, evidence-retrieval, and need-timing consequences.The candidates share the same state, evidence, and history, allowing pairwise preferences without a baseline or critic.Source: D CORPUS, EXPORT RULES AND TRAINING… · p. 25
  • The export reads need graphs to value decisions, while the trainer does not; prompts are reconstructed from recorded artifacts and retrieved paragraphs are truncated at 1,200 characters.The export ran with a flat consequence-score threshold of 0.05.Source: D CORPUS, EXPORT RULES AND TRAINING… · p. 25
  • Imitation targets stopping when required evidence is already present, otherwise selecting the best sampled question above a fixed floor; states with no qualifying question are omitted.Preference training adds synthesized stop contrasts for unfinished states that imitation would drop.Source: D CORPUS, EXPORT RULES AND TRAINING… · p. 25
  • Development comparisons use paired coverage differences at gate-matched spend, but they are selection readings rather than confirmatory results.None of the 23 checkpoints passed the pre-committed selection rule as written; the carried-forward checkpoint was chosen on a separate pre-committed ground.Source: D CORPUS, EXPORT RULES AND TRAINING… · pp. 25–26
  • The final preference arm combines question pairs, recorded stop contrasts, and synthesized stop contrasts, with 31,473 of its 36,349 preference-file pairs included in training.The table gives adapter settings and preference-training settings for the arms.Source: Table 8 · p. 26

E HELD-OUT RESULTS IN FULL

On held-out runs, the trained questioner improves required-evidence coverage against the same model, prompted, across the reported suites, though the trained-arm contrasts are exploratory. The section also examines question diversity, ordering, and whether out-of-order retrieval corresponds to answer harm.

E HELD-OUT RESULTS IN FULL

  • The trained-arm contrasts are exploratory because no endpoint of the preregistered stage names the trained questioner as a treatment.The results are therefore reported as point estimates and intervals without a probability value.Source: E HELD-OUT RESULTS IN FULL · p. 27
  • At gate-matched spend, the trained questioner is above the same model, prompted, on all three suites; at the shared cap of eight, it is above it on MuSiQue and within two points on the other two.Its mean question counts are 4.20, 3.05, and 2.32, compared with comparator charges of 3.93, 2.87, and 2.11.Source: E HELD-OUT RESULTS IN FULL · p. 27
  • Pooling rollout seeds makes question diversity appear lower than when the same runs are grouped by task and seed.The pooled held-out values are 0.458 to 0.600, while per-task-and-seed values are 0.805 to 0.849.Source: E HELD-OUT RESULTS IN FULL · p. 27
  • The reported out-of-order excess on MuSiQue is largely incidental retrieval, not questions that explicitly ask for the child need.The asked-for component is −0.012 [−0.042, +0.014], while the incidental component is +0.069 [+0.023, +0.115].Source: E HELD-OUT RESULTS IN FULL · p. 28
  • A task-level comparison finds no answer-harm signal where the trained questioner resolves more needs out of order, but does not establish that disorder helps.The strata are selected by the trained questioner’s own behavior and may instead be tasks where it reaches more.Source: E HELD-OUT RESULTS IN FULL · p. 29
  • Independent rollouts reproduce the positive coverage differences against the same model, prompted, on MuSiQue and StrategyQA.The independent-rollout differences are +0.119 and +0.082, versus +0.112 and +0.070 in Table 2.Source: E HELD-OUT RESULTS IN FULL · p. 29

F THE EIGHT CONTROLS

This appendix compares registered controls with the trained questioner under spend-matched rules. The controls vary in how many questions they ask and whether they preserve the questioner’s evidence-seeking behavior.

F THE EIGHT CONTROLS

  • +0.3474 [+0.3033, +0.3910] on MuSiQue and +0.6325 [+0.5923, +0.6757] on StrategyQA: the trained questioner leads random questions at equal spend.The comparisons cover 176 MuSiQue tasks retained by the control and 200 StrategyQA tasks.Source: F THE EIGHT CONTROLS · p. 30, Figure 7 · p. 31
  • With both arms at the lower question count, the trained questioner leads depth-one and evidence-blind ablations on MuSiQue, but neither separates on StrategyQA.On MuSiQue the differences are +0.043 and +0.064 for depth one, and +0.084 and +0.087 for evidence blind, across the two seeds; every interval excludes zero.Source: F THE EIGHT CONTROLS · p. 30, Figure 7 · p. 31
  • +0.0220 [+0.0114, +0.0335]: at equal spend, the trained questioner’s split leads the single-model variant, although they are equivalent at the shared cap of eight.The per-suite equal-spend differences are +0.0257 [+0.0145, +0.0398] on MuSiQue and +0.0186 [+0.0029, +0.0380] on StrategyQA.Source: F THE EIGHT CONTROLS · p. 30, Figure 7 · p. 31
  • Answer-length and compute-budget controls ask nothing, so their positive coverage deltas do not rule out those explanations.Their deltas of +0.885 and +0.884 are true but not evidence; the controls leave the explanations unfalsified.Source: F THE EIGHT CONTROLS · p. 30

G STOPPING IN FULL

The stopping analysis finds a persistence gain but a trade-off in stopping when tasks are done and in question length. It tests training compositions, budget ceilings, and whether evidence gains translate into answer gains.

G STOPPING IN FULL

  • The trained questioner asks more often at unfinished states, but stops at finished states only 0.427 and 0.545 of the time and fails the question-length criterion on both suites.Its unfinished-state asking rates rise by +0.076 and +0.079; questions are four and a half and nine and a half words longer than the comparator’s.Source: G STOPPING IN FULL · p. 32, Figure 8 · p. 32, Table 15 · p. 32
  • Imitation omits states where no sampled question clears its floor, leaving the policy without a lesson for distinguishing hard unfinished states from finished ones.The exporter refuses 5,030 states beside 14,302 exported rows on the reference export, and 24,538 beside 43,837 on the pooled export.Source: G STOPPING IN FULL · p. 33
  • Stop contrasts alone raise persistence but buy no decided coverage gain; the final stage moves a question-pair-trained policy toward persistence while retaining coverage gains.The final stage and its starting policy gain +0.1284 and +0.1177 on MuSiQue, a point ordering rather than a tested difference.Source: G STOPPING IN FULL · p. 33, Table 16 · p. 33
  • Across five retrieval ceilings, the trained questioner is stopped by the retrieval cap less often than the prompted comparators and GPT-OSS-120B, but it also stops on unfinished work.The cap-stopped share falls from 34% to 8% on MuSiQue and 19% to 4% on StrategyQA, versus 13% to 69% for the comparators.Source: G STOPPING IN FULL · pp. 33–34, Figure 9 · p. 34
  • +0.102 [+0.074, +0.134] at the cheapest ceiling on MuSiQue: the trained questioner leads GPT-OSS-120B, prompted, while StrategyQA results diverge at higher ceilings.On StrategyQA the trained questioner trails GPT-OSS-120B from the second ceiling upward, including −0.027 [−0.050, −0.002] at eight and −0.056 [−0.079, −0.035] at twenty-four.Source: G STOPPING IN FULL · p. 34
  • Answer-node imitation covers the answer-bearing need less often than its matched control on all three suites, even when the answer node is directly supplied.The deficits are −0.095 [−0.148, −0.048], −0.075 [−0.125, −0.033], and −0.293 [−0.358, −0.233]; on 2WikiMultiHopQA, the nameable-node deficit exceeds the prerequisite-hidden deficit.Source: G STOPPING IN FULL · pp. 34–35

G.1 WHERE THE EVIDENCE GAIN IS LOST BEFORE THE ANSWER

  • Only MuSiQue has extra answer-node coverage to convert, and its implied gain is below detectable size; the stronger-reader test decides none of twelve cells.The stronger reader is Claude Opus 5, given retrieved paragraphs uncut and instructed to answer only from them.Source: §G.1 · p. 35, Table 17 · p. 35

H TRANSFER IN FULL

This section reports tool-use transfer and audits the validity and scope of the pair-ordering and need-graph evaluations. It finds transfer evidence in retail, alongside limits on what the benchmarks and validation checks establish.

H TRANSFER IN FULL

  • Retail task success rises in every seed and prompt variant, and gold-object hit rate and recall rise while the trained questioner asks slightly fewer questions.Against GPT-OSS-120B, prompted, retail follow-up turns fall and success rises in both prompt variants; airline differences in follow-up turns and success are undecided.Source: H TRANSFER IN FULL · pp. 36–37, Table 18 · p. 36, Table 20 · p. 37
  • In an exploratory Claude Sonnet 5 fork campaign, the questioner-drafter protocol needs 4.47 fewer retail follow-up turns, but the airline interval contains zero at every clustering unit.The campaigns were untrained and read on truncated records; Figure 10 reports fewer runaways on both domains, while only retail’s interval excludes zero.Source: H TRANSFER IN FULL · pp. 37–38, Figure 10 · p. 37

I.1 THE MODEL PANEL, AND ONE REPLACEMENT

  • The pair-ordering validation is mixed: two retained model raters agree with the rule above chance, but a replaced member is not distinguishable from chance and agreement varies by pair type.The two retained coefficients are 0.230 and 0.350; rule agreement with the panel majority is lower for pairs where the chosen side resolved no required need.Source: §I.1 · pp. 38–39, Table 21 · p. 38

I.2 TWO HUMAN RATERS

  • Two human raters pass hidden checks and align with the model panel overall, but the preference ordering is less reliable than judging which candidate reaches further beyond the user’s statement.On 280 matched pairs, retained model raters score 0.878 [0.798, 0.939] on reaching further and 0.556 [0.373, 0.709] on the better next question.Source: §I.2 · p. 39, Table 22 · p. 39

I.3 WHAT READERS MAKE OF THE NEED GRAPHS

  • Need-node readings cover only 1.3% of one suite, and optional-node assumptions can overstate coverage; random edge deletions can hide depth effects but did not reverse them.On 2WikiMultiHopQA, 96.2% [91.5, 98.4] of nodes labeled optional are actually required; the graph reading covers one suite, with no node-level reading on the other two.Source: §I.3 · pp. 39–40

I.4 DO THE EDGES GATE RETRIEVAL? AN AUTOMATED PROBE

  • Only 20 of 1,040 prerequisite edges pass the retrieval-gating test, so depth reflects where the benchmark decomposition places evidence rather than a barrier to retrieving it beforehand.With refuted edges removed, depth-weighted recall gains remain decided, whereas graphs retaining only validated edges are nearly flat.Source: §I.4 · p. 40, Table 23 · p. 40

I.5 GOLD DEFECTS, EXCLUSIONS, AND WHAT REMAINS UNMEASURED

  • Success figures exclude episodes whose reference rewards conflict with recorded outcomes, and the evaluation does not measure whether proactive questions harass simulated users.The section also leaves open whether need graphs survive users who are heterogeneous, impatient, or working from shifting instructions.Source: H TRANSFER IN FULL · p. 38, §I.5 · p. 41

J SCALE AND FAMILY

The scale ladder does not support ordering the trained arms by parameter count, and the 32-billion-parameter preference gain does not persist under length matching. Two prompt searches found nothing better than the shipped template.

J SCALE AND FAMILY

  • Parameter count does not order the trained arms: middle-size seeds depart in opposite directions on MuSiQue and StrategyQA, and the ladder is exploratory.The middle-size seeds are not separable; the comparison’s minimum detectable difference is 0.054.Source: J SCALE AND FAMILY · p. 41
  • At each arm’s own stop, eleven of fourteen checkpoint-by-suite cells go negative, so the ladder supports efficiency per question at gate-matched spend, not more final evidence.The prompted smallest model never stops at a covered state, unlike the middle and largest models.Source: J SCALE AND FAMILY · p. 41
  • On FRAMES, the same model, prompted, trails GPT-OSS-120B further at every ceiling, with an interval excluding zero from ceiling twelve upward.This comparison indicates the shortfall is size, not absence of training.Source: J SCALE AND FAMILY · p. 41
  • The 32-billion-parameter final checkpoint gains +0.031 [+0.001, +0.063] in ranking accuracy, but its length-matched gain is +0.009 [−0.029, +0.044].The 8B arm trained the same way gains +0.059 [+0.017, +0.096] on length-matched pairs; the two scales are not tested against each other.Source: J SCALE AND FAMILY · p. 42, Table 25 · p. 42
  • Three Granite 3.3 8B imitation seeds improve held-out coverage at equal spend on MuSiQue and 2WikiMultiHopQA, but trail the prompted model at their own stops on all three suites.Granite has no preference-trained checkpoint, so this is a first-rung reading, not a family contrast.Source: J SCALE AND FAMILY · p. 42

K PROMPT SEARCH OVER THE QUESTIONER

  • Neither prompt search found a better template: the first winner’s apparent gain was reproduced by length padding, and the second lost to the shipped template and length control.The second search used disjoint search, selection, and verdict slices; its design’s minimum detectable difference was 0.1585.Source: K PROMPT SEARCH OVER THE QUESTIONER · p. 42

L THE RUN SET ASIDE, AND FOUR COMPLETE RUNS

The four complete final-stage runs show positive equal-spend coverage gains across all three suites, while the reported questioner remains the average of seeds 1 and 2. The accompanying material records the run exclusion, prompt and firewall details, and the study’s scope boundaries.

L THE RUN SET ASIDE, AND FOUR COMPLETE RUNS

  • One preference-stage run was set aside after a trainer defect reloaded its starting weights, leaving it trained for only about the final tenth of the stage.It was excluded based on its adapter-distance measurement before outcomes of the other two runs were read; it was also the run selected on development stopping behavior.Source: L THE RUN SET ASIDE, AND FOUR COMPLETE… · pp. 42–43
  • The question-pair arm has nearly the trained questioner’s equal-spend coverage gains, but loses on StrategyQA at the shared cap of eight and on the widest constructed breadth shape.It asks fewer questions than the trained questioner and stops under three questions on 57%, 92% and 80% of runs.Source: L THE RUN SET ASIDE, AND FOUR COMPLETE… · p. 43
  • +0.113, +0.070 and +0.053 are the pooled equal-spend coverage gains across the three suites for four complete runs, each with an interval excluding zero.The runs differ by less than half of each run’s sampling error; the reading is exploratory, and the reported questioner remains the average of seeds 1 and 2.Source: L THE RUN SET ASIDE, AND FOUR COMPLETE… · p. 43, Table 26 · p. 43

M FURTHER RELATED WORK

  • Among related approaches to missing-information seeking, Q&D’s distinguishing label credits each question with the evidence its continuation actually retrieves.The appendix also notes that standard direct preference optimization was kept without comparing noisy-label variants.Source: M FURTHER RELATED WORK · pp. 43–44

N PROMPTS AND THE GOLD FIREWALL

  • The prompt favors latent required needs and stops when all are resolved, but string-leak coverage is absent for some suites and cannot bound paraphrase or gold entering prompts.The four firewall layers catch different leak paths, but coverage is per suite by design.Source: N PROMPTS AND THE GOLD FIREWALL · pp. 44–45

O.1 TERMS

  • Horizontal proactivity pursues an already nameable need; vertical proactivity pursues one made nameable by the need just resolved.The need graph records prerequisite edges and is recovered mechanically from benchmark decompositions.Source: §O.1 · p. 46

O.2 COMMON QUESTIONS

  • The trained questioner improves held-out evidence coverage over the prompted same model on all three suites, and in retail beats GPT-OSS-120B with fewer customer follow-ups.The paper also states that extra evidence does not yet reach final answers and training teaches what to ask more readily than when to stop.Source: §O.2 · p. 47

O.3 WHERE THIS WORK SITS

  • The work connects proactive agents and missing-need detection with question asking, agentic retrieval, decomposition, stepwise preference training, and simulated users.These connections are discussed as related lines of work rather than as a single unified approach.Source: §O.3 · pp. 47–48

Related papers

Loading 2609.37236v1…