Source-linked AI summary
Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent Messages
Chih-Hsuan Yang, Anjir Ahmed Chowdhury, Cheng-Hau Yang, Weijian Zheng, Fernando Llorente, Xiaolong Ma, Xinyang Li, Eliu A. Huerta, Ian T. Foster, Rajeev Thakur
TL;DR
Answer correctness does not reveal whether a message will help downstream reasoning. Using DHD to compare integration with and without fixed messages, the paper finds wrong-helpful messages across all benchmark-model combinations, with more than four in ten correctness-changing cases helping.
Problem
Reliability proxies do not establish whether a message’s reasoning will help or harm downstream integration, because correctness does not determine trajectory value.
Method
DHD caches five independently generated messages and measures each message’s trajectory value by comparing matched integration runs with and without it.
Results
More than four in ten correctness-changing wrong-answer messages are helpful across all five benchmarks and both model families.
Takeaways & Limitations
Answer correctness is informative but insufficient for deciding whether agents should keep or remove a message during downstream reasoning.
Takeaways & Limitations
Trajectory-value labels are context-specific, and repetition reduces but does not eliminate uncertainty; same-problem evidence does not establish unseen-problem generalization.
Abstract
from arXiv · showhide
Multi-agent reasoning systems often use agreement, confidence, or automated scores to decide which messages should shape a final answer. Such filtering assumes that a message likely to be correct is also worth keeping. Yet a wrong answer can contain a useful decomposition, constraint, or scientific principle. We test this distinction with Diverse Hypothesis Deliberation (DHD), a controlled measurement protocol that caches five independently generated messages and replays the same downstream solver, called the integrator, with each message available or hidden. The replay comparison measures a message's trajectory value: whether making the message available helps or harms subsequent reasoning. Across five mathematics and science benchmarks and two openly available model families, gpt-oss-120b and gemma-4-31B-it, wrong-helpful messages appear in every benchmark-model combination. Among wrong-answer messages that change final correctness, more than four in ten changes are helpful in each model. Controlled repeats show that the number of repeatable message effects is unlikely to arise from replay variation alone (p=0.0002). A focused intervention on repeatable wrong-helpful messages finds that the complete message works best, while retaining its reasoning preserves more success than retaining only its answer; the source of the complete-message advantage remains open. Within the same problem, repeated trajectory-value evidence also identifies a better keep-or-remove choice than answer correctness alone. Answer correctness is therefore informative but does not determine trajectory value. DHD measures this missing property and produces reusable labels for learning when agents should listen.
Introduction
The introduction distinguishes a message’s answer correctness from its context-specific trajectory value: whether making the whole message available helps or harms downstream reasoning. It presents DHD as a fixed-pool replay protocol for measuring this distinction and reports that wrong-helpful messages occur broadly, with reasoning often contributing more than an attached wrong answer.
- Conceptual distinction: Proposal correctness asks whether a message’s answer is right, whereas trajectory value asks whether the whole message helps or harms subsequent reasoning.A correct answer can accompany misleading reasoning, while a wrong answer can contain a useful decomposition, constraint, or scientific principle.
- Practical contribution: DHD’s repeated value evidence supports message-selection labels that identify when agents should preserve or prioritize messages beyond answer correctness alone.The introduction frames these labels as reusable and context-indexed for learning message-selection behavior.
- Measurement protocol: DHD fixes five independently generated messages and compares matched replays in which one message is available versus hidden.The protocol creates the message pool before observing answers or downstream effects and applies no filtering by answer, confidence, or later effect.
- Empirical findings: Wrong-helpful messages appear in every benchmark–model combination, and more than four in ten wrong-answer messages that change final correctness produce a helpful change.The study covers five mathematics and science benchmarks and two independently developed model families.
- Empirical findings: A controlled Gemma intervention finds more evidence that retaining reasoning benefits integration than retaining the attached wrong answer.The test separately hides reasoning and answer fields to probe why wrong-helpful effects persist.
Related Work
Prior work scores reasoning by correctness, revises solvers through feedback, and selects or aggregates among multiple reasoning paths. DHD instead measures the contextual contribution of each complete cross-agent message by replaying it within a fixed generated pool and pairing its effect with answer correctness.
- Correctness supervision and refinement: Correctness supervision scores steps or search states by association with correct solutions, while self-refinement and critique revise solvers using feedback.Models may resist or misuse even strong feedback.
- Imperfect information can still help: Failed trajectories and incorrect demonstrations can still provide useful training or in-context-learning signals, motivating trajectory value as a contextual complement.Trajectory value measures which cross-agent message helps or harms in its current context.
- Multiple reasoning paths and selection: Selection and pruning methods maintain multiple reasoning paths and seek better or cheaper final solutions, whereas DHD fixes a generated pool and tests each complete message’s contribution.The comparison covers self-consistency, search, hypothesis generation, candidate ranking, confidence-weighted consensus, and communication pruning.
- Multi-agent aggregation and contribution: Multi-agent debate, role-based collaboration, and reconciliation aggregate reasoning paths, while DHD measures contribution per message and separates candidate availability from successful integration.Related work also studies communication topology and agent-level importance.
- Replay attribution and positioning: Replay-based removal can attribute whole-agent contributions and failures, but DHD pairs each message’s replay effect with its own answer correctness and conditions trajectory value on the observed pool.Unlike Shapley-style attribution, DHD’s leave-one-out contrast does not average across coalitions.
Method
DHD separates a message’s answer correctness from its trajectory value by replaying fixed, independently generated messages with a downstream integrator. Matched visibility comparisons classify each message as helpful, neutral, or harmful in context.
- Definitions: Proposal correctness evaluates only a message’s proposed answer, whereas trajectory value measures the downstream consequence of exposing its reasoning and answer together.The two properties are evaluated on the complete structured message, not on its answer field alone.
- Replay comparison: Helpful means only the message-visible run is correct, harmful means only the hidden-message run is correct, and neutral means both runs have the same correctness.Observed effects are represented as ∆i,K ∈ {+1, 0, −1}.
- Protocol: DHD recruits five complementary roles to generate independent structured messages once, then replays selected cached subsets through a separate integrator.Hypothesizers see only the problem and assigned role; roles and messages are never regenerated during replay.
- Replay contexts: Single-message replay tests whether one message creates a successful route from independent solving, while in-pool leave-one-out tests usefulness, redundancy, or harm amid other messages.The K=0 baseline is an independent solver, and in-pool LOO is used for K=2, . . . , 5.
- Replay comparison: Matched replay pairs hold the problem, model, prompt, cached messages, ordering, and evaluation fixed while changing only whether the target message is visible.This contrast measures whole-message availability in context, although removal also changes prompt length and token positions.
- Labels: Crossing correctness with replay effects yields six categories, including wrong-helpful and correct-harmful, demonstrating that correctness is neither necessary nor sufficient for positive trajectory value.The method separately reports correctness-flip rate and, among flips, the share that are helpful.
Experimental Setup
The study evaluates DHD across five mathematics and science benchmarks using standardized role-specific decoding and answer-equivalence evaluation. Its leave-one-out scans, controlled replay repeats, and robustness audits quantify message effects while preserving matched comparisons and explicit measurement costs.
- Benchmarks: Five benchmarks cover open-answer mathematics, mixed-format exams, college science, text-only biology, and materials science across 6,666 problems.The benchmarks are Omni-MATH-2 (4,181), JEEBench (515), SciBench (580), LAB-Bench (741), and MaScQA (649).
- Standardized decoding: Hypothesizers use temperature 0.7, while recruiters, integrators, and evaluators use temperature 0 under shared evaluation settings.A separate gpt-oss-120b evaluator applies the same answer-equivalence procedure across both model families, without exposing ground truth or evaluator judgments to reasoning agents.
- Replay controls: Controlled replay samples 1,000 problems per model family and produces 35 observations per problem to estimate message effects and ordinary replay disagreement.Each of five matched blocks evaluates the same full-pool input twice and each of five one-message removals.
- DHD protocol: A K=5 leave-one-out scan reuses five cached messages for six integrator outcomes: one full-pool result and five one-message removals.Message-level analyses require rendered messages and complete matched comparisons; missing replays are never imputed as neutral.
- Measurement effort: DHD is a measurement protocol: controlled repeated replay uses 35 integrator outcomes per sampled problem, while four Omni-MATH-2 protocols average roughly 18K to 616K logged tokens per problem.Recruitment and the five messages are generated once, and statistical analyses make no model calls.
- Evaluator robustness: Robustness replay finds 94.2–96.6% pairwise agreement on individual answers, while simultaneous six-cell-label agreement is 71.6–78.6% for OSS and 82.4–88.7% for Gemma.Every evaluator recovers both off-diagonal categories.
Results
DHD shows that message availability, integration, and trajectory-value identification are distinct: final accuracy can conceal opposing message effects, while wrong-answer messages can still help or harm. Repeated replay and same-problem selection further show that trajectory value is measurable beyond proposal correctness.
- Integration outcomes: Moving from K=0 to K=5 changes macro-average accuracy by +1.6 points for OSS and +0.3 for Gemma, with mixed benchmark directions.Averaged final accuracy can cancel helpful, harmful, and neutral message effects.
- Integration outcomes: Correct candidates appear in 76.0–95.4% of problems, yet candidate availability exceeds K=5 accuracy by 4.2–36.2 points.Availability does not establish usable reasoning or successful integration.
- Trajectory value: Among flips caused by wrong-answer messages, 41.9% of OSS events and 45.3% of Gemma events are helpful.Wrong answers make harm more likely but do not determine whether the complete message helps or harms; wrong-helpful observations occur in all ten benchmark–model cells.
- Repeatability: Controlled repeats find p = 0.0002 for the aggregate effect count, showing message-availability effects beyond ordinary replay variation in both model families.Exact-input full-pool calls disagree on final correctness in 7.3% of OSS pairs and 2.1% of Gemma pairs.
- Selection: Same-problem one-removal selection improves accuracy by 1.68 points for OSS and 2.61 for Gemma, versus 0.94 and 1.00 for proposal correctness.The repeated trajectory-value evidence identifies a better keep-or-remove choice within the same problem.
Discussion
The discussion shows that answer correctness does not determine trajectory value: wrong messages can help downstream reasoning, often through their reasoning rather than their attached answers. DHD measures this distinction while deliberately separating message generation, selection, and integration rather than maximizing accuracy.
- Empirical findings: More than four in ten wrong-answer messages that change final correctness produce a helpful change in both model families.Controlled repetition detects message-availability effects beyond ordinary replay variation, including multiplicity-controlled Gemma wrong-helpful cases across all five benchmarks.
- Empirical findings: Component masking localizes more observed benefit to message reasoning than to the attached wrong answer.This supports the distinction between answer correctness and reusable reasoning content.
- Implications: Repeated trajectory-value estimates contain decision-relevant information beyond correctness, although deployment would require predicting value before evaluation and transferring to unseen problems.The study defines this prediction target but does not yet provide that deployment capability.
- Limitations: DHD is a measurement protocol rather than an accuracy-maximizing protocol, because its fixed role-diverse pool separates generation, selection, and integration.Editing, filtering, or regenerating messages might improve accuracy but would entangle the stages DHD is designed to separate.
Limitations
The study’s trajectory-value labels depend on the message, pool, integrator, and evaluation context rather than being intrinsic text properties. Its findings may not generalize across unseen problems, interactive or heterogeneous agents, frontier models, or tasks without stable ground truth.
- Context dependence: Trajectory-value signs can change for fixed problems and messages because labels describe a message–pool–integrator context, not an intrinsic text property.Repetition reduces but does not eliminate this uncertainty.
- Evaluation limits: The fifth-block analysis measures same-problem opportunity rather than generalization to unseen problems.LOO also hides a whole message in one fixed prompt order, while the smaller masking diagnostic only begins to separate reasoning from its answer field.
- Scope of agents and models: Nested K prefixes are not randomized agent additions, and one model family fills all reasoning roles within each run.Results may not transfer to interactive debate, heterogeneous agent models, or frontier models.
- Task and evaluator limits: Results may not transfer to tasks without a stable ground-truth answer, while Gemma’s ratings condition on complete replay records and compound labels are more evaluator-sensitive than individual answers.The Appendix provides sensitivity tests, sample flow, and evaluator audits.
Conclusion
Across five benchmarks and two model families, wrong-answer messages can still improve final reasoning, so answer correctness alone should not determine whether a message is discarded. DHD measures this distinction by replaying fixed messages with and without integration, while diagnostics suggest reasoning contributes more than the wrong answer field.
- More than four in ten correctness flips involving wrong-answer messages improve the final result across five benchmarks and two model families.
- DHD measures message value by holding candidate messages fixed and comparing downstream integration with and without each message.
- Controlled replay detects message-availability effects beyond ordinary replay variation in both model families, while component masking more consistently supports reasoning as the benefit source.Multiplicity-controlled wrong-helpful cases are recovered for Gemma, not OSS.
Appendix Overview
The appendix provides a compact evidence map for the paper’s central claims, covering protocol details, system-level scope and costs, complete results, and offline selection and integration. It also includes cross-benchmark trace examples and robustness analysis of replay variation and repeatable effects.
- Evidence map: The appendix maps the paper’s central claims across protocol definitions, role information boundaries, system scope, costs, complete results, and offline selection and integration.DHD is the measurement protocol defined in the main paper, while the system-level section covers four protocols, two models, and five benchmarks.
- Evidence map: Real Trace Examples Across Benchmarks provides paired examples from every benchmark.
- Evidence map: Controlled Repeated Replay Robustness measures replay variation and repeatable effects.
Component Masking: Hiding Reasoning and Answer Fields
The section situates message-level mechanism analysis alongside same-problem selection opportunity and evaluation procedures covering agreement, missing data, provenance, and artifact structure.
- Component Masking: Hiding Reasoning and Answer Fields: Message-level mechanisms are examined through matched replay pairs and integrator uptake.
- Component Masking: Hiding Reasoning and Answer Fields: Same-problem selection opportunity is tested with Cross-Fitted One-Removal Opportunity.
- Component Masking: Hiding Reasoning and Answer Fields: Evaluation procedures record evaluator agreement, missing-data treatment, provenance, and the artifact schema.
Prompt Interface Details
DHD uses four separate model calls with shared benchmark templates and distinct information boundaries: recruiter, hypothesizer, integrator, and evaluator. Replay changes only which cached messages the integrator sees, enabling trajectory-value estimates in different comparison contexts.
- The recruiter receives the problem and proposes five dynamic, problem-specific solver roles without access to the ground-truth answer.
- Each hypothesizer receives the problem and one assigned role, then produces structured reasoning, assumptions, checks, uncertainty, and a possible final answer independently.Hypothesizers do not see other messages, evaluator output, or ground truth.
- The integrator receives the problem and selected cached messages, synthesizing, correcting, combining, or rejecting them rather than voting or selecting a proposed answer.It outputs one final answer in the benchmark’s expected format and is not told which proposed answer is correct.
- The evaluator sees only the submitted final answer and evaluator-only ground-truth material, producing a correctness label without solving or choosing messages.
- Single-message and leave-one-out replays estimate trajectory value under different contexts by changing only the integrator’s visible message subset.Leave-one-out retains the fixed original message set while hiding one message; full-K contains all K cached messages, and no replay regenerates roles or messages.
Complete Results and Analysis Scope
This section defines the complete analysis scope, including benchmark deduplication and the distinction between protocol-level accuracy and message-level replay analyses. Replay analyses condition on rendered five-message pools and preserve problem-level dependence during resampling.
- Model scope: Model-family abbreviations are OSS for gpt-oss-120b and Gemma for gemma-4-31B-it.These abbreviations are used to organize the complete results by model family.
- Benchmark scope: 649 unique MaScQA problems remain after collapsing one exact duplicate, while SciBench retains 580 distinct questions after resolving three reused source identifiers.SciBench disambiguation uses problem-text hashes.
- Analysis scope: Protocol accuracy measures complete-workflow success across all assigned problems, whereas message-level replay conditions on rendered five-message pools and required removal outcomes.The protocol and LOO samples therefore differ by analysis conditioning rather than unexplained attrition.
- Resampling: Resampling draws problem identifiers with replacement and retains every K=5-by-removal observation for each problem, preventing repeated messages from being treated as independent examples.This preserves within-problem dependence in the replay analyses.
System-Level Protocol and Cost Context … Evaluation Procedure and Artifact Schema
The paper separates message-level trajectory-value measurement from end-to-end protocol accuracy, using replay-based diagnostics and recorded outcomes to show that answer correctness is informative but insufficient for selecting useful messages. Across robustness checks, trace audits, component interventions, and cross-fitted decisions, repeated message-specific evidence improves keep-or-remove choices while revealing heterogeneous mechanisms and substantial cost differences.
- System-Level Protocol and Cost Context: DHD measures available-versus-hidden effects of fixed five-message pools, whereas system-level comparisons evaluate complete protocols with different prompts, samples, computational costs, and inference budgets.A complete K=5 leave-one-out scan uses one full-pool integration plus five removals; controlled repeated replay uses 35 integrator outcomes per sampled problem.
- Offline Selection and Integration Diagnostics: Offline analyses reuse complete recorded outcomes from 6,666 OSS and 6,017 Gemma problems to compare single-message selection, full-pool integration, interference, synergy, and proposal-correctness diagnostics.Budget-one policies expose the integrator to one selected recorded message, while K=5 integration exposes it to all recorded messages; these are outcome diagnostics rather than deployable expected-policy estimates.
- Offline Selection and Integration Diagnostics: Correctness-based selection improves over random but leaves 7.1 OSS points and 5.5 Gemma points to the best recorded outcome, missing wrong-answer routes and differing outcomes among correct-answer messages.The result identifies information needed by a value-aware rule, not a deployable routing policy.
- Real Trace Examples Across Benchmarks; Omni-MATH-2; JEEBench; SciBench; LAB-Bench; MaScQA: Trace examples across five benchmarks show wrong-helpful messages supplying reusable principles, decompositions, or checks that repair downstream answers despite incorrect proposals, alongside correct-harmful messages that propagate errors or create interference.Examples include an omitted boundary case, a wrong box direction, a one-point GC overestimate, and incorrect or conflicting scientific assumptions.
- Controlled Repeated Replay Robustness: Replay stability differed across domains: identical-input calls disagreed on 363 of 5,000 pairs (7.3%), with rates ranging from 1.6% on MaScQA to 17.5% on LAB-Bench.The rate includes integrator-output and subsequent-evaluation variation, so individual LAB-Bench replays warrant greater caution.
- Controlled Repeated Replay Robustness: Repeated comparisons found weak cross-integrator correspondence, with Pearson r = 0.095 and only 10 of 20 jointly nonzero messages retaining the same direction, while historical labels agreed on 88.98% of 34,748 events.The historical pairing had weighted Cohen’s κ = 0.448 and its lowest agreement was 73.36% on LAB-Bench; disagreements were neutral versus non-neutral.
- Cross-Fitted One-Removal Opportunity; Integrator Uptake in Matched Replay Pairs; Evaluation Procedure and Artifact Schema: Cross-fitted repeated effects selected the full pool in 81.3% of OSS folds and 92.5% of Gemma folds, while message-specific gains exceeded correctness-only gains: 1.68 versus 0.94 OSS points and 2.61 versus 1.00 Gemma points.Across variants, the largest one-sided p-value was 0.002; the analysis quantifies same-problem repeated-evidence opportunity rather than demonstrating an online selector.