PaperlayerView exact revision on arXiv

Source-linked AI summary

When Upstream Messages Override Correct Answers: A Controlled Study of Multi-Agent LLM Collaboration

Yaxin Gong, Gangyi Zhang, Chongming Gao, Leyang Shen, Chenxiao Fan, Jiakai Wang, Dong Wang, Yang Liu, Wenjie Wang, Xiangnan He

arXiv:2609.36855v1cs.AI

TL;DR

Prior work had not clearly separated the benefits of multi-agent communication from harms caused by incorrect messages. In controlled experiments across five benchmarks and five receivers, the authors find that messages can help or displace correct answers, with audited harmful overrides usually copying the upstream’s specific wrong answer.

  • Problem

    Prior work had not clearly separated the benefits of communication from the damage caused by incorrect messages in multi-agent systems.

    Source: §1 · pp. 1–2

  • Method

    The authors compare downstream answers with no message, the upstream message, or its opposite conclusion while fixing downstream evidence across five benchmarks and five receivers.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    Up to 32% of correct answers are overridden when an erroneous message is present, and 94% of audited harmful cases copy the upstream’s specific wrong answer.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Removing messages detected as likely wrong can recover some overridden correct answers, while the authors conclude communication should be selective.

    Source: §1 · p. 2

    The experiments use discrete, verifiable answers; whether substitution extends to open-ended generation, iterative debate, or longer chains remains untested.

    Source: §6 · p. 10

Abstract

from arXiv · show

Multi-agent LLM systems rely on message passing among specialized agents to accomplish complex tasks. However, an upstream agent may provide useful information or an incorrect answer that causes a downstream agent to override a correct answer supported by its own evidence. Prior work has not clearly separated the benefits of communication from the damage caused by incorrect messages. We study this problem with controlled experiments across five benchmarks and five receivers, keeping the downstream task and evidence fixed while comparing answers under three conditions: no message, the upstream agent's original message, or a message with the opposite conclusion. Our experiments reveal three key findings. First, messages often help when the downstream agent would otherwise answer incorrectly. Second, messages can also hurt: when the downstream agent would answer correctly without a message, an incorrect upstream message changes the answer in up to 32% of cases. Third, in 94% of audited harmful cases, the downstream agent copies the upstream's specific wrong answer--a pattern we term answer substitution. Removing unreliable messages recovers part of the lost accuracy, suggesting that communication should be selective based on upstream reliability and the evidence already available to the downstream agent.

1 INTRODUCTION

Multi-agent messages can propagate errors and override a downstream agent’s correct answer even when it has sufficient independent evidence. This study uses controlled, item-level comparisons to separate message benefits from harms and characterize when errors displace correct answers.

1 INTRODUCTION

  • Prior studies examined conformity, sycophancy, and knowledge conflicts, but did not isolate message effects with fixed evidence and controlled manipulation in real pipelines.The paper identifies a need for item-level causal analysis of communication in multi-agent systems.Source: §1 · pp. 1–2
  • Up to 32% of correct answers are overridden by an erroneous message, and 94% of audited overrides copy the upstream’s specific wrong answer.The authors call this pattern substitution; matched controls attribute it to message content rather than position, source label, or an additional reasoning opportunity.Source: §1 · p. 2
  • Messages help when the downstream agent would otherwise answer incorrectly, but their benefits and harms nearly cancel when it has independent evidence.More accurate upstream models reduce the number of affected items without eliminating per-item displacement.Source: §1 · p. 2
  • Removing messages detected as likely wrong, or replacing the downstream model, can recover a portion of overridden correct answers.The authors present selective communication as a lever for improving multi-agent reliability.Source: §1 · p. 2

2 ANALYSIS

The analysis studies a controlled two-agent handoff, defining measurable message effects and the behavioral pattern of substitution. It poses questions about conditional message value, directional displacement, and alternative explanations.

2.1 EXPERIMENTAL DESIGN

  • A single-message draft-review handoff pairs the upstream message with task evidence at the downstream agent; a 2 × 2 design measures message value and its interaction with evidence.Substitution is defined as switching from a correct answer without the message to the peer’s specific wrong answer after receiving it.Source: §2.1 · p. 3

2.2 RESEARCH QUESTIONS

  • The research questions test whether messages become harmful when the upstream errs, whether displacement targets the peer’s specific wrong answer, and whether reversing the peer conclusion shifts the receiver’s answer.The authors note that positive evidence–message interaction alone could reflect information redundancy, so directional and conclusion-sensitive shifts provide key evidence.Source: §2.2 · p. 3

2.3 ALTERNATIVE EXPLANATIONS TO NARROW

  • Matched controls test whether displacement instead arises from message order, teammate labeling, a second inference call, or scoring and formatting effects.These alternatives could produce apparent harm without the message content driving the answer change.Source: §2.3 · p. 4

3 EXPERIMENTS

Across five benchmarks and five receivers, messages help when upstream answers are correct but can harm when they are wrong. Conclusion-reversal and recovery experiments test how message content shapes answers and whether targeted interventions can restore accuracy.

3.1 SETUP

  • The base design crosses evidence present or absent with message shown or hidden across five benchmarks and five receivers.The primary upstream and receiver are gpt-4o-mini; L2W has few natural upstream errors, limiting statistical power in some analyses.Source: §3.1 · p. 4

3.3 MESSAGE CAUSES DIRECTED HARM

  • 11.1% of independently solvable items (297/2,667) are overridden across 25 cells, with a maximum single-cell rate of 32%.In 77 of 82 audited transitions (94%), the final answer matches the peer’s specific wrong answer; 90% of transitions with per-item correctness data occur when the upstream answer is wrong.Source: Figure 3 · p. 5, §3.3 · pp. 5–6

3.2 THE SAME MESSAGE HELPS AND HURTS UNDER DIFFERENT CONDITIONS

  • Across all 25 receiver–benchmark cells, messages help when upstream answers are correct but hurt when they are wrong.The conditional pattern persists across four upstream models, although stronger upstreams reduce the number of affected items.Source: §3.2 · p. 5

3.4 THE ANSWER FOLLOWS THE MESSAGE CONTENT

  • Across all 11 tested cells, corrupting a correct conclusion lowers accuracy and correcting a wrong conclusion restores it.Minimal edits to the answer and conclusion lines alone have an independent causal effect in four of nine cells, with effect sizes varying by receiver and task.Source: Figure 4 · p. 6, §3.4 · p. 6

3.5 NARROWING ALTERNATIVE EXPLANATIONS

  • Source-label changes do not reduce overrides, and delayed receipt preserves benefits from correct messages but fails to restore evidence use when upstream answers are wrong.Matched-call controls show more correct-to-wrong transitions with message exposure than without it; alternative scoring reduces but does not eliminate overrides.Source: §3.5 · p. 7

3.6 RECOVERY AND ITS LIMITS

  • Correct error locations yield the largest recovery gains, while removing messages and replacing receivers helps on upstream-wrong items but harms on upstream-correct items.Receiver replacement adds 10.9–24.1 percentage points on upstream-wrong items; practical use requires gating interventions on error detection, and the oracle-based experiments provide an upper bound.Source: §3.6 · pp. 7–8, Table 1 · p. 8

4 CHAIN-OF-THOUGHT DIAGNOSIS

The chain-of-thought analysis tests whether step-by-step reasoning counters overrides and uses reasoning traces to examine how receivers handle conflicting evidence. CoT does not reliably prevent overrides, and audited traces show evidence engagement despite following a peer’s wrong answer.

4 CHAIN-OF-THOUGHT DIAGNOSIS

  • The analysis uses chain-of-thought both as a potential defense and as traces of how receivers process conflicting information.The proposed test is whether explicit step-by-step reasoning helps receivers re-engage their own evidence.Source: §4 · p. 8
  • CoT does not reliably eliminate overrides: effects vary across models and tasks, and none of the HotpotQA changes reaches significance.On LBMusique, even the strongest reduction leaves the model overriding nearly half of items it would answer correctly alone.Source: §4 · p. 8
  • Annotators found receivers cited relevant evidence yet followed the peer’s wrong answer, with near-perfect agreement (AC1 = 0.98).The traces argue against a pure capacity account that predicts failure to engage with evidence; whether anchoring or another account explains the pattern remains open.Source: §4 · p. 8

5 RELATED WORK

The related work situates message substitution within research on multi-agent error propagation, social influence, knowledge conflict, and self-correction. This paper distinguishes its setting by testing a single message handoff while holding downstream evidence fixed.

5 RELATED WORK

  • Prior multi-agent studies document error propagation, while this work tests why wrong answers persist when a downstream agent has sufficient evidence to correct them.The study focuses on a single message handoff, described as the atomic unit of sequential pipelines.Source: §5 · pp. 8–9
  • Unlike peer-conformity studies, the experiments hold independent task evidence fixed while manipulating the message for item-level causal attribution.Whether the observed displacement shares a mechanism with group conformity remains open.Source: §5 · p. 9
  • The CoT analysis connects self-correction and reasoning-faithfulness work: receivers can cite correct evidence yet adopt a peer’s wrong conclusion.The paper states that this failure is not for lack of evidence engagement.Source: §5 · p. 9

6 DISCUSSION AND CONCLUSION

The discussion argues that message value depends on the receiver’s evidence state and upstream correctness, while controlled results do not resolve how receivers follow erroneous peers. Its scope is limited to oracle-quality evidence and discrete, verifiable tasks, leaving imperfect retrieval and broader generation settings open.

6 DISCUSSION AND CONCLUSION

  • The controlled results show a directional shift toward the peer’s specific wrong answer, but do not distinguish evidence replacement from excessive weighting of the peer claim.Receivers cite correct evidence yet follow the peer’s wrong answer in trace-level observations.Source: §6 · p. 9
  • Weighted mean τ = −0.6 pp (95% CI [−4.1, +3.3]): help and harm nearly cancel, and message removal is net-harmful on the full item set.The discussion cautions that pipelines should not stop passing messages; the value depends on receiver evidence and upstream correctness.Source: §6 · p. 9
  • All experiments use oracle-quality evidence, so substitution incidence may differ in deployments with imperfect retrieval.Source: §6 · p. 9
  • The findings are demonstrated on discrete, verifiable-answer tasks; extension to open-ended generation, iterative debate, and longer chains remains untested.Preserving communication benefits while protecting independent judgment when upstream messages are wrong remains open.Source: §6 · p. 10

A.1 BENCHMARK DETAILS

The benchmark overview defines “Upstream err.” as the fraction of items where the upstream agent, gpt-4o-mini, answers incorrectly.

A.1 BENCHMARK DETAILS

  • “Upstream err.” denotes the fraction of benchmark items answered incorrectly by the upstream agent, gpt-4o-mini.Source: Table 2 · p. 14

A.2 DECOMPOSITION OF KIMI-K2.6 UPSTREAM ON LBM

The LBM decomposition shows that positive message value on items with incorrect upstream answers comes from receivers that cannot solve those items alone; on independently solvable items, wrong conclusions can cause harm.

A.2 DECOMPOSITION OF KIMI-K2.6 UPSTREAM ON LBM

  • +10.5 pp message value with independent evidence on LBM items where kimi-k2.6’s upstream answer is wrong; the capability split shows help on hard items and harm on easy ones.Source: Table 3 · p. 14, §A.2 · p. 14
  • 91% (20/22) of items where the wrong message helped had receiver scores of zero without the peer message.These receivers could not solve those multi-hop questions without guidance.Source: §A.2 · p. 14
  • In 86% of those items, the gold answer appeared in kimi’s reasoning chain despite its incorrect final answer; receivers extracted intermediate steps, not the conclusion.Source: §A.2 · p. 14
  • Kimi’s mean reasoning length was 1,640 characters versus gpt-4o-mini’s approximately 400, yet its wrong conclusions could still override correct answers on easy items.Source: §A.2 · p. 14

A.3 PER-ITEM TRANSFER MATRIX (BIRD)

The BIRD per-item breakdown shows that wrong upstream messages harmed some correct answers but helped none in the reverse direction; a separate audit finds that harmful transitions usually adopt the peer’s specific wrong answer.

A.3 PER-ITEM TRANSFER MATRIX (BIRD)

  • 11 of 92 items were harmed, 0 were helped in the reverse direction, and 81 showed no change when the gpt-4o-mini receiver received a wrong upstream message on BIRD.Among unchanged items, 6 were correct and 75 incorrect under both conditions.Source: §A.3 · p. 14
  • 77 of 82 audited correct-to-wrong transitions (94%) adopted the peer’s specific wrong answer rather than producing an unrelated error.The audit spans four HotpotQA receivers and one LBM receiver.Source: §A.3 · pp. 14–15

A.4 BEHAVIORAL TAXONOMY

Table 4 classifies cases where the upstream answer is wrong and the receiver has independent evidence, distinguishing unconditional copying from following an error despite being able to solve the item alone.

A.4 BEHAVIORAL TAXONOMY

  • Table 4 reports behavioral-classification percentages for gpt-4o-mini, with sample sizes of 92 for BIRD, 70 for LBM, 100 for HQA, and 22 for DROP.Percentages are rounded independently; “Follows upstream error” combines unconditional copying with cases where the receiver can solve alone but still follows.Source: Table 4 · p. 15
  • 38–86% of cases involve unconditional following, where the receiver copies the upstream error but cannot solve the item independently.Table 4 covers items with an incorrect upstream answer and independent evidence available to the receiver.Source: §A.4 · p. 15, Table 4 · p. 15
  • In a further 6–23% of cases, the receiver can solve the item alone but still follows the upstream error when messaged.The passage identifies this as the most direct evidence for substitution.Source: §A.4 · p. 15

A.5 BIRD-HARD SUBSET

On the harder BIRD-Hard subset, the substitution interaction replicates: independent evidence reduces message value from substantial to near-zero.

A.5 BIRD-HARD SUBSET

  • +28.0 without independent evidence versus +1.3 with it yields Γ = +26.7 on BIRD-Hard, consistent with ΓBIRD = +25.3.BIRD-Hard uses the same pipeline and scoring as BIRD on 150 harder items, with 68.0% upstream errors.Source: §A.5 · p. 15

A.6 L2W RESULTS

This section presents supplementary analyses of how upstream messages interact with receivers’ evidence across benchmark cells, including recovery strategies, conclusion reversals, and responses to erroneous messages. It also describes controls for isolating message content and the QA evaluation measures, including L2W token-level F1 and a 0.5 threshold for binary decisions.

A.6 L2W RESULTS; A.7 DROP RESULTS

  • On L2W’s 16 naturally upstream-wrong items, message value is +46.6 without independent evidence and +21.8 with evidence (Γ = +24.8); DROP’s τ(with evidence) remains +23.2 pp.L2W conclusion reversal changes accuracy by +46.0 [25.0, 67.9] when correcting erroneous conclusions and −72.1 [−81.4, −61.9] when corrupting correct ones; DROP has the largest Γ of the five benchmarks.Source: §A.6 · p. 15, §A.7 · p. 15, Table 5 · p. 16

A.8 RECOVERY EXPERIMENT: INTERVENTION COMPARISON

  • Under gated recovery, removing the message and using a different-family receiver yields the largest reported gains: +9.3 pp on BIRD and +7.5 pp on LBM.Generic warnings improve BIRD by at most +0.7 pp, and answering before reviewing the message is statistically indistinguishable from simply omitting it on all three benchmarks; message removal is non-negative on upstream-wrong items but can discard useful signals overall.Source: §A.8 · pp. 16–17, Table 8 · p. 17, Table 9 · p. 17
  • On BIRD, the capability-matched same-family gpt-4.1-mini yields +17.4 pp recovery, while same-family and cross-family replacement means are statistically indistinguishable (25.0% vs. 27.2%, p = 0.50).Solo accuracy explains most recovery variance across replacement receivers (R2 = 0.77, p = 0.002); the results support using a different model, not a causal role for family diversity.Source: §A.8 · pp. 17–18, Table 10 · p. 18

A.9 MESSAGE VALUE AND INTERACTION (FULL 20-CELL TABLE); A.10 PER-ITEM BOOTSTRAP CIS FOR τ(WITH EVIDENCE)

  • All 20 receiver–benchmark cells show significantly positive Γ (p < 0.0001), and 13 of 20 τ(with evidence) estimates are significantly negative, with six surviving Bonferroni correction.Independent evidence sharply reduces the marginal value of the upstream message; the τ confidence intervals are paired-bootstrap estimates.Source: §A.9 · p. 18, Table 11 · p. 19, Table 12 · p. 19

A.11 CONDITIONAL DECOMPOSITION OF τ(WITH EVIDENCE); A.12 UPSTREAM MODEL DIVERSITY

  • Across the primary 20 cells, messages turn 274 of 2,184 independently solvable items wrong (12.5%); including DROP, the total is 297/2,667 (11.1%).All eight upstream–benchmark Γ values are significant (p < 0.0001), while τ(with evidence) is non-negative in all eight; indiscriminate removal is therefore not guaranteed to improve performance.Source: §A.12 · pp. 18–20, Table 14 · p. 20, Table 15 · p. 21

A.13 CONCLUSION-REVERSAL EXPERIMENT: FULL EFFECT SIZES; A.15 VALIDATION OF MESSAGES WITH REVERSED CONCLUSIONS

  • The six reported conclusion-reversal contrasts are significant, and reversing conclusions preserves 60–73% of tokens and 63–71% of evidence citations.The procedure changes connective reasoning to support the new conclusion; message lengths are closely matched by median ratio (0.97–1.13).Source: Figure 7 · p. 21, Table 16 · p. 21, §A.15 · p. 23

A.15 VALIDATION OF MESSAGES WITH REVERSED CONCLUSIONS

  • The reversal audit found 93.8% of messages passed all three semantic criteria, with 99.7% evidence grounding and 99.6% free of rewriting artifacts.When the upstream answer is correct, corrupting its conclusion performs worse than showing no message on all five benchmarks.Source: §A.15 · pp. 22–23, Table 19 · p. 22, Table 20 · p. 23

A.16 COT TRACE CLASSIFICATION PROTOCOL; A.17 COT TRACE EXAMPLES

  • Among 60 strong-model traces, annotators agreed on 59 (98.3%); representative cases show receivers following a peer despite recognizing conflicting evidence or adopting its answer through fabricated or altered reasoning.The classification covers 112 natural-error overrides across four models on HotpotQA and LBM; the six examples include evidence-engaged deference, invented justification, and reinterpretation to fit the peer’s answer.Source: §A.16 · pp. 23–24, §A.17 · pp. 24–25

A.17 COT TRACE EXAMPLES

  • In illustrative traces, receivers selectively used evidence to support a peer’s wrong answer, even when their unaided answer or reasoning identified the correct answer.Examples include switching from Laurence Olivier to Rhett Butler, selecting Birmingham City’s earlier league win over its later cup win, and answering Kenya after reasoning that Mara Region is in Tanzania.Source: §A.17 · p. 25

A.18.1 QUANTITATIVE RESULTS: NATURAL-ERROR VS. INJECTED-CONFLICT OVERRIDE RATES

  • CoT reduced injected-conflict overrides for deepseek from 45% to 9% (−36 pp) and kimi from 15% to 1% (−14 pp), but did not reliably reduce natural-error overrides.Three of four models showed no natural-error change beyond ±2 pp, while gpt-5.4 rose from 15% to 32%; the pattern replicated on LBM. Figure 8 gives the HotpotQA comparison, and Table 23 reports the LBM results.Source: Figure 8 · p. 26, §A.18.1 · p. 26, Table 23 · p. 26

A.18.2 WEAK MODELS ARTICULATE NO SUBSTANTIVE REASONING

  • In natural-error override cases, gpt-4o-mini and gpt-5.4 produced no substantive reasoning in 76% and 87% of cases despite a step-by-step prompt; deepseek-v3.2 and kimi-k2.6 produced substantive reasoning in every case.Additionally, 97% of deepseek traces and 100% of kimi traces were evidence-engaged: the receiver cited the correct evidence passages yet followed the peer’s wrong answer.Source: §A.18.2 · p. 26, §A.18.3 · p. 26

A.19 MINIMAL-EDIT CONCLUSION REVERSAL

  • On LBM, minimal conclusion edits caused 12/48 gpt-4o-mini and 13/51 deepseek correct-to-wrong flips (both p < 0.001); kimi’s 3/49 effect was not significant (p = 0.25).HotpotQA showed significant flips for gpt-4o-mini (9/81, p = 0.004) and deepseek (10/80, p = 0.002), while BIRD showed no significant sensitivity. The edit changed only the conclusion and at most one supporting sentence while preserving at least 95% of token content; the effect was attenuated for kimi and absent on BIRD.Source: §A.19 · pp. 26–27, Table 24 · p. 27

A.20 OUTPUT STABILITY UNDER REPEATED INDEPENDENT RUNS

  • Majority-vote classification of independent solvability agreed with single-run classifications for more than 96% of items overall.The receiver was classified as independently correct if it answered correctly in at least two of three evidence-only runs; all 25 cells were run three times independently.Source: §A.20 · p. 28

A.21 MATCHED-REVIEW CONTROL (HOTPOTQA)

  • With matched review prompts on HotpotQA, correct-to-wrong transitions exceeded reverse transitions 34 to 14 for gpt-4o-mini (p = 0.006) and 47 to 5 pooled across receivers on upstream-wrong items (p < 0.001).Because both branches used the same review step, with a neutral placeholder in the hidden branch, the pooled result supports message content—not merely a second review—as the source of damage.Source: §A.21 · pp. 28–29, Table 26 · p. 28

A.22 SOURCE-LABEL ATTRIBUTION CONTROL

  • On HotpotQA, no pairwise comparison among teammate, tool, unlabeled, and evidence-priority messages was significant; explicit evidence-priority instructions did not reduce harm.On LBM, samples were small and comparisons were nonsignificant. In the source-label experiment, 49/51 transitions adopted the peer’s specific wrong answer, showing that message content mattered more than its attribution.Source: §A.22 · pp. 29–30

A.24 RELEVANCE TO DEPLOYED MULTI-AGENT ARCHITECTURES

  • AFlow search found a two-node generator-to-finalizer architecture on HotpotQA and 2WikiMultihopQA, while a three-layer pilot showed message influence persisted at arbitration.AFlow achieved test-set F1 of 0.768 and 0.753, respectively, above single-agent baselines of 0.201 and 0.145; the pilot attributed the arbitration answer to an upstream message in 7 of 16 full-pipeline cases.Source: §A.24 · p. 30

A.26 PROMPT TEMPLATES

  • QA benchmarks (LBM, HotpotQA, and L2W) use token-level F1 with a 0.5 threshold for binary decisions.BIRD uses execution accuracy, while DROP uses exact match.Source: §A.26 · p. 32
Loading 2609.36855v1…