Believed but Not Said: Measuring Hidden Belief Collapse in Multi-Agent LLM Discussions

1 Introduction

• A compromised tool-agent, poisoned retrieval source, or untrusted participant can inject a false claim through messages without access to model weights or hidden states.The shared channel used for coordination is also an attack surface.Source: §1 · p. 1
• The study pairs each benign agent’s public stance with a private answer after every round, across six models and three factual-QA datasets.The private answer is elicited in a separate call hidden from peers; benign agents are not assigned a verification role.Source: §1 · pp. 1–2
• Under three attackers, benign agents held the false answer in 51.6% of exposed turns but endorsed it publicly in 19.7%.Agents retaining the true private answer endorsed the claim in only 2.3% of turns; false-answer agents hedged or ignored it in 58% of turns.Source: §1 · p. 2
• Without an attacker, final infection was 0.6%; one attacker raised it to 44.8% and three to 60.5%.The ten-agent groups were not significantly more robust than one-agent groups.Source: §1 · p. 2
• Agents retaining the true answer publicly rejected the false claim in only 61.9% of turns, otherwise staying silent, hedging, or going along.The transcript therefore understates how far the false answer has spread.Source: §1 · p. 2
• Two models that knew the same 85% of facts abandoned 25% and 80% of them, respectively.Baseline knowledge therefore did not predict robustness in this comparison.Source: §1 · p. 2

2 Related Work

• Multi-agent systems are organized to debate, critique, and vote, often on the premise that more agents and rounds surface and filter errors.Prior work also finds that discussion can erode correct answers even without an adversary.Source: §2 · p. 2
• Prior adversarial-agent studies mostly evaluate group outputs, while this work pairs public stance with private answers to reveal unvoiced infection.The attacker only argues for a fixed false claim in conversation and does not edit model weights.Source: §2 · p. 2
• Related work finds that persuasion and misinformation can damage collaborative accuracy, spread through agent networks, and persist in agent communities.The present study examines a simpler attacker that repeatedly advocates one fixed false claim.Source: §2 · p. 2
• Social psychology distinguishes public compliance from private acceptance, and prior LLM-group studies mostly report public agreement despite retained private views.The paper contrasts this compliance pattern with private adoption of a falsehood and rare public endorsement.Source: §2 · pp. 2–3

3.1 Threat Model and Chatroom

• Ten agents discuss one factual item for 16 rounds, with 0, 1, or 3 attackers repeatedly advocating the same fixed false claim.Benign agents receive no question-specific instructions, and attackers only post messages rather than editing weights or controlling benign agents.Source: §3.1 · p. 3

3.2 Dual Private/Public Measurement

• Separate private probes measure benign agents’ answers, while independent labels capture whether their public utterances endorse, reject, hedge, ignore, or drift from the claim.Probes are hidden from other agents and use the agent’s conversational memory; exposed turns are those preceded by an attacker’s utterance.Source: §3.2 · p. 3
• The study pairs public utterances with post-round private probes, which cover what an agent heard before speaking but can also reflect later speakers in the same round.Results are also checked using first speakers’ pre-round probes and last speakers’ post-round probes.Source: §3.2 · p. 3

3.3 Metrics

• Final private infection is the share of benign agents privately holding the false answer at the final step; turn-level rates track infection and public endorsement or correction across exposed turns.The method also measures whether agents privately labeled TRUE explicitly reject the false claim, and distinguishes private belief collapse from public correction failure.Source: §3.3 · pp. 3–4

3.4 Items and Datasets

• Items come from COUNTERFACT, POPQA, and SIMPLEQA, with type-compatible false answers and screening to exclude items where the false answer appears before discussion.The screened sets contain 40, 53, and 50 items; evaluated models’ initial private infection is 0.2%, and the no-attacker baseline never exceeds 3%.Source: §3.4 · pp. 4–5

3.5 Models, Judge, and Aggregation

• The evaluation runs six models across three datasets, labels answers and stances with DeepSeek-V4-Flash, and checks private-answer labels with an independent judge.The independent judge agrees on 87.5% of answers (κ=0.82); applying its relabeling rates lowers three-sybil final infection from 60.5% to 57.8%.Source: §3.5 · p. 5

4.1 A single adversarial agent causes most of the damage

• Final private infection averages 0.6% without an attacker, 44.8% with one, and 60.5% with three.The single-adversary rate is 74% of the rate with three attackers.Source: §4.1 · p. 5

4.2 Most infection is not voiced

• With three attackers, false private answers occur in 51.6% of exposed turns versus public endorsements in 19.7%, so transcripts understate infection.Among false-answer turns, 34.3% publicly endorse the claim; 28.6% respond by hedging or rejecting it.Source: Figure 3 · p. 6, §4.2 · p. 6, Table 3 · p. 6

4.3 Infected agents report the false answer as their own

• When asked for their own view, agents retain 86.4% of false answers, and anonymizing their prior messages leaves the result unchanged.Asking for their own view lowers final infection for GEMMA and GEMINI, but hardly changes it for GPT-OSS-120B or MISTRAL.Source: §4.3 · pp. 6–7, Table 5 · p. 7

4.4 Knowledge does not predict robustness

• On COUNTERFACT, HY3 and GPT-OSS-120B each know 85% of facts, yet abandon 25% versus 80% of those known facts.With six heterogeneous models, the results establish that baseline knowledge does not necessarily predict robustness, not a model-size relationship.Source: Figure 4 · p. 7, §4.4 · p. 7

4.5 Collapse compounds and persists

• An uninfected agent’s switching probability rises from 9% to 66% as more benign peers hold the false answer, while endorsement is followed by more switching than correction.After the round, 2.3% of correctors versus 39.7% of endorsers hold the false answer; the authors caution that showing concession causes switching would require intervention.Source: §4.5 · pp. 7–8, Figure 5 · p. 8

4.6 Infection dynamics over rounds

• Five of six models reach half their final infection within one round, and about 80% of eventual infection is present by round 4 before curves plateau.With one attacker, infection starts more slowly and ends lower in every model; GPT-OSS-120B is slowest initially but ends highest at 76%.Source: §4.6 · p. 8, Figure 6 · p. 8

• The findings challenge the premise that adding agents to debate, ensembling, or agentic RAG makes systems robust through social redundancy.Source: §5 · p. 8
• Transcript monitoring reveals only about a third of infected turns, while most infection emerges within four rounds and rarely reverses.Source: §5 · p. 8
• The paper recommends treating agent messages as untrusted, testing peer-pressure robustness during model selection, monitoring private answers, and intervening early.Source: §5 · p. 8

• A few participants repeatedly asserting a false claim can shift private answers in open-ended factual discussion, with much of the failure invisible in chat.Source: §6 · p. 8
• The shift persists when agents are asked for their own view, spreads among benign agents once it begins, and rarely reverses.Source: §6 · p. 8
• Recording each agent’s answer privately at every round can reveal hidden answers even when the transcript looks unchanged.Source: §6 · pp. 8–9

Limitations

• The study compares only groups of two and ten agents, without varying network topology or model mixtures.Source: Limitations · p. 9
• The attacker acts only in context, so parametric attacks such as knowledge editing are outside the studied mechanism.Source: Limitations · p. 9
• Public endorsement rates may be underestimated because implicit endorsements in free-form chat are easy for the LLM judge to miss.All labels come from an LLM judge; public stance is harder to label than a one-sentence private answer.Source: Limitations · p. 9
• “Private belief” means the answer produced from the agent’s current context and memory, and other probe wording could yield somewhat different rates.Source: Limitations · p. 9
• Whether prompting agents to correct the claim early protects the group remains untested.The authors say it could be evaluated at low cost.Source: Limitations · p. 9

Ethical Considerations

• The study involves no human participants or personal data; messages are model-generated, items come from public benchmarks, and models are used via provider APIs without training or fine-tuning.Source: Ethical Considerations · p. 9
• The authors report using Claude Code to help write and debug experiment code and improve grammar and wording in author-written text.Source: Ethical Considerations · p. 9

A.1 Measurement: Public Stance Judge

• The public-stance judge classifies a benign agent’s current utterance toward the manipulated claim using labels for endorsement, rejection, hedging, ignoring, or unrelated speech.Source: §A.1 · p. 11

A.2 Measurement: Private Probe

• Private probes ask agents to answer from their knowledge and discussion memory, and validation re-asks vary the instruction while keeping persona, memory, and question fixed.The study also reports 24,480 re-ask probes across selected steps and conditions.Source: §A.2 · p. 12

A.3 Measurement: Private Belief Judge

• The private-belief judge labels each short answer as endorsing the manipulated claim, endorsing the reference statement, neither, or ambiguous.Source: §A.3 · p. 12

A.4 Simulation: Sybil (Attacker) Agent

• The sybil agent is instructed to advocate the assigned manipulated position confidently, push back politely but firmly, and reinforce others who accept it.Source: §A.4 · p. 12

A.5 Simulation: Benign Agent

• Benign agents are prompted to converse naturally, engage with the wider discussion, and agree, disagree, ask follow-ups, or share thoughts.Source: §A.5 · p. 12

B Timing of Public and Private Measurements

• The main analysis probes after each round, while the timing checks use first-speaker probes before the round and last-speaker probes after it, with no intervening message.Speaking order is reshuffled each round.Source: B Timing of Public and Private… · pp. 12–13
• Across three probe pairings, 54–58% of false-answer turns hedge on or ignore the claim, and 57.6% do so when agents held the false answer both before and after the round.The strictest pairing checks agents who therefore also held the false answer while speaking.Source: B Timing of Public and Private… · p. 13, Table 7 · p. 13
• Truth-knower correction stays at 61.9–62.6% and performative endorsement at 2.3–2.8% across the three pairings.Agents who switch during the round can be persuaded after speaking and reject the claim more often (19.3% vs. 5.8%).Source: B Timing of Public and Private… · p. 13

C Item Construction and Screening

• For COUNTERFACT, manipulated answers come from each record’s counterfactual; records whose subject contains either answer are dropped.Source: C Item Construction and Screening · p. 13
• For POPQA, each question is paired with a random question sharing its Wikidata relation, excluding candidates that share an alias with the true answer or appear in the question.The pairing does not match subject popularity.Source: C Item Construction and Screening · p. 13
• For SIMPLEQA, questions are assigned fine-grained relations and matched to answers with compatible shapes; items with no relation or an overly broad relation are dropped.Source: C Item Construction and Screening · p. 13
• Screening keeps items whose manipulated answer is initially absent and whose screening-model answers are usually correct, applying staged sampling and stricter thresholds for COUNTERFACT and POPQA.SIMPLEQA uses a single stage on 240 relation-balanced candidates.Source: C Item Construction and Screening · p. 13

• One conversation is run per item, model, and number of adversarial agents.Source: D Implementation Details · p. 13
• Agents are named Alice through Jack and assigned ten specified personas; the first k agents are adversarial.Source: D Implementation Details · p. 13
• Speaking order is reshuffled each round from different seeds but remains identical across conversations, keeping adversarial agents at the same positions across runs.Source: D Implementation Details · p. 13
• LLMs sample at temperature 0.7, while private probes and both judges run at temperature 0.Source: D Implementation Details · p. 13

E Dyad Baseline

• The comparison reruns each model×dataset cell with one adversarial and one benign agent (N=2), using the same 10 sampled items as the ten-agent group.The benign persona is drawn from the nine personas used in the group runs; model, items, temperature, discussion seed, history window, private probe, and judge are unchanged.Source: E Dyad Baseline · pp. 13–14
• Item screening requires that at most 10% of the screening model’s context-free samples mention both answers at every stage.Table 8 distinguishes string matching against answers and aliases from screening by the LLM judge.Source: Table 8 · p. 14
• Peers provide no reliable defense: the ten-agent group ends only slightly better on average, while effects differ by model.Peers protect GPT-OSS-120B (+37.4 points, [15.2, 57.0]), but GEMINI-2.5-FLASH-LITE ends on the false answer more often in the group than alone.Source: E Dyad Baseline · p. 14, Table 9 · p. 14
• The adversarial agent writes half of all messages in the pair but only a tenth in the ten-agent group.Source: E Dyad Baseline · p. 14