Multi-agent discussion gains less when dissent is withheld

1 INTRODUCTION

• Discussion outcomes in prior LLM studies conflict, with reports of gains, no effect, and task-dependent effects of conformity (Du et al., 2024; Liang et al., 2024; Chen et al., 2024; Smit et al., 2024; Wang et al., 2024b; Kaesberg et al., 2025; Zhang et al., 2024).The paper identifies the factors behind these divergent outcomes as unclear.Source: §1 · pp. 1–2
• The paper proposes a model in which discussion can outperform majority voting only in regimes determined by withholding rate and single-agent accuracy.These quantities separate cases where discussion outperforms voting from cases ending in incorrect consensus.Source: §1 · p. 2
• Four recurring behaviors ground the model: withholding dissent, internalizing stated answers, reconsidering after visible dissent, and correcting toward the correct answer.Withholding suppresses correction triggers, while internalization can turn conformed statements into private beliefs.Source: §1 · p. 2
• Discussion can recover from an incorrect initial majority only below a critical withholding rate c*, placing outcomes in regimes defined by single-agent accuracy p and withholding c.The model uses this threshold to distinguish recovery from incorrect consensus.Source: §1 · p. 2
• Across benchmarks, discussion gains diminish as teams move farther above the critical threshold; honesty instructions reduce withholding, while cohesion instructions push teams farther above it.The reported relationship spans HiddenBench, MedEInst, and MuSiQue.Source: §1 · p. 2
• Incorrect consensus can result from silent dissent or from voiced dissent failing to change answers when internalization is high and net correction is low.The paper reports that disabling reasoning lowers internalization and raises net correction, increasing c*; reasoning agents may also keep contrary evidence private.Source: §1 · p. 2

2.1 AGENTS AND UPDATE RULES

• Agents hold binary private beliefs and make binary public statements about correctness, beginning with independent accuracy p; the analysis uses a mean-field limit.Only public statements are visible to other agents, and agents effectively see random samples of peers.Source: §2.1 · p. 3, §2.2 · p. 3
• When an agent's belief differs from the local majority, it conforms with probability c or states its belief; conforming can be internalized with probability a, while stated dissent can trigger reconsideration and correction.Reconsideration occurs with probability proportional to visible dissent and reaches the correct answer with probability r.Source: §2.1 · p. 3, §2.2 · p. 4

2.2 CRITICAL WITHHOLDING RATE

• The correct minority grows when (1 − c)γ > c · a, yielding the critical withholding rate c* = γ/(γ + a).Here γ = ρ(2r − 1); correction raises the threshold and internalization lowers it.Source: §2.2 · p. 4
• For γ > 0, the all-correct state is stable at any c; above c*, two stable states coexist, and initial accuracy selects the destination across phase boundary p*(c).The recovery regime is p*(c) < p < 1/2.Source: §2.2 · p. 4
• The recovery threshold requires local majority rule with q ≥ 3; with only one observed counterpart, no threshold appears.The analysis considers a small correct minority near the state where nearly all agents are incorrect.Source: §2.2 · p. 4

3 PARAMETER ESTIMATION

• Public statements are read from discussion rounds, while private beliefs are measured with isolated queries asking the same question without showing the answer to others.The elicited private response is treated as a measurement counterpart, not as a claim about an LLM's actual internal belief.Source: §3 · p. 5
• A dissent opportunity occurs when the prior private answer differs from the visible majority; stating the majority measures withholding, and matching it in the private query measures internalization.The two answers are elicited in separate calls within the same round.Source: §3 · p. 5
• Bayesian inference estimates withholding c, internalization a, and net correction γ, then yields posterior estimates of the margin between withholding and c*.The margin indicates which side of the incorrect-consensus threshold an LLM system occupies.Source: §3 · p. 5

4.1 HIDDEN PROFILE TASK

• On the hidden profile task, gains shrink as withholding rises above the setting’s threshold; switching reasoning off lowers internalization and raises net correction and discussion gains.The task requires pooling private clues to reveal the correct answer; the figure compares withholding, threshold, and gain across instructions and reasoning settings.Source: §4.1 · pp. 5–6, Figure 2 · p. 6
• When withholding was fixed experimentally, forcing c = 1 nearly removed discussion gains, while forcing c = 0 raised gains only for gpt-4o-mini.The other three models already agreed privately and publicly and had no sizeable gain to begin with.Source: §4.1 · p. 6

4.2 EVALUATION ON OTHER BENCHMARKS

• Gains decline as the withholding margin rises on HiddenBench and MedEInst, while the relationship is unclear on MuSiQue, where net correction is always negative.Discussion gains are large on HiddenBench and small on MuSiQue and MedEInst.Source: §4.2 · p. 7
• Benchmark and instruction shift withholding by comparable amounts, with higher withholding where private clues provide weaker grounds to dissent.Withholding is lowest on HiddenBench and highest on MedEInst.Source: §4.2 · p. 7
• Reasoning models share evidence supporting their own choices and withhold evidence pointing to the correct answer; GPT-4o-mini pools more decisive evidence but uses it less effectively.Reasoning models have higher utilization but lower pooling, while GPT-4o-mini gains more from discussion despite being the weaker model.Source: §4.2 · p. 7
• The ordering of settings by withholding margin persists when rates and outcomes are estimated on separate tasks and rounds, although the model underestimates recovery from wrong majorities.Those recoveries occur when hidden clues reach shared evidence, a path absent from the model.Source: §4.2 · p. 7

5 RELATED WORK

• Prior studies disagree on whether discussion improves accuracy, and the model links these divergent outcomes to whether withholding stays below a critical threshold.Below the threshold teams can recover from incorrect initial majorities; above it, recovery is unlikely.Source: §5 · pp. 7–8
• LLM conversation logs provide public and private responses for estimating withholding, internalization, and net correction, though private responses are not treated as true beliefs.Prior work also reports that LLMs withhold unshared information.Source: §5 · p. 8
• Many consensus models lack ground truth or assume full information sharing; this model extends truth-seeking analysis using four repeatedly observed LLM behaviors.Related approaches study public conformity, majority-rule dynamics, herding, and consensus thresholds.Source: §5 · pp. 8–9

6 DISCUSSION

• Discussion gains diminish as withholding approaches the critical threshold and fall to zero above it, helping explain conflicting prior findings as tasks operating in different regimes.The framework decomposes withholding and discussion effectiveness by LLM, benchmark, and instruction.Source: §6 · p. 9
• The study covers abstracted tasks, short discussions, homogeneous agents, fixed communication, and single sessions; its reconsideration premise is not identifiable from logs.Private answers are operational measurements and are not guaranteed to reflect internal beliefs.Source: §6 · p. 9
• Task design should account for withholding and its threshold, reasoning-related internalization, and how information is distributed and kept private.The authors suggest discussion protocols provide useful, but not excessive, information and training that incentivizes sharing and transparent reasoning.Source: §6 · p. 9

AI USE STATEMENT

• Generative AI helped refine text, write verified experimental and analysis code, search literature, and discuss ideas; the authors reviewed AI-assisted work and take responsibility for the paper.LLMs were also the agents in every experiment.Source: AI USE STATEMENT · p. 9

• The experiments use a Snakemake workflow, with code, configurations, and prompts provided in an anonymized supplementary archive.Source: REPRODUCIBILITY STATEMENT · p. 9
• Appendix B.1 lists experimental parameters, benchmarks, task and setting counts, exclusion criteria, call budgets, and the nine LLMs and their providers.Source: REPRODUCIBILITY STATEMENT · p. 9
• Runs were made between August and September 2026; hosted proprietary endpoints may later resolve to different model versions under the same identifier.Source: REPRODUCIBILITY STATEMENT · p. 10

• The publication was supported by SUNY System Administration using the SUNY AI Platform and by the SUNY-IBM AI Research Alliance.Source: FUNDING · p. 10

A.1 DERIVATION

• Because the four behavioral rules run in one direction and break detailed balance, the model uses a maximum-caliber measure over trajectories rather than an energy over instantaneous states.Once the four constraints are fixed, the maximum-caliber solution is unique; parsimony therefore depends on the constraint list.Source: §A.1 · p. 15

A.2 PHASE BOUNDARIES

• Numerical fixed-point analysis confirms the closed-form critical withholding rate; the nearby jump point is distinct and occurs slightly earlier.Increasing q lowers the jump point but leaves the critical rate unchanged, while internalization lowers the jump point to realistic withholding rates. Recovery does not occur when r ≤ 1/2, even with zero withholding; the correct consensus remains stable for every c and a when γ > 0.Source: §A.2 · p. 16

A.3 K ALTERNATIVES

• For K alternatives, the threshold formula is unchanged, and dispersed incorrect answers allow recovery at higher withholding rates than binary choice.Plurality is taken over visible statements; disagreement among incorrect answers can trigger reconsideration that reaches the correct answer with probability r.Source: §A.3 · p. 16

B.1 TASKS AND PROMPTS

• The experimental protocol elicits independent round-0 answers, then runs three discussion rounds with public recommendations, private answers, and evidence sharing.Tasks include a hidden-profile benchmark with controlled initial accuracy, HiddenBench, a retrieval variant of MuSiQue, and adapted MedEInst cases where only some agents hold the crucial clue.Source: §B.1 · p. 17
• Round-0 accuracy was p = ninf/N, below one half except when ninf = 3; frequent off-rule public answers were excluded.An off-rule answer matched neither the agent’s private answer nor the visible majority.Source: §B.1 · p. 21
• Manipulation checks verified designed round-0 accuracy, selection of the dominant candidate within each agent’s field of view, and ability to count observations.Source: §B.1 · p. 21

B.2 ESTIMATORS

• Withholding and internalization rates are estimated with a hierarchical Beta-Binomial model that accounts for task heterogeneity.Source: §B.2 · p. 22
• Net correction is inferred from directed belief-update flows, with likelihood tempering used to correct overconfidence from correlated agent-rounds.Source: §B.2 · p. 22
• γ measures observed belief flow rather than dissent's causal effect; reconsideration also varies between tie and majority rounds.Source: §B.2 · pp. 22–23
• When no agent enters a round with the correct answer, the data provide no information about v, so γ is unidentified and its sign is not interpreted.Source: §B.2 · p. 22
• On synthetic logs, estimator errors are small and intervals cover true values, but real-data coverage may be lower.Source: §B.2 · p. 23

B.3 DECOMPOSITION OF WITHHOLDING RATES

• The withholding decomposition adds LLM, benchmark, instruction, interaction, and residual effects on the logit scale, under stated distributional assumptions.Source: §B.3 · p. 23
• Figure 4 maps evidence pooling and utilization by model and instruction on the hidden profile task, with 95% Wilson intervals.Source: Figure 4 · p. 24
• With reasoning on, internalization rises or stays, while net correction and discussion gain are lower in all six models.Source: Figure 5 · p. 24

C ADDITIONAL RESULTS

• On HiddenBench, discussion gains fall as the margin ˆc−c∗ grows.This independently designed benchmark uses tasks no agent can solve individually and an eliminative aggregation rule.Source: C ADDITIONAL RESULTS · p. 24
• The HiddenBench result suggests the margin relationship is not specific to the authors’ task construction or aggregation procedure.The passage attributes this scope to HiddenBench’s independent design and eliminative aggregation.Source: C ADDITIONAL RESULTS · p. 24
• Figure 6 compares discussion gains across transparent, control, and forced-majority settings for four LLMs on two tasks.Points and intervals are observations; gray bands are predictions registered before the experiment.Source: Figure 6 · p. 25

C.1 ADDITIONAL ANALYSES

• Forced-majority nearly eliminated discussion gain; transparency increased it only for gpt-4o-mini, whose public and private answers differed in control.The experiment tested four LLMs on hidden profile and HiddenBench tasks.Source: §C.1 · p. 25
• The ordering of settings by margin remained consistent when rates and outcomes were estimated on separate tasks and rounds.The task-split analysis used 20 random splits.Source: §C.1 · p. 25
• Posterior predictions tracked teams whose initial majority was correct but underpredicted recovery from incorrect majorities, which occurred when hidden evidence reached shared evidence.The model does not include that evidence-driven recovery path.Source: §C.1 · pp. 25–26, Figure 8 · p. 26
• Figure 9 partitions withholding rounds by whether visible evidence supported, tied, or opposed the majority and whether private answers moved with it.In reasoning models, visible evidence and private answers were on the majority side in most rounds; divergence appeared mainly for gpt-4o-mini and instruction C.Source: §C.1 · p. 26, Figure 9 · p. 27