The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate

• Multi-agent architectures are motivated by the prospect that smaller open-weight models can collectively approach frontier-model reasoning under rising training costs.Source: §1 · p. 1
• The evaluation targets identical 7B-8B agents using standard plurality voting, without specialized personas, roles, or external orchestration.Peer debate is compared with isolated self-correction and stochastic noise injection to isolate peer influence from revision prompts and context length.Source: §1 · pp. 1–2
• Up to 85.5% modal sycophancy degraded accuracy below baselines while inflating team consensus to 90.1%.Source: §1 · p. 2
• Up to 32.3% oracle gaps show plurality voting discards correct answers already present in teams’ generation pools.The gap reflects peer-induced sycophancy and consensus failure rather than an absence of correct reasoning.Source: §1 · p. 2
• 2.1×-3.4× more tokens were required by debate than isolated self-correction for statistically comparable or worse accuracy.Routing rationales across agents and rounds creates O(N×K) overhead and up to 23,816 tokens per problem.Source: §1 · p. 2

2.1 Multi-Agent Architectures and the Evaluation Gap

• Specialized personas and static, easier benchmarks make it difficult to attribute heterogeneous-system gains to semantic communication rather than prompt engineering.Source: §2.1 · p. 2
• This study strips away persona engineering and evaluates identical-agent plurality voting on dynamically perturbed GSM-Hard and MMLU-Hard, exposing groupthink and contextual fragility.Source: §2.1 · pp. 2–3

2.2 Sycophancy and Alignment Artifacts

• Prior work links inter-agent sycophancy to premature consensus and lower accuracy but does not separate mechanistic pathways, communication-density thresholds, or non-semantic controls.Source: §2.2 · p. 3

2.3 Inference Economics and Context Scaling

• Debate context scales as O(N×K), creating prompt-routing overhead that can produce comparable or worse performance than self-correction at a fraction of the cost.Source: §2.3 · p. 3
• The paper complements token-optimization work by testing whether homogeneous debate generates accuracy gains commensurate with its communication cost.Source: §2.3 · p. 3

3 Methodology and Experimental Design

• The evaluation harness replaces peer communication with stochastic noise and self-reflection controls to isolate semantic communication from inference scaling.Source: §3 · p. 3

3.1 Task Selection

• GSM-Hard tests algorithmic mathematics and MMLU-Hard tests difficult scientific reasoning and logic under zero-shot standardized output formatting.GSM-Hard dynamically changes problem numbers, while MMLU-Hard covers difficult STEM and professional subjects with randomized answer-option order.Source: §3.1 · pp. 3–4

3.2 Model Selection

• Three 7B-8B instruct models are evaluated, with Qwen2.5-7B serving as the primary anchor and smaller fixed-seed subsets used for cross-model validation.The primary anchor uses 1,017 items; Llama-3.1-8B and Ministral-3-8B use 100-item subsets with wider confidence intervals.Source: §3.2 · p. 4

3.3 Architectural Baselines and Controls

• Teams contain N=10 agents over R=3 rounds, comparing peer debate, unrelated-rationale noise injection, self-correction, and a zero-shot single-agent baseline.Debate supplies K=9 peer traces, whereas self-correction supplies only the agent’s prior answer and the shared revision prompt.Source: §3.3 · pp. 4–5

3.4 Multi-Agent Interaction Protocol

• Agents generate at T=0.4 and top_p=0.95, receive truncated peer rationales ordered by confidence, and produce final outputs through plurality voting.A confidence-mass tie-breaker resolves strict plurality ties; confidence ordering may introduce a held-constant primacy-bias confound.Source: §3.4 · p. 5

3.5 Evaluation Metrics: Robustness and Economics

• The study measures vulnerability, recovery, modal sycophancy, team and oracle accuracy, oracle gap, consensus strength, token economics, and paired statistical significance.Token economics sums prompt and output tokens, while McNemar’s test with continuity correction uses p<0.05 as the significance threshold.Source: §3.5 · p. 5

4 Results

• Debate consistently failed to improve isolated reasoning and imposed severe overhead across the evaluated models and benchmarks.Table 1 reports systemic accuracy and token economics; the comparisons generally favor self-correction or show no significant debate advantage.Source: §4 · p. 5, §4.1 · p. 5, Table 1 · p. 6

4.1.1 Effectiveness of Peer Communication.

• 48.3% versus 20.7% accuracy on Ministral-3-8B GSM-Hard favored self-correction over debate, while debate used 23,816 versus 10,707 tokens per problem.The debate result was also below Ministral’s 24.7% zero-shot baseline; Llama debate likewise showed no significant improvement over self-correction.Source: §4.1.1 · p. 6, Table 1 · p. 6
• 66.7% versus 60.7% accuracy on Qwen2.5-7B MMLU-Hard significantly favored self-correction, while Llama’s directional advantage was not statistically significant.Ministral also favored self-correction at 65.3% versus 62.0%; Llama scored 47.7% versus 45.0%, with p=0.131.Source: §4.1.1 · p. 6, Table 1 · p. 6

4.1.2 Process Loss and Behavioral Micro-Dynamics.

• Debate inflated consensus to 90.1% for Qwen2.5-7B on MMLU-Hard and destabilized Ministral reasoning to a 70.0% vulnerability rate on GSM-Hard.Qwen debate consensus exceeded self-correction, whereas Ministral consensus fell under debate from 55.2% to 42.3%.Source: Table 2 · p. 6, §4.1.2 · p. 7

4.2 The Oracle Gap

• The oracle gap reached 32.3% for Ministral on GSM-Hard and 24.3% for Llama on MMLU-Hard, showing that debate discarded correct answers generated by agents.Ministral independently generated the correct answer in 53.0% of GSM-Hard cases but achieved 20.7% team accuracy; Llama generated correct answers in 69.3% of MMLU-Hard cases but achieved 45.0%.Source: §4.2 · p. 7, Table 3 · p. 7

4.3 The Asymmetry of the Communication Cost

• Debate cost 2.1×–3.4× more tokens than self-correction, reaching 28,631 tokens per problem despite comparable or worse accuracy.The overhead came primarily from O(N×K) prompt routing, while output-token generation remained relatively stable.Source: §4.3 · pp. 7–8, Table 4 · p. 8

4.4 Robustness Analysis

• The main debate degradation exceeded three standard deviations in the highest-variance comparison, supporting a structural rather than stochastic failure interpretation.For Ministral on GSM-Hard, the 27.6-point debate/self-correction gap exceeded 3σ despite self-correction variance of 4.8 percentage points.Source: §4.4 · p. 8
• High conformity appeared with minimal peer exposure: Qwen modal sycophancy was already 69.4% at K=2 on GSM-Hard and 77.2% at K=2 on MMLU-Hard.Increasing K raised these rates to 73.4% and 85.5%, respectively, while debate never exceeded self-correction accuracy.Source: §4.4 · p. 8

Prompt Engineering and Cognitive Diversity

• The findings directly concern unguided homogeneous debate, not all compound AI architectures, although homogeneous critic sub-swarms can underlie heterogeneous systems.The authors argue that inherited groupthink could affect systems built on such critic panels.Source: §5 · pp. 8–9
• Persona or adversarial prompting might not remove the problem, but these generalization arguments are explicitly speculative and require structured-debate baselines.The proposed concerns include increased oracle gaps, longer prompts, verbose rationales, and persistent shared error distributions.Source: §5 · p. 9

Limitations and Future Work

• The empirical evaluation is restricted to Qwen2.5, Ministral-3, and Llama-3.1 models in the 7B–8B parameter class.Whether the observed dynamics generalize to frontier-scale models remains unresolved, despite preliminary Qwen2.5-32B replication suggesting amplified sycophancy.Source: §5 · p. 9
• Frontier-model debate may face a prohibitive economic and latency bottleneck because the measured routing overhead scales as O(N×K).The paper leaves open whether frontier-model swarm cost-to-accuracy tradeoffs can justify this overhead.Source: §5 · p. 9
• Reasoning-augmented models such as Qwen3 and DeepSeek-R1 were not evaluated, so their interaction with peer rationales remains an open question.Potential robustness gains must be weighed against their substantially higher per-agent reasoning-token costs.Source: §5 · p. 9
• The stochastic control uses coherent rationales from unrelated problems, so it cannot fully separate context perturbation from incidental transfer of useful computational patterns.A pure random-token or shuffled-sentence baseline is proposed for a stricter non-semantic comparison.Source: §5 · p. 9
• All experiments use N=10 and one revision prompt template, leaving team-size and dissent-oriented prompt sensitivity unresolved.The authors propose varying N and testing alternative prompts that explicitly encourage dissent.Source: §5 · p. 9

6 Conclusion

• Across three 7-8B models and two high-difficulty benchmarks, debate exhibits conformity, contextual fragility, and consensus collapse, with modal adoption up to 85.5%, vulnerability up to 70.0%, and oracle gaps up to 32.3 percentage points.Source: §6 · p. 9
• 2.1×–3.4× higher token cost lets debate achieve accuracy statistically comparable to or worse than isolated self-correction, while a single agent with a 10× output budget can match debate at lower cost.Source: §6 · p. 9
• The paper concludes that isolated self-correction offers a more favorable cost-accuracy tradeoff, without claiming that all multi-agent architectures are ineffective.Source: §6 · p. 9

A Experimental Prompts

• GSM-Hard and MMLU-Hard use standardized RATIONALE/FINAL/CONF outputs, with peer revisions asking agents to evaluate team responses and update or retain their answers.Source: §A.1 · p. 10, §A.2 · p. 11

B Round-by-Round Evolution

• 60.7% debate accuracy versus 66.7% self-correction at R3 on MMLU-Hard shows debate can end below isolated reasoning despite 90.1% consensus and 74.3% oracle accuracy.For Qwen2.5-7B, the team becomes increasingly confident in wrong answers while correct answers are destroyed or outvoted.Source: B Round-by-Round Evolution · p. 11
• 73.4% modal sycophancy for Qwen on GSM-Hard and 80.5% for Ministral on MMLU-Hard increase across rounds, indicating that additional debate deepens conformity.Ministral’s GSM-Hard vulnerability reaches 70.0% as debate accuracy falls from 27.7% to 18.3% before a slight R3 recovery.Source: B Round-by-Round Evolution · p. 11
• 14pp separates Ministral’s R3 self-correction and debate oracle trajectories, as self-correction rises to 67.0% while debate collapses to 53.0%.Self-correction wins more GSM-Hard items than debate for every model, by 5.9× for Ministral, 1.6× for Qwen, and 1.3× for Llama.Source: B Round-by-Round Evolution · p. 11, Table 5 · p. 13, Table 6 · p. 13
• 96% of Ministral items solved by self-correction but not debate had at least one correct debate agent at R1, showing failure where the needed knowledge was already available.The corresponding subset averaged 4.8 correct agents out of 10, compared with 2.3 among all R3 debate-wrong items.Source: B Round-by-Round Evolution · pp. 12–13

C Confidence Calibration and Inflation

• 98.1 mean confidence paired with 14.4% agent-level accuracy yields an 83.7pp overconfidence gap for Ministral under GSM-Hard debate.Even Qwen self-correction on MMLU-Hard retains a 31.3pp gap, while debate consistently shrinks confidence discrimination relative to self-correction.Source: C Confidence Calibration and Inflation · p. 13, Table 7 · p. 13
• Confidence is nearly unrelated to correctness before communication, with Round-1 point-biserial correlations of only 0.024–0.166 and 78.5–95.6% of scores at 100.Within-team confidence dispersion is often zero, making confidence-descending ordering effectively arbitrary; majority-following therefore drives adoption more than positional primacy.Source: C Confidence Calibration and Inflation · p. 13, Table 8 · p. 13

D First-Peer vs. Modal-Peer Sycophancy

• Modal adoption exceeds first-peer adoption for Qwen on MMLU-Hard, 85.5% versus 79.1%, and GSM-Hard, 73.4% versus 67.1%.The close alignment nevertheless indicates that both majority-following and first-presented-peer influence can shape answer adoption.Source: D First-Peer vs. Modal-Peer Sycophancy · p. 14, Table 9 · p. 14

E Per-Subject MMLU-Hard Breakdown

• College physics is the most debate-vulnerable MMLU-Hard subject across models, with deficits of -12.5pp for Qwen, -2.1pp for Ministral, and -20.8pp for Llama.Llama falls from 60.4% under self-correction to 39.6% under debate, consistent with disruption of multi-step quantitative reasoning.Source: E Per-Subject MMLU-Hard Breakdown · p. 14, Table 10 · p. 14
• Qwen loses accuracy on every evaluated MMLU-Hard subject under debate, ranging from -2.8pp in professional accounting to -12.5pp in college physics.Its broad degradation is reported alongside 85.5% modal sycophancy on MMLU-Hard.Source: E Per-Subject MMLU-Hard Breakdown · p. 14
• Llama shows polarized domain sensitivity, with mild-to-moderate losses up to -6.7pp in college math but no change in formal logic.The paper reports that isolated domain-specific gains are outweighed by failures on complex reasoning tasks such as college physics.Source: E Per-Subject MMLU-Hard Breakdown · p. 14

F Extended Generation Baseline

• Ministral’s extended baseline reaches 70% on one setting, exceeding self-correction at 65.3%, but this is the only model where it outperforms all other approaches.The comparison uses a different prompt, and models self-terminate before exhausting the tenfold token budget.Source: F Extended Generation Baseline · pp. 14–15
• 66.7% accuracy with 619 tokens beats debate’s 60.7% with 17,401 tokens for Qwen on MMLU-Hard, while self-correction remains the strongest condition overall.The extended single-agent baseline achieves accuracy comparable to or better than debate at lower cost, but Ministral on GSM-Hard falls to 18.7% versus 24.7% zero-shot under the verification prompt.Source: F Extended Generation Baseline · pp. 14–15, Table 11 · p. 15

G Run-Level Variance

• Observed debate degradation exceeds stochastic variation across benchmarks and models, with all main-body differences exceeding within-condition variance by more than 3σ.Ministral’s GSM-Hard self-correction has the largest reported standard deviation at σ=4.8 percentage points, yet its debate score remains drastically lower.Source: G Run-Level Variance · p. 15, Table 12 · p. 15

• The communication-density ablation tests K∈{2,4,9} against K=0 self-correction using Qwen2.5-7B at T=0.4 across GSM-Hard and MMLU-Hard.Each communicating agent observes a random subset of peers for K=2 or K=4, while K=9 is fully connected.Source: H Ablation: Communication Density · p. 15, Table 13 · p. 16
• 69.4% sycophancy appears at K=2 on GSM-Hard, remaining near 73.4% at K=9; MMLU-Hard rises from 77.2% to 85.5%.The onset of conformity is rapid, with further peer exposure amplifying it more on MMLU-Hard.Source: H Ablation: Communication Density · p. 15
• 90.1% consensus at K=9 on MMLU-Hard exceeds 81.6% at K=2, synchronizing agents on the same answer as network density increases.At K=2, fragmented peer samples instead produce a 22.3pp oracle gap, compared with 13.7pp at K=9.Source: H Ablation: Communication Density · p. 16
• 17,401 tokens at K=9 versus 7,854 at K=2 on MMLU-Hard, while neither debate configuration exceeds isolated self-correction’s 66.7% accuracy at 6,170 tokens.Communication cost scales with peer inclusion without a corresponding accuracy advantage.Source: H Ablation: Communication Density · p. 16

• 80.3% oracle accuracy at T=0.7 versus 75.3% at T=0.4 on MMLU-Hard shows that higher temperature creates a more capable starting pool.Round 1 consensus simultaneously falls from 87.2% to 84.1% on MMLU-Hard and from 58.4% to 49.5% on GSM-Hard.Source: I Ablation: Generation Diversity · p. 16
• By Round 3, consensus reaches 87.2% on MMLU-Hard and 82.5% on GSM-Hard at T=0.7, narrowing the gap with T=0.4 despite greater initial diversity.The additional diversity is absorbed by debate without translating into accuracy.Source: I Ablation: Generation Diversity · p. 16
• On GSM-Hard, modal sycophancy increases from 73.4% to 76.8% (+3.4pp) at higher temperature, while MMLU-Hard changes from 85.5% to 80.9% (−4.6pp).Sycophancy remains above 70% in both conditions and is near-ceiling on MMLU-Hard at T=0.4.Source: I Ablation: Generation Diversity · p. 16, Table 14 · p. 16
• 61.7% debate accuracy versus 60.0% self-correction on MMLU-Hard at T=0.7 is not significant (McNemar p=0.597), while GSM-Hard favors self-correction 60.7% to 59.3%.This was the study’s only numerical debate advantage, and peer communication provided no reliable accuracy benefit at either temperature.Source: I Ablation: Generation Diversity · p. 16
• 17,401 versus 17,373 tokens on MMLU-Hard and 18,240 versus 18,398 on GSM-Hard show nearly identical debate expenditure across temperatures.The token burden is therefore structural to the debate architecture rather than dependent on generation diversity.Source: I Ablation: Generation Diversity · p. 16

J Output Budget Sensitivity and Invalid Answer Escalation

• 82.2% of Ministral outputs are invalid at 350 tokens under Round 3 debate on MMLU-Hard, reducing the effective ensemble from 10 to approximately 1.8 valid agents.The outputs contain rationales but exhaust the budget before required FINAL and CONF tags.Source: J Output Budget Sensitivity and… · p. 16, Table 15 · p. 17
• At 1,024 tokens, Ministral’s invalidation rate falls to 0% and the debate–self-correction accuracy gap narrows from 26.3pp to 3.3pp, identifying format collapse as the main source of apparent degradation at 350 tokens.The true modal sycophancy rate simultaneously rises from an apparent 19.5% to 80.5%.Source: J Output Budget Sensitivity and… · p. 17, Table 16 · p. 17
• Peer rationales inflate Ministral’s reasoning chains, producing format failure under constrained budgets and sycophantic conformity when budgets are sufficient.The paper links both outcomes to difficulty resisting or efficiently integrating peer context.Source: J Output Budget Sensitivity and… · p. 17

K Scaling to Larger Models

• 95.4% modal sycophancy and 96.5% consensus in Qwen2.5-32B exceed the 7B model’s 73.4% and show near-total debate lock-in.Between Rounds 2 and 3, zero debate items change their team answer, indicating a fixed point.Source: K Scaling to Larger Models · p. 17, Table 17 · p. 17
• 63.0% self-correction versus 58.6% debate accuracy widens the 32B gap to 4.4pp, while debate consumes 3.1× more tokens.The larger model’s base accuracy is 51.5% versus 25.6%, yet debate remains less accurate and more expensive.Source: K Scaling to Larger Models · p. 17
• The 32B scaling comparison is preliminary: it uses one benchmark, three runs, and 100 items per run, requiring broader replication for definitive conclusions.Future work is planned across both benchmarks and 70B+ models.Source: K Scaling to Larger Models · pp. 17–18

• Negative synergy occurs in 6 of 8 condition–task pairs, with Qwen+Llama debate reaching 33.7% on GSM-Hard, 25.1pp below homogeneous Qwen debate.Deficits are also negative across the reported MMLU-Hard debate and self-correction comparisons.Source: L Heterogeneous Team Experiments · p. 18
• 62.0% Llama+Ministral debate accuracy on GSM-Hard exceeds homogeneous Llama by 22.7pp and Ministral by 41.3pp, but this positive synergy does not replicate on MMLU-Hard.Self-correction under the same composition improves only 1.7pp over Ministral self-correction.Source: L Heterogeneous Team Experiments · p. 18
• On MMLU-Hard, Qwen+Llama oracle accuracy falls from 74.7% to 65.0% under debate but rises to 81.3% with self-correction; Llama+Ministral shows the same divergence.Model diversity therefore does not protect against knowledge destruction in this setting.Source: L Heterogeneous Team Experiments · p. 18, Table 19 · p. 18
• 64.6%–85.0% modal sycophancy in heterogeneous teams is comparable to homogeneous rates, and correct-to-wrong flips exceed wrong-to-correct flips on MMLU-Hard.The agent-level dynamics remain consistent with conformity and knowledge loss rather than mitigation.Source: L Heterogeneous Team Experiments · p. 18, Table 20 · p. 18
• Qwen+Llama debate uses 16,275 tokens versus 5,407 for self-correction (3.0×), while Llama+Ministral uses 20,904 versus 9,337 (2.2×).Mixing models preserves the debate cost disadvantage.Source: L Heterogeneous Team Experiments · p. 18
• These heterogeneous-team results use 100 items per task, two compositions, and a single run, so additional models, samples, and structured protocols are needed to establish generality.Source: L Heterogeneous Team Experiments · p. 18

Experimental setup

• The reasoning-model assessment evaluated Qwen3-8B-think, a model with explicit chain-of-thought, under the main-experiment protocol: N=10 agents, K=9 peers, and 3 rounds.Table 21 reports performance, defining Base as a single agent (N=1), R1 as 10-agent plurality voting without communication, R3 as after 3 rounds, and Cost as mean total tokens per item at R3.Source: M Reasoning Model Results · p. 18, Table 21 · p. 19

Benchmark results

• Oracle accuracy fell under debate on both tasks, from 75.0% to 73.0% on GSM-Hard and from 93.0% to 91.0% on MMLU-Hard.Self-correction kept GSM-Hard oracle accuracy at 75.0% and increased MMLU-Hard oracle accuracy to 94.0%.Source: M Reasoning Model Results · p. 19
• 72.0% debate accuracy underperformed the 73.0% no-communication baseline on GSM-Hard, whereas MMLU-Hard improved to 90.0% from 86.0%.Self-correction reached 73.6% on GSM-Hard and 87.0% on MMLU-Hard.Source: M Reasoning Model Results · p. 19

Agent-level dynamics

• Answer changes were only 2.2–6.1% for Qwen3-8B-think, but 93.8–97.4% of agents matched the modal peer answer and Round 3 consensus reached 98.0–98.5%.These conformity and consensus levels exceeded those observed for non-reasoning 7–8B models.Source: M Reasoning Model Results · p. 19, Table 22 · p. 19

Token cost

• Debate used 22,346 tokens per GSM-Hard item versus 12,461 for self-correction and 1,811 for a single agent.Across conditions, debate cost 1.6–1.8× more than self-correction, with longer reasoning traces raising absolute costs.Source: M Reasoning Model Results · p. 19

Limitations

• These findings are limited to 100 items per task and a single reasoning model, with broader validation planned using multiple reasoning models and larger samples.Planned models include DeepSeek-R1 and larger Qwen3 variants.Source: M Reasoning Model Results · p. 19