Source-linked AI summary
Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate
Andrea Wynn, Harsh Satija, Gillian Hadfield
TL;DR
Multi-agent debate is proposed to improve LLM reasoning, but evidence about heterogeneous agent groups and debate failure modes remains limited. This paper evaluates debate across diverse tasks and model capabilities, finding that accuracy can decline over successive rounds even when stronger agents are the majority. The results identify sequential revision, social influence, sycophancy, task type, and model capability as contributing factors.
Problem
The paper asks whether multi-agent debate reliably improves reasoning, especially when agents differ in capability, and examines why debates can fail to reach correct majority answers.
Method
The paper systematically evaluates iterative multi-agent debate across three datasets and heterogeneous LLM agents, then analyzes sequential revision, social influence, and sycophancy.
Results
Group accuracy often declines over successive debate rounds, even when most agents perform well individually or weaker models can access stronger models’ reasoning.
Takeaways & Limitations
Naive debate can amplify errors and cause heterogeneous agents to converge on wrong answers together, so debate should be applied carefully to the task and setting.
Takeaways & Limitations
The evaluation assumes that every dataset question has a ground-truth answer in the possible answer set.
Abstract
from arXiv · showhide
While multi-agent debate has been proposed as a promising strategy for improving AI reasoning ability, we find that debate can sometimes be harmful rather than helpful. Prior work has primarily focused on debates within homogeneous groups of agents, whereas we explore how diversity in model capabilities influences the dynamics and outcomes of multi-agent interactions. Through a series of experiments, we demonstrate that debate can lead to a decrease in accuracy over time - even in settings where stronger (i.e., more capable) models outnumber their weaker counterparts. Our analysis reveals that models frequently shift from correct to incorrect answers in response to peer reasoning, favoring agreement over challenging flawed reasoning. We perform additional experiments investigating various potential contributing factors to these harmful shifts - including sycophancy, social conformity, and model and task type. These results highlight important failure modes in the exchange of reasons during multi-agent debate, suggesting that naive applications of debate may cause performance degradation when agents are neither incentivised nor adequately equipped to resist persuasive but incorrect reasoning.
1 Introduction
This work evaluates whether multi-agent debate reliably improves reasoning, finding that debate can instead amplify errors and degrade performance, especially with heterogeneous agents.
- Motivation and Contributions: Multi-agent debate can degrade performance, producing worse final answers than a single agent acting alone.The failures arise systematically when agents agree reflexively instead of challenging flawed reasoning.
- Motivation and Contributions: Longer debates may progressively worsen performance, even when stronger agents outnumber weaker agents.Introducing a weaker model can disrupt a stronger model’s performance.
- Motivation and Contributions: The paper identifies task type and complexity, agent diversity and capability, and social influence as factors mediating whether debate helps or hurts.It recommends applying debate carefully to the task and setting of interest.
- Motivation and Contributions: The evaluation covers three datasets and compares debate with majority voting while examining performance across successive rounds.The study also extends debate beyond homogeneous agent populations.
- Motivation and Contributions: The analysis investigates how sequential revision, social influence, and sycophancy contribute to correct answers becoming corrupted during debate.These analyses motivate future work on improving reasoning exchange in multi-agent systems.
2 Related Work
Prior research presents multi-agent debate and collaboration as ways to improve reasoning, but also documents failures involving conformity, persuasion, and social influence.
- Debate and Collaboration: Multi-agent debate was proposed for scalable oversight and later used for iterative reasoning exchange among LLM agents.These forms include judge- or verifier-mediated conversations and multi-agent deliberation.
- Debate and Collaboration: Theoretical work reports a “tyranny of the majority” effect in which minority agents conform to the majority answer regardless of correctness.This conformity can create an echo chamber effect.
- Debate and Collaboration: The paper tests a theoretical expectation that diversity in model abilities should improve debate performance, finding empirical evidence that this often fails in practice.This extends prior work on opinion diversity into heterogeneous model settings.
- Debate and Collaboration: Role-based debates and explicitly adversarial debates can improve answers when agents constructively challenge one another.These approaches differ from setups in which agents merely exchange reasons or seek agreement.
- Failure Modes: Prior work also warns that judges may be persuaded by rhetoric and that human-like dynamics can bias group outputs.This paper builds on those observations by studying heterogeneous debate across varied tasks.
- Collaborative Frameworks: Other collaborative systems assume that decentralized, multi-turn interaction produces richer reasoning and more efficient cooperation.One cited system reports reaching goals with 60% fewer steps than a lone agent.
3 Setting: Multi-agent debate
The study models debate as iterative answer generation in which agents revise responses using other agents’ reasoning, with majority voting producing the final answer.
- Problem Setting: Each question has a ground-truth answer, and agents generate answers from a task’s question and possible answer set.Accuracy is evaluated against the ground truth.
- Single-Agent Setting: In the single-agent setting, an underlying LLM maps a question to a generated answer, with accuracy as the main metric.Task-specific prompts may also be supplied to the model.
- Multi-Agent Debate: Debate begins with independent agent responses containing answers and reasoning, then iteratively revisits the question using other agents’ summarized responses.Each agent also conditions revision on its own previous response.
- Multi-Agent Debate: The debate runs for T rounds, after which majority voting across the agents’ final responses determines the group answer.Group accuracy is evaluated against the ground truth.
4 Experimental Setup
Experiments evaluate multi-agent debate across three reasoning datasets using several model families, standardized prompts, repeated random samples, and multiple seeds.
- Datasets: The experiments cover CommonSenseQA, MMLU, and GSM8K, spanning commonsense, broad knowledge, and multi-step mathematical reasoning.CommonSenseQA additionally tests a task not previously evaluated in multi-agent debate.
- Models: The models are GPT-4o-mini, LLaMA-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.2 from distinct model families.The setup aligns with prior multi-agent debate work.
- Experimental Protocol: Experiments use 100 random samples per task, report results over 5 random seeds, and run T = 2 debate rounds.The maximum generation length is 2048 tokens.
- Prompts: Task prompts request accurate answers with bullet-point reasoning and a final answer formatted as (X).The prompt examples cover all three experimental datasets.
- Prompts: Debate prompts provide other agents’ solutions as additional advice and ask each model to examine both its own and others’ reasoning before updating.A general system prompt frames the model as a helpful assistant.
5 Results
Across three tasks, debate often reduces group accuracy rather than improving it, including configurations where stronger models outnumber weaker ones. Table 1 compares majority-vote performance without reasoning exchange against performance after debate, while Figure 1 shows degradation across debate rounds.
- 5.1 Effectiveness of Debate: Debate can harm performance compared with selecting the majority vote from agents’ initial responses.The comparison uses a w/o Debate baseline and an After Debate condition following exchange of reasons.
- 5.1 Effectiveness of Debate: The experiments compare single-agent results with debate procedures using 100 random samples and five seeds, reporting means and standard errors.Table 1 distinguishes majority voting without exchanged reasoning from majority voting after debate.
- 5.1 Effectiveness of Debate: Debate can decrease accuracy even when stronger models outnumber weaker models.This effect is reported for groups containing more strong models than weak ones.
- 5.2 Performance Degradation during Debate: Across MMLU, CommonSenseQA, and GSM8K, performance is evaluated as a function of debate rounds.The groups contain language models with varying individual task performance.
- 5.2 Performance Degradation during Debate: In many group configurations, performance decreases as debate progresses, especially for mixed-capability groups on MMLU and CommonSenseQA.The degradation occurs despite those groups having a majority of stronger agents.
6 Failure Modes of Debate
The analysis examines why sequential reasoning exchange can harm debate performance, focusing on answer flips, social influence, and sycophancy. Across settings, correct answers are often changed to incorrect ones, while a correctness-payoff prompt does not reliably prevent these harmful shifts.
- 6.1 Does exchange of reasoning help in sequential revision?: Debate can actively mislead initially correct agents: correct-to-incorrect transitions exceed incorrect-to-correct transitions across subsequent rounds.Most initially incorrect agents do not improve, and performance degrades over rounds.
- 6.1 Does exchange of reasoning help in sequential revision?: Social pressure dominates sequential revision, with agents showing lower resistance to disagreement-induced harmful flips after the second round.Agents that resist flipping correct answers in round 1 exhibit larger correct-to-incorrect shifts after round 2.
- 6.2 Are agents influenced by social effects?: The likelihood of a correct-to-incorrect flip is highest when the ego agent is isolated and decreases as more peers agree with its answer.The relationship varies substantially across datasets and models.
- 6.3 Are agents sycophantic?: The correctness-payoff prompt does not significantly reduce correct-to-incorrect flips and sometimes increases them.The intervention therefore does not resolve the observed debate failures on its own.
- Cross-cutting factors: Answer-changing behavior varies substantially across models and tasks, indicating that capability, dataset, and task complexity shape debate dynamics.The reported analyses include CommonSenseQA, MMLU, and GSM8K.
7 Discussion
The discussion concludes that multi-agent debate does not uniformly improve reasoning and can amplify errors, especially in heterogeneous groups. It attributes failures to multiple interacting factors and motivates debate protocols that favor critical evaluation over superficial consensus.
- 7 Discussion: Group accuracy can decline over successive debate rounds even when most agents perform well individually or weaker models access stronger agents’ reasoning.Additional reasoning exchange may amplify rather than correct mistakes.
- 7 Discussion: Heterogeneous groups frequently converge on wrong answers through interacting effects involving sequential revision, social conditioning, sycophancy, model capability, and task type.The paper presents these as correlated and contributing factors rather than a single cause.
- 7 Discussion: More principled debate protocols should promote critical evaluation over consensus, including assessing reasoning soundness and using confidence or credibility signals.The discussion also suggests rewarding independent verification and discouraging unsupported conformity.