PaperlayerView exact revision on arXiv

Source-linked AI summary

Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?

Hyeong Kyu Choi, Xiaojin Zhu, Sharon Li

arXiv:2508.17536v2cs.CLcs.MA

TL;DR

The paper investigates whether MAD’s effectiveness comes from inter-agent debate or from aggregating multiple outputs. It separates voting from communication, combines benchmark experiments with a stochastic theoretical framework, and finds that majority voting accounts for most gains while correction-biased updates improve debate.

  • Problem

    MAD’s advances leave unclear whether performance gains come primarily from meaningful inter-agent communication or from aggregating multiple agent outputs.

  • Method

    The paper compares majority voting with multi-round debate across benchmarks and models debate as a Bayesian stochastic process whose belief updates form a martingale.

  • Results

    Across seven benchmarks, majority voting accounts for most MAD performance gains, while debate alone preserves expected correctness and correction-biased interventions improve debate performance.

  • Takeaways & Limitations

    Simple ensembling methods such as majority voting remain strong and often more reliable alternatives to MAD, while preserving correct answers is a design direction for improving debate.

  • Takeaways & Limitations

    The initial analysis focuses on homogeneous agents and assumes the correct answer has the largest initial belief.

Abstract

from arXiv · show

Multi-Agent Debate~(MAD) has emerged as a promising paradigm for improving the performance of large language models through collaborative reasoning. Despite recent advances, the key factors driving MAD's effectiveness remain unclear. In this work, we disentangle MAD into two key components--Majority Voting and inter-agent Debate--and assess their respective contributions. Through extensive experiments across seven NLP benchmarks, we find that Majority Voting alone accounts for most of the performance gains typically attributed to MAD. To explain this, we propose a theoretical framework that models debate as a stochastic process. We prove that it induces a martingale over agents' belief trajectories, implying that debate alone does not improve expected correctness. Guided by these insights, we demonstrate that targeted interventions, by biasing the belief update toward correction, can meaningfully enhance debate effectiveness. Overall, our findings suggest that while MAD has potential, simple ensembling methods remain strong and more reliable alternatives in many practical settings. Code is released in https://github.com/deeplearning-wisc/debate-or-vote.

1 Introduction

The paper asks whether MAD’s gains come from inter-agent communication or simply aggregating multiple outputs. Across seven benchmarks, it finds majority voting explains most gains, while theory characterizes debate as a martingale and motivates correction-biased interventions.

  • Motivation: MAD combines multiple agents with iterative discussions, but its underlying source of effectiveness remains unclear.The paper frames the central question as whether communication or output aggregation drives observed gains.
  • Motivation: If ensembling explains most gains, majority voting can replace more complex MAD designs while reducing computational and architectural overhead.The motivation connects this possibility to avoiding additional system complexity.
  • Approach: The study isolates ensembling by comparing initial-response majority voting with performance after multiple debate rounds across seven benchmarks.Majority Voting measures the aggregated output before debate, while final MAD performance includes iterative communication.
  • Theory: A stochastic DCM-based framework models agent uncertainty and proves that debate induces a martingale over belief in the correct answer.The martingale means expected belief remains unchanged over debate rounds, with belief evolution driven by stochastic peer influence.
  • Implications: Correction-biased belief updates preserve correct responses across debate rounds and produce notable improvements over standard MAD.The proposed interventions give correct responses more influence than misleading ones.
  • Findings: Across seven representative benchmark datasets, majority voting is reported as effective as MAD, including broader settings with larger, heterogeneous, and open-ended agents.The contribution statement extends the comparison beyond the initial homogeneous benchmark setup.

2 Preliminaries

MAD uses multiple language-model agents that independently answer and then iteratively update responses using peer outputs. The paper formalizes both direct majority voting and graph-structured debate, comparing debate rounds with aggregation before communication.

  • Multi-Agent Debate: MAD is a collaborative framework in which multiple language-model agents iteratively exchange responses to solve tasks such as question answering or text generation.A final answer is typically obtained through an aggregation mechanism such as majority voting.
  • Formalization: Each agent is modeled as a stochastic function that produces an initial response from the input.The formalization denotes the input space by X, output space by Y, and initial response for agent i by y_i,0.
  • Majority Voting: Majority voting directly aggregates the agents’ initial responses through a voting function, typically returning the most frequent answer.The resulting prediction is y_0 = V(y_1,0, ..., y_N,0).
  • Multi-Agent Debate: Debate adds T rounds in which each agent observes responses from graph-defined neighbors at the previous round and updates its answer.The neighbor set includes the agent itself and may include all agents in a fully connected setting.
  • Multi-Agent Debate: The iterative debate process is represented as repeated composition of a single-round debate function, followed by final aggregation after T rounds.The formal expression uses D^(T)(x; R_i) for each agent’s response after repeated updates.
  • Experimental Setting: The study adopts simultaneous parallel updates and homogeneous agents to isolate inter-agent communication against majority voting.Heterogeneous settings are deferred to a later section.

3 Is Debate Really Necessary? A Closer Look at Debate vs. Voting

The study tests whether MAD’s effectiveness comes from debate or from aggregating multiple agent outputs. Across benchmark comparisons and an agent-count ablation, majority voting accounts for most observed gains and often matches MAD.

  • Research Question: The paper dissects MAD into multi-agent ensembling and inter-agent communication to measure whether debate adds benefits beyond aggregation.The section presents empirical evidence that simple majority voting accounts for most observed gains.
  • Experimental Setup: Table 1 compares single-agent, MAD, and majority-voting performance across seven benchmark datasets using Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct.The comparison includes 2-, 3-, and 5-round debates among five agents.
  • Key Observations: Majority voting performs on par with MAD in most cases and does not reliably surpass the simpler voting strategy.MAD consistently outperforms the single-agent baseline, but voting remains competitive across the reported comparisons.
  • Key Observations: With the more capable Qwen2.5-32B-Instruct model, overall MAD performance improves while majority voting still accounts for most gains.The paper interprets this pattern as evidence that model ensembling, rather than iterative debate, largely drives MAD effectiveness.
  • Ablation: Increasing the number of Qwen2.5 agents from N = 1 to N = 5 generally improves performance, suggesting an ensemble contribution.The ablation varies the number of participating agents in each debate round.

4 Theoretical Analysis

The framework models agent responses and belief updates probabilistically, then analyzes majority voting and debate as distinct decision processes. It shows that majority voting can become highly reliable as the agent count grows, whereas debate belief trajectories form a martingale under the stated Bayesian setting.

  • Modeling assumptions: Agents sample beliefs from a Dirichlet distribution and generate categorical responses, providing the stochastic basis for the analysis.The framework represents each agent with a belief vector and uses belief sampling followed by response generation.
  • Majority voting: Majority voting selects the most probable answer with a lower-bounded success probability when the agent count exceeds a threshold determined by the number of options and belief margin.The theorem defines the margin as ∆ = θ̄1 − θ̄2 and requires N > K/∆2.
  • Majority voting: As the number of agents scales, majority voting’s success probability asymptotically approaches 1 even when the correct answer is only marginally more probable.This magnifying effect applies while the correct answer remains the most probable choice, with a sharper finite-agent analysis also provided.
  • Debate dynamics: Under Bayesian debate updates, each agent incorporates neighbors’ response counts into its Dirichlet parameters, making each debate round a conjugate Bayesian update.The update is αi,t = αi,t−1 + ci,t, where ci,t records neighbors’ selections.
  • Debate dynamics: The expected belief in the correct answer remains unchanged across debate rounds because each agent’s belief trajectory forms a martingale.The empirical mean-accuracy trajectories are essentially flat across rounds, consistent with the theoretical martingale property.
  • Implications: In homogeneous, fully connected settings, debate alone does not necessarily improve initial accuracy, while generalized heterogeneous collective intelligence can also exhibit martingale behavior.The framework is additionally generalized to open-ended tasks through distributional or similarity-based representations.

5 How Does Theory Inform Improved Design of MAD?

The theory motivates interventions that bias debate updates toward correct signals, departing from neutral martingale behavior. Experiments include oracle-style correction and majority-vote-guided modifications.

  • Without additional bias toward correct signals, debate’s martingale dynamics do not guarantee convergence to the truth.
  • The oracle variant locks an agent’s belief after it produces the correct answer, preserving that response in subsequent rounds.
  • In Decentralized MAD with T = 5 rounds, MMLU (Form. Log.) accuracy increases from 0.5000 to 0.6825 under MAD-oracle.
  • MAD-oracle is infeasible without access to the true answer, so it serves as an upper bound for biasing belief updates toward correct signals.
  • Practical modifications use majority responses as a proxy for ground truth to steer belief updates without direct oracle access.

6 Extended Experiments to General Settings

Extended experiments test whether majority voting remains comparable to debate with larger models, heterogeneous agents, and open-ended generation. The larger-model results confirm the earlier pattern on GSM8K and HellaSwag.

  • The extended evaluation examines larger models, heterogeneous agents, and open-ended question formats.
  • On Qwen2.5-32B-Instruct, Majority Voting remains comparable to multi-agent methods on GSM8K and HellaSwag.
  • The larger-model results suggest the majority-voting claim is not limited to smaller language models.
  • Heterogeneous-agent experiments construct distinct personas using persona sets selected for college mathematics and clinical knowledge.
  • Open-ended evaluation studies MAD on CNN/DailyMail summarization, where applying Majority Voting is not straightforward.

7 Related Works

Related work presents MAD as a promising multi-agent approach while also documenting concerns about its effectiveness. Prior studies report failure modes, inconsistent gains, insufficient self-correction, and sensitivity to hyperparameters.

  • Multi-agent systems surveys review state-of-the-art LLM-based approaches, among which MAD is prominent for improving single-agent benchmarks.
  • MAD’s proposed strengths include iterative discussion intended to improve factual accuracy and reasoning quality.
  • One prior analysis identifies 14 distinct failure modes in MAD systems.
  • Prior studies report that MAD does not consistently outperform single-agent approaches and that agents are not sufficiently self-corrective.
  • Other work reports that MAD performs no better than advanced single-agent reasoning methods and is sensitive to hyperparameters.

8 Conclusion

The paper concludes that most MAD gains arise from majority voting rather than debate itself. Its martingale framework explains this result and motivates preserving correct answers during debate.

  • Experiments across seven benchmarks indicate that most MAD performance gains stem from majority voting rather than debate.
  • The theoretical framework characterizes debate dynamics as a martingale that preserves each agent’s expected success probability over time.
  • The findings position majority voting as a strong and often more reliable ensembling strategy.
  • The paper identifies preserving correct answers during inter-agent debate as a direction for improving MAD design.
  • Experiments use sampling temperature 1.0, nucleus sampling probability 0.9, and a maximum of 512 generated tokens.

A.2 Dataset Details

The experiments cover arithmetic, mathematical reasoning, factual question answering, natural language inference, commonsense reasoning, alignment labeling, and abstractive summarization, with dataset-specific prompts and standardized answer formats.

  • Task Coverage: The tasks span arithmetic calculation, multistep mathematical reasoning, professional medical and formal logical reasoning, sentence completion, commonsense question answering, preference labeling, and summarization.HH-RLHF asks agents to select the more helpful and less harmful example from chosen/rejected pairs.
  • MAD Template: The MAD template presents peer responses from the previous round alongside the agent’s latest opinion and asks it to revise its answer using that advice.The template assumes three agents for brevity and includes responses from agents 2 and 3 plus the agent’s own previous response.
  • Prompt Formatting: Prompts require a standardized final-answer format, with task-specific examples for arithmetic, GSM8K, multiple-choice datasets, and CNN/DailyMail.CNN/DailyMail prompts place the summary after “# Summary #”, while other tasks use curly-braced final-answer formats.
  • Agent Personas: Heterogeneous personas are assigned through system prompts describing roles such as assistant, mathematician, economist, programmer, lawyer, and psychologist.The persona descriptions specify domain-oriented capabilities for each assigned role.

C Special Case of Theorem 1

This special case formalizes majority voting when agents independently produce Bernoulli-correct answers and derives a concentration-based lower bound for voting success. The broader framework also models debate beliefs with Dirichlet–multinomial updates and characterizes martingale behavior.

  • Majority Voting: Each agent independently answers correctly with probability p0, and the empirical mean X̄ represents the fraction of agents voting correctly.The analysis uses independent Bernoulli variables Xi ∼ Bernoulli(p0).
  • Majority Voting: When p0 > 0.5, majority-vote success is bounded using Hoeffding’s inequality for independent variables bounded between 0 and 1.The proof rewrites the concentration bound in terms of the empirical correctness mean.
  • General Bound: The general theorem defines the margin Δ as the difference between the mean probability of the correct answer and the strongest competing answer.The theorem states a lower bound under a condition involving the number of agents and Δ.
  • General Bound: If the empirical vote distribution is within L1 distance Δ of the mean distribution, majority voting selects the correct answer.The proof establishes the contrapositive: an incorrect majority implies L1 deviation at least Δ.
  • Debate Model: Under the Dirichlet–multinomial model, agents sample responses from a fixed mean distribution, reducing the relevant counts to a multinomial distribution.The fixed mean is denoted by θ̄, and agents draw answers independently from it.
  • Debate Model: For debate beliefs pi,t, the theorem shows that the expected belief at the next round equals the current belief when its stated condition holds.This conditional-expectation property is the martingale definition used in the proof.

D.3.1 When does (1) in Theorem 2 Hold?

The martingale condition is guaranteed for homogeneous fully connected agents, isolated agents, and isolated homogeneous cliques, but not necessarily for line or ring topologies. Heterogeneous fully connected systems require a separate collective-belief condition.

  • Sufficient Conditions: The martingale condition holds for homogeneous fully connected agents, fully isolated agents, and isolated cliques whose agents are homogeneous.These are presented as sufficient conditions for equation (1).
  • Sufficient Conditions: In homogeneous fully connected MAD, agents share identical initial beliefs and identical count-vector updates, so their beliefs remain aligned.This setting corresponds to the paper’s core decentralized MAD configuration.
  • Sufficient Conditions: Fully isolated agents update only from their own previous answers, making the process equivalent to iterative self-refinement while retaining the martingale property.Each agent’s belief parameters evolve independently in this extreme topology.
  • Limitations: The sufficient-topology analysis is incomplete because other debate graph structures may also satisfy the martingale condition.A complete characterization is left for future work.
  • Counterexamples: Line and ring graphs may violate the martingale condition because local neighborhoods can produce unequal belief updates.The ring topology is the Sparse MAD baseline and is empirically reported to satisfy the property approximately.
  • Heterogeneous Agents: For heterogeneous fully connected agents, individual beliefs are not necessarily martingales, although collective belief can be one when belief magnitudes are equal and connectivity is complete.The collective-belief condition is stated as sufficient for the martingale property.

F Proper Evaluation Matters

The study argues that evaluation methodology, especially free-form answer extraction, can substantially alter measured MAD performance. Standardized extraction improves reliability and can change comparisons with majority voting.

  • Evaluation Reliability: Answer-extraction methods can significantly affect measured performance and may reverse conclusions about MAD versus majority voting.Rule-based parsing can mark a correct model response as incorrect because of formatting failures.
  • Evaluation Reliability: The authors standardize extraction by instructing agents to append final answers in a common format before selecting the majority answer.The protocol is intended to reduce parsing failures and improve evaluation reliability.
  • Evaluation Results: On GSM8K, the revised evaluation protocol outperforms MAD evaluated with the prior extraction strategy from [2].The authors state that the protocol also improves single-agent accuracy.
  • Evaluation Results: Additional evaluations with three GPT-4 agents across four benchmarks show trends consistent with the open-source model experiments.These results are presented as supporting the generality of the findings.
  • Limitations: The study primarily examines the Simultaneous Talk protocol and assumes agent homogeneity in its theoretical framework.Sequential protocols, summarizer-mediated protocols, and heterogeneous agents may exhibit different dynamics and remain future-work directions.
Loading 2508.17536v2…