PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Multi-Agent Debate with Sparse Communication Topology

Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, Eugene Ie

arXiv:2406.11776v1cs.CL

TL;DR

Multi-agent debate improves reasoning but grows costly as agents and rounds expand input context. This paper studies sparse communication topologies and extends debate to alignment labeling; sparse configurations match or exceed standard MAD performance while reducing costs.

  • Problem

    Multi-agent debate can improve reasoning and factuality, but more agents and rounds expand input context, while prior approaches let every agent communicate with all others.

    Source: §1 · p. 1

  • Method

    The paper studies sparse communication graphs for multi-agent debate and evaluates the approach on reasoning and alignment-labeling tasks.

    Source: §3.1 · p. 3, §7 · p. 8

  • Results

    Sparse MAD matches or exceeds standard MAD while reducing computational costs; on MATH it improves accuracy by +2% and cuts average reasoning input-token cost by over 40%.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    Topology design matters: the authors report better overall performance when stronger LLMs occupy more connected agents in multi-LLM debates.

    Source: §1 · p. 2, §7 · p. 8

    The authors identify potential misuse, including biased information or misinformation, and substantial computational demands that could affect energy use and environmental sustainability.

    Source: §9 · p. 9

Abstract

from arXiv · show

Multi-agent debate has proven effective in improving large language models quality for reasoning and factuality tasks. While various role-playing strategies in multi-agent debates have been explored, in terms of the communication among agents, existing approaches adopt a brute force algorithm -- each agent can communicate with all other agents. In this paper, we systematically investigate the effect of communication connectivity in multi-agent systems. Our experiments on GPT and Mistral models reveal that multi-agent debates leveraging sparse communication topology can achieve comparable or superior performance while significantly reducing computational costs. Furthermore, we extend the multi-agent debate framework to multimodal reasoning and alignment labeling tasks, showcasing its broad applicability and effectiveness. Our findings underscore the importance of communication connectivity on enhancing the efficiency and effectiveness of the "society of minds" approach.

1 Introduction

Multi-agent debate can improve reasoning and factuality, but its context costs rise with the number of agents and rounds. This study finds that sparse communication can preserve or improve performance while reducing costs, and extends the approach to alignment labeling and mixed-model debates.

1 Introduction

  • Neighbor-connected MAD improves MATH accuracy by +2%, matches GSM8K accuracy, and reduces average reasoning input-token cost by over 40% versus fully connected MAD.The introduction also reports alignment-labeling improvements of +0.5% in helpfulness and +1.0% in harmlessness, with costs reduced by 50.0% and 53.3%, respectively.Source: §1 · p. 2
  • On Anthropic-HH alignment labeling, sparse MAD improves helpfulness by +0.5% and harmlessness by +1.0%, while reducing costs by 50.0% and 53.3%, respectively.The paper presents this as an extension of MAD to alignment labeling and reports effectiveness compared with single-agent setups.Source: §1 · p. 2
  • In non-regular graphs, assigning stronger LLMs to agents with higher centrality consistently yields better performance.The authors also report that weaker models are progressively strengthened through interaction with stronger models.Source: §1 · p. 2
  • The paper evaluates sparse MAD for text-only and multimodal reasoning, alignment labeling, and mixed-LLM debates, and offers case-study insights into why sparse communication can work.The introduction identifies these as contributions alongside improved effectiveness and efficiency over standard MAD.Source: §1 · p. 2

2 Related Work

Related work spans prompting and multi-agent methods for reasoning, multimodal reasoning, and AI feedback. The paper distinguishes its focus on sparse communication topology from prior MAD work emphasizing agent diversity.

Multi-Agent Debate

  • Prior MAD studies diversify debates through agent roles, multiple LLMs, or subgroups, whereas this work investigates sparse communication topology.The paper also extends MAD to reasoning and alignment tasks.Source: §2 · p. 2

LLM and Multimodal Reasoning

  • Reasoning approaches surveyed include Chain-of-Thought, problem decomposition, reasoning ensembles, and verification, while multimodal studies use prompt-based methods to improve visual reasoning.The paper notes that existing multimodal methods can be complex to design and implement.Source: §2 · p. 2

AI Feedback

3 Method

The method represents debate communication as a graph and studies sparse topologies, then uses a multi-round process in which agents exchange connected peers’ prior responses and vote on consensus.

3.1 Communication Topology

  • Communication topology is a graph G = (V, E): agents are vertices, and an edge indicates that connected agents can access each other’s previous-round solutions.The study focuses on static graphs, with exploratory dynamic-graph experiments reported in an appendix.Source: §3.1 · p. 3
  • Graph density D is the ratio of edges to the maximum possible; fully connected MAD has D = 1, while neighbor-connected MAD has |E| = |V| and is sparse.Lower D indicates a sparser graph.Source: §3.1 · p. 3
  • The main topology experiments use regular, permutation-invariant graphs with six agents, balancing sparsity range against the higher cost of studying larger systems.The paper also reports additional four-agent experiments in an appendix.Source: §3.1 · p. 3

3.2 Multi-Agent Debate Process

  • Agents first generate independent responses; in later rounds they critique or refine their answers using connected peers’ previous responses, then a majority vote selects consensus if answers differ.The debate uses simultaneous-talk communication to facilitate asynchronous computation.Source: §3.2 · p. 3

4 Experiments Setup

The experiments cover text reasoning, multimodal reasoning, and alignment labeling with different models and baselines. They measure task accuracy and input-token cost, and use variance-reduction procedures to compare topologies more reliably.

4.1 Tasks

  • The task suite includes MATH and GSM8K text reasoning, free-form MathVista multimodal questions, and Anthropic-HH helpfulness and harmlessness labeling.MATH uses the algebra linear 1d composed sub-task; MathVista uses free-form questions for consistency.Source: §4.1 · p. 3

4.2 Models

  • GPT-3.5 is used for text reasoning, GPT-4 for multimodal reasoning, and GPT-3.5 plus Mistral 7B for alignment; experiments sample 100 GPT examples or 500 Mistral examples.GPT-4 is excluded from other tasks because it costs approximately 10 times as much as GPT-3.5; Mistral is excluded from other tasks for inferior zero-shot arithmetic reasoning.Source: §4.2 · p. 3

4.3 Baselines

  • Sparse MAD is compared with Chain-of-Thought, self-consistency, and standard fully connected MAD using simultaneous-talk communication.The standard baseline is denoted MAD (D = 1).Source: §4.3 · p. 4

4.4 Evaluation Metrics

  • Reasoning quality is measured by accuracy against ground truth, alignment-labeling accuracy against human annotations, and inference cost by input-token count.The paper links token count to pay-per-token pricing for advanced LLMs.Source: §4.4 · p. 4

4.5 Variance Reduction

  • To reduce variance, the study lowers decoding temperature to 0.25 for text arithmetic tasks and reuses fixed initial agent responses across topology designs.The authors report that most variance arises from first-round individual responses and say this procedure improves result reliability.Source: §4.5 · p. 4

5 Experiments: MAD with Single LLM

Sparse multi-agent debate (MAD) generally matches or exceeds fully connected MAD while reducing inference costs across text, multimodal, and alignment tasks. The experiments also examine how reference-solution quality and debate length relate to sparse MAD’s performance.

5.1 MAD on Text Reasoning Tasks

  • Sparse MAD matches or beats fully connected MAD on MATH and GSM8K; inference costs fall by up to −41.5% and −43.5%, respectively.Both setups outperform Chain-of-Thought and self-consistency on these tasks.Source: §5.1 · p. 4, Table 2 · p. 5

5.2 MAD on Multimodal Reasoning Task

  • On MathVista, the best sparse MAD configuration improves accuracy +1.2% over fully connected MAD and +6.4% over self-consistency, while reducing total tokens by up to 33.1%.Excluding input image tokens, total token usage is reduced by 40.6%.Source: §5.2 · p. 5, Table 3 · p. 5

5.3 MAD on Alignment Labeling Tasks

  • On alignment labeling tasks, MAD outperforms single-agent methods, while most sparse configurations match or exceed fully connected MAD and reduce costs by up to −53.3% for GPT-3.5 and −53.5% for Mistral 7B.Sparse MAD improves performance by approximately +0.5% to +1.0% with GPT-3.5 and about +1.1% to +1.5% with Mistral 7B, depending on task.Source: §5.3 · pp. 5–6
  • GPT-3.5 has lower helpfulness alignment accuracy than Mistral 7B; the authors attribute the difference to their training corpora and leave few-shot mitigation for future work.The proposed few-shot mitigation is a hypothesis and was not explored in this work.Source: §5.3 · p. 6

5.4 Why Does Sparse MAD Work?

  • For easier questions, more reference solutions improve answer likelihood, but for harder questions, additional references tend to mislead agents.The probability analysis used three GSM8K questions and varied reference-solution correctness; Figure 3 presents the results.Source: §5.4 · p. 6, Figure 3 · p. 7
  • Sparse MAD tends to sustain debate for more rounds before all agents converge on the same answer.Effective debate rounds are counted until all agents reach consensus; Figure 4 compares topologies across reasoning and alignment tasks.Source: §5.4 · p. 6, Figure 4 · p. 7

6 Experiments: MAD with Multiple LLMs

The section examines topology design when agents use different LLMs, finding that stronger-model placement at higher-centrality nodes performs better and helps weaker agents gain accuracy sooner.

6 Experiments: MAD with Multiple LLMs

  • When agents use different LLMs, permutation invariance is broken, so a regular graph may not be optimal.The section considers a network of N agents, with M stronger-model agents and N − M weaker-model agents.Source: §6 · p. 7
  • The comparison tested isotropic topologies with the stronger LLM at low or high centrality, corresponding to degree 1 or degree 5.The table reports isotropic MAD accuracy averaged over debate rounds.Source: Figure 5 · p. 7, §6 · p. 7, Table 6 · p. 7
  • +3.0% improvement resulted from placing GPT-3.5 at degree 5, versus +1.8% at degree 1, in harmlessness labeling.The experiment used six agents: one GPT-3.5 and five Mistral 7B models.Source: Table 6 · p. 7, §6 · p. 7
  • With degree 5, the stronger agent raises weaker agents’ average accuracy in one debate round; with degree 1, information spreads over two rounds and loses more information.Figure 6 plots weaker agents’ average accuracy across debate rounds.Source: §6 · p. 7, Figure 6 · p. 8

7 Conclusion

The paper concludes that sparse communication can preserve or improve debate performance while reducing computational costs. It extends debate to alignment labeling and finds that stronger models at more connected nodes improve overall performance.

7 Conclusion

  • Sparse multi-agent debate configurations achieve comparable or superior performance to standard debate while greatly reducing computational costs.The conclusion also reports benefits of sparse configurations over fully connected ones in alignment labeling.Source: §7 · p. 8
  • The framework is extended to alignment labeling, where debate outperforms single-agent setups and self-consistency, and sparse debate offers benefits over fully connected configurations.The conclusion presents this as an additional evaluation of the framework.Source: §7 · p. 8
  • Assigning stronger LLMs to more connected agents enhances overall performance, while case studies offer insights into sparse debate’s effectiveness.The paper identifies further mechanism analysis and optimal topology design as future research directions.Source: §7 · p. 8

8 Ethical Considerations

The ethical considerations describe the data and tools used and limit AI assistance to writing polish and grammar correction.

8 Ethical Considerations

  • The framework used publicly available datasets designed for academic research and avoided personal or confidential data.The authors state that they followed ethical guidelines in using these datasets.Source: §8 · p. 8
  • The authors report using only public APIs with appropriate licensing.They state that this was intended to ensure lawful and ethical use of external tools.Source: §8 · p. 8
  • AI tools were used only to polish writing and correct grammar, not to generate research content or ideas.The authors describe this as preserving the work’s authenticity and originality.Source: §8 · p. 8

9 Limitations

The authors identify limits in graph settings, data coverage, theoretical understanding, and topology design, and note potential misuse and resource costs. An additional temperature experiment reports little accuracy change and slightly greater cost savings at T = 0.25.

9 Limitations

  • The study uses static graphs, zero-shot prompt design, publicly available GPT and Mistral models, and regular graphs, limiting coverage of dynamic networks, diverse agent populations, and other graph configurations.The authors recommend studying dynamic graphs, more models, and varied connectivity.Source: §9 · p. 8
  • A subset of academic datasets, chosen under data-access and computational constraints, may not represent broader real-world diversity and limits generalization across domains and contexts.The authors recommend including a wider range of datasets.Source: §9 · p. 8
  • The paper lacks a rigorous theoretical proof for why sparse connectivity can improve performance and has no definitive method for choosing an optimal topology.The authors say this limits generalization and confident application across settings; potential explanations remain open.Source: §9 · pp. 8–9
  • The authors identify risks of biased information or misinformation and substantial computational-resource requirements, with possible effects on energy use and environmental sustainability.They call for robust, trustworthy, and energy-efficient systems.Source: §9 · p. 9

B Additional Experiments with Different Temperature

  • At T = 0.25, multimodal accuracy showed almost no difference from T = 1, while cost savings were slightly greater.The reported performance at T = 1 is in Table 3; Table 7 shows the cost savings at T = 0.25.Source: B Additional Experiments with… · p. 12

C Additional Experiments with 4 Agents

A regular graph with four agents has two configurations, and both show similar GSM8K accuracy.

C Additional Experiments with 4 Agents

D ProbMAD: MAD with Probablistic Topology

ProbMAD assigns a probability to whether agents see prior-round reference solutions; its reported GSM8K performance is comparable to fully connected MAD, with cost savings similar to sparse topologies. Sparse MAD also increases effective debate rounds on both Mistral preference tasks, while agents exhibit four common response types to peer solutions.

D ProbMAD: MAD with Probablistic Topology

  • ProbMAD is comparable to fully connected MAD in performance, with cost savings similar to sparse topologies on GPT-3.5 GSM8K.Its communication probability is D; the authors note that further work is needed to compare probabilistic and deterministic sparsity and explain the mechanism.Source: D ProbMAD: MAD with Probablistic… · p. 12, Table 9 · p. 13

E Rounds of Effective Debate for Mistral 7B

F Types of Agent Behaviors

Related papers

Loading 2406.11776v1…