Source-linked AI summary
Do We Need Complex Topology Control? Distinct-Peer Random Routing Improves Cost-Efficiency in Sparse Multi-Agent Debate
Boxuan Wang, Zhuoyun Li, Xiaowei Huang, Yi Dong
TL;DR
The paper asks whether complex topology control improves sparse multi-agent debate beyond simple routing and stopping. It evaluates routing and stopping separately, finding that distinct-peer random routing is competitive and that short debate reduces token use while maintaining competitive accuracy. The authors recommend benchmarking advanced topology controls against these simple baselines before justifying their added complexity.
Problem
The paper asks whether topology control improves accuracy or reduces token cost beyond simple sparse routing and short debate.
Source: §1 · p. 1Method
The study uses a two-axis framework separating peer routing from stopping, comparing reference topologies, routing controls, stopping policies, and their combination.
Source: §1 · p. 1Results
Random-NoDup provides competitive accuracy without learned or state-aware routing, while short debate further improves token savings.
Source: §5 · p. 4Takeaways & Limitations
Advanced topology controls should be benchmarked against distinct-peer random routing baselines before their added complexity is warranted.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Multi-agent debate (MAD) has emerged as a promising paradigm for improving the reasoning accuracy of large language models (LLMs) through iterative peer interaction. Communication topology plays a central role in this process, motivating increasingly sophisticated mechanisms that learn, adapt, or dynamically reconfigure agent interactions to improve accuracy or reasoning reliability. Meanwhile, prior studies suggest that much simpler sparse communication can already achieve competitive performance at substantially lower cost. In this work, we take a closer look at sparse MAD and ask whether complex topology control is actually necessary to improve collective reasoning. We find that a simple random-without-replacement routing policy, which lets each agent debate with two distinct and newly sampled peers at every round, provides a surprisingly strong baseline and consistently improves the accuracy-cost trade-off of sparse MAD. Building on this observation, we further study deliberation stopping and show that lightweight stopping can substantially reduce inference cost while preserving competitive accuracy. Our results suggest that sophisticated topology control such as learned topology adaption should be evaluated against strong simple routing and stopping baselines before its additional complexity is justified.
1. INTRODUCTION
The introduction asks whether complex topology control improves sparse multi-agent debate beyond simple routing and short debate. It outlines a routing-and-stopping evaluation and highlights Random-NoDup as a strong accuracy–cost baseline.
1. INTRODUCTION
- The paper asks whether topology control improves accuracy or reduces token cost beyond simple sparse routing and short debate.Prior work found sparse communication could match or exceed fully connected debate accuracy with fewer tokens, while repeated interaction need not outperform majority voting over initial responses.Source: §1 · p. 1
- The two-axis framework separates peer routing from debate duration, comparing reference topologies, routing controls, stopping-only policies, and their combination.A matched-protocol temporal-edge adaptation of AgentPrune serves as a learned comparator.Source: §1 · p. 1
- Random-NoDup has higher observed macro accuracy than Ring at similar token cost, while its short version has the highest observed macro accuracy and lowest mean token use among tested methods.The study evaluates routing and stopping as separate controls on the accuracy–cost trade-off.Source: §1 · p. 1, Figure 1 · p. 1
- The study introduces Random-NoDup as a distinct-peer sparse routing baseline and reports that it is highly competitive, with stopping further improving efficiency.Its evaluation measures final-answer accuracy and total input plus output tokens per question.Source: §1 · p. 1
2. RELATED WORK
Prior work motivates economical sparse communication and increasingly selective or optimized debate topologies, but leaves open how simple random routing performs under controlled conditions.
2. RELATED WORK
- Sparse debate has been found to match or outperform fully connected debate while reducing computation, and moderate sparsity may suppress error diffusion while preserving beneficial information flow.Other work studies selective peers, edge pruning, and lightweight diversity or confidence interventions; additional debate is not automatically useful.Source: §2 · p. 2
- Topology-control research optimizes or generates communication graphs through adaptive structures, guided diffusion, reinforcement learning, and small-world priors.The paper positions its comparison as complementary: it tests whether these controls offer a clear advantage over simple routing under a matched sparse protocol.Source: §2 · p. 2
- The related-work discussion identifies an open comparison: the strength of a simple random sparse baseline when peer collisions and deliberation length are controlled.The study compares against Sparse MAD and an AgentPrune-style learned temporal-edge adaptation while evaluating stopping.Source: §2 · p. 2
3. PRELIMINARIES AND METHODS
The methods compare sparse routing rules and stopping controls under a shared multi-agent debate protocol, alongside a learned topology-control baseline. Routing varies which peers communicate, while stopping varies how long debate continues.
3.1. Sparse debate and reference topologies
- The setup uses n homogeneous agents and at most T answer rounds: agents answer independently in round 1, then later updates read k peers’ responses from the previous round.Source: §3.1 · p. 2
- Reference routing contrasts persistent local peers, a persistent local peer plus a resampled shortcut, and two distinct resampled peers per round.Previously contacted peers remain eligible, and the graph varies across rounds without using agent states to select peers.Source: §3.1 · p. 2, Figure 2 · p. 2
3.2. Routing and stopping controls
- Routing controls include preset or random switching and uncertainty-guided selection, while stopping returns a fixed-horizon majority or uses uncertainty and stable transitions to choose a stopping round.The uncertainty measure reflects answer disagreement and changes rather than correctness; parameters are selected using held-out calibration sets.Source: §3.2 · p. 2
3.3. Learned temporal-edge adaptation
- The learned AgentPrune-style baseline assigns probabilities to candidate edges, updates them from task rewards, and prunes low-value edges.Its sparse-MAD comparison uses disjoint calibration questions while keeping agents, prompts, synchronous updates, received degree, and aggregation rule unchanged.Source: §3.3 · pp. 2–3
4. EXPERIMENTS
The experiments compare peer-routing and stopping policies in sparse multi-agent debate. Random-NoDup remains competitive with controlled routing, while short debate reduces token use; across models, the cost savings are more consistent than accuracy preservation.
4.1. Experimental Setup
- The setup uses eight homogeneous agents, two peer messages per communication round, and evaluates accuracy and inference-token cost across five benchmarks.Full debate runs for 14 rounds; short Random-NoDup stops at round 3 after two communication updates.Source: §4.1 · p. 3
4.2. RQ1: Does Topology Control Improve Routing?
- Full-round Random-NoDup has higher macro accuracy than all tested routing controllers and the learned-edge adaptation, without a consistent accuracy–cost advantage from additional control.Uncertainty-guided routing leads on GSM8K and learned adaptation on MMLU-Pro, but those task-specific gains do not carry across tasks.Source: §4.2 · p. 3
4.3. RQ2: Does Stopping Improve the Trade-off?
- Short Random-NoDup achieves accuracy comparable to longer and adaptive alternatives while using substantially fewer tokens.Macro accuracy is close to its observed peak during rounds 2–5, whereas later rounds add tokens without consistent accuracy improvement.Source: §4.3 · p. 4, Figure 3 · p. 4
4.4. Ablations on Cross-model Performance
- Across four model variants, short Random-NoDup uses the fewest tokens for every model–task pair and has the highest macro accuracy for three models.One model loses accuracy on GSM8K when debate stops at round 3; cost savings transfer more consistently than accuracy preservation.Source: §4.4 · p. 4, Table 2 · p. 4
5. CONCLUSION
The paper evaluates sparse multi-agent debate by separating peer routing from stopping. Its results support distinct-peer random routing as a competitive baseline and short debate as a source of token savings, while recommending comparisons against these baselines before added topology complexity is justified.
5. CONCLUSION
- Random-NoDup provides competitive accuracy without learned or state-aware routing, while short debate further improves token savings.The two-axis framework separates peer routing from stopping and compares reference, adaptive, joint-control, and learned temporal-edge approaches.Source: §5 · p. 4
- Across tested comparisons, added routing control shows no detected accuracy advantage over Random-NoDup, whereas stopping consistently reduces token use.Stopping’s accuracy effect still depends on the model and task; the authors recommend benchmarking advanced topology methods against these simple baselines.Source: §5 · p. 4
COMPLIANCE WITH ETHICAL STANDARDS
The work reports no human participants or newly collected human-subject data.
COMPLIANCE WITH ETHICAL STANDARDS
- Experiments use public benchmarks and model inference conducted under applicable terms of use.The authors report no human participants or newly collected human-subject data.Source: COMPLIANCE WITH ETHICAL STANDARDS · p. 5