Source-linked AI summary
Scaling Large Language Model-based Multi-Agent Collaboration
Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, Maosong Sun
TL;DR
The paper investigates whether continuously adding collaborative agents can produce scalable gains comparable to neural scaling, addressing limited evidence from small multi-agent systems. It introduces MACNET, which organizes agents in directed acyclic graphs and topologically orchestrates their reasoning. Evaluations report effective collaboration among over a thousand agents, irregular topologies outperforming regular ones, and logistic performance growth with earlier collaborative emergence.
Problem
Prior multi-agent studies usually involve fewer than ten agents, leaving the performance impact of continuously adding collaborative agents largely overlooked.
Method
MACNET uses directed acyclic graphs to organize agents and topologically orchestrate their interactive reasoning for autonomous task solving.
Results
MACNET supports effective collaboration among over a thousand agents, with irregular topologies outperforming regular ones and overall performance following a logistic growth pattern as agents scale.
Takeaways & Limitations
Collaborative scaling can extend inference-time procedural thinking beyond small agent groups, with collaborative emergence occurring earlier than previously observed neural emergence.
Takeaways & Limitations
The scaling-law fitting reflects only a general pattern from network scale and requires future characterization using profiles, tools, communication protocols, or social routing.
Abstract
from arXiv · showhide
Recent breakthroughs in large language model-driven autonomous agents have revealed that multi-agent collaboration often surpasses each individual through collective reasoning. Inspired by the neural scaling law--increasing neurons enhances performance, this study explores whether the continuous addition of collaborative agents can yield similar benefits. Technically, we utilize directed acyclic graphs to organize agents into a multi-agent collaboration network (MacNet), upon which their interactive reasoning is topologically orchestrated for autonomous task solving. Extensive evaluations reveal that it effectively supports collaboration among over a thousand agents, with irregular topologies outperforming regular ones. We also identify a collaborative scaling law--the overall performance follows a logistic growth pattern as agents scale, with collaborative emergence occurring earlier than traditional neural emergence. We speculate this may be because scaling agents catalyzes their multidimensional considerations during interactive reflection and refinement, thereby producing more comprehensive artifacts. The code is available at https://github.com/OpenBMB/ChatDev/tree/macnet.
1 INTRODUCTION
The paper asks whether continuously adding collaborative agents can produce scalable performance gains analogous to neural scaling. It introduces a DAG-based collaboration design and reports effective collaboration beyond a thousand agents.
- Motivation: Multi-agent collaboration often surpasses standalone intelligence through interactive reflection and refinement, but prior studies usually involved fewer than ten agents.Only limited work extended to several dozen agents.
- Approach: MACNET organizes agents in a directed acyclic graph, assigning supervisory critics to edges and compliant actors to nodes.This functional bipartition promotes role specialization and prevents information backflow.
- Approach: Agents interact in topological order, iteratively refining artifacts while propagating only the refined artifact rather than the entire dialogue.The design suppresses global broadcasting and context explosion.
- Findings: MACNET supports effective collaboration among over a thousand agents, and irregular topologies outperform regular ones across extensive downstream evaluations.The evaluations use chain, tree, and graph families with six representative variants.
- Findings: Overall performance follows a logistic growth pattern as agents scale, with collaborative emergence occurring earlier than previously observed neural emergence.The authors speculate that scaling agents promotes multidimensional considerations during reflection and refinement.
2 MULTI-AGENT COLLABORATION NETWORK
MACNET uses a directed acyclic graph to assign specialized agents and orchestrate artifact refinement in topological order. Memory control retains concise artifacts to reduce context growth and improve scalability.
- 2.1 NETWORK CONSTRUCTION: MACNET organizes agents into a graph whose nodes and edges encode entities and their directed relationships for collaborative task solving.The framework uses graph structure to orchestrate agent interactions.
- 2.1 NETWORK CONSTRUCTION: A directed acyclic graph prevents information backflow and avoids task-specific cycle-breaking designs.The paper links acyclicity to greater generalizability and adaptability across contexts.
- 2.1 NETWORK CONSTRUCTION: The framework studies chain, tree, and graph topologies, divided into six representative sub-topologies.Chains are linear, trees branch independently, and graph structures support more arbitrary connectivity.
- 2.1 NETWORK CONSTRUCTION: Supervisory critics are assigned to edges while compliant actors are assigned to nodes, creating a functional bipartition of labor.The assignment is intended to promote role specialization and progressive task solving.
- 2.2 INTERACTIVE REASONING: Topological ordering places each source actor before its edge critic and destination actor, establishing orderly information propagation.Each node is visited only after its dependencies have been traversed.
- 2.2 INTERACTIVE REASONING: Each connected critic–actor pair engages in iterative multi-turn reflection and refinement, producing a refined artifact for subsequent interaction.Critics provide instructions and actors provide or refine artifacts.
- 2.3 MEMORY CONTROL: Long-term memory retains only the final artifact from current dialogue, while short-term memory supports context-aware decisions within an interaction.This controls each agent’s context visibility.
- 2.3 MEMORY CONTROL: Without memory control, context length grows quadratically with network size; the mechanism reduces this growth from quadratic to linear.The analysis presents this as suppressing context explosion and enabling larger networks.
3 EVALUATION
Evaluations compare MACNET with established baselines across heterogeneous downstream tasks and examine how topology, direction, density, and scale affect collaboration. MACNET generally improves performance, supports large networks, and reveals task- and topology-dependent trade-offs.
- 3.1 DOES OUR METHOD LEAD TO IMPROVED PERFORMANCE?: MACNET-CHAIN consistently surpasses all baselines across most metrics, with artifacts repeatedly reflected and refined to mitigate inaccuracies and hallucinations.The default comparative setting uses the simplest topology, chain.
- 3.1 DOES OUR METHOD LEAD TO IMPROVED PERFORMANCE?: MACNET automatically generates cooperative networks through simple topology and scale hyperparameters, avoiding task-specific customization required by some alternatives.The passage contrasts this with GPTSWARM’s extensive node- and edge-level customization requirements.
- 3.2 HOW DO DIFFERENT TOPOLOGIES PERFORM AGAINST EACH OTHER?: Topology effectiveness varies by task: chains suit software development, whereas trees are more suitable for creative writing.The authors associate software development with sequential processes and creative writing with divergent structures.
- 3.2 HOW DO DIFFERENT TOPOLOGIES PERFORM AGAINST EACH OTHER?: Irregular topologies often outperform regular ones, while random topologies consume approximately 51.92% less time than mesh topologies.The authors hypothesize that overly dense interactions cause information overload, whereas random connections can create small-world shortcuts.
- 3.2 HOW DO DIFFERENT TOPOLOGIES PERFORM AGAINST EACH OTHER?: Divergent topologies substantially outperform reversed convergent counterparts, because divergence supports varied artifact discussion while convergence complicates artifact integration.The authors recommend maximizing divergence and minimizing convergence to reduce degradation during aggregation.
- 3.3 COULD A COLLABORATIVE SCALING LAW BE OBSERVED?: MACNET’s scaling trend suggests balancing topology shape and scale: a node magnitude of 24 appears reasonable, while more agents can improve performance if collaboration is effective.The authors note efficiency advantages for sparse topologies and performance advantages for dense ones.
4 RELATED WORK
Prior work frames neural scaling as performance growth with model, data, and compute scale, while extending LLMs into autonomous agents through added capabilities. Multi-agent collaboration builds on these agents by using interactive reflection, refinement, and diverse expertise.
- Neural scaling laws relate performance or loss to model size, dataset size, training time, and compute.
- Autonomous-agent research augments LLMs with capabilities including memory, tool use, procedural planning, and role playing.
- Multi-agent collaboration uses interactive reflection and refinement to combine the distinct expertise of diverse agents.
5 CONCLUSION
The study introduces MACNET to scale multi-agent collaboration through graph-organized reasoning and finds that large, irregular networks can improve task solving. It reports logistic collaborative scaling, earlier collaborative emergence than neural emergence, and a boundary on scaling.
- MACNET organizes agents with graphs and orchestrates their reasoning for autonomous task solving.
- Over a thousand agents can collaborate effectively, and irregular topologies outperform regular ones.
- Overall performance follows a logistic growth pattern as the number of agents increases, with collaborative emergence occurring earlier than neural emergence.
- The study reports limits on the scaling horizon and proposes inference-time agent collaboration as a possible shortcut around resource-intensive retraining.
J J Hopfield. Neural Networks and Physical Systems with Emergent Collective Computational
This section contains bibliography entries spanning neural scaling laws, multi-agent collaboration, debate, and related agent systems. It also cites work on directed acyclic graphs, topological sorting, and large-scale network control.
- Related methodological references address directed acyclic graphs and topological sorting of networks.
- Several cited papers study collaborative or simulated interactions among language-model agents.
- The bibliography includes research on multi-agent debate and persuasive debating for truthful answers.
- Additional citations cover large-scale network control, self-refinement, and agent operating systems.
- The references include work on scaling laws for neural language models.