Source-linked AI summary
Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies
Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulić, Anna Korhonen, Sercan Ö. Arık
TL;DR
Designing effective multi-agent systems is difficult because prompt sensitivity, agent interdependence, and combinatorial topology choices complicate joint optimization. MASS analyzes these factors, then interleaves local prompt optimization, pruned topology search, and global prompt optimization. Across diverse tasks, MASS-optimized systems substantially outperform manual and automated alternatives, while the authors identify further topology expansion as an avenue for improvement.
Problem
MAS design must jointly address sensitive prompts and combinatorial topology choices, while prompt optimization is complicated by dependencies among agents and sparse rewards.
Method
MASS interleaves block-level prompt optimization, influence-weighted topology search in a pruned space, and workflow-level prompt optimization conditioned on the selected topology.
Results
MASS-optimized systems achieve state-of-the-art performance and substantially outperform manual MAS baselines and automatically generated alternatives across reasoning, multi-hop understanding, and code-generation tasks.
Takeaways & Limitations
Effective MAS design depends on optimizing prompts together with influential topologies, and MASS-derived systems provide design principles for future MAS development.
Takeaways & Limitations
MASS's topology space may omit other useful topologies, and pruning redundant agent communications could further improve performance.
Abstract
from arXiv · showhide
Large language models, employed as multiple agents that interact and collaborate with each other, have excelled at solving complex tasks. The agents are programmed with prompts that declare their functionality, along with the topologies that orchestrate interactions across agents. Designing prompts and topologies for multi-agent systems (MAS) is inherently complex. To automate the entire design process, we first conduct an in-depth analysis of the design space aiming to understand the factors behind building effective MAS. We reveal that prompts together with topologies play critical roles in enabling more effective MAS design. Based on the insights, we propose Multi-Agent System Search (MASS), a MAS optimization framework that efficiently exploits the complex MAS design space by interleaving its optimization stages, from local to global, from prompts to topologies, over three stages: 1) block-level (local) prompt optimization; 2) workflow topology optimization; 3) workflow-level (global) prompt optimization, where each stage is conditioned on the iteratively optimized prompts/topologies from former stages. We show that MASS-optimized multi-agent systems outperform a spectrum of existing alternatives by a substantial margin. Based on the MASS-found systems, we finally propose design principles behind building effective multi-agent systems.
1. Introduction
Multi-agent systems improve complex-task solving by coordinating agents with prompts and interaction topologies, but designing both is difficult. MASS addresses this challenge by interleaving prompt and topology optimization across a structured search space.
- MAS coordinate multiple agents through topologies that provide diverse perspectives or roles, including verification and debate.
- Prompt sensitivity can amplify across cascaded agents, while topology design and the combined prompt-topology space require substantial manual experimentation.
- Existing automation studies exemplars, agent scaling, or topology generation, but leave the factors most important for MAS performance insufficiently understood.
- MASS analyzes MAS design factors, identifies influential prompts and a small subset of influential topologies, and prunes the search space.
- MASS interleaves block-level prompt warm-up, workflow topology optimization, and workflow-level prompt optimization.
- MASS-optimized systems outperform manually crafted and automatically generated alternatives across reasoning, multi-hop understanding, and code-generation tasks.
2. Designing Multi-Agent Systems
MAS design has block-level and workflow-level dimensions: prompts shape individual agent roles, while topologies determine agent composition and arrangement. The design space includes reusable collaboration blocks and task-specific tool or summarization components, but prompt optimization is difficult because agents are interdependent.
- Block-level design optimizes individual agents within functional building blocks, while workflow-level design selects agent types, quantities, and arrangements.
- Prompts define agent roles, behavioral instructions, and optional few-shot demonstrations, making them a primary block-level optimization target.
- MAS prompt optimization is difficult because intermediate ground-truth outputs are unavailable, agent dependencies increase complexity, and reward signals become sparse as agent count grows.
- The search space contains Aggregate, Reflect, and Debate blocks for parallel prediction, iterative verification, and fully connected opinion exchange.
- Custom Summarize agents support long-context abstraction, while Tool-use inserts capabilities such as retrieval or code execution.
3. Mass: Multi-Agent System Search
MASS reduces MAS design complexity by optimizing prompts locally, searching influential workflow topologies within a pruned space, and then jointly optimizing prompts globally for the selected workflow.
- 1) Block-level prompt optimization: MASS first warms up the initial predictor and individually optimizes each topology block with manageable minimum-agent configurations.
- 2) Workflow topology optimization: The workflow topology stage evaluates agent arrangements and prunes dimensions whose measured influence yields low selection probability.
- 2) Workflow topology optimization: Only a small fraction of topologies improve MAS performance, motivating influence-based pruning rather than unrestricted topology search.
- 2) Workflow topology optimization: MASS samples valid workflows from the pruned space under an agent-count budget and constructs them in a predefined summarize–reflect–debate–aggregate order.
- 3) Workflow-level prompt optimization: The final stage jointly optimizes prompts across all agents after selecting the best topology, adapting prompts to orchestration interdependence.
4. Experiments
The experiments evaluate MASS across reasoning, long-context understanding, and coding tasks against manual and automatic baselines, showing gains from interleaved prompt and topology optimization. Ablations and trajectories examine which stages matter, how search behaves, and what architectures and design principles emerge.
- Setup: MASS is evaluated on reasoning, long-context understanding, and coding tasks using Gemini 1.5 Pro and Flash, with further validation on Claude 3.5 Sonnet and Mistral Nemo.The evaluation includes MATH, DROP, HotpotQA, MuSiQue, 2WikiMultiHopQA, MBPP, HumanEval, and LiveCodeBench.
- Setup: The comparison includes single-agent, self-consistency, self-refinement, debate, ADAS, and AFlow baselines under comparable inference cost and a maximum of 10 agents.Table 1 reports task-appropriate accuracy, F1, or pass@1 metrics, while AFlow* uses Claude 3.5 Sonnet as optimizer and Gemini 1.5 Pro as executor.
- Main results: 78.8% and 74.3% average performance are achieved by MASS on Gemini 1.5 Pro and Flash, respectively, with consistent improvements also observed on Claude 3.5 Sonnet and Mistral Nemo.MASS substantially improves over common multi-agent systems that scale agents without optimizing collaborative prompts.
- Ablating optimization stages: 6% average gain separates block-level optimization from single-agent APO, followed by 3% from topology optimization and approximately 2% from workflow-level prompt optimization.The ablation also reports that prompt optimization beforehand and search-space pruning are both critical for effective topology exploration.
- Cost-effectiveness: MASS steadily improves validation performance through interleaved prompt and topology search, while AFlow shows greater variance and ADAS can become trapped in over-complex topologies.MASS’s optimization within stages (1) and (2) can be completely parallelized, unlike the iterative trajectories of ADAS and AFlow.
- Best-found MAS architectures & Design principles: On MATH, debate is initially strongest, but aggregation with more parallel agents later outperforms debate, after which workflow-level prompt optimization finds the best predictor.The resulting design principles emphasize optimizing individual agents before composition, selecting influential topologies, and jointly optimizing workflow prompts.
5. Related Work
Related work studies agent cooperation and automated multi-agent design through different topologies, optimization procedures, and search spaces. The supplied discussion positions MASS as searching both topology and prompt design, addressing prompt optimization that prior work has relatively understudied.
- Forms of LLM-based agentic systems: LLM-agent cooperation commonly uses parallel exploration, serial refinement, aggregation, or more complex debate topologies.Self-consistency represents parallel scaling, reflective agents represent serial refinement, and aggregation summarizes multiple opinions.
- Forms of LLM-based agentic systems: Mass automatically searches effective topologies among established parallel, serial, aggregation, debate, and extended connection patterns.The paper frames topology as the coordination structure through which multiple agents cooperate.
- Automatic optimization for MAS: Automatic MAS optimization includes learnable agent policies, synthesized trajectories, dynamic activation, hyperparameter optimization, connection optimization, and code-based topology generation.These approaches require more sophisticated search spaces and algorithms because multi-agent optimization is more complex than single-agent optimization.
- Automatic optimization for MAS: Prior automatic agent-design work has relatively under-studied prompt design, whereas MASS explicitly searches both prompt and topology spaces.The distinction concerns whether optimization accounts for how agents are prompted as well as how they are connected.
6. Conclusion
The paper analyzes MAS design spaces, then introduces MASS to efficiently search a pruned space by interleaving prompt and topology optimization. MASS-optimized systems outperform existing manual and automated approaches across extensive tasks, motivating design principles for future MAS.
- MASS analyzes MAS design spaces to identify influential prompts and redundant search-space components.These findings support searching within a pruned design space.
- MASS interleaves prompt and topology optimization in a multi-stage framework for generating high-performing MAS.The framework is agnostic to customized prompt optimizers and topology design spaces.
- MASS-optimized MAS significantly outperform existing manual and automated approaches across an extensive set of tasks.
- The strongest systems discovered by MASS yield design principles intended to guide future MAS development.
A. Limitations and future work
MASS remains extensible beyond its current topology and search configurations, with additional topologies, communication pruning, and advanced optimization algorithms identified as future opportunities.
- MASS’s topology space covers serial, parallel, and mixture connections, but other topologies may further improve performance.The debate topology uses fully connected communication across agents.
- Pruning redundant agent communications may improve the efficiency of the strongest MASS-found design.
- Bayesian optimization and other advanced search algorithms may improve MASS sample efficiency in more complex design spaces.
B.1. Datasets
The evaluation uses datasets spanning competition mathematics, symbolic reasoning, long-context multi-hop question answering, and code generation, with task and topology specifications reported separately.
- The datasets cover competition-level mathematics and discrete, symbolic reasoning over paragraphs.The specified datasets are Hendryck’s MATH and DROP.
- HotpotQA, MuSiQue, and 2WikiMultiHopQA evaluate long-context reasoning using standardized LongBench versions.
- The evaluation includes MBPP and additional code-generation datasets listed in the dataset specification.
- Table 2 reports dataset splits, topology search spaces, and MASS-optimized MAS, while Table 3 reports search dimensions for each topology.Table 3 identifies the minimum topology as the building block optimized in MASS Stage (1).
- Figure 8 visualizes topology building blocks and the best MASS-discovered topologies from Gemini 1.5 Pro.
B.2. Baselines
The baselines include prompting, self-consistency, self-refinement, debate, and automated topology or workflow design methods, evaluated under specified reproduction settings.
- CoT uses zero-shot prompting, while SC samples diverse reasoning traces and applies rule-based majority voting, reported as SC@9.
- Self-Refine combines a predictor with a self-reflector, stopping when the reflection is correct and allowing up to five rounds.The maximum is 11 calls in the worst case.
- Multi-Agent Debate uses three agents for three debate rounds, followed by an aggregator, totaling ten agents.
- ADAS is reproduced with Gemini 1.5 as optimizer and evaluator, using 30 search rounds and three validation evaluations per round.
- AFlow searches workflows with Monte-Carlo Tree Search, but its comparison is not completely fair because its original optimizer is Claude 3.5 Sonnet.AFlow uses Gemini 1.5 Pro as executor; out-of-time errors are reported as ‘-’.
- MASS constructs different topologies sequentially using a simple predefined rule, while finding configuration impacts outweigh ordering substantially.
C. Additional experiments
Across model backbones, MASS recovers performance when transferred agent prompts and basic topologies degrade, while task-family analyses reveal topology patterns and task-specific differences.
- Generalization across LLM backbones: Mass recovers performance and improves substantially over the initial agent when transferred prompts and basic topologies cause degradation on Claude 3.5 Sonnet.The Claude setup uses standard HotpotQA instead of LongBench because of context-window differences.
- Generalization across LLM backbones: Mass demonstrates consistent improvements over baselines on Mistral-Nemo-12B across a representative subset of coding tasks.The subset is evaluated to save resources.
- Discussion on Mass-found topologies: Topology patterns vary by task family: debating benefits factual multi-hop tasks, more exploration benefits MATH and DROP, and reflection with tool use appears in coding tasks.The reported patterns include HotpotQA, MuSiQue, 2WikiMQA, MATH, and DROP.
- Discussion on Mass-found topologies: Even within a task family, the best configurations differ, supporting automatic optimization to identify influential topologies for customized search spaces.The passage presents this as a distinctive advantage of MASS.
C.3. Cost analysis
MASS has staged computation costs, while its optimized systems can offer stronger token-effectiveness than baselines and comparable training cost to auto-agent methods.
- Cost analysis: The 1PO cost scales linearly with the number of building blocks J.Its cost is computed from prompt candidates, evaluation rounds, and agent API calls.
- Cost analysis: The 3PO cost scales linearly with the number of agents in the network.3PO performs prompt optimization on the best-found workflow.
- Cost analysis: MASS training cost is comparable to the training cost of auto-agent baselines.Table 7 also reports inference cost per query and reference baseline costs.
- Cost analysis: MASS can generate a Pareto-front of optimized designs with stronger token-effectiveness than baselines, allowing selection of more cost-efficient workflows.The primary optimization objective remains maximizing task performance within the same training budget.
C.4. Graph optimization baseline
Compared with graph optimization methods, MASS attributes larger gains to prompt optimization, while remaining compatible with alternative prompt optimizers and retaining prompt templates that improve performance.
- Graph optimization baseline: Graph optimization improves inference efficiency more effectively by sparsifying fully connected graphs than by improving task performance.This comparison is made on the overlapping task set from the original GPTSwarm work.
- Graph optimization baseline: MASS brings gains of 8% on MATH and 6% on HumanEval over graph optimization methods.These are representative reasoning and coding tasks.
- Ablation on prompt optimizers: MASS is plug-and-play with arbitrary prompt optimizers and is agnostic to the specific optimizer used.MIPRO is integrated as a representative optimizer for joint instruction and exemplar optimization.
- Ablation on prompt optimizers: Including data summaries, task demonstrations, and past instructions in MIPRO prompts improves final performance, so MASS retains those prompt components.The observation is reported from the prompt-template analysis.