PaperlayerView exact revision on arXiv

Source-linked AI summary

At Equal Inference Cost, Multi-Agent Structure Does Not Beat a Single Frozen Agent

David Dylan, Aoife Brennan, Cian Murphy, Niamh O'Sullivan, Conor Kelly, Saoirse Walsh

arXiv:2609.04217v1cs.MA

TL;DR

Multi-agent pipelines are often compared with single agents under unequal language-model-call budgets, leaving their structural benefit unclear. This paper introduces call-matched evolution of a frozen-backbone Planner-Executor-Critic team and finds no statistically significant advantage over evolving a single agent, despite higher evaluation cost.

  • Problem

    Existing comparisons typically fix environment rollouts rather than total language-model calls, leaving the benefit of multi-agent structure under equal inference cost unresolved.

  • Method

    MA-EVOLVE evolves Planner, Executor, and Critic prompts by per-role coordinate ascent over a shared frozen backbone under an identical total-call budget.

  • Results

    Across ALFWorld and WebShop, multi-agent structure provides no clear benefit: on ALFWorld the team scores 0.769 versus 0.754 for the single agent, p = 0.80, while WebShop evolution is null and the team trends worse.

  • Takeaways & Limitations

    At equal inference cost, the realized improvement comes from evolving the executor, while planner and critic prompts add calls without corresponding benefit.

  • Takeaways & Limitations

    The study evaluates multi-agent evolution in the regime of a single frozen backbone, so its findings are scoped to that setting.

Abstract

from arXiv · show

Multi-agent LLM pipelines, such as Planner-Executor-Critic teams, often report gains over single agents, but these gains usually come with higher inference cost because the team makes multiple model calls per environment step. Existing automated methods search over roles, topologies, and prompts, but typically compare teams against single agents at equal environment rollouts, giving the team extra compute. We instead fix the total number of language-model calls and ask whether evolving a multi-agent team still beats evolving a single agent under the same budget. We introduce MA-Evolve, which represents a Planner-Executor-Critic team as three evolvable role prompts and optimizes them by per-role coordinate ascent over a shared frozen 7B backbone. On ALFWorld, evolving a single executor significantly improves over the unevolved agent, while the full team achieves the highest mean but is not statistically better than the single agent: 0.769 versus 0.754, p = 0.80, despite using 1.8 times more evaluation calls. Leave-one-in analysis shows that the realized value comes entirely from the executor; the planner and critic evolve to empty or low-impact prompts and rarely change the executor's action. With 2-3 times free compute, the team only matches the single agent, and on WebShop evolution is null while the team trends worse. Under equal inference cost, multi-agent structure adds cost without clear benefit.

I. INTRODUCTION

This paper asks whether evolving a multi-agent team beats evolving a single agent when total language-model calls are held fixed on a frozen backbone. On ALFWorld, the team has the highest mean but no statistically separable advantage, costs 1.8× more evaluation calls, and derives its realized value from the executor.

  • Motivation and evaluation: The iso-call protocol fixes total language-model calls rather than environment rollouts, preventing multi-agent comparisons from receiving extra compute at no cost.The paper reports search calls, evaluation calls per task, and tokens for every arm.
  • Method: MA-EVOLVE evolves Planner→Executor→Critic role prompts over a single frozen backbone using per-role coordinate ascent, mutating one prompt while holding the others fixed.The agents are not fine-tuned; natural-language role prompts are the optimized objects [5].
  • Empirical result: 0.769 versus 0.754 (Δ=+0.015, p=0.80): the Planner→Executor→Critic team is not statistically separable from the evolved single agent while using 1.8× more evaluation calls per task.Single-agent executor evolution improves over Stock by +0.097 (p=0.021).
  • Interpretation and scope: On a frozen 7B backbone, role co-adaptation drives redundancy while 1.8−2.5× call overhead starves iso-call evolution, so planner and critic add cost without corresponding gain.The introduction also reports a consistent WEBSHOP null and contrasts iso-call control with an environment-rollouts team-best-case regime.

II. RELATED WORK · A. Multi-Agent LLM Frameworks

Multi-agent LLM frameworks improve complex task completion by orchestrating multiple model instances, with roles, topology, and protocols forming the design space for evolution. However, orchestration is usually hand-designed, motivating automated evolution and enabling exact iso-call comparison with a single ReAct agent.

  • II. RELATED WORK: The paper organizes related work into five themes spanning frameworks, automated design, population methods, reinforcement learning, and benchmarks with overoptimization.These themes position multi-agent frameworks within the broader literature reviewed by the paper.
  • A. Multi-Agent LLM Frameworks: Multi-agent orchestration reliably improves complex task completion relative to scaling one model, defining this paper’s evolvable design space.The design space includes agent roles, topology, and communication protocol.
  • A. Multi-Agent LLM Frameworks: Role-specialized collaboration assigns agents distinct personas and structured workflows across multi-agent systems.Examples include AutoGen’s multi-agent conversations [1] and CAMEL’s role-playing agents [10].
  • A. Multi-Agent LLM Frameworks: The literature also includes recurring multi-agent orchestration paradigms and surveys documenting their rapid growth.The supplied passage identifies role-specialized collaboration as one recurring paradigm and notes survey coverage of the field.
  • A. Multi-Agent LLM Frameworks: Existing multi-agent orchestration is almost always hand-designed, leaving a gap for methods that automatically evolve system structure and prompts.This gap connects framework design to the evolution literature.
  • A. Multi-Agent LLM Frameworks: A planner→executor→critic team generalizes an existing ReAct rollout through multiple calls, enabling an exact iso-call comparison with a single ReAct agent.The structural correspondence makes the equal-inference-cost comparison precise.

B. Multi-Agent and System Evolution … E. Benchmarks and Goodhart

The paper situates multi-agent evolution within graph-based system search, evolutionary population methods, and weight-trained multi-agent reinforcement learning. It evaluates these ideas on interactive benchmarks while guarding against metric gaming through held-out performance, role-influence measurements, and significance testing.

  • B. Multi-Agent and System Evolution: Automated multi-agent design models language-model systems as optimizable graphs of role nodes and communication edges, spanning GPTSwarm, dynamicnetwork, ADAS, AFlow, and AgentSquare [25], [26], [27], [28].ADAS searches directly in code space, while AFlow searches code-represented workflows with Monte-Carlo tree search.
  • C. Population, Co-Evolution, and Quality-Diversity: Cooperative and competitive co-evolution, quality-diversity, NEAT, CMA-ES, and population-based training provide frameworks for role co-adaptation, diversity, topology-content evolution, and strategy adaptation,, [49],,,, [53].These methods frame whether populations preserve useful diversity or collapse toward homogenized behavior.
  • C. Population, Co-Evolution, and Quality-Diversity: Language-model-as-operator methods retain population dynamics while replacing hand-coded variation with a frozen model, including Evolution through Large Models and Language-Model-Crossover.The supplied passage identifies this line as inheriting evolutionary population dynamics while using the language model as the mutation operator.
  • D. Multi-Agent Reinforcement Learning: Multi-agent reinforcement learning instead updates weights through centralized critics, value factorization, self-play, or learned communication channels [61], [62], [63], [64],,, [67], [68], [69].The learned communication-channel literature is presented as the closest contrast to evolving a natural-language protocol over a frozen model.
  • E. Benchmarks and Goodhart: ALFWorld uses sparse binary rewards, WebShop uses dense [0, 1] attribute-matching scores, and τ-bench provides a tool-agent-user testbed; newer benchmarks target coordination, competition, and deception [8], [9], [78], [79], [80], [81], [82], [83],.These environments supply executable rewards for evaluating interactive agent systems.
  • E. Benchmarks and Goodhart: Metric gaming, sycophantic agreement, and diversity collapse can inflate evolution fitness, so the paper reports held-out performance, measures role influence and inter-role content, and uses significance tests [89], [90].The team’s highest mean is treated as evidence requiring statistical interpretation rather than as a leaderboard result.

F. Distinction from Concurrent Single-Locus and Co-Evolution Studies · III. PROBLEM FORMULATION

The paper distinguishes inter-agent role credit assignment from prior single-agent localization and evaluates single versus multi-agent structures under an equal language-model-call budget. Its formulation treats roles as prompts over a frozen backbone, explicitly prices teams’ extra calls, and uses leave-one-in evolution to localize team value.

  • F. Distinction from Concurrent Single-Locus and Co-Evolution Studies: The study extends single-agent credit assignment from text-harness slots to inter-agent roles, asking whether team value concentrates in one role and survives the team’s extra calls.The related single-agent line finds the reflection/control slot load-bearing; the paper transfers that localization instrument to team roles.
  • III. PROBLEM FORMULATION: The comparison uses a frozen language-model backbone, with each role defined by the backbone paired with a natural-language prompt.The single agent is one executor role whose prompt is the only evolvable string.
  • III. PROBLEM FORMULATION: A team composing N role-calls per action incurs more language-model calls than a single agent, making equal-environment-rollout comparisons computationally confounded.Here N=3 for plan, execution, and criticism, subject to role-specific call cadences.
  • III. PROBLEM FORMULATION: The iso-call invariant first records the single agent’s total search calls, B, then budgets every team arm to stop when cumulative calls reach the same B.This makes a team win evidence that structure pays for its additional calls rather than receiving extra inference for free.
  • III. PROBLEM FORMULATION: Under equal call budgets, a team making N calls per step receives 1/N as many environment rollouts during search as the single agent.Reflection calls are identical per arm and therefore cancel in the comparison.
  • III. PROBLEM FORMULATION: The primary comparison is held-out success at equal search calls, while the evaluation-calls-per-task multiplier separately prices the team’s inference cost.The multiplier is reported in the same table as the headline comparison.
  • III. PROBLEM FORMULATION: Leave-one-in arms evolve only one role prompt while freezing the others, and compare each role’s gain over the unevolved Stock agent with the full team.These comparisons localize each role’s standalone marginal value under the same iso-call budget.

IV. METHOD: MA-EVOLVE · A. The Team Genome

MA-EVOLVE combines a fixed Planner→Executor→Critic team genome with unchanged per-role reflective evolution and CallMeter iso-call budgeting. The genome uses three evolvable role prompts around a shared reason-and-act rollout, while planning and criticism cadences remain fixed.

  • IV. METHOD: MA-EVOLVE: MA-EVOLVE comprises a Planner→Executor→Critic genome, per-role coordinate-ascent optimization, and CallMeter enforcement of the iso-call budget.The optimizer reuses the single-agent reflective-evolution engine unchanged; Figure 2 diagrams the pipeline and Algorithm 1 specifies the optimizer.
  • A. The Team Genome: The fixed-topology team surrounds the existing reason-and-act rollout, isolating composed-call count as the comparison variable against a single ReAct agent.The environment and rollout loop remain unchanged.
  • A. The Team Genome: The Planner reads the task and observation at initialization and every J steps, emitting a sub-goal prepended to the Executor’s preamble.The sub-goal remains active until the next planning step.
  • A. The Team Genome: The Executor proposes each environment action from the trajectory, Planner sub-goal, and latest Critic note, matching the single agent’s role.Its prompt generalizes the single agent’s evolvable preamble.
  • A. The Team Genome: The Critic activates on repeated or null-effect observations, producing a one-line correction injected into the Executor’s next-step preamble.It inspects the latest action and observation.
  • A. The Team Genome: The genome is G = (pplan, pexec, pcrit): three ownership-disjoint evolvable strings, while planning cadence J and the stall trigger are fixed hyperparameters.The rollout wrapper composes the three role calls into one environment action.

B. Per-Role Coordinate-Ascent · C. CallMeter: Enforcing the Iso-Call Budget

MA-EVOLVE evolves Planner, Executor, and Critic prompts by coordinate ascent over a shared frozen backbone while enforcing an iso-call budget. CallMeter anchors the budget to single-agent search and limits team rollouts according to its three calls per step.

  • B. Per-Role Coordinate-Ascent: The architecture uses one frozen backbone for Planner, Executor, and Critic, with the team spending three LLM calls per environment step versus one for a single agent.The Planner emits sub-goals every J steps, the Critic issues corrections on stalls, and the Executor produces the environment action.
  • B. Per-Role Coordinate-Ascent: The genome consists of three role prompts, and the optimizer cycles through Planner, Executor, and Critic coordinates using reflective prompt evolution.MA-EVOLVE adds the multi-agent rollout and iso-call cap to the reused reflective optimizer.
  • B. Per-Role Coordinate-Ascent: Coordinate ascent improves one role prompt at a time while holding the other two fixed, retaining candidates only when held-out validation performance does not decrease.Reflective evolution uses the joint trajectory to assign credit to the role being mutated.
  • C. CallMeter: Enforcing the Iso-Call Budget: The single-agent search runs first, and its cumulative LLM-call count defines the shared iso-call budget B for subsequent team searches.The budget is based on language-model calls rather than environment rollouts.
  • C. CallMeter: Enforcing the Iso-Call Budget: CallMeter snapshots call counts before and after rollout batches and terminates team evolution when cumulative calls reach B, limiting the team to 1/N the single agent’s rollouts.For this architecture, N=3 calls are spent per environment step.
  • C. CallMeter: Enforcing the Iso-Call Budget: Algorithm 1 initializes all three prompts empty, allocates each role a per-role budget of B/|R|, and cycles coordinate ascent under the global cap.Each role is evolved with the remaining prompts fixed, and a candidate is accepted only if validation score does not decrease.
  • C. CallMeter: Enforcing the Iso-Call Budget: During execution, the Planner generates a sub-goal every J steps, the Critic generates a stall correction, and the Executor converts the trajectory plus both signals into one action.All role outputs are produced by the shared frozen backbone.

V. EXPERIMENTAL SETUP

Experiments use ALFWorld as the primary benchmark and WebShop as a dense-reward secondary benchmark, with every role served by one frozen Qwen2.5-7B model under deterministic greedy decoding.

  • Benchmarks: ALFWorld evaluates 134 held-out household tasks with binary success rewards and a 30-step cap, while WebShop evaluates 80 held-out shopping sessions with dense attribute-match scores and a 15-step cap.WebShop tests whether multi-agent effects depend on reward density and can reveal small effects hidden by ALFWorld’s coarse reward.
  • Frozen backbone: A single frozen Qwen2.5-7B model [92] serves every role in every arm, using greedy decoding at temperature 0 for deterministic, reproducible trajectories.

VI. RESULTS

On ALFWorld, evolving the executor raises held-out success, while the full team reaches the highest mean at 1.8× cost but is not statistically better than the single agent. On WebShop, evolution is null and the team trends worse; ALFWorld gains concentrate on hard task families without per-type structural value.

  • ALFWorld aggregate results: 0.754 versus Stock’s 0.657 shows that single-agent evolution lifts ALFWorld success by +0.097 (p=0.021).The evolved executor prompt gives concrete action guidance, including opening receptacles before taking objects, taking before placing, using exact action templates, and confirming cool/heat/use subtasks.
  • WebShop results: 0.245 versus 0.242 shows null single-agent evolution on WebShop (p=0.96), while the team reaches 0.185, ∆=−0.060 versus single (p=0.12).The team again costs more calls per task, and the direction is mildly harmful on the dense benchmark.
  • ALFWorld task-family breakdown: 0.111→0.444 on examine-in-light and 0.652→0.826 on heat&place show that ALFWorld’s evolution gains concentrate on hard families.On easier families, arms are near ceiling; the team tracks the single agent on every family and never separates, so structure adds no per-type value.
  • ALFWorld aggregate results: 0.769 is the team’s highest ALFWorld mean, but its +0.015 margin over the single agent is not significant (p=0.80) at 1.8× cost.The team beats Stock by +0.112 (p=0.018), while using 32.2 evaluation calls per task versus 18.0 for the single agent.

VII. ANALYSIS: LOCALIZATION, BOTH REGIMES, AND COLLAPSE · A. The Value Localizes to the Executor

The analysis localizes the team’s realized value to the evolving single-agent prompt and shows that the full team is not meaningfully better. Figure 4 reports nearly equal significant gains for the single agent and full team, with only a small, nonsignificant structural contribution.

  • VII. ANALYSIS: LOCALIZATION, BOTH REGIMES, AND COLLAPSE: The per-role analysis tests where the team’s realized value resides and why its extra roles fail to pay.The section uses per-role localization, a comparison across both budget regimes, and collapse instrumentation.
  • VII. ANALYSIS: LOCALIZATION, BOTH REGIMES, AND COLLAPSE: The section combines localization, both-budget-regime comparison, and collapse instrumentation to explain the team’s limited additional value.Figure 4 provides localization results, while Figure 5 provides collapse instrumentation.
  • A. The Value Localizes to the Executor: +0.097 versus +0.112 are the single agent’s and full team’s gains over Stock, with overlapping bootstrap intervals and a +0.015 structural difference (p=0.80).These are the only two arms whose gains are significant.
  • A. The Value Localizes to the Executor: Evolving only the Executor recovers a substantial gain over Stock, reaching +0.075 in the leave-one-in analysis.This arm is labeled LOI:exec.
  • A. The Value Localizes to the Executor: Evolving only the Critic also recovers a substantial gain over Stock, reaching +0.097 with p=0.037.This arm is labeled LOI:crit.
  • A. The Value Localizes to the Executor: Figure 4 compares each arm’s gain over Stock using paired bootstrap intervals.The intervals support the reported near-equivalence between the single agent and full team.

B. Both Budget Regimes Agree · C. Why the Roles Do Not Pay: Collapse and Starvation

Across both equal-call and fixed-rollout budgets, the evolved team gains no separable advantage over the single agent. The mechanism is role collapse and budget starvation: only the Executor evolves, while Planner and Critic add calls without changing outcomes.

  • B. Both Budget Regimes Agree: 0.739 = 0.739: with fixed environment rollouts and free 2–3× compute, the evolved team only ties the single agent.Extra compute does not produce a separable advantage under the env-rollouts regime.
  • B. Both Budget Regimes Agree: 0.657/0.754/0.769/0.731 are the exact ALFWorld column means for the four headline and Executor-only arms, with evolution gains concentrated in hardest families.Examples include examine-in-light rising from 0.111 to 0.444 and heat&place from 0.652 to 0.826.
  • B. Both Budget Regimes Agree: +0.015 team-over-single gain on ALFWorld is not significant (p=0.80), despite single-agent evolution gaining +0.097 and the full team +0.112 over Stock.The paired 95% bootstrap comparison finds the structural marginal contribution equal to noise.
  • C. Why the Roles Do Not Pay: Collapse and Starvation: Only the Executor prompt evolves under the iso-call budget, reaching approximately 420 characters, while Planner and Critic prompts remain empty across clean runs.Coordinate ascent accepted no non-trivial steering-role content because it did not improve validation performance on the frozen 7B.
  • C. Why the Roles Do Not Pay: Collapse and Starvation: Planner and Critic influence rates are 0.14 and 0.41, respectively, while the Critic restate-rate is 0.00, so neither role materially changes the Executor’s action.The Critic never rewrites the action’s content.
  • C. Why the Roles Do Not Pay: Collapse and Starvation: The team uses the highest evaluation cost, adding 14 calls per task over the single agent, without higher success.The steering roles consume extra calls but do not improve outcomes.
  • C. Why the Roles Do Not Pay: Collapse and Starvation: Under the iso-call cap, spending 1.8–2.5× more calls per step reduces each role’s search budget to B/N, starving steering-role mutations.Coarse binary fitness rarely accepts mutations within that reduced slice, causing the steering roles to freeze empty.

VIII. DISCUSSION · IX. CONCLUSION

Under equal language-model-call budgets on a frozen 7B backbone, adding planner and critic roles increases inference cost without a statistically separable benefit over an evolved executor. The conclusion is bounded to one backbone, topology, and optimizer rather than multi-agent systems generally.

  • VIII. DISCUSSION: The team attains the highest mean, but its margin over the single agent is not statistically separable (p=0.80) and costs 1.8× the calls.The authors characterize structure as paying cost rather than delivering a separable benefit at iso-call.
  • VIII. DISCUSSION: On a frozen shared 7B, the executor prompt can absorb planner and critic functions, leaving steering roles empty because they add no validation-improving content.The proposed mechanism is limited role-differentiation headroom in the shared frozen substrate, not failed evolution.
  • VIII. DISCUSSION: The result is a boundary, with scale, additional budget, and alternative conditions identified as levers that could make role differentiation and structure valuable.Role-influence and inter-role-content meters are intended to detect emerging differentiation and test whether the team-minus-single margin tracks it.
  • VIII. DISCUSSION: Cost-aware multi-agent design is complementary, but on a frozen small backbone the cheapest informative cell—the single executor—is also the accuracy-per-call winner.The iso-call invariant and call meter are presented as safeguards against mistaking cost for performance.
  • IX. CONCLUSION: MA-EVOLVE tests whether a Planner→Executor→Critic team of frozen agents pays for itself by optimizing role prompts with coordinate ascent under a call-metered cap.The method reuses a single-agent reflective evolution engine unchanged while holding total language-model calls rather than environment rollouts fixed.
  • IX. CONCLUSION: The conclusion is limited to one frozen Qwen2.5-7B backbone, one Planner→Executor→Critic topology, and one coordinate-ascent optimizer.The paper does not claim transfer to larger or heterogeneous backbones, and debate, ensemble, or query-adaptive topologies remain out of scope.

APPENDIX A THE EVOLVED EXECUTOR PROMPT · APPENDIX B PER-CELL DETAILS AND HYPERPARAMETERS · APPENDIX C THE WEBSHOP NOTE

The appendices show that iso-call evolution concentrated entirely in the executor, while equal search compute starved multi-agent arms of rollouts and WebShop evolution remained null, with the team performing worst at higher cost.

  • APPENDIX A THE EVOLVED EXECUTOR PROMPT: Under the iso-call budget, only the Executor prompt evolved; Planner and Critic prompts froze empty.The resulting control text emphasized opening closed receptacles, taking objects before placing them, exact action templates, and verifying held objects and appliances.
  • APPENDIX B PER-CELL DETAILS AND HYPERPARAMETERS: 2448−2673 search calls were consumed by every evolved arm, but the equal budget yielded 64 team rollouts versus 132 for the single agent.The iso-call cap therefore held while multi-agent structure reduced search-side environment exposure.
  • APPENDIX B PER-CELL DETAILS AND HYPERPARAMETERS: Planner influence was 0.14 and Critic influence was 0.41, while the Critic never rewrote content despite the steering roles firing hundreds of times.The instrumentation is analysis-only and does not alter the success metric.
  • APPENDIX B PER-CELL DETAILS AND HYPERPARAMETERS: The measured ALFWorld cost multiplier was 1.8× rather than 3× because repeated or null-effect calls made effective calls per step average below the worst-case N=3.The appendix identifies Stock as the unevolved ReAct agent, single as executor-only evolution, and team as three-role coordinate ascent over the frozen Qwen2.5-7B backbone.
  • APPENDIX C THE WEBSHOP NOTE: On dense WebShop, the team scored 0.185, a ∆=−0.060 versus single with p=0.12, while Stock scored 0.242 and single scored 0.245 with p=0.96.The appendix cautions that these absolute scores are not comparable to specialized WebShop leaderboards because all cells share one pipeline and rollout.
  • APPENDIX C THE WEBSHOP NOTE: WebShop corroborates the primary finding: evolution is null, and the team mildly hurts performance while operating at higher cost.The dense secondary is described as matching the ALFWorld pattern, with the team paying more and not benefiting.
  • APPENDIX C THE WEBSHOP NOTE: 0.185 was the team’s WebShop score, versus 0.245 for single and 0.242 for Stock, while the team used 7.6 versus 5.1 evaluation calls.The team was the worst headline cell and the highest-cost arm; single-agent evolution was null (0.245 vs. 0.242, p=0.96).
Loading 2609.04217v1…