PaperlayerView exact revision on arXiv

Source-linked AI summary

MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks

Zixuan Ke, Yifei Ming, Austin Xu, Ryan Chin, Xuan-Phi Nguyen, Prathyusha Jwalapuram, Jiayu Wang, Semih Yavuz, Caiming Xiong, Shafiq Joty

arXiv:2601.14652v5cs.AIcs.CLcs.MA

TL;DR

Automatic MAS design lacks scalable global orchestration and principled evidence about when MAS improve on SAS. MAS-Orchestra addresses this with holistic function-calling RL, while MASBENCH controls MAS–SAS comparisons across five task axes; together, they support consistent benchmark improvements and more than 10× efficiency over strong baselines.

  • Problem

    Automatic MAS design is limited by sequential code-level orchestration that scales poorly, while the benefits of MAS over SAS remain uncertain across task conditions.

  • Method

    MAS-Orchestra trains an orchestrator with function-calling RL to generate a complete MAS holistically, and MASBENCH evaluates MAS–SAS behavior across Depth, Horizon, Breadth, Parallel, and Robustness.

  • Results

    MAS-Orchestra achieves consistent improvements on diverse public benchmarks and lies on the performance–cost Pareto frontier, with more than 10× efficiency over strong baselines.

  • Takeaways & Limitations

    MAS benefits depend on task structure, verification protocols, and model capabilities rather than holding universally, motivating controlled configuration and evaluation.

Abstract

from arXiv · show

While multi-agent systems (MAS) promise elevated intelligence through coordination of agents, current approaches to automatic MAS design under-deliver. Such shortcomings stem from two key factors: (1) methodological complexity - agent orchestration is performed using sequential, code-level execution that limits global system-level holistic reasoning and scales poorly with agent complexity - and (2) efficacy uncertainty - MAS are deployed without understanding if there are tangible benefits compared to single-agent systems (SAS). We propose MASOrchestra, a training-time framework that formulates MAS orchestration as a function-calling reinforcement learning problem with holistic orchestration, generating an entire MAS at once. In MAS-Orchestra, complex, goal-oriented subagents are abstracted as callable functions, enabling global reasoning over system structure while hiding internal execution details. To rigorously study when and why MAS are beneficial, we introduce MASBENCH, a controlled benchmark that characterizes tasks along five axes: Depth, Horizon, Breadth, Parallel, and Robustness. Our analysis reveals that MAS gains depend critically on task structure, verification protocols, and the capabilities of both orchestrator and subagents, rather than holding universally. Guided by these insights, MAS-Orchestra achieves consistent improvements on public benchmarks including mathematical reasoning, multi-hop QA, and search-based QA, while achieving more than 10x efficiency over strong baselines. Together, MAS-Orchestra and MASBENCH enable better training and understanding of MAS in the pursuit of multi-agent intelligence.

1. Introduction

Automatic MAS design faces code-level orchestration that scales poorly and lacks global reasoning, alongside uncertainty about when coordination benefits MAS over SAS. MAS-Orchestra addresses these issues with holistic function-calling RL, while MASBENCH evaluates MAS benefits systematically across task structures.

  • Motivation: Existing automatic MAS approaches use executable-code orchestration that incurs overhead and becomes difficult to scale as subagents and interactions grow complex.The formulation also encourages local, step-wise optimization rather than global reasoning about MAS structure.
  • Motivation: No principled quantitative framework predicts when adding subagents improves performance or degrades it relative to simpler single-agent alternatives.Practitioners therefore often rely on heuristics when assessing MAS value.
  • MAS-Orchestra: MAS-Orchestra introduces degree of MAS (DoM), allowing coordination to be configured according to task needs because not all problems benefit from MAS.The formulation supports low DoM with up to one subagent and high DoM with unconstrained agent count.
  • MAS-Orchestra: MAS-Orchestra formulates orchestration as function-calling reinforcement learning, encapsulating complex goal-oriented subagents while allowing the orchestrator to generate their complete configuration holistically.The orchestrator selects subagents and connections through create_agent and create_flow rather than reproducing internal execution details.
  • MASBENCH: MASBENCH characterizes tasks along Depth, Horizon, Breadth, Parallel, and Robustness to systematically analyze when and why MAS outperform SAS.The benchmark is designed to support controlled comparisons of MAS behavior across task structures.
  • Results: More than 10× efficiency over strong baselines accompanies strong performance on mathematical reasoning, multi-hop question answering, and multi-step search-based QA.The evaluation covers three orchestrator settings and five subagent settings across different model sizes and families.

2. Related Work

Related work distinguishes single-agent and multi-agent systems and contrasts sequential training-time orchestration with MAS-Orchestra’s holistic alternative. The paper positions holistic orchestration as enabling plan-level global coordination while supporting complex subagents and controlled MAS–SAS comparisons.

  • From Single to Multi-Agent Systems: SAS uses one reasoning locus and sequential control loop, whereas MAS contains multiple goal-directed LLM-backed agents coordinating through structured communication.MAS system behavior emerges from collective reasoning across agents with potentially distinct contexts, objectives, and tools.
  • Automatic Training-time Orchestration: Training-time orchestration seeks to avoid expensive inference-time adaptation by directly training an orchestrator, with most prior approaches using sequential multi-step RL.MAS-GPT is identified as a non-RL exception that reports lower performance than inference-time adaptation methods.
  • Automatic Training-time Orchestration: MAS-Orchestra is presented as the first training-time holistic orchestration approach, generating a complete multi-agent system in a single decision step.This lets the orchestrator reason at the plan level rather than over execution trajectories.
  • Automatic Training-time Orchestration: Holistic orchestration enables global coordination across subagents, avoids error accumulation from intermediate states, and aligns training objectives with end-task performance.The orchestrator does not observe intermediate states or partial results; orchestration quality is evaluated through the final output.

3. MAS-Orchestra Framework

MAS-Orchestra formulates orchestration as function-calling reinforcement learning, generating a complete multi-agent system in one decision step while abstracting subagent internals. Its reward is based on final-answer correctness, and GRPO compares sampled orchestrations within groups.

  • MAS-Orchestra maps each task input to a latent orchestration that encodes a complete MAS reasoning process and produces an answer.
  • The user-selected DoM level constrains the allowable orchestration space, including subagent count and inter-agent topology.Low DoM permits at most one subagent without explicit topology, while remaining distinct from SAS.
  • Subagents are exposed through callable signatures while their internal reasoning and execution details remain abstracted from the orchestrator.A deterministic parser instantiates configured subagents and executes their specified connections.
  • The orchestrator generates the entire orchestration in a single decision step and receives no intermediate execution states or partial results.Orchestration quality is evaluated through the final output.
  • The task-level reward is based on final-answer correctness, with subtask correctness incorporated when required by the verification protocol.
  • GRPO samples groups of orchestrations and updates the policy using their relative rewards through a clipped policy-gradient objective.

4. Proposed Analysis Framework for MAS

The analysis framework compares SAS and MAS under controlled task structures and verification protocols using five axes: Depth, Horizon, Breadth, Parallel, and Robustness. MASBENCH instantiates these axes from dependency graphs and provides reproducible, axis-specific evaluation splits.

  • The framework was created because no dedicated benchmark rigorously evaluates the distinction between single-agent and multi-agent systems in controlled settings.
  • MAS gains may reflect coordination and specialization or increased effective compute from ensembling, with outcomes depending on task structure and verification protocol.
  • Five-axis evaluation framework: Depth, Horizon, Breadth, and Parallel capture distinct dependency, decomposability, and coordination patterns, while Robustness tests reliability against incorrect intermediate information.Horizon evaluates intermediate and final outputs because long-horizon reasoning requires carrying and reusing intermediate results.
  • Five-axis evaluation framework: Each axis value represents the number of subtasks satisfying its definition, approximately indicating task complexity; larger values indicate higher complexity.
  • MASBENCH construction: MASBENCH defines instances with questions and dependency graphs, deriving the four structural axes from graph properties and Robustness from adversarial notes.The adversarial notes inject incorrect upstream information into subtask descriptions.
  • MASBENCH construction: MASBENCH uses primarily iGSM-generated instances with non-overlapping training and testing splits to control structural complexity and avoid template-level leakage.
  • MASBENCH construction: The benchmark covers all five axes with values from 2 to 12 and provides separate axis-specific training and test splits.The reported train/test counts are Depth 3,993/1,195; Horizon 2,174/567; Breadth 2,000/676; Parallel 1,807/567; and Robustness 3,000/600.

5. What Factors Affect MAS Performance?

The controlled analyses examine how task structure, orchestrator initialization, and sub-agent reasoning effort affect MAS relative to SAS. MAS benefits vary with sub-agent capability, coordination demands, orchestration style, and context limits.

  • Evaluation axes: MAS outperforms SAS across most subtask structures with Qwen-7b sub-agents, except on Depth, where strict sequential dependence limits coordination benefits.A single sequential chain-of-thought can reduce branching and coordination overhead for strongly interdependent subtasks.
  • Evaluation axes: MAS gains diminish across Depth, Horizon, Breadth, and Parallel when GPT-120b (low) is the sub-agent.Coordination cost and cross-agent error propagation can offset potential gains when the sub-agent is stronger.
  • Evaluation axes: MAS consistently outperform SAS under data poisoning, while SAS accuracy collapses to near zero.Delegation, cross-verification, and a moderating final-answer agent help identify adversarial signals and provide corrective guidance.
  • Evaluation axes: MAS are most effective at the edge of sub-agent competence, when explicit decomposition and moderation expose reasoning capacity.The paper identifies this regime when sub-agents are capable but cannot reliably internalize complex task structure, especially for non-sequential or adversarial tasks.
  • Orchestrator initialization: Instruction-tuned LLM initialization outperforms RLM initialization because it produces more flexible delegation and coordination structures.RLM orchestrators tend to generate simpler systems, sometimes with only one sub-agent, whereas instruction-tuned orchestrators support more varied delegation.

6. Evaluation on Public Benchmarks

MAS-Orchestra is evaluated on diverse public benchmarks spanning mathematical reasoning, multi-hop question answering, and multi-step search. It consistently outperforms strong baselines, generalizes out of distribution, and adapts orchestration to task structure and cost.

  • Benchmark setup: MAS-Orchestra is evaluated on five benchmarks covering math, multi-hop QA, and multi-step search-based QA.The benchmark suite includes AIME24, AIME25, GPQA, HotpotQA, and BrowseComp+; GPQA also tests out-of-domain generalization from math-focused training data.
  • Benchmark setup: The evaluation fixes candidate sub-agents to comparable workflows while varying tools and prompting, including CoT, self-consistency, Debate, Self-refine, and Search.This design supports controlled comparison across orchestration systems using a shared LLM backbone for candidate sub-agents.
  • Configuration: MAS-Orchestra uses low DoM for largely sequential math and reasoning tasks and higher DoM for tasks requiring broader coordination.The configurations encode task- and model-dependent deployment rather than treating MAS as a default replacement for SAS.
  • Overall results: MAS-Orchestra consistently outperforms strong baselines across all evaluated benchmarks and demonstrates robust OOD generalization.The reported gains are attributed to orchestration and transfer to unseen problems rather than sub-agent strength alone.
  • Learned orchestration: MAS-Orchestra learns single-agent delegation under low DoM and parallel search followed by aggregation under high DoM.On BrowseComp+, it typically invokes DeepResearchAgent for three to four parallel searches before combining results with an aggregation agent.
  • Overall results: MAS-Orchestra lies on the performance–cost Pareto frontier, achieving higher accuracy at lower or comparable cost.The paper links this efficiency to dynamically matching MAS designs and agent configurations to task structure.

7. Conclusion

The paper concludes that MAS-Orchestra enables holistic, function-calling RL orchestration, while MASBENCH supports controlled MAS–SAS comparisons across five axes. The reported evidence ties MAS benefits to task structure, verification protocols, and model capabilities, with consistent public-benchmark improvements and Pareto-frontier efficiency.

  • Conclusion: MAS-Orchestra formulates automatic MAS design as function-calling reinforcement learning, while MASBENCH evaluates MAS–SAS differences across five axes.The five axes are Depth, Horizon, Breadth, Parallel, and Robustness.
  • Conclusion: The analyses show that MAS benefits are shaped by task structure, verification protocols, and model capabilities rather than applying universally.The conclusion summarizes the controlled comparisons and public-benchmark findings without treating MAS as uniformly superior.
  • Conclusion: MAS-Orchestra achieves consistent improvements on diverse public benchmarks and lies on the performance–cost Pareto frontier.The conclusion connects benchmark gains with favorable accuracy-cost positioning.
  • Benchmark examples: The benchmark materials include independent problems that can be solved in parallel, illustrating the Parallel axis.The supplied examples explicitly state that each problem is independent and can be solved in parallel.

A.2. Public Benchmarks

The public-benchmark appendix reports dataset statistics and notes that training stops after MAS-Orchestra converges. Consequently, the full reported training dataset may not be consumed.

  • Dataset statistics: Table A.2 reports training and test data statistics across datasets.The table is identified as the source for dataset-level training and test counts.
  • Training coverage: Training stops once MAS-Orchestra converges, so the limited training steps may not consume the entire reported dataset.This is an explicit training-process limitation attached to the dataset statistics.

B. Training Setup

The training setup specifies key hyperparameters for training and reinforcement learning, while benchmark correctness is evaluated with task-specific procedures.

  • Math correctness is evaluated using string matching via Hugging Face’s Math-Verify.
  • Other benchmarks use Llama-3.3-70B-Instruct as an LLM-as-a-judge to determine correctness.

C.1. MAS vs. SAS for Robustness Axis

The robustness comparison evaluates SAS and MAS on the same problem instance using Qwen-7b as orchestrator and GPT-120b as sub-agent or agent.

  • Figure C.1 reports Avg@8 for the Robustness setting with GPT-120b (low) as the sub-agent.SAS performance is too low to be visible in the figure.
  • Robustness comparisons use Qwen-7b as the orchestrator and GPT-120b as the sub-agent or SAS agent.

C.2. Robustness and Adversarial-aware Training.

Robustness performance depends on adversarial-aware training, while the orchestrator adapts its structure to task complexity without necessarily improving over SAS.

  • Explicit inclusion of adversarial data is necessary to improve robustness performance.Without adversarial examples, robustness remains nearly zero and comparable to SAS performance.
  • Combined training receives twice as many training steps as separate training because its dataset is naturally larger.The experiments are not intended to fully converge, and additional combined-training steps may yield further gains.
  • Structural alignment with task structure does not necessarily translate into performance gains over SAS.
  • The orchestrator generates more sub-agents as task complexity increases.This indicates adaptation to increasing complexity, while the benchmark covers axis values from 2 to 12.
  • Within each axis, orchestration patterns remain similar across values despite increasing sub-agent counts.The observed stability suggests scaling without changing the underlying orchestration structure.

C.3.2. GIVEN VALUE, OBSERVATIONS ACROSS AXES.

Across axis settings, MAS-Orchestra adapts the number, arrangement, and timing of sub-agents to task structure, producing sequential, parallel, or structurally diverse systems.

  • Depth and Breadth: Depth and Breadth typically converge to sequential structures with lengths of 1 to 5 and widths up to 2.Generated systems commonly follow Parse → Solve → Verify → Final Answer.
  • Parallel: Parallel converges to parallel MAS structures whose width typically matches the specified Parallel value.This mirrors independent sub-task structure.
  • Figure C.2 reports agent statistics for axis values 2, 4, and 6 from left to right.
  • Horizon: Horizon produces structural diversity because tasks differ in when intermediate results should be synthesized.The orchestrator varies sub-agent number and ordering across horizon settings.
  • Robustness: Robustness tends toward parallel MAS structures that distribute sub-tasks and help isolate or mitigate adversarial or incorrect information.Additional verification or moderation components may support this isolation.

C.4. Separate vs. combined training.

Combined training generalizes effectively across MASBENCH axes, while performance declines predictably as task complexity increases. Additional analyses compare orchestrator architectures, context limits, and generated agent structures.

  • Separate vs. combined training: Combined-axis training produces highly similar performance across individual axes, indicating effective generalization when the orchestrator is trained on data from all five axes.The comparison uses combined data from Depth, Horizon, Breadth, Parallel, and Robustness.
  • Separate vs. combined training: Adversarial training data substantially improves Robustness performance compared with training without adversarial samples.This comparison is reported for models trained on combined data, with and without adversarial examples.
  • Orchestrator architecture: Even at matched model size, the DS-7b RLM consistently underperforms Qwen-7b across MASBENCH axes, suggesting that the current RLM is poorly suited as an orchestrator.The comparison controls for the 7B model size and uses a similar number of training steps.
  • Complexity generalization: As task complexity increases, performance decreases both in-distribution and out-of-distribution, indicating consistent complexity-generalization behavior.Training and evaluation extend to higher held-out axis values for Depth, Breadth, Parallel, Horizon, and Robustness.
  • Generated agent structures: RLM-based orchestrators tend to generate simpler MAS, sometimes using one sub-agent, whereas instruction-tuned LLM orchestrators more often generate three-agent systems.These patterns are reported for the agent statistics in Figure C.6.
  • Experimental setup: The section also documents the compared sub-agent implementations and baseline execution setup used in the experiments.The appendix materials summarize sub-agents, their orchestration decisions, and the use of official baseline code.

D.2. Additional Observations on Overall Results

Additional observations show that MAS-Orchestra compares favorably with inference- and training-time baselines while adapting its generated systems to task structure. Its efficiency advantage includes lower calls, token use, cost, and wall time, although some baseline behaviors affect comparisons.

  • Inference-time orchestration baselines: AFlow is the strongest inference-time baseline but still underperforms MAS-Orchestra, while inference-time methods such as AFlow and MAS-Zero are computationally costly.MAS-Zero requires an orchestrator larger than 32B parameters to produce meaningful results.
  • Training-based orchestration baselines: MAS-GPT often falls back to Self-Consistency when it fails to generate a valid MAS, and this fallback substantially contributes to its gains over ToolOrchestra.Training-based orchestration baselines perform worse than many inference-time methods in the reported observations.
  • Training-based orchestration baselines: ToolOrchestra often refuses to call its sub-agent, whereas MAS-Orchestra directly generates MAS and consistently utilizes sub-agents under the DoM formulation.Refusal degrades performance when the sub-agent is stronger than the orchestrator.
  • Generated MAS across benchmarks: HotpotQA commonly receives one SearchAgent plus a final CoTAgent, suggesting that one search step is sufficient for these relatively simple questions.The observation is contrasted with the possibility of parallel searches.
  • Generated MAS across benchmarks: MAS-Orchestra dynamically adapts MAS designs to task substructure and delegates execution to effective agent configurations, although generated designs may not perfectly match the underlying structure.The paper attributes imperfect matching partly to diverse underlying LLM capabilities.
  • Cost comparison: More than 10× efficiency is achieved in LLM calls, token count, and overall cost, with substantially faster wall time than the compared systems.Wall-time results also reflect implementation choices such as asynchronous sub-agent execution; the detailed statistics are reported for AIME24 and GPQA.
Loading 2601.14652v5…