MasRouter: Learning to Route LLMs for Multi-Agent Systems

Yanwei Yue; Guibin Zhang; Boyang Liu; Guancheng Wan; Kun Wang; Dawei Cheng; Yiyan Qi

Source-linked AI summary

MasRouter: Learning to Route LLMs for Multi-Agent Systems

Yanwei Yue, Guibin Zhang, Boyang Liu, Guancheng Wan, Kun Wang, Dawei Cheng, Yiyan Qi

arXiv:2502.11133v1Feb 16, 2025cs.LGcs.MA

TL;DR

MAS routing must select not only LLMs but also collaboration modes and agent roles, because existing routing methods focus on single-agent scenarios. MasRouter unifies these decisions through a cascaded controller network that progressively constructs a MAS. Across experiments, it improves performance, reduces overhead, and integrates with mainstream MAS frameworks.

Problem
Existing LLM routing methods customize model selection for queries but focus on single-agent scenarios, while MAS additionally requires collaboration-mode and role decisions.
Method
MasRouter uses a cascaded controller network for collaboration-mode determination, progressive role allocation, and agent-level LLM routing.
Results
MasRouter surpasses RouterDC by 3.51% on average, reduces HumanEval overhead from 0.363$ to 0.185$, and achieves 17% ∼28% fewer cost when integrated with mainstream MAS.
Takeaways & Limitations
MasRouter provides an inductive, plug-and-play routing approach that balances MAS effectiveness and efficiency while generalizing to unseen LLM backbones and collaboration modes.
Takeaways & Limitations
Existing routing methodologies mainly focus on single-agent scenarios, and their unawareness of inter-agent topology constrains applicability to complex tasks and scalability in larger systems.

Abstract

from arXiv · show

Multi-agent systems (MAS) powered by Large Language Models (LLMs) have been demonstrated to push the boundaries of LLM capabilities, yet they often incur significant costs and face challenges in dynamic LLM selection. Current LLM routing methods effectively reduce overhead in single-agent scenarios by customizing LLM selection for each query, but they overlook the critical decisions regarding collaboration modes and agent roles in MAS. In response to this challenge, we first introduce the problem of Multi-Agent System Routing (MASR), which integrates all components of MAS into a unified routing framework. Toward this goal, we propose MasRouter, the first high-performing, cost-effective, and inductive MASR solution. MasRouter employs collaboration mode determination, role allocation, and LLM routing through a cascaded controller network, progressively constructing a MAS that balances effectiveness and efficiency. Extensive experiments demonstrate that MasRouter is (1) high-performing, achieving a $1.8\%\sim8.2\%$ improvement over the state-of-the-art method on MBPP; (2) economical, reducing overhead by up to $52.07\%$ compared to SOTA methods on HumanEval; and (3) plug-and-play, seamlessly integrating with mainstream MAS frameworks, reducing overhead by $17.21\%\sim28.17\%$ via customized routing. The code is available at https://github.com/yanweiyue/masrouter.

1 Introduction

MASR addresses the gap between single-agent routing and the broader decisions required to construct effective, efficient multi-agent systems. MasRouter unifies collaboration-mode selection, role allocation, and agent-level LLM routing in a cascaded framework, with experiments showing performance and cost benefits.

Motivation: Existing LLM routing methods select models per query but primarily support single-agent scenarios, leaving MAS collaboration decisions unresolved.MAS also requires suitable collaboration modes and a division of labor across agents.
Problem Definition: MASR assigns each query an appropriate collaboration mode, efficient agent roles, and an LLM for each agent.This definition integrates the main structural and model-selection decisions of a multi-agent system.
Method: MasRouter integrates a collaboration-mode determiner, role allocator, and LLM router into a unified cascaded routing framework.The components use a variational latent variable model, a structured probabilistic cascade, and multinomial LLM recommendation, respectively.
Method: MasRouter progressively constructs a high-performing, resource-efficient MAS and can integrate with mainstream multi-agent systems.The approach is designed to balance effectiveness and efficiency while reducing inference cost.
Experimental Validation: 3.51% average improvement over RouterDC, overhead reduction from 0.363$ to 0.185$ on HumanEval, and 17% ∼28% fewer cost demonstrate the reported benefits.These results span performance, economy, and plug-and-play integration across five benchmarks.

2 Related Work

Related work distinguishes fixed multi-agent architectures from dynamic networks and reviews routing methods developed for single LLMs. Existing routing methods mainly overlook inter-agent topology, limiting their applicability and scalability for complex multi-agent tasks.

Multi-Agent System: Fixed agentic networks use manually crafted architectures spanning debate, collaboration, and competitive paradigms.These architectures are pre-established rather than configured dynamically.
Multi-Agent System: Dynamic agentic networks configure structure and communication strategies from real-time feedback and observations.Prior work uses MCTS and evolutionary algorithms to discover effective agent configurations.
Single LLM Routing: Single-LLM routing methods balance computational cost and model performance by customizing model selection for each query.Early approaches include binary routing, sequential pipelines, and preference-driven routing.
Single LLM Routing: Existing routing methods focus on single-agent scenarios and remain unaware of inter-agent topology.This constrains their applicability to more complex tasks and limits scalability in larger systems.

3 Formalization

This section formalizes MASR as query-conditioned selection of a multi-agent system from available LLM backbones, agent roles, and collaboration modes. Its objective jointly balances MAS performance and cost.

3.1 Notation Establishment: The MAS search space comprises available LLM backbones, predefined agent roles, and collaboration modes such as Chain, Tree, and LLM-Debate.
3.1 Notation Establishment: A multi-agent system combines several LLM-powered agents with distinct identities that work together collaboratively.
3.1 Notation Establishment: Each MAS instance selects its LLM backbones, roles, and collaboration mode, with k denoting the number of LLM agents.
3.2 Definition of MASR: MASR maps the search space to a MAS tailored to query Q through a mapping function f.
3.2 Definition of MASR: The probability π(S) represents the likelihood of selecting multi-agent system S conditioned on Q.
3.2 Definition of MASR: Given benchmark queries and oracle answers, the ideal MASR optimizes a strategy that jointly balances performance and cost.
3.2 Definition of MASR: The objective combines MAS utility, which measures performance, with system cost, which quantifies expected LLM calls, API cost, and token cost, weighted by λ.

4 MasRouter

MasRouter constructs a task-adaptive multi-agent system by cascading collaboration-mode determination, role allocation, and agent-level LLM routing. It jointly optimizes response quality and multi-agent cost using performance and cost feedback.

Overall Framework: MasRouter samples a customized MAS by sequentially selecting collaboration patterns, agent roles, and LLM backbones for each query.The sampled system is executed before the selection modules are jointly optimized from performance/cost feedback.
Overall Framework: The controller network maps each query to a distribution over candidate MAS configurations and uses solution likelihood to guide system selection.Fθ computes the distribution of S conditioned on Q, while O(·|·) denotes the likelihood of obtaining an answer by executing S.
4.1 Collaboration Mode Determination: A variational latent model determines collaboration modes by encoding query semantics and decoding probabilities of collaborative patterns.The latent representation uses a text encoder and a refinement function to embed relationships between queries and candidate collaboration patterns.
4.1 Collaboration Mode Determination: The selected collaboration mode and query embedding determine the agent count through a learnable complexity mapping capped by a maximum-agent hyperparameter.The number of agents is computed as k = ⌈δ(H) · γ⌉.
4.2 Agent Role Allocation: The role allocator progressively assigns roles using a structured probabilistic cascade conditioned on the query, collaboration mode, and previously assigned roles.This ordering models role interdependencies, such as programming preceding testing and debugging.
4.3 Agent LLM Routing: The LLM router assigns backbones to agents as a multinomial distribution conditioned on the query, collaboration mode, and selected roles.LLM compatibility is computed from embeddings of the query, collaborative patterns, selected roles, and candidate LLMs; Gamma-function approximation smooths gradients through the agent-count calculation.
4.4 Optimization: MasRouter optimizes correct-solution probability while minimizing token expenditure through a cost-weighted objective and policy-gradient optimization.C(·) evaluates multi-agent-system cost, and λ controls the effectiveness-efficiency trade-off.

5 Experiments

MasRouter is evaluated across five diverse benchmarks against single-agent, fixed-topology, and routing baselines, showing strong performance, cost efficiency, integration, and inductive behavior. Ablations and sensitivity analyses examine which modules and parameters drive the performance-cost trade-off.

Performance: MasRouter achieves the best performance across five datasets, outperforming RouterDC by 3.51% on average.On MBPP, it exceeds AgentPrune and AFlow by 8.20% and 1.80% at pass@1, respectively.
Performance & Cost: MasRouter reaches the MBPP cost-effectiveness Pareto front, improving performance over AFlow by 1.8% ∼8.0% while reducing inference overhead by 40.22% ∼43.78%.
Training Resources: MasRouter saves training resources on MMLU by 69.57% versus GPTSwarm and 83.51% versus AFlow.The savings arise because it avoids exhaustive traversal and validation of every agentic structure.
Plug-in Integration: MasRouter improves MAD by 1.55% at pass@1 on HumanEval while reducing integrated cost by 17.21% ∼28.17%.When integrated with MAD on MMLU, it saves 6.17 ∼7.63$ in inference cost.
Inductive Ability: After adding Deepseek-v3, MasRouter selects it 12.17% of the time on MMLU and 27.19% on MATH.Accuracy rises on MMLU from 84.25% to 85.40% and on HumanEval from 90.62% to 91.41%.
Ablation & Sensitivity: Removing cost evaluation raises overhead by 54.09% and 41.62% without significantly changing performance.Increasing the agent limit from 2 to 6 improves performance from 88.50% to 90.62%, while increasing it to 10 incurs 1.5× per-query cost for marginal gains.

6 Conclusion

The paper introduces MASR for query-level allocation of collaboration patterns, agent roles, and LLMs, and presents MasRouter as a customized routing solution. MasRouter progressively builds adaptive roles and selects task-domain-proficient LLMs to balance effectiveness and efficiency.

MASR allocates collaboration patterns, agent roles, and LLMs for each query to construct a customized MAS.
MasRouter progressively builds mutually adaptive roles and selects an LLM proficient in the task domain.The resulting system balances effectiveness and efficiency.
MasRouter is presented as a high-performing, economical, and inductive MASR solution.

A Notations

The notation appendix provides a reference table for commonly used symbols and records the overall MasRouter workflow in Algorithm 1.

Table 4 collects the notations commonly used throughout the manuscript.
Algorithm 1 summarizes the overall workflow of MasRouter.

C Case Study

The case study visualizes the customized MAS structures generated by MasRouter for queries with varying difficulty across five benchmarks.

Tables 5 to 9 visualize MasRouter’s customized MAS designs for varying query difficulties on five benchmarks.

D Detailed Cost-Performance Data

This section compares baseline overhead and performance on MBPP and HumanEval, including a HumanEval Pareto-front visualization.

The evaluation reports baseline inference overhead and performance on the MBPP dataset.
The evaluation also reports baseline inference overhead and performance on the HumanEval dataset.
Figure 6 presents the HumanEval results as a scatter plot with a Pareto front.

D.2 Training Cost

This section compares the training overhead of MasRouter with state-of-the-art methods on MATH and MMLU.

The comparison focuses on training overhead rather than inference overhead.
MasRouter is compared against state-of-the-art methods that require training.
The training-cost comparison covers the MATH and MMLU datasets.

E The Module Profile

The module profiles describe the models and roles used by MasRouter, while its workflow progressively determines collaboration mode, allocates roles, and routes LLMs. The profile also includes benchmark accuracies, model costs, and implementation-specific role configurations.

Role profiles: MasRouter constructs role pools using GraphRouter and Macnet, selecting three distinct roles for each task.
Workflow: The workflow samples a latent representation, predicts collaboration-mode probabilities, and dynamically determines the agent count.
Workflow: After determining the collaboration mode, MasRouter selects agent roles through cascaded inference.
Workflow: MasRouter aggregates query, task, and role context before assigning LLMs through multinomial sampling.
Optimization: The optimization objective combines task utility with a cost penalty and updates controller parameters using policy gradients.
Module profiles: The profiles record model costs, benchmark accuracies, and role-specific prompts, aggregation methods, output formats, and post-processing settings.