Source-linked AI summary
One Model, Many Minds: Unlocking Multi-Agent Synergy in a Single Agent via Mixture of Roles
Zhichen Zeng, Huiyuan Chen, Jingru Cheng, Juan Zha, Ming Liu, Ying Chen, Xiyuan Yang, Chaosheng Dong, Haiyang Zhang, Hanghang Tong
TL;DR
Single-agent methods are efficient but fixed, while MAS provides dynamic perspectives through costly multi-turn communication. MoRe learns steering roles and composes them per query for single-turn inference, matching MAS performance while improving over single-agent baselines and reducing token cost.
Problem
Single-agent specialization is fixed and non-adaptive, whereas MAS requires multi-turn interactions that increase context length and inference cost.
Method
MoRe learns a diversified steering-vector codebook and uses a query-aware router to compose multiple roles into one vector while keeping the backbone LLM frozen.
Results
MoRe outperforms single-agent specialization baselines by 2.2% on average, achieves performance competitive with MAS, and reduces token cost by 20×.
Takeaways & Limitations
MoRe combines the efficiency of single-agent steering with the adaptivity of multi-agent collaboration in single-turn inference.
Takeaways & Limitations
MoRe requires access to backbone hidden states, is not directly applicable to black-box API models, and keeps one query-level persona fixed throughout generation.
Abstract
from arXiv · showhide
Specializing Large Language Models (LLMs) toward distinct abilities underpins successes ranging from personalized assistants to multi-agent systems (MAS). Single-agent paradigms rely on pre-defined personas or steering vectors to induce specialization, yet they impose a single fixed specialization that fails to adapt to diverse queries. Conversely, MAS achieves dynamic multi-perspective problem solving by orchestrating agents with distinct text-based roles, but fusing these specializations requires multi-turn interactions that inflate context length and inference cost. To address these limitations, we propose Mixture of Roles (MoRe), which adaptively composes multiple specializations into a single steering vector for single-turn inference. Specifically, MoRe learns a diversified codeboox of steering vectors, each of which encodes a latent role. A query-aware router dynamically fuses the codebook into a steering vector that encompasses multiple roles. By steering the backbone LLM with the composed vector, MoRe enables multi-perspective specialization in a single-agent, single-turn inference process. The proposed MoRe can be efficiently trained via a three-stage SFT curriculum and GRPO post-training, while the backbone LLM remains frozen. Experiments across reasoning and personality benchmarks show that MoRe outperforms single-agent baselines by 2.2% on average, and achieves performance on par with MAS while reducing token cost by 20x.
1 Introduction
Single-agent specialization is efficient but static, while MAS supports dynamic multi-perspective reasoning at the cost of multi-turn computation. MoRe addresses this trade-off by composing query-specific roles in activation space during single-turn inference.
- Single-agent role-playing and activation steering use predefined specializations that impose a static perspective and limit query-specific adaptivity.
- MAS integrates distinct roles through multi-turn interactions, but incurs heavy computation and long contexts.
- The paper asks whether one agent can achieve dynamic multi-perspective specialization within single-turn inference.
- MoRe learns a steering-vector codebook and uses a lightweight router to compose query-specific roles into one steering vector.
- 2.2% average improvement over single-agent specialization baselines accompanies performance competitive with MAS and 20× lower cost.
2 Related Works
Prior specialization methods apply fixed roles, whereas MAS fuses perspectives through expensive text communication. MoRe retains multi-perspective specialization while moving role fusion into activation space with query-aware routing.
- Textual prompting and activation steering specialize LLMs toward targeted behaviors, expertise, or cognitive modes.
- Existing specialization methods use predefined roles or apply one fixed role uniformly across queries.
- MAS frameworks orchestrate multiple LLM agents with distinct prompt-defined roles and interaction structures.
- MAS gains come from fusing specialized perspectives through multi-turn text communication, whose inference cost grows with agents and rounds.
- MoRe fuses steering directions in activation space during single-turn inference using a query-aware router.
3 Methodology
MoRe replaces static single-role steering with a query-specific composition of candidate role vectors, enabling multi-perspective specialization in a single frozen-LLM inference process. Its codebook, router, curriculum SFT, and GRPO training operate without updating the backbone.
- Composer Design: MoRe maps each query to a dynamic steering vector by combining candidate role vectors from a learnable codebook.A lightweight router scores candidates and fuses selected vectors into the query-specific composition.
- Specialization Codebook: Each codebook candidate encodes a specialization direction and is warm-started from contrastive activation directions before end-to-end refinement.Remaining entries are initialized as convex mixtures of the warm-started candidates.
- Query-Aware Router: Top-K routing composes only a few candidates per query, assigning zero weight to unselected experts instead of averaging the full codebook.The composed vector is formed from the selected candidates using softmax routing weights.
- Curriculum Supervised Fine-Tuning: The three-stage SFT curriculum separately warms up the codebook, trains the router with load balancing, and then jointly optimizes both components.Stage 1 trains individual candidates, stage 2 trains routing with LCE + λlbLlb, and stage 3 adapts both to query-dependent compositions.
- GRPO Post-Training: GRPO shifts the action space from tokens to candidate role subsets, making rewards attributable to the router while keeping the backbone frozen.Rollouts sample steering-vector sets, evaluate frozen-LLM responses, and update the router using group-normalized advantages.
4 Experiments
Experiments show that MoRe combines strong reasoning and personality performance with substantially lower token use than multi-agent systems. Analyses further indicate that its codebook, query-aware router, and steering position each contribute to performance and adaptability.
- Reasoning Results: MoRe ranks Top-2 across two backbone LLMs while consuming significantly fewer tokens.The evaluation compares Llama-3.1-8B-Instruct and Qwen3-8B against single-agent and MAS baselines.
- Reasoning Results: MoRe outperforms the best single-agent baseline ActAdd by 0.96 on Llama and 0.97 on Qwen.The improvement is more pronounced on tasks requiring longer reasoning chains.
- Reasoning Results: MoRe matches MAS performance while using approximately 1/20 as many tokens on average.MoRe requires a single model call, unlike MAS approaches using repeated calls, discussions, and communication.
- Personality Results: MoRe achieves state-of-the-art PersonalityBench performance across backbones and personalities, consistently ranking first.Against the best competitor, average outperformance is 0.23 on Llama and 0.61 on Qwen.
- Learned Components: Learned codebook candidates form distinct role-specific representation clusters and mostly low-similarity steering directions.The candidates become diverse without an explicit diversity regularizer.
- Learned Components: The router distributes weights across experts, adapts routing preferences by dataset, and shares patterns across related tasks.Math benchmarks emphasize the mathematician-initialized expert, while MedQA combines teacher, skeptic, and doctor experts.
- Ablation Studies: MoRe outperforms ablated variants, with the learned codebook contributing more than the router alone.The codebook supplies diverse directions, while the router selects and combines them per input.
- Ablation Studies: Steering the final prompt token is more effective than steering all prompt, first prompt, or response tokens.The final prompt token summarizes preceding context and bridges steering to the subsequent reasoning trajectory.
5 Conclusion
MoRe enables dynamic multi-perspective specialization in single-agent, single-turn inference by composing a diversified codebook of specializations with a router, consistently outperforming single-agent methods and reducing token cost versus MAS.
- MoRe learns a diversified specialization codebook and uses a router to compose query-specific multi-perspective steering vectors.
- MoRe combines single-agent steering efficiency with multi-agent collaboration’s adaptivity.
- MoRe consistently outperforms existing single-agent specialization methods across reasoning and personality benchmarks.
- MoRe reduces token cost by 20× compared to MAS.
Limitations
The paper reports that role effects are task-dependent and that MoRe benefits from query-dependent composition, while its scope is limited by access, interpretability, and routing granularity constraints.
- MoRe requires access to backbone hidden states and is not directly applicable to black-box API models.
- Learned candidate vectors are latent despite prompt-based initialization, limiting their interpretability.
- MoRe fixes a query-level persona throughout generation, while finer-grained token- or reasoning-step routing may add adaptivity and computational overhead.
- Role effectiveness is highly task-dependent, with the strongest predefined role varying across tasks, backbones, and steering methods.
- MoRe generally approaches or surpasses the best predefined-role performance by composing complementary candidate steering vectors according to each query.
A.2 Balancing Performance–Efficiency
MoRe occupies a favorable performance–efficiency region across models and datasets, approaching MAS performance at far lower token cost while preserving the benefits of query-dependent composition.
- MoRe consistently occupies the upper-left performance–token region across models and datasets, indicating a favorable accuracy–cost balance.
- MoRe achieves the best performance on MATH with Llama while using substantially fewer tokens than all MAS baselines.
- Compared with single-agent methods, MoRe uses a similar number of tokens while generally achieving stronger performance, especially on MATH and GSM8K.
- MAS baselines can achieve higher accuracy on some tasks, but typically require hundreds or thousands of completion tokens per sample.
- MoRe approaches MAS performance with one to two orders of magnitude lower token cost.
- Steering effects persist and become candidate-specific, with angular change increasing from approximately 12° at the steering layer to over 40° at the final layer.
B Experiment Details
The experiments define role-based and single-agent baselines, compare them with multi-agent systems, and train MoRe’s composer using GRPO with soft, task-aligned rewards.
- The role design covers seven persons: mathematician, software engineer, data scientist, logician, teacher, skeptic, and medical doctor.
- MoRe’s composer is post-trained with GRPO using group-relative advantages while the backbone LLM remains outside the router’s optimization signal.
- The reward design uses exact-match, format, semantic-similarity, and partial-credit signals for multiple-choice, free-form QA, and long-chain reasoning tasks.
- The experiments evaluate Vanilla, Text, CAA, ActAdd, Prompt FT, Billy, NPTI, and IRIS as single-agent baselines.
- The experiments evaluate LLM Debate, AutoGen, AgentVerse, and DyLAN as multi-agent baselines.
B.4 Datasets
The evaluation uses established reasoning and personality benchmarks, alongside publicly available models, datasets, and baselines. The paper also discusses dynamic specialization and risks beyond the evaluated settings.
- Reasoning benchmarks: Five reasoning benchmarks cover general knowledge, mathematics, and professional medical expertise.The benchmarks include MMLU, TriviaQA, MATH, GSM8K, and MedQA.
- Personality evaluation: PersonalityBench evaluates specified personality traits using the Big Five framework and opposing trait aspects.It covers Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.
- Scope and risks: Learned steering vectors may amplify undesirable biases or unintended behavioral changes on inputs outside the training distribution.The experiments focus on established reasoning and personality benchmarks.
- Data and artifacts: The work builds on publicly available datasets, pretrained language models, and existing single-agent and multi-agent baselines without collecting human-participant data.The authors state that all external artifacts are credited to their original publications and repositories.
E.3 Data Contains Personally Identifying Info Or Offensive Content
The described materials do not intentionally introduce personally identifying information. PersonalityBench uses synthetic situational questions that may involve sensitive interpersonal or emotional scenarios.
- Personally identifying information: The work does not collect or introduce new personally identifying information.The reasoning benchmarks mainly contain general knowledge, mathematical, and professional examination questions.
- Sensitive content: PersonalityBench contains synthetically constructed situational questions based on the Big Five personality framework.Questions may describe interpersonal conflicts, emotional states, or other sensitive situations needed to evaluate personality expression.
- Privacy boundary: The authors report that PersonalityBench questions do not contain private information about identifiable individuals, to their knowledge.The statement applies to the synthetic situational questions used for personality evaluation.
- Documentation: Appendix B documents the evaluated models, datasets, baselines, evaluation protocols, and hardware to support reproducibility.Dataset statistics are summarized separately in Appendix F.
F Statistics For Data
The statistics cover five reasoning datasets and PersonalityBench across knowledge, mathematics, medical question answering, and personality expression. Dataset sizes and evaluation metrics vary by benchmark.
- MMLU: MMLU contains approximately 16,000 multiple-choice questions across 57 subjects and uses exact match.Its domains span humanities, social sciences, natural sciences, and professional subjects.
- TriviaQA: TriviaQA contains approximately 95,000 question–answer pairs and uses LLM-as-a-judge accuracy.It covers open-domain factoid question answering across diverse topics.
- Mathematics benchmarks: MATH has 7,500 training and 5,000 test problems, while GSM8K has 7,473 training and 1,319 test problems.Both datasets evaluate mathematical reasoning, with MATH emphasizing competition-level problems and GSM8K grade-school word problems.
- MedQA: MedQA contains 12,723 multiple-choice medical questions and uses exact match.The English subset primarily contains questions based on the United States Medical Licensing Examination.
- PersonalityBench: PersonalityBench contains 180,000 training instances and scores responses from 1 to 5 using LLM-as-a-judge.Its evaluation set has approximately 90 situational questions per personality trait.
G Computational Experiments
The computational evaluation uses frozen 8-billion-parameter backbones and optimizes only a small steering codebook and query-aware router. Experiments report benchmark accuracy, ranks, personality scores, and hyperparameter sensitivity.
- Backbone models: Two approximately 8-billion-parameter backbones are evaluated: Llama-3.1-8B-Instruct and Qwen3-8B.Both backbone models remain frozen throughout training.
- Trainable components: Only the candidate steering-vector codebook and lightweight query-aware composer are optimized.The codebook has 0.04M parameters and the router has 1.32M parameters, together constituting 0.017% of the backbone model.
- Hardware and settings: All computational experiments run on 8 NVIDIA A100 GPUs with 40GB memory.The experimental settings are described in Section 4.1.
- Hyperparameters: The default configuration uses steering strength α = 0.2, codebook size N = 10, router Top-K K = 3, and load-balancing weight λlb = 0.1.Sensitivity is studied over alternative values for all four hyperparameters.
- Metrics: Reasoning evaluation reports per-dataset accuracy, average accuracy, and average rank, while PersonalityBench reports trait scores and cross-trait standard deviation.Personality scores are produced by an LLM judge.