When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail

Xiaoxiao Li

Source-linked AI summary

When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail

Xiaoxiao Li

arXiv:2601.04748v2Jan 8, 2026cs.AIcs.MA

TL;DR

The paper asks whether single-agent skill libraries can preserve multi-agent modularity while reducing communication costs, and how skill selection scales as libraries grow. It formalizes compilation from multi-agent systems to skills and studies selection capacity, finding efficient small-library operation but sharp degradation beyond a threshold, especially among semantically confusable skills.

Problem
The paper addresses how to retain modular reasoning benefits while reducing multi-agent coordination costs and how skill-selection ability changes as skill libraries expand.
Method
The paper compiles multi-agent capabilities into single-agent skills and analyzes selection separately from execution across controlled skill libraries using a cognitive-science capacity perspective.
Results
54% lower token consumption and 50% lower latency were achieved on average with similar accuracy to multi-agent counterparts on GSM8K, HumanEval, and HotpotQA; selection accuracy later drops sharply beyond a critical threshold.
Takeaways & Limitations
Skill-based agents can provide efficient modularity when libraries remain small, while semantic confusability and capacity limits require structured hierarchical routing at scale.
Takeaways & Limitations
The evaluation uses synthetic skill libraries, measures selection rather than end-to-end task performance, covers two OpenAI models, and uses relatively simple hierarchy designs.

Abstract

from arXiv · show

Multi-agent AI systems have proven effective for complex reasoning. These systems are compounded by specialized agents, which collaborate through explicit communication, but incur substantial computational overhead. A natural question arises: can we achieve similar modularity benefits with a single agent that selects from a library of skills? We explore this question by viewing skills as internalized agent behaviors. From this perspective, a multi-agent system can be compiled into an equivalent single-agent system, trading inter-agent communication for skill selection. Our preliminary experiments suggest this approach can substantially reduce token usage and latency while maintaining competitive accuracy on reasoning benchmarks. However, this efficiency raises a deeper question that has received little attention: how does skill selection scale as libraries grow? Drawing on principles from cognitive science, we propose that LLM skill selection exhibits bounded capacity analogous to human decision-making. We investigate the scaling behavior of skill selection and observe a striking pattern. Rather than degrading gradually, selection accuracy remains stable up to a critical library size, then drops sharply, indicating a phase transition reminiscent of capacity limits in human cognition. Furthermore, we find evidence that semantic confusability among similar skills, rather than library size alone, plays a central role in this degradation. This perspective suggests that hierarchical organization, which has long helped humans manage complex choices, may similarly benefit AI systems. Our initial results with hierarchical routing support this hypothesis. This work opens new questions about the fundamental limits of semantic-based skill selection in LLMs and offers a cognitive-grounded framework and practical guidelines for designing scalable skill-based agents.

1 Introduction

This section presents skills as an efficient way to internalize specialized agent behaviors, while showing that skill selection has a sharp capacity limit as libraries grow. The central challenge is preserving modular reasoning benefits without multi-agent communication overhead.

Motivation: Multi-agent systems improve reasoning through specialized agents that communicate explicitly, but repeated coordination incurs substantial computational overhead.The motivating question is whether modular reasoning can be retained at lower cost.
Skills as Internalized Agents: Skills encode specialized behaviors as selectable actions, enabling a single agent to replace distributed agent roles while eliminating inter-agent communication overhead.The compilation view treats each agent’s behavior as a skill within a unified context.
Efficiency: 54% lower token consumption and 50% lower latency were achieved on average while maintaining similar accuracy to multi-agent counterparts across GSM8K, HumanEval, and HotpotQA.These results apply when skill libraries remain small.
Scaling Limits: Skill selection accuracy remains high below a critical library-size threshold, then drops sharply rather than degrading gradually.The study evaluates controlled libraries ranging from 5 to 200 skills and describes this behavior as a phase transition.
Scaling Limits: Selection degradation is driven by semantic confusability among skills, not library size alone.The result connects LLM selection behavior to similarity-based interference in human memory retrieval.
Hierarchical Routing: The work identifies hierarchical routing as a way to mitigate scaling limits by organizing skills into structured categories.This provides cognitive-grounded design guidance for scalable skill-based agents.

2 Problem Formulation: Multi-Agent Systems to Single-Agent with Skills

This section formalizes multi-agent systems and single-agent systems with skills, then defines compilation as transforming specialized agent capabilities into selectable skills while internalizing coordination.

2.1 Multi-Agent Systems: An agent is specified by a role description defining its identity and expertise, together with a behavioral policy governing reasoning and response generation.
2.1 Multi-Agent Systems: A multi-agent system consists of specialized agents, a communication graph, and a coordination protocol that initializes, routes, and terminates execution.Its execution appends agent outputs to a shared history and routes subsequent actions through the protocol.
2.2 Single-Agent with Skills (SAS): A single-agent system with skills replaces agent routing with skill selection over semantic descriptors, then executes the selected skill within the evolving task history.The selector operates over descriptors rather than complete procedural specifications.
2.2 Single-Agent with Skills (SAS): Each skill combines a semantic descriptor, an execution policy, and an execution backend that may be internal model reasoning or an external tool.Separating selection from execution isolates descriptor-based selection complexity from execution variability.
2.3 The Compilation Problem: Compilation maps a multi-agent system to a single-agent system by distilling each agent’s specialized function into one or more skills.The compiler is described as capability decomposition, backend assignment, and topology internalization.
2.3.1 Phase 1: Capability Decomposition: Capability decomposition extracts discrete functional units from each agent’s role description independently of implementation details.These capabilities can represent tasks such as code review or weather-data retrieval.
2.3.2 Phase 2: Backend Assignment: Backend assignment chooses external execution when a capability requires grounding and internal execution when it is purely cognitive, while generating a semantic descriptor for retrieval.
2.3.3 Phase 3: Topology Internalization: Topology internalization converts explicit communication edges into implicit input/output constraints and handover instructions within skill definitions.The resulting skill library is optimized to reduce cognitive load from retrieval complexity and generated-skill context consumption while preserving functional equivalence.

3 Experiments on MAS-to-SAS Compilation

The experiments characterize when MAS-to-SAS compilation is possible and test whether compiled single agents preserve performance while reducing computational cost. Across benchmark comparisons, compilation largely preserves accuracy while reducing tokens, latency, and API calls, but benefits are smaller for very long outputs near context limits.

3.1 Conditions for Compilability: Compilation requires serializable communication, shared history without private state, and a homogeneous backbone; it fails with true parallelism, private information, adversarial objectives, or heterogeneous capabilities.The scope is limited to systems satisfying these compilability conditions.
3.2 Experimental Setup: The experiments evaluate internally managed skills on three benchmarks using GPT-4o-mini for all agents, measuring accuracy, total tokens, latency, and API calls.The compiled SAS performs equivalent computation through structured skill invocation within a single API call.
3.2.2 Results: Accuracy stays within −2.0% to +4.0% of MAS across benchmarks, with a +0.7% average improvement after compilation.On HotpotQA, SAS outperforms MAS by 4.0%, plausibly because unified context improves information integration.
3.2.2 Results: 53.7% average token reduction comes from eliminating repeated context across agent calls, with the largest savings on HotpotQA (58.4%) and GSM8K (56.2%).HotpotQA benefits most because retrieved passages were repeatedly passed between agents.
3.2.2 Results: 49.5% average latency reduction follows from replacing 3–4 sequential API calls with one, while GSM8K improves by only 28.7% because reasoning computation dominates communication.Latency reductions are largest on HotpotQA (60.9%) and HumanEval (58.9%).
3.2.2 Results: API calls fall from 3–4 for MAS to exactly 1 for SAS, reducing per-request overhead and yielding a 75% decrease for HotpotQA.The single-call design combines modular skill behavior with monolithic execution.
3.2.2 Results: Compilation provides smaller benefits for tasks requiring very long outputs that approach context limits, a limitation not explored in depth here.This caveat applies even when the formal compilability conditions hold.

4 The Skill Scaling Hypothesis: A Cognitive Science Perspective

The paper models skill-selection scaling as a capacity-limited process shaped by both library size and semantic interference. It predicts a sharp accuracy transition beyond a capacity threshold and motivates hierarchical organization as a mitigation.

Hierarchical mitigation: The framework predicts that hierarchical processing can manage complexity by narrowing choices into smaller stages within reliable capacity limits.This draws on cognitive accounts of chunking, menu breadth, and stepwise option elimination.
Problem: The scaling problem asks how selection accuracy degrades as the skill library grows, while separating search-space expansion from semantic interference.Selection accuracy isolates choosing the correct skill from execution fidelity and external noise.
Scaling hypothesis: The hypothesis assigns degradation to an effective capacity threshold κ and semantic interference I(S) among skills.The model treats these as distinct contributors to selection difficulty.
Predictions: Below κ, accuracy should remain high; beyond κ, it should decline sharply and super-linearly rather than gradually.The predicted phase transition follows the regime change from |S| ≪ κ to |S| ≫ κ.
Predictions: Semantic similarity is predicted to create errors even at fixed library size, so highly confusable small libraries and large distinct libraries can fail for different reasons.The formulation treats capacity limits and similarity interference as partially independent failure modes.

5 Experiments for Scaling-law

Controlled experiments vary synthetic skill-library properties across two models to test capacity, confusability, policy complexity, and hierarchical routing. Results show sharp scaling degradation, strong confusability effects, no meaningful policy-complexity effect, and substantial gains from hierarchy at large library sizes.

H1: Non-linear Phase Transition: Above 90% accuracy at |S| ≤20 falls to approximately 20% at |S| = 200, with super-linear decay exponents γ of 1.72 and 1.56.The fitted capacity thresholds are κ = 91.8 for GPT-4o-mini and κ = 83.5 for GPT-4o.
H2: Confusability-Driven Errors: Adding one competitor per skill reduces accuracy by 7–30%, while two competitors cause 17–63% degradation.With no competitors, accuracy remains 100% at |S| = 20, compared with approximately 95% for mixed-similarity libraries.
H3: Instructional Saturation: Simple, medium, and complex execution policies produce largely overlapping accuracy curves and no meaningful policy-complexity effect.All three conditions still show the same sharp decline beyond |S| = 50.
Practical guidelines and limitations: The authors recommend monitoring library size, minimizing semantic overlap, using confusability-aware hierarchy, and writing distinctive descriptors.They also identify synthetic data, selection-only evaluation, limited model coverage, and simple hierarchy designs as limitations.

6 Conclusion and Future Directions

The paper frames skill-based modularity as an efficient alternative to multi-agent coordination, while showing that skill selection has capacity limits. It identifies hierarchical organization and broader evaluation as important directions for addressing these limits.

Skill-based systems offer a middle ground between monolithic prompting and expensive multi-agent coordination, but their efficiency depends on keeping skill libraries small.The paper links this constraint to inherent scaling limits in skill selection.
Selection accuracy remains stable below a critical threshold and then degrades sharply, with semantic confusability contributing to the decline.The observed pattern is described as non-linear and analogous to capacity limits in human decision-making.
Hierarchical organization may help mitigate capacity limits as skill libraries grow.The paper presents this as a hypothesis supported by preliminary experiments rather than a settled conclusion.
Future directions: Future work should test broader model families, naturally occurring skill libraries, and how selection errors affect end-to-end task outcomes.These directions address universality, ecological validity, and practical viability.
Future directions: Learned routing, formal theories of capacity limits, and connections to information-theoretic or mechanistic perspectives are also identified as open problems.The paper presents these as ways to improve scaling and strengthen the scientific grounding of the observations.

A Related Work

The related-work discussion situates the paper at the intersection of cognitive science, tool use, multi-agent systems, and scaling laws. Its distinguishing focus is the largely understudied scaling behavior of skill selection.

The paper connects its contribution to cognitive science, tool use in LLMs, multi-agent systems, and scaling laws.
Its central distinction from adjacent work is a focus on how skill-selection behavior scales.The paper characterizes this as having received limited systematic study.
The paper presents skill-selection scaling as central to agent-system design.

A.1 Tool Use in LLMs

Prior tool-use research enables LLMs to invoke external capabilities, while later systems address large-scale tool selection and planning. This paper distinguishes skills from tools and studies selection limits rather than proposing another selector.

Tool Use in LLMs: Toolformer showed that LLMs can learn to use calculators, search engines, and translation systems through self-supervision.
Tool Use in LLMs: ReAct established interleaved reasoning and tool-use actions, supporting dynamic decisions about invoking external capabilities.
Tool Use in LLMs: ToolLLM scaled tool selection to over 16,000 real-world APIs using neural retrieval and depth-first search for multi-step planning.
Tool Use in LLMs: Skills differ from tools because they are internalized capabilities with rich semantic descriptors and execution policies, rather than primarily atomic external APIs.The paper argues that this richer structure increases selection-related cognitive load.
Tool Use in LLMs: Rather than introducing a new selection mechanism, the paper characterizes fundamental limits on selection accuracy.

A.2 Multi-Agent LLM Systems

Prior multi-agent and skill-management systems organize specialized roles, workflows, memories, or skills for complex tasks. This paper bridges these traditions by treating some multi-agent systems as compilable into single agents with skill libraries.

Multi-Agent LLM Systems: AutoGen, MetaGPT, CAMEL, and related systems use role assignment, structured workflows, or agent interaction to support multi-agent problem solving.
Multi-Agent LLM Systems: Voyager, Generative Agents, and DEPS address skill acquisition, memory, planning, or sub-goal ranking in agent systems.
Multi-Agent LLM Systems: The paper provides a theoretical bridge by compiling certain multi-agent systems into equivalent single-agent systems with skill libraries.
Multi-Agent LLM Systems: The compilation is beneficial when skill libraries remain within capacity, but problematic when those libraries exceed selection capacity.
Multi-Agent LLM Systems: Hierarchical multi-agent organization is presented as a mitigation for skill libraries that grow beyond capacity.

A.3 Scaling Laws and Emergent Abilities

Prior scaling-law work has examined model, data, compute, loss, downstream-task, and agent-system scaling, but this paper shifts attention to the number of skills in an agent’s action space. It connects this scaling behavior to semantic confusability and cognitive-psychology principles while introducing a multi-agent-to-single-agent compilation procedure.

Prior Scaling Work: Earlier work studied scaling with model size, data, compute, loss, downstream tasks, and agent-system structure.The cited studies include trade-offs involving agent quantity, coordination structure, and task properties.
Action-Space Scaling: This paper studies scaling with action-space size: the number of skills an agent must select among.The paper distinguishes this focus from prior work centered on model scale.
Semantic Confusability: The paper argues that action-space scaling is mediated by semantic confusability and relates LLM behavior to cognitive-psychology principles.The cited principles come from Hick, Shepard, and Sweller.
MAS-to-SAS Compilation: Algorithm 3 compiles a multi-agent graph into a unified skill library by extracting agent capabilities and assigning tool-backed or internalized skill representations.The procedure takes a multi-agent graph and compiler model as inputs, then initializes a unified skill library.

C Experimental Details

The experiments use a synthetic skill library organized across domains, with each skill represented by an identifier, natural-language descriptor, and execution policy. The accompanying table provides examples of these domain-specific skills.

Skill Library: The synthetic skill library contains skills from multiple domains.Table 4 presents representative examples from each domain.
Skill Representation: Each skill has a unique identifier, a natural-language descriptor, and an execution policy.These fields define the representation used for example skills in the library.
Skill Examples: Table 4 provides example skills from different domains in the synthetic library.The table is used to illustrate the library’s cross-domain composition.

C.2 Task Examples

The task examples pair generated tasks with ground-truth skill mappings and construct semantic competitors with similar functionality but different phrasing. The experiments illustrate policy complexity, flat selection, and two-stage hierarchical selection using concrete prompts and categories.

Task Construction: Tasks are generated from domain-specific templates with randomized parameters and paired with ground-truth skill mappings.Table 5 summarizes example tasks and their correct skills.
Semantic Competitors: Semantic competitors have near-identical functionality but different surface forms or phrasing.This design creates semantic confusability for H2 experiments.
Policy Complexity: H3 varies policy complexity across three levels, illustrated with the “Calculate Sum” skill.Figure 6 shows the three policy-complexity examples.
Hierarchical Routing: The hierarchical example routes a numerical-summation task through categories before selecting a skill within the selected category.The prompt includes Summation, Averaging, Email Writing, Sentiment Analysis, and Name Extraction as available categories.