Source-linked AI summary
MASkills: Continual Skills Optimization for Multi-Agent LLM Systems
Huaiyuan Yao, Xiaoou Liu, Charles Fleming, Tianlong Chen, Hua Wei
TL;DR
Multi-agent LLM systems need better ways to improve continually from interaction experience because unstructured memories are difficult to invoke, refine, and scale. MASkills instead optimizes structured agent skills with credit assignment, aggregation, stabilized language-space updates, and skill evolution, achieving consistent improvements across memory and agentic benchmarks while remaining scoped to mainly cooperative settings with fixed roles and topologies.
Problem
Existing experience memories record past trajectories but lack reliable reusable policies, invocation conditions, and scalable organization for continual multi-agent improvement.
Method
MASkills optimizes multi-agent skill libraries through skill-conditioned credit assignment, hierarchical aggregation, momentum-smoothed language-space editing, and refinement, induction, consolidation, and pruning.
Results
MASkills consistently improves memory and agentic tasks across HotpotQA, LoCoMo, and GAIA, with strong performance across the evaluated benchmarks.
Takeaways & Limitations
Continual skill optimization offers a structured route for agents to refine and reuse procedural knowledge from interaction experience in multi-agent systems.
Takeaways & Limitations
Experiments mainly cover cooperative settings with relatively fixed agent roles and communication topologies, leaving dynamic, adversarial, and open-world environments underexplored.
Abstract
from arXiv · showhide
LLM-based multi-agent systems have shown strong performance on complex tasks, yet continual improvement from interaction experience remains challenging. Existing self-reflection methods build experience memories, but memories are mostly hard to invoke, refine, or scale, while agent skills offer a more actionable unit: structured procedural knowledge that specifies when to act, how to act, and which resources or tools to use. We introduce MASkills, a continual learning framework that optimizes multi-agent LLM systems through agent skills. MASkills presents a new agent-optimization pipeline that integrates skill-conditioned credit assignment, hierarchical credit aggregation, and momentum-smoothed optimization, enabling agent skill libraries to evolve through refinement, induction, consolidation, and pruning. Experiments on HotpotQA, LoCoMo, and GAIA demonstrate the effectiveness of MASkills across multiple agentic tasks. Our code is available at https://github.com/DaRL-GenAI/MASkills
1. Introduction
Continual improvement remains difficult because experience memories record past trajectories without reliably specifying reusable policies, invocation conditions, or scalable organization. MASkills addresses this gap by optimizing structured agent skills for coordinated multi-agent behavior and evolving them through language-space updates.
- Motivation: Experience memories preserve past trajectories but lack reusable action policies, reliable invocation conditions, and scalable organization as noisy records accumulate.Useful lessons can become mixed with redundant or stale free-form memories.
- Motivation: Skills package procedural knowledge about when to act, how to act, and which resources or tools to use.Their progressive-disclosure design supports discovery, on-demand loading, refinement, and scalability.
- Challenges: Existing reusable skill methods remain predominantly single-agent and do not optimize skills under multi-agent coordination dynamics.Team-level utility depends on interactions among agents and their skills.
- Challenges: Multi-agent skill evolution requires credit assignment to the responsible skill, hierarchical aggregation across coordination structures, and discrete high-risk language-space updates.Skill effects may occur at only a few timesteps and become entangled with other agents’ behaviors.
- Approach: MASkills performs policy improvement directly in agents’ skill spaces through a unified language-space policy-gradient-style framework.The framework is designed for decentralized execution and continually evolves reusable skill artifacts.
- Contributions: MASkills evolves agent skill libraries through skill-conditioned credit assignment, aggregation, optimization, refinement, induction, consolidation, and pruning.Held-out validation and rollback support stable updates, with evaluation on HotpotQA, LoCoMo, and GAIA.
2. Related Work
Prior work develops reusable procedural abstractions and multi-agent frameworks, but these lines of research have generally not jointly optimized skills for team coordination. MASkills builds on this gap between single-agent skill evolution and multi-agent continual improvement.
- Agent Skill Discovery: Voyager introduced an expanding skill library, while MemSkill and EvoSkill developed learnable memory operations and iterative textual skill discovery.These methods use controllers, executors, designers, proposers, or skill-builders to refine reusable procedures.
- Multi-agent LLM Systems: Multi-agent frameworks such as AutoGen and MetaGPT use division of labor to improve coverage on complex tasks.Most rely on hand-engineered roles and prompts, while self-improving systems rewrite prompts globally or share memories.
3. Problem Formulation
MASkills formulates cooperative multi-agent behavior under decentralized partial observability and re-parameterizes agent policies through evolving skill libraries. Skills are structured artifacts that provide procedural instructions and optional resources, while language-space editing updates those libraries from aggregated feedback and historical directions.
- Cooperative Multi-agent Setting: MASkills models a team of N LLM agents in a cooperative Dec-POMDP with decentralized execution and a shared team objective.Each agent maps local observations to actions, and the team optimizes expected discounted return.
- Skill-space Optimization: Each agent’s policy is re-parameterized through a skill library that serves as the learnable component of decentralized behavior.The libraries contain evolving skill artifacts rather than conventional network parameters.
- Skill Artifacts: A skill artifact is a structured file-system representation containing metadata, procedural instructions, and auxiliary resources.Resources may include scripts, references, assets, and configuration files.
- Skill Artifacts: This artifact structure supports concise procedural abstraction, actionable execution guidance, hierarchical disclosure, and reusable skill composition.Implementations may use a directory with an instruction file, optional resource folders, and sometimes a separate skill.yaml metadata file.
- Skill-space Optimization: MASkills directly optimizes skill libraries rather than classical policy parameters using language-space policy-gradient-style editing.The analogy is conceptual: rollouts sample trajectories, critics provide advantage-like feedback, and editing operators act as pseudogradient updates.
4. MASkills Framework
MASkills treats reusable skill artifacts as the policy objects of a continual multi-agent optimization loop. The framework records skill invocations, assigns counterfactual credit, stabilizes language-space updates, and evolves libraries through several operators with held-out validation.
- Skill Evolution Operators: The resulting feedback drives refinement, induction, consolidation, and pruning-like skill-library transformations, including localized edits, new abstractions from hard cases, and merging of overlapping skills.The supplied passages explicitly describe refinement, induction, and consolidation; the pipeline overview also names pruning.
- Multi-Agent Skill Execution: Agents autonomously select and compose skills during decentralized execution, producing explicit skill traces that support later skill-level credit assignment.Skill metadata is exposed at rollout time, while detailed skill resources are loaded when selected.
- Skill-Conditioned Credit Assignment: A centralized language critic assigns each invoked skill counterfactual credit relative to omitting that skill, while residual agent credit captures unexplained effects and possible needs for new skills.The critique distinguishes helpful, redundant, harmful, generalizable, or specialized skill behavior.
- Stabilized Language Gradient Descent: MASkills converts noisy trajectory critiques into stable language-space gradients through structured edit extraction, hierarchical aggregation across interaction levels, and momentum across optimization cycles.Aggregation merges recurring patterns, resolves conflicts, removes redundancy, and summarizes coordination utility.
- Validation and Rollback: Candidate skill updates are evaluated on held-out validation tasks before commitment, providing rollback-based protection against regressions from noisy critiques or overfitting to recent trajectories.The validation mechanism functions as a trust-region-style constraint in skill space.
5. Experiments
The experiments evaluate MASkills on diverse cooperative benchmarks, then test skill quality, transferability, topology robustness, backbone generalization, and component contributions. MASkills shows strong performance across tasks, transferable skills, competitive behavior across coordination structures and backbones, and degradation when optimization modules are removed.
- Experimental Setup: MASkills is evaluated on HotpotQA, LoCoMo, and GAIA, covering multi-hop reasoning, long-horizon dialogue memory, and general AI assistant tasks.The cooperative environments use decentralized execution, specialized agents, peer-to-peer communication, and shared team objectives.
- Main Results: Across all three benchmarks, MASkills achieves strong and consistent performance, outperforming relevant baselines on HotpotQA and achieving the strongest overall performance on GAIA.On LoCoMo, it improves long-context memory retrieval and response quality across single-hop and multi-hop settings.
- Skill Quality and Transferability: Continually optimized skills substantially outperform prompt-generated skills across evaluated LoCoMo and GAIA settings, improving LoCoMo F1 and BLEU and achieving the strongest GAIA performance.Prompt-generated skills provide only marginal improvements over directly prompting agents.
- Skill Quality and Transferability: Transferred skill libraries improve downstream performance across all evaluated source-target pairs without additional optimization, including GAIA-to-HotpotQA reasoning and LoCoMo-derived long-horizon memory reasoning.These results suggest that MASkills captures reusable behavioral patterns.
- Robustness and Generalization: MASkills maintains competitive performance across centralized, decentralized peer, and hierarchical coordination topologies, but the best topology depends on the task.Decentralized peer coordination is strongest on HotpotQA and GAIA, whereas centralized coordination performs best on LoCoMo; hierarchical coordination is generally intermediate.
- Robustness and Generalization: MASkills maintains competitive performance across proprietary and open-source language-model backbones, while ablations show that removing any optimization component degrades performance.Validation rollback has the largest ablation impact, indicating that reverting unstable updates is important for stable continual optimization.
6. Conclusion
MASkills optimizes multi-agent LLM systems in skill space, enabling agents to refine and reuse skills from interaction experience. Across multiple benchmarks, it consistently improves memory and agentic tasks.
- MASkills optimizes multi-agent LLM systems in skill space rather than parameter space.
- Skill-conditioned credit assignment and evolution let agents continually refine and reuse skills from interaction experience.
- Experiments across various benchmarks show consistent improvements on memory and agentic tasks.
Limitations
The evaluation mainly covers cooperative environments with fixed roles and communication topologies, while expanding skill libraries may create scalability challenges.
- Current experiments focus mainly on cooperative settings with relatively fixed agent roles and communication topologies.Dynamic, adversarial, and open-world environments remain underexplored.
- Skill-library growth may create challenges for retrieval, consolidation, and coordination efficiency.
- Future extensions include adaptive organizational structures, competitive games, decentralized coordination, hierarchical organization, retrieval compression, and lifelong learning.
Ethical Considerations
Continual skill refinement can amplify undesirable model behaviors and create privacy and misuse risks. The paper recommends validation safeguards, human oversight, privacy practices, and benign-use restrictions.
- Automatically evolved skills may amplify factual errors, social biases, unsafe tool use, or overconfident reasoning inherited from underlying LLMs.
- Held-out validation, rollback, consolidation, and pruning mitigate risks but do not replace human oversight in high-stakes applications.
- Deployments using user data should apply data minimization, anonymization, access control, and dataset and tool license compliance.
- Improved coordination and tool use could enable harmful automation without safeguards, so the paper recommends benign use cases and monitoring.
AI usage
The authors used LLMs during preparation to assist with code implementation and improve manuscript grammar and style. They report reviewing and verifying the final content, contributions, and analyses.
- LLMs assisted with code implementation during preparation of the work.
- LLMs also helped refine the manuscript’s grammatical clarity and stylistic flow.
- The authors state that they rigorously reviewed and verified the final content, technical contributions, and experimental analyses.