Source-linked AI summary
Coalition-Aware Skill Reliability for Self-Evolving Agents
Qiyan Zhao, Xiaofeng Zhang, Bo Liu, Minda Chen, Wei Xiong, Jingyang Chen, Guanting Ye, Wenhao Yu, Xiaosong Yuan, Shijie Han, Da-Han Wang, Jianmin Ji, Fei Huang, Xu-Yao Zhang
TL;DR
Existing skill-based self-evolving agents do not establish whether accumulated skills make positive mechanistic contributions, especially when interactions and deployment domains change. The paper audits skills across coalitions and domains, then introduces coalition-aware selection and label-free masking. Across four benchmarks, both interventions consistently improve task performance and cross-domain generalization, while the analysis shows that reliability is coalition- and domain-dependent.
Problem
Existing work largely focuses on skill lifecycle operations while leaving whether accumulated skills make positive mechanistic contributions unresolved.
Method
The paper audits skill reliability across bank compositions and domains, uses sampled Shapley marginals for CASS, and masks skills with harmful retrieval effects using unlabeled target queries in u-SMCO.
Results
CASS and u-SMCO consistently improve task performance and cross-domain generalization across LoCoMo, LongMemEval, HotpotQA, and ALFWorld.
Takeaways & Limitations
Skill reliability is a joint property of the skill, its surrounding bank, and the deployment domain rather than an intrinsic individual attribute.
Takeaways & Limitations
Outcome-only gates are structurally blind to coalition-level interactions, so isolation-based skill evaluation cannot establish reliability.
Abstract
from arXiv · showhide
Agent skills, structured artifacts distilled from interaction trajectories and dynamically reused from skill banks, have become a central mechanism for enabling large language model (LLM)-based self-evolving agents to learn from past experience. Yet existing work has largely focused on the operational aspects of skills, such as acquisition, evolution, and retrieval, while leaving a more fundamental reliability question unresolved: Do accumulated skills in an agent's skill bank actually make positive mechanistic contributions? We investigate this question through systematic skill-bank audits across alternative bank compositions and deployment domains, measuring the resulting changes in agent behavior. These audits reveal two recurring reliability failures: coalition pollution, where bank-level gains conceal negative coalition-level skill contributions, and cross-domain utility reversal, where source-beneficial skills reverse their effects after transfer. These findings motivate two reliability interventions: coalition-aware skill selection during skill accumulation and label-free skill masking after transfer. Coalition-Aware Skill Selection (CASS) selects more reliable candidate skills for the current bank using sampled Shapley marginals. Unsupervised Skill-Masked Coalition Optimizer (u-SMCO) masks transferred skills whose exclusion improves retrieval quality on unlabeled target-domain data. Agentic experiments on LoCoMo, LongMemEval, HotpotQA, and ALFWorld show that CASS and u-SMCO consistently improve task performance and cross-domain generalization over strong skill-based self-evolving agent baselines. Beyond accuracy, coalition-conditioned reliability modeling reduces sensitivity to noisy outcome-reward fluctuations during reinforcement learning and exposes the limits of isolation-based skill evaluation.
1 Introduction
Self-evolving agents reuse distilled skills, but existing lifecycle methods leave unresolved whether accumulated skills make positive mechanistic contributions. Audits identify coalition pollution and cross-domain utility reversal, motivating coalition-aware selection and label-free masking interventions.
- Background: Skill-based agents distill interaction trajectories into reusable artifacts that are stored in persistent banks and retrieved at inference time.These artifacts support learning from both successful and failed interactions without additional parameter updates.
- Motivation: Existing work emphasizes skill acquisition, evolution, retrieval, composition, and generalization while largely leaving mechanistic contribution reliability unexamined.The central question is whether accumulated skills actually contribute positively when invoked.
- Audit: Systematic audits vary skill-bank composition across evolution and transfer, comparing pre- and post-acceptance banks and in-domain versus out-of-domain deployment.The audit is termed Skill Mechanistic Reliability Audit (SMRA).
- Failure modes: Coalition pollution occurs when aggregate gains admit skills whose individual contribution is near zero or negative because contributions depend on coalition composition.The audit also finds that incumbent skill marginals can shift after a candidate is accepted.
- Failure modes: Cross-domain utility reversal occurs when skills useful in the source domain become unreliable or harmful after transfer to the target domain.The same accumulated bank can therefore receive different reliability verdicts across deployment domains.
- Interventions: CASS selects candidates using sampled coalition Shapley marginals, while u-SMCO masks transferred skills when their removal improves retrieval quality on unlabeled target queries.Together, these interventions target skill accumulation and post-transfer deployment respectively.
- Results: Experiments on LoCoMo, LongMemEval, HotpotQA, and ALFWorld show consistent improvements in task performance and cross-domain generalization over strong self-evolving baselines.Coalition-aware signals also mitigate reward noise during training.
2 Related Work
Research on self-evolving agents has progressed from external-memory trajectories toward structured, reusable skills and stronger agent scaffolds. Related work also studies reliability and distribution shift, while this paper distinguishes itself through coalition-conditioned causal attribution and targeted interventions.
- Memory and skills: Memory-based agents store interaction trajectories externally, but redundancy and noise motivate distilling experience into reusable skills.This shift supports more structured reuse than raw trajectory retrieval.
- Memory and skills: Agent Skills encode reusable decision-making strategies that can be retrieved at inference time without updating model parameters.They distill knowledge from successful and failed interaction trajectories.
- Reliability: SkillLens shows that textually plausible skills can still have negative effects through utility-grounded reliability analysis.The paper extends this line beyond text-based utility estimation.
- Reliability: This work models reliability as a joint property of the skill, surrounding bank, and deployment domain, using causal contributions rather than intrinsic per-skill judgments.It also proposes interventions instead of stopping at diagnosis.
- Methodological foundations: The approach combines coalition-level contribution attribution based on Shapley values with reliability analysis under distribution shift.Prior distribution-shift work motivates skepticism that source competence predicts target behavior.
3 Skill Mechanistic Reliability Audit
SMRA audits skill contributions by varying bank composition and deployment domain, exposing coalition pollution and cross-domain utility reversal. It also shows why outcome-only acceptance gates cannot reliably identify which skills help or harm.
- Self-evolution: An outcome-only gate accepts a candidate when aggregate reward increases, but this bank-level signal does not identify other skills’ contributions.The gate accepts when Ge > 0 and leaves the resulting bank’s other members unconstrained.
- SMRA formulation: SMRA evaluates skills through paired full-bank and knockout interventions, then generalizes marginal contributions across alternative skill coalitions.Positive knockout deltas indicate support, while negative values mean removing the skill improves performance.
- Coalition pollution: The baseline gate accepted a bank transition with only +0.006 online gain, although capture_activity_preferences contributed ∆ = −0.005 while insert contributed ∆ = +0.084.This mismatch defines coalition pollution: non-contributing skills can be retained when bank-level gains conceal coalition-level contributions.
- Reliability limits: Accept/reject margins of 0.006 to 0.02 are close to reward-estimate error bars of ≈0.004, while single-skill audits resolve contributions only to approximately ±0.04.The audit resolution is an order of magnitude coarser than the margins used for gate decisions.
- Reliability interventions: CASS restores coalition information during evolution, whereas u-SMCO uses unlabeled target-domain retrieval quality to address transferred skills’ domain-dependent utility.Together, these interventions reflect reliability as a property of the skill, its surrounding bank, and its deployment domain.
- Cross-domain utility reversal: Insert’s source contribution is +0.084 on LoCoMo but reverses to −0.035 on HotpotQA, whose 95% confidence interval includes zero.Because the bank is held fixed, the contrast isolates deployment-domain dependence; the result is interpreted as directional reversal rather than significant target harm.
4 Method
The method introduces CASS for coalition-aware skill selection during evolution and u-SMCO for label-free skill masking after transfer. CASS samples coalition knockouts to estimate bank reliability, while u-SMCO masks skills whose removal improves target-domain retrieval quality.
- CASS and u-SMCO address coalition pollution during evolution and cross-domain utility reversal after transfer, respectively.
- 4.1 CASS: Coalition-Aware Skill Selection: CASS estimates coalition-conditioned reliability by sampling diverse skill coalitions and averaging their knockout margins before combining the result with online outcome reward.Coalitions are sampled with uniformly chosen sizes and subsets; joint removal losses are charged equally to coalition members.
- 4.1 CASS: Coalition-Aware Skill Selection: CASS retains a candidate bank only when its combined score exceeds every score observed so far.The gate preserves the baseline best-so-far rule while changing the scalar used for ranking candidate banks.
- 4.2 u-SMCO: Unsupervised Bank Masking: u-SMCO scores each skill by target-domain retrieval quality using unlabeled queries, rather than relying on task labels or training-reward signals.Its mask score compares retrieval quality with the full bank against retrieval quality after removing one skill.
- 4.2 u-SMCO: Unsupervised Bank Masking: u-SMCO greedily masks the lowest-scoring skill and stops when no score falls below threshold τ, requiring only unlabeled queries and raw contexts.Memory rebuilding dominates the reported computational cost at O(K^2) rebuilds and 6–10 minutes per mask step.
5 Experiments
The experiments evaluate matched interventions for coalition pollution and cross-domain utility reversal, showing that coalition-aware selection and label-free masking improve reliability-critical behavior while adding limited overhead.
- Experimental setup: CASS differs from MemSkill only in the gate, while u-SMCO runs before inference under a fully matched protocol across four domains.Evaluation reports mean ± std over three seeds under a fixed judge.
- CASS results: 1/12 candidate skills were accepted by CASS versus 6/12 by MemSkill across four gate decisions per seed and three seeds.The outcome-gate additions were collectively net-negative by −1.38 pp on average despite satisfying the admission criterion.
- u-SMCO results: Skill_IN was masked on 5/6 trained banks without target labels, and masking improved every bank by 7.17 pp for MemSkill versus 3.59 pp for CASS on average.The result identifies label-free cross-domain utility reversal and indicates that polluted banks require more post-transfer removal.
- Discussion: Outcome-only gates cannot distinguish bank-level improvement from coalition-level contribution, making isolation-based skill evaluation structurally insufficient.Theorem 1 places this limitation below the resolution of a single-skill audit, motivating coalition scoring.
- Signal analysis: Retrieval quality correlated with the label-based ranking at ρ = +0.76, whereas selection-policy entropy produced an anti-correlated ranking at ρ = −0.18.The retrieval-quality signal is measured on the target domain and decoupled from training reward.
- Cost: CASS adds eight coalition evaluations per outer epoch, about 5% of a 20-hour run, while u-SMCO performs O(K2) memory rebuilds before deployment and adds no inference-time cost.The reported overhead is limited to training or pre-deployment processing.
6 Conclusion
The paper concludes that skill reliability depends jointly on the skill, its surrounding bank, and its deployment domain, motivating coalition-aware training and label-free post-transfer masking.
- Conclusion: Reliability is a coalition-level property whose contribution depends on the surrounding bank and deployment domain.Outcome-only gates are provably blind to coalition pollution and cross-domain utility reversal.
- Conclusion: CASS scores sampled coalitions during training, while u-SMCO masks transferred skills using unlabeled target queries.Across three seeds, both interventions improve the reliability-critical regimes they target.
- Future directions: The paper identifies adaptive coalition budgets and extensions beyond textual skills as directions for future work.