Source-linked AI summary
CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification
Hanrong Zhang, Shicheng Fan, Henry Peng Zou, Yankai Chen, Zhenting Wang, Jiayu Zhou, Chengze Li, Wei-Chieh Huang, Yifei Yao, Kening Zheng, Xue, Liu, Xiaoxiao Li, Philip S. Yu
TL;DR
Professional open-ended tasks require structured skills beyond isolated tools, yet human authoring is costly and inconsistent and existing self-evolving methods do not construct complex multi-file packages. CoEvoSkills co-evolves an iterative Skill Generator with a Surrogate Verifier that provides actionable feedback without hidden test content. On SkillsBench, it achieves the highest reported pass rate, outperforms five baselines, and transfers strongly across additional LLMs.
Problem
Professional open-ended tasks require coordinated multi-step workflows and artifacts, while human-authored skills are costly, difficult to scale, and inconsistently effective.
Method
CoEvoSkills co-evolves a Skill Generator that iteratively refines multi-file skills with a Surrogate Verifier that supplies actionable feedback without hidden test content.
Results
71.1% pass rate, +40.5pp over the no-skill baseline, is achieved on SkillsBench while outperforming five baselines and transferring to six additional LLMs.
Takeaways & Limitations
Self-evolved skills capture agent-relevant reasoning and tool-use strategies and encode reusable task structure that transfers across model families.
Takeaways & Limitations
Skill evolution uses task-relevant background knowledge available at initialization and does not yet autonomously seek additional open-world knowledge.
Abstract
from arXiv · showhide
Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks that simple tool invocations cannot address. A tool is a single, self-contained function, whereas a skill is a structured bundle of interdependent multi-file artifacts. Currently, skill generation is not only label-intensive due to manual authoring, but also may suffer from human--machine cognitive misalignment, which can lead to degraded agent performance, as evidenced by evaluations on SkillsBench. Therefore, we aim to enable agents to autonomously generate skills. However, existing self-evolving methods designed for tools cannot be directly applied to skills due to their increased complexity. To address these issues, we propose CoEvoSkills, a self-evolving skills framework that enables agents to autonomously construct complex, multi-file skill packages. Specifically, CoEvoSkills couples a Skill Generator that iteratively refines skills with a Surrogate Verifier that co-evolves to provide informative and actionable feedback without access to ground-truth test content. On SkillsBench, CoEvoSkills outperforms five baselines on both Claude Code and Codex, and generalizes strongly to six additional LLMs. The code is publicly available at https://github.com/Zhang-Henry/CoEvoSkills.
1 Introduction
Professional open-ended tasks require coordinated multi-step workflows and artifacts beyond isolated tool calls. CoEvoSkills targets costly, inconsistent human-authored skills by enabling autonomous skill evolution, achieving strong benchmark performance and rapid quality gains.
- Motivation: Professional tasks require agents to decompose goals, coordinate tools, recover from failures, and validate intermediate outputs across tightly coupled artifacts.These tasks are difficult because decisions are long-horizon and reliable environmental feedback is often sparse or delayed.
- Agent skills: Skills package workflow instructions, executable scripts, and domain references, extending beyond the capabilities of simple tools.SkillsBench reports consistent performance gains from well-crafted skills across professional domains, including software engineering and scientific analysis.
- Motivation: Human-authored skills are labor-intensive, difficult to scale, and uneven in effect, with Natural Science showing degraded performance after skill integration.The paper attributes this inconsistency to possible human–machine cognitive misalignment between expert-designed workflows and LLM execution behavior.
- Contributions: CoEvoSkills surpasses human-curated skills within five evolution iterations, according to the reported quality-improvement trajectory.Figure 2 presents quality across five evolution rounds, highlighting iterative improvement rather than one-shot generation.
- Contributions: 71.1% pass rate, +40.5pp over the no-skill baseline, makes CoEvoSkills the strongest result among five SkillsBench baselines.The framework also transfers to six additional LLMs, producing approximately 35–44 percentage-point gains over their respective no-skill baselines.
2 Related Work
Prior work establishes reusable agent skills and benchmarks them across professional domains, while self-evolving methods largely remain limited to tools, APIs, heuristics, or prompt templates. These limitations leave construction of executable, multi-file skill packages unresolved.
- LLM Agent Skills: Agent skills are reusable modules with explicit applicability and termination conditions, distinct from atomic tools and one-off plans.SkillsBench evaluates reusable skills across diverse professional domains using deterministic task verifiers.
- Self-evolving LLM Agents: Most self-evolving pipelines produce single tools, function APIs, or prompt heuristics rather than complete multi-file skill packages.AutoSkill and AutoRefine extract reusable knowledge as prompt templates instead of executable packages.
3 Method
CoEvoSkills co-evolves a multi-file Skill Generator and an informationally isolated Surrogate Verifier, alternating skill refinement with verifier test escalation. Hidden oracle feedback remains opaque, while surrogate diagnostics provide actionable guidance for iterative improvement.
- Framework overview: CoEvoSkills alternates Skill Generator refinement with Surrogate Verifier evolution to construct complex multi-file skill packages.The generator executes candidate skills, while the verifier evaluates outputs and supplies feedback for subsequent revisions.
- Co-evolution loop: When oracle evaluation exposes a mismatch between surrogate and hidden rewards, the verifier strengthens its tests without ground-truth test content, and the generator refines the skill.Successful skills can be deployed; otherwise, the binary oracle signal triggers another co-evolution iteration and preserves the best oracle-scoring snapshot.
- Problem formulation: A skill is a structured bundle of domain-specific instructions, executable scripts, and reference materials that conditions the agent’s policy.The agent selects actions according to πθ(at | ht, S), with the skill influencing behavior over the execution trajectory.
- Surrogate verification: The ground-truth oracle returns only an opaque pass/fail signal, so CoEvoSkills uses deterministic surrogate assertions to produce dense reward and failure diagnostics.The surrogate verifier reports failed tests, root-cause analysis, and actionable revision suggestions without access to hidden oracle criteria or test content.
- Skill Generator: The Skill Generator initializes context with the task instruction, observable background knowledge, and a domain-agnostic skill-creator meta-skill.At each revision, it updates the current skill using accumulated surrogate-verifier diagnostics.
4 Experiments
CoEvoSkills is evaluated on SkillsBench across baseline comparisons, models, and professional domains. It outperforms human-curated and self-generation baselines, transfers across models, and improves most domains through iterative evolution.
- Experimental Setup: SkillsBench evaluation covers 85 tasks with deterministic binary pass/fail verification, using pass rate as the primary metric.
- Experimental Setup: The study compares CoEvoSkills with five baselines, including no-skill, one-pass self-generation, chain-of-thought-guided generation, and human-curated skills.
- Skill Quality Comparison: 71.1% pass rate: CoEvoSkills exceeds the no-skill baseline (30.6%) by +40.5pp and human-curated skills (53.5%) by +17.6pp on Claude Opus 4.6 with Claude-Code.
- Cross-Model Transferability: 35–44 percentage points: Opus-evolved skills improve pass rates across six additional models, while GPT-5.2’s self-evolved skills reach 69.8% versus 65.0% with transferred skills.Model-matched evolution retains a 4.8pp advantage for GPT-5.2.
- Per-Domain Results: Self-evolved skills outperform human-curated skills in 9 of 11 domains, with the largest margins in Finance (+56.9pp) and Cybersecurity (+23.2pp).Energy and Robotics show diminishing returns where human-curated skills already perform well.
- Evolution Dynamics: Pass rate rises from 30.6% at the no-skill baseline to 75% by round 5, surpassing human-curated skills at round 3 with 63%.Tasks require an average of 4.1 verification cycles and 2.4 Ground Truth Oracle rounds to converge.
5 Conclusion
CoEvoSkills enables agents to self-generate skills that outperform human-curated and baseline alternatives while transferring across models. Its current scope starts from task-relevant background knowledge, motivating open-world skill evolution as future work.
- CoEvoSkills addresses one-shot generation unreliability and missing ground-truth feedback through co-evolutionary optimization.
- On SkillsBench, CoEvoSkills substantially outperforms human-curated skills and self-generation baselines while demonstrating strong transferability.
- The current framework evolves skills from task-relevant background knowledge available at initialization, leaving open-world knowledge acquisition as a next step.
A Ablation Studies
The ablations show that surrogate verification and background context both improve skill evolution, with diagnostic feedback producing the largest gain. The full framework combines these components with iterative evolution.
- Framework settings: The full CoEvoSkills framework combines iterative skill evolution, surrogate verification, and pre-installed multi-file skill packages.Ablations use Claude Code with Claude Opus 4.6.
- Surrogate verification: 30.0pp: removing the surrogate verifier lowers pass rate from 71.1% to 41.1%.Without structured failure diagnostics, the generator receives only opaque oracle pass/fail feedback and cannot perform targeted repairs.
- Background context: Background context raises pass rate from 30.6% to 42.4%, while full CoEvoSkills reaches 71.1%.Context provides an evolutionary substrate that co-evolution converts into structured, verified, executable skills.
B Evolution Iteration Analysis
Across 85 evolution tasks, surrogate verification handles much of the iteration process while most tasks reach oracle evaluation quickly. The stated iteration budgets are sufficient for the majority of tasks, although failures cluster at higher iteration counts.
- Task-level iteration distributions: 85 evolution tasks are analyzed through distributions of verification cycles and Ground Truth Oracle rounds.Verification cycles include surrogate failures and oracle evaluations; oracle rounds count only surrogate-passed cases.
- Task exclusions: Two tasks are excluded because required infrastructure cannot be reproduced reliably, rather than because of model outcomes.The exclusions involve authenticated Google services and a CUDA-capable GPU with 5,000 PyTorch training steps.
- Oracle convergence: Over 60% of tasks converge within 2 Ground Truth Oracle rounds, with a mean of 2.4 rounds.The 10 tasks failing to achieve a perfect oracle score cluster at 5 or more verification cycles.
- Verification cost: Approximately 40% of iterations are resolved by the surrogate verifier alone: 2.4 of 4.1 average verification cycles escalate to the oracle.The surrogate verifier therefore absorbs most iteration cost before authoritative evaluation.
C Case Study: Exoplanet Transit Period Detection
The exoplanet case study traces CoEvoSkills from a flawed BLS implementation to a TLS-based skill that passes all oracle tests. It also shows how surrogate diagnostics and oracle feedback jointly guide both repairs and algorithm replacement.
- Task setup: The task detects exoplanet orbital periods from TESS lightcurves with stellar variability and requires 5-decimal accuracy across four deterministic tests.All four tests must pass for reward = 1.0.
- Evolution trace: The skill evolves through 4 script rewrites across 6 host interventions, with Ground Truth Oracle scores progressing 75% →75% →100%.The evolution trace records surrogate and oracle outcomes at each round.
- Versions 1–2: Version 1’s BLS implementation fails all 15 surrogate tests because its duration range generates noise from micro-transit-like features.The task does not reach Ground Truth Oracle evaluation at this stage.
- Versions 1–2: Version 2 reaches 75% oracle accuracy, but BLS lacks 5-decimal precision because its grid resolution is too coarse.Widening the duration range and adding an end-to-end pipeline do not remove the precision limitation.
- Version 3: After median-filter detrending still yields 75%, the agent recognizes that parameter tuning within BLS is insufficient.The precision issue persists regardless of the detrending method.
- Version 4: 100%: Version 4 switches to TLS, Savitzky–Golay filtering, and two-stage period search, producing a 4/4 oracle result.TLS uses a realistic limb-darkened transit model, while the two-stage search refines a broad period sweep.
- Verification architecture: Surrogate tests catch implementation bugs and regressions, but the oracle reveals BLS’s hidden precision ceiling and drives algorithm replacement.The surrogate verifier checks functional correctness, not numerical precision against hidden ground truth.
- Skill structure: The evolved package contains a 64-line procedure document and 142-line utility module, achieving 100% across 5 fresh Opus 4.6 trials.It consolidates a 9-step pipeline and 9 executable functions, whereas five human-curated skills total 1,096 documentation lines.
D Key Prompts
The prompts organize agents into an ordered loop of skill evolution, task execution, and summarization, while enforcing reusable, portable, self-reflective skill construction. A structured self-generation baseline without external verification reaches only 30.7% pass rate.
- D.1 Evolution Agent System Prompt: The appendix presents the prompts governing CoEvoSkills, including the Evolution Agent and Skill Generator system prompts.The prompts are shown verbatim with minor formatting adjustments.
- D.1 Evolution Agent System Prompt: The Evolution Agent follows three phases: evolve skills, execute the task using them, and summarize changes for later iterations.The workflow requires skill design, self-reflection, and producing outputs by importing skill functions.
- D.1 Evolution Agent System Prompt: The command interface requires analysis, planning, and command objects, with optional task completion and bounded command wait durations.Commands specify exact keystrokes and wait times; no command may wait longer than 60 seconds.
- D.1 Evolution Agent System Prompt: The Skill Generator loads relevant reusable skills, creates new skills through skill-creator, and stores them as evo-prefixed multi-file packages.Skills contain workflows, domain expertise, references, SKILL.md, and independently testable utility functions.
- D.1 Evolution Agent System Prompt: Mandatory progress tracking requires /root/progress.md to record ordered phases and prevents task completion until every phase is checked off.The phases include environment discovery, skill creation, self-reflection, execution, repair, and summary.
- D.1 Evolution Agent System Prompt: Skills must internalize provided documentation, ground logic in environment data, and remain portable without external reference files.The prompts require encoding documented concepts into SKILL.md and scripts rather than linking to outside documents.
- D.1 Evolution Agent System Prompt: Task outputs must be produced by importing and calling skill utility functions, while self-reflection verifies requirements and portable import examples before execution.The prompt specifically rejects standalone code duplicating skill logic.
- D.5 CoT-Guided Self-Generation Prompt: The structured five-step self-generation baseline still lacks external verification feedback and achieves only a 30.7% pass rate.Its performance is comparable to the no-skill baseline.