CODESKILL: Learning Self-Evolving Skills for Coding Agents

• The paper frames procedural skills as a compact, reusable form of experience that provides actionable guidance beyond episodic memory or raw trajectory storage.Prior agent systems store episodes, retrieve related cases, or summarize trajectories, whereas skills specify recurring situations and suitable actions.Source: §1 · p. 1
• Existing skill-construction methods rely heavily on fixed prompts and heuristic criteria, leaving evidence selection, skill abstraction, and skill-bank revision unclear for SWE agents.Long-horizon, tool-driven trajectories contain heterogeneous feedback that makes reusable procedural knowledge difficult to distinguish from task-specific or accidental details.Source: §1 · p. 1
• CODESKILL extracts reusable procedural knowledge, evolves skills from new or failed experience, and maintains the bank by adding, merging, or dropping candidates.The framework is designed to keep skills reusable and actionable for future software-engineering tasks.Source: §1 · pp. 1–2
• CODESKILL learns skill management as a policy using supervised warm-starting followed by GRPO reinforcement learning with downstream execution and rubric-based skill-quality feedback.The hybrid reward combines verifiable task outcomes with judgments of skill quality and behavior-skill alignment.Source: §1 · p. 2
• CODESKILL improves average pass rate by 9.69 over no-skill and by 4.01 over the strongest prompt-based or memory baseline across EnvBench, SWE-Bench Verified, and Terminal-Bench 2.These gains correspond to relative improvements of about 33% and 11%, respectively, and hold across in-domain and out-of-distribution software-engineering tasks.Source: §1 · p. 2

2.1 Self-evolving Long-term Memory

• Long-term-memory systems retain reflections, experiences, utility estimates, principles, or workflows, while coding-agent methods retrieve or summarize prior repair trajectories and issue contexts.These approaches demonstrate benefits from past experience but primarily focus on retrieval or summarization.Source: §2.1 · p. 2

2.2 Agent Skills & Automatic Skill Management

• Agent skills provide actionable procedural guidance through reusable workflows, instructions, scripts, or tool-use knowledge rather than merely recalling past interactions.Automatic construction methods distill lessons, workflows, and failure patterns into skill banks that can be reused and updated.Source: §2.2 · p. 2
• Existing skill methods often improve the solver, train skill use, or co-evolve skills with the agent, whereas CODESKILL learns how to generate and maintain skills for a frozen downstream coding agent.This shifts the learning target from skill use alone to adaptive skill management itself.Source: §2.2 · pp. 2–3
• Static skill injection does not always improve coding agents because skill usefulness depends strongly on task fit and context, motivating adaptive management.The related-work discussion identifies this dependency as a reason to avoid fixed skill-management strategies.Source: §2.2 · pp. 2–3

3.1 Problem Formalization

• CODESKILL learns a policy that updates a skill bank to improve the downstream performance of a frozen coding policy.The policy maps trajectory evidence and skill-bank context to operations that update the bank, while the downstream policy itself remains unchanged.Source: §3.1 · p. 3

3.2 Skill Management Loop

• The skill bank represents both task-level strategies and event-driven guidance for recurring execution events.Task-level skills encode high-level procedures, whereas event-driven skills specify reactions to failures, error messages, test outputs, and other recurring events.Source: §3.2 · p. 3
• Skill extraction distills reusable knowledge from trajectories, and skill evolution revises existing skills when new evidence reveals missing cases, better procedures, or failure modes.Extraction can produce task-level or event-driven skills, while evolution can revise a skill or skip when evidence does not support an update.Source: §3.2 · p. 3
• Skill-bank maintenance adds, merges, or drops candidate skills to keep the bank compact while retaining useful and complementary knowledge.Candidates are compared with similar existing skills; redundant, weakly grounded, overly specific, or non-transferable candidates can be dropped.Source: §3.2 · pp. 3–4

3.3.1 Supervised Warmup

• Supervised warmup uses teacher-generated operation targets before reinforcement learning, producing an initial policy that follows the operation schema.Targets cover extraction, evolution, and maintenance using trajectory evidence, optional skills, and retrieved bank context.Source: §3.3.1 · p. 4

3.3.2 RL for CODESKILL

• Reinforcement learning combines rubric-based quality rewards with execution and alignment feedback from a frozen coding policy, optimized through GRPO.Quality evaluates grounding, reusability, specificity, format, and actionability; execution measures verifier improvement over a no-skill baseline, while alignment checks whether the rollout follows the skill.Source: §3.3.2 · pp. 4–5
• A three-stage curriculum progressively trains extraction, evolution, and full skill-bank maintenance, reusing earlier-stage outputs as later training context.This progression follows the skill-bank lifecycle and addresses the multiple operation types and sparse verifiable feedback.Source: §3.3.2 · p. 5

4.1 Experimental Setup

• The evaluation uses held-out splits for EnvBench and SWE-Bench Verified, holds out Terminal-Bench 2 for out-of-distribution testing, and reports pass rate plus efficiency metrics.EnvBench also reports average issue count, while all benchmarks report average reasoning steps on solved instances.Source: §4.1 · p. 6

4.2 Main Results

• 39.26 average pass rate versus 29.57 for no-skill, while outperforming stronger baselines after RL training.CODESKILL achieves the best pass rate on all benchmark groups and exceeds the strongest baseline by 11.4% relative gain on average.Source: §4.2 · pp. 6–7
• Under the Qwen3.5-35B-A3B policy, CODESKILL reduces average reasoning steps on solved instances from 44.12 for no-skill to 35.15, the lowest among compared methods.Skill-based methods generally reduce reasoning cost compared with the no-skill baseline.Source: §4.2 · p. 7
• CODESKILL generalizes to held-out Terminal-Bench 2, raising pass rate from 25.88 to 34.12 and exceeding the strongest baseline by 3.53 pass rate.SFT-CODESKILL does not improve over no-skill on Terminal-Bench 2, whereas RL with downstream feedback improves transfer to unseen task types.Source: §4.2 · p. 7
• With GPT-5.4-mini as the downstream policy, CODESKILL improves average pass rate by 8.93 over no-skill and by 2.87 over the strongest prompt-based or memory baseline.The result supports transfer beyond the Qwen3.5-35B-A3B policy used for optimization.Source: §4.2 · p. 7

4.3 Ablation & Analysis

• Event-driven and task-level skills provide complementary knowledge, and their combination improves performance over either single-granularity variant.The lifecycle ablation compares progressively enabled skill granularities and operations under the Qwen3.5-35B-A3B policy.Source: §4.3 · pp. 7–8
• Adding skill evolution raises average pass rate from 38.63 to 40.75, while full maintenance reduces pass rate by about 2% but shrinks the bank from 1252 to 676 skills.Maintenance therefore controls redundancy and prevents skill-bank growth while preserving most downstream utility.Source: Table 2 · p. 7, §4.3 · p. 8
• As related skills accumulate, merge and drop decisions become more frequent, and the skill bank becomes nearly stable instead of growing without bound.Add decisions increase when entering a new benchmark, whose distinct instructions are not covered by the existing bank.Source: §4.3 · p. 8, Figure 3 · p. 8

• CODESKILL learns to extract, evolve, and maintain reusable skills from coding-agent trajectories as a learnable management policy.Source: §5 · p. 8
• Reinforcement learning combines verifiable downstream feedback with rubric-based skill-quality judgments to optimize skill management.Source: §5 · p. 8
• CODESKILL improves over no-skill execution, prompt-based skill management, and memory-based baselines on EnvBench, SWE-Bench Verified, and Terminal-Bench 2.Source: §5 · p. 8

CODESKILL

• Natural-language instruction skills limit expressiveness because they cannot include executable scripts, APIs, tool definitions, or other structured resources.The evaluated environments use fixed tools and execution interfaces, but richer representations may provide stronger guidance when agents extend their toolsets.Source: CODESKILL · p. 9
• The action space handles one operation over one candidate skill at a time, preventing direct joint revision, splitting, or coordinated multi-skill updates.The restriction simplifies training, reward assignment, and skill-bank updates; future work could support multi-skill and multi-operation updates.Source: CODESKILL · p. 9

• Supervised fine-tuning teaches the action schema and basic skill-management behavior before reinforcement learning further optimizes the policy with downstream feedback.Training experiments use four H100 80GB GPUs.Source: A Training Details · p. 11

A.1 Supervised Fine-Tuning

• Trajectory sources are normalized into task context, interleaved reasoning/action/observation steps, and final outcome information despite differing logging formats.Sources include mini-SWE-agent SWE-smith rollouts and EnvBench ReAct-style trajectories.Source: §A.1 · p. 11
• Teacher targets cover extraction, evolution, and maintenance actions, including reusable-skill or skip decisions, skill revision, and add, merge, or drop operations.Evolution examples pair retrieved existing skills with new, failed, or partially successful trajectories exposing missing conditions, procedures, or failure modes.Source: §A.1 · pp. 11–12
• Qwen3.5-4B-Instruct is fine-tuned with LoRA on completion tokens to produce valid skill-management outputs before reinforcement learning.The training stage uses the qwen3_nothink template and teaches extraction, evolution, and maintenance behavior.Source: §A.1 · p. 12

A.2 Reinforcement Learning

• Three RL phases progress from skill extraction, to joint extraction and evolution, to the full extraction–evolution–maintenance action space.Skills and trajectories produced earlier are reused to construct later-stage prompts and candidate skill-bank entries.Source: §A.2 · p. 12
• 660 unique task instances are covered across the curriculum, which introduces extraction, evolution, and maintenance prompts in lifecycle order.Source: §A.2 · p. 12
• The curriculum runs for 500 GRPO update steps: 130 in phase 1, 120 in phase 2, and 250 in phase 3.Reward-side rollouts use the frozen Qwen3.5-35B-A3B coding policy, while rubric and alignment judgments use GPT-5.4-mini.Source: §A.2 · p. 12

B Reward Design

• Execution reward measures a generated skill’s improvement over an instance-specific no-skill baseline computed from four frozen-policy rollouts.The baseline is the average verifier score for the task instance.Source: B Reward Design · p. 12
• Rubric rewards use action-specific judges for extraction, evolution, maintenance, and behavior alignment, with the score equal to satisfied yes/no questions divided by all rubric questions.The resulting reward is normalized to [0, 1], while dimensions receive relative weight according to their number of questions.Source: B Reward Design · p. 12

C Experimental Details

• Each evaluation instance produces about one task-level and three event-driven candidate skills before online maintenance.The frozen downstream policy then solves tasks with retrieved skills injected as prior knowledge.Source: C Experimental Details · p. 12
• Separate dense retrieval indexes match task-level and event-driven skills to different query signals within the same benchmark and granularity.Retrieval documents contain each skill’s title, when_to_apply, and rules, while same-instance skills are filtered to prevent leakage.Source: C Experimental Details · p. 13
• Task-level skills are retrieved once from task and repository context, whereas event-driven skills are retrieved online from execution signals such as errors, actions, observations, and test outputs.Newly extracted or evolved skills are passed to maintenance, which may add, merge, or drop them.Source: C Experimental Details · p. 13
• The prompt-based baseline preserves CODESKILL’s pipeline, action space, retrieval, downstream policy, and decoding settings while replacing its learned manager with fixed prompt-based decisions.Comparisons also include no-skill execution and a subtask-level memory baseline.Source: C Experimental Details · p. 13
• The subtask-memory reproduction segments trajectories using rule-based action and execution boundaries, summarizes segments into memory items, and retrieves them into the downstream prompt.The reproduction is used because the original implementation is not publicly available.Source: C Experimental Details · p. 13

D Skill Examples

• CODESKILL focuses on natural-language instruction files containing a title, granularity label, when_to_apply condition, and actionable rules.Task-level skills encode procedures for tasks or related task families, while event-driven skills provide local guidance for recurring execution signals.Source: D Skill Examples · p. 13

E Prompts

• Figures 4 and 5 provide representative task-level and event-driven skills from the maintained bank, while Figures 6–9 specify templates for extraction, evolution, and maintenance.Source: Figure 4 · p. 14, Figure 5 · p. 15, Figure 6 · p. 16
• Figures 10–14 provide rubric templates for task-level extraction, event-driven extraction, skill evolution, merge decisions, and behavior alignment.The prompt appendix also includes the skill-bank maintenance merge judge.Source: Figure 10 · p. 20, Figure 11 · p. 21, Figure 12 · p. 22