SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles

• Growing libraries can return noisier context, crowd out useful entries, and retain skills that become counterproductive as the policy improves.The paper calls a previously stable skill whose fitness later falls below the retirement threshold delayed obsolescence.Source: §1 · pp. 1–2
• SkillForge pre-retires low-fitness seed skills, fine-tunes on surviving skills, then co-evolves the skill library and policy through a fitness-driven lifecycle.The lifecycle includes trial, active, stable, and retired states, with selective retirement, stabilization, and LLM-guided mutation.Source: §1 · p. 2, Figure 1 · p. 2
• SKILLFURNACE bundles 5k+ annotated records covering retirement-filtered SFT trajectories, fitness-annotated evolved libraries, and human-labeled retirement failure categories.Source: §1 · p. 2
• Up to 7.8% relative improvement in aggregate success rate over the strongest baseline, with a compact library, across three interactive agent benchmarks.The benchmarks span embodied control, web navigation, and Search-Augmented QA.Source: §1 · p. 2

• Skill-library methods and memory-augmented RL repositories are described as monotone: entries persist and cannot be corrected as the policy evolves.SkillForge instead retires low-fitness skills and mutates borderline ones alongside online RL.Source: §2 · p. 3
• LLM evolutionary optimization methods target static text artifacts against stationary held-out metrics in offline loops.Source: §2 · p. 3
• SkillForge evolves skills that augment agent context during interactive rollouts, where online policy training makes skill fitness non-stationary.This motivates retiring and reshaping skills as the policy changes.Source: §2 · p. 3

3.1 Group Relative Policy Optimization

• GRPO samples groups of episodes per task, computes binary-reward group-normalized advantages, and updates the policy using token-level clipping and a trajectory-level KL penalty.Action tokens are generation targets, while agent contexts and observations condition them.Source: §3.1 · p. 3

3.2 Skill-Augmented RL

• SkillRL retrieves the top-K skills whose applicability conditions match the task type and concatenates them into the agent context.Each skill consists of a name, behavioral principle, and applicability condition.Source: §3.2 · pp. 3–4
• The agent context combines the task and retrieved skills with a sliding window of recent observation–action pairs and the current observation; episodes receive binary reward.The resulting context–action pairs serve as SFT supervision units, and the cold-start checkpoint initializes RL and acts as its reference policy.Source: §3.2 · p. 4

4.1 SKILLFURNACE Initialization

• Seed skills are evaluated under the base model, low-fitness skills with sufficient use are pre-retired, and successful trajectories using surviving skills form the SFT data.The resulting checkpoint initializes RL and serves as its reference policy.Source: §4.1 · p. 4
• 0.67 average fitness among retired skills falls by 0.28 at retirement, and 57.2% first reach stable, empirically marking delayed obsolescence.Source: §4.1 · p. 4

4.2 Skill Fitness Tracking

• Fitness tracking preserves preliminary evaluation counts and credits each retrieved skill with the episode’s binary reward, recording usage and successful use.Source: §4.2 · pp. 4–5

4.3 Skill Lifecycle Management

• The lifecycle assigns skills trial, active, stable, or retired states; hysteresis requires a stable skill to demote before it can be retired.Trial skills must first accumulate enough uses to become active, protecting new skills from premature retirement.Source: §4.3 · p. 5
• Moderately fit, sufficiently used active skills enter a mutation pool, with lower-fitness skills more likely to be selected as parents for LLM-generated variants.Each child enters in the trial state.Source: §4.3 · p. 6

4.4 Training Integration

• Table 2 reports ALFWorld and WebShop performance percentages, distinguishing WebShop’s partial-credit Score from binary task-success Succ.Source: Table 2 · p. 7
• Each training step updates the policy using the current library, while forging updates the library from accumulated fitness that shifts with the policy trajectory.The forging cycle runs every E validation epochs, with retirement and mutation budgets and a skill cap.Source: §4.4 · pp. 6–7

5.1 Experimental Setup

• The evaluation covers ALFWorld, WebShop, and Search-Augmented QA, and compares context or memory methods, policy-optimizing RL, memory-augmented RL, and frontier reference models.The Search-Augmented QA setting spans seven datasets.Source: §5.1 · p. 7

5.2 Main Results

• 92.4% on ALFWorld and 78.4% on WebShop improve over SkillRL by 2.8% relative and 7.8% relative, respectively; Search-Augmented QA reaches 48.7 ± 0.3% micro-averaged accuracy, highest overall.The accuracy comparison spans the three evaluated environments.Source: §5.2 · p. 8, Figure 2 · p. 8
• Beyond step 75, SKILLFORGE reaches ∼90% success while SkillRL oscillates between ∼80–88% over the remaining training budget.Both methods rise from ∼60% to ∼70% between steps 50 and 65.Source: §5.2 · p. 8, Figure 2 · p. 8

5.3 Ablation Study

• Removing the full lifecycle produces the largest relative performance drops on every environment, peaking at 7.4% on WebShop.The single-component drops sum to more than the joint removal drop, consistent with sub-additive interactions.Source: §5.3 · pp. 8–9, Table 3 · p. 8
• Pre-retirement produces the largest single-component drops: 1.8% on ALFWorld, 4.5% on WebShop, and 2.5% on Search-Augmented QA.The ablation compares removing individual forging stages across all three environments.Source: §5.3 · pp. 8–9, Table 3 · p. 8
• On WebShop, pre-retirement and mutation lift the static 72.6% baseline to 75.9%; adding retirement reduces the library from 121 to 95 skills and closes the remaining 3.2% relative gap to 78.4%.Without retirement, the ALFWorld library grows beyond 130 skills, while SKILLFORGE outperforms it with 100.Source: §5.3 · p. 9

5.4 Forging Dynamics

• On ALFWorld, 200 steps yield 100 non-retired skills from 132 created, with 32 retired; the final checkpoint has 76 stable and 24 active or trial skills.The library cap binds near step 170, after which mutations and retirements balance.Source: §5.4 · p. 9, Figure 3 · p. 9
• On ALFWorld, the sub-δretire fitness fraction falls from 7.5% to 2.0%, while the middle-band fraction rises from 18.9% to 22.0% and mean fitness stays near 0.73.Retirement depletes the low-fitness tail, while mutation adds children with a neutral prior of 0.5.Source: §5.4 · p. 9, Figure 4 · p. 9

6 Conclusion

• SKILLFORGE co-evolves skills and policy through online RL, delivering up to 7.8% relative improvement over the strongest memory-augmented RL baseline across three environments while keeping the active library compact.The environments are ALFWorld, WebShop, and Search-Augmented QA.Source: §6 · p. 9
• SkillFurnace bundles 5k+ annotated records covering retirement-filtered SFT trajectories, evolved skill libraries with fitness annotations, and retirement events with human-annotated failure categories.The conclusion identifies procedural rigidity as the dominant mode by which once-useful skills become active interference.Source: §6 · p. 9
• Procedural rigidity emerges as the dominant mode by which once-useful skills turn into active interference, described as the empirical signature of delayed obsolescence.Source: §6 · p. 9

Training configuration

• The three environment-specific initial libraries contain 44 ALFWorld, 66 WebShop, and 41 Search-Augmented QA skills; evaluation uses 130 unseen ALFWorld tasks and 500 held-out WebShop tasks.Search-Augmented QA test-set sample sizes are reported in Table 10; training uses GRPO on 64 NVIDIA H200 GPUs.Source: B Implementation Details · p. 15, Table 5 · p. 15

Forging hyperparameters

• Pre-retirement uses δpre = 0.3, below runtime δretire = 0.4, because its unadapted base policy has systematically lower success rates than the RL policy.The pre-RL filter leaves skills with fewer than three uses in the initial library; runtime fitness uses a five-use warm-up and defaults to 0.5 when usage is too low.Source: B Implementation Details · p. 15

Lifecycle well-definedness

• Lifecycle guards are disjoint, and each cycle orders promotion and demotion before retirement, then stabilization before mutation.A skill can be demoted and retired in one cycle because demotion precedes retirement.Source: B Implementation Details · p. 15

Mutation weighting

• Mutation selects eligible parents with weight proportional to 1 − fi, giving at most a 2× preference to lower-fitness skills within the default [0.4, 0.7] range.The teacher prompt supplies skill metadata, fitness, and up to eight recent failure examples; its response is one JSON skill variant.Source: B Implementation Details · pp. 16–17

Threshold sensitivity

• Across the swept ALFWorld thresholds, overall success varies within ±1.4%; δretire = 0.4 lies near the peak between removing useful borderline skills and retaining low-fitness skills.The sensitivity sweep varies the retirement threshold and library cap around their defaults.Source: B Implementation Details · p. 16, Table 7 · p. 16

Compute resources

• A 200-step run is estimated at 24 hours for ALFWorld, 18 for WebShop, and 72 for Search-Augmented QA, totaling 7,296 GPU-hours across environments.Pre-retirement adds about six hours per environment; teacher-model calls run asynchronously during validation.Source: B Implementation Details · pp. 16–17

C SKILLFURNACE Dataset Details

• SKILLFURNACE contains 5,852 annotated records: 1,013 SFT trajectories, 4,521 library snapshots, and 318 retirement events across three environments.The SFT trajectories come from preliminary Qwen2.5-7B-Instruct rollouts, snapshots track evolving libraries, and retirement events are drawn from primary and additional training runs.Source: C SKILLFURNACE Dataset Details · p. 17
• The 1,013 SFT trajectories retain successful rollouts whose retrieved skills avoid the pre-retired set, with 412 from ALFWorld, 348 from WebShop, and 253 from Search-Augmented QA.The preliminary evaluation uses 500, 450, and 400 episodes per environment, respectively; released trajectories are ordered context–action pairs with retrieved-skill IDs attached.Source: C SKILLFURNACE Dataset Details · pp. 17–18
• Snapshots record each skill’s fitness, lifecycle state, usage, success, generation, and lineage; across environments, 46.4% of final non-retired skills are mutation descendants.Forging runs every 10 steps over 200 training steps, producing 21 snapshots per run; Table 8 defines the library-composition measures.Source: C SKILLFURNACE Dataset Details · p. 18, Table 8 · p. 18
• Retirement records capture fitness history, lifecycle transitions, lineage, and human-labeled failure modes; mean lifespan is 38.0 steps, with retirements spread across training.Overly rigid ordering dominates in every environment; Table 9 also reports environment-specific categories.Source: C SKILLFURNACE Dataset Details · p. 18, Figure 5 · p. 19, Table 9 · p. 19
• The three dataset components are released as linked JSON files with a schema document under CC-BY-4.0.Global skill IDs link retirement events to library snapshots and SFT-retrieved skills to step-0 snapshots.Source: C SKILLFURNACE Dataset Details · p. 19

D Search-Augmented QA Full Per-Dataset Results

• The reported 48.7% aggregate accuracy is a micro-average over 51,713 samples from the seven Search-Augmented QA test sets.The seven comprise in-domain NQ and HotpotQA plus five out-of-domain datasets.Source: D Search-Augmented QA Full Per-Dataset… · p. 19
• SKILLFORGE achieves the best or second-best accuracy on every Search-Augmented QA sub-dataset, with gains on both in-domain and out-of-domain splits.The largest absolute gains over the strongest baseline are +3.4 on Bamboogle and +0.7 on HotpotQA; Table 10 reports the seven datasets and micro-averaged overall accuracy.Source: D Search-Augmented QA Full Per-Dataset… · p. 19, Table 10 · p. 20

E.1 Deepest Forging Lineages

• Most mutation lineages stabilize within one or two generations, while some extend to nine; early variants are often retired as refined descendants survive.Figure 6 depicts root skills and descendants, marking retired variants red and stable variants blue.Source: §E.1 · p. 20, Figure 6 · p. 20

E.2 Failure Taxonomy Details

• Overly rigid ordering accounts for 74.5% of retirement events, while only 3.1% are labeled redundant with policy; 96.9% contain prescriptions inconsistent with successful rollouts.The second most common category is overly specific at 9.4%; contradicts-environment failures occur exclusively in Search-Augmented QA.Source: §E.2 · pp. 20–21, Figure 7 · p. 21

E.3 WebShop: Hidden-Constraint Verification

• In WebShop, a stable child targets a narrower task niche, while a variant referring to an absent Description tab is filtered at step 78.The stable child’s aggregate fitness is 0.70 versus its parent’s 0.84, but within-niche accuracy is what matters for the lifecycle.Source: §E.3 · p. 21

E.4 ALFWorld: Mutation Outcomes

• An ALFWorld mutation that adds pickup verification succeeds, whereas a rigid target-before-lamp ordering fails and is retired.The two variants came from the same parent skill.Source: §E.4 · p. 22

E.5 Search-Augmented QA: Query Reformulation Lineage

• In Search-Augmented QA, forging retires a rigid three-step query escalation while a simpler identity-verification variant survives.The surviving variant is described as simpler, more general, and more effective.Source: §E.5 · p. 22

F KL Regularization under a Library-Conditioned Context

• Because the library enters through context rather than policy parameters, the KL divergence remains evaluated at the same context as skills evolve.The adopted fixed reference uses β = 10^-3 as a soft regularizer; re-anchoring each forging cycle is presented as a compatible alternative.Source: F KL Regularization under a… · p. 22

G Limitations

• Each of the K retrieved skills shares one binary episode reward, so the estimator cannot distinguish individual contributions when multiple skills are co-retrieved.Applicability-conditioned retrieval is used to mitigate this approximation.Source: G Limitations · p. 22
• The highest co-retrieval frequency for any skill pair remains below 15% of episodes per environment, but shared credit may slow convergence where co-retrieval is unavoidable.In practice, each skill’s fitness estimate is dominated by episodes where it was retrieved alone.Source: G Limitations · p. 23
• All experiments use a 7B base model, and lifecycle dynamics at larger scales remain untested; mutation quality also depends on the external teacher’s instruction-following ability.A weak teacher may produce low-quality mutations that the trial gate must filter.Source: G Limitations · p. 23

H Discussion

• 38.0 training steps is the average retirement-event lifespan, compared with a 200-step training budget.The cumulative fitness estimate is intentionally conservative to avoid premature retirement after transiently hard task batches.Source: H Discussion · p. 23
• 64.8% of ALFWorld, 52.7% of WebShop, and 62.9% of Search-Augmented QA mutant skills survive the lifecycle.The three environments generated 88, 55, and 70 mutant skills, respectively.Source: H Discussion · p. 23
• 1.2–3.5% success drop without pre-retirement is attributed to SFT data quality, not library size.The resulting SFT policy is the starting point for subsequent retirement and mutation.Source: H Discussion · p. 23
• 132, 121, and 111 skills accumulate without retirement, versus active-library sizes of 100, 95, and 85 with the full method.The fixed forging schedule keeps total generated children largely invariant; retirement controls which remain active.Source: H Discussion · p. 23
• 1.9–5.7% absolute gains over SkillRL are significant at p < 10^-3 on ALFWorld, WebShop, and Search-Augmented QA.SkillForge’s mean success rates are 92.4 ± 0.5%, 78.4 ± 0.4%, and 48.7 ± 0.3% on those environments, respectively.Source: H Discussion · p. 24
• SkillFurnace supports research on skill-quality prediction and failure-aware generation with SFT trajectories, fitness-annotated libraries, and failure-annotated retirement events.The collection contains only these artifact types and introduces no risks beyond those inherent to LLM agents acting in interactive environments.Source: H Discussion · p. 24