PaperlayerView exact revision on arXiv

Source-linked AI summary

Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agents

Jie Ma, Zhipeng Qian, Yufei Ma, Zihan Liang, Jiayi Ji, Qingpeng Cai, Ben Chen, Peng Jiang, Xiaoshuai Sun

arXiv:2610.11781v1cs.CV

TL;DR

Existing self-evolving skill libraries mainly accumulate new skills, leaving the validity and applicability of stored skills less tested. Skill-V represents skills as versioned, falsifiable contracts and uses environment outcomes to guide additions and boundary revisions. Across ALFWorld and WebShop, it reports high success rates while maintaining a compact library.

  • Problem

    Existing self-evolving skill libraries primarily accumulate skills, while stored skills may be inapplicable or have mis-specified operational boundaries.

    Source: §1 · pp. 1–2

  • Method

    Skill-V uses versioned, falsifiable skill contracts and environment outcomes to guide skill addition and boundary revision, with revisions validated against semantic constraints and observed evidence.

    Source: §1 · p. 2

  • Results

    Skill-V reports success rates of 95.3% on ALFWorld and 85.9% on WebShop while correcting boundary errors and avoiding severe library bloat.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Reliable skill evolution involves testing and correcting stored knowledge as well as expanding coverage.

    Source: §1 · p. 2

    The framework uses discrete text-based contract criteria; visual or continuous control settings require new multimodal evaluation mechanisms.

    Source: A LIMITATIONS AND BROADER IMPACTS · p. 12

Abstract

from arXiv · show

Interactive agents can turn experience into reusable skills, yet existing self-evolving skill libraries primarily improve by accumulating new knowledge. Failures may lead to new skills, while previously stored skills are less often revisited as new evidence arrives. However, growth alone does not ensure reliability, as a retrieved skill may be inapplicable under the current task conditions, and an existing skill may encode a mis-specified operational boundary. Reliable skill evolution therefore requires not only adding knowledge, but also testing and revising what is already stored. We introduce Skill-V, a verifiable self-evolving skill library. To make stored knowledge testable, we propose representing skills as versioned, falsifiable contracts that link semantic intent to observable behavioral criteria. We use environment outcomes to drive library evolution. Specifically, task failures motivate skill addition, while disagreements between contract evaluations and task outcomes guide revisions to existing skill boundaries. To validate these revisions, we require them to preserve protected semantic constraints and satisfy non-regression criteria for rubric-outcome metrics on historical replay evidence. Finally, we employ an applicability-aware filter to exclude candidates judged confidently inapplicable to the current task. Across ALFWorld and WebShop, Skill-V achieves success rates of 95.3% and 85.9%, respectively, while maintaining a more compact skill library than growth-oriented baselines. Applicability-aware filtering reduces incorrect skill invocations, and outcome-grounded revisions correct mis-specified skill boundaries without degrading performance on previously observed evidence. These results show that reliable skill evolution requires more than accumulating experience: the library must learn which knowledge to retain, when to revise it, and when it should be applied.

1 INTRODUCTION

Existing self-evolving skill libraries largely add skills without revisiting whether stored knowledge remains valid. Skill-V treats skills as testable contracts, using task outcomes to guide additions and boundary revisions; it achieves high success rates on ALFWorld and WebShop while maintaining a compact library.

1 INTRODUCTION

  • Existing self-evolving libraries primarily accumulate new content, while stored skills may be inapplicable or encode operational boundaries that are too broad or too narrow.The resulting problem is deciding when existing knowledge should be trusted, not only whether the library contains enough strategies.Source: §1 · pp. 1–2
  • Skill-V represents each skill as a versioned, falsifiable contract linking semantic intent and reusable procedures to applicability conditions and observable behavior.Rubric acceptance of a failed trajectory suggests an overly permissive boundary; rejection of a successful one suggests an overly restrictive boundary.Source: §1 · p. 2
  • Task failures motivate adding skills, while disagreements between rubric judgments and task outcomes identify existing contracts for revision.Candidate revisions must preserve semantic identity and avoid regressions on previously observed evidence.Source: §1 · p. 2
  • Skill-V retrieves semantically related skills, filters those confidently inapplicable to the task, then uses interaction trajectories and outcomes as evidence for library evolution.This links skill use with testing whether stored knowledge should be retained, expanded, or revised.Source: §1 · pp. 2–3
  • The paper frames self-evolving skill learning as jointly expanding knowledge coverage and maintaining the reliability of existing skills.Its approach combines contracts, outcome-grounded addition and revision, evidence-preserving validation, and applicability-aware use.Source: §1 · p. 3
  • Skill-V achieves success rates of 95.3% on ALFWorld and 85.9% on WebShop while correcting boundary errors and avoiding severe library bloat.The reported analyses also find that it mitigates catastrophic forgetting.Source: §1 · p. 3

2 RELATED WORK

Prior work develops external memory and evolving skill libraries, but approaches differ in whether they focus on acquisition, curation, or maintenance. Skill-V centers maintenance on testing stored skills against environment outcomes and revising boundaries when evidence warrants it.

Memory and skill reuse

  • Memory methods retain trajectories, reflections, or distilled knowledge, while Voyager organizes experience as executable skills; Skill-V focuses on testing and correcting procedural knowledge already in memory..Source: §2 · p. 3

Self-evolving skill learning

  • Recent self-evolving methods study skill acquisition, credit assignment, or curation, whereas Skill-V asks whether stored skills remain applicable and behaviorally well-specified as evidence arrives.The cited methods include SkillRL, Skill1, Skill-R1, and SkillOS.Source: §2 · p. 3

Reliable skill maintenance

  • Skill-V uses disagreements between skill rubrics and environment outcomes to identify boundary errors, admitting revisions only when semantic intent is preserved and observed counterexamples are not regressed on.The paper positions this as a way to mitigate catastrophic forgetting in lifelong learning agents.Source: §2 · p. 3

Self-evolving skill learning

  • Figure 2 depicts Skill-V's loop from retrieving and filtering task-relevant skills through policy interaction to library evolution using trajectories and outcomes.Failures add semantic skills; rubric–outcome disagreements provide counterexamples for contract revision, subject to deterministic validation.Source: Figure 2 · p. 4

3 METHOD

Section 3 frames Skill-V as a closed loop in which environment outcomes guide skill-library updates, while applicability filtering governs which retrieved skills reach the policy.

3 METHOD

  • Skill-V closes the loop between skill use and library evolution: outcomes drive skill additions or boundary revisions, and inference filters skills for task applicability.Source: §3 · p. 4

3.1 PROBLEM FORMULATION

  • The agent retrieves reusable strategies from an external, versioned library and conditions its policy on selected skills; experience can add, revise, or leave that library unchanged.Source: §3.1 · p. 4

3.2 SKILLS AS VERIFIABLE CONTRACTS

  • Each skill pairs semantic intent and procedure with a versioned rubric of applicability conditions and observable criteria, making its operational boundary testable while preserving its represented knowledge.Skills may begin with no executable rubric; an applicable-evidence-based proposal can initialize one if it preserves semantic intent and meets the contract specification.Source: §3.2 · pp. 4–5

3.3 OUTCOME-GROUNDED SKILL EVOLUTION

  • Failures can expose missing strategies and add semantic-form skills with executable rubrics pending, while rubric–outcome disagreements flag potentially mis-specified contracts for targeted revision.Source: §3.3 · pp. 5–6
  • Revisions must preserve semantic constraints and executable support, avoid worsening disagreement and false negatives or lowering balanced accuracy on replay evidence, and improve a metric when changing a hard criterion.Replay evidence is a bounded set of applicable trajectories; rejected candidates leave the current version in place.Source: §3.3 · p. 6

3.4 APPLICABILITY-AWARE SKILL USE

  • Semantic similarity retrieves expanded candidates, then deterministic checks and a confidence-thresholded judge remove only confidently inapplicable skills; uncertain candidates may remain before top-K selection.Source: §3.4 · p. 6

4 EXPERIMENTS

Experiments on ALFWorld and WebShop compare Skill-V with static and evolving skill-library baselines. Skill-V achieves 95.3% ALFWorld success and 85.9% WebShop success, while maintaining a compact library.

4.1 EXPERIMENTAL SETUP

  • Experiments use ALFWorld success across six task types and WebShop average score and success rate, comparing Skill-V with a broad baseline suite.The suite includes memory-augmented RL methods such as SkillRL, RetroAgent, and Skill1.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • Against GRPO, Skill-V raises ALFWorld overall success from 77.6% to 95.3% and WebShop success from 66.1% to 85.9%.Table 1 reports the main results across both domains.Source: Table 1 · p. 7, §4.2 · p. 7
  • Against SkillRL, Skill-V raises ALFWorld overall success from 89.9% to 95.3% and WebShop success from 72.7% to 85.9%.Skill-V stores approximately 200 entries, compared with Skill1’s 5,000-skill ALFWorld library, and exceeds Skill1 on Look, Clean, and Heat success.Source: §4.2 · pp. 7–8, Table 1 · p. 7

4.3 ABLATION STUDIES

  • With the policy fixed, ALFWorld success rises from 78.59% without external skills to 88.99% with the initial library and 95.30% with the evolved library.The evolved library includes newly acquired skills and revised existing contracts.Source: §4.3 · p. 8, Table 2 · p. 8
  • Full Skill-V reaches 95.3% success, versus 68.8% for Acquisition Only and 67.2% when revision is limited to failed trajectories.Failure-based revision addresses permissive contracts, while success-based revision addresses restrictive ones.Source: §4.3 · p. 8, Table 3 · p. 8
  • Without evidence-based validation, 9 of 46 revisions regress on replay metrics; full Skill-V’s gate raises overall task success from 75.0% to 95.3%.The gate also requires revisions to satisfy all remaining acceptance conditions.Source: §4.3 · pp. 8–9, Table 4 · p. 9
  • On ALFWorld, semantic retrieval alone yields 71.9% success, deterministic contract checks raise it to 89.1%, and omitting the LLM judge lowers full-method performance.The judge labels 29.6% of retrieved candidates confidently inapplicable during training.Source: Table 5 · p. 9, §4.3 · p. 9
  • Adding rubric-based process reward yields 93.8% success, below the default task-reward-only configuration’s 95.3%.The authors retain the simpler task-reward-only objective for the main experiments.Source: Table 6 · p. 9, §4.3 · p. 9

5 CONCLUSION

Skill-V shifts skill-library evolution from accumulating knowledge to maintaining it through outcome-grounded revisions and applicability filtering.

5 CONCLUSION

  • Skill-V achieves state-of-the-art performance across ALFWorld and WebShop while maintaining a highly compact library and circumventing catastrophic forgetting.Source: §5 · p. 9
  • Task failures drive skill addition, while contract-environment disagreements trigger targeted revisions to skill boundaries, with an evidence gate keeping updates non-regressive.Source: §5 · p. 9
  • During inference, an applicability-aware filter excludes confidently inapplicable candidates to prevent irrelevant skills from misguiding the policy.Source: §5 · p. 9

A LIMITATIONS AND BROADER IMPACTS

The paper identifies scope limits in contract evaluation, skill retirement, and historical replay, and outlines extensions for safety assessment.

A LIMITATIONS AND BROADER IMPACTS

B IMPLEMENTATION DETAILS

Implementation separates a training library updated from training trajectories from a synchronized, read-only evaluation copy.

B IMPLEMENTATION DETAILS

B.1 END-TO-END TRAINING PROCEDURE

The training loop uses environment rewards to optimize the policy and terminal outcomes to test the external skill library, which evolves every five training steps in the experiments.

B.1 END-TO-END TRAINING PROCEDURE

  • Environment rewards optimize the policy, while terminal outcomes test the external skill library; library evolution occurs every F = 5 training steps in the experiments.Source: §B.1 · p. 13
  • Contract diagnosis compares applicable rubric judgments with environment outcomes, and evidence selection prioritizes disagreements while retaining supporting successes and failures.Source: §B.1 · p. 13
  • Proposed contract updates undergo schema and traceability checks, semantic-preservation checks, executable-support checks, and evidence non-regression validation before activation.Source: §B.1 · p. 13

B.2 STORED SKILLS AS VERSIONED CONTRACTS

Skill-V stores each skill as a versioned contract separating its semantic identity from its procedure and executable rubric. Revisions adjust operational boundaries against replay evidence while preserving semantic constraints and requiring non-regression.

B.2 STORED SKILLS AS VERSIONED CONTRACTS

  • A versioned contract separates a skill’s semantic identity, reusable procedure, executable rubric, and evolution record.The semantic identity sets the boundary revisions cannot overwrite; the rubric translates intended behavior into criteria checked against trajectories.Source: §B.2 · pp. 13–14
  • Revisions are replayed on the same observed support before commitment, with skill intent held fixed.Table 8 illustrates tightening a boundary that admits failed behavior and relaxing one that rejects successful behavior.Source: §B.2 · p. 13, Table 8 · p. 15
  • Contract criteria are grounded in observable task, action, observation, or trajectory evidence rather than hidden policy reasoning.The evidence interface specifies criterion identifiers, severity, weight, match mode, source, operator, and pattern.Source: §B.2 · p. 14
  • Each committed version records its source skill and the experience motivating the change.The initial libraries use a compact sentence-rubric schema, normalized at load time into criterion groups; semantic constraints are not executable unless a conservative proxy is compiled.Source: §B.2 · pp. 14–15
  • The two revision cases show boundary calibration in either direction, with a new version committed only after replay evidence establishes non-regression.The examples distinguish adding an executable requirement from using successful replay to challenge an over-restrictive detector; matchers use a predefined declarative operator set, not model-generated code.Source: §B.2 · pp. 14–15

B.3 EXECUTABLE RUBRIC EVALUATION

Skill-V evaluates applicable contract criteria against executed trajectories and compares rubric judgments with environment outcomes. Those signals support failure-driven acquisition and targeted contract revision, subject to structural checks and replay-based non-regression criteria.

B.3 EXECUTABLE RUBRIC EVALUATION

  • For each task, Skill-V evaluates valid contracts’ applicability and then checks their execution criteria against the trajectory.When no applicable contract has executable criteria, the aggregate score and judgment are undefined and no rubric-outcome evidence is generated.Source: §B.3 · pp. 15–16

B.4 FAILURE-DRIVEN SKILL ACQUISITION

  • Failed training trajectories prompt proposals for reusable skills, which enter the library as semantic contracts before executable criteria are compiled.Pending skills can be retrieved and prompted to the policy, but cannot produce rubric judgments until compilation succeeds; rejected compilation leaves them pending.Source: §B.4 · p. 16

B.5 OUTCOME-GROUNDED CONTRACT DIAGNOSIS

  • Comparing rubric judgments with task outcomes identifies disagreement cases, which the updater prioritizes alongside selected agreement cases.It retrieves relevant contracts using failed criteria, semantic overlap with the task, and pending compilation as relevance signals.Source: §B.5 · p. 16, Table 9 · p. 16
  • Revisions may change applicability, procedures, or executable criteria, but must copy every protected semantic constraint exactly.This permits repair of a skill’s operational boundary while prohibiting silent replacement of its identity.Source: §B.5 · p. 17

B.6 DETERMINISTIC REVISION VALIDATION

  • Before replay, proposals undergo checks of schema, criterion identifiers, matchers, weights, traceability, source identity, and version order.Validated proposals are then tested on the observed evidence pool associated with the update.Source: §B.6 · p. 17
  • An evidence-preserving revision must lower or maintain disagreement and false-negative rate, raise or maintain balanced accuracy, and retain executable support.Changing a hard criterion additionally requires strict improvement in at least one metric; accepted revisions replace the contract in place, while rejected ones leave it unchanged.Source: §B.6 · p. 17

B.7 SEMANTIC RETRIEVAL AND APPLICABILITY FILTERING

  • Semantic retrieval is followed by applicability judgments; only confidently inapplicable candidates are filtered before selection within the original skill budgets.The judge removes NOTAPPLICABLE predictions only at confidence ≥0.8; applicable and uncertain candidates remain eligible, and service failure falls back to embedding Top-K.Source: §B.7 · p. 17

C.1 ENVIRONMENTS AND METRICS

Evaluation covers embodied household tasks in ALFWorld and online shopping in WebShop, using task success and, for WebShop, average task score. Selected step-150 checkpoints achieve 95.3% ALFWorld success and 92.5 average score with 85.9% success in WebShop.

C.1 ENVIRONMENTS AND METRICS

  • ALFWorld evaluates embodied household tasks using success rates for seven task categories and the complete set, with success derived from positive terminal reward.WebShop evaluates product search and purchase against natural-language requests, reporting average task score on a 0–100 scale and overall success; success for contract diagnosis requires normalized score ≥0.99.Source: §C.1 · p. 18
  • 95.3% overall success rate for ALFWorld; WebShop reports an average score of 92.5 and an 85.9% success rate at the selected step-150 checkpoints.The results use the selected checkpoint rather than a separate peak checkpoint for each metric.Source: §C.1 · p. 18

C.2 TRAINING AND EVALUATION CONFIGURATION

The reported runs use a shared training configuration and initially compiled skill libraries, with external models handling retrieval judgments and library maintenance. At step 150, accepted revisions coexist with acquisition-driven library growth, and checkpoint diagnostics characterize rubric-outcome agreement.

C.2 TRAINING AND EVALUATION CONFIGURATION

  • Both domains use GRPO with eight rollouts per instruction, a 10^-6 learning rate, 0.01 KL coefficient, and 0.1 invalid-action penalty.Policies start from the corresponding 7B SFT checkpoint and train across eight GPUs; Table 10 gives domain-specific settings and per-update budgets.Source: §C.2 · p. 18, Table 10 · p. 18

C.3 INITIAL SKILL LIBRARIES

  • Initial libraries contain 55 ALFWorld entries and 66 WebShop entries, including separately stored common mistakes.The retrievable contracts are compiled offline; unsupported semantic requirements remain protected but are disabled as executable proxies, avoiding claims that deterministic matching fully verifies richer semantics.Source: §C.3 · pp. 18–19

C.4 EXTERNAL MODELS

  • Qwen3-Embedding-0.6B handles semantic retrieval, while deepseek-v4-flash handles applicability judgments, skill proposals, and rubric compilation or revision.The separate 7B policy is the only model updated by gradient-based reinforcement learning; updater responses are locally checked and cannot bypass validation gates.Source: §C.4 · p. 19

D.1 LIBRARY EVOLUTION STATISTICS

  • ALFWorld’s step-150 library had 162 general skills and 32 task-specific skills, with 10 accepted revisions; WebShop held 153 entries and 14 accepted revisions.Accepted revisions replace versions in place, so ALFWorld’s 150-entry net growth came entirely from acquisition; revisions themselves do not increase library size.Source: §D.1 · p. 19, Table 11 · p. 19

D.2 CHECKPOINT DIAGNOSTICS

  • ALFWorld’s low aggregate disagreement at step 150 describes rubric-outcome consistency, not causal policy quality.The outcomes are trajectory-level and multiple contracts can apply to one task; these diagnostics do not replace standard task-success metrics.Source: §D.2 · p. 19, Table 12 · p. 20

E PROMPT TEMPLATES; E.1 APPLICABILITY JUDGMENT

  • Runtime prompts substitute tasks, candidate skills, trajectory cases, diagnostics, and current contracts; applicability judgments consider task objective and declared scope.The judge labels each candidate applicable, not applicable, or uncertain, without requiring its precondition to hold in the current state.Source: E PROMPT TEMPLATES · p. 20, §E.1 · p. 20

E.2 FAILURE-DRIVEN SEMANTIC ACQUISITION

  • Failure-driven acquisition prompts the external model with up to five failed trajectories and existing skill titles to propose actionable reusable skills.Each proposal includes a title, principle, and explicit application condition.Source: §E.2 · p. 20

E.3 CONTRACT COMPILATION AND REVISION

  • Contract revision uses outcome disagreements to target operational boundaries while preserving protected semantic constraints and requiring replay-based evidence preservation.Rubric-pass/task-fail cases may call for added or conditionalized criteria; rubric-fail/task-pass cases may call for narrower applicability or relaxed soft criteria.Source: §E.3 · p. 20
Loading 2610.11781v1…