PaperlayerView exact revision on arXiv

Source-linked AI summary

MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

Zefeng Wang, Minxi Yan, Jinhe Bi, Sikuan Yan, Volker Tresp, Yunpu Ma

arXiv:2607.05297v1cs.AI

TL;DR

Skill self-improvement leaves the procedure that edits task skills fixed, despite the diversity of tasks agents face. MetaSkill-Evolve co-evolves branch-local task skills and meta-skills on fast and slow timescales using the same pipeline. Across three benchmarks, it improves held-out test accuracy over the raw backbone by +23.54 / +16.09 / +1.92 points on OfficeQA / SealQA / ALFWorld.

  • Problem

    Existing skill evolution improves task skills while leaving the improvement procedure fixed, although branches may need to revise how they diagnose failures.

    Source: §1 · p. 2, §3.2 · p. 4

  • Method

    MetaSkill-Evolve attaches a meta-skill that parameterizes each branch’s five-step task-skill procedure and evolves it every H fast iterations.

    Source: §3.2 · p. 4, §3.5 · p. 5

  • Results

    +23.54 / +16.09 / +1.92 points in held-out test accuracy over the raw Gemma-4 31B backbone on OfficeQA / SealQA / ALFWorld.

    Source: §5 · p. 9

  • Takeaways & Limitations

    The results support treating an agent’s improvement policy as an object of the same search, within a bounded recursive setup.

    Source: §5 · p. 9

    Transfer to open-ended, long-horizon real-world tasks with noisier feedback is untested, and the five-agent pipeline and meta-update horizon remain fixed.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill $s$ and a branch-local meta-skill $m=(\psi,\sigma,\alpha,\pi,\varepsilon)$ whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.

1 Introduction

Reusable skills can help agents tackle open-ended tasks, but fixed skills cannot anticipate task diversity and existing self-improvement leaves the procedure that edits them unchanged. MetaSkill-Evolve instead evolves both the task skill and the branch-local procedure that improves it.

1 Introduction

  • Existing self-improvement rewrites task skills from failure traces but keeps the improvement procedure fixed, so branches cannot revise how they diagnose errors.The same fixed procedure is applied to different failures, such as misread tables and faulty calculations.Source: §1 · p. 1
  • Evolution search has two distinct targets: current skill utility U(s) and the meta-productivity P(m | s) of the policy generating descendants.A branch with a lower current score may be more promising if its policy reliably improves later scores.Source: §1 · pp. 1–2
  • MetaSkill-Evolve adds a branch-level meta-skill that co-evolves on a slower timescale through the same pipeline that rewrites task skills.The figure contrasts this with single-level evolution, where task skills evolve but the meta-process remains fixed.Source: Figure 1 · p. 2

RESEARCH QUESTION

MetaSkill-Evolve asks whether an improvement procedure can itself evolve alongside the task skills it produces. It uses a branch-local meta-skill and a slower update loop, and improves held-out accuracy over the raw backbone on OfficeQA, SealQA, and ALFWorld.

Framework

  • MetaSkill-Evolve assigns each branch a task skill, a meta-skill, and an iteration history, with task skills evolving every iteration and meta-skills every H iterations.The slow-loop update is driven by how much the branch’s descendants improved over the preceding H iterations.Source: RESEARCH QUESTION · pp. 2–3
  • The five meta-skill components parameterize diagnosis, cross-branch retrieval, child-budget allocation, edit proposal, and application and verification of approved edits.The same five-agent pipeline refines the meta-skill files as well as the task-skill files, without an added architectural component.Source: RESEARCH QUESTION · pp. 2–3

Evaluation

  • +23.54 / +16.09 / +1.92 points in held-out test accuracy over No-Skill on OfficeQA / SealQA / ALFWorld, respectively.The gains over Single-Level Evolution are +6.38 / +8.05 / +1.92 points, respectively.Source: RESEARCH QUESTION · pp. 2–3
  • Across both QA benchmarks, performance improves monotonically from No-Skill through Static and Single-Level to Ours; on ALFWorld’s held-out split, the backbone is near ceiling, margins are small, and the static skill is roughly neutral.Source: RESEARCH QUESTION · p. 3

Search

  • The frontier score combines task utility, meta-productivity, and branch novelty to favor productive, underexplored branches.Setting a score weight to zero can leave selection prone to stagnated high-utility nodes, a single lineage, or noisy single-child gains.Source: RESEARCH QUESTION · p. 3

2 Related Work

Related work studies reusable skills and ways to improve them, as well as recursive systems that modify code or prompts. MetaSkill-Evolve distinguishes itself by evolving branch-local skill files and the procedure that edits them.

Skill-based agents and skill self-improvement

  • Skill self-improvement uses execution feedback to rewrite reusable procedures, while a parallel line builds skill libraries rather than a single artifact.The cited work spans reflection, distillation, analyze–propose–evolve, reinforcement-learning rewrites, and trajectory- or memory-based methods.Source: §2 · p. 3

Recursive self-improvement

  • Prior recursive systems improve code or prompts under one global policy, whereas MetaSkill-Evolve recursively refines skill files under a branch-local, five-agent policy.The paper characterizes this as a bounded, one-level recursion that adds no model.Source: §2 · p. 3

Prompt, textual, and evolutionary optimisation

  • Prompt and program optimisation typically rewrites one artifact under a fixed rule; MetaSkill-Evolve lets the rule diverge across branches and evolve on its own timescale.Its frontier score also uses meta-productivity as a quality-diversity descriptor for improvement-policy diversity.Source: §2 · p. 3

3 MetaSkill-Evolve

MetaSkill-Evolve evolves task skills quickly and branch-local meta-skills more slowly, applying the same improvement pipeline to both. Its search uses a persistent graph and a frontier score to select branches for further evolution.

3.1 Problem Formulation

  • A task skill specifies procedures, tools, and heuristics for a task, and its utility is expected reward estimated by held-out validation accuracy.The reward scores predictions against reference outputs.Source: §3.1 · p. 4

3.2 Branch State and Meta-Skill

  • Each branch pairs a task skill with five meta-skill policies for diagnosis, retrieval, allocation, proposing, and executing edits; the same pipeline can refine those policies recursively.The task skill is the fast-loop target, while meta-productivity measures the expected per-child improvement over proposals.Source: §3.2 · p. 4

3.3 Evolution Graph and Frontier Selection

  • A persistent DAG preserves branch provenance and failed edits as retrieval inspiration, while its frontier score balances utility, trajectory quality, and visitation cooling.Neutral or regressing children cannot become parents, but remain available to the Retriever.Source: §3.3 · pp. 4–5

3.4 Fast Timescale: Task-Skill Evolution

  • The fast loop restores the selected branch’s snapshots, diagnoses its worst-scoring training example, proposes edits, and evaluates child gains on validation data.The Analyzer, Retriever, Allocator, Proposer, and Evolver respectively diagnose, retrieve inspirations, set a child budget, propose edits, and write and verify them.Source: §3.4 · p. 5

3.5 Slow Timescale: Meta-Skill Evolution

  • Every H fast iterations, the slow loop aggregates descendant outcomes into a synthetic meta-failure trace and applies the pipeline to branch-local meta-skill files.All five files are co-edited, and successive children build on the preceding child’s edits.Source: §3.5 · pp. 5–6

4 Experiments

Experiments compare skill-evolution strategies on three benchmarks using a shared frozen backbone and held-out tests. MetaSkill-Evolve leads overall, while ablations and horizon sweeps show domain-specific component and update-frequency effects.

4.1 Setup

  • OfficeQA, SealQA, and ALFWorld use a shared frozen Gemma-4 31B backbone, and the held-out test partition is never observed during evolution.The benchmark files are stratified into training, validation, and held-out test partitions.Source: §4.1 · pp. 6–7

4.2 Main Results

  • MetaSkill-Evolve raises held-out accuracy over No-Skill by +23.54 points on OfficeQA, +16.09 on SealQA, and +1.92 on ALFWorld.The QA progression is monotonic across No-Skill, Static, Single-Level, and MetaSkill-Evolve; ALFWorld begins near ceiling at 92.31%.Source: Figure 3 · p. 7, §4.2 · p. 7

4.3 Component Ablations

  • Every typed-component ablation underperforms the full system; α is most important on OfficeQA (55.32→35.58, −19.7 pts), while π dominates SealQA (45.26→36.84).The ablation figure covers the two QA benchmarks; ALFWorld ablations are reported in Table 3.Source: §4.3 · pp. 7–8, Figure 4 · p. 8

4.4 Meta-Update Horizon

  • H=2 is best on every benchmark; OfficeQA loses 9.1 points from H=2 to H=8 (48.94→41.35→39.84), while SealQA and ALFWorld are nearly flat through H=4.The sweep holds meta-updates fixed at three, so total iterations scale as 3H; H=2 differs from the five-iteration default used in Tables 1 and 3.Source: Figure 5 · p. 8, §4.4 · pp. 8–9

5 Conclusion

MetaSkill-Evolve recursively evolves branch-local task skills and meta-skills on fast and slow timescales using the same pipeline. It improves held-out test accuracy over the raw Gemma-4 31B backbone on OfficeQA, SealQA, and ALFWorld.

  • Held-out test accuracy gains over the raw Gemma-4 31B backbone are +23.54 points on OfficeQA, +16.09 on SealQA, and +1.92 on ALFWorld.The slow loop contributes +6.38, +8.05, and +1.92 points, respectively.Source: §5 · p. 9
  • Each branch carries a task skill and a five-component meta-skill that parameterizes the improvement pipeline.The meta-skill components are ψ, σ, α, π, and ε.Source: §5 · p. 9
  • The same pipeline refines task skills on a fast loop and meta-skills on a slower loop, without an additional model or training.Source: §5 · p. 9

Limitations

The evaluation covers three curated benchmarks, leaving transfer to open-ended, long-horizon tasks with noisier feedback untested. The implementation uses a five-agent pipeline, branch-specific meta-skill snapshots, progressive skill disclosure, and role-specific meta-skill prompts.

Limitations

  • Transfer to open-ended, long-horizon real-world tasks with noisier feedback is untested; the five-agent pipeline’s roles and wiring remain fixed, and meta-updates use a fixed horizon H.Source: Limitations · p. 9

A Five-Agent Pipeline Details

  • The pipeline assigns failure diagnosis to the Analyzer, candidate retrieval to the Retriever, search budgeting to the Allocator, edit proposals to the Proposer, and file edits and verification to the Evolver.All five agents share a frozen base model and load the skill catalog as context.Source: A Five-Agent Pipeline Details · p. 12

B Meta-Skill Representation and Skill-Catalog Disclosure

  • Each branch’s meta-skill components are stored and snapshotted with its task skill, so expansion restores that branch’s lineage-specific improvement policy.Agents first see a compact skill catalog and load full skill files or resources on demand.Source: B Meta-Skill Representation and… · p. 12

C Meta-Skill Prompt Templates

  • The initial diagnosis policy instructs the Analyzer to classify failures, distinguish skill-addressable failures from base-model limits, and assign a short failure tag.Source: C Meta-Skill Prompt Templates · p. 12

D Hyperparameter Sensitivity

The horizon sweep favors the tightest tested spacing, while cross-branch retrieval helps at moderate levels but becomes noisy when overly aggressive. The reported defaults set H=2 and limit cross-branch inspirations.

  • Defaults set horizon H=2 with five fast iterations (two meta-updates), and child budget K ranges from 1 to 3, initially 2, which the Allocator adapts per step.Source: D Hyperparameter Sensitivity · p. 12
  • Cross-branch retrieval uses probability 0.2, with same-branch and cross-branch inspiration limits of 3 and 2, respectively.Source: D Hyperparameter Sensitivity · p. 13
  • Moderate cross-branch retrieval consistently improves OfficeQA and SealQA, while very aggressive retrieval introduces irrelevant inspirations from structurally different failures.Source: D Hyperparameter Sensitivity · p. 13
  • H=2 is best on every benchmark; wider meta-update gaps can let stale rewrites overwrite productive edits.The sweep holds the number of meta-updates fixed at three, so the iteration budget scales as 3H.Source: Table 2 · p. 13

E Full Ablation Tables

The component ablations compare held-out test accuracy across the three benchmarks, with the Evolver always executing. On ALFWorld, the proposal policy is the dominant component, while removing cross-branch retrieval or freezing meta-updates returns accuracy to the single-level baseline.

  • Table 3 expands the component-ablation summary across all three benchmarks using the same partition as Table 1.Source: E Full Ablation Tables · p. 13, Table 3 · p. 13
  • On ALFWorld, removing the proposal policy π drops held-out test accuracy from 94.23 to 86.54, making it the dominant component.With −π, the Evolver consumes raw analysis instead of a structured proposal.Source: E Full Ablation Tables · p. 13, Table 3 · p. 13
  • On ALFWorld, removing cross-branch retrieval or freezing meta-updates each yields 92.31, exactly matching the single-level baseline.No cross-branch retrieval retains same-branch candidates; no meta-updates freezes the slow loop.Source: E Full Ablation Tables · p. 13, Table 3 · p. 13

F Evaluation Protocol: Native Held-Out Test

The evaluation partitions each benchmark by category into train, validation, and test sets. The evolution loop never sees the held-out test partition, so test accuracy measures generalization to unseen rows.

G Train/Validation Split Ratio Sensitivity

Split-ratio sweeps show that more training data yields modest gains, while larger validation sets improve ALFWorld results. The sweeps use a common held-out test subset for comparison.

G Train/Validation Split Ratio Sensitivity

  • All sweep configurations are rescored on a common held-out test subset formed by intersecting test items across runs, so absolute scores differ slightly from the main results.The common subsets are smaller than the benchmark sets used for the main results.Source: G Train/Validation Split Ratio… · p. 13
  • With validation fixed at 0.10, ALFWorld accuracy rises from 87.7 to 88.7 to 90.6 as training data increases, while SealQA and OfficeQA remain essentially flat within noise.The sweep uses five outer iterations; the reported SealQA and OfficeQA variation is ±1 to 4 items on n ≈87 to 107.Source: G Train/Validation Split Ratio… · p. 13, Table 4 · p. 14
  • On ALFWorld, enlarging validation from 0.10 to 0.25 improves the evolved skill by +2.0, +4.3, and +2.5 points at training ratios 0.05, 0.10, and 0.15.On high-baseline ALFWorld, a small validation set may expose too few failures for the analyzer and the diagnosis-to-selection cycle to act on; enlarging it makes more failures available and improves held-out performance.Source: G Train/Validation Split Ratio… · pp. 13–14, Table 5 · p. 14
Loading 2607.05297v1…