PaperlayerView exact revision on arXiv

Source-linked AI summary

From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement

Leijun Zhou, Yanke Yu, Zhihao Liu, Qu Xiang, Tiancheng Hu, Yiru Dong, Yihan Wu, Yangxi Xiang, Xuejun Wu, Xi Chen, Buyue Qian, Yaowei Zheng, Junhao Hu

arXiv:2610.14625v1cs.AI

TL;DR

Recursive self-improvement research has many methods and broad classifications, but less systematic guidance on how design choices interact or how specific learning mechanisms can inspire new ideas. This survey organizes methods in a nine-dimension taxonomy, examines learning-mechanism connections, and reviews evaluation settings. It concludes that these analyses can help identify limitations and suggest research directions, while noting that evaluation conditions complicate comparisons.

  • Problem

    Existing broad classifications describe RSI methods but do not fully explain their design choices and interactions, and surveys offer less systematic guidance on reusing specific deep-learning mechanisms.

    Source: §1 · p. 3

  • Method

    The survey develops a nine-dimension taxonomy and examines traditional deep-learning mechanisms and evaluation settings as sources for RSI research ideas.

    Source: §1 · pp. 3–4

  • Results

    The survey places existing methods in a shared design space and uses specific learning mechanisms to explore new RSI research ideas.

    Source: §1 · p. 4, §6 · p. 19

  • Takeaways & Limitations

    Examining taxonomy dimensions and their interactions can reveal limitations of current mechanisms and suggest new design directions.

    Source: §3 · p. 6, §6 · p. 19

    Comparisons across methods require accounting for evaluation mode, benchmarks, example order, and feedback, which vary across studies.

    Source: §5.4 · p. 19

Abstract

from arXiv · show

Recursive self-improvement (RSI) offers a route to AI systems that participate in their own continued development. Recent methods allow language agents to retain changes to their parameters, memory, skills, and code for subsequent execution and improvement. We examine how these methods can inform new research ideas through two complementary perspectives. First, we introduce a unified taxonomy with nine design dimensions organized into four groups: update targets, update organization, update schedule and scope, feedback and selection. This taxonomy places existing methods in a shared design space, where examining individual dimensions and their interactions can inspire new research ideas. Second, we examine how existing RSI methods draw on traditional deep learning, for example through optimization, regularization, data augmentation, continual learning, and adversarial learning. We develop further ideas by examining more specific learning mechanisms and drawing on evolutionary computation. We also review how existing benchmarks assess self-improvement under online, offline, and offline-to-online settings. Together, these perspectives help researchers develop new RSI research ideas from existing designs and learning mechanisms. RSI papers and resources are available at https://prism-shadow.github.io/awesome-rsi/.

1 Introduction

RSI systems modify and retain changes for subsequent execution and improvement, while broad classifications leave design choices and interactions underexplained. The survey frames a unified taxonomy and analysis of learning mechanisms as sources of new RSI research ideas.

1 Introduction

  • Existing surveys [6] [7] classify methods by targets, autonomy, or loop closure, but broad categories do not explain design choices or their interactions.Methods that revise agent code can differ in who proposes changes, how earlier versions are reused, and what feedback determines retention.Source: §1 · p. 3
  • The unified taxonomy organizes nine design dimensions into four groups and places existing methods in a shared design space for comparison and idea generation.The groups are update targets, update organization, update schedule and scope, and feedback and selection.Source: §1 · pp. 3–4
  • The survey reviews online, offline, and offline-to-online benchmarks and evaluation settings to explain what they measure and motivate shared protocols.It also presents an accompanying website for browsing and filtering methods along the taxonomy dimensions.Source: §1 · p. 4
  • The survey examines how RSI adapts traditional deep-learning mechanisms, then develops further research ideas from specific mechanisms and evolutionary computation.Examples of existing connections include optimization, regularization, data augmentation, continual learning, and adversarial learning.Source: §1 · p. 4

2 Background

The background defines agentic systems and their task-execution cycle, then formalizes recursive self-improvement as a loop that retains changes for later rounds. Its nine settings describe update timing and scope, candidate construction, feedback, and retention.

2 Background

  • RSI requires an agentic system to modify itself, retain changes, and use the updated system on subsequent tasks and further self-modification.The background first introduces the agentic system and task-execution loop, then formalizes the self-improvement loop and connects it to the taxonomy.Source: §2 · p. 4

2.1 Agentic systems

  • An agentic system combines a language model with a harness of tools, context, memory, skills, and executable code that coordinates model calls and actions.The environment is the external system the agent can observe and affect.Source: §2.1 · p. 4

2.2 The ReAct Loop

  • The ReAct Loop interleaves reasoning and acting; repeated tool use produces observations and execution traces that can inform later system changes.The harness assembles model-call context and executes requested tool operations during each iteration.Source: §2.2 · p. 4

2.3 The RSI Loop

  • The RSI loop connects self-modification to later execution and improvement through candidate generation, feedback, and retention of selected changes.Retained resources that are not replaced remain available, and successive rounds can support further candidate construction.Source: §2.3 · pp. 5–6
  • Update schedule and scope specify whether improvement is offline, online, or offline-to-online, how often updates occur, and where retained resources are reused.Triggers can include a step, event, complete trajectory, or batch; experience scope distinguishes task-specific reuse from reuse across tasks or domains.Source: §2.3 · p. 5
  • Candidate generation draws on current resources, task material, execution records, and feedback; its settings specify the modified artifact, updater roles, and use of earlier versions.Candidates may revise memory, context, skills, or code, or update model parameters using generated learning material [13] [14] [15] [5].Source: §2.3 · p. 5
  • Candidate feedback assesses a proposed configuration using resources, execution records, or further results, with settings specifying its source and form.Feedback can include test outcomes, scores, or textual assessments.Source: §2.3 · pp. 5–6
  • Retention uses feedback and an acceptance criterion based on artifact validation, task results, or aggregate performance to decide which changes enter the next configuration.The loop’s update procedure U covers a complete improvement round, and t counts rounds while D collects the nine design settings.Source: §2.3 · pp. 5–6

3 A Unified Taxonomy of Agent Self-Improvement

The taxonomy organizes RSI methods as combinations of design choices across four groups and nine dimensions. Locating methods in this shared space supports comparison, analysis of interactions, and development of new research ideas.

3.1 Overview of the unified taxonomy

  • The four groups are update targets (artifact), update organization (updater, topology), schedule and scope (mode, frequency, experience scope), and feedback and selection (source, type, acceptance).Source: §3.1 · p. 6

3.2 Update targets

  • Update targets distinguish model weights from non-parametric resources such as context, memory, skills, harness code, and other retained artifacts.A method can update several artifacts together.Source: §3.2 · p. 6

3.3 Update organization

  • Update organization captures who forms a change—Self, Teacher, or Joint—and how new versions depend on earlier ones through Sequential, Tree, or Graph topologies.Source: §3.3 · p. 9

3.4 Update schedule and scope

  • Update schedule and scope distinguish offline, online, and offline-to-online modes; step, event, trajectory, or batch triggers; and general versus specialized reuse.Source: §3.4 · pp. 9–10

3.5 Feedback and selection

  • Feedback and selection classify guidance by source, form (score or non-score), and retention basis, including validation, performance metrics, or no separate validation.Source: §3.5 · p. 10

3.6 Ideas from the unified taxonomy

  • A logging edit can enable later failure repair yet be rejected by immediate-score acceptance, motivating criteria that value contributions to subsequent self-improvement.Source: §3.6 · p. 11

4 Insights from Traditional Deep Learning

The section connects established learning mechanisms to how RSI systems revise skills, memories, models, and code, then develops new research ideas from more specific mechanisms in deep learning, broader machine learning, and evolutionary computation.

4.1 Connections in Existing RSI Methods

  • SkillAdam uses revision history and performance changes to guide what to edit and how much, while TextGrad propagates task feedback backward through a computation graph to suggest component-level revisions.TextGrad uses a language model to generate upstream suggestions, then an optimizer revises selected variables such as instructions and code.Source: §4.1.1 · p. 11, §4.1.2 · p. 12
  • MemRL uses task rewards to update memory utility, SEAL improves the generation of self-edit materials through downstream performance, and RISE constructs distillation targets from the model’s own training trajectory.These methods apply feedback to memory selection, update production, or subsequent supervision, respectively.Source: §4.1.3 · pp. 12–13, §4.1.4 · p. 13, §4.1.5 · p. 13
  • RSI regularization controls revision complexity or masks skill content, aiming to limit overfitting to improvement tasks or dependence on particular instructions.RRSI limits independent edits and removes components without gains; SkillEvoReg proposes revisions using partially masked skills.Source: §4.1.6 · p. 13
  • TTCS generates task variants and learns from answer agreement, while SEEK uses historical examples to screen new skill revisions for improvement without excessive regression.The first also updates its question synthesizer using solver feedback; the second retains examples as an acceptance check.Source: §4.1.7 · p. 14, §4.1.8 · p. 14
  • SkillProx simplifies skills using measured utility, DMoA learns agent routing from predictive entropy, and ACE couples code generation with adversarial test generation.These examples apply pruning, ensemble learning, and adversarial interaction to RSI while keeping different components fixed or revisable.Source: §4.1.9 · p. 14, §4.1.10 · p. 14, §4.1.11 · p. 15

4.2.1 Further Developments in Traditional Deep Learning

  • The lottery ticket hypothesis motivates testing whether a smaller initial set of agent resources can support comparable later capabilities, while adversarial masking could expose skill dependencies and prompt alternative procedures.A masking agent could hide content that a web research agent relies on, then use resulting failures to encourage other extraction approaches.Source: §4.2.1 · pp. 15–16

4.2 From Established Methods to New RSI Ideas

  • Continual backpropagation motivates reinitializing low-utility neurons to preserve learning of new features, while Freeze-Thaw Bayesian Optimization suggests using predicted performance and uncertainty to allocate rounds across improvement branches.Paused branches would retain agent state and revision history for later continuation.Source: §4.2.1 · p. 16, §4.2.2 · p. 16

4.2.3 Insights from Evolutionary Algorithms

  • Evolutionary modularity suggests separating procedures that change independently so an agent can adapt retrieval or output skills while retaining other reusable procedures.The example separates changing document retrieval from changing report production as task requirements vary.Source: §4.2.3 · p. 16

5 Evaluation and Benchmarks

The section organizes RSI evaluation by when improvement occurs relative to testing: online, offline, or offline-to-online. It reviews representative benchmark designs and notes that differing tasks, ordering, feedback, and scoring conditions complicate comparisons, motivating shared protocols.

5 Evaluation and Benchmarks

  • Benchmarks are grouped as online, offline, or offline-to-online according to whether improvement occurs during evaluation, before it, or in both phases.The review compares task organization, available feedback, and evaluation goals within each setting; Table 1 compares representative studies.Source: §5 · p. 17, Table 1 · p. 17

5.1 Online evaluation

  • Online benchmarks examine continued learning through task streams, related task families, changing task sequences, and repeated research experiments.Examples test reuse of accumulated memory or skills, retention across domains, and successive revisions informed by experimental feedback.Source: §5.1 · p. 18

5.2 Offline evaluation

  • Offline benchmarks test whether improvement acquired on training tasks transfers to held-out tasks, including specified procedural combinations, revised harnesses, and exploration-derived information.Designs include feedback on training responses, unseen task evaluation, and frozen harness code at test time.Source: §5.2 · p. 18

5.3 Offline-to-online evaluation

  • Offline-to-online benchmarks combine resources acquired before evaluation with further updates during testing, varying how closely later tasks relate to prior ones.AWM learns workflows from training examples and test tasks, while Mem2Evolve initializes memory from benchmark tasks or GAIA before continued use.Source: §5.3 · p. 19

5.4 Toward unified evaluation

  • Reported gains are difficult to compare directly because studies vary in evaluation mode, benchmark, example order, and feedback conditions.The section recommends shared task organization, feedback, and scoring rules within each mode, plus reporting performance and gain over the same pre-improvement system.Source: §5.4 · p. 19

6 Conclusion

The conclusion presents the survey’s taxonomy and learning-mechanism review as complementary routes to new RSI research ideas.

6 Conclusion

  • The survey reviews RSI methods through a unified taxonomy and examines their connections to traditional deep learning.It also extends the search for research ideas to more specific learning mechanisms and beyond traditional deep learning.Source: §6 · p. 19
  • The taxonomy places methods in a shared high-dimensional design space for understanding and comparison.Examining individual dimensions and their interactions can reveal limitations of current mechanisms.Source: §6 · p. 19
  • Concrete examples show how theories and mechanisms from established learning methods can suggest new RSI design directions.The conclusion includes inspiration beyond traditional deep learning.Source: §6 · p. 19

A Literature Catalogue

The appendix collects the RSI methods, benchmarks, and software systems discussed in the survey.

A Literature Catalogue

  • The appendix gathers methods, benchmarks, and software systems discussed in the survey.The catalogue applies the unified taxonomy to methods and summarizes the reviewed evaluation settings for benchmarks.Source: A Literature Catalogue · p. 31
  • The method catalogue uses the unified taxonomy introduced in Section 3.Source: A Literature Catalogue · p. 31
  • The benchmark catalogue summarizes evaluation settings reviewed in Section 5.Source: A Literature Catalogue · p. 31

A.1 Method papers

Table 2 classifies method papers under the unified taxonomy, recording their choices across its dimensions and briefly describing update mechanisms.

A.1 Method papers

  • Table 2 records each method’s choices across the nine dimensions and briefly describes its update mechanism.The table presents method papers and their classifications under the unified taxonomy.Source: §A.1 · p. 31, Table 2 · p. 31
  • The table uses multiple labels to capture the resources and variants studied in each paper.Its resource labels include parameters, harness code, context, memory, skills, and other artifacts.Source: §A.1 · p. 31
  • Its headings include method, targets, organization, updater, topology, schedule and scope, mode and frequency, source and type, acceptance, and update mechanism.Source: §A.1 · p. 32

AlgoEvo

The AlgoEvo entry describes persistent experience cards that support consolidation of cross-task patterns into reusable design skills for later discovery.

AlgoEvo

  • The [29] entry lists context, memory, and skills as targets and identifies a self updater.Source: AlgoEvo · p. 32
  • Persistent experience cards retain experience for later use.Source: AlgoEvo · p. 32
  • Patterns supported across multiple tasks are consolidated into reusable design skills for subsequent discovery.Source: AlgoEvo · p. 32

AutoTTS

The supplied method entries describe memory-based reuse and an agent that discovers test-time scaling controllers using execution-trace feedback.

AutoTTS

  • An LLM agent discovers test-time scaling controllers for other LLMs using an environment built from pre-collected reasoning trajectories and probe signals.Source: AutoTTS · p. 32
  • Execution-trace feedback is used to revise controller programs, which transfer to held-out benchmarks and model scales.Source: AutoTTS · p. 32
  • Another entry retrieves completed web trajectories to guide later tasks.Source: AutoTTS · p. 32
  • One memory entry separates planning and execution memories, attributes task outcomes before storing experience, and updates the relevant bank for later tasks.Source: AutoTTS · p. 32
  • That memory approach guides later tasks without changing model weights.Source: AutoTTS · p. 32

COBRA-Skills

The COBRA-Skills entry allocates skill evaluations with a contextual bandit and evolves a skill population using observed task rewards.

COBRA-Skills

Continual Harness

The Continual Harness uses environment and LLM feedback without validation to refine prompts, subagents, and skills. It also updates memory during episodes and can optionally co-learn model weights from process-reward-labeled rollouts.

Continual Harness

Cradle

This part surveys methods for agent self-improvement through updates to memory, context, skills, tools, harnesses, prompts, code, policies, and models, guided by accumulated experience, feedback, validation, or selection. It also covers benchmarks for evaluating continual improvement across online, offline, and mixed settings, and software systems for agent self-improvement.

Cradle / Dynamic Cheatsheet / EvoSkill-GUI

  • Cradle [88], Dynamic Cheatsheet [94], and EvoSkill-GUI [100] preserve observations, strategies, or trajectory critiques as reusable skills and memory for later interactions.Cradle creates executable skills from reusable procedures; EvoSkill-GUI revises skill packages during execution and after trajectory-level critique, retaining recovery experience.Source: Cradle · p. 32, Dynamic Cheatsheet · p. 33, EvoSkill-GUI · p. 34

DarwinX / DemoEvolve / Ecdysis / EnvHarness / EvolveNet

  • DarwinX [89], DemoEvolve, Ecdysis, EnvHarness, and EvolveNet revise harnesses or environments, using repeated evaluation, training-set gains, solvability checks, or regression tests to control retention.DemoEvolve uses competent human trajectories to diagnose sparse, noisy failures; EvolveNet combines evidence-backed changes with explicit scopes before broadcasting a version.Source: DarwinX · p. 32, DemoEvolve · p. 33, Ecdysis · p. 34

DCE + SRCL / DMoA

  • DCE + SRCL co-evolves a student and privileged self-teacher through on-policy distillation and verified concise rewrites, while DMoA learns routing weights from output entropy with language models frozen.The rewritten material is used to improve subsequent reasoning rounds; DMoA applies its learned agent selection to later queries.Source: DCE + SRCL · p. 33, DMoA · p. 33

Designer-RSI

  • Designer-RSI [37] admits procedural-skill changes only when matched replays improve outcomes without regressions, while its coding-agent archive and skill populations retain useful descendants and combine parent capabilities.The skill-evolution variant also adapts skill-generation operators using measured utility.Source: Designer-RSI · p. 33

Dream-RSI

  • Dream-RSI [93] replays recorded discovery trees to revise exploration-policy code, then deploys the selected policy to guide fresh searches and expand improvement history.Replaying the trees makes policy evaluation and revision cheaper.Source: Dream-RSI · p. 33

Escher-Loop / Evo-Harness

  • Escher-Loop evolves populations of task and optimizer agents, with optimizers revising task programs and themselves and task-agent scores guiding optimizer evolution; Evo-Harness compiles useful one-shot experience into structured skills.Evo-Harness evolves a shared harness using separate local workloads.Source: Escher-Loop · p. 34, Evo-Harness · p. 34

EvolveTrade / EvoOntology

  • EvolveTrade periodically rewrites a trading agent’s textual policy from decisions, tool calls, rationales, and returns, while EvoOntology [99] evolves data-agent content, schema, and tools from grounded probes and traces.EvoOntology retains changes through paired validation.Source: EvolveTrade · p. 34, EvoOntology · p. 34

EvoTrainer / Experience Funnel

  • EvoTrainer promotes policy-training versions through performance and behavioral checks, while Experience Funnel alternates validation-gated experience-state updates with policy distillation and carries state and student weights forward.EvoTrainer also stores reusable lessons and skills from rollouts and diagnostic code.Source: EvoTrainer · p. 35, Experience Funnel · p. 35

ExpGraph

  • ExpGraph learns retrieval from an experience graph and improves over no-memory runs while leaving the task executor frozen.Source: ExpGraph · p. 35

ForeDreamer

  • ForeDreamer evolves forecasting lessons and evidence-processing instructions and tools, validating updates and reusing tools across procedural branches.Source: ForeDreamer · p. 35

G-Zero

  • G-Zero co-evolves a GRPO-trained Proposer and DPO-trained Generator: the Proposer finds queries and hints, while the Generator internalizes them without an external judge.Source: G-Zero · p. 35
  • G-Zero pools code changes, traces, and outcomes across archived agents to evolve harnesses, revise prompts from execution feedback, and recombine candidates through Pareto-based search.Source: G-Zero · p. 35

Growing Harness; Gödel Agent

  • Growing Harness repairs a shared executable harness with failure-localized edits and rolls back changes that harm held-out success; Gödel Agent rewrites its executable logic, including its self-modification logic, and selects better descendants.Source: Growing Harness · p. 35, Gödel Agent · p. 35

Harness-of-Harness; Harness-Zero; HarnessBank

HarnessPAI; HarnessX

  • HarnessPAI revises robot programs from rollout videos and diagnostics, validating repairs across scenes and saving successful ones as skills; HarnessX evolves typed primitives from traces and can turn trajectories into training signals.Source: HarnessPAI · p. 36, HarnessX · p. 36

HyperSkill; MediSkill-Evo

  • HyperSkill revises reusable context files and retrieval code with historical context-engineering skills, then validates candidate context; MediSkill-Evo writes clinical experience to typed knowledge banks after checks and constrains later actions with validated knowledge [119].The HyperSkill meta-agent combines historical skills for a base agent; MediSkill-Evo checks provenance, scope, and process before retaining knowledge.Source: HyperSkill · p. 37, MediSkill-Evo · p. 37

Mem2Evolve; Memp

  • Mem2Evolve co-evolves experience and asset memories, using prior lessons to create tools or expert agents whose use generates new experience; Memp distills trajectories into revisable step-level instructions and procedures [120].Memp continuously adds, corrects, and deprecates entries as experience accumulates.Source: Mem2Evolve · p. 37, Memp · p. 38

MemRL; Mendel Gödel Machine

  • MemRL [3] updates memory utility from task rewards and combines it with semantic relevance for retrieval, while Mendel Gödel Machine adds reaction-norm mutation and cross-lineage hybridization to archive search.The latter uses evidence from multiple tasks or another lineage to improve coding-agent descendants.Source: MemRL · p. 38, Mendel Gödel Machine · p. 38

Meta-Harness; MM-Zero

  • Meta-Harness searches for task-specific harness code using prior candidates’ code and execution traces, while MM-Zero trains Proposer, Coder, and Solver roles to generate multimodal data from zero seed images.MM-Zero’s Proposer poses visual concepts and questions, and its Coder renders them as executable code.Source: Meta-Harness · p. 38, MM-Zero · p. 38

ModularRSI; MoMHa

  • ModularRSI evolves five harness modules from execution experience and validates patches before integration; MoMHa proposes harnesses from historical code and traces, retaining those that improve a joint accuracy, safety, and token-cost objective [66].MoMHa also turns deployment failures into source-code revisions and tests rebuilt agents on captured tasks before promoting fixes through consent.Source: ModularRSI · p. 38, MoMHa · p. 38

MoMHa; Procedural Graphs

  • MoMHa also converts deployment failures into code revisions and tests fixes on captured tasks; Procedural Graphs builds graphs from successful and failed trajectories and validates graph edits [131].MoMHa promotes fixes through consent.Source: MoMHa · p. 38, Procedural Graphs · p. 40

NavHarness; NeoHorse-1

  • NavHarness carries navigation experience across fresh sessions through revisable spatial memory, handovers, notes, and skills, while NeoHorse-1 reallocates the next training mixture according to updated capability gaps [125].NavHarness uses observations and model-based completion checks; NeoHorse-1 organizes experience for post-training.Source: NavHarness · p. 39, NeoHorse-1 · p. 39

OpenSkill; Prime Agent

  • Policy and prompt optimization use evolved prompts; prompt-induced gains are distilled into model parameters, while a persistent RLM-style harness carries prompts and subagent specifications across trajectories so later work can build on earlier context.Source: OpenSkill · p. 39, Prime Agent · p. 39

Procedural Graphs; SimSkill

  • Procedural Graphs turn relevant subgraphs into step-level guidance for later tasks, while SimSkill consolidates action–critic episodes into reusable code skills and linked knowledge pages.Source: Procedural Graphs · p. 40, SimSkill · p. 41

R-Few; R-Zero

  • R-Few uses Challenger–Solver self-play with human-labeled examples and an adaptive difficulty curriculum, while R-Zero uses edge-of-ability questions and majority-vote pseudo-labels without external data.R-Few’s mixed human and synthetic training data under its curriculum mitigates drift and diversity collapse.Source: R-Few · p. 40, R-Zero · p. 40

ReasoningBank; Recuris

  • ReasoningBank retrieves self-judged reasoning strategies for later tasks and improves its memory bank with test-time rollouts, whereas Recuris uses long-horizon execution to localize memory failures before validation-gated skill-memory updates.Recuris’s updates are made by a fixed meta-agent.Source: ReasoningBank · p. 40, Recuris · p. 40

Machine

  • Machine co-evolves an agent and its evaluator with stable within-epoch selection, and revises a prompt-level harness using pairwise feedback across versions.Its harness evolution is regularized to reduce overfitting to evolution tasks.Source: Machine · pp. 40–41

RSIAgent; SAGEAgent

  • RSIAgent updates reusable memory from verified experience and freezes it for downstream execution; SAGEAgent revises acquisition rules using cost and uncertainty feedback to improve diagnostic requests.SAGEAgent’s downstream agent is frozen.Source: RSIAgent · p. 41, SAGEAgent · p. 41

Self-Harness; SePO

  • Self-Harness proposes bounded edits from recurring failures and merges only those passing held-in and held-out regression tests, while SePO evolves system prompts and task-agent prompts.SePO also searches coding-agent versions using pairwise code judgments and task evaluations to reduce expensive benchmark runs.Source: Self-Harness · p. 41, SePO · p. 41

SkillAA; SkillAdam

  • SkillAA attributes failures to skill-graph nodes and applies validated local edits with rollback, while SkillAdam uses an adaptive edit budget and retains changes that gain without unacceptable regressions.SkillAdam also regularizes repeated skill updates.Source: SimSkill · p. 42, SkillAA · p. 42, SkillAdam · p. 42

SkillEvoReg; SkillGLoW

  • SkillEvoReg uses skill dropout, complexity control, and counterexample checks to limit overfitting and regressions; SkillGLoW consolidates task-local solving procedures into reusable skill priors validated through execution.Source: SkillEvoReg · p. 42, SkillGLoW · p. 42

SkillGLoW / SkillHEX

  • SkillGLoW tests explicit failure hypotheses against instance-level evidence, while SkillHEX shares cached-output evidence across skill-revision branches to guide budget-limited test-time improvement.Source: SkillGLoW · p. 42, SkillHEX · p. 42

SkillPivot / SkillProx

  • SkillPivot compares a failed trajectory with a teacher’s continuation from the same prefix and retains localized edits after preservation and generalizability checks.SkillProx revises skills from execution feedback and removes or consolidates low-utility content to lower complexity while preserving task performance [57].Source: SkillPivot · p. 42, SkillProx · p. 42

SkillRefine / SkillRevise

  • SkillRefine repairs refinery-planning skills using documentation, expert records, execution constraints, and reference comparisons [142], while SkillRevise returns the best tested candidate within a bounded budget.SkillRevise diagnoses defects from execution traces and verifier feedback and applies repair principles.Source: SkillRefine · p. 42, SkillRevise · p. 43

SkillSmith / SoL-Pi

  • SkillSmith jointly edits skills and executable tools, validates bundled changes progressively, and records anti-patterns [28]; SoL-Pi combines retained mechanisms into a lower-cost agent under capability constraints.Source: SkillSmith · p. 43, SoL-Pi · p. 43

TextGrad

  • TextGrad separates environmental facts from conditional procedures, then uses task feedback to retire misleading rules, resolve contradictions, and update both banks for later tasks [22].Source: TextGrad · p. 43

VisPlay

  • A single VLM jointly serves as an image-conditioned Questioner and multimodal Reasoner, using GRPO with diversity and difficulty rewards to improve visual reasoning from unlabeled images.The method is described as requiring no validation data.Source: VisPlay · p. 44

When Rules Learn / WikiSkill

  • When Rules Learn tests combinations of query-rewriting rules and removes those that fail to improve legal-retrieval metrics; WikiSkill’s persistent wiki survives rejected skill edits [148].The WikiSkill passage says later skill updates can build on that persistent wiki.Source: When Rules Learn · p. 44, WikiSkill · p. 44

A.2 Benchmarks

  • Table 3 organizes benchmark settings by updated resources, testing organization, feedback, and measures; PATH-Bench uses repeated probes and reports task performance, transfer, and forgetting.The table also includes PAST-Bench and ContinualSkillBench, which use fresh-session learning episodes and difficulty-ordered tasks, respectively.Source: §A.2 · pp. 44–45, Table 3 · p. 44
  • Online benchmarks test research, data-curation, implementation, and repeated-attempt workflows, including MLS-Bench [156], Curation-Bench [157], AutoLab [158], and EdgeBench [159].Their setups include ML-method research loops, data-curation policies, time-bounded implementation improvement, and long-running tasks with judge feedback.Source: §A.2 · p. 45
  • Offline benchmarks cover incremental context ingestion for MemoryAgentBench [160], feedback training followed by held-out tests for MemoryBench [72], harness optimization for VeRO-Bench [161], and skill generation from seed tasks followed by testing for SkillLearnBench [162].Source: §A.2 · p. 45
  • Other benchmarks test agent development and transfer: SEAGym [164] uses training-task updates with frozen-snapshot validation and transfer tests, while GDPevo [73] recombines training rules in held-out workflows.EvoMemBench [165] is described as online within/across episodes.Source: §A.2 · pp. 45–46
  • Benchmarks span online research-agent loops, offline held-out evaluations, and protocols combining online updates with offline transfer or adaptive feedback.They report measures such as task performance, gain, cost, and sometimes time.Source: §A.2 · p. 46

A.3 Software systems

  • Table 4 presents software systems for agent self-improvement.Source: Table 4 · p. 46

Related papers

Loading 2610.14625v1…