Source-linked AI summary
Self-Evolving Skills via Surrogate-Guided Solve-and-Reproduce
Jiale Liu, Pinze Ren, Yuqi Xia, Huan Wang, Zhenlin Zhao, Siming Dong
TL;DR
Self-evolving skills can underperform curated or no-skill baselines and can pass during evolution without reproducing at deployment. reSolve freezes each candidate for fresh-agent execution, adds blind surrogate verification, and guides beam search; on the stated fixed harness, it reaches 74.9% mean-of-3, +14.8 points over the 60.1% curated baseline. The paper also identifies scope boundaries and failure cases for self-evolution.
Problem
Self-evolution can produce weak skills and reward one lucky trajectory even though deployment requires a stochastic fresh agent to reproduce the solution.
Method
reSolve combines solve-and-reproduce, a blind surrogate verifier, and verifier-guided beam search over a solution-construction graph.
Results
74.9% mean-of-3, +14.8 points over the 60.1% curated baseline, was achieved by reSolve Self-Evolved on the stated fixed harness.
Takeaways & Limitations
Self-refinement is the most effective self-evolution setting reported, while from-scratch evolution reaches the no-skill frontier range but remains below the curated baseline.
Takeaways & Limitations
Fresh-agent stochasticity remains, and candidate-specific surrogate suites make eR a within-beam ranking heuristic rather than a grader-reward proxy.
Abstract
from arXiv · showhide
Agent skills are portable packages of instructions and resources an agent consults at deployment. Self-evolving them fails in two ways today. First, skills evolved from scratch underperform human-curated ones and, on a weak model, using no skill at all. Second, an evolution-time pass records one lucky trajectory that a fresh stochastic agent often fails to reproduce at deployment. We present reSolve, a per-task, oracle-in-the-loop framework built on three components. It decouples interactive solving from a self-contained deliverable that is independently re-executed in a fresh container, a protocol we call solve-and-reproduce. It enhances the sparse reward signal with a surrogate verifier that cannot access hidden tests or reference answers. It then runs verifier-guided beam search over a solution-construction graph. Within a fixed harness, a cheap model self-evolves skills that reach $74.9\%$ mean-of-3, $+14.8$ points over the $60.1\%$ human-curated baseline, exceeding the strongest official curated-skill result ($67.3\%$, GPT-5.5/OpenHands). We also report observed failure cases and domain-level results, including performance on the 14 Natural Science tasks, to clarify when the approach does and does not help.
1 Introduction
Agent skills package procedural knowledge for reuse, but self-evolution can underperform curated or no-skill baselines and reward lucky authoring trajectories rather than reproducible deployment. reSolve addresses these problems with solve-and-reproduce, surrogate-guided beam search, and empirical evaluation of when self-evolution helps.
- Agent skills are portable instruction, procedure, and code packages that provide procedural knowledge without changing model weights.They can be inspected, edited, reused across tasks, and attached to different agents.
- Good skills improve agent performance, with reported gains including 3.3× more unique Minecraft items and 24.6% and 51.1% relative success-rate increases on Mind2Web and WebArena.
- Self-evolution is limited by weak candidates that can fall below no-skill performance and by evolution-time passes that capture search luck.Plausible but incorrect candidates may be retained when a weak generator-verifier pair produces apparently successful results.
- The reproduction gap arises because a skill is deployment advice, yet current frameworks reward one successful trajectory rather than fresh-agent re-execution.A skill can pass on one model or trajectory and fail to reproduce on another stochastic deployment agent.
- reSolve decouples interactive solving from a frozen deliverable, independently re-executes it, and uses a blind surrogate verifier with verifier-guided beam search.The grader’s full-pass signal gates continuation while surrogate feedback supplies denser guidance without exposing hidden tests or reference answers.
- SkillsBench evaluation characterizes when self-evolution pays off, finding self-refinement most effective while from-scratch evolution remains below the curated baseline.The study also reports failure cases and limitations of self-evolution.
2 Related Work
Related work develops portable procedural memory, experience-based skill artifacts, grounded feedback, and search over executable agent behavior. reSolve combines these directions around reproducible deployment of a frozen task-specific package.
- Prior work represents procedural knowledge as reusable skills, libraries, directories, APIs, and trainable external state learned from agent experience.These approaches span executable programs, hierarchical skill banks, static directories, and bounded document edits.
- The reported gains from external procedural knowledge depend on skill quality, motivating methods that construct and improve such artifacts reliably.
- Iterative refinement improves outputs more reliably when feedback is grounded in environment traces, external tools, or executable tests rather than language-only self-correction.
- Search-based methods retain multiple prompt, workflow, program, or architecture candidates under evaluative signals instead of committing to one refinement trajectory.
- Executable behavior and self-verification improve inspectability but do not ensure successful reuse, as deployment can fail through wrong API selection or incorrect inputs.
- Figure 1 situates reSolve as a five-stage loop combining authoring, fresh execution, dual evaluation, and search control.The figure also shows blind verifier checks and a bounded beam-search configuration.
3 Method
reSolve evolves one task-specific skill package by freezing each candidate, testing it with a fresh deployment agent, and using hidden grading plus blind surrogate feedback to guide beam search. The method explicitly evaluates the package boundary rather than the authoring trajectory.
- 3 Method: reSolve has a producer author a candidate, freezes it, fresh-executes it in a new container, evaluates it, and refines it when it does not fully pass.
- 3.1 Problem setup: A task consists of an instruction, initial workspace, and hidden grader reward, while full pass means the candidate completely satisfies the grader.The producer cannot access hidden tests, reference answers, grader code, or grader-side diagnostics.
- 3.1 Problem setup: A candidate skill is a frozen directory whose instructions, resources, and scripts are read by a fresh agent alongside the original task and workspace.Freezing the package does not freeze the later stochastic execution trajectory.
- 3.2 Solve-and-Reproduce: The remaining reproduction gap means fresh-agent stochasticity persists, so reSolve requires every candidate to cross the fresh-agent boundary before crediting a pass.
- 3.2 Solve-and-Reproduce: Solve-and-reproduce discards the producer’s conversation, tool history, and authoring workspace, then mounts only the frozen package in a restored fresh container.The hidden grader runs after the new agent has independently produced output files.
- 3.3 Blind surrogate verifier: A blind verifier generates executable checks from the task and accessible workspace, producing a surrogate score and textual failure descriptions without hidden-grader information.Checks can test files, formats, numerical ranges, and table schemas; refinement uses the resulting repair descriptions.
- 3.3 Blind surrogate verifier: The verifier score ranks candidates within the beam, while hidden full reward stops search when a candidate passes.Algorithm 1 initializes candidates, evaluates them, retains top surrogate-scoring skills, and returns a full-pass or fallback package.
- 3.3 Blind surrogate verifier: Because each candidate receives a fresh check suite, surrogate scores are not strictly comparable and serve as ranking heuristics rather than grader-reward proxies.
4 Experiments
Across 86 SkillsBench tasks, reSolve improves performance and deployment reliability, with the largest gains concentrated in DeepSeek-V4-Pro and in tasks where curated skills are refined. Its benefits vary by base-model capability and task domain, while execution remains vulnerable to semantic divergence across trials.
- Main results: 74.9% mean-of-3 lifts DeepSeek-V4-Pro from the 60.1% reSolve Curated baseline, a +14.8-point gain that exceeds the strongest official curated-skill result of 67.3%.Self-evolution also raises pass@3 from 68.6% to 79.1%.
- Main results: 25 of 86 tasks improve after self-evolution, with 20 reaching perfect scores across all three deployment trials and 11 large-gain tasks contributing roughly two thirds of total improvement.The gains are heavy-tailed rather than uniform, functioning primarily as rescue on tasks where curated skills fail or pass intermittently.
- Cross-model generalization: 41.4% is Gemma 4 31B’s self-evolved score versus 36.2% with curated skills, while DeepSeek-V4-Pro reaches 74.9% versus 60.1%.Evolution improves 10 Gemma tasks compared with 25 DeepSeek tasks, and the leaderboard movement is from 24th to 20th for Gemma versus 14th to 1st for DeepSeek.
- Ablation studies: 44.0% mean reward from-scratch evolution trails the full framework by 30.9 points but improves the 30.6% no-skill baseline by 13.4 points.The results support refinement from a curated anchor as more effective than generation from scratch.
- Ablation studies: 59.4% mean reward for greedy search is 15.5 points below the full framework, while disabling surrogate ranking yields 66.0%, 8.9 points lower.These ablations indicate contributions from search breadth and depth, plus surrogate-guided parent selection.
- Execution stability: 56 of 86 tasks pass all three deployment trials with reSolve Self-Evolved, compared with 37 for reSolve Curated and 12 with no skill.Full-pass trial slots increase from 72 to 148 to 188 across the same condition ladder.
- Execution stability: Graded failures dominate non-passing trials: reSolve Self-Evolved has 57 graded failures versus 13 flagged errors among its non-passing slots.A frozen package still permits different deployment trajectories, and failed trials tend to use more tool steps than passing trials.
- Domain results: 87.6% mean reward on the 14 Natural Science tasks is well above the 74.9% overall average, while finance, cybersecurity, and software engineering range from 51.7% to 55.6%.The strongest categories generally involve well-specified deliverables and scripted pipelines; more open-ended exploration corresponds to lower reward.
5 Conclusion
reSolve combines solve-and-reproduce, a blind surrogate verifier, and verifier-guided beam search to help agents self-evolve skills. On 86 SkillsBench tasks, it exceeded curated and no-skill baselines, while gains varied with the evolving model's capability.
- Results: 74.9% mean-of-3 was reached by reSolve Self-Evolved on 86 SkillsBench tasks with DeepSeek-V4-Pro, exceeding the 60.1% curated baseline by 14.8 points.It was also 7.6 points above the strongest official curated-skill entry, although those rows used different harnesses.
- Method: Solve-and-reproduce freezes each candidate package and certifies it only when a fresh agent reproduces the solution in a new container.This separates skill authoring from skill use at deployment.
- Method: A blind surrogate verifier provides dense scores and failure descriptions, while verifier-guided beam search exposes search breadth and depth as explicit test-time-compute parameters.The hidden grader continues to gate full-pass continuation while withholding hidden tests, reference answers, and grader-side diagnostics.
- Results: The benefit transferred to Gemma 4 31B with smaller gains of 5.2 versus 14.8 points, indicating that the evolving model's capability affected the improvement.The supplementary run reported smaller gains than the DeepSeek-V4-Pro result.
- Analysis: Ablations ranked the curated anchor first, beam search second, and surrogate ranking third, with all three intervals excluding zero.From-scratch evolution still raised the no-skill baseline from 30.6% to 44.0%.