SkillNEO: Non-Elitist Evolutionary Optimization of Agent Skills

1 INTRODUCTION

• A bundle’s reward can hide partial progress when coupled instructions, scripts, and resources require several interdependent edits.For example, correcting input processing while leaving output construction incomplete can leave whole-task reward unchanged or lower.Source: §1 · pp. 1–2
• Reward-improvement requirements can exclude partial repairs from further editing, motivating separate decisions about deployment readiness and repair value.Non-elitist selection permits a child to receive further edits while the best available version remains deployed.Source: §1 · p. 2
• SkillNEO compares parent-child and sibling executions to identify procedures to preserve and failures to repair, and revisits earlier versions using later evidence.Comparisons guide coordinated edits to instructions, scripts, and resources, including reuse across branches.Source: §1 · p. 2
• 49.1% to 66.4%: SkillNEO raises GPT-5.4 Mini’s average reward across 86 tasks into the original-skill range of GPT-5.5 and Luna.Final evaluation uses the same task instances; GPT-5.5 and Luna’s original-skill references are 66.1% and 67.3%, respectively.Source: §1 · p. 2

2 RELATED WORK

• Agent skills package reusable guidance, programs, workflows, and policies; prior work also studies their transfer across models and further skill development.The related work situates skills alongside memory, auditing, scientific workflows, and agent evolution.Source: §2 · pp. 2–3
• Skill-evolution methods vary in which candidates can receive further edits, including offspring replacement, score thresholds, archives, and multiple populations.SkillNEO selects ordinary parents from eligible, fully measured children; earlier bundles can return through a separate archive-revisiting rule.Source: §2 · p. 3
• Execution-guided methods extract workflows, compare successes and failures, or revise skills from trajectories; SkillNEO links executions to saved bundle versions.It compares repeated runs, parent-child and sibling executions, supports joint edits to instructions and files, and can use later executions to revise saved bundles.Source: §2 · p. 3

3 SKILLNEO FOR NON-ELITIST EVOLUTION OF AGENT SKILLS

• SkillNEO separates deployment selection from development selection; comparisons of related executions identify procedures to preserve and failures to address.A version can receive further edits even when it scores below the deployed incumbent.Source: §3 · p. 3, Figure 2 · p. 4

3.1 PROBLEM FORMULATION

• The objective is to improve a frozen agent’s expected reward on a task using a revised bundle, with estimates from finite executions guiding development.After development, the selected bundle is frozen for cross-model reuse without repeating skill development for the receiver.Source: §3.1 · p. 4

3.2 NON-ELITIST SKILL DEVELOPMENT

• Ordinary parent selection promotes eligible, fully evaluated children from the current generation; previous parents do not compete, and children need no parent-relative gain.A child may receive another edit despite a reward too low for deployment; deployment is decided separately.Source: §3.2 · p. 5

3.3 COMPARATIVE EVIDENCE FOR SKILL REVISION

• The Proposer compares repeated, parent-child, and sibling executions to identify unresolved failures and procedures worth preserving.The Skill Editor can make coordinated changes across instructions, scripts, and resources; validation checks candidate files before task evaluation.Source: §3.3 · p. 5

3.4 EVALUATION, DEPLOYMENT, AND ARCHIVE REVISITING

• Repeated evaluation informs deployment, while archive revisiting can return development to an earlier version after a HOLD with a weaker child.The incumbent remains deployed as the parent set changes; the frozen output bundle is later used for cross-model reuse.Source: §3.4 · pp. 5–6

4.1 EXPERIMENTAL SETUP

• Development uses 86 SkillsBench tasks across eight categories, with Mini performing task execution, proposal generation, and skill editing.Final evaluation uses the same task and input instances; 41 bundles are modified and 45 remain byte-identical to their originals.Source: §4.1 · p. 6

4.2 PERFORMANCE ACROSS TASKS AND DOMAINS

• 49.1% to 66.4%: SkillNEO raises Mini’s average reward across 86 tasks, within GPT-5.5 and Luna’s original-skill range.On the 41 modified tasks, Mini scores 67.8% against 31.5% with original skills, with 29 gains, 12 ties, and no observed losses.Source: §4.2 · p. 7
• Mini improves across all eight domains, with gains ranging from 5.6 pp in industrial / physical tasks to 27.4 pp in software engineering.Both domains start near 48.3%.Source: §4.2 · p. 7, Figure 3 · p. 7
• Revised workflows can still leave requirements unchecked or unenforced, including emergency-mode verification and required component evidence.Some models also misread input images or fail to save required output files.Source: §4.2 · pp. 7–8, §4.4 · p. 10

4.3 SEARCH PATHS AND EXECUTION EVIDENCE

• On the 41 modified tasks, lower-scoring versions often contribute to later deployed skills: 22 of 29 gain paths use a non-incumbent parent.Across 41 searches, 163 lower-scoring versions receive further edits, and 45 have a later deployed descendant.Source: §4.3 · p. 8

4.4 CROSS-MODEL REUSE

• Frozen Mini-developed bundles raise Luna from 63.9% to 74.4% (+10.5 pp) on 41 modified tasks; GPT-5.5 scores 71.1% against its 60.5% published reference (+10.6 pp).The models improve on six shared tasks and 20 distinct tasks in total.Source: §4.4 · p. 9

• SkillNEO separates the bundle used for deployment from versions retained for development, guiding revisions with comparative execution evidence.The paper recommends testing evolved skills on new instances within their intended task scope and studying how to accumulate improvements in a shared skill library.Source: §5 · p. 10

A LIMITATIONS AND FUTURE DIRECTIONS

• The complete-framework experiments do not isolate the effects of non-elitist selection, comparative execution evidence, or cross-branch reuse.The authors call for matched-budget comparisons and suggest evaluating stopping criteria that track development-population progress.Source: A LIMITATIONS AND FUTURE DIRECTIONS · p. 14
• Development and final evaluation use the same tasks and inputs, so effectiveness on unseen tasks or inputs remains unestablished.The authors call for tests on new task variants.Source: A LIMITATIONS AND FUTURE DIRECTIONS · p. 14
• Mini performs every model-based development role, leaving open how developer capability and search depth jointly shape skill improvement.The authors ask whether deeper search can compensate for limited capability and which capabilities remain necessary.Source: A LIMITATIONS AND FUTURE DIRECTIONS · p. 14
• SkillNEO evolves task-specific bundles; a shared library would need to manage interactions between accumulated procedures and previously learned skills.The authors propose studying combinations of broadly useful procedures with task-specific guidance.Source: A LIMITATIONS AND FUTURE DIRECTIONS · p. 14

B DEVELOPMENT PROTOCOL

• Development starts from each SkillsBench task’s supplied skill bundle, and Harbor executes tasks while SkillNEO compares traces, edits bundles, evaluates candidates, and selects versions.The bundle retains the SKILL.md document and optional tools and resources.Source: B DEVELOPMENT PROTOCOL · p. 14
• The planned configuration requests six children, promotes up to three, measures promoted children and the incumbent over three repeats, and allows five generations.A full generation with six admitted children and three promotions contains 15 task executions; the complete planned schedule totals 78.Source: B DEVELOPMENT PROTOCOL · p. 14, §B.1 · p. 14

B.1 SEARCH CONFIGURATION

• Screening ranks complete children by one-run official reward, while deployment compares fully replicated records under matching task, model, environment, and verifier settings.Promoted children receive three runs, and the incumbent receives three fresh runs in the same comparison batch.Source: §B.1 · p. 15
• On a negative HOLD, archive revisiting can replace the ordinary parent set with eligible earlier survivors; search also stops after two consecutive holds or other stated conditions.The proposer can use eligible parents and graft donors, while later prompts discourage directions previously measured as net losses.Source: §B.1 · pp. 14–16

• Final evaluation uses the frozen task-bundle assignment and model settings specified for the study.Source: §B.3 · p. 17
• GPT-5.4’s 51.7% original-skill reference uses 87 tasks, OpenHands, and xhigh reasoning, so its +13.8 pp comparison with the 86-task Codex result spans different settings.Source: §B.3 · p. 17
• Reported SkillsBench averages are 66.4% for SkillNEO, 53.42% for MUSE-Autoskill with GPT-5.5, and approximately 52% for Mini with SPARK skills, each under its own task set and protocol.SkillAudit reports 73.9% using Opus 4.8 for execution, auditing, editing, and final evaluation.Source: §B.3 · p. 17
• SkillNEO and SkillAudit refine supplied skills, whereas MUSE-Autoskill and SPARK generate skills; SkillNEO also permits continued editing without parent-relative improvement.SkillNEO uses benchmark rewards during evolution, while SkillAudit excludes external task rewards.Source: §B.3 · pp. 17–18

• The 41 modified tasks divide into 29 Mini-positive and 12 Mini-tied tasks using a ±0.05 pp tie band; 45 tasks retain their original bundles.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 18
• 13.8 pp and 18.3 pp are GPT-5.5’s and Luna’s gains, respectively, in the Mini-improved group; the Mini-tied group shows a 2.8 pp GPT-5.5 gain and an 8.3 pp Luna loss.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 18, Table 6 · p. 18
• Eleven of 12 Mini-tied bundles were adopted with higher official reward than the incumbent, although final evaluation could still tie the original bundle.The final ties comprise four ceiling cases, three partial-score cases, and five zero-score cases.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 18
• Among the 41 modified tasks, 14 reach 100.0% average reward with all three receivers, 15 with one or two, and 12 with none.Three tasks receive zero with all three receivers.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 18
• Six modified tasks gain for both GPT-5.5 and Luna, while 20 gain for at least one; four show opposite nonzero directions.Three named tasks decrease for both receivers.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 18, Table 7 · p. 18
• Luna’s modified and retained subsets contribute approximately +4.99 and −0.39 pp to its +4.60 pp full-cohort difference.GPT-5.5’s corresponding subset contributions are +5.05 and −1.55 pp, yielding +3.50 pp.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 19, Table 8 · p. 19
• Among 17 modified tasks with perfect original Luna reward, 16 retain perfect reward and one receives less; among 16 with partial original reward, seven improve, three tie, and six lose reward.Of eight tasks with zero original reward, four improve and four tie; that group has the largest mean gain.Source: C ANALYSIS ACROSS RECEIVING MODELS · p. 19, Table 9 · p. 19

• Mini improves in every domain, whereas Luna’s domain-level differences vary across gains and losses.Table 10 reports domain-level improvement and transfer differences, with task counts for all and modified subsets.Source: Table 10 · p. 19, D DOMAIN RESULTS · p. 19

D.1 RESOLVED AND REMAINING WORKFLOW OBSTACLES

• SimPO Code Reproduction reward rises from 0.0 to 100.0 when environment setup precedes installation, logging, and testing.Original runs used Python 3.12 and stopped at the version assertion, although their 100 loss values matched passing final runs.Source: §D.1 · p. 20
• Data-to-D3 reward rises from 0.0 to 100.0 after the skill adds rendered-page interaction checks, which expose and help correct a repeated-selection bug.All final executions perform interaction checks and pass all 15 tests.Source: §D.1 · p. 20
• Adaptive Cruise Control rises from 33.3 to 66.7, but an unavailable helper path and omitted emergency-behavior checks leave a run failing safety and scenario checks.The bundled checker cannot verify the complete controller.Source: §D.1 · p. 20
• Manufacturing Codebook Normalization finishes at 0.0 despite two completed runs passing 15 of 16 tests, as they admit contextual evidence and retain mismatched component assignments.A further run times out without the required output.Source: §D.1 · p. 20

E COMPLETE TASK-LEVEL RESULTS

• Table 11 covers all 86 tasks, with each row representing one task and its complete collection of skills.Modified rows use evolved bundles; retained rows use byte-identical original bundles, and Mini’s retained rows reuse the same original three-run score in both columns.Source: Table 11 · p. 20, E COMPLETE TASK-LEVEL RESULTS · p. 22

F CROSS-MODEL EXECUTION ANALYSIS

• The matched Luna comparison on 41 modified tasks yields 11 gains, 23 ties, and seven losses.The task-level table reports mean reward for each task and distinguishes evolved bundles from byte-identical retained bundles.Source: Table 11 · p. 20, F CROSS-MODEL EXECUTION ANALYSIS · p. 22, E COMPLETE TASK-LEVEL RESULTS · p. 22

F.1 REUSABLE WORKFLOW IMPROVEMENTS AND RECEIVER EXECUTION

• On six of Luna’s 11 improved tasks, Mini’s final mean reward remains below 100.0.The analysis examines Software Dependency Audit and Sales Pivot Analysis using frozen final bundles across four models.Source: §F.1 · p. 22
• On Software Dependency Audit, Mini rises from 0.0 to 66.7 and Luna from 0.0 to 100.0 with the final bundle, which corrects scan scope and reports three HIGH-severity findings.Mini’s remaining failure uses CVSS 9.8 instead of the required 8.1 and truncates a fix list; all three receivers preserve the required fields.Source: §F.1 · p. 22
• On Sales Pivot Analysis, Luna’s evolved-skill runs filter to eight full input names, exclude four out-of-scope records, and build pivots over 2,450 regions, satisfying both requirements.Mini passes state validation in all runs but twice omits actual pivot objects; GPT-5.5 retains the excluded category in all three runs.Source: §F.1 · p. 22
• The Sales Pivot skill coordinates dependent operations by cleaning missing income markers before quartiles and saving, reopening, and checking the source sheet before attaching pivot objects.Mini’s successful final execution computes quartiles over valid income values and maps labels back to merged data.Source: §F.1 · p. 22

F.2 UNRESOLVED FAILURES IN LUNA REUSE

• Energy and Grid each fall from two successes to none after evolved skills replace benchmark demand Pd with Pd + Gs, despite passing schema and feasibility checks.The revised rule fails cost optimality in the high-cost direction, showing that passing development evidence did not establish the rule’s correctness.Source: §F.2 · pp. 22–23
• Organize falls from two successes to none because evolved runs place documents below the source directory, while the verifier treats those documents as unsorted leftovers.The stronger self-check audits the destination tree but leaves source-directory clearance unspecified.Source: §F.2 · p. 23
• Civ6 executions find competitive legal layouts but can time out without saving the required JSON; the workflow lacks a checkpoint to save each new best layout.Two other runs time out with valid outputs and receive 0.95 normalized adjacency reward.Source: §F.2 · p. 23
• Drone’s quintic trajectories mismatch the oracle’s cubic interpolation, explaining 13 common command failures; revised skills do not resolve the planner choice.Scoring counts successful commands individually, so enclosing pytest groups pass even at low reward.Source: §F.2 · p. 23
• JPG receipt workbooks retain month-token perception errors despite deterministic parsing and image review, which cannot verify recognition accuracy.Five failed executions misread one or two month tokens on the same receipts across conditions.Source: §F.2 · p. 23
• The only failed R2R execution passes local checks but fails verifier settling time because the local and verifier metrics use different bands and time origins.The task specifies a settling target without fully defining its measurement convention.Source: §F.2 · p. 23

F.3 ADDITIONAL RECEIVER EVIDENCE

• GPT-5.5 receiver failures include unsaved Paratransit reports below the 443-trip minimum, a GRPO source-pattern penalty despite a correct numeric value, and Lake Warming attribution errors.React failures violate the external-API minimum checkout time, while SEC failures concern newly acquired positions or reported holdings.Source: §F.3 · p. 24

G CHANGES TO SKILL ARTIFACTS

• Across 86 tasks, 62 of 228 task-skill pairs change (27.2%); among 41 modified tasks, 62 of 125 pairs change (49.6%), while total source length grows by 1.5%.Table 13 counts core instructions and executable support separately; core instructions become shorter on average while support scripts grow.Source: G CHANGES TO SKILL ARTIFACTS · p. 24, Table 13 · p. 24

H SEARCH HISTORIES AND DEVELOPMENT PATHS

• 45 of 247 lower-scoring candidates later had a deployed descendant, and 18 of those continuations required at least two edits.35 of the 45 candidates lie on a final deployment’s primary-parent path.Source: H SEARCH HISTORIES AND DEVELOPMENT PATHS · p. 25
• Eight retained graft nodes enter final paths across six tasks, while adopted lineages average 3.07 edits from root to deployment.Graft proposals identify donors outside their tasks’ final paths; 11 of 75 graft candidates were removed as duplicates.Source: H SEARCH HISTORIES AND DEVELOPMENT PATHS · p. 25, Table 15 · p. 26
• 61 of 87 adopted post-generation-one edits use non-incumbent parents, spanning 30 of 41 adopted lineages.Among the 29 tasks with improved final Mini reward, 22 use a non-incumbent parent on the final path.Source: H SEARCH HISTORIES AND DEVELOPMENT PATHS · p. 26
• Paratransit’s final path revisits generation-three d06, whose child f08 is selected in generation five, raising Mini’s score from 0.0 to 33.3.Luna remains at 0.0.Source: H SEARCH HISTORIES AND DEVELOPMENT PATHS · p. 26
• Wyckoff uses later execution evidence to revise the original bundle; the final e03 bundle raises Mini from 75.7 to 100.0 while Luna remains at 100.0.The revision adds file-based analysis guidance, and the later execution uses it to batch-analyze files.Source: H SEARCH HISTORIES AND DEVELOPMENT PATHS · p. 27

I.1 A PERSISTENT FAILURE MOVES THE REPAIR UPSTREAM

• In Software Dependency Audit, moving the repair upstream to prohibit -include-dev-deps yields a 100.0% mean reward after reporting edits alone leave scan scope unverified.The three revised executions use production scope, write three rows, and check the saved count and key set.Source: §I.1 · p. 27

I.2 WORKFLOW CONSOLIDATION ON A RETAINED BRANCH

• In Gravitational Wave Detection, consolidating the workflow on retained e04’s branch produces f04, whose 20 Hz search runs pass every test and score 100.0%.The zero-reward e04 runs passed eight of nine tests but used a 15 Hz conditioning cutoff instead of the specified 20 Hz cutoff for waveform generation and matched filtering.Source: §I.2 · pp. 27–28

I.3 HELPER REPAIR AND DIRECT WORKFLOW EXECUTION

• In PDDL Airport Planning, the agent directly executes the prescribed writing, parsing, and validation workflow through the planning library, and the saved plan passes official verification.The proposal’s helper was initially omitted, then added after validation feedback, but the agent could not locate it in the first execution.Source: §I.3 · p. 28

I.4 A NON-DEPLOYED SIBLING CONTRIBUTES AN EXECUTED PROCEDURE

• b04 scored 100.0% mean reward, while sibling b05 scored 33.3%; its third successful execution prompted c04 to borrow its PDF procedure.c04 retained b04’s spreadsheet skill and adopted b05’s continuation-aware PDF extraction; two executions passed, while the third failed state validity, yielding a 66.7% search-time mean reward.Source: §I.4 · p. 28
• Generation two retained b04 after its historical-score check but selected c04 for development, and the final bundle descended from c04’s lineage.That primary-parent path includes the revision importing b05’s procedure (Figure 4b).Source: §I.4 · p. 28
• b09 compressed a 485-line penetration-testing guide to 31 lines on reproducing an unauthenticated channel-open attempt and verifying rejection, preserving perfect final scores on Mini and Luna.The revision added no script.Source: §I.4 · p. 28

J PROPOSER AND SKILL EDITOR PROMPT EXCERPTS

• The Proposer is instructed to diagnose grader failures through execution traces, separate recoverable errors from causes of task loss, and use passing runs to specify actionable edits.The prompt distinguishes the grader’s verdict, which determines the score, from rollouts, which help explain the cause and reveal what a successful run did instead.Source: J PROPOSER AND SKILL EDITOR PROMPT… · pp. 29–30
• The prompts rank causes by runs lost because of them, not by how often they appear, and prioritize clustered graded values by tracing divergent computation paths and encoding the passing path.A cause shown in passing runs is not automatically task-losing; the proposed procedure should pin the passing ordering, normalization, or formula rather than the target value.Source: J PROPOSER AND SKILL EDITOR PROMPT… · pp. 30, 34
• Later-generation prompts direct distill to encode the winning run’s method and allow executable files for repeatable derivations or checks that prose cannot supply.The file and the instruction that makes it reachable are treated as one edit.Source: J PROPOSER AND SKILL EDITOR PROMPT… · p. 32
• Proposals must connect a repair to a failing run step and a passing-run contrast, then specify an executable change rather than merely naming the topic.The prompt asks for the divergence, the action that would change, and consistency with successful behavior; it also requires proposals to be checked against runs.Source: J PROPOSER AND SKILL EDITOR PROMPT… · pp. 34–35, §J.1 · p. 36

J.1 SKILL EDITING FROM A PROPOSAL

• The rewrite example narrows Trivy scanning to the offline production-dependency scope after runs using --include-dev-deps produced a dev-inclusive CSV that failed the expected-set test.The proposed correction follows a passing scan command without the dev-deps flag.Source: §J.1 · pp. 35–36
• The editor makes one intent real in the skill, changing what the agent does at the loss point while preserving unrelated file content byte for byte.Parts needed to complete that intent, such as a script and the procedure that invokes it, count as one edit; unrelated intents should not be bundled.Source: §J.1 · pp. 36–37
• Skill-body guidance should use agent-observable triggers, teach how to obtain required information rather than cache task answers, and make errors observable before submission when needed.The agent cannot see verifier output; a runnable file can provide a check that instructions alone may not ensure is followed.Source: §J.1 · pp. 36, 38