PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

Qiankai Xu

arXiv:2609.38372v1cs.AIcs.LG

TL;DR

Existing self-evolving harness methods often rely on a separate proposer using a human-designed harness and evolve a separate harness per benchmark, leaving questions about self-improvement and generalization across domains. This paper has the same frozen model solve tasks and directly edit its own harness, using multi-task and continual training stages. The evolved harness improves on the seed across in- and out-of-distribution benchmarks, surpassing Codex on the former and matching it on the latter.

  • Problem

    Many self-evolving harness methods use a separate proposer on a fixed human-designed harness and evolve a separate harness per benchmark, while generalization to unseen tasks remains a concern.

    Source: §1.1 · pp. 1–2, §1.2 · p. 3

  • Method

    The same frozen model solves tasks and directly edits the harness it runs on, with evolution organized as multi-task pretraining followed by continual training.

    Source: §1.3 · p. 3, §3.4 · p. 7

  • Results

    4.48 points above the seed on in-distribution benchmarks, where the evolved harness surpasses Codex, and 12.64 points above the seed on out-of-distribution benchmarks, where it is on par with Codex.

    Source: §4.2 · pp. 8–9, §6 · p. 13

  • Takeaways & Limitations

    Continual training on a new domain further improves the harness’s score in that domain.

    Source: §6 · p. 13

    The second stage uses only one new domain, so whether the conclusions extend to other domains remains to be tested.

    Source: §6 · p. 12

Abstract

from arXiv · show

A harness is the code around a language-model agent that organizes prompts, calls tools, manages context, and controls execution. As models grow stronger, recent work has begun to let agents improve their own harnesses, a line of work known as self-evolving harnesses. In most existing methods, a separate proposer running on a human-designed harness modifies the solver's harness, and a separate harness is evolved for each benchmark. Real-world tasks come from many domains, so both the evolution and the evaluation of a harness should cover a diverse range of tasks. We propose a framework close to recursive self-improvement: the same frozen model, on the same version of the harness, first solves tasks as the solver and then, as the proposer, reads the complete run records and directly edits the harness that runs it. Each evolution batch draws tasks from five benchmarks in different domains. To measure generalization, training and held-out tasks are strictly separated, and we additionally evaluate on five out-of-distribution benchmarks never used during evolution. We frame the evolution process as deep-learning training with two stages, multi-task pretraining and continual training. Starting from a 49-line seed harness, the harness obtained at the end of the first stage improves the average score by 4.48 points on the in-distribution benchmarks and by 12.64 points on the out-of-distribution benchmarks, surpassing Codex on the former and matching it on the latter. In the second stage, continued evolution on Claw-Eval, one of the out-of-distribution benchmarks, further raises the score on that benchmark from 66.17 to 68.06, exceeding Codex. We also provide an in-depth analysis of the mechanisms that emerged during evolution, including output truncation, history compaction, and independent review.

1 Introduction

The paper proposes a self-evolving harness that the same frozen model uses to solve tasks and then directly modify, with evolution and evaluation designed to test generalization across domains. It frames this process as two-stage training and reports gains over the seed harness across in- and out-of-distribution benchmarks.

1 Introduction

  • Prior self-evolving methods commonly use a separate proposer on a fixed human-designed harness and evolve a separate harness per benchmark, motivating a shared, more general setup.The paper identifies the ability of frontier models to improve their own harness beyond human-designed ones as an open question.Source: §1 · p. 1, §1.1 · p. 2

1.1 Self-Improvement Framework with Minimal Human Priors

  • The framework uses the same frozen model and harness for solving and proposing, with the proposer reading complete run records and directly editing the harness without restricting edit scope.The proposer itself runs on the harness it modifies, so edits to the solver harness also change the proposer’s working environment.Source: §1.1 · p. 2, Figure 1 · p. 2

1.2 Generalization of Harness Updates

  • Disjoint training and held-out splits, evaluation on five unseen benchmarks, and mixed-domain batches are used to assess generalization beyond evolution tasks.The approach also tightens the edit budget over iterations and prompts the proposer to prioritize generalization over task memorization.Source: §1.2 · p. 3

1.3 Harness Evolution as Training

  • Harness evolution is framed as training an external state: code edits act like parameter updates, while solving and grading batches provide forward-pass and supervision roles.The new-line budget is treated like a learning rate, and the experiments use multi-task pretraining followed by continual training with replay.Source: §1.3 · p. 3
  • About 4.5 points higher on average than the seed harness on held-out in-distribution benchmarks and 12.6 points higher on five unseen benchmarks, where it is on par with Codex.The in-distribution average also exceeds Codex; the out-of-distribution benchmarks were never used during evolution.Source: §1.3 · p. 3
  • The harness grows from 49 lines to 430 after the first stage and 524 after the second, developing output truncation, history compaction, independent review, and later visual inspection.The framework’s contributions include comparisons against the seed harness and Codex across ten benchmarks, alongside analysis of how mechanisms emerged and affected performance.Source: §1.3 · p. 3

2 Related Work

Related work spans agents that rewrite their own code, methods that evolve harnesses, and optimization-style approaches that update prompts or skill documents. The paper distinguishes its approach through the shared model and harness, broader task mixture, and training-inspired evolution design.

Self-improving agents and harness evolution

  • Self-improving-agent work includes systems that rewrite agent logic or code, while later harness-evolution work introduces candidate banks, population merging, and module-wise evolution.The paper places its work in a field that has increasingly focused on harnesses.Source: §2 · p. 4

Harness gains and model capability

  • Prior studies report that harness gains vary with model capability, including larger benefits for weaker models and better results when a strong optimizer organizes the process itself.The cited work also reports that the strongest tier may gain less than mid-tier models from updates.Source: §2 · pp. 4–5

Evolution tasks and optimization-style evolution

  • Related evolution work varies in task diversity and optimization design, while comparisons also emphasize evaluating methods under the same budget.Some methods use feedback from multiple same-type benchmarks; optimization-style approaches include bounded edits, annealed edit counts, and continual evolution.Source: §2 · p. 5

3 Method

The method lets a frozen model solve tasks and then edit the harness that runs it, using full run records and checks on each update. Evolution proceeds from multi-benchmark pretraining to continued training on a new domain, with notes and edit budgets carrying state across iterations.

3.1 Problem Setup

  • The framework treats harness source code as an optimization target: task solving and grading provide the update signal, and the proposer rewrites the harness.The harness state includes both the source code and accompanying evolution notes.Source: §3.1 · p. 5

3.2 Solver, Proposer, and Seed Harness

  • The same frozen model and harness version play solver and proposer, while the 49-line seed is a minimal loop with a bash tool and no truncation or compaction.The proposer receives complete run records and may modify any harness file or add and delete files.Source: §3.2 · p. 6

3.3 One Iteration: Rollout, Update, and Check

  • Each iteration rolls out the task batch, packages run records for the proposer to edit the harness, then checks the update and starts a repair session if it fails.Checks require harness-only edits, a non-notes change, compliance with the new-line budget, and a successful mock-model run.Source: §3.3 · p. 6

3.4 Two Stages: Multi-Task Pretraining and Continual Training

  • Stage 1 draws 12 previously unused tasks per benchmark per iteration across five benchmarks, uses each training task once, and keeps held-out tasks out of evolution.Stage 2 switches to new-domain tasks and mixes in old success-anchor tasks, rerunning them so the proposer can see effects on original tasks.Source: §3.4 · pp. 6–7

3.5 Optimizer State and Learning-Rate Schedule

  • Evolution notes carry forward and revise prior judgments, while an iteration-specific line-addition budget follows a decaying schedule and is rewarmed when stage 2 begins.The first-stage budget decreases from 500 to 100 lines; stage 2 begins at 200 and then lowers to 100.Source: §3.5 · p. 7

4 Experiments

The experiments compare evolved harnesses with Seed and Codex on held-out tasks across in-distribution and never-used benchmarks, then continue training on Claw-Eval. Iteration 5 leads on the in-distribution average and is close to Codex out of distribution; Iteration 7 improves further on Claw-Eval.

4.1 Experimental Setup

  • The evaluation uses five varied first-stage benchmarks, five never-used out-of-distribution benchmarks, and a common GPT-5.6 Sol model configuration for Seed, Iteration 5, and Codex.Claw-Eval training tasks are reserved for the second stage; small held-out sets are evaluated repeatedly and averaged.Source: §4.1 · pp. 7–8, Table 3 · p. 8
  • Evaluations pair a fixed anti-cheating instruction and delayed access to answers and grading tests with trajectory audits; no reported record was confirmed as cheating.Flagged trajectories and unusually high- or low-scoring records receive additional review.Source: §4.1 · p. 8

4.2 Main Results

  • 60.17 average across the five in-distribution benchmarks puts Iteration 5 4.48 points above Seed and 2.99 above Codex.Its gains are concentrated on Terminal-Bench 2.1 (66.67 to 83.91) and SWE-Bench Pro (51.00 to 61.00); Seed scores highest on OfficeQA Pro and τ 3-Bench banking.Source: §4.2 · pp. 8–9, Figure 3 · p. 9
  • 70.45 average across five out-of-distribution benchmarks puts Iteration 5 12.64 points above Seed and within one point of Codex (70.57).It scores 90.00 on BrowseComp-Plus, versus Seed’s 34.00 and Codex’s 93.00; excluding that benchmark, its four-benchmark average is 65.56, versus 63.76 for Seed and 64.97 for Codex.Source: Figure 3 · p. 9, §4.2 · p. 9
  • Seed remains strongest on OfficeQA Pro and τ 3-Bench banking, where the reported gaps are attributed to ambiguous questions and task-level judgment or calculation errors, not harness mechanisms.Codex’s OfficeQA Pro errors are described as ambiguous; its banking failures span tasks and involve business judgment and calculation.Source: §4.2 · p. 9

4.3 Continual Training on Claw-Eval

  • 68.06 on Claw-Eval raises Iteration 7’s overall score from Iteration 5’s 66.17, while passed tasks increase from 52 to 57.The gain comes mainly from multimodal tasks, whose average score rises from 39.3 to 47.8.Source: §4.3 · pp. 9–10, Figure 4 · p. 10

5 Analysis

The analysis traces how harness mechanisms emerged to address execution and review failures, then examines their effects across tasks and the costs of unrestricted, unvalidated evolution.

5.1 How the Harness Evolved

  • Iteration 1 added truncation, command time limits, history compaction, and image viewing after seed failures; later rounds revised mechanisms, while Table 4 tracks iteration-level code changes.The seed's failures included roughly 16 MB of command output, hung commands, and image tasks that required pixel-by-pixel parsing.Source: §5.1 · pp. 10–11, Table 4 · p. 10

5.2 Output Truncation

  • Seed output-overflow aborts occurred on nine of ten benchmarks, while Iteration 5 had none; it solved 57 of 65 affected BrowseComp-Plus tasks.The seed also aborted on 12 SWE-Bench Pro tasks because of long test or build output, and Iteration 5 solved seven of them.Source: §5.2 · p. 11

5.3 History Compaction

  • On BrowseComp-Plus task q0285, Iteration 5 compacted history three times across 64 calls and answered correctly after the seed aborted from context overflow.The seed's search output reached about 3.19 million characters before its fourth request exceeded the input limit.Source: §5.3 · p. 11

5.4 Independent Review

  • During review, Iteration 5 corrected an OfficeQA Pro year alignment from 1960 to 1959, changing its calculation from 0.77667 to 0.88525 and passing grading.The independent review could gather evidence, recompute, and revise the answer when it found a problem.Source: §5.4 · p. 11

5.5 Visual Inspection Budget

  • On 34 held-out multimodal tasks, Iteration 5 missed delivery deadlines 22 times; Iteration 7's image quotas and deadlines eliminated timeouts, and M097 scored 1.0 versus 0.2.Iteration 7 capped image requests and imposed time-based delivery cutoffs.Source: §5.5 · pp. 11–12

5.6 Regressions Relative to the Seed

  • On τ 3-Bench banking, Iteration 5 passed four fewer runs than Seed; compaction deleted customer requests in all 11 affected runs, and configure_run shortened conversations in four runs.Two configure_run cases were cut off before the conversation finished; on OfficeQA Pro, the gap instead involved ambiguous questions, and all three harnesses gave the same wrong answers.Source: §5.6 · p. 12

5.7 Costs of Simplicity and Freedom

  • Unvalidated local edits can cause cross-task regressions, while unrestricted scope still yielded few mechanisms and omitted skills, memory, and sub-agents.The proposer focuses on one batch's common failures, and updates are accepted if they run without comparing old and new scores.Source: §5.7 · p. 12

6 Discussion and Conclusion

The paper proposes a self-evolving harness in which the same frontier model solves tasks and directly modifies the harness that runs it. The evolved harness improves average scores across in-distribution and out-of-distribution benchmarks, while the authors identify limits to the evidence and avenues for further improvement.

6 Discussion and Conclusion

  • Evidence is limited to one strong model and one new domain in the second stage, while stochastic runs and few held-out tasks make some benchmark differences uncertain.The authors caution that small differences should be interpreted carefully.Source: §6 · p. 12
  • The framework has the same frontier model solve tasks and directly modify the harness running it, without a human-designed analysis step in the outer loop.The authors describe this as a self-evolving harness framework.Source: §6 · p. 12
  • The evolved harness surpasses Seed and Codex on in-distribution average score and scores 12.64 points above Seed and on par with Codex out of distribution.Continual training on a new domain further improves the score in that domain.Source: §6 · p. 13
  • The analysis identifies side effects from local edits, missing update validation, and narrowing edit directions; validating updates affordably and evolving along several paths are proposed as promising directions.Source: §6 · p. 13

A Seed Harness Code

The seed harness is a 49-line implementation that gives the agent a bash tool and repeatedly handles model replies and tool calls until the model finishes.

A Seed Harness Code

  • The 49-line seed harness defines an autonomous-agent system prompt and a bash tool, then runs a model-call loop until no tool calls remain.Source: A Seed Harness Code · p. 16
  • For bash calls, the harness returns standard output, any standard error, and the exit code to the conversation as a tool message.Source: A Seed Harness Code · p. 16
  • Calls to unrecognized tools produce an “unknown tool” message, while model replies and tool results are appended to the message history.Source: A Seed Harness Code · p. 16

B Update Task Prompt

The update task asks the agent to improve the harness that will run both the next task batch and its next editing round. It provides prior run records and persistent notes, while emphasizing generalization and allowing broad harness changes subject to acceptance checks.

B Update Task Prompt

  • The update task supplies current harness code and notes plus complete records of prior tasks, including successful and failed runs, scores, grader outputs, and trajectories.From the second iteration of each stage, it also includes the previous update session's run record.Source: B Update Task Prompt · p. 17
  • The proposer is instructed to make improvements that generalize to unseen tasks, avoiding memorized solutions or wording and favoring general methods.Source: B Update Task Prompt · p. 17
  • The proposer may reshape the harness by adding or changing components such as tools, prompts, memory, context management, or subagents.Source: B Update Task Prompt · p. 17
  • Each edited harness runs the next task batch and the next editing task, repeating round after round.Source: B Update Task Prompt · p. 17
  • A persistent notebook can track prior edits, their expected and observed generalization, current evidence, ineffective ideas, and other useful lessons, but not individual task specifics.Source: B Update Task Prompt · p. 17
  • Files written under the agent log directory are collected into each trial record, letting the model retain trajectories or notes it judges useful for future rounds.Source: B Update Task Prompt · p. 17
  • Each model API request is limited to 258,400 input tokens before it fails with a context-length error.Source: B Update Task Prompt · p. 17
  • Edits must pass workspace, nonempty-change, smoke-test, and line-budget checks, and the prompt prohibits obtaining answers from solution-bearing sources.Source: B Update Task Prompt · p. 17

C Mechanism Trigger Rates

Figure 5 reports the share of tasks on each benchmark that triggered Seed context-overflow aborts or Iteration 5 mechanisms. The trigger patterns differ across task types: truncation is common in retrieval and document tasks, while independent review is common in non-interactive tasks.

C Mechanism Trigger Rates

Related papers

Loading 2609.38372v1…