PaperlayerView exact revision on arXiv

Source-linked AI summary

Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents

Tianxin Wei, Zhan Shi, Minhua Lin, Bing He, Zewen Liu, Yisi Sang, Yuanchen Bei, Xuying Ning, Jiaru Zou, Ting-Wei Li, Xiao Lin, Yanjun Zhao, Chi Wang, Benoit Dumoulin, Dakuo Wang, Jingrui He, Hanqing Lu

arXiv:2608.15071v1cs.AIcs.CL

TL;DR

Agents often encounter novel tasks only once, producing noisy execution contexts that existing experience-based methods do not reliably convert into reusable lessons. Evo-Harness formulates online harness learning and compiles single-shot contexts into skill updates, consistently outperforming existing experience-based methods across five complex benchmarks.

  • Problem

    Existing experience-based methods rarely address one-shot learning from noisy, novel-task executions or validate continuous improvement on complex realistic tasks.

  • Method

    Evo-Harness incrementally updates a structured external harness by compiling individual execution contexts into reusable general and topic-level skill guidance.

  • Results

    EVO-HARNESS consistently outperforms existing experience-based methods across five complex benchmarks.

  • Takeaways & Limitations

    Online harness learning offers an analytical lens for how frozen agents produce, organize, transfer, and apply reusable experience in realistic task streams.

  • Takeaways & Limitations

    The study covers single-agent systems using natural-language guidance in text-based and tool-use environments, not embodied or multi-agent systems or executable-code harnesses.

Abstract

from arXiv · show

Learning from experience is critical for developing capable, self-improving large language model (LLM) agents. Existing methods typically extract knowledge from accumulated trajectories via reflection, memory, rules, or skills. However, agents in realistic environments continuously encounter novel tasks, often offering only a one-shot opportunity to improve. These executions yield rich but highly noisy contexts, entangling broadly useful lessons with task-specific artifacts. Critically, prior works rarely validate their effectiveness on complex real-world tasks or isolate the underlying drivers of improvement. To address these gaps, we formulate online harness learning, where a frozen agent improves by continually updating a structured harness across sequential tasks. This formulation enables a systematic study of key self-improvement factors through our proposed Evo-Harness. At its core, context-to-harness skill compilation distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation. To demonstrate the efficacy of one-shot skill compilation, we evaluate across five realistic benchmarks (TerminalBench2, SWE-bench, CL-Bench, -bench, WebArena-Infinity). Our extensive analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly. Our code is available at https://github.com/A-EVO-Lab/a-evolve/tree/release/evo-harness.

1 Introduction

Evo-Harness formulates online harness learning, in which a frozen agent incrementally updates an external structured harness to convert one-shot executions into reusable skill guidance. It introduces context-to-harness skill compilation and systematically analyzes harness evolution across five realistic benchmarks.

  • Motivation: Failures often remain isolated, causing agents to repeat similar mistakes when executions are not converted into reusable lessons.
  • Limitations of prior work: Existing experience-based methods learn from accumulated trajectories using reflection, memory, or skills, but often lack rigorous validation under realistic constraints and systematic analysis of improvement drivers.
  • Online harness learning: Online harness learning lets a frozen agent incrementally update a structured external harness across sequential tasks, progressively distilling executions into actionable future guidance.
  • Evo-Harness: Evo-Harness compiles single-shot execution contexts into actionable general- and topic-level skill updates that integrate failures, feedback, constraints, and operational procedures.
  • Evaluation and analysis: Evo-Harness is evaluated under sequential continuous learning across TerminalBench2, SWE-bench, CL-Bench, τ-bench, and WebArena-Infinity, while analyzing evolver designs, feedback types, and transfer settings.

2 Related Work

Related work spans self-improvement through reflection, accumulated experiences, evolving agent artifacts, and increasingly operational skill-centered methods. Evo-Harness frames skill harnesses as interpretable structures for filtering noisy single-shot reflections and studying continuous online adaptation.

  • Evolving and self-improving LLM agents: Self-improvement methods use local scalar or verbal feedback, accumulated trajectories, and synthesized insights, interaction rules, or transferable directories.These approaches learn from interaction and feedback through iterative revision or experience mining.
  • Evolving and self-improving LLM agents: Other approaches evolve higher-level agent artifacts, including prompts, reasoning banks, memory structures, and executable code skills.The literature therefore spans both passive artifact evolution and more operational skill-centered evolution.
  • Memory and skill harnesses for agent experience: External memory systems preserve long-term observation histories, organize memories adaptively, and update or retrieve evidence across modalities.These systems treat memory as an external medium for preserving agent experience.
  • Memory and skill harnesses for agent experience: Evo-Harness treats the skill harness as an interpretable medium for studying continuous online adaptation, not merely as a performance-enhancing artifact.It compiles single-shot reflections into structured general and topic skills to filter task-specific artifacts from noisy real-world executions.

3 EVO-HARNESS: Context-to-Harness Skill Compilation

EVO-HARNESS enables online harness learning by compiling noisy, one-shot execution contexts into reusable guidance for future task solving. A frozen solver remains fixed while reflection and evolution update the harness with cross-task patterns and task-type procedures.

  • Framework: EVO-HARNESS compiles noisy execution signals into structured guidance that shapes later planning, acting, verification, and recovery.Unlike experience retrieval, it transforms execution contexts into reusable harness entries rather than recalling past cases.
  • Online Harness Learning: After each task batch, execution contexts are reflected into candidate memories, then evolved into harness updates while solver parameters remain fixed.Contexts include inputs, action trajectories, outcomes, and available feedback such as verifier results, unit tests, tool diagnostics, or judge feedback.
  • Framework: The external harness is a compact control scaffold selected and injected into the frozen solver’s context before each task.It distills prior executions instead of storing raw trajectories and affects behavior through selection and injection.
  • Execution Reflection: Reflection creates candidate memories only for failures or negative feedback, targeting incorrect assumptions, missing constraints, ineffective tools, weak verification, and failed recovery.This focus avoids unnecessary updates from successful executions that may contain task-specific details.
  • Harness Evolution: Evolution filters noisy or redundant memories and applies ADD, MERGE, REVISE, or SKIP edits instead of appending candidate memories directly.The resulting guidance can be promoted, merged, or revised when it is reusable and compatible with the current harness.
  • Reusable Guidance: The evolved harness develops cross-task patterns and localized task-type procedures through batch-level comparison and integration of candidate memories.The modular formulation decouples the solver, evolver, feedback source, and initial harness state; with |B_i| = 1, it reduces to task-type updating.

4 Experiments

Experiments across five diverse agent benchmarks show that EVO-HARNESS consistently improves frozen solvers, especially on tasks with reusable procedural structure. Diagnostic studies show that gains depend on guidance scope, model pairing, online updating, and externally grounded feedback.

  • Experimental setup: EVO-HARNESS is evaluated on WebArena-Infinity, TerminalBench2, SWE-bench Lite, CL-Bench, and TAU-Bench using Claude Opus 4.6 as the default solver and evolver.The study also compares representative experience-, memory-, and skill-based adaptation baselines under the same task split and evaluation protocol.
  • Overall effectiveness: EVO-HARNESS achieves the best results across all five benchmarks and improves over No-Evolve and prior experience-based baselines, with especially large gains on TerminalBench-2.TerminalBench-2 requires file inspection, command execution, error interpretation, and recovery, indicating that evolution helps when tasks expose reusable procedural structure.
  • Category-level effectiveness: On CL-Bench, EVO-HARNESS improves every solver model, with Opus 4.7, Opus 4.6, Opus 4.5, Kimi-K2.5, and GPT-OSS gaining +3.7, +4.5, +3.8, +1.1, and +0.8, respectively.Procedural Task Execution is the most consistent category: Opus 4.6 and Opus 4.5 each gain +9.8 points, whereas Empirical Discovery & Simulation has less stable gains.
  • Harness components: Localized task-type guidance is more useful on CL-Bench, where Topic Only reaches 33.70 versus General Only at 30.28, while broader guidance is nearly sufficient on SWE-bench Lite.The full EVO-HARNESS setting performs best on both benchmarks, reaching 34.02 on CL-Bench and 67.00 on SWE-bench Lite.
  • Online updating and transfer: Online updating performs best: No-Evolve reaches 68.8, Train-Split Transfer reaches 73.4, and Online-Updated reaches 75.0.Training-split skills transfer across tasks and models, while cross-model evolution can help but is not always optimal relative to using the same model for solving and evolution.
  • Feedback grounding: Self-Generated feedback decreases CL-Bench from 29.54 to 27.96 and SWE-bench Lite from 63.67 to 61.67, whereas environment-provided Minimal and Standard feedback are more reliable overall.Standard feedback is best on CL-Bench at 34.02, while Minimal feedback is best on SWE-bench Lite at 67.33 versus Standard feedback at 67.00.

5 Conclusion

The paper formulates online harness learning to study how frozen LLM agents improve through sequential, one-shot executions. Evo-Harness implements this setting by compiling noisy execution contexts into reusable external harnesses and using them to analyze online self-improvement.

  • Online harness learning studies how frozen LLM agents improve from sequential, one-shot executions.
  • Evo-Harness compiles noisy execution contexts into a reusable external harness for future task solving.
  • The harness serves both as a skill-compilation pipeline and an analytical lens for understanding online self-improvement.

Limitations · Appendix

The study is limited to single-agent LLM systems in text-based and tool-use environments, using natural-language harness guidance. It does not cover embodied or multi-agent settings, nor executable-code or structured-program harnesses.

  • Limitations: The evaluation focuses on single-agent LLM systems rather than embodied or multi-agent agents.Excluded settings may require additional interaction, coordination, and environmental feedback.
  • Limitations: Covered environments include text-based, tool-use, web, command-line, and software-engineering tasks.
  • Limitations: Embodied agents are not evaluated in this study.Their harnesses may involve additional forms of interaction and environmental feedback.
  • Limitations: Multi-agent systems are not evaluated because their harnesses may require coordination and additional environmental feedback.
  • Limitations: The harness is instantiated with natural-language guidance.
  • Limitations: Executable code skills and structured programs are not covered as alternative harness formats.

A Benchmark Details

The benchmarks share an online protocol in which tasks arrive as a stream, the solver uses the current harness, and the evolver updates it after observing outcomes. They span stateful web interaction, command-line work, repository repair, adaptive reasoning, and customer-service tool use, with executable, test-based, rubric-based, or database-state evaluation.

  • Common online protocol: Across benchmarks, tasks are processed as a stream, the solver uses the current harness, and the evolver updates it after observing task outcomes.This is the common online protocol summarized in Table 5.
  • WebArena Infinity: WebArena Infinity uses 80 hard, email-centered web tasks requiring stateful browser interaction, with final application state checked programmatically.Tasks include creating labels, changing settings, applying email filters, and coordinating calendar/email information.
  • Terminal-Bench v2: Terminal-Bench v2 evaluates 89 isolated-shell tasks across command-line domains, determining success through executable verifiers inside task-specific Docker images.Domains include software engineering, data processing, scientific computing, security, and system administration.
  • SWE-bench Lite: SWE-bench Lite contains 300 real GitHub issues from 12 Python repositories, with solver patches evaluated by repository tests.The solver edits code before submitting the final patch.
  • Reasoning and tool-use benchmarks: CL-Bench contains 1899 adaptive reasoning tasks graded with rubric criteria, while TAU-Bench contains 165 airline and retail customer-service tool-use tasks evaluated by final database state.CL-Bench covers domain knowledge reasoning, empirical discovery and simulation, procedural task execution, and rule system application.

B Harness Artifacts Examples

Representative artifacts show Evo-Harness converting failed executions into reusable, verifier-grounded skills across web interaction, executable debugging, software repair, and tool use. The learned harnesses target distinct failure modes, including state verification, identity grounding, recovery policies, test expectations, and complete operation execution.

  • WebArena: verified setting: In WebArena task_h10, the evolved run passed after checking existing label state, learning to disable rather than delete labels and verify both operations independently.The baseline left “Support Ticket” enabled despite claiming completion.
  • WebArena: grounded recipients: In WebArena task_h65, the evolved run passed by using role-derived recipients in CC, learning authoritative identity sources and exact email preservation across apps.The baseline used plausible names with wrong domains.
  • Terminal-Bench: executable recovery: In Terminal-Bench task adaptive-rejection-sampler, the evolved run passed after diagnosing multiple failures and learned to shrink repeated failures into minimal reproducers before restoring complexity.The baseline’s formal tests failed after repeated reruns of the same command.
  • SWE-bench: test-grounded repair: In SWE-bench task django__django-11133, the evolved run passed FAIL_TO_PASS: 1/1 and PASS_TO_PASS: 64/64 by anchoring patches in complete failing-test assertions and targeted tests.The baseline timed out after source search and failed test attempts.
  • TAU-Bench: complete tool execution: In TAU-Bench task retail_task_5, the evolved run passed after retrieving the order and applying all requested changes, learning to enumerate operations, select tools by status, and verify full coverage.The baseline verifier failed consistently with partial or wrong-tool execution.

C Which Tasks Are Easier to Improve? · D What the Harness Actually Learns · E Prompt Templates

The merged sections examine which task failures improve after harness evolution, what reusable behaviors the harness learns, and how proposal, curation, and generalization prompts implement context-to-harness compilation.

  • C Which Tasks Are Easier to Improve?: Matched fail-to-pass cases are grouped by observable failure mode to identify which task types turn around after harness evolution.Table 6 provides representative task identifiers from the runs.
  • D What the Harness Actually Learns: Figure 7 presents real skill excerpts supported by matched fail-to-pass cases, pairing each turnaround family with its core learned behavior.The evidence line identifies the relevant turnaround family, while the rule line states the learned behavior.
  • E Prompt Templates: Across experiments, the solver reflects on execution context to propose candidate lessons, and the evolver curates them against the existing harness.This shared proposal-and-curation logic underlies the representative prompt templates.
  • E.1 Solver-Side Proposal Prompt: The solver proposal prompt analyzes evaluation results, verifier or rubric feedback, trajectory signals, compressed trajectories, and related skills to identify missing actions and propose reusable skills.It selects a broad topic, chooses NEW, ENHANCE, or NONE, writes a trigger-style description, and filters aggressively.
  • E.2 Evolver-Side Curator Prompt: The evolver curator chooses ACCEPT, MERGE, or SKIP for each proposal while favoring nonduplication, budget compliance, clear triggers, concise actionable content, and multi-task usefulness.The prompt applies a generalizability test against multiple unseen tasks.
  • E.3 General Skill Curator: The general skill curator updates skills only when repeated patterns span multiple contexts, encoding context-independent procedures for planning, verification, recovery, or tool use.General skills must avoid context-specific references and capture cross-task guidance.

F Experimental Details … H.3 Artifact Use Consistent With Intended Use

The experiments use benchmark-specific adapters and sandboxed environments, while Evo-Harness stores and retrieves inspectable procedural skills under a bounded budget. The study uses public artifacts and standard model APIs, releases its code and skill libraries, and reports intended-purpose, cited, and licensed artifact use.

  • F Experimental Details: Benchmark-specific adapters support WebArena, Terminal-Bench, SWE-bench, TAU-Bench, and CL-Bench in their respective execution and evaluation environments.WebArena uses persistent browser state; Terminal-Bench uses Docker images and executable verifiers; SWE-bench uses isolated repositories and unit tests; TAU-Bench uses its official tool-use verifier; CL-Bench uses rubric-based judging.
  • F Experimental Details: The harness is an inspectable Markdown skill library with YAML metadata, retrieval triggers, actionable procedures, optional execution evidence, and cross-task or task-type scope.Claude Sonnet 4.5 retrieves relevant skills before task execution.
  • F Experimental Details: A batch size of 16 and a maximum of 5 general skills plus 5 skills per task-type topic constrain retrieval and encourage merging overlapping guidance.The budget is intended to avoid overly detailed or task-specific records.
  • G Potential Risks: Evo-Harness injects procedural skills covering navigation, tool use, verification, and recovery into prompts, with all benchmarks running in sandboxes without real user, external-account, or production-system access.The evaluated setting therefore poses minimal practical risk beyond standard benchmark execution.
  • H Use Or Create Scientific Artifacts: The study uses five public benchmarks—WebArena-Infinity, TerminalBench2, SWE-bench Lite, CL-Bench, and TAU-Bench—without modifying them, and uses Claude and other foundation models through standard AWS Bedrock APIs.The EVO-HARNESS pipeline code and evolved skill libraries will be released upon publication.
  • H.1 Cite Creators Of Artifacts: All benchmarks and models are properly cited in the main text.This section explicitly states that artifact creators are cited.
  • H.2 Discuss The License For Artifacts: The benchmark resources are publicly available under their respective licenses, including MIT, Apache-2.0, and a custom evaluation-only license, while the pipeline code will use the MIT License.WebArena Infinity and TAU-Bench use MIT licenses; WebArena uses Apache-2.0; CL-Bench uses a custom evaluation-only license.
  • H.3 Artifact Use Consistent With Intended Use: All benchmarks are used to evaluate agent capabilities as intended, and foundation models are accessed through standard APIs in accordance with their terms of service.The statement covers artifact use consistent with intended purpose and API terms.

H.4 Data Contains Personally Identifying Info Or Offensive Content … J AI Assistants In Research Or Writing

The benchmarks use synthetic or sanitized scenarios and, to the authors’ knowledge, contain no personally identifying information or offensive content. Experiments use fixed seeds, full benchmark task sets, three-run averages, and frozen LLM backbones, while AI assistants only polished the manuscript.

  • H.4 Data Contains Personally Identifying Info Or Offensive Content: Benchmarks use synthetic or sanitized scenarios, and SWE-bench uses public GitHub issues; none are known to contain personally identifying information or offensive content.
  • I.1 Compute Budget: The experimental suite spans multiple benchmarks, solver models, evolution settings, and ablation configurations, with total API costs on the order of $100K.A single benchmark configuration typically uses 16 parallel workers.
  • I.1 Compute Budget: Software-engineering and command-line tasks take longer because they require repository setup, Docker execution, test running, or verifier calls.Web and tool-use benchmarks are usually faster.
  • I.2 Experimental Setup And Hyper-params: Key hyperparameters are described in Appendix F, and task shuffling uses fixed random seeds (42) to ensure reproducibility.
  • I.3 Descriptive Statistics: Pass rate (%) is the primary metric across all benchmarks, with results reported over each benchmark’s full task set and averaged over three runs.
  • I.4 Parameters For Packages: The software environment uses AWS Bedrock with AWS infrastructure services, Python 3.10+, PyTorch 2.0+, CUDA 12.0+, Docker, and Ubuntu 24.04.
  • J AI Assistants In Research Or Writing: The paper studies LLMs using frozen LLM backbones for inference, while reported numerical results come from experimental code and are checked by the authors.The models are not used to create benchmark labels or fabricate experimental results.
  • J AI Assistants In Research Or Writing: AI assistant tools polished the manuscript but did not decide scientific claims, select reported results, or replace author verification.
Loading 2608.15071v1…