PaperlayerView exact revision on arXiv

Source-linked AI summary

Recursive Harness Self-Improvement

Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, Yujin Tang

arXiv:2607.15524v1cs.LGcs.AI

TL;DR

Provider-built harnesses must generalize across diverse users and tasks, making continual updates prohibitively labor- and cost-intensive. RHI treats the harness as a prompt-level specification and iteratively refines it through trajectory-local self-comparison using pairwise preference feedback. Across synthetic research tasks, few-shot RHI achieves higher win rates than stronger test-time-scaling settings and reduces inference cost by up to 60% relative to the ultracode baseline.

  • Problem

    Provider-built harnesses must generalize across diverse users and tasks, making continual updates prohibitively labor- and cost-intensive.

  • Method

    RHI treats the harness as a prompt-level specification and iteratively refines it through trajectory-local self-comparison using pairwise preference feedback.

  • Results

    Across synthetic research tasks, few-shot RHI achieves higher win rates than stronger test-time-scaling settings and reduces inference cost by up to 60% relative to the ultracode baseline.

  • Takeaways & Limitations

    RHI offers a practical approach for task-specific optimization of user-constructed harnesses to improve the quality of agent execution traces.

  • Takeaways & Limitations

    Because the coding agent is black-box, embedding analysis cannot directly establish how textual harness changes affect the model’s internal computation or output distribution.

Abstract

from arXiv · show

Under model--harness co-evolution, harnesses are not merely inference-time scaffolds but data-generating components whose execution traces can shape future foundation models. This motivates harness-in-the-loop learning: optimizing harnesses for both immediate agent performance and the quality of traces used for future model training. However, continually updating provider-built scaffolds is costly and labor-intensive. We therefore investigate whether optimizing user-constructed harnesses in a task-specific manner can improve execution-trace quality while remaining computationally lightweight and requiring only a few update iterations. To this end, we introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback over its own revision history. Across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations suffice to substantially raise the performance ceiling of low-reasoning-effort agents, exceeding the corresponding maximum-reasoning-effort setting while reducing inference cost by up to 60%. We show that these gains arise primarily from improved task-specific context management through more effective inter-agent information flow rather than longer reasoning traces. Finally, we formalize this behavior as an information-theoretic hypothesis for RHI's implicit optimization objective, suggesting RHI as a practical algorithm for continual learning within the paradigm of model--harness co-evolution.

1. Introduction

The paper frames harness–model co-evolution around improving execution traces from user-constructed harnesses. It introduces RHI, a lightweight prompt-level self-improvement method that raises low-effort agents’ performance while reducing inference cost.

  • Introduction: The work targets user-constructed harnesses because provider-built harnesses must generalize broadly, making continual updates labor- and cost-intensive, while task-specific optimization enables practical specialization.The broader motivation is improving execution-trace quality within the harness–model co-evolutionary feedback loop.
  • Introduction: RHI represents the harness as a prompt-level specification of agent roles, instructions, inter-agent contracts, and workflow hops.Roles and instructions define task allocation, contracts specify communication, and hops govern when reasoning is invoked.
  • Introduction: RHI performs lightweight trajectory-local self-comparison, comparing each harness revision with its immediate predecessor and using preference history to guide subsequent revisions.This replaces expensive population-level harness search and requires only a few update iterations to raise low-reasoning-effort performance plateaus.
  • Introduction: RHI’s gains primarily reflect improved task-specific context management rather than longer reasoning traces, as communication contracts and workflow hops become more task-specific.Output-token usage remains nearly constant across iterations, while cache usage and inference cost often decrease.

2. Preliminary

The paper formalizes harness optimization as selecting a task-specific harness whose generated repositories outperform competing harnesses under multi-criterion evaluation. It then frames existing search methods as finite-population approximations whose candidate-wise execution and evaluation costs limit continual user-specific optimization.

  • A fixed-language-model coding agent maps a task prompt and harness to a generated code repository.
  • Open-ended repositories require evaluation across functional correctness, task alignment, reproducibility, and code quality rather than a single reliable scalar metric.
  • The ideal task-specific harness maximizes expected pairwise win rate against harnesses sampled from a reference distribution.
  • Existing population-based methods make optimization tractable by restricting harness representations and replacing the reference distribution with finite candidate populations.Examples span harness code, workflow graphs, agent programs, prompts, and modular LLM programs, including Meta-Harness, AutoHarness, ADAS, GPTSwarm, OPRO, and TextGrad.
  • Every additional candidate requires a fresh black-box execution and evaluation, making population search prohibitive for users continually specializing agents across open-ended tasks.Wang et al. (2026) report that accounting for search cost causes automatic harness evolution to fail to consistently outperform simple test-time scaling baselines.

3. Recursive Harness Self-Improvement

Recursive Harness Self-Improvement (RHI) iteratively refines a task-specific, prompt-level harness using pairwise feedback on successive agent outputs and accumulated self-history. Its trajectory-local objective makes harness optimization computationally lightweight while preserving the ideal objective’s utility ordering under a standard pairwise-preference model.

  • RHI algorithm: RHI updates a task-specific harness by having an agent solve the task, comparing its output with the previous output, storing the preference, and using an LLM optimizer to revise the harness.Figure 2 summarizes this loop, while Algorithm 1 presents the complete procedure.
  • Computationally lightweight objective: RHI compares each current harness only with its immediate predecessor, requiring one new execution trace and one pairwise evaluation per iteration instead of population-wide or quadratic comparisons.Cached outputs are reused, reducing the cost of both agent execution and pairwise evaluation.
  • Trajectory-local objective: Under a standard pairwise-preference model, the trajectory-local objective targets the same latent utility ordering as the ideal population objective, providing a noisy local-ascent signal.Winning revisions are encouraged, while losing revisions are discarded or revised.
  • Self-history: RHI accumulates preference feedback across revisions so each update conditions on self-history rather than a single noisy comparison, improving robustness under a limited update budget.The accumulated history acts as a momentum-semantic signal while retaining one new agent execution and one pairwise evaluation per iteration.
  • Recursive self-improvement: The harness optimizer indirectly aligns revisions with the evaluation prompt through evaluator-generated pairwise feedback, without directly observing that prompt.This defines RHI as recursive self-improvement through the preference history induced by its own previous revisions.

4. Benchmark & Evaluation

RHI is evaluated on 30 synthetic, open-ended ML research tasks across quantitative finance, robotics, and pharmaceutical ML using standardized repository deliverables and pairwise LLM evaluation.

  • Benchmark construction: The benchmark contains 30 tasks spanning quantitative finance, robotics, and pharmaceutical machine learning, with 10 tasks per domain.An LLM transforms domain-relevant industry job postings into research-style prompts.
  • Benchmark construction: Tasks combine coding, domain reasoning, empirical analysis, and ML experimentation, requiring complete repositories with research reports, visualizations, metrics, and indexed file paths.Core deliverables are research_report.md, .png visualizations, metrics.json, and index.json.
  • Evaluation protocol: Repositories are compared pairwise by an LLM evaluator using only task-specified deliverables, keeping the evaluator prompt within approximately 30–40% of its maximum input length.The standardized deliverable structure provides a common evaluation interface while avoiding impractical inclusion of every generated artifact.
  • Evaluation protocol: The evaluator judges deliverable coverage, numerical and empirical rigor, reproducibility, and presentation when selecting the preferred repository.These criteria assess compliance, methodological consistency, reproducibility resources, and output quality.

5. Experiment

Across progressively stronger base models, a few RHI iterations surpass same-family test-time scaling baselines while reducing inference cost by up to 60%. The gains are associated with improved context management rather than longer generations.

  • Experimental setup: RHI evaluates the same coding agents with task-specific textual harness specifications, comparing them against higher-reasoning-effort agents without RHI across three base models.The reported metrics include pairwise performance, normalized cost, output-token count, and cache read/write usage.
  • Main results: After two iterations, sonnet-4.6-high+H [2] wins 20 of 30 comparisons against sonnet-4.6-max, while one or two iterations suffice to surpass stronger Opus baselines.Opus-4.7 surpasses both xhigh and max after one iteration; Opus-4.8 surpasses all test-time scaling baselines after two iterations.
  • Harness comparison: Opus-4.8-high+H [2] outperforms opus-4.8-ultracode, indicating that task-specific prompt-level harness optimization can exceed a fixed provider-built multi-agent harness.The result supports adapting multi-agent workflows to heterogeneous tasks through user-constructed harnesses rather than relying on a single system-level harness.
  • Mechanism: RHI’s performance gains are not primarily driven by longer generations, because output-token usage remains nearly constant while performance improves across iterations.Normalized output-token usage ranges from 1.71 to 1.86 for Sonnet-4.6 and from 1.42 to 1.81 for Opus-4.8; the gains instead track more efficient KV-cache usage.
  • Cost efficiency: Up to 60% lower normalized cost is achieved by RHI while improving performance across all three base models.For opus-4.8-high+H [2], cost is 23% below opus-4.8-max and 60% below opus-4.8-ultracode; sonnet-4.6-high+H [2] is 7% below sonnet-4.6-max, and opus-4.7-high+H [1] is 18% below opus-4.7-max.

6. Ablation Study

RHI improves weaker agents beyond train-time scaling, although it does not consistently match stronger-model baselines. Ablations indicate that gains arise mainly from task-specific coordination, especially contract and hop refinement, motivating a correlational information-theoretic objective hypothesis.

  • 6.1 Beyond Train-Time Scaling: RHI substantially improves sonnet-4.6-high+H, with gains plateauing after 2–4 iterations, but does not consistently close the gap to opus-4.7 baselines.Figure 8 compares sonnet-4.6-high+H across iterations against opus-4.7-high and opus-4.7-xhigh.
  • 6.2.1 Whole-Harness Analysis: RHI’s updates progressively diverge from the initial harness, with cumulative similarity decreasing from 0.82 to 0.75, while later transitions become more incremental.The first update has lower transition similarity (0.82) than later transitions (0.97, 0.98, 0.99), indicating the largest semantic change occurs initially.
  • 6.2.1 Whole-Harness Analysis: RHI produces systematic, task-dependent harness changes: the largest semantic separation occurs in contracts, followed by hops and instructions, while roles are less separable.Whole-harness embeddings separate the initial and improved harnesses, but cannot identify responsible components; component analysis finds contracts most task-dependent across domains.
  • 6.2.2 Component-Level Analysis: Contracts specialize within a few iterations, with consecutive similarity rising from 0.48 to 0.72, whereas roles, instructions, and hops evolve more gradually.The results suggest pairwise feedback provides a stronger or more effective signal for contract refinement than for other components.
  • 6.3 Implicit Objective: The proposed implicit objective hypothesizes that prompt-targeted coordination increases task information in contracts and hops while reducing redundancy among harness components.Table 2 shows monotonic increases for contracts and hops and decreases for roles; Table 3 reports debiased total-correlation reductions from 4.84 to 3.63 nats and 3.51 to 2.62 nats in two configurations.
  • 6.3 Implicit Objective: This objective is a plausible, correlational hypothesis rather than proof of a unique latent objective, because the evidence relies on embedding-based estimators.The analysis models update trajectories as guided by an implicit preference over task-specific components without claiming to identify the optimizer’s true objective.

7. Related Work

RHI is a bounded, prompt-level form of recursive self-improvement that optimizes a reusable multi-agent harness around a fixed foundation model. It differs from prior harness, workflow, prompt, and multi-agent methods in its optimization object and trajectory-local preference-based revision.

  • Multi-agent systems and test-time scaling: RHI separates coordination improvements from test-time scaling by revising system organization instead of merely adding agents, calls, or longer generations.This distinction addresses the possibility that multi-agent gains arise from increased computation rather than better coordination.
  • Recursive self-improvement and harness-level RSI: RHI rewrites and reuses a prompt-represented harness from self-comparison history while keeping the base model, evaluator, and optimizer fixed.Its recursion operates over the harness trajectory rather than the model itself.
  • Harness optimization and system scaling: Unlike executable-code or population-based system optimizers, RHI performs one trajectory-local comparison per iteration using LLM preference feedback, including for open-ended tasks.Meta-Harness searches executable harness code, whereas TTHE evolves parallel branches and selects them with execution-derived proxies.
  • Prompt and pipeline optimization: RHI optimizes a multi-agent harness containing roles, instructions, hops, and communication contracts rather than a single prompt or modular language-model call.Its ablations associate gains primarily with workflow and contract components rather than longer or better single-agent prompts.

8. Conclusion

The paper argues that harness–model co-evolution should improve agent execution traces as post-training data for future foundation models, presenting RHI as a practical task-specific approach. Future work will study how to internalize these traces into future foundation models.

  • Harness–model co-evolution should prioritize improving agent execution traces because they can serve as post-training data for future foundation models.
  • Recursive Harness Self-Improvement (RHI) offers a practical approach by task-specifically optimizing user-constructed harnesses.
  • Future work will investigate how the resulting execution traces can be effectively internalized into future foundation models.

A. QnAs

The results show that explicit multi-agent harness specifications improve cost–performance over built-in coordination, while multi-agent execution itself adds value beyond role-and-instruction prompting. RHI’s previous-harness comparison retains earlier history through a compressed momentum signal.

  • Q1: Although each update compares with the immediately preceding harness, D_x^(i) compresses comparisons across harness history and preserves earlier information as a momentum signal.Thus, earlier harness history is retained rather than discarded in the update rule.
  • Q2: RHI prioritizes multi-agent harness components because built-in coordination is unreliable and prompt-represented multi-agent harnesses outperform prompt-represented single-agent multi-persona harnesses.These observations motivate focusing improvement on contracts and hops rather than components specific to a single agent.
  • Q2: Figure 13 shows that explicit multi-agent harnesses outperform built-in coordination and single-agent multi-persona execution on Elo and cost–performance.Built-in multi-agent harnesses incur higher cost yet lower Elo than single-agent defaults, whereas the explicit harness improves Elo at lower or comparable cost and beats single(unionOur).
  • Q2: The gains require multi-agent execution beyond adding roles and instructions, relying on specialized agents, independent contexts, and explicit orchestration.single(unionOur) remains competitive and can beat single(default), but multi(ours) performs best.

B. Examples of RHI harness · C. Harness optimizer prompt · 1. Following query solved with claude coding

The appendix illustrates how RHI refines a domain-specific agent harness across five iterations for a pharmaceutical machine-learning task. The revised harness emphasizes structured specialist collaboration, explicit interface contracts, reproducibility checks, and targeted recall before final acceptance.

  • B. Examples of RHI harness: RHI evaluates harnesses H[i] for i∈0, 1, 2, 3, 4 on a pharmaceutical ML task using sonnet-4.6-high, with outputs included in Figure 5a.The example task implements an SE(3)-equivariant network for side-chain χ-angle prediction from backbone context using PDB-derived data, angular-error evaluation, and profiling.
  • B. Examples of RHI harness: The initial harness uses a domain-specific agent team spanning orchestration, structural data, geometric ML, generative modeling, training scale, force fields, evaluation, communication, portfolio execution, and quality safety.These specialists cover scientific design, implementation planning, benchmarking, publication, cross-functional execution, and compliance review.
  • B. Examples of RHI harness: Later harness revisions replace broad delegation with multi-round orchestration, acceptance rubrics, parallel design fan-out, reconciled interface contracts, evidence review, targeted patches, and final gating.The workflow defines contracts for datasets, model inputs and outputs, metrics, ablations, profiling, plots, and reports before downstream implementation or claims.
  • B. Examples of RHI harness: The refined harness explicitly addresses prior failures involving incorrect deliverable paths, unsupported RCSB acquisition claims, missing provenance, weak ablations, invalid profiling, and inconsistent quantitative reporting.It requires exact paths such as src/profile.py, verifiable data manifests, leakage checks, frozen results contracts, and independent consistency review.
  • B. Examples of RHI harness: The harness requires independent validation of code, schemas, plots, provenance, exact files, reproducibility, scientific claims, equivariance language, profiling, leakage, and compliance before acceptance.Artifact and quality reviewers inspect the workspace during early and final gates and can trigger targeted corrections when evidence is missing or inconsistent.
  • B. Examples of RHI harness: The revised design formalizes machine-checkable HandoffV2 communication, versioned artifacts, evidence bundles, acknowledgements or change requests, and mandatory dependencies among data, modeling, evaluation, and reporting agents.RequiredPathManifest, DatasetRecord v2, DataProvenanceManifest v1, SplitLeakageReview v1, ModelRegistry v1, ModelIOContract v2, AblationGrid v2, and ResultsContract v2 structure the workflow.

2. Current submission code repo from Claude Code (evidence only)

The current submission repository comparison finds the two submissions indistinguishable, with identical artifact coverage and shared quantitative-quality defects. Neither submission is superior, while both require corrections to consistency, stability analysis, JSON validity, and potential feature leakage.

  • Repository comparison: The submissions are tied because the provided evidence shows identical workspaces, artifacts, strengths, and weaknesses.Both point to the same workspace path and matching repository contents, so neither submission is better.
  • Repository comparison: Both repositories materially satisfy the artifact checklist, including the report, indexed deliverables, nine plots, organized code, pinned dependencies, and results files.The required PNGs are present under deliverables/plots/, with run_pipeline.py as the entry point and the principal JSON artifacts available.
  • Quantitative quality: Both reports contain quantitative inconsistencies: they misstate GBM significance and primary-model identity, and claim ElasticNet beats models it trails at 5-day IC.The reported 5-day GBM IC is 0.039 with p=0.122; ElasticNet’s 5-day IC is −0.0458 versus Ridge’s 0.0896 and GBM’s 0.0391.
  • Limitations: Both fail substantive pre/post-2020 stability analysis, include non-standard JSON NaN tokens, and leave potential feature leakage unresolved in proxy shock construction.The stability fields are empty or null, while the excerpts do not prove post-event shock features are safely lagged before overlapping return prediction.

3b. Pairwise history delta checklist (recurring issues to fix) · 4. Instructions

The checklist prioritizes recurring historical defects, while the instructions revise the multi-agent design to create measurable advantages over single-agent execution through structured communication, feedback, and verification.

  • 3b. Pairwise history delta checklist (recurring issues to fix): The revision process prioritizes recurring history issues, including metric inconsistency, robustness gaps, null/NaN defects, reproducibility gaps, and weak validation or testing.Higher counts indicate issues repeated more often in the revision history.
  • 4. Instructions: The design update preserves the query’s original intent and deliverables while addressing weaknesses identified by evidence.
  • 4. Instructions: Output-to-orchestrator contracts should produce structured downstream inputs, and orchestrators should recall subagents with narrower scopes and updated acceptance criteria.
  • 4. Instructions: The revised design must create genuine multi-agent advantages through specialist parallelism, cross-agent validation, inter-agent communication, and conflict-resolution loops.
  • 4. Instructions: Feedback must be specific and evidence-grounded, citing concrete files, metrics, or history signals rather than generic coordination claims.
  • 4. Instructions: The proposed response must provide a replacement design block, enumerate changes, explain mechanistic benefits, identify evidence, state expected impacts, and define verification checks.
  • 4. Instructions: The replacement design block must begin with the specified instruction to create an agent team from candidate agents for solving the problem.

D. Task examples

The appendix presents representative examples from a 30-task evaluation benchmark spanning robotics, pharmacy, and quantitative research.

  • D. Task examples: The evaluation benchmark contains 30 tasks across three domains.The domains are robotics, pharmacy, and quantitative research.
  • D. Task examples: Each domain contributes 10 tasks to the benchmark.
  • D. Task examples: The appendix provides representative examples of the benchmark’s task domains.

D.1. ML Research Task (Robotics)

This robotics task requires a reproducible fine-tuning experiment for a small vision-language model on robot-relevant instruction classification. It evaluates accuracy, calibration, and paraphrase robustness using a public or curated action-caption dataset.

  • Task: The experiment fine-tunes a small vision-language model for robot-relevant instruction classification, including action type, object category, and spatial relation.EPIC-KITCHENS captions aligned with actions or a curated Something-Something V2 subset may serve as the dataset.
  • Evaluation: Evaluation covers accuracy, calibration, and robustness to paraphrases.Required visualizations include per-class accuracy, confusion matrices, calibration reliability, paraphrase robustness drop, and learning curves.
  • Deliverables: Deliverables comprise a conference-style research report, indexed outputs, plots, reproducible code, dataset preparation, fine-tuning, evaluation, paraphrase generation, and requirements.txt.Paraphrases may be generated with a public paraphrase model or rule-based templates.

D.2. ML Research Task (Quantitative)

This quantitative research task tests whether Wikipedia pageviews improve short-term return prediction for Dow 30 equities, using leakage-safe evaluation and a market-neutral backtest. It also requires placebo testing, reproducible artifacts, and comprehensive predictive and portfolio metrics.

  • Task and data: The task investigates whether Wikipedia pageviews improve next-day and next-week return or volatility prediction for Dow 30 constituents, with SPY as the benchmark.The data universe uses Wikipedia-listed Dow 30 constituents and SPY, with prices and trading volume from Yahoo Finance.
  • Feature construction: Features include abnormal pageview surges, attention momentum, day-of-week adjustments, and interactions with earnings dates.The task also permits optional macroeconomic controls and requires documenting fallback procedures if earnings-calendar scraping is unstable.
  • Modeling and evaluation: Evaluation uses leakage-safe modeling, purged time-series cross-validation, and a market-neutral long/short attention-strategy backtest with transaction costs.A causal-style placebo shifts pageviews by +7 days and checks whether the predictive signal disappears.
  • Deliverables: Deliverables include a conference-style research report, reproducible Python code, visualizations, mappings, model comparisons, and metrics covering prediction, risk, trading, and placebo performance.Required metrics include IC, RankIC, Predictive R2, Sharpe ratio, beta, maximum drawdown, turnover, transaction costs, and placebo-test metrics.

D.3. ML Research Task (Pharmacy)

This task evaluates whether fusing ESM-2 protein sequence embeddings with PDB-derived graph features improves enzyme commission number prediction over sequence-only and structure-only representations. It requires reproducible data, modeling, evaluation, and reporting deliverables.

  • Task: The experiment compares sequence-only, structure-only, and fused representations for predicting enzyme commission numbers from protein sequences and structures.It focuses on proteins with known structures and uses ESM-2 embeddings with graph features derived from the Protein Data Bank.
  • Data sources: The study uses public UniProt/Swiss-Prot enzyme annotations and PDB mappings obtained through the public SIFTS database.
  • Success criterion and deliverables: Success is assessed by whether multimodal fusion improves prediction performance, using macro and micro F1, hierarchical accuracy across EC levels 1–4, and coverage statistics.Deliverables include a conference-style report, indexed files, evaluation plots, reproducible Python code, and metrics in results/metrics.json.

E. Evaluator Prompt

The evaluator uses Leval to compare two completed submissions pairwise under a strict, evidence-driven quantitative-research rubric. Its prompt treats the task specification as the sole contract, uses workspace contents only as evidence, and requires checking explicitly requested artifacts.

  • Pairwise comparison: Leval compares two completed submissions as a senior quantitative researcher, judging empirical rigor, leakage discipline, uncertainty, baselines, reproducibility, and consistency between narrative and numeric artifacts.The evaluator is instructed to be strict, practical, and evidence-driven.
  • Evaluation rules: The task block is the only production contract, while workspace sections provide evidence only and cannot justify invented file contents.The prompt also instructs the evaluator to use every explicitly required artifact named in the task.
  • Example task: The example task requires evaluating robot-plan success-probability calibration using Robosuite data, predictive modeling, ECE, AUROC, and decision utility.The task specifies varying execution difficulty and asks whether to execute or replan.
  • Example task: The required deliverables include a conference-style report, index, five named plots, reproducible code, and metrics covering AUROC, average precision, and ECE.The prompt names reliability, temperature, ROC, decision-utility, and difficulty-bin visualizations.

F. Distribution of normalized cost, output tokens, and cache read/write

Figure 14 summarizes the distributions of normalized cost, output tokens, and cache read/write across 30 ML synthetic tasks, with boxplot trends consistent with the corresponding mean values.

  • Figure 14’s boxplots show distributions of normalized cost, output tokens, and cache read/write across 30 ML synthetic tasks, consistent with mean values in Figures 5b, 6c, and 7e.
Loading 2607.15524v1…