PaperlayerView exact revision on arXiv

Source-linked AI summary

TTHE: Test-Time Harness Evolution

Jun Nie, Yonggang Zhang, Jun Song, Qianshu Cai, Dahai Yu, Yike Guo, Xinmei Tian, Bo Han

arXiv:2607.08124v1cs.SEcs.LG

TL;DR

Existing approaches generally freeze agent workflows before evaluation, leaving open whether a harness can adapt during evaluation from its own unlabeled traces. TTHE evolves executable harnesses around a frozen LLM and improves fixed ReAct-style baselines across evaluated tasks, while its transductive protocol does not establish forward generalization.

  • Problem

    Most agent harnesses are fixed during evaluation, while existing adaptation approaches do not modify the persistent control program from unlabeled test-time traces.

    Source: §1 · pp. 1–2

  • Method

    TTHE maintains and refines candidate executable harnesses from unlabeled execution traces, then commits a candidate that persists, with the model weights fixed and gold withheld during adaptation.

    Source: §1 · p. 2, §4.1 · p. 4, §4.2 · p. 5

  • Results

    TTHE improves fixed ReAct-style baseline harnesses across diverse execution-grounded tasks, including text-to-SQL, competitive programming, software engineering, data-science coding, and agentic tool use.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Within the studied transductive setting, executable harnesses provide a practical test-time adaptation state, while proxy reliability remains a key challenge for robust evolution.

    Source: §7 · p. 10

    The transductive protocol evaluates a harness on the same batch whose traces informed its selection and does not establish forward generalization to subsequent batches.

    Source: §6 · p. 10

Abstract

from arXiv · show

The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures. Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time. This limits adaptation when the test distribution, failure modes, or tool interactions differ from those seen during development. We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs. We introduce Test-Time Harness Evolution (TTHE), which treats the executable harness as the state of test-time adaptation. During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces, without gold labels or task-specific supervision; a judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs. Crucially, TTHE does not update model weights, require gold labels, or train a separate adaptation model: solver, proposers, and judge are different roles and harnesses around the same frozen LLM, so all adaptation occurs through changes to the surrounding program. Across text-to-SQL, competitive programming, software engineering, data-science coding, and agentic tool-use tasks, TTHE improves fixed ReAct-style baseline harnesses, yielding persistent, inspectable improvements rather than a pre-searched workflow or per-query retries. These results recast test-time adaptation for LLM agents as evolution over executable control programs and identify execution-derived proxy reliability as a central challenge for robust unsupervised agent improvement.

1 INTRODUCTION

LLM agents’ behavior depends on their executable harness, yet harnesses are generally fixed during evaluation. TTHE evolves that harness from unlabeled execution traces, and improves fixed ReAct-style baselines across diverse execution-grounded tasks while exposing proxy reliability as a key challenge.

1 INTRODUCTION

  • Execution traces provide unlabeled evidence about operating procedures, including repeated normalization inconsistencies, unverified patches, and omitted results.Such traces are available during evaluation and do not require gold labels.Source: §1 · pp. 1–2
  • Existing test-time methods adapt weights, individual responses, or workflows searched before evaluation, leaving the persistent control program fixed during testing.TTHE asks whether that program can instead adapt from its own traces without labels, human repair signals, or an additional repair model.Source: §1 · p. 2
  • TTHE evolves the executable harness during evaluation from unlabeled traces, then commits a candidate whose program persists to govern subsequent inputs.Gold labels are consulted only after selection for measurement.Source: §1 · p. 2
  • The adaptation loop uses different solver, proposer, and judge roles around the same frozen LLM, changing the surrounding program rather than model weights.It does not introduce a separately trained adaptation model.Source: §1 · pp. 2–3
  • Executable harnesses can express context construction, multi-sample generation, tool-mediated grounding, contract checks, and conditional recovery branches.The flexibility comes with incomplete execution evidence: successful execution can still yield an incorrect answer or a proxy-incompatible choice.Source: §1 · p. 2
  • TTHE improves fixed ReAct-style baseline harnesses across diverse execution-grounded tasks, while performance is non-monotonic in search budget and judge reliability remains a bottleneck.Trace audits identify cases where the judge commits a plausible but incorrect program under imperfect proxy signals.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work optimizes prompts, workflows, or agent source code, but most such systems evaluate a fixed artifact after optimization. TTHE instead adapts the executable harness within evaluation using execution-derived proxies without task-level gold.

Self-evolving agents and harness optimization

  • Most prompt, pipeline, and source-rewriting systems optimize against development objectives, curated failures, or external success signals before evaluating a fixed artifact.TTHE moves harness optimization inside evaluation, where candidates are judged from execution-derived proxies on the current unlabeled batch.Source: §2 · p. 3

Test-time adaptation

  • Unlike test-time training or bounded configuration search, TTHE changes executable harness code using native task traces and preserves the result as inspectable source.Code edits can add grounding, verification, or recovery logic beyond parameter or hyperparameter settings.Source: §2 · p. 3

Response-level inference and execution feedback

  • Response-level methods improve answers within fixed scaffolds, whereas TTHE makes grounding, verification, and repair persistent executable policies learned without labels.Its judge adjudicates execution traces without a labeled correctness signal.Source: §2 · pp. 3–4

3 PRELIMINARIES

The preliminaries define a harness as a program that wraps a fixed model and produces outputs with execution traces. Gold labels define the measurement objective but remain unavailable during adaptation, which changes the harness using inputs, traces, and execution probes.

3 PRELIMINARIES

  • A harness H wraps a fixed model and execution environment, producing an output-trace pair for each input; gold labels are reserved for evaluation.The measurement objective J⋆(H) is expected task accuracy, with correctness defined according to the task rather than literal output equality.Source: §3 · p. 4
  • At each adaptation step, the current unlabeled batch and candidate harnesses’ traces inform proposals, while a judge commits a harness to carry forward.The judge may use inputs, traces, and additional execution probes without assuming a calibrated scalar reward.Source: §3 · p. 4
  • The adapted state is the executable scaffold, not model parameters or normalization statistics.This distinguishes TTHE’s program changes from test-time training and adaptation.Source: §3 · p. 4

4 METHOD

TTHE adapts an executable harness during evaluation using unlabeled execution traces and proxy signals, then commits a harness that persists across batches. Its loop evolves diverse candidate branches around a frozen solver and selects among them without gold labels.

4.1 OVERVIEW

  • TTHE evolves candidate harnesses on each unlabeled batch using execution evidence, then commits one harness that carries forward to the next batch.The judge selects from final branches using execution health, trace-level signals, round-trip consistency, and public-test outcomes when available; gold is reserved for post-commit measurement.Source: §4.1 · p. 4, Figure 1 · p. 5

4.2 HARNESS AND ADAPTATION STATE

  • The adaptation state is executable harness code wrapping a frozen solver, rather than a prompt template or fixed menu of workflow switches.The harness controls context construction, tool use, intermediate validation, and recovery; proposers can make program-level changes such as grounding, decomposition, contract checks, or execution-feedback repair.Source: §4.2 · p. 5

4.3 THE OPTIMIZATION SIGNAL

  • TTHE gives its proposer and judge detailed gold-free traces and imperfect, task-dependent proxies instead of collapsing evidence into one scalar reward.The proxies include execution health, round-trip consistency, and public-test pass rate; public tests are incomplete evidence, and agreement can repeat a shared model error.Source: §4.3 · pp. 5–6

4.4 TEST-TIME HARNESS EVOLUTION

  • Each batch evolves G parallel branches for R rounds; proposers edit their own parent harnesses using other branches’ traces, and invalid children fall back to their parents.The loop observes harness executions and returns traces, cached outputs, and proxy signals before proposing edits.Source: §4.4 · p. 6
  • Branch lineages and distinct edit objectives support compounding changes, but improvement requires both generating better behavior and having the judge commit it.The judge selects only among final-round branches, may probe or re-execute candidates, and never sees gold; reported accuracy is transductive because the selected harness is measured on the batch used for selection.Source: §4.4 · p. 6

4.5 IMPLEMENTATION AND ADAPTATION BOUNDARY

  • The solver, proposers, and judge use the same frozen backbone, so adaptation comes from code changes around the solver rather than a separate adaptation model.Gold labels, reference outputs, and hidden tests are excluded from adaptation and used only for post-selection measurement.Source: §4.5 · p. 6
  • Fixed resource and wall-clock budgets, candidate exclusions, re-execution-backed judging, and empirical audits bound the open-ended code search.The exclusions cover malformed or non-terminating candidates.Source: §4.5 · p. 7

5 EXPERIMENTS

The experiments evaluate TTHE against fixed ReAct-style baselines on hard slices spanning code, SQL, and agentic tool use. They test cross-task performance, emergent policies, and how search, accumulation, and candidate selection shape results.

5.1 TASKS, MODELS, AND PROTOCOL

  • The evaluation uses fixed hard slices across four code and query-generation tasks, with gold used for measurement rather than adaptation.The baseline is unchanged ReAct; execution accuracy means SQL result-set equality or passing all hidden code tests.Source: §5.1 · p. 7, §5.2 · p. 7, §5.3 · p. 7

5.4 EFFECTIVENESS ACROSS TASKS

  • TTHE raised accuracy from 12.0% to 50.0% on BIRD, 30.0% to 38.3% on LiveCodeBench, 20.0% to 35.0% on SWE-bench Verified, and 38.0% to 44.0% on DS-1000.The BIRD slice was selected from baseline misses, so its low starting accuracy is by construction; the task slices are not controlled cross-task comparisons.Source: Figure 2 · p. 8, §5.4 · p. 8

5.7 ABLATION STUDY

  • Search and batch-size effects were non-monotonic; BIRD accuracy peaked at 50.0% for B=10, while accumulation reached 50.0% versus 44.0% after resets.More rounds helped only at G=3; accumulation’s edge was concentrated in middle batches, notably B2.Source: Figure 3 · p. 8, §5.7 · p. 9, Table 3 · p. 9
  • TTHE also improved BIRD under MiMo V2.5 (32.0%→52.0%) and Kimi K2.5 (28.0%→48.0%), with each backbone improving over its own baseline.Because the slice is conditioned on DeepSeek failures, absolute scores are not a model ranking.Source: §5.4 · p. 8, Table 2 · p. 9

5.5 BEYOND CODE: AN AGENTIC TOOL-USE DOMAIN

  • On claw-eval, an accumulating harness raised mean task score from 48.9% to 69.8% (+20.9 points), improving on 28 of 30 tasks.This is a graded score, unlike the binary pass@1 measures on the code domains, so its magnitude is not directly comparable.Source: §5.5 · p. 8, Figure 2 · p. 8

5.6 EMERGENT INFERENCE POLICIES

  • Evolved policies included reproduce-first repair on SWE-bench, database-grounded value normalization on Text-to-SQL, and conditional verification on LiveCodeBench.These policies were selected from label-free execution traces.Source: §5.6 · pp. 8–9

5.7 ABLATION STUDY

  • The gain from G=3, R=3 over G=1 cannot be attributed to proposer count alone because the branches also had distinct role prompts.The three roles were conservative-repair, exploratory, and adversarial.Source: Table 1 · p. 8, §5.7 · p. 9

5.8 WHERE THE GAINS ARE LOST: COVERAGE AND SELECTION

  • A judge-restricted oracle reached 64.0% versus the committed harness’s 50.0%, while an oracle over all generated candidates reached only 70.0%.The gaps indicate both selection regret and limited candidate coverage; roughly fifteen tasks were never produced by any candidate.Source: §5.8 · p. 10

6 LIMITATIONS

The evaluation protocol is transductive: each batch’s unlabeled traces select a harness that is evaluated on that batch. The authors identify unresolved questions about forward generalization, judge selection, evaluation breadth, and deployment scope.

6 LIMITATIONS

  • The transductive protocol evaluates a harness on the same batch whose unlabeled traces selected it, so it does not establish forward generalization.Prequential evaluation on the next batch and shuffled-batch reruns are proposed to test forward transfer and distinguish cross-batch accumulation from batch-local adaptation.Source: §6 · p. 10
  • Selection relies on a learned agentic judge, and the authors propose varying proposer and judge access to test whether raw traces enable the discovered policies.The selection-regret analysis identifies the judge as a bottleneck.Source: §6 · p. 10
  • The evidence base lacks multi-seed evaluation and compute-matched baselines, and results come from a restricted, sandboxed environment.The authors identify best-of-N execution selection and development-time optimizers as comparison baselines, and say open-world or safety-critical use would require additional guardrails.Source: §6 · p. 10

7 CONCLUSION

TTHE updates the agent-controlling program using evaluation-time execution, without changing model weights or revealing gold labels. Across the transductive setting studied, it improves baseline harnesses, while imperfect proxy signals can lead the judge to commit regressions.

7 CONCLUSION

  • TTHE improves baseline harnesses across different tasks without changing model weights or revealing gold labels.The evolved programs produce inspectable policies for grounding, verification, and repair.Source: §7 · p. 10
  • Evaluation-time execution updates the program controlling the agent, making the executable harness a practical test-time adaptation state.The conclusion frames this finding within the transductive setting studied.Source: §7 · p. 10
  • Imperfect proxy signals can lead an agentic judge to commit regressions, making proxy reliability the key challenge for robust evolution.Source: §7 · p. 10

A TASK CONSTRUCTION

The evaluation uses selected hard slices to make harness failures observable within a feasible test-time budget. Slice construction draws on benchmark metadata or reference artifacts, which are not passed to TTHE.

A TASK CONSTRUCTION

  • Selected hard slices make harness failures observable under a feasible test-time budget.Source: A TASK CONSTRUCTION · p. 14
  • The BIRD stream spans multiple databases, while LiveCodeBench uses hard problems from a recent contamination-controlled release window.Source: A TASK CONSTRUCTION · p. 14
  • SWE-bench instances are selected by gold-patch complexity, and DS-1000 instances by reference-solution length.Source: A TASK CONSTRUCTION · p. 14
  • Benchmark metadata or reference artifacts construct the fixed evaluation slice but are never passed to TTHE.Source: A TASK CONSTRUCTION · p. 14

B DISCOVERED HARNESS POLICIES

TTHE retains evolved programs as source code and discovers task-specific changes to the agent’s control procedure. The policies span grounding, verification, repair, and tool-use safeguards rather than one-off answer corrections.

B DISCOVERED HARNESS POLICIES

  • The evolved programs are retained as source code, and their policies change control procedures across instances rather than append one-off answer corrections.Source: B DISCOVERED HARNESS POLICIES · p. 14
  • Text-to-SQL policies probe databases for value grounding, repair date formats, check output shape, and remove accidental limits on list queries.Source: B DISCOVERED HARNESS POLICIES · p. 14
  • LiveCodeBench policies rank multiple candidate programs on public tests, repair from expected-versus-observed outputs, and recover from repeated failure with a fresh start.Source: B DISCOVERED HARNESS POLICIES · p. 14
  • SWE-bench policies reproduce issues before debugging, localize root causes before editing, verify edits, and prevent empty patches with an anti-paralysis failsafe.The passage also points to Figure 5.Source: B DISCOVERED HARNESS POLICIES · p. 14
  • DS-1000 policies enforce insertion-only output contracts, deterministic self-checks, and guards against redefining supplied variables.Source: B DISCOVERED HARNESS POLICIES · p. 14
  • Claw-eval policies use single-line tool payloads, per-record identifier citations with completeness checks, exact-identifier service chaining, and prohibit reporting unexecuted actions.Source: B DISCOVERED HARNESS POLICIES · p. 14

C.1 TASK, METRIC, AND PROTOCOL

The agentic-tool evaluation measures harness evolution on local multi-service workflows using a held-out grader. Adaptation proceeds across batches without task scores entering the in-loop process.

C.1 TASK, METRIC, AND PROTOCOL

  • Claw-eval tasks require multi-service office work through mock services, with correctness scored on completion, safety, and communication criteria.The rubric combines these criteria into one score.Source: §C.1 · p. 14
  • The study retains 112 of 139 tasks dependent only on local deterministic services, then selects 30 where frozen-baseline scores are below 0.60.Source: §C.1 · p. 14
  • The solver is deepseek-v4-flash in OpenClaw, and its system prompt and skills form the harness edited as source.Source: §C.1 · p. 14
  • The 30 tasks run in six batches as one general harness evolves from baseline; graders are disabled in-loop and used only on the committed harness.Source: §C.1 · p. 14

C.2 PER-BATCH TRAJECTORY AND DISCOVERED POLICIES

The committed harness improves aggregate task score across the evaluated stream, while trace-derived policies add service-action and reporting rules. The SQL and SWE-bench excerpts illustrate task-specific debugging changes.

C.2 PER-BATCH TRAJECTORY AND DISCOVERED POLICIES

  • 48.9→69.8% aggregate mean task score (+20.9 points), with improvement on 28 of 30 tasks.Baseline-to-committed-harness mean task scores by stream-ordered batch were 29→49, 36→70, 42→62, 46→78, 49→76, and 56→84 (%).Source: §C.2 · p. 14
  • Trace-derived rules add single-line tool payloads, record-identifier completeness checks, exact-identifier service chaining, and grounding reported actions in executed tool calls.Source: §C.2 · pp. 14–15
  • The SQL harness grounds filter values in database casing, infers expected output shape, and retries generation with targeted failure feedback before a final shape fix.Its checks cover structural errors, execution errors, empty results, and wrong column counts.Source: Figure 4 · p. 15, §C.2 · p. 15
  • The SWE-bench policy uses reproduce-first debugging, root-cause localization, post-edit verification, and a repair branch for empty patches.The repair branch reruns the agent with a sharper prompt when the first rollout returns an empty patch.Source: §C.2 · p. 15, Figure 5 · p. 15
  • Because batches are ordered by ascending baseline difficulty, the slice supports per-task before/after comparisons but not isolated cross-batch accumulation claims.Source: §C.2 · p. 15
Loading 2607.08124v1…