PaperlayerView exact revision on arXiv

Source-linked AI summary

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

Michael Nguyen, Wei Chen Tan, Nurul Aisyah Hassan, Arvind Raman, Li Hua Lim, Ahmad Faiz Razak

arXiv:2609.02889v1cs.CL

TL;DR

Existing harness-evolution methods usually optimize agent scaffolding as one flat string, leaving unclear which components carry optimization value. HARNESSEVO decomposes the harness into four slots and applies reflective evolution with LOI/LOO attribution under an iso-budget. The results localize ALFWorld’s useful value in reflection/control, show that uniform budget splitting can suppress it, and establish a task-contingent null on WebShop.

  • Problem

    Existing reflective prompt-evolution methods usually optimize the harness as one flat string, so they do not reveal which component carries its optimization value.

  • Method

    HARNESSEVO decomposes the harness into four separately evolvable slots and applies coordinate-ascent reflective optimization with LOI/LOO credit assignment under a matched total rollout budget.

  • Results

    On ALFWorld, overall success ties the stock and flat-string baselines at 0.657 versus 0.642 and 0.642, while reflection/control alone yields a +0.119 LOI gain; WebShop remains null.

  • Takeaways & Limitations

    Harness-optimization value can be localized to one component, so credit assignment should precede structured evolution and budget should not be split uniformly below the optimizer’s working point.

  • Takeaways & Limitations

    The study uses one frozen Qwen2.5-7B backbone, two benchmarks, and one coordinate-ascent reflective optimizer, limiting claims about scaling, task coverage, and alternative schemes.

Abstract

from arXiv · show

A growing body of work improves frozen large language models (LLMs) as agents by evolving their harness: the textual scaffolding around the model, including persona, strategy, format rules, and control heuristics. Existing reflective prompt-evolution methods usually optimize this harness as one flat string. We instead ask where the optimization value actually resides. We introduce HARNESSEVO, which decomposes the harness into four separately evolvable slots: role, task-strategy, tool/format-rules, and reflection/control. Using the same reflective optimizer under an iso-budget setting, we pair this decomposition with leave-one-in and leave-one-out attribution to measure the contribution of each slot. On ALFWorld with a frozen 7B backbone, HARNESSEVO does not significantly improve the overall binary success rate over either the stock harness or flat-string evolution: 0.657 versus 0.642 and 0.642, respectively. However, the slot-level analysis reveals that nearly all useful optimization value is localized in the reflection/control slot, which achieves a leave-one-in gain of +0.119. The other slots are individually null. We further show that uniform budget splitting is harmful: allocating 64 rollouts across four slots leaves only 16 per slot, below the optimizer's effective search floor, causing every slot to freeze at its empty seed. Concentrating the budget on the high-credit control slot recovers the lost gain, reaching 0.761 with half the split budget. The effect is task-contingent. On WebShop, all slots freeze empty and all methods tie, indicating a genuine absence of recurrent, verbalizable control failures rather than budget starvation. Overall, our results suggest that harness value is localized, uniform budget splitting can be actively harmful, and credit assignment should precede structured agent-evolution.

1 Introduction

The paper decomposes harness optimization to locate where value resides, finding that ALFWorld gains concentrate in reflection/control while uniform budget splitting suppresses them. The effect is task-contingent: WebShop shows no recoverable slot value.

  • Motivation: Flat-string evolution cannot identify which harness component contributes to improvement, motivating a four-slot decomposition and credit-assignment protocol.The slots are role/persona, task-strategy, tool/format-rules, and reflection/control heuristics.
  • ALFWorld results: On ALFWorld, the full structured method reaches 0.657 held-out success versus 0.642 for both the stock harness and flat-string evolution, with neither difference significant.The reported McNemar p-values are 0.617 versus stock and 0.480 versus flat evolution.
  • ALFWorld results: +0.119 LOI gain comes from evolving reflection/control c4 alone, while role, strategy, and format are individually null.The c4 comparison is significant at p = 0.0046, whereas the other slot gains are +0.030, +0.007, and +0.007, all nonsignificant.
  • Budget-splitting trap: +0.015 is the full structured gain, far below the +0.164 sum of four single-slot gains because 64 rollouts split across four slots gives 16 per slot, below the optimizer’s accept-and-rescore floor.Every slot therefore froze at its empty seed, producing strong sub-additivity.
  • Budget allocation: 0.761 is reached by a control-only run using 32 rollouts, recovering a +0.119 gain that uniform splitting discarded.The 64-rollout all-to-control replication reaches 0.724 with a +0.082, borderline gain, so the paper makes a conservative claim about magnitude.
  • Task contingency: On WebShop, all slots freeze empty and full, baseline, and targeted control runs remain null, indicating task-contingent absence of recurrent verbalizable control failures rather than budget starvation.The reported dense scores are 0.518, 0.539, and 0.545, all nonsignificant.

2 Related Work

Prior work evolves prompts, contexts, workflows, and agent reasoning, but generally treats optimized text as a whole. This paper positions slot-level credit assignment as the missing analysis of where harness value resides.

  • Prompt and agent self-evolution: Automatic prompt methods such as APE, OPRO, Promptbreeder, DSP, and DSPy optimize instruction text or program prompts, typically without attributing value to individual spans.The related work distinguishes prompt optimization from the paper’s slot-level question.
  • Reflective prompt evolution: GEPA is the principal foil: it reflects on rollout traces, mutates one prompt string, gates candidates on minibatches, and maintains an instance Pareto frontier.HARNESSEVO reuses this reflective optimizer slot-by-slot rather than evolving one undifferentiated string.
  • Context and experience evolution: Context-evolution methods induce workflows, curate evolving context, manage memory, or build skill libraries rather than measuring marginal value under a shared harness budget.These methods form a complementary self-improvement strand.
  • Agent scaffolding: ReAct, Reflexion, Tree-of-Thoughts, self-consistency, and self-refinement structure reasoning-and-acting behavior, providing conceptual background for the harness components studied here.The paper’s agent runs a ReAct-style reasoning-and-action loop.
  • Interactive benchmarks: ALFWorld and WebShop ground evaluation in interactive household manipulation and instruction-conditioned web shopping with dense attribute-match rewards.The benchmarks are selected to examine environment-dependent harness value.
  • Credit assignment: Credit assignment is the paper’s central analytical instrument, extending ablation-style component analysis to prompt-harness slots under a shared budget.The paper contrasts this with decomposition methods that restructure computation without measuring per-part marginal value.
  • Reward and alignment: Goodhart and reward-over-optimization work motivates scrutiny of the signal being optimized, while this paper studies what a black-box harness optimizer extracts from that signal.The authors distinguish optimization-side analysis from reward-side alignment research.
  • Recursion: Recursion is orthogonal to this study because the optimizer is held fixed and the empirical claims concern a single-pass coordinate-ascent procedure.The paper does not iterate the meta-improvement process.

3 Problem Formulation

The problem formulation treats harness optimization as searching text for a frozen-model agent under a rollout budget, then attributes gains to four semantically disjoint slots. LOI and LOO quantify marginal contribution and loss when a slot is removed.

  • Frozen-model agent: A frozen-model agent uses language model M without gradient updates, acts in environment E over discrete steps, and receives environment-verified episode outcomes.Behavior is conditioned on harness text injected at a fixed prompt-construction point.
  • Harness-optimization problem: H⋆ maximizes expected held-out success ρ(H) subject to a rollout budget B, with selection on validation data and reporting on a disjoint held-out set.The objective is defined for deterministic temperature=0 rollouts.
  • Baseline: The stock harness H0 is the empty string, making structured and flat-string methods start from the same neutral baseline and spend the same budget.Every method must earn harness content through search.
  • Component decomposition: H = (c1, c2, c3, c4) renders four mutually exclusive slots in fixed order, concatenating non-empty slots and dropping empty ones.The neutral render is ε.
  • Component decomposition: c1 role/persona specifies agent disposition, c2 task-strategy specifies high-level task decomposition, and c3 tool/format-rules specifies action templates and syntax.Each slot excludes the semantic scope of the others.
  • Component decomposition: c4 reflection/control-heuristics specifies loop discipline and self-correction, including avoiding failed repetitions, verifying completion, backtracking, and stopping.It contains no persona, strategy, or low-level syntax.
  • Credit assignment: LOI and LOO attribute slot value by evolving only slot ck or all slots except ck, respectively, then evaluating held-out performance.A localized slot has large positive LOI with other slots’ LOI gains near zero; LOO measures loss when that slot is removed from the full harness.
  • Credit assignment: Sub-additivity means the jointly optimized harness gains far less than the sum of individually optimized slot gains.The notation and comparison are summarized in Table 1.

4 Method: HARNESSEVO

HARNESSEVO decomposes the harness into four slots and evolves them by coordinate ascent under an iso-budget, then uses LOI and LOO probes for credit assignment.

  • Method: HARNESSEVO combines a four-slot harness, coordinate-ascent evolution, and LOI/LOO credit assignment.The slots are evolved separately with an existing reflective optimizer, while the attribution probes assess slot sufficiency and necessity.
  • Architecture: The harness comprises role, strategy, format, and control slots rendered into one string at the fixed ReAct injection point.The frozen model and rollout path remain shared across methods, isolating changes to injected harness text and structure.
  • Coordinate-ascent evolution: Coordinate ascent evolves one slot while freezing earlier slots and holding later slots at empty seeds.Each slot uses a slot-scoped reflection prompt, and the validation-selected candidate is frozen before the next coordinate step.
  • Budget allocation: The per-slot budgets sum to the flat method’s total budget, enforcing an iso-budget comparison.The algorithm evenly splits the total budget across slots and caps each optimizer run at its assigned budget.
  • Credit assignment: LOI evolves one slot alone, whereas LOO evolves all slots except one to estimate sufficiency and necessity.Each resulting harness is evaluated on the held-out test set under the shared rollout, with paired statistical comparisons.

5 Experimental Setup

The experiments use a frozen 7B model, held-out ALFWorld and WebShop evaluations, iso-budget comparisons, targeted budget arms, and paired uncertainty testing.

  • Backbone: All experiments use a single frozen Qwen2.5-7B model without weight updates, with deterministic rollout decoding.The same model serves as task policy and reflection model; mutation sampling uses higher temperature as in GEPA.
  • Benchmarks and splits: ALFWorld uses 30 development tasks for training and validation and 134 disjoint held-out test tasks with binary success.Episodes run for up to 40 steps under environment-verified evaluation.
  • Benchmarks and splits: WebShop uses 80 held-out sessions and a dense attribute-match score.The benchmark evaluates purchasing-task performance with scores in the [0, 1] range.
  • Budgets: B = 64 rollouts is the primary iso-budget: flat evolution uses one string, while HARNESSEVO allocates 16 rollouts per slot.Additional arms use 32 rollouts for single-slot LOI and 64 for all-to-control; B120 provides corroboration.
  • Methods: The comparison includes stock, flat-string evolution, full HARNESSEVO, LOI, LOO, and all-to-control budget arms.All methods share the identical rollout path.
  • Metrics and statistics: Held-out rates receive 10,000-resample bootstrap 95% confidence intervals and paired significance tests.McNemar’s test is used for binary ALFWorld outcomes, while WebShop uses a paired bootstrap on dense scores.

6 Results

At B=64, HARNESSEVO ties the stock and flat-string baselines overall, while attribution localizes the useful ALFWorld optimization value to c4 reflection/control. Concentrating budget on c4 recovers a substantial gain that uniform splitting discards.

  • Main comparison: 0.657 held-out success for HARNESSEVO-full versus 0.642 for both stock and flat-string evolution; all three methods tie statistically.Neither HARNESSEVO comparison is significant, and the baselines are co-best.
  • Per-component attribution: +0.119 LOI gain identifies c4 reflection/control as the only load-bearing slot on ALFWorld.The gain is significant (p = 0.0046), with 22 of 28 discordant tasks favoring the evolved control.
  • Per-component attribution: The other three slots are individually null, while removing any slot from the near-empty full harness produces no significant change.The largest reported leave-one-out change is 0.067 in magnitude.
  • Interaction and sub-additivity: +0.015 full-harness gain versus +0.164 summed LOI gains demonstrates strong sub-additivity.The structured method captures roughly one-tenth of the value offered by its components individually.
  • Budget corroboration: At B120, HARNESSEVO-full reaches 0.627 versus 0.672 for stock and 0.642 for flat evolution, with all pairwise differences nonsignificant.The control slot remains the most positive single-slot direction at +0.052, but this is treated as a noisy robustness check.
  • Budget allocation: 0.761 held-out success results when c4 receives 32 rollouts, recovering a +0.119 gain with half the four-way split’s total budget.At 64 rollouts, all-to-control reaches 0.724, a positive but borderline result (p = 0.0725).

7 Analysis: Credit Assignment and the Budget-Splitting Trap

The LOI analysis localizes ALFWorld’s harness value almost entirely in reflection/control, while uniform budget splitting suppresses that gain by freezing every slot at the empty seed. Concentrating budget on c4 recovers substantial gains, whereas WebShop remains null even with concentrated control budget.

  • 7.1 The value is localized to control: +0.119 LOI gain comes from reflection/control c4; role, strategy, and format are individually null on ALFWorld.The other three slots sum to +0.044 and are individually nonsignificant.
  • 7.2 Sub-additivity and the accept-and-rescore floor: 16 rollouts per slot leaves all four slots frozen at the empty seed because the accept-and-rescore floor lies between 16 and 32 rollouts.At B=64, an accept-bearing iteration costs approximately 16 rollouts.
  • 7.2 Sub-additivity and the accept-and-rescore floor: 0.761 held-out success is reached by control-only LOI at 32 rollouts, exceeding the four-way split’s 0.657 with half its total budget.The 64-rollout all-to-control run reaches 0.724 and is borderline significant.
  • 7.3 Task contingency: Δ=+0.444 on look_at_obj_in_light and Δ=+0.226 on clean_then_place concentrate gains where recurrent loop and multi-step control failures occur.The task-type pattern tracks the content of the learned control rule.
  • 7.3 Task contingency: 0.535 versus 0.518 on WebShop remains null under all-to-control, showing that its null is not caused by budget starvation.The control slot froze empty because WebShop lacks a recurrent, verbalizable control failure for a general heuristic to fix.

8 Discussion

The discussion argues for localizing harness value before optimizing it: structured decomposition is useful as a diagnostic microscope, but uniform splitting can push every component below the optimizer’s working point. The observed benefit is task-contingent and corresponds to environment-grounded self-correction rather than reward exploitation.

  • 8 Discussion: A cheap single-slot LOI probe can identify the high-credit component before concentrating a fixed optimization budget there.The paper presents decomposition as a localization instrument rather than a drop-in method for improving agents.
  • 8 Discussion: Uniform budget splitting can make structured evolution worse than targeted single-slot evolution by dividing each component below the accept-and-rescore floor.The trap follows from the optimizer’s minimum budget requirement, not from harness decomposition itself.
  • 8 Discussion: Harness optimization helps when failures recur across instances and are describable as general control rules; otherwise, the paper expects a null.ALFWorld and WebShop bracket this task-contingent pattern.
  • 8 Discussion: The localized value is grounded self-correction that fixes real control errors, generalizes across task types, and does not exploit the reward.The benchmark with the more gameable reward produced a null rather than an apparent shortcut.

9 Conclusion

HARNESSEVO uses four-slot evolution and leave-one-in/leave-one-out attribution to locate harness value, but its full structured method ties stock and flat evolution because budget splitting freezes every slot. Concentrated control optimization recovers the discarded ALFWorld gain, while the conclusions remain bounded by the tested backbone, benchmarks, and optimizer.

  • 9 Conclusion: HARNESSEVO combines four-slot coordinate ascent under an iso-budget with leave-one-in and leave-one-out credit assignment.The instrument is designed to identify where harness optimization value resides.
  • 9 Conclusion: 0.657 / 0.642 / 0.642 is the held-out success comparison for structured, stock, and flat-string evolution, respectively, with no significant overall improvement.The localized ALFWorld c4 gain is +0.119, while the other slots are individually null.
  • 9 Conclusion: 0.761 at half the split budget recovers the discarded gain through targeted control-only evolution.The full-budget control-only result is 0.724 and borderline significant.
  • 9 Conclusion: The study uses one frozen Qwen2.5-7B backbone and two benchmarks, so localization may differ at larger scale or across a broader task space.The authors explicitly flag backbone scaling and broader benchmark coverage as open questions.
  • 9 Conclusion: The conclusions depend on one coordinate-ascent structured scheme and one reflective optimizer, whose accept-floor drives the observed sub-additivity.Alternative slot taxonomies, joint evolution, or optimizers might distribute value differently.

A.1 Audited frozen slot texts

The audited slot texts show that the successful ALFWorld c4 candidate encodes compact environment-grounded control corrections, while WebShop’s slots remain empty. This contrast supports localization of useful value in actionable self-correction rules.

  • A.1 Audited frozen slot texts: The successful ALFWorld c4 text instructs the agent to verify outcomes, skip repeated dead ends, open receptacles, confirm holding, and stop when appropriate.The candidate is a compact rule set targeting recurrent control failures.
  • A.1 Audited frozen slot texts: The full-budget ALFWorld c4 run froze a longer pseudo-code variant of the same control rules.Its elaboration preserves the same sequence of checks and corrections.
  • A.1 Audited frozen slot texts: WebShop’s control and other slots all froze empty under both four-way splitting and all-to-control.The empty-versus-substantive contrast accompanies WebShop’s null result.
  • A.1 Audited frozen slot texts: The B120 corroboration arm keeps the control direction as the largest single-slot gain, while all differences remain nonsignificant.The corroboration also preserves the system’s sub-additive pattern.

A.2 B120 corroboration arm (full table)

The B120 arm serves as corroboration rather than an independent significant result. At this higher budget, the control slot remains the most positive single-slot direction, while the full system is sub-additive.

  • B120 corroboration: 0.672 is stock success, compared with 0.642 for flat evolution and 0.627 for HARNESSEVO-full; all pairwise tests are nonsignificant.These values come from the higher-budget B120 configuration.
  • B120 corroboration: +0.052 is the control slot’s B120 leave-one-in gain, but it is nonsignificant.The control direction remains the most positive single-slot direction in this robustness arm.
  • B120 corroboration: The B120 arm is included to check the ordering of per-slot value, not to claim an independent significant result.Its role is corroborative rather than a separate confirmation of improvement.
  • Experimental setup: The B120 evaluation uses a frozen Qwen2.5-7B backbone with deterministic task rollouts and sampled reflection mutations.The same frozen backbone serves as both task policy and reflection model.
  • Experimental setup: ALFWorld evaluation uses binary environment-verified success on 134 disjoint held-out test tasks, while WebShop uses a dense attribute-match score on 80 test sessions.The two benchmarks use different outcome metrics and held-out test sets.
  • Experimental setup: The optimizer uses iso-budget reflective mutation, validation selection, and final freezing of the best-validation candidate; budgets tested were B=64 and B=120.Single-slot leave-one-in runs use half the budget, with all-to-control runs at 32 and 64 rollouts.
Loading 2609.02889v1…