HELIX: Model-Harness Co-evolution for Recursive Self-Improvement

1 Introduction

• A harness changes what the model observes, which actions it can take, how it recovers, and when it stops, so agent behavior depends on the executed model–runtime pair.Across improvement rounds, verified trajectories shape the next model, and the updated model conditions the next harness portfolio.Source: §1 · p. 4
• A 65-candidate LCB matrix identified a stronger fixed harness and up to 58.0% more post-hoc portfolio coverage than Pi, with sibling artifacts yielding 438 records.The records include SFT, critic, filter, and preference data; selected candidates received repeated-run and official SWE-bench validation.Source: §1 · pp. 4–5

1.1 Contributions

• HELIX contributes an auditable build–update–rebuild formulation, a bounded source-derived harness-modification substrate, and a verified handoff of sibling rollouts into model-training records.The substrate uses typed ports, atoms, recipes, product shells, and policies, with pre-execution checks and evidence capture.Source: §1.1 · p. 5

2 Model–Harness Co-evolution through Build–Update–Rebuild

• Recursive improvement is verifier-grounded: verified outcomes determine both model-update data and evidence for rebuilding the harness portfolio, while tasks, objectives, and verifiers remain externally specified.The model–harness pair, rather than the model alone, is treated as the state being improved.Source: §2 · p. 5

2.1 Two coupled feedback timescales

• Within tasks, the harness mediates context, actions, execution, feedback, and stopping; across rounds, resulting evidence supports a model update and a rebuilt portfolio.The task-level interaction loop and cross-round improvement loop operate at distinct timescales.Source: §2.1 · p. 5

2.2 Two outputs of a harness-evolution round

• A harness-evolution round evaluates both the best fixed harness for current execution and the candidate portfolio’s learning value from verified sibling contrasts.Matched outcomes can reveal distinctions such as a resolved patch versus no action, or a target-passing patch that causes a regression.Source: §2.2 · p. 6

2.3 From verified experience to the next system state

• Verified round evidence supplies model-update records and also guides which components to retain, prune, or propose when rebuilding for the updated model.Rebuilding matters because harness utility depends on the model; training from only one harness can also over-specialize a model to its tool vocabulary and termination behavior.Source: §2.3 · pp. 6–7

2.4 Fixed-harness performance versus portfolio yield

• Fixed-harness task coverage with K attempts and post-hoc oracle coverage over a candidate set are distinct quantities for reporting execution performance and portfolio breadth.The former is defined for a verifier and fixed harness; the latter is defined for candidates produced in a build round.Source: §2.4 · p. 7

3 HELIX System

• HELIX reports fixed coverage, successful attempt slots, and oracle coverage separately because oracle coverage is only a routing ceiling unless success can be selected before seeing the answer.Equation 10 measures potential positive-outcome yield from an observed candidate portfolio.Source: §3 · p. 8

3.1 The harness as an explicit intervention variable

• Harnesses are explicit interventions composed across eight dimensions spanning the model-facing interface and environment-facing control plane, not single product labels.The dimensions include shell, session and hooks, config, prompt, tools, turn loop, acceptance, and runtime policy.Source: §3.1 · p. 8

3.2 Source-traceable ports, atoms, and recipes

• Ports, atoms, packs, recipes, and product shells provide reusable, source-traceable components whose compiled expansion stays linked to each rollout.Common atoms offer product-neutral behavior, personality atoms preserve source-specific semantics, and the compiler and lockfile retain the expansion used. Full product contracts expose 96 ports, bindings, and swap points; a neutral Minimal contract supports compact runtime and conformance checks.Source: §3.2 · p. 8

3.3 Pre-execution checks and recipe enumeration

• Conformance, boundary, and source-purity checks validate compiled recipes before evaluation; enumeration defines 1,024 or 4,096 candidates, while the live experiment uses a structured 65-candidate slice.Deterministic smoke screening selects candidates for live evaluation, keeping each rollout linked to its intended intervention; the experiment does not evaluate all possible recipes.Source: §3.3 · pp. 8–9

3.4 Model–harness interaction at runtime

• At runtime, the harness mediates each model action by constructing observations and action interfaces, checking and executing actions, and returning environment feedback.The supplied loop begins with session creation or restoration and hooks, then request normalization and prompt/context building, followed by provider streaming and tool-call collection.Source: §3.4 · p. 9

4 Design Benefits

• Runtime execution checks permissions, runs tool batches, evaluates acceptance and finish reasons, continues or stops, and persists traces and workspace effects.The runtime can apply result hooks and truncate or retry where required; stopping may occur at acceptance, a request boundary, or the configured step limit.Source: §4 · p. 10
• Harness policies and model actions jointly determine which transitions, tools, and stopping checks shape an observed trajectory.Context compaction, tool-input normalization, permissions, continuation cadence, and assistant-part protocols can change trajectories without changing model weights.Source: §4 · p. 10

3.5 The evidence plane preserves intervention identity

• Recipe identity persists from compilation through runtime evidence, linking each performance comparison and exported record to its declared intervention.Runs join recipes and lockfiles with model–task–attempt identity, runtime traces, workspace effects, verifier outcomes, and policy evidence; the same identity enables fixed-runtime comparison and matched sibling-data construction.Source: §3.5 · p. 10

4.1 Bounded modification makes build and rebuild practical

• HELIX evolves harnesses through bounded recipe substitutions, reusing stable-port atoms, contracts, and shared compiler, execution, and evidence paths across candidates and model updates.This makes build and rebuild component-level operations rather than requiring each candidate to be implemented as a new agent.Source: §4.1 · pp. 10–11

4.2 Persistent identity enables cross-round attribution

• Persistent recipe, model, task, and attempt identity ties declared composition and execution evidence to outcomes, enabling attribution among model, harness, and interaction effects across rounds.Conformance, boundary, and source-purity checks expose undeclared behavior that could otherwise change the meaning of an intervention.Source: §4.2 · p. 11

4.3 Evidence-linked siblings carry experience into model updating

• Matched sibling groups preserve intervention identity and evaluation evidence, distinguishing resolutions, regressions, near misses, no-action runs, and alternative solutions for model updating.The portfolio acts as a curriculum operator, while external execution evidence grounds which experiences should be learned from, contrasted, or filtered.Source: §4.3 · p. 11

5 Experimental Design

• Bounded component reuse supports repeatable rebuilds, persistent identity makes cross-round effects attributable, and evidence-linked siblings preserve interaction data for model updates.Sibling rollouts retain intervention identity and outcome evidence for SFT, critic, filter, or preference learning; sibling groups stay together in train/dev splits to prevent paired or near-identical trajectories from crossing the split boundary.Source: §5 · p. 12, Table 1 · p. 12

5.1 Evaluation goals and research questions

• The evaluation asks whether evolution finds a stronger fixed harness, what additional coverage and diversity the full portfolio exposes, and what verified sibling outcomes can support model updates.A 65-candidate portfolio is evaluated under a shared model label, with selected members receiving repeated-run, cross-benchmark, and artifact-quality analysis.Source: §5.1 · p. 12

5.2 LiveCodeBench-derived repair subset

• The primary LCB-derived matrix uses 100 repair fixtures (26 easy, 24 medium, 50 hard) and compares Pi Mono with 64 source-pure recipes across 6,500 one-attempt slots [11].Each fixture asks the agent to modify a local solution.py; success requires an observed normalized bash call running test_solution.py with python or python3 and corresponding passing output.Source: §5.2 · p. 13
• A repeated-run follow-up evaluates Pi and two OpenCode-family members for ten attempts per task, producing 3,000 slots.These source-pure interventions were chosen for mechanism analysis, not because they were the top two candidates in the one-attempt matrix; the follow-up measures tasks with at least one success and successful slots.Source: §5.2 · p. 13

5.3 Selected-member SWE-bench Verified follow-up

• The SWE-bench [12] follow-up samples 55 instances and gives Pi, two leading fixed candidates, and two OpenCode-family members two attempts each, producing 550 slots.With a 200-step maximum and local concurrency of one, the official evaluator labels task resolution; the matrix supplies outcomes, patches, and sibling traces for RQ3 data analysis.Source: §5.3 · p. 13

5.4 Shared model and attempt protocol

• All SWE-bench and LiveCodeBench rollouts use MiniMax-M2.7-highspeed with deterministic=false.Source: §5.4 · p. 13

6.1 RQ1—Execution today: can evolution surpass the baseline?

• 52/100 versus 50/100 tasks in the complete matrix, 75/100 versus 72/100 in repeated LCB runs, and 46/55 versus 44/55 on SWE-bench show stronger fixed harnesses, with the preferred member changing by protocol.The best fixed candidate’s matrix gain is a 4.0% relative improvement over Pi; selected-member follow-ups deepen the evidence but do not establish a universal winner.Source: §6.1 · pp. 13–14, Figure 3 · p. 14

6.2 RQ2—Execution breadth from rapid harness evolution

• 79/100 tasks and 1,014 trace-strict successes across 6,500 slots give the 65-candidate portfolio 58.0% more coverage than Pi, but this post-hoc union is not a fixed-harness score.The union covers 29 more tasks than Pi’s row in the one-attempt matrix.Source: §6.2 · p. 14, Figure 3 · p. 14
• Only two of 64 evolved candidates exceed Pi and 23 solve no tasks; the selected-member SWE union covers 49/55 instances versus 46/55 for its best fixed member.The evidence supports complementarity, not online routing; typed gates, screening, and adaptive selection are needed to exploit it.Source: §6.2 · pp. 14–15

6.3 RQ3—Learning data for the next model update

• Of 411 officially resolved slots, only 205 are both production-only and path-clean, showing that official resolution alone does not establish a clean training positive.Across 550 evaluated slots, 543 contain nonempty patches; 411 are resolved, 132 gaps, and seven empty, while 248 nonempty patches contain symlink/path noise.Source: §6.3 · p. 15
• LCB recompositions show repair-closure evidence but mixed efficiency signs, and both recompositions regress on another task despite traces sharing Pi’s read–read–write–bash shape.The changed dimensions move together, so these cases support harness-conditioned trajectories and graded sibling labels, not a single-atom causal explanation or a “more tools” mechanism.Source: §6.3 · p. 17
• Sibling cases recover clean positives, regression negatives, semantic near misses, and patch preferences: Sphinx supplies a verified positive, Pytest exposes regressions, and Xarray contrasts clean but semantically different patches.For Pytest, the Pi patch passes both targets but breaks two regression tests, whereas the OpenCode-config sibling passes both targets and all 123 PASS_TO_PASS tests; for Xarray, the contrast does not prove config caused the outcome.Source: §6.3 · pp. 16–17
• A 200-slot slice yields 438 verified training rows, including 154 preference pairs; the split contains 339 train and 99 dev rows with no instance crossing the boundary.The 154 pairs include minimal-cleanup versus larger-cleanup (46), clean versus noisy resolved (44), minimal-clean versus larger clean (32), resolved versus regression (14), resolved versus target misses (14), and resolved versus no-action (4).Source: §6.3 · p. 17, Table 3 · p. 17

• 200 curated sibling slots yield 438 structured SFT, critic, filter, and preference records from successful, failed, noisy, and alternative trajectories.The audit maps verifier and patch-quality labels to multi-purpose records, not only clean SFT examples.Source: §7 · p. 18, Figure 4 · p. 18
• Harness evolution produces two outputs—expanded execution for the current model and verified sibling data for updating it—that reconnect through build–update–rebuild.Source: §7 · p. 18

7.1 The return from update to rebuild makes improvement recursive

• Three dependencies make improvement recursive: harnesses shape trajectories, verified trajectories shape the next model, and updated capabilities change the conditions for rebuilding.Evidence from a round informs both model supervision and harness retention, pruning, and proposals; the harness preferred for one model may not remain best after an update.Source: §7.1 · pp. 18–19

7.2 The model–harness pair has two improvement frontiers

• 52/100 fixed candidate and 79/100 portfolio union show distinct performance and data frontiers, with deeper validation changing which harness is preferred.The performance frontier favors a reliable, safe, affordable fixed harness; the data frontier favors portfolios that produce verified positives, informative failures, alternatives, and contrasts, so the best deployment harness need not be the best exploration policy.Source: §7.2 · p. 19

7.3 Portfolio coverage exposes execution headroom and data yield

• Portfolio coverage measures verified data yield and bounds routing opportunity, but realizing the oracle-union gap requires a pre-outcome selector, which the experiments do not evaluate.Candidate successes on baseline-missed tasks can supply positive trajectories subject to outcome, regression, and hygiene gates; possible selector signals include task features, early traces, or a cheap critic.Source: §7.3 · p. 19

7.4 Verified failures enrich model-update data

• 23 of 65 candidates score zero under trace-strict status, yet failures can identify incompatible regions to prune or provide negatives for critics and proposal policies.To prevent low-quality runs from teaching interface artifacts or verifier exploits, data builders should retain harness identity, verifier rationale, outcome vector, and failure class for routing, weighting, or exclusion.Source: §7.4 · pp. 19–20

8.1 Recursive self-improvement and model–harness co-evolution

• Prior recursive self-improvement ranges from behavior refinement to model and research-process changes [4], including Gödel Machines [5], Darwin Gödel Machine [6], and harness-specific loops [2] [3].HELIX instead recomposes behavior from heterogeneous open-source harnesses while preserving source identity through runtime evidence and verifier-labeled sibling records.Source: §8.1 · p. 20

8.2 Harnesses as agent–computer interfaces

• ReAct [16], Toolformer [17], and SWE-agent [10] establish that actions, tool use, and agent–computer interfaces shape behavior; HELIX extends intervention to the complete runtime.That runtime includes session semantics, configuration, turn control, acceptance, and policy; HELIX does not claim priority for treating runtime design as part of the agent.Source: §8.2 · p. 20

8.3 Automated harness and workflow evolution

• Unlike workflow- and agent-program-evolution systems [7] [8] [9], HELIX retains nonselected candidates as graded sibling trajectories for regression, near-miss, artifact-quality, and preference examples.AutoFlow optimizes natural-language workflows, ADAS includes Meta Agent Search, and AFlow uses Monte Carlo tree search for code-represented workflow optimization.Source: §8.3 · pp. 20–21

8.4 Interaction data and executable software benchmarks

• LiveCodeBench [11], SWE-bench [12], SWE-Gym [13], SWE-smith [14], and Learn-by-Interact [15] cover executable software evaluation, training environments, synthetic tasks, or interaction-derived training data.HELIX connects these strands through a source-traceable build-to-update handoff.Source: §8.4 · p. 21
• HELIX links source-traceable harness interventions to evidence about current execution and matched sibling records for subsequent model updating.The handoff makes the relationship explicit and auditable at the level of source-derived harness components and matched records.Source: §8.4 · p. 21

9.1 Scope of the evaluation

• Evaluation is limited to coding agents, selected AtCoder examples with public tests, 55 Verified SWE tasks, and automated labels that can miss invalid patches or misclassify correct ones.Some cases still require human review.Source: §9.1 · p. 21, §9.4 · p. 22

9.2 What causes an improvement

• Because each evolved harness changes several components at once, HELIX measures whole-harness effects but cannot identify which component caused an improvement.The paper recommends future one-component-at-a-time experiments with other components held fixed.Source: §9.2 · p. 21

9.3 Effects of attempts and selection

• Oracle coverage is measured after outcomes are known, and differing attempt budgets and member selection make the 65-candidate, LCB, and SWE rankings not directly comparable.The matrix uses one attempt per candidate–task pair, versus ten for selected-member LCB and two for SWE; future comparisons should use common budgets, separate evolution and final-evaluation tasks, and equal total compute.Source: §9.3 · p. 21

9.5 Data scale and model updating

• HELIX materializes model-update data but does not train an updated model; scaling the sibling dataset and developing reliable RSI training methods remain necessary to evaluate the loop.Generating broad portfolios and repeated rollouts also requires substantial token budgets; training must use heterogeneous sibling signals while controlling regressions and preserving general capability.Source: §9.5 · p. 22

9.6 Safety and maintenance

• Unsafe commands, file changes, secret leakage, and unsafe permissions from combining harness components motivate isolated workspaces, limited permissions and network access, and trace redaction.Integrations and safety checks also need retesting when upstream harnesses change.Source: §9.6 · p. 22

10 Conclusion

• The recursively improving system is Z_t = (M_t, Φ_t): the harness portfolio shapes model behavior and experience, which can update the model and alter preferred harnesses.HELIX treats the transition from Z_t to Z_{t+1} as the defining structure of recursive self-improvement for interactive agents.Source: §10 · p. 22, model–harness co-evolution · p. 22

model–harness co-evolution.

• Build–update–rebuild returns the updated model to harness evolution, making harness improvement serve both current execution and subsequent system improvement.Scaling experience, reliable model updates, and comparisons of held versus rebuilt harnesses across rounds remain next steps toward repeated improvement.Source: model–harness co-evolution · p. 22

A Selected-Member Follow-up Results

• The selected-member follow-up tables report deeper evaluations of candidates that were already members of the 65-candidate evolution portfolio.Source: A Selected-Member Follow-up Results · p. 23, model–harness co-evolution · p. 23

A.1 Repeated LCB evaluation

• Table 4 reports repeated-run validation of selected portfolio members using trace-strict LCB outcomes, with tasks and slots counted by successful attempts.Tasks require at least one success in ten attempts; slots count successes among 1,000.Source: Table 4 · p. 23

A.2 Official SWE-bench evaluation

• Table 5 reports official SWE-bench validation with two slots per task; its union measures portfolio coverage, not fixed-harness performance.Resolved slots count successful attempts, while the final column counts tasks with at least one resolved attempt.Source: Table 5 · p. 23

B Verifier-Grounded Case Traces

• The case traces reconstruct two paired cases from persisted rollout transcripts, extracted patches, and official evaluator outputs, retaining decision-relevant steps while omitting repetition.Source: B Verifier-Grounded Case Traces · p. 24

B.1 Pytest: target success with regressions

• Both Pytest patches pass the target tests, but the Pi patch removes the existing longrepr schema while the sibling preserves it.The Pi patch stores only chain and sections; the sibling also retains reprcrash and reprtraceback.Source: §B.1 · p. 24, Table 6 · p. 24
• Official regression failures query a field removed by the Pi patch, making the paired trajectories a resolved-versus-regression supervision pair despite target-test success.The failures are not incidental test noise.Source: §B.1 · p. 24

B.2 Xarray: a clean semantic near miss

• The Xarray gap falls back to attrs[0] when only one attribute exists, whereas the resolved patch falls back to an empty mapping.The official target isolates this fallback semantic despite matched patch hygiene and passing ad hoc checks.Source: §B.2 · p. 25, Table 7 · p. 25
• The Xarray negative is structurally clean and preserves regression tests, yet encodes the wrong target fallback semantics, so patch cleanliness alone is insufficient as a training label.Source: §B.2 · p. 25