PaperlayerView exact revision on arXiv

Source-linked AI summary

Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions

Haochen Luo, Yi Huang, Sichun Luo, Fengyuan Liu, Lei Li, Zefa Hu, Junlan Feng, Qi Liu

arXiv:2607.03935v1cs.AI

TL;DR

Self-evolving systems often optimize task solutions while keeping the surrounding harness fixed. HASE lets one model generate solutions and edit selected harness components in a shared action space. Across the reported tasks, it matches or exceeds larger-model baselines in some settings and repairs evaluators before reaching state-of-the-art circle-packing performance.

  • Problem

    Self-evolving systems often keep the harness fixed, although harness components shape what models observe and how candidate solutions are scored.

    Source: §1 · p. 1

  • Method

    HASE lets a single model generate task solutions and edit selected harness components, with evaluator repairs guided by real-world feedback.

    Source: §1 · p. 1, §5 · p. 8

  • Results

    HASE matches the GPT-OSS-120B system on text classification, outperforms its reported alpha-mining baseline, and reaches state-of-the-art performance after evaluator repair in geometric algorithm discovery.

    Source: §5 · pp. 8–9

  • Takeaways & Limitations

    HASE demonstrates a unified agentic approach to co-evolving the model policy, harness, and task solution across the reported task settings.

    Source: §5 · pp. 8–9

    Editable components are whitelisted, evaluator edits require detected divergence, and phase review can reject useful but more invasive modifications.

    Source: §6 · p. 9

Abstract

from arXiv · show

Self-evolving frameworks usually optimize task solutions while treating the surrounding harness as fixed. We introduce Harness-Aware Self-Evolving (HASE), an agentic reinforcement-learning framework in which a single model can generate task solutions or edit selected harness components in a multi-turn action space. HASE enables a single Qwen3-8B model to match the text-classification performance of a GPT-OSS-120B model that uses Claude Code as the harness proposer. In alpha factor mining, HASE outperforms the reported GPT-OSS-120B baseline. HASE also repairs imperfect evaluation components and converges to state-of-the-art performance in circle-packing algorithm discovery. These results show that HASE improves the harness and the solution through one unified agentic process.

1 Introduction

Self-evolving systems often improve task solutions while holding the harness fixed, even though harness components shape what models observe and how solutions are scored. HASE lets one model co-evolve solutions and selected harness components, and the paper reports strong performance across several tasks.

1 Introduction

  • Fixed harnesses can contain incomplete evaluators or poor search guidance, while the harness shapes what the model observes, which actions it can take, and how solutions are scored.Treating the harness as immutable prevents the agent from improving its computational environment.Source: §1 · p. 1
  • HASE gives a single model one action space for generating task solutions and editing selected harness components.It can improve guidance components freely, while evaluator edits remain anchored by real-world feedback.Source: §1 · p. 1
  • HASE evaluates a Qwen3-8B model on online text classification, alpha factor mining, and geometric algorithm discovery, reporting performance comparable to much larger model-based systems.It also reports evaluator repair and state-of-the-art performance in algorithm discovery.Source: §1 · p. 1

2 Related Work

Prior self-evolving systems commonly modify solutions or search processes while leaving the harness fixed. HASE makes selected harness components editable and combines harness editing with task solving in one action space.

2.1 Self-Evolving Algorithms

  • Many self-evolving algorithms use LLMs to propose or mutate artifacts and rely on external evaluators, while test-time RL adapts the model under a fixed harness.HASE addresses the shared assumption that prompts, context management, tools, and evaluators remain immutable.Source: §2.1 · p. 2

2.2 Harness Engineering

  • Meta-Harness optimizes guidance components with an external coding-agent proposer, while HASE also studies evaluator repair and unifies editing with task solving.On online text classification, the paper reports comparable performance from an 8B model trained with HASE.Source: §2.2 · p. 2

3 HASE Methodology

HASE places task-solution generation and selected harness edits in one multi-turn action space. It evolves guidance components through reviewed candidate edits and permits evaluator repair only when real-world feedback identifies mismatches.

3 HASE Methodology

  • HASE co-evolves model weights, the harness, and task solutions, with phase-level review selecting harness edits.The same Qwen3-8B policy proposes solutions and edits; the controller validates, reviews, and commits its patches.Source: §3 · p. 2, Figure 1 · p. 3

3.1 Two Types of Harness

  • Guidance components shape prompts and context, while local evaluators provide proxy scores and immutable real-world evaluators define task validity.Some tasks use only guidance edits when evaluators are already reliable or too risky to repair automatically.Source: §3.1 · pp. 2–3

3.2 Unified Action Space

  • At each step, the model can submit a solution or use sandbox tools to inspect and modify the editable harness.There is no separate hardcoded planner for choosing between solving and editing.Source: §3.2 · p. 3

3.3 HASE Algorithm: an Agentic Reinforcement-Learning View

  • HASE trains on local-evaluator rewards, while guidance edits earn credit through solutions that improve task scores and evaluator edits through increased mismatch-set agreement.The persistent harness is updated only after phase-level review; evaluator patches also require mismatch evidence and non-regression checks.Source: §3.3 · p. 4

3.4 Guidance Harness Evolution

  • Guidance edits are used within their rollout, shortlisted as candidates, then filtered and reviewed before a stable edit is committed for later phases.A proof-of-concept score can shortlist candidates, but does not alone determine the final commit.Source: §3.4 · p. 4

3.5 Evaluation Harness Evolution

  • 2.635983 matches the AlphaEvolve state of the art after HASE detects and repairs two evaluator divergences in circle packing.Mismatches are added to a set for patch review; agreement on observed cases does not guarantee correctness for future solutions.Source: §3.5 · pp. 4–5, Figure 2 · p. 5

4 Experiments

Across guidance-harness and evaluation-harness experiments, HASE co-evolves harness components and task solutions, matching or exceeding larger-system results in the reported settings. The experiments show evaluator repair followed by policy improvement, as well as gains from evolving search guidance.

4.1.1 Circle packing

  • 2.635983 matches AlphaEvolve's state-of-the-art circle-packing score after oracle disagreements expose omitted validity checks and drive evaluator repairs.The repairs add boundary and overlap checks, then tighten the overlap tolerance to the task-specified 10^-12.Source: §4.1.1 · pp. 5–6

4.1.2 Heilbronn Triangles

  • 0.036082 by Phase 8 is the fully valid co-evolved Heilbronn score, versus 0.021880 for the repaired-evaluator frozen-policy run.The frozen run plateaus after the exploit gap closes; RL updates let the co-evolved policy internalize repaired constraints.Source: §4.1.2 · p. 6, Figure 3 · p. 6

4.2.1 Experiment: Text Classification

  • The discovered harness brings Qwen3-8B to the Meta-Harness operating point on single-attempt test classification, despite weaker standalone few-shot performance.The comparison uses an untouched test set, with the committed harness frozen at test time.Source: §4.2.1 · p. 7, Table 2 · p. 7

4.2.2 Experiment: Alpha Factor Mining

  • 14.35% AER vs. 7.89% and 1.70 IR vs. 0.80; HASE's RankIC 0.0302 closely matches GPT-OSS-120B's 0.0318 under the CogAlpha-aligned protocol.HASE exceeds every other baseline in Table 3.Source: Table 3 · p. 8, §4.2.2 · p. 8
  • The alpha-mining guidance harness first widened factor-pool access, then replaced unbounded context growth with IC-ranked retrieval under a fixed context budget, leaving the financial evaluator unchanged.The editable components were the prompt priors and factor-pool view.Source: §4.2.2 · p. 8, Figure 4 · p. 8

5 Conclusion

HASE co-evolves a model’s policy, task solutions, and selected harness components in one agentic process. Across four task settings, it reports comparable or stronger performance against large-model baselines and state-of-the-art performance in geometric discovery.

5 Conclusion

  • HASE lets one model improve task solutions and selected harness components through a shared multi-turn action space.It supports guidance-harness improvement and evaluator repair when local proxy scores diverge from oracle scores.Source: §5 · p. 8
  • Evaluator edits are triggered by proxy-oracle divergence, letting HASE improve its working environment without unconstrained reward hacking.Guidance components can be improved, while evaluation components are repaired when the local evaluator’s score diverges from the real-world evaluator’s score.Source: §5 · p. 8
  • Across four task settings, HASE matches GPT-OSS-120B in text classification, outperforms its reported alpha-mining baseline, and reaches state-of-the-art performance in geometric discovery.In geometric algorithm discovery, HASE repairs imperfect evaluation components before converging to state-of-the-art performance.Source: §5 · pp. 8–9

6 Limitations

The evaluation covers four task settings but is not an exhaustive scaling study. HASE’s conservative edit controls trade generality for safety and can reject useful, more invasive modifications.

6 Limitations

  • Experiments span four task settings, but do not constitute an exhaustive scaling study; harder follow-up settings remain for each domain.The authors identify this as a boundary on the scope of the reported experiments.Source: §6 · p. 9
  • Whitelisted edits, evaluator-divergence requirements, and phase-boundary review trade generality for safety and can reject useful, more invasive modifications.The authors suggest expanding editable surfaces and strengthening tests while preserving different governance for guidance and evaluator edits.Source: §6 · p. 9

A.1 S2D Qualitative Tool-Evolution Examples

HASE modifies retrieval and memory tools during training to address misleading examples from the initial recency-based retrieval harness. The examples show tool changes followed by corrected disease classifications.

A.1 S2D Qualitative Tool-Evolution Examples

  • The initial recency-based few-shot retrieval often returns misleading cases, motivating HASE to modify retrieval and memory tools during training.The modified sandboxed tool supports stronger retrieval and memory construction.Source: §A.1 · p. 11
  • A customized symptom-overlap scorer replaces irrelevant pneumonia and UTI examples with dengue and typhoid cases, enabling a correct dengue prediction on the second attempt.The initial prediction was pneumonia; the second-attempt reward was R = 0.5.Source: §A.1 · p. 11
  • A stronger lexical-similarity retrieval method exposes diagnostic links to joint pain, guiding a second-attempt correction from fungal infection to psoriasis.Across phases, the harness also evolves toward symptom extraction, label-grouped summaries, and diagnostic-indicator memory.Source: §A.1 · p. 11

A.2 S2D Training-Time Agentic Behaviour

The S2D training-time table describes inspection and harness-editing behavior after Phase 0. The accompanying passages explain candidate shortlisting, test controls, and a repaired Heilbronn solution shown alongside one accepted by a weak evaluator.

A.2 S2D Training-Time Agentic Behaviour

  • After Phase 0, the model consistently performs multi-turn inspection and harness edits; tool edits count overwrite/replace, while read operations count cat/ls/grep.The final test run uses committed files without test-time edits, fine-tuning, retries, or retrieval over the test set.Source: Table 4 · p. 11, §A.4 · p. 12

A.3 PoC-Guided Candidate Shortlisting

  • Positive PoC guidance edits enter a candidate pool, but phase-boundary review commits the interface-valid candidate with the best reviewed validation score, not the highest single-trajectory score.The PoC score records a downstream improvement for the same trajectory; it is not itself a commit rule.Source: §A.3 · p. 11

A.4 S2D Protocol Controls

  • The S2D comparison uses 200 training, 50 validation, and 212 test instances across 22 classes; only training instances are available as the retrieval pool.Phase-boundary review uses validation accuracy, and the final test run has no retries or test-set retrieval.Source: §A.4 · pp. 11–12

A.5 Heilbronn Exploit Visualization

  • Figure 5 contrasts an invalid solution accepted by the weak evaluator with the repaired valid solution.The visualization presents the evaluator-exploit case and the corrected solution side by side.Source: §A.5 · p. 12

B Rigor of the Alpha-Mining Evaluation Pipeline

The appendix documents evaluation-infrastructure controls intended to help interpret the alpha-mining results as alpha rather than pipeline noise.

B Rigor of the Alpha-Mining Evaluation Pipeline

  • The appendix describes controls applied at each evaluation stage to distinguish alpha-mining results from pipeline noise.It documents the infrastructure underlying the alpha-mining results in Section 4 (Experiment 3).Source: B Rigor of the Alpha-Mining Evaluation… · p. 12

B.1 Universe Construction and Data Preprocessing

The pipeline uses a curated CSI300 universe and tests factors for future-information leakage, including dependence on future rows being present in the input.

B.1 Universe Construction and Data Preprocessing

  • CSI300 index codes in common OHLCV dumps can dominate volume-based universe selection and corrupt cross-sectional rank statistics and IC measurements.The index aggregate volume is one to two orders of magnitude larger than any individual constituent.Source: §B.1 · p. 12
  • The evaluation uses 754 curated CSI300 constituents and the full 300-stock universe per window rather than a top-50 volume slice.Typically 150–260 instruments survive per slice after NaN filtering.Source: §B.1 · p. 12

B.2 Future-Information Leakage Detection

  • A causal factor may use observations through day t, but not later rows; future-information leakage can inflate IC and portfolio returns with unavailable signals.Examples include directly using tomorrow’s return or negative-index wraparound near the start of an array.Source: §B.2 · p. 12
  • The detector perturbs rows after each test date and rejects a factor if any finite output in the unchanged past shifts beyond 10−10.It uses seeded per-stock randomwalk OHLCV data and applies asymmetric per-stock multipliers and shifts while maintaining basic OHLC consistency.Source: §B.2 · p. 12
  • A second check truncates the input at several endpoints and compares overlapping prefixes; all factors in the main-text alpha-mining table pass both checks.This tests whether factor output depends on future rows being physically present in the input array.Source: §B.2 · p. 12

B.3 Cross-LLM Independent Leakage Audit

Four LLM judges and manual inspection found no future-information use in the reported factors. The S2D headline is reported across five runs under a frozen-harness inference protocol.

B.3 Cross-LLM Independent Leakage Audit

  • The visual contrasts a phase-0 placement that cheats the weak evaluator with a proxy score of 0.02708 but oracle score of 0.0000 against a valid phase-8 placement.The phase-0 points are out of bounds; the phase-8 points are strictly inside the containing triangle, with the bottleneck minimum area highlighted.Source: Figure 5 · p. 13
  • The audit asked judges to classify whether each per-day, per-stock factor function uses information from t′ > t when computing its value at t.Each judge received the full factor source code and was asked to return LEAK or CLEAN with a brief justification.Source: §B.3 · p. 13
  • 120/120 judge verdicts were CLEAN across four LLMs, and the reported factor implementations were also manually inspected for future-information use.The judges were Gemini, ChatGPT, DeepSeek, and Claude Sonnet.Source: §B.3 · pp. 12–13

C S2D Statistical Reporting

  • The S2D split has 212 instances, and the headline HASE result is reported as mean ± std across five independent evaluation runs.The final policy and committed harness are fixed; test-time editing, fine-tuning, retry, and retrieval over test examples are disallowed.Source: C S2D Statistical Reporting · p. 13

D Compute and Reproducibility

This section documents the compute setup and preserves representative evolved artifacts, including evaluators, alpha-mining pool views and prompts, and classification-tool retrieval implementations ranging from a basic stub to TF-IDF/symptom matching and keyword overlap with label symptom maps.

D Compute and Reproducibility

  • Experiments used Qwen3-8B with vLLM greedy decoding and VeRL GRPO training, while S2D and alpha-mining runs used 4 GPUs across 3 RL phases.Each phase lasted 3 epochs; rollout conversations were capped at 15 user turns and 22,000 tokens to prevent vLLM KV-cache exhaustion during FSDPvLLM sleep/wake cycling.Source: D Compute and Reproducibility · p. 13

E Raw Evolved Harness Artifacts

  • The appendix preserves representative evolved circle-packing evaluators that check square-boundary constraints and reject overlapping circle pairs.The displayed variants use numerical tolerances of 1e-8 and 1e-12 for pairwise-distance checks.Source: E Raw Evolved Harness Artifacts · p. 14

pool_viewer_phase0.py

prompt_phase2.txt

  • Prompts evolved from a generic request for high-quality factors to guides targeting 10-day forward returns, validation across samples and regimes, and hybrid momentum–inverse-volatility factors.One guide reports the best factor from a previous phase had real-world IC 0.0327 and a 10-factor portfolio returned 6.79% annually; another reports 0.0309 and 8.59%, respectively.Source: prompt_phase0.txt · p. 16, prompt_phase2.txt · p. 17, prompt_phase3.txt · p. 17

prompt_phase8.txt

  • Later prompts broadened factor-search guidance to multi-timescale volatility, volume-normalized liquidity, cross-sectional signals, and regime monitoring.Phase 8 proposes combining inverse 20-day average price range with volume relative to its 20-day average, and testing 10-, 20-, and 30-day timeframes.Source: prompt_phase6.txt · pp. 18–19, prompt_phase8.txt · p. 20

G Evolution Details of the Text Classification

  • The initial classification stub stores a knowledge base and retrieves its last n examples.It also declares a present_memory method, but the supplied passage does not show its implementation.Source: G Evolution Details of the Text… · p. 21

classification_tools_phase1.py

  • Phase 1 retrieves top-n examples by TF-IDF cosine similarity and builds label-grouped memory from symptom matches with overlap above 0.5.Symptom extraction uses keyword categories, including fever and muscle pain; the overlap threshold is applied to query and example symptoms.Source: classification_tools_phase1.py · pp. 21–22

classification_tools_phase2.py

  • Phase 2 ranks examples by shared-word count normalized by the longer text, then presents query words shared with symptoms mapped to each label.Retrieval returns the top n ranked examples; memory generation iterates over the supplied label set and reports common terms when present.Source: classification_tools_phase2.py · pp. 22–23

Related papers

Loading 2607.03935v1…