PaperlayerView exact revision on arXiv

Source-linked AI summary

SIA: Self Improving AI with Harness & Weight Updates

Prannay Hebbar, Yogendra Manawat, Samuel Verboomen, Alesia Ivanova, Selvam Palanimalai, Kunal Bhatia, Vignesh Baskaran

arXiv:2605.27276v2cs.AIcs.CL

TL;DR

Existing self-improvement approaches update either an agent’s harness or its model weights, leaving the two levers isolated. SIA jointly updates both and strictly outperforms harness-only iteration across three contrasting domains.

  • Problem

    Harness-update and test-time-training approaches leave the agent’s harness and model weights fixed, respectively, so their combined effect remains untested.

  • Method

    SIA uses a Feedback-Agent to jointly evolve a task-specific agent’s scaffold and RL-adapted weights across downstream tasks.

  • Results

    +20.1 pp on LawBench accompanies SIA-W+H’s strict outperformance of SIA-H on every task.

  • Takeaways & Limitations

    Harness and weight updates occupy distinct change spaces, with each contributing gains unavailable from the other alone.

  • Takeaways & Limitations

    The coupled system can reach verifier-strong fixed points that are fragile under perturbations to either component.

Abstract

from arXiv · show

Humans are the bottleneck in building and improving AI. Both the models and the agents that wrap them are written, tuned, and corrected by people. The long-horizon goal of an AI that can figure out how to improve itself remains open. Two largely disjoint research lines attack this bottleneck. The harness-update school has a meta-agent rewrite the scaffold of a task-specific agent (its tools, prompts, retry logic, and search procedure) while the model weights are held fixed. The test-time training school uses hand-written RL pipelines to update the model's own weights on task feedback while the harness is held fixed. These two silos operate in isolation. We propose SIA, a self-improving loop in which a language-model agent (the Feedback-Agent) updates both the harness and the weights of a task-specific agent. We evaluate across three contrasting domains: Chinese legal charge classification, low-level GPU kernel optimisation, and single-cell RNA denoising. Combining both levers outperforms scaffold iteration alone on all three benchmarks. SIA-W+H achieves 25.1% over prior SOTA on LawBench, 12.4% faster GPU kernels than prior SOTA (1,017 vs 1,161 μs), and 20.4% over prior SOTA on denoising. Harness updates make the model agentic, shaping how it searches and acts, while weight updates build the domain intuition that no prompt or scaffold can instil.

1. Introduction

SIA addresses the human bottleneck in AI improvement by enabling a Feedback-Agent to update both a task-specific agent’s scaffold and model weights from only a task specification and verifier. Across three contrasting domains, the combined SIA-W+H configuration outperforms harness updates alone.

  • Motivation: The paper frames SIA as a concrete step toward AI that can determine how to improve itself, addressing progress otherwise limited by human design and tuning.Humans currently design and post-train models while engineering agents’ scaffolds, prompts, debugging, and tuning.
  • Contribution: SIA enables a Feedback-Agent to improve both a task-specific agent’s scaffold and model weights without further human intervention, given a task specification and verifier.The system produces an evolved scaffold and RL-adapted LoRA weights.
  • Research gap: Automated self-improvement has split into harness updates that hold model weights fixed and test-time training that holds the harness fixed.Harness methods rewrite prompts, tools, retries, and extraction logic, whereas test-time training updates weights through hand-written reinforcement-learning pipelines.
  • Research gap: Harness edits commonly improve software-engineering hygiene but rarely supply domain-specific reasoning beyond what prompting the fixed base model can elicit.The cited examples include parsing, retries, and dispatch improvements.
  • Empirical scope: SIA-W+H strictly outperforms SIA-H across LawBench classification, TriMul CUDA optimization, and single-cell RNA-seq denoising.Figure 1 compares Baseline, SIA-H, and SIA-W+H on accuracy, speedup, and denoising mean-squared-error norm.

2. Research Questions · 3. Background and Preliminaries

The paper asks whether jointly updating an agent’s harness and model weights surpasses harness-only iteration across contrasting domains, and what distinct changes each lever produces. It defines the agent components, meta-agents, and trajectory-based feedback loop underlying this investigation.

  • 2. Research Questions: The central questions test whether combining harness and weight updates exceeds harness-only improvement across contrasting domains, and whether each lever changes the agent in distinct ways.The mechanism question specifically concerns domain knowledge from weight updates versus external infrastructure changes from harness iteration.
  • 3.1. Agent and its components.: A task-specific agent is a program that takes a task instance and produces an answer.Its components include the language model, system prompt, tool-dispatch logic, answer extraction, and grader.
  • 3.1. Agent and its components.: The scaffold, or harness, is every fixed non-weight component, including prompts, tool dispatch, answer extraction, and supporting infrastructure.The underlying model weights are separate from this fixed code.
  • 3.2. Meta-agent vs. task-specific agent.: SIA uses meta-agents whose outputs are agents: M generates the initial scaffold from the task specification and reference implementations, while F improves later scaffolds.The task-specific agent is the scaffold at generation g that executes on the evaluation dataset.
  • 3.2. Meta-agent vs. task-specific agent.: The Feedback-Agent receives the prior scaffold, its execution trajectory, and performance metrics to synthesise an improved scaffold.This establishes the iterative harness-improvement role of F.
  • 3.3. Trajectory and feedback loop.: Unlike aggregate-metric-only improvement, F reads complete structured execution logs containing prompts, responses, tool calls, tool results, and extracted answers.This trajectory-level feedback enables diagnosis of specific failure modes rather than reaction to summary statistics.
  • 3.3. Trajectory and feedback loop.: Each generation runs the current scaffold on the evaluation dataset inside a sandbox and captures its trajectory for the feedback loop.The sandbox permits read-only dataset access and read/write access to a working directory.

1. Execution.

In the execution phase, F receives the agent’s source code, τg, metrics Eg, and optionally sample task descriptions to discourage overfitting to a single instance.

  • 1. Execution.: F receives Ag’s source code, τg, metrics Eg, and optional sample task descriptions that discourage single-instance overfitting.

2. Analysis.

The Feedback-Agent produces both a prose improvement report and a next-generation agent, linking proposed changes to the subsequent agent iteration.

  • F emits an improvement report containing prose analysis and proposed changes.
  • F also emits the next-generation agent A_g+1.
  • Together, these artefacts pair analysis and proposed modifications with the resulting agent iteration.

3. Improvement.

This section formalizes the improvement process using generation-indexed agent scaffolds, evaluation feedback, execution trajectories, and trainable policies.

  • The notation indexes agent scaffolds by generation and records performance metrics, error logs, and execution trajectories for each generation.
  • The task is specified by a benchmark description and sample instances, with performance evaluated on a designated dataset.
  • The improvement formulation distinguishes a trainable current policy from a frozen reference policy and includes multiple rollouts per state during reinforcement-learning training.

4. Related Work

Prior work separates harness mutation from test-time weight adaptation, whereas SIA combines both levers in one self-improving loop. Its related-work gap is explicitly characterized along whether systems edit the harness and/or model weights.

  • Harness-update systems: Darwin Gödel Machine, Meta-Harness (Lee et al., 2026), Hyperagents, AI Scientist (Lu et al., 2024), and AutoResearcher primarily vary scaffold editing or hold the scaffold fixed.Hyperagents makes the meta-mechanism editable but leaves model weights fixed; AI Scientist and AutoResearcher keep the scaffold fixed across runs.
  • SIA’s gap: SIA addresses the separation between the two silos by treating weight updates as a second lever alongside scaffold rewrites under Feedback-Agent control.The supplied implementation passage describes weight updates as one of two selectable actions, while the related-work comparison identifies the corresponding research gap.
  • Test-time training: Test-time training work adapts model weights through rollouts or gradients, including Learning to discover at test time (Yuksekgonul et al., 2026), TTT (Akyürek et al., 2024), and TTRL (Zuo et al., 2025).SIA reuses the entropic-utility objective and LoRA-based training stack from Learning to discover at test time.
  • SIA’s gap: SIA is, to our knowledge, the only entry that updates both the scaffold and the weights in a single self-improving loop.Table 1 compares self-improving and automated agents by whether they edit the harness and model weights.

5. Method

SIA is a configurable self-improvement loop in which a Meta-Agent initializes a task-specific scaffold and a Feedback-Agent interleaves harness and weight updates after executions. Harness updates evolve the scaffold with weights fixed, while training updates modify weights via an RL method selected by the Feedback-Agent.

  • System architecture: SIA combines three LLM components—Meta-Agent, Task-Specific Agent, and Feedback-Agent—in a loop that updates either the harness or model weights.The Meta-Agent initializes the task-specific scaffold; after each execution, the Feedback-Agent observes trajectory and performance and selects the next action.
  • Two levers, one loop: The Feedback-Agent freely interleaves harness and weight updates rather than enforcing sequential phases.Figure 2 depicts alternating decisions across a seven-step example, with metric gains from both update types.
  • Update mechanisms: A harness update evolves the scaffold while holding model weights fixed, whereas a training update applies an RL-based weight update chosen by the Feedback-Agent.Harness changes affect the scaffold; the task-specific agent uses the base model for harness steps or an RL-adapted checkpoint for training steps.
  • Harness updates: Each harness-update step follows Execution → Analysis → Improvement, changing only the scaffold A_g while rollouts come from the current model π_θ.The Meta-Agent is conditioned on diverse task specifications during scaffold generation to mitigate overfitting to a single benchmark instance.

6. Experiments

SIA is evaluated on three contrasting law, systems, and biology benchmarks chosen to enable direct comparison with prior self-improving AI systems. Across tasks, the Feedback-Agent switches from scaffold iteration to weight updates when harness progress stalls, isolating harness-only and combined performance.

  • Benchmark scope: Three contrasting benchmarks span Chinese legal charge classification, low-level GPU kernel optimisation, and single-cell RNA denoising, supporting direct comparison with prior work.The benchmarks are commonly used to evaluate other self-improving AI systems.
  • Experimental protocol: Across all tasks, SIA reports SIA-H for harness-only updates and SIA-W+H after adding weight updates, with weight updates beginning when scaffold progress stalls.The initial scaffold is generated around gpt-oss-120b, and both variants use the same verifier.
  • LawBench: 50.0% accuracy was reached on LawBench after scaffold updates reorganized the solution around TF-IDF + LinearSVC, a 36.5-point gain over the initial run.The Feedback-Agent then switched to weight updates after accuracy gains levelled off.
  • GPU kernel optimisation: 1.14× speedup was achieved for TriMul after harness iterations refined CUDA kernels using memory-layout hints, compilation flags, and retry logic before the trajectory plateaued.TriMul is memory-bandwidth-limited and requires H100-specific optimization such as tensor-core scheduling, shared-memory tiling, and register-pressure management.
  • Single-cell RNA denoising: 20% gain lifted denoising mse norm to 0.289 after GRPO introduced np.clip + np.rint rounding, a structural transformation absent from every scaffold-only iteration.The transformation enforces non-negative integer imputed counts, a biological invariant.

7. Discussion

SIA-W+H strictly outperforms harness-only SIA-H across LawBench, TriMul, and MAGIC denoising, showing that harness and weight updates provide complementary gains. Harness updates improve how the agent searches and acts, while weight updates internalise task-specific knowledge through direct gradient pressure.

  • Ablation: +20.1 pp on LawBench, 91.9% runtime reduction on TriMul (12,483 →1,017 µs), and 20% on denoising distinguish SIA-W+H from SIA-H across every task.The two levers occupy distinct change spaces—external scaffold versus internal parameters—so neither saturates the gains available from the other.
  • Harness updates: Harness iteration adds tools, tighter parsers, search procedures, retry policies, and prompt structure while keeping the model checkpoint fixed.Observed scaffolds included structured answer extraction and reranking for LawBench, CUDA-diagnostic feedback and timing for TriMul, and batched configuration and score parsing for denoising.
  • Adaptive training: The Feedback-Agent selects training approaches from trajectory observations rather than following a fixed RL procedure.The reported tasks used PPO with GAE on LawBench, entropic advantage weighting on TriMul, and GRPO on denoising; broader algorithm selection is deferred to v2.
  • Weight updates: Weight updates internalise task-specific, verifier-aligned knowledge by changing the model’s prior over solutions through direct gradient pressure.They sharpened LawBench charge disambiguation, produced H100-specific TriMul kernel patterns, and added domain knowledge beyond scaffold edits.

8. Limitations

SIA’s coupled harness search and RL weight updates optimize against the same fixed verifier, creating a co-evolutionary Goodhart failure. Their joint fixed point is a Nash equilibrium between mutually history-blind optimizers rather than a verifier-maximizing solution.

  • 8. Limitations: SIA’s harness search and RL weight updates both optimize against the same fixed verifier V, creating a coupled co-evolutionary Goodhart problem.Each update process changes the distribution encountered by the other.
  • 8. Limitations: The harness finds scaffolds exploitable by the current policy, while weights train on data collected through a scaffold that later changes.This mutual distribution shift means neither optimizer accounts for the other’s update history.
  • 8. Limitations: The coupled system converges to a Nash equilibrium between two mutually history-blind optimizers, not necessarily a point that maximizes V.Thus, optimizing the shared verifier does not guarantee reaching its global maximum.

9. Future Work

Future work will make SIA’s self-improvement policy learnable across tasks and replace coarse alternation with finer-grained switching between harness and weight updates.

  • Meta-RL over the action-selection policy: The Feedback-Agent could learn lever selection with meta-RL across tasks, treating action-outcome trajectories as transitions in an outer MDP.This would improve attribution of when to use harness versus weight updates through experience rather than a frozen LLM prior.
  • More interleaved training and harness switching: Finer-grained interleaving could trigger weight updates during harness search and resume exploration after gradient steps, reducing lag after plateaus.The current loop alternates between harness search and weight updates in discrete, coarse-grained rounds.
Loading 2605.27276v2…