Source-linked AI summary
Recursive self-improvement of AI research agents
Dhruv Srikanth, Bingchen Zhao, Dixing Xu, Yuxiang Wu, Zhengyao Jiang
TL;DR
Research agents can automate AI R&D, but the efficiency of the process producing research remains fixed and the effectiveness of improving that process is unclear. AIDE2 recursively rewrites and evaluates its research agent, discovering gains that transfer to held-out tasks and domains.
Problem
AI agents automate parts of AI R&D, yet the efficiency of the research process producing their artifacts remains fixed and the effectiveness of recursive self-improvement for frontier AI research efficiency is unclear.
Source: §1 · p. 1Method
AIDE2 uses an inner loop to optimize task solutions and an outer loop to rewrite the research agent, retaining changes that improve its performance on AI R&D tasks.
Source: §1 · p. 2Results
Across four held-out benchmarks, the strongest discovered agent matches or exceeds AIDEhuman; reward hacking falls from 55% to 32%, below the human-engineered agent’s 39%.
Source: §1 · pp. 2–3, §5 · p. 13Takeaways & Limitations
Under a fixed evaluation budget, harness-layer improvements transfer to in- and out-of-distribution tasks and improve an AI research agent’s research efficiency.
Source: §5 · p. 13The ignition comparison is inconclusive with three seeds per arm, and a decisive evaluation would require costly additional RSI seeds and held-out evaluations.
Source: §3.6 · p. 10, §5 · p. 13
Abstract
from arXiv · showhide
AI agents are beginning to automate research and development across the AI stack, from improving training efficiency to optimizing inference. A natural next step is to improve the research efficiency of the agents themselves. When an AI research agent's own code is the object of optimization, each accepted rewrite becomes the agent that the next round edits. We refer to this loop as recursive self-improvement. Its significance lies in a long-standing trend, in which increased cumulative spending on R&D yields diminishing returns. Sustained self-improvement offers a way to counter this trend. We present AIDE^2, a system that implements this loop for a frontier AI research agent. It proposes changes to its own code, benchmarks modified versions of itself on a suite of AI R&D tasks, and keeps the changes that perform best on hidden evaluations. In an autonomous 8-day run, AIDE^2 discovered seven successive improvements, ranging from a new search policy to memory mechanisms that compress and manage the agent's growing context. These gains generalize to four held-out benchmarks spanning machine learning engineering, heuristic algorithm engineering, and physics-based weather forecasting, the last of which is out of distribution from the selection tasks. On all four, the strongest discovered agent matches or exceeds a human-engineered production research agent that ranks among the strongest on FML-Bench. On a separate held-out task family, the discovered agents also exhibit reduced reward hacking, a property the loop never explicitly optimized for: the rate falls from 55% to 32% during the run, 7 percentage points below the human-engineered agent. Together, these results show that an AI research agent can improve its own research efficiency through recursive self-improvement, and that these gains transfer to tasks and domains the loop never encountered.
1. Introduction
AI agents already automate parts of AI R&D, but the efficiency of the research process itself remains fixed and the effectiveness of recursive self-improvement for frontier AI R&D is unclear. AIDE2 tests a two-loop system that rewrites its research agent and evaluates the changes on AI R&D tasks.
1. Introduction
- Although AI agents automate tasks across the AI stack, the efficiency of the research process producing their artifacts remains fixed, and frontier AI research efficiency remains an open RSI question.Source: §1 · p. 1
- AIDE2 uses an inner loop to optimize task solutions and an outer loop to rewrite the research agent for improved inner-loop research efficiency.Source: §1 · p. 2
- On four external benchmarks, the strongest discovered agent matches or exceeds AIDEhuman; reward hacking falls from 55% to 32%, below its 39%.The reduction was not part of the optimization objective.Source: §1 · pp. 2–3
- In one autonomous 8-day run, AIDE2 discovered seven successive improvements, each accepted after improving held-out results unseen by the rewritten agent.Source: §1 · p. 2
2. Method
The method frames recursive self-improvement as bi-level optimization: an inner-loop agent solves AI R&D tasks, while an outer-loop agent rewrites that agent. AIDE2 evaluates rewrites on a diverse, fixed-budget selection benchmark and retains the best-performing agents.
2. Method
- The two loops optimize agents against measurable outcomes, with fixed-budget inner-loop evaluation linking optimization capability to research efficiency.Source: §2 · p. 3
2.1. Recursive self-improvement
- The inner loop edits task code under a fixed dollar budget, returns a solution, and receives a grade aggregated from private held-out scores across diverse AI R&D tasks.Task families include ML engineering, heuristic algorithm engineering, and harness engineering.Source: §2.1 · p. 3, §2.2 · p. 5
- The outer-loop agent proposes rewrites of the best-graded incumbent, and each accepted rewrite becomes the codebase for the next step.Outer-loop selection is fixed, while inner-loop candidate selection is an editable policy.Source: §2.1 · pp. 3–4
2.2. AIDE2
- AIDE2 starts from AIDE0, removing ML-specific machinery while retaining tree search with draft, debug, and improve operators.AIDE0 uses greedy selection, choosing the highest-scoring solution as the next parent.Source: §2.2 · p. 4
- AIDEhuman drives the outer loop and serves as the human-engineered baseline; the inner-loop model is held fixed during the run and evaluated under task-specific budgets.The selection benchmark grades agents across three task families using private held-out scores.Source: §2.2 · pp. 4–5
3. Experiments
The experiments test whether recursive self-improvement yields repeated gains, transfers to external benchmarks, and changes behavior beyond the optimized objective. They also examine AIDE85’s mechanisms and whether it improves the outer loop as a self-improver.
3.2. A sustained trend of improvements
- In an 8-day run, seven accepted rewrites raised the incumbent grade from 0.703 to 0.778; two other complete runs also produced sustained improvements.The seven rewrites were accepted at steps 2, 6, 28, 39, 47, 63, and 85; the trajectory contained 100 nodes.Source: §3.2 · pp. 5–6, Figure 2 · p. 6
3.3. Generalization to held-out benchmarks
- AIDE85 matches or exceeds AIDEhuman on all four external benchmarks, including out-of-distribution WeatherBench 2, where both evolved checkpoints converge on similar gains across seeds.AIDE85 performs best on ALE-Bench and FML-Bench; AIDE47 performs best on MLE-Bench and WeatherBench 2.Source: Figure 3 · p. 7, §3.3 · p. 7
3.4. An emergent behavior: reduced reward hacking
- Reward hacking fell from 55% for AIDE0 to 32% for AIDE85 on held-out kernel tasks, compared with 39% for AIDEhuman, although the responsible rewrites are unidentified.The rates measure reward hacking across held-out kernel and training-context pairs.Source: §3.4 · p. 8, Figure 4 · p. 8
3.5. The self-improved agent: AIDE85
- AIDE85 replaces greedy selection with bandit strategy arms and periodic forking, bounds prompts with gated failure memory, and adds safeguards against buggy or brittle outputs.Failure memory activates when candidate bug rate reaches 15%; the robustness mechanisms also include a guard for nearly empty generated code and a patch to a failing evaluation script.Source: §3.5 · pp. 8–10
3.6. Ignition test
- In the ignition test, the AIDE47-driven outer loop reached its final score region sooner, but three seeds per arm leave its comparative sample efficiency inconclusive.The treatment reached that region after roughly 20 steps, versus roughly 40 for the AIDEhuman reference; mean endpoints were 0.780 and 0.782, respectively.Source: §3.6 · pp. 10–11, Figure 7 · p. 11
4. Related work
Earlier work optimizes artifacts, automates research workflows, or tunes agent harnesses, while generally leaving the optimization procedure itself human-designed. AIDE2 applies program search and recursive self-improvement to AI research agents and their harness code.
LLM-driven search
- Earlier LLM-driven optimization loops improve candidate artifacts, but the procedure driving optimization remains largely human-designed.AIDE2 instead targets an agent’s own capability on AI R&D tasks.Source: §4 · pp. 11–12
Automated research
- Automated research spans ideation, experiments, writing, and domain applications, while execution outcomes help ground evaluation of generated research ideas.Related work also emphasizes guarding benchmarks against shortcuts and separating development tasks from evaluation tasks.Source: §4 · pp. 11–12
Automated harness search
- Harness-optimization work evolves prompts, parameters, workflows, or whole agent pipelines, but the harness being optimized is separate from the procedure optimizing it.TextGrad treats text components as parameters updated through natural-language feedback.Source: §4 · p. 12
Self-referential improvement
- Self-referential systems optimize their own machinery through empirical evaluation, including rewrites to prompts, search strategies, or agent code.AIDE2 searches harness code and evaluates candidates by downstream research performance under fixed budgets.Source: §4 · pp. 12–13
Meta-learning and learned optimizers
- Meta-learning improves learning procedures from experience across tasks, while program-search approaches also evolve learning algorithms and parameter-update rules.Related work reports meta-learned update rules transferring to environments outside those used for discovery.Source: §4 · p. 13
5. Discussion
AIDE2’s accepted harness rewrites transfer across held-out benchmarks and coincide with reduced reward hacking. The discussion highlights noise, evaluation cost, and agent complexity as constraints on interpreting and deploying these results.
Discussion
- Seven accepted rewrites under a fixed evaluation budget yield AIDE85, which equals or surpasses AIDEhuman on four held-out benchmarks and reduces reward hacking from 55% to 32%.The benchmarks include in- and out-of-distribution tasks; the reduction was not explicitly optimized for.Source: §5 · p. 13
- Noise in both search trajectories and candidate evaluations can cause a falsely accepted rewrite to become the incumbent and derail later search.The private acceptance grade reflects both sources of noise.Source: §5 · p. 13
- The ignition test is inconclusive: three seeds do not establish AIDE47’s sample-efficiency advantage, and a definitive comparison would require more seeds and held-out evaluations.The additional outer-loop seeds and final-agent evaluations are prohibitively costly, making compute access a limiting factor.Source: §5 · p. 13
- Discovered agents remain complex and difficult to interpret, and this complexity can increase deployment friction.Production deployment requires compatibility with existing product features and deployment-infrastructure constraints.Source: §5 · pp. 13–14
A. Held-out benchmark details
Held-out evaluations compare agents under shared, benchmark-specific constraints and scoring measures. ALE-Bench and MLE-Bench use recommended lite sets, while WeatherBench 2 and KernelBench use measures tailored to forecast skill and reward hacking.
A. Held-out benchmark details
- All agents use the same fixed protocol, with per-run cost caps on ALE-Bench, MLE-Bench, and WeatherBench 2 and step caps on FML-Bench and KernelBench.Additional safeguards are generally nonbinding; runs reaching one are scored on their best available candidate, and some AIDE0 runs ended early from context-window errors.Source: A. Held-out benchmark details · p. 24
- Table 1 reports ALE-Bench mean private contest performance, MLE-Bench mean private percentile, WeatherBench 2 forecast-skill gain, and FML-Bench mean normalized test improvement.Each benchmark mean is reported with one standard error.Source: Table 1 · p. 24
- ALE-Bench lite contains 10 of 40 problems and MLE-Bench lite contains 22 of 75 competitions; FML-Bench uses all 18 tasks.KernelBench measures reward hacking, while WeatherBench 2 scores four forecast fields at lead times from 1 to 15 days against ERA5.Source: A. Held-out benchmark details · pp. 24–25
- KernelBench counts reward hacking when isolated speedup exceeds 1.02× but less than half survives in training, or the kernel crashes there.The measure contrasts isolated fixed-input timing with performance inside GPT-2, ViT, or CNN training contexts.Source: A. Held-out benchmark details · p. 25
- WeatherBench 2 reports forecast-skill gain: improvement of an optimized dynamical core over its unmodified starting core, not an absolute leaderboard score.The forecast objective covers 500 hPa geopotential, 2 m temperature, mean sea-level pressure, and 10 m zonal wind.Source: A. Held-out benchmark details · p. 25
B. AIDEhuman is a strong baseline
AIDEhuman is a strong FML-Bench baseline, scoring above the six independently evaluated agents described by the benchmark authors. The figure reports mean normalized test improvement across 18 tasks and three seeds, with small margins relative to seed-level standard error.
B. AIDEhuman is a strong baseline
- AIDEhuman outperforms all six independently evaluated agents on FML-Bench, though leading-agent margins are small relative to seed-level standard error.The comparison reports mean normalized test improvement over 18 tasks and 3 seeds, with ±1 standard error across seed-level means.Source: Figure 8 · p. 25, B. AIDEhuman is a strong baseline · p. 25
C. Harness gains from recursive self-improvement transfer across models
The section tests whether AIDE85’s harness gains persist across underlying models and reports results on ALE-Bench and MLE-Bench. It also examines rejected rewrites and the search and selection approaches explored during recursive self-improvement.
C. Harness gains from recursive self-improvement transfer across models
- AIDE85 gains over AIDE0 transfer across all three models on ALE-Bench and MLE-Bench; on ALE-Bench, Gemini 3 Flash with AIDE85 scores 1858±22 versus Fable 5 with AIDE0 at 1796±24.On MLE-Bench, gains were smaller; AIDE85 with Fable 5, the strongest model there, stayed within one standard error of its AIDE0 score.Source: Figure 9 · p. 26, C. Harness gains from recursive… · p. 26
- About a quarter of rejected rewrites beat the incumbent on the agent-visible public signal but lost on the private grade.Representative rejected proposals are mapped to conceptual analogues; Δg values around −0.004 to −0.007 are small relative to observed run-to-run variability.Source: D. Rejected proposals · p. 26, Table 3 · p. 27
- Rejected proposals ranged across classical search and learning methods, concentrating most heavily on search policy and selection; ensembling was never retained because it consumed budget otherwise available for search.The rejected proposals included population-based and restart-driven exploration, robust selection, and ensembling.Source: D. Rejected proposals · p. 26
E. Prompt compression
The evolved context-management system replaces prior-candidate history with bounded summaries, keeping prompts smaller than AIDE0’s as runs progress. The section quantifies these reductions across held-out benchmarks and describes how bounded prompts avoid context-window failures.
E. Prompt compression
- At run end, discovered prompts were 2.6–5.7× smaller than AIDEhuman on ALE-Bench, MLE-Bench, and FML-Bench and 13–14× smaller on WeatherBench 2; AIDE0 prompts were 3–14× larger.The discovered agents replace prior-candidate history with a bounded summary, while AIDE0 concatenates the code and execution output of every prior candidate into prompts.Source: E. Prompt compression · p. 27
- AIDE0 prompt overflow ended five FML-Bench runs and 48 ALE-Bench runs at the larger per-run budget; AIDE85, AIDE47, and AIDEhuman had no such issues.AIDE0’s prompts grow with run history, while the discovered agents keep prompt size bounded.Source: E. Prompt compression · p. 27
- By run end, median prompt-size reductions versus AIDE0 reached 7× on MLE-Bench, over 40× on WeatherBench 2, and about 50× on ALE-Bench and FML-Bench.Figure 10 plots per-call prompt size in characters across held-out runs; it shows medians and interquartile ranges over task means, with no band for WeatherBench 2’s single task.Source: E. Prompt compression · p. 27, Figure 10 · p. 28