AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement

Yizhe Chi; Wenyi Li; Deyao Hong; Xiaoqiu Wang; Mingju Gao; Kaisen Yang; Bingxiang He; Youjie Zheng; Calvin Xiao; Qinhuai Na

Source-linked AI summary

AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement

Yizhe Chi, Wenyi Li, Deyao Hong, Xiaoqiu Wang, Mingju Gao, Kaisen Yang, Bingxiang He, Youjie Zheng, Calvin Xiao, Qinhuai Na

arXiv:2608.20318v1Aug 20, 2026cs.AIcs.CLcs.LG

TL;DR

AI4AI-Bench addresses the lack of a benchmark that isolates whether agents design training algorithms, a capability relevant to recursive self-improvement. It evaluates code changes across ten repositories; results show limited overall scores and that most submissions leave the learning procedure unchanged.

Problem
No benchmark isolates whether an agent changes how a run is executed or how the model learns, although recursive self-improvement depends on improving the training process inherited by successors.
Source: §1 · p. 1
Method
AI4AI-Bench evaluates agents editing training code across ten repositories, with a four-hour development period followed by a clean-start run and a fixed evaluation.
Source: §1 · p. 3
Results
Across 29 configurations on ten tasks, the mean score was 0.166 and the best system reached 0.250; 141 of 263 submissions left learning unchanged, while those that changed it averaged 0.226 against 0.126.
Source: §1 · p. 3
Takeaways & Limitations
The algorithmic design level is where the distance to a better algorithm is closed, yet agents reach it least often; greater reasoning effort mostly increases their willingness to do so.
Source: §1 · p. 3
The score gap between submissions that do and do not change the learning procedure is not randomized, and on some tasks proxy and final evaluation draw from the same corpus.
Source: §2.3 · p. 5, §4.1 · p. 9

Abstract

from arXiv · show

Recursive self-improvement (RSI) asks whether an AI system can improve the process that produces AI systems, so that the next system inherits the improvement. That process is the training algorithm: a better objective or update rule improves the compute\mbox{-}capability exchange rate for every subsequent run, including the one that produces the next agent. Whether RSI is feasible therefore turns on whether an agent can design training algorithms. No benchmark isolates that ability: existing suites are won by collecting data or by tuning hyperparameters, and none tells a change to how a run is executed apart from a change to how the model learns. We present AI4AI\mbox{-}Bench, 10 frozen research repositories spanning 10 training algorithm families. In each task, an agent has 4 hours on one B300 to rewrite the training algorithm; its code is then rerun from scratch for up to 12 hours and scored by a fixed evaluator hidden from the agent, against the repository's original algorithm under the same procedure. Because the 10 metrics are incommensurable, every task is mapped onto one scale on which $0$ is an uninformative model, $0.1$ is the algorithm the repository ships, and $1.0$ is the task optimum. Across 29 configurations of 6 systems on all 10 tasks the mean score is $0.166$, and the best system reaches $0.250$: even the strongest closes under a fifth of the distance between the algorithm that was already there and the optimum. The submissions show where that distance went: most never change how the model learns at all, and the minority that do average $0.226$ against $0.126$ for the rest. More reasoning effort mostly buys the willingness to go there, taking that minority from $8\%$ of submissions to $64\%$ and the mean score from $0.094$ to $0.196$. We release the task suite, the evaluators and every scored submission, so that the measurement can be repeated as these systems change.

1 Introduction

Recursive self-improvement depends on improving training algorithms, whose gains can carry into later runs. AI4AI-Bench tests whether agents make such algorithmic changes, rather than only changing how a run is executed.

Motivation

Existing benchmarks emphasize prediction, data assembly, or system components, and do not isolate whether an agent changes how a run executes or how the model learns.The introduction identifies this distinction as the question the RSI loop turns on.Source: §1 · pp. 1–2
An algorithmic change rewrites the loss or update rule, whereas a hyperparameter is a number the training algorithm takes as given.Source: §1 · p. 2

Benchmark design

AI4AI-Bench covers ten repositories and training-algorithm families, giving agents four hours to edit code before a clean-start run of up to twelve hours is scored.The final evaluation is unavailable during development; agents can consult a fast proxy metric.Source: §1 · p. 3

Results

Across 29 configurations of six systems on ten tasks, the mean score is 0.166 and the best system reaches 0.250, against 0.1 for the shipped algorithm and 1.0 for the task optimum.The benchmark maps task metrics onto a common scale.Source: §1 · p. 3
Of 263 submissions that changed something, 141 left the learning procedure unchanged; the 122 that changed it averaged 0.226 against 0.126 for the rest.Algorithmic changes include changes to the objective, supervision signal, learning rule, or data.Source: §1 · p. 3
More reasoning effort raised the share of submissions reaching the learning procedure from 8% to 64% and the mean score from 0.094 to 0.196.The paper describes willingness to reach the algorithmic layer as most of what greater reasoning effort buys.Source: §1 · p. 3

2 AI4AI-Bench

AI4AI-Bench tests whether agents can rewrite training algorithms across ten frozen research repositories, while fixing the evaluation and comparing submissions with each repository’s original procedure. Its shared scoring scale places the shipped algorithm at 0.1 and the task optimum at 1.0.

2.1 Task formulation

The agent may rewrite the training loop, objective, optimizer, data pipeline, or schedule, but cannot change the evaluation itself.The final metric and evaluator are fixed in advance; the agent submits source code, which is run from scratch for up to 12 hours.Source: §2.1 · pp. 3–4

2.2 AI4AI-Bench algorithmic tasks

The ten frozen repositories span ten training-algorithm families and require reproducible metrics within a half-day budget on one B300.Weight averaging and one-shot pruning are included despite having no training horizon.Source: §2.2 · p. 4

2.3 Protocol

Exploration lasts four hours on one B300, and only the edited source—not trained weights, cached state, or notes—carries into verification.Training tasks run from initialization for up to 12 hours; non-training tasks execute once to produce a model.Source: §2.3 · p. 5

2.4 Baselines

The baseline is each repository’s own algorithm run with the same hardware, budget, evaluator, and evaluation asset as the submission.The source code is the only difference between baseline and submission.Source: §2.4 · p. 5

2.5 Scoring

The shared score places the shipped algorithm at 0.1, with scores below it measuring shortfall and scores above it measuring progress toward the task optimum.For perplexity, the scoring transformation uses −log rather than raw perplexity so the metric aligns with likelihood tasks.Source: §2.5 · p. 6

3 Experiments

Across 290 task-configuration cells, scores remain close to the bottom of the scale, with 124 below the repository baseline. Six systems were evaluated as combinations of model, harness, and reasoning effort.

3.1 Setup

The study evaluates 29 configurations of six model-harness systems, each attempting all ten tasks for 290 cells.A system is the combination of model, harness, and reasoning effort.Source: §3.1 · p. 6

3.2 Results

0.166 is the mean score across 290 cells, while Claude Opus 5 at medium effort reaches the study’s best configuration average of 0.288.The strongest system averages 0.250; 124 cells fall below the 0.1 repository-algorithm baseline.Source: §3.2 · p. 6
The six systems’ average scores range from Claude Opus 5 at 0.250 to GPT-5.6 Luna at 0.117, all within the bottom quarter of the scale.The range changes the score without changing the overall low-scoring regime.Source: §3.2 · p. 6
Median exploration cost ranges from $48 for Luna to $434 for Sol, but cost does not order systems by score.Opus 5 leads at a median cost of $181, less than half the second-placed system’s spending.Source: §3.2 · pp. 6–7

4 Analysis

Submissions that modify how the model learns score higher than those changing only how the run proceeds, while greater reasoning effort makes learning-layer changes more common. The examples show agents diagnosing task mechanisms and building measurable tools before changing training procedures.

4.1 Most submissions change how the run goes, not how the model learns

Among 263 classifiable submissions, 141 stayed entirely on the run side and 122 touched how the model learns; submissions matched 3.13 change families on average, which can overlap.Run-side families include rate or batch size, checkpoint selection, and trainable capacity; learning-side families include loss, supervision, update rule, and training data.Source: §4.1 · p. 7
Learning-procedure submissions averaged 0.226 versus 0.126 for run-side submissions, but the comparison is not randomized and does not establish an effect of making those changes.The difference remained 0.182 versus 0.128 after dropping agentic RL.Source: §4.1 · pp. 8–9

4.2 Reasoning effort buys nerve, and nerve is what pays

As reasoning effort increased, learning-layer submissions rose from 8.0% to 64.0%, while mean score rose from 0.094 to 0.196.The higher-effort score remained about a tenth of the distance from the shipped algorithm to the task optimum.Source: §4.2 · p. 10
In the Codex grid, median exploration rose from 4 to 16 evaluations and $1.69 to $34.60 per task; 19 cells still scored zero.Eight zero-score cells ended without a usable patch, while eleven completed patches failed the run contract.Source: §4.2 · p. 10

4.3 What the submissions that reached the algorithmic layer did

A one-shot-pruning submission replaced a procedure that only pruned weights with a three-stage training pipeline, reducing perplexity from the repository’s 53.4 to just over 13.The stages changed weight selection and updating, applied layerwise distillation, then fine-tuned with masked knowledge distillation.Source: §4.3 · p. 10
A weight-averaging submission built a faster evaluation rig, then measured learned coefficients at 0.7020 against a uniform average at 0.6880 on its own rig.The rig reduced scoring time from roughly 190 seconds to 0.38 seconds; greedy soup scored 0.7025.Source: §4.3 · p. 10
On multi-turn agentic RL, submissions reaching a perfect score replaced GRPO with imitation learning from optimal moves, with one adding dagger-style policy rollouts.They generated boards, labeled each step with its optimal move, and fine-tuned on that supervision.Source: §4.3 · p. 11
Across the three examples, agents built something measurable before acting: a solver, a faster evaluation rig, or a diagnosis of overwritten activations.The paper identifies this as the capability missing from most submissions: reading training dynamics, naming a failing mechanism, and changing it.Source: §4.1 · p. 10, §4.3 · p. 11

5 Related work

Prior automated-ML work either searches predefined configurations or demonstrates algorithm discovery in compact spaces, while broader agent benchmarks mix learning-rule changes with other levers. AI4AI-Bench isolates and verifies whether repository-level gains come from changing the training algorithm.

Systems engineering for self-improving AI

Systems methods speed up fixed computations but are bounded by hardware ceilings; AI4AI-Bench instead treats kernel, parallelization, and communication edits as systems changes.Once a fixed computation reaches its compute or memory-bandwidth roofline, systems tuning has no remaining headroom.Source: §5 · p. 11

Data engineering for self-improving AI

Data-centric methods select, repair, reweight, or generate corpora for an otherwise unchanged trainer, so the successor inherits data rather than a learning rule.The section notes constraints from finite human-generated text and degradation when generated data is recursively reused.Source: §5 · p. 11

Benchmarks for automated ML research

Earlier automated-ML research includes compact-space demonstrations that search the update rule, but not a benchmark of general research agents improving algorithms in existing repositories.The section contrasts learned optimizers, optimizer search, AutoML-Zero, and Lion with repository-level agent evaluation.Source: §5 · p. 11
AI4AI-Bench isolates algorithmic design by rerunning submitted source from a clean start and classifying whether gains came from execution, data, or the training algorithm.The section presents it as the first benchmark to make this step the object of evaluation.Source: §5 · p. 12
Existing agent benchmarks mix learning-rule changes with execution, data, capacity, and hyperparameter gains, and tend to make tuning and engineering easier than method invention.Even autoresearch edits behave largely like hyperparameter optimization in controlled comparison.Source: §5 · pp. 11–12

6 Conclusion

AI4AI-Bench evaluates whether agents improve the algorithm that trains a successor. Across its tasks, scores remain close to the shipped algorithms, and most submissions do not change how models learn.

Conclusion

Across 29 configurations of six systems on all ten tasks, the mean score was 0.166; the repository’s shipped algorithm scored 0.1, the task optimum was 1.0, and the best system reached 0.250.Source: §6 · p. 12
Of 263 submissions that changed anything, 141 left the learning procedure untouched; the 122 that changed it averaged 0.226 against 0.126 for the rest.The algorithmic changes include changes to objectives, supervision signals, learning rules, or data.Source: §6 · p. 12
More reasoning effort raised the share of submissions reaching the algorithmic layer from 8% to 64%, while mean score rose from 0.094 to 0.196.The strongest setting still reached only a tenth of the way past the algorithm it started from.Source: §6 · p. 12
The authors characterize current agents as recovering a competent default rather than designing past it, and release the benchmark materials and scored submissions for repeated measurement.The release includes the task suite, evaluators, and every scored submission.Source: §6 · p. 12

A The ten tasks

The ten tasks freeze repositories and ask agents to improve each repository’s own training algorithm, with task-specific assets and proxy and final metrics. The suite spans training methods as well as checkpoint combination and pruning.

Task contract

Across all tasks, the contract is to improve the algorithm applied to each repository’s own model, with a cheap exploration metric and a separate final metric.The proxy-to-final relationship varies by task: it may be a subsample of the final protocol or a different benchmark.Source: A The ten tasks · p. 17

OpenR1 — supervised fine-tuning

OpenR1 fine-tunes a 1.5B coder model on 8,005 CodeForces rows; candidates can alter training signal, masking, or objective, and final scoring uses 175 LiveCodeBench problems.The proxy is livecodebench_public_pass_at_1; final scoring uses ten samples per problem at temperature 0.2 and top-p 0.95.Source: A The ten tasks · p. 17

RAGEN — multi-turn agentic RL

RAGEN trains a Qwen2.5-3B policy on Sokoban with online multi-turn GRPO, while board construction, curriculum, rollouts, reward shaping, and updates are open to change.Its proxy is a four-bank solve rate; the final metric is a held-out 512-board solve rate under different fixed seed protocols.Source: A The ten tasks · p. 17

OPD and BTRM

OPD distills a 1.5B student from a mounted teacher, using MATH-500 as proxy and AIME 2024/2025 for final scoring; BTRM trains a scalar reward model on preference pairs.For OPD, AIME inputs are not mounted during exploration; for BTRM, 512 of 2,985 pairs are visible and the remainder are held out.Source: A The ten tasks · p. 17

DPO and DDPO

DPO aligns a merged Zephyr/Mistral-7B model and is scored on IFEval; DDPO fine-tunes Stable Diffusion v1.5 with LoRA against frozen CLIP aesthetic reward.DDPO opens prompt construction, reward shaping, auxiliary losses, updates, and trainable parameters; its proxy uses 64 images and final scoring uses 256.Source: A The ten tasks · p. 17

NPO and DiGress

NPO unlearns TOFU forget10 and reports extraction strength and model utility via a balanced score; DiGress trains a discrete graph diffusion model on QM9 without hydrogens.NPO exploration does not mount the retain-90 anchor or final-role data; the supplied DiGress passage identifies its fast metric as validity, uniqueness, and novelty product.Source: A The ten tasks · p. 17

Model Soup and OWL

Model Soup combines 72 fixed CLIP checkpoints, while OWL prunes OPT-6.7B in one pass; candidates choose combination coefficients or pruning criteria, respectively.Neither shipped recipe trains a model, but submissions may add gradient-based search to soup or distillation fine-tuning to pruning.Source: A The ten tasks · p. 18

B One task contract in full

The reproduced RAGEN contract fixes the scoring environment and final seeds while allowing broad changes to training. It specifies exploration, fresh formal retraining, checkpoint requirements, and how to interpret stochastic results.

Exploration

The exploration budget is four hours; continue meaningful experiments while usable budget remains, and submit early only when no such experiment can be completed and interpreted.A good, loadable, reproducible fallback alone does not establish that exploration is complete.Source: B One task contract in full · p. 18

Formal replay and submission

Formal retraining starts fresh for up to 12 hours, regenerates boards, and excludes exploration rollouts and checkpoints; accepted loadable checkpoints are evaluated independently.If more than three valid checkpoints are produced, only the three with greatest progress values are accepted, and the official result is the best valid final score.Source: B One task contract in full · p. 18

Evaluation boundary

The public four-bank proxy and held-out 512-board final metric use different fixed seed protocols, so each should be compared only within its own tier.The contract labels both metrics higher-is-better.Source: B One task contract in full · p. 18

Task boundaries

Candidates may change board construction, curriculum, rollouts, reward shaping, objectives, and updates, but the scoring engine, action decoding, final seeds, and evaluation behavior are fixed.Formal replay uses the fixed policy and frozen Sokoban evaluator and excludes external or hidden-final data.Source: B One task contract in full · pp. 18–19

Stochasticity

Board and policy training are stochastic, so the contract calls for preserving per-board outcomes and judging small claimed improvements against available uncertainty.It also says one training seed is not a complete noise estimate.Source: B One task contract in full · p. 19

Reference results

The shipped RAGEN solution scores 87/512 boards (0.169922), compared with 60/512 (0.117188) for the fixed policy start, a difference of +27/512 (+0.052734).The listed training time is 2746.19 seconds and final scoring time is 339.15 seconds.Source: B One task contract in full · p. 19

Work surface

The editable work surface includes training-board generation, rollout and data collection, reward and advantage computation, loss reduction, optimization, checkpoint merging, and environment integration.Systems gains matter only when the resulting checkpoint is evaluated under the unchanged final protocol.Source: B One task contract in full · p. 19

Experiment handling

The contract requires tracking outcomes and failures, and warns that a smoke or startup check is not evidence of performance.Candidates with environment, merge, or load failures, non-finite loss, or collapse should be stopped.Source: B One task contract in full · p. 19