PaperlayerView exact revision on arXiv

Source-linked AI summary

Athanor: An Agent-Native System for Autonomous Post-Training Research

Yufa Zhou, Yichen Wang, Jiakun Fan, Ziqi Shu, Zhichao Liu, Anru R. Zhang, Neiwen Ling, Zijun Wei

arXiv:2610.21055v1cs.AIcs.CLcs.LGcs.MA

TL;DR

Post-training is an expensive research search, and no stable scaling law is known when its recipe changes. Athanor lets agents autonomously search post-training recipes, and a four-agent team matched or exceeded the vendor’s instruct model across development and held-out suites.

  • Problem

    Post-training shapes model capabilities, but it remains an expensive iterative search, and no stable compute-to-performance law is known when recipes change.

    Source: §1 · p. 2, §2.2 · p. 5

  • Method

    Athanor lets agents search executable post-training recipes across checkpoint lineages, using public data without teacher models and evaluating a generalist across six capabilities and a sealed suite.

    Source: §1 · p. 2, §2.1 · p. 4

  • Results

    67.3 versus 66.5 on the development suite and 48.2 versus 47.9 on a held-out suite: the team matched or surpassed the vendor’s instruct release.

    Source: §1 · p. 4, §9 · p. 17

  • Takeaways & Limitations

    The authors identify diversity of ideas, rather than execution, as the clearest observed limit to autonomous post-training.

    Source: §9 · p. 17

    The result comes from one campaign with one team and one 4B base model; transfer to larger models or other model families is untested.

    Source: §8 · p. 17

Abstract

from arXiv · show

Recursive self-improvement, in which AI systems build stronger AI systems, is one promising route to superintelligence. We study a weaker form that can be measured today: frontier agents post-training a small raw base model with public data, as a test of their research potential. We build Athanor, an open agent-native system for autonomous post-training research (AutoPostTrain) on real GPUs. We give the agents full research autonomy, low-friction engineering tools, and efficient starter code. The agents carry out the research themselves: they read datasets and evaluation results, build their training data, write and modify training algorithms, tune hyperparameters, and analyze the results to decide what to try next. They work in a sandbox and cannot use external teacher models. Athanor's services supply frozen evaluation, managed jobs, decontaminated data, a persistent research record, and a held-out suite the agents are unaware of. In a 104-hour campaign, a team of four Claude Opus 5.5 agents, each on an eight-GPU node and given no research advice, improves Qwen3-4B-Base from 15.9 to 67.3 on a six-task development suite and from 9.2 to 48.2 on a held-out suite, against 66.5 and 47.9 for the official Qwen3-4B-Instruct-2507. To our knowledge, this is the first demonstration that a team of frontier agents can autonomously post-train a raw base model into a cross-domain generalist model that matches or surpasses the official instruct version on average. The campaign's full record shows that the agents reach this level by recombining known methods and each other's models: SFT on public reasoning data, RL with a rollout change one agent introduced, weight merges across agents, and rounds of self-distillation. With execution handled, today's agents already have the engineering and research ability to conquer this task. We hope Athanor provides a foundation for scaling autonomous AI research.

1 Introduction

The paper introduces Athanor for autonomous post-training research and tests whether agents can turn a raw base model into a cross-domain generalist. In a 104-hour campaign, four agents reach the vendor instruct release’s level on both development and held-out suites, while the campaign record reveals how their research progressed.

1 Introduction

  • Post-training remains an expensive iterative search, and published reports typically record the final recipe rather than the research trajectory that produced it.The paper studies this evidence-driven search over data mixtures, objectives, algorithms, and hyperparameters.Source: §1 · p. 2
  • The setting requires one generalist checkpoint to improve six capabilities from a raw base model, without a teacher model, and to pass an unseen held-out evaluation.Prior work often trains separate specialists, permits teacher models, or lacks a sealed held-out evaluation.Source: §1 · p. 2
  • The campaign record shows agents recombined known methods, shared ideas and checkpoints, converged early on reasoning data, and rarely checked answer correctness.The agents identified issues of format, length, scorability, and difficulty in their data, but rarely checked whether answers were correct.Source: §1 · p. 4
  • Athanor gives agents research autonomy while managing experiment execution, frozen evaluation, decontaminated data, and a durable record of requests and results.Agents inspect data and model behavior, write their own data and training programs, and choose what to run; the environment runs, measures, and recovers experiments.Source: §1 · pp. 3–4
  • In 104 hours, four agents post-trained Qwen3-4B-Base into a cross-domain generalist matching or surpassing the vendor instruct release, scoring 67.3 vs 66.5 on development and 48.2 vs 47.9 held-out.The campaign cost about $12,000 and used no teacher model.Source: §1 · p. 4

2 On AutoPostTrain

AutoPostTrain treats post-training research as a team of agents searching executable recipes over checkpoints, guided by noisy benchmark feedback. Its difficulty comes from a broad, state-dependent search space and delayed, incomplete evidence about recipe quality.

2.1 Problem Formulation

  • AutoPostTrain searches executable recipes over a growing pool of checkpoints, with agents sharing history and proposing recipes from one shared pool.Executing a recipe yields a checkpoint; benchmark evaluations provide noisy score vectors that only estimate latent capability.Source: §2.1 · pp. 4–5

2.2 What Makes AutoPostTrain Hard

  • Recipe search is combinatorial and state-dependent: stages, hyperparameters, data, and starting checkpoints interact, while benchmark feedback is non-differentiable.Each new checkpoint expands the admissible recipe set, making the outer search stochastic black-box hill climbing.Source: §2.2 · p. 5
  • A researcher agent’s prior focuses proposals, but scores are useful only when linked to the checkpoint lineage that produced them.Recipe effects depend on their starting checkpoint.Source: §2.2 · p. 5
  • Post-training feedback is delayed, noisy, and incomplete, and the paper asks how research output and understanding grow with experiments and compute.Training and evaluation precede feedback; finite benchmarks can blur nearby candidates and cover only part of a capability.Source: §2.2 · p. 5

3 Athanor

Athanor implements autonomous post-training research through concurrent researcher and environment loops over a shared record. It leaves research decisions to agents while supplying protected evaluation, persistent state, and background execution services.

3.1 The Autoresearch Loop

  • Each researcher reads the shared record, analyzes evidence with its own tools, and submits jobs, evaluations, data, or artifacts; services execute requests and return results.The researcher loop and environment services run concurrently, recording updates in the same history.Source: §3.1 · p. 6, §3.2 · p. 6

3.2 The Research Environment and Its Roles

  • Athanor supplies decontaminated public data, editable training starter code, and frozen development and held-out evaluation, while agents cannot use external teacher models.It does not prescribe a research procedure; agents retain research decisions, while software enforces the sandbox and sealed evaluation.Source: §3.2 · pp. 6–7

3.3 Keeping the Loop Fast and Reliable

  • Execution services allocate resources, run and recover jobs, and evaluate outside the researcher’s sandbox; durable lineage and background jobs preserve evidence and keep experiments in flight.The record links requests to code, data, input checkpoints, outputs, and measurements, enabling sessions to resume and teammates to reuse checkpoints.Source: §3.3 · p. 7

4 AutoPostTrain at Scale

In a 104-hour campaign, four agents post-trained Qwen3-4B-Base to match or exceed Qwen3-4B-Instruct-2507 on the final development and held-out suite means. The campaign also documents task-level differences and the execution costs and conditions behind that result.

4.1 Experimental Setting

  • Four Claude Opus 5.5 agents each used an eight-GPU node for 104 hours, with public data, no external model calls, and no human research advice.Each suite has six capability tasks; held-out tasks measure the same capabilities on different material, and the final measurement uses longer output limits and sampled decoding.Source: §4 · p. 7, §4.1 · pp. 7–8

4.2 Model Improvement

  • 67.3 on development and 48.2 on the held-out suite, versus 66.5 and 47.9 for Instruct-2507, with paired 95% intervals containing zero.These are the final-measurement means; the intervals are −0.8 to +2.4 and −2.0 to +2.6, respectively.Source: Table 1 · p. 8, §4.2 · p. 9
  • Under the campaign protocol, the submission scores 66.1 ± 0.8 on development versus 66.4 for Instruct-2507, and 46.7 ± 1.3 held-out versus 48.0 ± 1.4.The held-out paired difference is not distinguishable from Instruct-2507; all four agent submissions reach a similar level.Source: §4.2 · p. 9
  • The submission leads Instruct-2507 on multi-step action and code but trails on logic and mathematics; action offsets those deficits in the overall means.Without multi-step action, it trails by 3.7 points on development and 4.2 on held-out; reasoning answers are also more often cut off at output limits.Source: §4.2 · p. 9

4.3 Execution Effectiveness

  • Jobs occupy 93% of GPU hours while agents wait on jobs for 87% of session time; GPUs compute for about 75% of available hours.The environment absorbs node outages, and submitted checkpoints, evaluations, and lineage are recovered after the campaign.Source: §4.3 · pp. 9–11, Figure 3 · p. 10
  • At $3.49 per GPU hour, the campaign costs about $11,800 total, including about $10,800 for GPU time; the final six frontier points cost about $8,000.The agents’ inference accounts for 9% of the total cost.Source: §4.3 · pp. 10–11, Figure 4 · p. 11

5 Trajectory Analysis

The campaign record traces how four agents combined known post-training methods and exchanged ideas and checkpoints. Their search converged early on one public reasoning corpus, while data inspection was more attentive to format and scorability than answer correctness; gains slowed after the first day.

5 Trajectory Analysis

  • The analysis follows 500 jobs, 255 requested full evaluations, and the lineages of 174 evaluated checkpoints to characterize the recipe and search.The record also includes agent transcripts and notes.Source: §5 · p. 11

5.1 Recombination

  • A quarter-greedy GRPO rollout change reduced validation loops from 86 to 18 and raised Agent 3’s checkpoint from 49.7 to 57.3.The change targeted greedy repetition loops that sampled rollouts rarely exhibited, and teammates adopted it within hours.Source: §5.1 · pp. 12–13, §5.2 · p. 14

5.2 Team Progress

  • From hour 21, every new best checkpoint was a merge of several agents’ models or was trained from such a merge; 28 of 32 checkpoints after hour 84 used all four agents’ training.Of 287 lineage links, 139 paths included a checkpoint fetched from a teammate.Source: §5.2 · p. 14

5.3 Recipe Convergence

  • Within the first hour, all agents chose AM-Thinking-v1-Distilled for 76–87% of their first mixtures’ public-data rows; alternative data lineages scored lower and were dropped.A new lineage trained on other data stayed near 55 while the team’s best was near 62, and merging it lowered logic scores.Source: §5.3 · pp. 14–15

5.4 Uneven Data Sense

  • Agents checked data format, length, scorability, and difficulty, but rarely checked answer correctness; a planned filter for questionable ground-truth answers was never run.They rejected overlong traces, removed 1,093 unscorable instruction rows, and fixed a deduplication step that marked every puzzle as a duplicate.Source: §5.4 · p. 15

5.5 Why the Gains Slow

  • The frontier reached 60 within the first day, while the remaining three days added about seven points, with the leading logic score rising from 49 to 65.No new training data entered the submission’s lineage in the last two days; gains came from model merges and self-distillation on model samples.Source: §5.5 · p. 15

6 Insights and Discussion

The campaign record turns post-training experiments into measurements tied to recipes and checkpoints, and the authors describe a broader loop in which agents could take on more parts of research. That loop would need checks against contamination and agents learning to please the evaluator.

Automated scientific discovery of post-training

  • The record supports measurements of post-training choices: averaging equally strong models from different lineages gained two to three points, moving a merge farther from the base traded knowledge for logic, and first-round self-distillation added about one point.The record preserves answers, repeated evaluations, model lineages, and teammates’ work, which let the authors trace the rollout change to scored answers showing repetition loops.Source: §6 · p. 16

Toward a broader autoresearch loop

  • A broader autoresearch loop could let agents construct evaluation suites and environments, generate and verify training data, and use recorded research trajectories as training data for future models.Each step that gives agents more of the loop would require checks against contamination and evaluator-pleasing behavior.Source: §6 · p. 16

7 Related Work

Prior work studies agent post-training, model-system improvement, and autonomous research teams, often in narrower settings. Athanor differs by having a team search for one generalist checkpoint without a teacher model and evaluate it on a held-out twin suite.

Automated post-training

  • PostTrainBench and Locus train benchmark specialists, while other systems use external model APIs or isolate fine-tuning and reinforcement-learning pipelines; Athanor instead searches for one generalist checkpoint without a teacher and with a held-out twin suite.RSI Arena uses competing agents and human preference rankings, rather than the described held-out-suite setup.Source: §7 · p. 17

Agents that improve AI systems

  • Earlier agents improve surrounding code, harnesses, or inference systems, whereas Athanor’s agents change model weights.Source: §7 · p. 17

Autonomous research and teams of agents

  • Prior team research finds that sharing can help when a verifier selects useful progress, while agents sharing a model, harness, and context can act alike; Athanor measures team convergence on real training compute.Related work also describes a single agent becoming stuck in a small idea basin.Source: §7 · p. 17

8 Limitations

The evidence comes from one campaign with one team of four identical agents and one 4B base model; the recipe’s transfer to larger models or other model families is untested. The agents used no teacher model, but much of their public training data had been distilled from stronger models.

One campaign, one base model, and one agent model

  • One campaign with one team and one 4B base model does not establish campaign-to-campaign variation or whether the recipe transfers to larger models or other model families.The four submissions were not independent repeats because the agents shared their work; a campaign without sharing would also be needed to measure the team’s contribution.Source: §8 · p. 17

Not yet at the vendor’s ceiling

  • The result reaches the vendor’s instruct release, not its higher-scoring reasoning releases, and public training data was partly distilled from stronger models.It therefore shows what agents can build from public resources, not from the base model alone.Source: §8 · p. 17

9 Conclusion

Athanor enables frontier agents to run autonomous post-training research end to end; in 104 hours, four agents brought Qwen3-4B-Base to scores level with or above the vendor’s instruct release on development and held-out suites. The campaign reached that result by composing known methods, and the authors identify idea diversity as the clearest observed limit.

9 Conclusion

  • 15.9 to 67.3 on the six-task development suite and 9.2 to 48.2 on the held-out suite, versus 66.5 and 47.9 for Qwen3-4B-Instruct-2507.The 104-hour campaign used public data and cost about $12,000 in GPU time and inference.Source: §9 · p. 17
  • The agents reached this level by composing known methods and fixing problems exposed by their own measurements.The campaign used public data and prohibited external teacher models, as described in the conclusion’s scope.Source: §9 · p. 17
  • With execution handled, the clearest limit observed was the diversity of the agents’ ideas.The authors suggest that agents could post-train models for automatically scored behavior when users have tokens and GPUs.Source: §9 · p. 17

A System Details

Athanor separates researchers’ decisions from experiment execution, preserves durable research state, and supports productive background jobs. Compared with neighboring systems, it is the only one described as fully supporting all six listed research and execution features.

A System Details

  • Execution and protected evaluation keep training and scoring services outside each researcher’s sandbox, returning only development scores and scored answers.Decontamination also runs outside the sandbox; rows overlapping development or held-out items are removed using a 13-word-sequence rule.Source: A System Details · p. 24
  • Durable workspaces, journals, evaluations, artifacts, and checkpoint lineage let sessions resume after node loss and make research histories auditable.Background jobs continue after an agent session ends, and state changes are sent back as session messages.Source: A System Details · pp. 24–25

B Feature Comparison with Neighboring Systems

  • Athanor is the only compared system with full support across all six research and execution feature columns.The comparison distinguishes model-weight post-training systems from research systems that mostly improve programs or single-task solutions.Source: B Feature Comparison with Neighboring… · p. 25, Table 4 · p. 25
  • The comparison defines generalist checkpoints as targeting multiple capabilities, and separates checkpoint lineage from research lineage and sealed final evaluation.Checkpoint lineage tracks parent–child training relationships; research lineage links hypotheses, artifacts, and evidence.Source: B Feature Comparison with Neighboring… · p. 25

C Experimental Details

The campaign starts from a raw base model, uses public and generated training data, and selects the submission on the development suite before scoring a held-out suite. The final measurement changes decoding and output limits; the contamination audit found no held-out items in scanned self-built datasets, but 47 datasets were not fully scanned.

C Experimental Details

  • The campaign protocol used two-seed reference measurements, fixed decoding and task-specific output limits, and three runs of the default starter.Mathematics averaged four samples per problem; the other tasks decoded greedily.Source: C Experimental Details · p. 26
  • The pool contained 447 public datasets, with 40,113 rows removed before the campaign for overlap with development or held-out items.The agents could also use further decontaminated public data, generated problems, and samples from their own checkpoints.Source: C Experimental Details · pp. 26–27
  • Agents could not call or download other models as teachers or receive human research advice, while teammates could inspect and reuse one another’s work and checkpoints.The operator fixed the selection rule in advance; the held-out suite was scored after the campaign and did not determine the submission.Source: C Experimental Details · p. 27
  • 48.2 was the submission’s held-out mean in the final measurement, compared with 46.7 under the campaign protocol.The final measurement used four seeds; the campaign-protocol result used two seeds, and the submission was selected by full development-suite score.Source: C Experimental Details · p. 27
  • Of 108 self-built datasets descended from the submission, 61 scanned in full contained no held-out item, while 47 were unavailable for a complete scan.The 47 were lost with local disks or exceeded scan limits; they were reformatted copies and mixtures of decontaminated-pool datasets, logic puzzles, and checkpoint samples.Source: C Experimental Details · pp. 27–28
  • The final measurement used sampled decoding and up to 32,768 tokens for mathematics, logic, code, and knowledge; all four final submissions were within 1.6 points of Instruct-2507 on both suites.Each model was evaluated with four seeds using the same evaluator build; the authors also checked Instruct-2507 with its own recommended settings.Source: C Experimental Details · p. 29, Table 10 · p. 29

D.1 Why the Submission Trails on Logic and Mathematics

The submission trails Instruct-2507 on logic and mathematics partly because answers are cut off at output limits, while the remaining mathematics gap is unexplained. The winning lineage combined fine-tuning, reinforcement learning, and data choices; several alternative attempts were dropped, and agents’ leads and workloads shifted over the campaign.

D.1 Why the Submission Trails on Logic and Mathematics

  • 17.5% of submission logic answers were cut off at 16,384 tokens versus 0.8% for Instruct-2507; mathematics cutoffs were 24.2% versus 9.8%.Among completed answers, logic scores were 76.4% versus 81.0%, while mathematics scores were 50.1% versus 47.2%; large logic puzzles concentrated the cutoffs.Source: §D.1 · p. 30, Figure 11 · p. 31
  • The agents’ self-distillation kept the shortest correct answer, but four fine-tuning rounds left final submissions’ logic cutoff shares at 17.5–18.9%.The authors attribute the limited learning signal on large puzzles to the model rarely getting them right and the training pools leaning toward smaller puzzles.Source: §D.1 · pp. 30–31
  • With a 32,768-token limit and sampled decoding, the submission’s logic score changed from 63.4 to 63.0, leaving final submissions 15–19 points below Instruct-2507.On mathematics, the longer limit narrowed but did not eliminate the gap: 38.4 against 44.2; the authors had not identified the remaining cause.Source: §D.1 · p. 31

D.2 The Winning Lineage

  • The winning lineage’s fine-tuning mixtures varied supplementary data, while reinforcement learning used 20–27 thousand public or generated problems per round with rule-based rewards.The data included verified code, rewritten science questions, checkable instruction-following data, chat, and ALFWorld expert trajectories; the logic share rose over rounds.Source: §D.2 · p. 31, Table 11 · p. 32
  • Several alternatives were dropped after weaker diagnostics: solver-trace fine-tuning, logic-heavy reinforcement learning, short-answer fine-tuning, length-penalty RL, and TIES merges.TIES scores were 64.5 and 64.7, below the best linear extrapolation at 65.5.Source: §D.2 · p. 31

D.3 Exchanges Between Agents

  • Figure 12 shows the agents’ best single evaluation so far: the lead changed hands several times, and all four agents ended with best scores between 65.8 and 66.9.Source: Figure 12 · p. 33

D.4 How the Agents Inspected Data

The appendix documents the agents’ data checks and campaign rhythm, then records post-training observations and a worked example of an agent diagnosing and changing a rollout loop.

D.4 How the Agents Inspected Data

  • Each agent inspected about 30–70 data rows in the first six hours, later 2–12, and ran at least six dataset or scorer checks; correctness checks still missed issues.The counts are lower bounds; agents also found answer formats the evaluator would not parse, while one left a suspicious ground truth unchecked and another marked solutions verified without comparing expected answers.Source: §D.4 · p. 33

D.5 The Campaign Agent by Agent

  • GPU work shifted from fine-tuning to GRPO, then mostly self-distillation sampling for Agents 2–4, with evaluations between jobs.The agents’ turns came in bursts as results arrived; single evaluations ran high, while multi-seed means appear in Table 9.Source: §D.5 · p. 33

D.6 Post-Training Observations

  • Equal-strength RL merges gained about 2.7 points, while later self-distillation rounds changed scores by at most 0.2.Scaling the merge raised logic scores from 56.4 to 63.0 while knowledge scores fell beyond factor 1.1; fine-tuning another merge scored 65.3 versus 65.5 before, but averaging it with a distilled model reached 65.9.Source: §D.6 · p. 34

E Research Trajectories

  • The appendix defines shorthand: dev6 is the development suite, task aliases name logic, knowledge, instruction-following, and action tasks, and RFT means rejection-sampling fine-tuning.A “soup” is an average of checkpoints; RESP is the rollout answer-length limit.Source: E Research Trajectories · p. 34

E.1 A Complete Agent Turn

  • Validation found that 26 of 26 cut-off instruction-following answers and 33 of 55 cut-off logic answers were loops, prompting mixed greedy and sampled training rollouts.The agent inspected trainer code, rebuilt logic and mathematics training problems, sampled a new dataset, and stopped the prior GRPO job.Source: §E.1 · p. 35

E.2 Further Traces

The further traces record experiments with merge scaling, self-distillation, alternate training data, and model-generated traces, including both useful results and ideas the agents dropped.

Finding (1): scaling a merge away from the base

  • Factor 1.3 scored 64.87, while factors 1.1–1.3 differed by about one seed’s noise; factor 1.2’s four-seed mean was 63.97.The first factor-1.2 evaluation scored 64.58, and the agent kept that factor for further evaluation.Source: §E.2 · p. 35

Finding (1): designing the self-distillation rounds

  • A first self-distillation round raised development score from 63.97 (four seeds) to 64.99 (three seeds), and Agent 2 reused Agent 3’s samples to fine-tune another merge.The agent sampled the best merge on selected prompts, kept verified-correct answers, preferred the shortest, and fine-tuned for one epoch at low learning rate.Source: §E.2 · p. 36

Finding (2): averaging two agents’ reinforcement-learning models

  • Averaging two near-57 RL models scored 59.95 and beat both parents on every task but alfworld; the resulting checkpoint became the team’s best, but was not on the submission lineage.The observation supports keeping distinct RL lineages for later averaging; averaging a strong model with weaker ones was neutral.Source: §E.2 · p. 36

Finding (3): a lineage without the shared corpus

  • A lineage trained on R1 traces plateaued near 55, and its merge diluted zebra performance, so Agent 3 returned to the shared-corpus lineage.The team’s prior SFT models used AM-Thinking traces; the alternate lineage was begun to increase soup diversity.Source: §E.2 · p. 36

Finding (4): rejecting a data source after reading model outputs

  • Solver-trace SFT scored 27.0 on zebra-only diagnostics versus about 49, and the model made unjustified deductions, so the data source was dropped.The agent had trained on 6,900 solver-generated traces and read the model’s answers before rejecting the source.Source: §E.2 · p. 36

Finding (4): checking the data before spending GPU hours

  • For large puzzles, the model had few short correct answers to learn from, so the compact-solution RFT idea was dropped without a training run.For 5x5 puzzles, only 3 of 157 solved puzzles had a correct answer under 8k tokens; for 6x4, only 3 of 121 did.Source: §E.2 · p. 36

Related papers

Loading 2610.21055v1…