PaperlayerView exact revision on arXiv

Source-linked AI summary

Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering

Junlin Yang, Che Jiang, Yu Fu, Tianwei Luo, Can Ren, Weizhi Wang, Kaikai Zhao, Hongyi Liu, Yuxin Zuo, Yuru Wang, Yuchen Fan, Kai Tian, Zhenzhao Yuan, Xiaojian Lin, Li Sheng, Rushi Qiang, Guoli Jia, Xingtai Lv, Ermo Hua, Dianqiao Lei, Youbang Sun, Ning Ding, Bowen Zhou, Kaiyan Zhang

arXiv:2607.28568v1cs.CL

TL;DR

AI systems that improve the process of building AI need executable testbeds and methods for learning from iterative feedback. This paper introduces OpenMLE and Frontis-MA1 for that setting, reaching 71.21% Medal Average with OpenMLE-Evo-Max1 and exceeding GPT-5.5 + Codex.

  • Problem

    Recursive self-improvement needs agents that build and improve AI systems, while machine learning engineering provides a concrete testbed with delayed, noisy execution feedback.

  • Method

    OpenMLE connects executable environments, execution-grounded operator post-training, and experience-guided long-horizon search through shared Draft, Improve, Debug, and Crossover operators.

  • Results

    71.21% Medal Average was achieved by Frontis-MA1-35B with OpenMLE-Evo-Max1, exceeding GPT-5.5 + Codex; under OpenMLE-Evo, Medal Average rose from 39.39% to 60.61%.

  • Takeaways & Limitations

    The released OpenMLE stack, models, datasets, and infrastructure provide a reproducible workflow for studying executable AI4AI and test-time evolution.

  • Takeaways & Limitations

    OpenMLE does not yet realize the full vision of recursive self-improvement and has five capability boundaries identified for future work.

Abstract

from arXiv · show

Recursive self-improvement (RSI) requires AI systems that improve the process of building AI (i.e., AI4AI); machine learning engineering (MLE) offers a concrete, executable testbed for studying this capability. We introduce OpenMLE, an open full-stack system for RSI research in MLE, spanning verifiable task environments with execution feedback (OpenMLE-Gym), operator learning (OpenMLE-RL), and long-horizon search (OpenMLE-Evo). On this stack we post-train Frontis-MA1 (35B) as a meta-evolution agent for MLE, aligning post-training and inference around four atomic program-evolution operators (Draft, Improve, Debug, Crossover): the same operators are trained via execution-grounded SFT and RL on data deduplicated against all evaluation benchmarks, then composed into long-horizon search, coupling learning and evolution in a single loop. On MLE-Bench Lite under a 12-hour per-task budget on one RTX 4090 capped at 12 GB VRAM, Frontis-MA1 (35B) improves Medal Average from 39.39% to 60.61% over its base model with OpenMLE-Evo, and reaches 71.21% with OpenMLE-Evo-Max (benchmark-independent experience priors and asynchronous search), exceeding GPT-5.5 + Codex and approaching GPT-5.6 Sol and the 2.8T Kimi K3. On held-out NatureBench Lite, both components transfer: with the framework fixed, swapping in the trained model raises Match-SOTA from 50% to 70%; with the model fixed, swapping in OpenMLE-Evo raises it from 20% to 50%. We release the model weights and the full OpenMLE stack to enable reproducible research on executable AI4AI toward RSI. Code: https://github.com/FrontisAI/OpenRSI

1. Introduction

The introduction frames machine learning engineering as a concrete AI4AI testbed for recursive self-improvement and presents OpenMLE as a full-stack system connecting executable environments, post-training, and long-horizon evolution. Using this stack, Frontis-MA1-35B substantially improves over its base model under a fixed MLE-Bench Lite budget, while evaluations separate model and harness contributions and test transfer.

  • Motivation: AI4AI uses AI systems to write code, run experiments, search designs, and help build improved AI systems, with recursive self-improvement as its more ambitious endpoint.The introduction presents capability growth as increasingly involving AI systems in the construction of subsequent AI systems.
  • Motivation: Machine learning engineering is a direct AI4AI instantiation because agents build real-world machine learning solutions and iteratively improve them through execution feedback.MLE trajectories can begin with valid pipelines and progress through repeated experiments toward competitive solutions.
  • OpenMLE: OpenMLE connects verifiable executable environments, execution-grounded post-training, and test-time evolution in one full-stack workflow for studying recursive self-improvement in MLE.Its components include OpenMLE-Gym, OpenMLE-ERL, and OpenMLE-Evo, covering task construction, stronger MLE behavior, and long-horizon search.
  • OpenMLE-Gym: 5,758 quality-gated executable tasks provide isolated execution, structured feedback, and task-specific evaluation for scalable training, search, and evaluation.OpenMLE-Gym unifies these tasks within the stack’s executable environment layer.
  • Results and evaluation: 39.39% to 60.61% Medal Average and 0.5828 to 0.7647 Human Rank: Frontis-MA1-35B improves over Qwen3.6-35B-A3B under the identical OpenMLE-Evo harness and a fixed 12 GPU-hour per-task budget.The paper evaluates gains at model, harness, and system levels on MLE-Bench Lite, while also providing transfer evidence on NatureBench Lite.

2. Problem Formulation

The formulation treats AI4AI as systems improving other AI systems, while meta-evolution reuses evolutionary trajectories to train the proposing model. It casts MLE as executable, feedback-driven program evolution and trains the same operator-conditioned policy through SFT and RL.

  • From AI for AI to meta-evolution: AI4AI optimizes AI systems that create or improve other AI systems, while evolution repeatedly modifies candidates using execution feedback.Meta-evolution adds a learning loop that reuses evolutionary trajectories to train the model proposing modifications.
  • Executable evolution for AI-building tasks: Each MLE task specifies a natural-language objective, data assets, submission contract, evaluator, and sandboxed execution environment for search.At each step, inference selects an operator and constructs its context from zero or more parent programs.
  • Executable evolution for AI-building tasks: Execution returns task scores, status, logs, artifacts, and runtime metadata, with scores converted into signed values so larger always indicates better outcomes.The task-specific evaluator maps execution results to the task score before normalization.
  • Executable evolution for AI-building tasks: OpenMLE stores candidates and feedback in task-local databases, repeatedly applies operators, and searches within a finite execution budget for the highest-scoring candidate.Its operator space contains Draft, Improve, Debug, and Crossover, while inference controls their global composition.
  • Learning to evolve from executable experience: Meta-evolution trains the operator-conditioned policy to assign higher probability to programs with stronger execution outcomes, using both SFT and RL.SFT quality-filters high-scoring programs, whereas RL weights sampled programs by processed rewards and entropic advantages; both update the same parameters.

3. OpenMLE-Gym: Building Scalable Verifiable Environments

OpenMLE-Gym provides a scalable, compute-backed environment for executable MLE tasks, coupling sandboxed program execution with task-specific evaluation and reproducible feedback. It standardizes heterogeneous task sources into quality-controlled packages, enabling post-training, search, and evaluation toward recursive self-improvement.

  • Environment design: OpenMLE-Gym hosts resource-intensive MLE tasks, executes candidate programs under controlled budgets, and returns reproducible diagnostics and rewards through task-specific evaluators.This substrate supports post-training, search, and evaluation for recursive self-improvement.
  • Task construction: OpenMLE-Gym combines Curated Anchors, Kaggle Dataset tasks, and Kaggle Competition tasks to balance task quality, scale, and coverage.The three source-specific paths occupy complementary positions in the quality–scale trade-off.
  • Task package contract: Each task package exposes public inputs and utilities while isolating hidden answers and providing metric.py for scalar execution feedback.The shared package contract gives heterogeneous tasks a common agent-facing interaction surface.
  • Execution backend: The backend schedules jobs across CPU/GPU Docker workers and records execution outcomes, including successful completion, runtime error, missing code, missing submission, scoring failure, and timeout.Records preserve status, score when available, logs, error type, runtime metadata, and workspace artifacts for reproducible scoring and scalable parallel execution.
  • Task construction: 5,758 executable tasks comprise 156 manually selected Curated Anchors, 3,362 Kaggle Dataset tasks, and 2,240 retained Kaggle Competition tasks.The pool spans tabular, text, time-series, image, and other modalities, with classification and regression alongside more engineering-intensive task types.

4. OpenMLE-ERL: Reinforcing Reusable Evolutionary Operators

OpenMLE-ERL trains reusable executable program-transformation operators rather than full search trajectories, combining execution-grounded SFT with online RL to improve solutions found under finite verification budgets. Its training data combines independent Draft solutions with evolutionary revisions, while task-package release is limited by licensing constraints.

  • SFT and RL: Execution-grounded SFT broadens executable behaviors, while RL reinforces better candidates using adaptive score bounds and entropic advantages rather than validity alone.This division addresses the need to improve both the breadth of reachable programs and the quality of repeated transformations.
  • Data release: 1,415 task packages include full data, while 4,343 additional tasks provide only prepare.py and metric.py because of licensing and copyright constraints.The scripts are released without redistributing the corresponding source data.
  • Reusable operator design: OpenMLE-ERL trains reusable operators over executable candidates, allowing different evolutionary search procedures to compose shared local skills under one sandbox protocol.The operators are Draft, Improve, Debug, and Crossover.
  • Budget-adaptive execution: Budget-adaptive collection stops at an accepted-example quota or execution limit, directing verification compute toward tasks with sparse successes.Executable evolutionary training must handle failed rewards, heterogeneous continuous metrics, long sandbox runtimes, and parent-dependent non-Draft actions.
  • Training corpus: 26,259 examples comprise the released SFT corpus: 17,245 complete Draft responses and 9,014 Improve, Debug, and Crossover trajectory steps.The parallel path samples complete solutions independently, while the evolutionary path retains useful local steps from high-quality trajectories.

LEARNING FROM EXECUTED ROLLOUTS

OpenMLE learns operator behavior from executed rollouts, retaining successful Draft and repair traces in a 26,259-example SFT corpus. Its RL pipeline improves learning efficiency through adaptive upper-tail rewards, asynchronous execution, and diverse fitness-informed parent selection.

  • SFT corpus construction: 26,259 selected examples form the released SFT corpus, combining threshold-passing Draft solutions with useful steps traced from repeated Debug repairs.The corpus uses a budget-adaptive stopping rule and two data-selection paths: Parallel Path and Evolutionary Path.
  • Reward shaping: Adaptive bounds remap raw task scores into processed rewards that preserve resolution around each task’s on-policy score frontier.This addresses incomparable score ranges across objectives such as accuracy and log loss.
  • Reward shaping: Entropic advantages amplify reward gaps near the top of each rollout group, directing more policy-learning signal to the best candidates.The entropic concentration parameter is selected under a fixed entropy/KL budget and replaces the usual GRPO-style group-normalized signal.
  • Rollout execution: Asynchronous rollouts let the trainer consume independently completed generation-and-execution groups, avoiding idle time caused by the slowest sandbox job.This targets execution latency, which varies substantially across tasks and candidate solutions.
  • Operator learning: Fitness-proportional parent sampling combines parent strength, child-reward variance, and cooling to target informative states while preserving search diversity.Cooling reduces repeated-visit dominance, exposing Improve, Debug, and Crossover to useful local contexts.

5. OpenMLE-Evo: Scaling Experience-driven Long-Horizon Search

OpenMLE-Evo turns evolutionary test-time search into experience-driven test-time learning by converting execution outcomes into reusable evidence that guides future exploration. It combines structured experience records, population-level search state, multi-factor parent selection, and operator-conditioned context for long-horizon program evolution.

  • Motivation: OpenMLE-Evo closes the loop between proposing, executing, learning from experience, and adapting future expansion over long horizons.The design targets AI-for-AI and recursive self-improvement, where generating more candidates alone is insufficient.
  • Search design: The search composes Draft, Improve, Debug, and Crossover within an AIRA-Evo-style population loop while redesigning how execution evidence is used.Unlike standard AIRA-Evo’s largely free-form memory, OpenMLE-Evo emphasizes structured, execution-grounded experience for search control.
  • Experience representation: Each evaluated candidate receives a deterministic experience card recording provenance, performance, execution outcome, and resource usage.The card provides a compact and consistently structured record for every search node.
  • Experience representation: The experience board tracks method families, family-wise best candidates, underexplored directions, repeated failures, score trends, and the parent graph.Together, the card and board preserve both node-level history and the surrounding search neighborhood.
  • Parent selection: OpenMLE-Evo selects parents using normalized validation score, positive improvement over the strongest parent, and method-family novelty.This policy preserves pressure toward high-quality solutions while allocating budget to meaningful progress and underexplored approaches.
  • Memory and context: After parent sampling, OpenMLE-Evo builds operator-conditioned context from deterministic records, recent ancestors, and direct siblings instead of appending the full free-form history.For Improve, the context includes score, parent-relative improvement, method family, runtime, rank, incumbent status, and direction novelty.

6. Experiments

Experiments evaluate OpenMLE-Evo on the 22-task MLE-Bench Lite split under a fixed 12-hour, single-RTX-4090 budget. Execution-grounded post-training improves Medal Average, while OpenMLE-Evo-Max further combines model training with reusable priors and asynchronous search.

  • MLE-Bench Lite: Under the identical OpenMLE-Evo harness, Frontis-MA1-35B improves Medal Average over its corresponding Qwen3.6 backbone by 21.22 percentage points.Each configuration uses three independent runs under a fixed 12-hour per-task budget on a single RTX 4090 with 12 GB VRAM.
  • MLE-Bench Lite: 71.21%: Frontis-MA1-35B with OpenMLE-Evo-Max exceeds GPT-5.5 with Codex by 3.03 percentage points.OpenMLE-Evo-Max extends OpenMLE-Evo with cross-task priors distilled from public competition artifacts and asynchronous multi-GPU parallel search, while keeping total sandbox compute unchanged.
  • MLE-Bench Lite: 18.18 percentage points: Frontis-MA1-30B reproduces the post-training gain over its Qwen3 backbone.The companion model tests whether the post-training improvement transfers across a second backbone and model scale.
  • MLE-Bench Lite: 66.67%: Frontis-MA1-30B with OpenMLE-Evo-Max reaches this Medal Average on MLE-Bench Lite.The same companion model reaches 53.03% with OpenMLE-Evo under the reported evaluation configuration.

B. Broader OpenMLE-Evo system context

Across evaluated models, OpenMLE-Evo achieves Medal Average results ranging from 27.27% to 65.15%, with Grok-4.5 performing best at 65.15%. LongCat-2.0 and several other models also exceed 50% Medal Average under OpenMLE-Evo.

  • Model results: 65.15% Medal Average is achieved by Grok-4.5 with OpenMLE-Evo, the highest listed result.Grok-4.5 records 22.00/22 and 0.8052 alongside this result.
  • Model results: 56.06% Medal Average is achieved by both LongCat-2.0 and Doubao Seed 2.1 Pro with OpenMLE-Evo.LongCat-2.0 records 21.00/22, while Doubao Seed 2.1 Pro records 20.33/22.
  • Model results: 51.52% to 54.55% Medal Average is achieved by Qwen3.7 Plus, DeepSeek-V4-Pro, DeepSeek-V4-Flash, and GLM-4.7 with OpenMLE-Evo.The listed results are 54.55% for Qwen3.7 Plus and DeepSeek-V4-Pro, and 51.52% for DeepSeek-V4-Flash and GLM-4.7.

C. General-purpose coding-agent references

OpenMLE-Evo improves Frontis-MA1 across model scale, search horizons, solution quality, and efficiency, outperforming general-purpose coding-agent references under matched evaluations. Gains also transfer across modalities and to NatureBench Lite, with contributions from both the post-trained model and adapted search framework.

  • Model and harness comparisons: 39.39% to 60.61% Medal Average: Frontis-MA1-35B improves over Qwen3.6-35B-A3B under the identical standard OpenMLE-Evo harness.It also outperforms MiniMax M3, Doubao Seed 2.1 Pro, and DeepSeek-V4-Pro under this common harness.
  • Model and harness comparisons: 71.21%: OpenMLE-Evo-Max raises Frontis-MA1-35B performance after injecting cross-task priors and widening parallel tree search.This exceeds GLM-5.2, MiniMax M3, and GPT-5.5 paired with Codex under the same enhanced harness.
  • Model and harness comparisons: 34.85% to 53.03% Medal Average: the 30B model improves over Qwen3-30B-A3B-Thinking-2507, then reaches 66.67% Medal Average and 0.8053 Human Rank with OpenMLE-Evo-Max.This controlled comparison supports gains beyond the primary 35B checkpoint.
  • Long-horizon search: 71.21% Medal Rate on final test versus 68.18% validation: OpenMLE-Evo-Max sustains improvement throughout long-horizon search.The final-test result is comparable to GPT-5.6 Sol and Kimi K3, which each achieve 72.73%.
  • Search efficiency: −41.7% total model-token consumption and −50.3% prompt tokens: OpenMLE-Evo reduces search cost versus original AIRA-Evo under matched checkpoint, seed, and 12-hour budget.Crossover mean usage falls 60.6%, while its 99th percentile falls 81.3%, consistent with operation-conditioned memory compressing histories.
  • Transfer and generalization: 10 percentage points in All S and 20 points in All M: Frontis-MA1-35B improves over its base on NatureBench Lite with the adapter fixed.With the base model fixed, OpenMLE-Evo improves over original AIRA-Evo by 10 points in All S; modality results raise mean Human Rank across all five groups without lowering group-level Medal Rate.

7. Related Work

Related work spans AutoResearch workflows, executable MLE benchmarks, inference-time evolutionary scaffolds, learning from executable experience, and AI-for-AI systems. OpenMLE-Evo distinguishes itself by explicitly training the same program-evolution operators used during search, rather than relying primarily on frozen-backbone or prompt-supplied transformation behavior.

  • AutoResearch systems and evaluation targets: AutoResearch systems automate hypothesis formation, experimentation, and artifact production through end-to-end workflows, agentic tree search, and compute-bounded training loops.Recent systems also investigate persistent project state, hypothesis-tree refinement, cross-task skill accumulation, and continuously evolving multi-agent workflows.
  • Executable MLE environments and scalable task resources: Executable MLE benchmarks extend classical AutoML by requiring LLM-based agents to write, execute, debug, and submit code-mediated solutions on realistic tasks.MLAgentBench and DSBench are cited as examples of this benchmark lineage.
  • Inference-time scaffolds and evolutionary search: Inference-time scaffolds amplify frozen models by allocating trials, maintaining search state, and selecting or transforming candidate programs with executable evaluators.Program-evolution systems use language models as mutation operators and evaluators as selection signals, while ThetaEvolve and TTT-Discover update behavior from test-time feedback.
  • Inference-time scaffolds and evolutionary search: OpenMLE-Evo explicitly composes operators trained for the same roles used during search, unlike methods that typically obtain transformation behavior from a frozen backbone or its prompts.The cited comparison frames OpenMLE-Evo as addressing a limitation of structured-search approaches.
  • Learning MLE agents from executable experience: A complementary line learns from executable experience through supervised or reinforcement-learning updates, including RLVR advances in reasoning and extensions to long-horizon agent tasks.This direction internalizes verifiable outcomes instead of retaining all improvement logic in an external scaffold.
  • AI-for-AI and trainable improvers: AI-for-AI broadens executable improvement toward models, operators, and harnesses, while search–learning systems return execution experience to generators through policy updates or evolutionary search with hindsight fine-tuning.These approaches connect evolutionary search with subsequent learning updates.

8. Limitations and Future Work

OpenMLE provides an executable path toward recursive self-improvement but does not yet realize the full RSI vision. Future work targets richer objectives, unified evolutionary and agentic search, broader participation in AI development, evolving the evolutionary system, and richer experience-guided expansion.

  • Richer objectives for improving the improver: OpenMLE primarily optimizes executed-solution outcomes, leaving research promise, generalizability, robustness, and computational worth insufficiently captured.The system is therefore better equipped to optimize solutions than to judge which ideas deserve pursuit.
  • Integrating evolutionary search with general coding agents: OpenMLE’s external evolutionary harness limits the range of model-initiated actions; future systems should unify population-based exploration with flexible agentic problem solving.This would combine evolutionary search with general coding agents in one framework.
  • Broader participation in AI development: Agents currently improve external machine learning artifacts and participate in only a limited part of AI development, especially excluding language-model improvement.Moving toward RSI requires agents to take part in a larger share of the AI development process.
  • Evolving the evolutionary system: OpenMLE evolves candidate solutions while leaving the evolutionary system largely fixed; future work should make the evolutionary system itself an object of evolution.This is identified as a further step toward recursive self-improvement.
  • Richer use of experience in node expansion: Experience-guided node expansion remains preliminary because parent selection uses only solution quality, parent-relative improvement, and method-family novelty.The framework shows structured experience can guide search-budget allocation, but richer use of experience remains necessary.

9. Conclusion

OpenMLE is an open full-stack system for training and deploying language-model agents that iteratively improve machine-learning solutions through executable feedback. Its model learning and search components provide complementary gains, including substantial improvement over the Qwen3.6-35B-A3B base.

  • OpenMLE system: OpenMLE supports agents that construct and iteratively improve machine-learning solutions through executable feedback.OpenMLE-Gym provides quality-gated tasks, isolated execution, and task-specific evaluation.
  • OpenMLE system: OpenMLE-ERL learns Draft, Improve, Debug, and Crossover transformations through execution-grounded supervised fine-tuning and reinforcement learning.OpenMLE-Evo composes these operators into long-horizon search using structured experience, multi-factor parent selection, and operator-conditioned mechanisms.
  • Results: 39.39% to 60.61%: Frontis-MA1-35B raises Medal Average over its Qwen3.6-35B-A3B base under the identical OpenMLE-Evo harness.The companion Frontis-MA1-30B reproduces the gain on a second backbone and scale.
  • Results: OpenMLE-Evo outperforms general-purpose coding-agent scaffolds across four frontier models and original AIRA-Evo on Frontis-MA1-35B.These matched comparisons isolate the contribution of the search harness across model settings.

10. Authors … B. OpenMLE-ERL Details

The appendix details OpenMLE’s automated task-construction pipeline and executable feedback infrastructure, while also listing the authors and affiliations. It shows how competition packages are built, validated, quality-gated, and executed in isolated workers that return repair-oriented diagnostics.

  • A.1. Task Construction and Selection: Competition construction uses an ordered state machine, with independent competitions dispatched concurrently and downstream stages gated on prerequisite success.The builder consumes candidate slugs remaining after upstream filtering.
  • A.1. Task Construction and Selection: The pipeline grounds generated task descriptions in recursively unpacked archives, preserved source assets, local file inventories, tabular probes, sampled rows, and textual documentation.This combines competition metadata with tool-assisted inspection rather than relying solely on web metadata.
  • A.1. Task Construction and Selection: Preparation executes prepare.py to create deterministic splits and expose inputs, then requires descriptions, training data, test inputs, sample submissions, and private answers before acceptance.Failed attempts clear partial process state.
  • A.1. Task Construction and Selection: Metric validation dynamically loads generated metric.py and scores public sample submissions against private test answers, failing on import, execution, missing-input, or missing-score errors.Metric generation and validation are separate stages.
  • A.1. Task Construction and Selection: Quality assessment applies hard structural and metric-execution gates before a semantic judge scores validity, data sufficiency, raw-data usage, and task completeness.Packages failing critical-file or metric smoke tests deterministically receive not_recommended.
  • A. OpenMLE-Gym Details: OpenMLE-Gym separates scheduling, isolated workers, shared job state, and returned feedback, executing candidate programs in CPU/GPU Docker environments against task data and evaluators.The execution backend exposes an API boundary between agent requests and structured worker feedback.
  • A.2. OpenMLE-Gym Execution Infrastructure: A failed sandbox job returns machine-readable status, runtime, artifacts, logs, and traceback diagnostics; evaluation is skipped and the score remains null when execution fails.These fields localize failures and provide concrete evidence for the next repair action.

B.1. SFT Data Generation … D. Supplementary Experiments

The supplementary sections detail OpenMLE’s executable data-generation, training, reward, rollout, and evolutionary-search mechanisms. They specify how trajectories are filtered and supervised, how search operators and experience records are structured, and how asynchronous execution and reward-hack detection support training and inference.

  • B.1. SFT Data Generation: Trajectory-step selection retains steps whose strategies, intermediate states, or error repairs are causally inherited by later steps and contribute to the endpoint.Cosmetic edits, blind retries, resource-saving reductions, and failed environment modifications or external-network accesses are discarded.
  • B.1. SFT Data Generation: 26,259 training examples form the released SFT corpus after message normalization, exact deduplication, chat templating, and exclusion of examples exceeding 32,768 tokens.The corpus contains 17,245 full responses and 9,014 trajectory steps.
  • B.1. SFT Data Generation: 74.0% of SFT examples are Draft, while Improve, Crossover, and Debug contribute 6.6%, 2.8%, and 16.5%, respectively.The corresponding counts are 19,436, 1,741, 742, and 4,340 examples.
  • B.2. SFT Training Configuration: OpenMLE uses executable SFT and verified RL configurations for both Frontis-MA1-30B and Frontis-MA1-35B.The supplied passages identify Tables 4 and 5 as summaries of the core SFT and RL settings, respectively.
  • B.4. Asynchronous Rollout: 50.8 minutes is the mean asynchronous rollout step time versus 97.0 minutes synchronously, a 1.91× ratio across 40 matched steps.Per-task exposure remained within ±2 steps of the run median in two representative asynchronous runs.
  • B.5. Reward Normalization and Entropic Advantage: Adaptive reward normalization rescales each task from successful historical programs and the current rollout group, while entropic post-processing supplies group advantages for RL updates.The adaptive view follows the score frontier as training improves and preserves reward differences among current candidates.
  • B.6. Detection and prevention of reward hacking.: Reward hacking can cause low-level plateaus, including randomly shuffling a sample submission; an o3-mini judge assigns detected hacks a reward of -0.5 before sandbox execution.The judge checks generated code before it runs in the sandbox.
  • C. OpenMLE-Evo Inference Details: OpenMLE-Evo uses Draft, Improve, Crossover, and Debug to create, revise, combine, and repair programs, with parent selection based on task-local stored fitness.Inference state is recorded through node-level experience cards and a task-global experience board recomputed from accumulated cards.

D.1. Repeated-Evaluation Statistics on MLE-Bench Lite · D.2. NatureBench Lite Task Composition · E. Simplified comparison of public release surfaces

The paper reports repeated-evaluation variability on MLE-Bench Lite, characterizes a fixed ten-task NatureBench Lite generalization subset, and audits fragmented public release surfaces under explicit artifact criteria.

  • D.1. Repeated-Evaluation Statistics on MLE-Bench Lite: Three evaluation epochs yield mean ± standard deviation for model–harness configurations with repeat-level records, reducing sampling and sandbox sensitivity.The standard deviation measures remaining run-to-run variability.
  • D.1. Repeated-Evaluation Statistics on MLE-Bench Lite: Codex, Claude Code, and Gemini CLI references were evaluated once, so they remain point estimates rather than repeated-evaluation statistics.Entries without ± lack archived repeat-level dispersion, and reported ± values are standard deviations, not confidence intervals.
  • D.2. NatureBench Lite Task Composition: The NatureBench Lite generalization study uses a fixed 10-task subset spanning all six scientific domains and diverse data structures.Covered structures include biological sequences, omics matrices, molecular structures, temporal signals, images, and tabular features.
  • D.2. NatureBench Lite Task Composition: Because NatureBench Lite contains ten tasks, each task changes All S or All M by ten percentage points.The subset favors moderately tractable tasks while retaining domain and data-structure coverage for rapid model–harness comparisons.
  • E. Simplified comparison of public release surfaces: The public release landscape is fragmented, and Table 11 compares representative MLE agents and resources by artifacts required to reproduce a full post-training stack.The comparison covers public accessibility and independent verifiability of release artifacts.
  • E. Simplified comparison of public release surfaces: Scores across audited systems are not strictly comparable because backbones, compute, wall-clock budgets, hardware, external resources, run counts, and aggregation procedures differ.This limitation applies to the representative systems summarized in the release-surface comparison.
  • E. Simplified comparison of public release surfaces: A release receives a tick for Data, Sandbox, Train code, Eval, or Weights only when the corresponding artifact was publicly accessible at audit time.Data requires reconstructible task or training data; Sandbox requires executable program-running or environment-construction code; Train code requires parameter-training entry points or configurations; Eval accepts runnable assets or official per-run grading reports.
Loading 2607.28568v1…