PaperlayerView exact revision on arXiv

Source-linked AI summary

MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Shangheng Du, Jinxin Shi, Shuaiyu Zhang, Yiming Wu, Junpeng Zhao, Shiyang Feng, Xin Li, Liang He, Xiangchao Yan, Bo Zhang

arXiv:2610.22344v1cs.AIcs.CLcs.LG

TL;DR

Long-horizon MLE training must select useful supervision from heterogeneous interaction trajectories and learn from experiment feedback that reflects several prior decisions. MLExpert combines a task factory and model-directed harness with hierarchical SFT and WIRL. It improves the base model’s MLE-Bench Lite medal rate and shows gains on other optimization tasks.

  • Problem

    Long-horizon MLE training must select useful supervision from heterogeneous trajectories and learn from intermediate experiment feedback that reflects multiple preceding decisions.

    Source: §1 · p. 2

  • Method

    MLExpert combines a model-directed interaction harness and executable Kaggle tasks with outcome-guided hierarchical SFT and WIRL, which rewards decisions between successive experiment results.

    Source: §1 · p. 2, §4.1 · p. 5

  • Results

    MLExpert improves the base model’s average medal rate from 37.9% to 72.7% on MLE-Bench Lite and also improves NatureBench Lite and scaffold and algorithm optimization performance.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results support treating long-horizon optimization decision making as a trainable model capability that generalizes across the reported tasks.

    Source: §7 · p. 11

Abstract

from arXiv · show

Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories and learn from intermediate outcomes. We propose a post-training method for autonomous long-horizon MLE optimization and apply it to train MLExpert, a 35B Mixture-of-Experts agentic model. We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation. Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE through outcome-guided supervision. Windowed Interaction RL (WIRL) treats the multi-turn decisions between successive experiment results as an execution window and assigns it a shared outcome-based reward. Asynchronous collection and policy correction support policy updates before the trajectory ends. On MLE-Bench Lite, MLExpert improves the average medal rate of the base model from 37.9% to 72.7%, achieving performance comparable to larger frontier models such as Kimi-K3 and GLM-5.3. Further experiments show that MLExpert improves performance on NatureBench Lite, agent scaffold optimization, and algorithm optimization, suggesting that the learned optimization capability generalizes to other tasks.

1. Introduction

Long-horizon MLE requires models to organize experiments and learn from feedback that reflects multiple preceding decisions. MLExpert combines model-directed interaction, outcome-guided SFT, and WIRL, improving benchmark performance and transferring to other optimization tasks.

1. Introduction

  • Existing systems often rely on predefined workflows, while long-horizon training must address heterogeneous tasks, noisy trajectories, intermediate feedback, and asynchronous policy mismatch.The paper frames model-directed experiment organization and sustained optimization as its focus.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • 37.9% to 72.7%: MLExpert raises its base model’s average medal rate on MLE-Bench Lite and performs comparably to larger models.The paper also reports improvements on NatureBench Lite and generalization to agent scaffold and algorithm optimization.Source: §1 · pp. 2–3
  • MLExpert is a 35B MoE agentic model trained to organize experiments and revise solutions based on execution feedback.The method combines a model-directed interaction harness, hierarchical agentic SFT, and Windowed Interaction Reinforcement Learning.Source: §1 · p. 2
  • The Kaggle-MLE Factory supplies executable tasks, while outcome-guided supervision selects useful trajectories and segments for SFT.WIRL assigns decisions between experiment results a shared outcome-based reward and supports updates during ongoing trajectories.Source: §1 · pp. 2–3

2. Preliminary

The paper formulates MLE as finding a high-scoring executable solution under a task-specific metric and time budget. In agentic MLE, a policy iteratively constructs and improves solutions through tool interaction.

2. Preliminary

  • An MLE task comprises a task description, data, and evaluation metric; its objective is to find the highest-scoring candidate solution within the task setting.A candidate is a complete ML pipeline, and the score is normalized so higher values indicate better performance.Source: §2 · p. 3
  • Agentic MLE represents the model as a policy that constructs, executes, and improves candidate solutions through tool interaction over a trajectory.Actions can include reasoning and tool calls, conditioned on task information and retained interaction history.Source: §2 · p. 3
  • The model uses experimental states to decide whether to analyze, revise, execute, or change direction, then selects a final solution from the interaction.Experimental states include code, execution records, and candidate solutions; final evaluation uses hidden test labels.Source: §2 · p. 3

3. Task Construction and Long-Horizon Interaction

The paper pairs an executable-task factory with a model-directed interaction harness to support long-horizon experimentation, trajectory collection, and MLExpert post-training.

3. Task Construction and Long-Horizon Interaction

  • The Kaggle-MLE Factory and Interaction Harness together provide executable tasks and model-directed interaction for hierarchical agentic SFT and WIRL.The harness manages experimental state and supports trajectory collection.Source: §3 · p. 3, Figure 2 · p. 4

3.1. Kaggle-MLE Factory: Constructing Executable MLE Tasks

  • The Kaggle-MLE Factory converts historical competitions into standardized executable tasks with public workspaces, hidden test splits, and unified evaluation.A multi-agent pipeline discovers and analyzes competitions, generates task data and evaluators, validates and repairs tasks, and excludes unresolved label or scoring issues after manual review.Source: §3.1 · pp. 3–4

3.2. Interaction Harness for Long-Horizon MLE Optimization

  • The Interaction Harness leaves experiment branching and optimization-path decisions to the model, using a revisitable tree of solution nodes and tools for analysis, revision, execution, history, and submissions.Nodes store candidate code, execution logs, and metadata; parent-linked revisions support continuation, backtracking, or changing direction.Source: §3.2 · p. 4
  • As histories grow, the harness summarizes earlier messages and can delegate analysis and diagnosis to a sub-agent; evidence can also be used to invalidate unreliable candidates.The paper specifically notes that overfitting or data leakage can inflate validation scores.Source: §3.2 · pp. 4–5

4. Agentic MLE Post-Training

The post-training method combines hierarchical agentic SFT with WIRL, which uses experiment outcomes to supervise decisions made between successive results and corrects for asynchronous collection.

4.1. Hierarchical Agentic SFT

  • Hierarchical agentic SFT first teaches general tool-use and code revision, then uses MLE and scientific-task trajectories to train algorithm design, analysis, and iterative optimization.The demonstrations include initial-solution, long-horizon optimization, and analysis-sub-agent trajectories.Source: §4.1 · p. 5
  • Outcome-guided selection retains useful trajectory prefixes, anchoring them at the first best submission when later degradation exceeds a threshold, and otherwise at the last submission.This removes degraded tails while preserving useful exploration and failure recovery before the peak.Source: §4.1 · p. 5
  • SFT samples preserve interaction context and train only on newly generated model outputs, masking tool feedback and inherited history from the loss.When context is compacted, retained history and its summary condition subsequent samples.Source: §4.1 · p. 5

4.2. Windowed Interaction Reinforcement Learning

  • WIRL groups decisions between successive experiment results into execution windows and assigns each window the corresponding experiment outcome for policy updates before the full trajectory ends.Task-specific medal thresholds provide a common quality reference, and a local trend term captures improvement or decline relative to the previous valid submission.Source: §4.2 · pp. 5–6
  • Window rewards penalize execution errors and unscorable submissions, reward valid submissions and medals, and adjust for local performance trend.WIRL uses the verifiable execution reward as a shared optimization weight for the main-model output tokens in a window, without group-relative normalization.Source: §4.2 · p. 6
  • Asynchronous collection permits windows from ongoing trajectories to enter training batches, while token-level policy correction weights in-range probability ratios and excludes out-of-range tokens.The correction addresses calls generated by rollout policies that may differ from the current training policy; loss is normalized by valid output tokens per window.Source: §4.2 · p. 6

5. Experiments

Across MLE-Bench Lite, NatureBench Lite, and other optimization tasks, MLExpert improves on its base model. Ablations, a case study, and cross-harness results examine the gains and their scope.

5.1. Experimental Setup

  • The primary benchmark is MLE-Bench Lite, a 22-task subset of historical Kaggle competitions; Any Medal is the primary metric, alongside medal tiers and Med+.Evaluations also include a ten-task NatureBench Lite subset and same-scale and larger frontier model baselines.Source: §5.1 · pp. 6–7

5.2. Main Results

  • MLExpert raises its base model's medal rate from 37.9% to 72.7%, highest among evaluated same-scale models and comparable to GLM-5.3 and Kimi-K3.Source: Table 1 · p. 7, §5.2 · pp. 7–8
  • With Claude Code, MLExpert's medal rate is 69.7% versus 42.4% for its base and exceeds the strongest evaluated same-scale baseline by 22.7 percentage points.Source: §5.2 · p. 8
  • On NatureBench Lite, Surpass-SOTA rises from 0.0% to 50.0%, while Match-SOTA rises from 40.0% to 70.0%.Source: §5.2 · p. 8, Table 2 · p. 8

5.3. Ablation Studies

  • Full WIRL reaches 72.7%, versus 56.1% for GRPO and 60.6% for PPO-style optimization; its variants reach 66.7% and 65.2%.The comparisons support the value of WIRL's local trend reward and policy correction.Source: Table 4 · p. 8, §5.3 · p. 9
  • 72.7% Any Medal after WIRL follows 47.0% after General Agent SFT and 62.1% after MLE-domain SFT, with gains at each stage.Source: Table 3 · p. 8, §5.3 · pp. 8–9

5.4. Case Study: Long-Horizon Optimization

  • On MLSP 2013 Birds, MLExpert improves ROC-AUC from 0.58 to 0.964, while MLExpert-SFT improves from 0.60 to 0.88.The case traces sustained experimentation from audio features and LightGBM through PANNs representations, AST fine-tuning, schedule adjustments, and snapshot averaging.Source: Figure 3 · p. 9, §5.4 · p. 9

5.5. Generalization to Other Optimization Tasks

  • On held-out AIME scaffold optimization, MLExpert reaches 43.7% accuracy versus 23.3% for Base; on 12 MLS-Bench-Lite tasks, mean verifier reward rises from 0.322 to 0.425.MLExpert outperforms Base on ten algorithm-component tasks, ties on one, and scores lower on one.Source: §5.5 · pp. 9–10, Table 5 · p. 10, Figure 4 · p. 10

6. Related work

Automated MLE research spans iterative refinement, structured long-horizon search, and collaborative or cross-domain systems. Recent training work addresses agent interaction capabilities, while MLExpert focuses on model-directed optimization over real-world MLE tasks.

6.1. Agents for Automated ML Research

  • MLE research systems use iterative refinement, structured search, collaboration, and persistent research states to organize automated experimentation.Examples include AIDE [1], MLE-STAR [2], MLEvolve [28], MARS [29], and Arbor [4] and Argus [5].Source: §6.1 · p. 10
  • Several training approaches target individual interaction steps or workflow components, whereas MLExpert trains model-directed long-horizon policies on real-world MLE tasks.WIRL uses execution windows to learn multi-turn decisions from delayed feedback under asynchronous updates.Source: §6.1 · p. 11

6.2. Post-Training Long-Horizon Agentic Models

  • Long-horizon agent training includes executable-environment learning, outcome-based intermediate decisions, multi-teacher distillation, and joint scaffold optimization.MLExpert’s approach joins this work by training for sustained MLE interaction.Source: §6.2 · p. 11, §6.1 · p. 11

7. Conclusion

The paper presents MLExpert and its training and task-construction infrastructure as a path toward autonomous long-horizon MLE. The Kaggle-MLE Factory screens and validates competition tasks, retaining 154 after human review.

7. Conclusion

  • MLExpert is a 35B MoE agentic model trained for long-horizon machine-learning experiments, and its results support treating optimization decision making as a trainable capability.The paper reports improvements over its base model on MLE-Bench Lite, comparable performance to larger frontier models, and generalization to NatureBench Lite, agent scaffold optimization, and algorithm optimization.Source: §7 · p. 11

A. Details of the Kaggle-MLE Factory

  • The Kaggle-MLE Factory begins from 11,649 Meta-Kaggle competition entries and screens candidates for relevance, data availability, evaluation feasibility, and task diversity.Default screening includes completed competitions, at least 50 participating teams, and datasets no larger than 200 GB.Source: A. Details of the Kaggle-MLE Factory · p. 15
  • After automated construction and validation, human review retained 154 of 160 registered tasks, excluding six with unresolved label or scoring issues.The pipeline checks task structure, submission consistency, labels, identifier overlap, data availability, and scoring validity, then returns errors for repair.Source: A. Details of the Kaggle-MLE Factory · p. 15

B. Interaction Harness Tools

The Interaction Harness provides tools for inspecting task environments, revising solutions, running experiments, retrieving prior states, and managing submissions.

B. Interaction Harness Tools

  • The harness supports environment inspection, candidate revision, experiment execution, prior-state retrieval, and submission management, with actions chosen from execution feedback.Table 6 summarizes the available tools and their purposes.Source: B. Interaction Harness Tools · p. 15, Table 6 · p. 15

C. Agentic MLE Post-Training Details

The appendix details hierarchical SFT data and supervision selection, WIRL’s window-level objective and implementation safeguards, and the training settings used for both stages.

C.1. Details of Hierarchical Agentic SFT

  • MLE-domain SFT draws on teacher trajectories from 154 Kaggle-MLE Factory tasks and 134 MLE-Dojo tasks, alongside scientific experimentation trajectories derived from 164 Nature-family papers.The retained corpus contains 636 trajectories: 412 MLE trajectories across 236 tasks and 224 scientific trajectories across 77 tasks.Source: §C.1 · pp. 15–16, Table 7 · p. 16
  • Outcome-guided SFT filters MLE trajectories using a 0.4 quality threshold and truncates degraded runs at the first best submission or retains the last submission otherwise.For a degraded run, the retained prefix includes the anchor update and response plus a randomly selected extension of up to ten complete tool interactions.Source: §C.1 · pp. 16–17
  • SFT samples are formed from trajectories, and the loss covers newly generated reasoning, responses, and tool calls while masking tool feedback and inherited history.The packed training dataset contains approximately 302.45 million tokens before repetition across eight epochs.Source: §C.1 · pp. 16–17

C.2. Details of Windowed Interaction Reinforcement Learning

  • WIRL defines a local objective over policy-generated interactions within execution windows, assigning each window the reward from its corresponding execution result.The objective is evaluated over a fixed distribution of window-start histories.Source: §C.2 · p. 17
  • WIRL’s local trend reward compares each valid submission with the previous one, while asynchronous collection uses token-level policy correction for calls generated under earlier policies.Updates exclude windows without trusted execution feedback; loss masking covers newly generated main-model tokens and handles empty windows without division by zero.Source: §C.2 · pp. 17–18

C.3. Hyperparameters

  • Both SFT stages and WIRL use cosine learning-rate decay, and WIRL uses no learning-rate warmup.The listed configuration corresponds to the medal_absolute_v1 WIRL reward formulation.Source: §C.3 · p. 18

D.1. MLE-Bench Lite Evaluation Details

MLE-Bench Lite results show higher medal rates after post-training, with lower learner waiting overhead during RL training. On NatureBench Lite, MLExpert improves both reference-match and reference-surpass rates over the base model.

D.1. MLE-Bench Lite Evaluation Details

  • Any Medal rises from 37.88% for the base model to 72.73% for MLExpert, while Gold rises from 22.73% to 50.00%.Under Claude Code, Any Medal is 69.7% for MLExpert versus 42.4% for the base model.Source: §D.1 · p. 18, Table 9 · p. 19
  • Any Medal rates increase from 37.88% for Base to 46.97% for General Agent SFT, 62.12% for MLE SFT, and 72.73% for WIRL.Rates are averaged over three evaluations; Table 10 expands the RL comparisons.Source: §D.1 · p. 18, Table 10 · p. 20
  • WIRL's mean learner waiting fraction is 39.3%, versus 87.6% for GRPO, a difference of 48.3 percentage points.The waiting fraction is time waiting for training data divided by waiting plus training time.Source: §D.1 · p. 20, Figure 5 · p. 20

D.2. NatureBench Lite

  • Under Claude Code, MLExpert improves Surpass-SOTA from 0.0% to 50.0% and Match-SOTA from 40.0% to 70.0% over the base model.Against Nex-N2.5-mini, MLExpert has a higher Surpass-SOTA rate (50.0% versus 20.0%) but lower Match-SOTA (70.0% versus 80.0%).Source: §D.2 · pp. 20–21

D.3. AIME Agent Scaffold Optimization

The scaffold-optimization evaluation tests whether optimized Python agent workflows transfer to held-out AIME years. WIRL produces the most accurate MLExpert scaffold, while SFT yields a lower-call alternative.

D.3. AIME Agent Scaffold Optimization

  • Optimizers develop scaffolds on AIME 2025, then evaluate frozen candidates on AIME 2023, 2024, and 2026 without held-out feedback.Each held-out year has 30 problems and three evaluation repetitions; accuracy requires an exact match to the reference integer answer.Source: §D.3 · p. 21
  • MLExpert-SFT achieves 38.1% held-out accuracy with 3.51 solver calls per problem, versus Kimi-K3's 40.0% with approximately 59% fewer calls for MLExpert-SFT.MLExpert-SFT also exceeds Majority@5, which achieves 24.8% with five calls per problem.Source: §D.3 · p. 21
  • WIRL raises held-out accuracy from 38.1% to 43.7%, a gain of 5.6 percentage points, with improvements across all three held-out years.The resulting scaffold uses 8.40 calls per problem, compared with 3.51 for MLExpert-SFT and 8.65 for Kimi-K3.Source: §D.3 · p. 21

D.4. MLS-Bench-Lite

Across 12 selected MLS-Bench-Lite tasks, MLExpert has a higher mean normalized reward than Base, though gains vary by task and include losses or ties.

D.4. MLS-Bench-Lite

  • The evaluation covers 12 selected tasks spanning 12 domains, using the same Agentic MLE harness, task descriptions, tools, and official task environments.Each task has a five-hour search budget and a maximum of 2,000 interaction steps.Source: §D.4 · pp. 21–22
  • Mean normalized reward is 0.425198 for MLExpert and 0.321543 for Base; MLExpert scores higher on ten tasks, ties on one, and scores lower on one.Scores come from the official Harbor verifier across 12 tasks.Source: §D.4 · p. 22, Table 12 · p. 23
  • The largest absolute reward gains occur in causal discovery, language-model demasking, and multi-objective optimization, while time-series forecasting is lower for MLExpert.Both models receive zero reward on Gaussian splatting densification; other tasks show smaller improvements.Source: §D.4 · p. 22

Related papers

Loading 2610.22344v1…