Source-linked AI summary
ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning
Zexi Liu, Yuzhu Cai, Xinyu Zhu, Yujie Zheng, Runkun Chen, Ying Wen, Yanfeng Wang, Weinan E, Siheng Chen
TL;DR
AI4AI agents need to exploit experience from solution exploration while reasoning, because existing approaches often leave these capabilities insufficiently integrated. ML-Master uses adaptive memory to connect parallel exploration with steerable reasoning, achieving a 29.3% average medal rate on MLE-Bench within 12 hours.
Problem
Existing AI4AI methods often fail to fully leverage exploration experience within analytical reasoning, while exploration and reasoning can become inefficient or stagnant when isolated.
Method
ML-Master integrates balanced multi-trajectory exploration and steerable reasoning through an adaptive memory mechanism that selectively captures exploration insights.
Results
29.3% average medal rate on MLE-Bench surpasses existing methods, with particularly strong performance on medium-difficulty tasks, achieved within a 12-hour limit.
Takeaways & Limitations
ML-Master demonstrates potential for advancing AI4AI by combining exploration and reasoning in a unified framework.
Takeaways & Limitations
Further ablation experiments were still ongoing and were to be reported in updated versions.
Abstract
from arXiv · showhide
As AI capabilities advance toward and potentially beyond human-level performance, a natural transition emerges where AI-driven development becomes more efficient than human-centric approaches. A promising pathway toward this transition lies in AI-for-AI (AI4AI), which leverages AI techniques to automate and optimize the design, training, and deployment of AI systems themselves. While LLM-based agents have shown the potential to realize AI4AI, they are often unable to fully leverage the experience accumulated by agents during the exploration of solutions in the reasoning process, leading to inefficiencies and suboptimal performance. To address this limitation, we propose ML-Master, a novel AI4AI agent that seamlessly integrates exploration and reasoning by employing a selectively scoped memory mechanism. This approach allows ML-Master to efficiently combine diverse insights from parallel solution trajectories with analytical reasoning, guiding further exploration without overwhelming the agent with excessive context. We evaluate ML-Master on the MLE-Bench, where it achieves a 29.3% average medal rate, significantly surpassing existing methods, particularly in medium-complexity tasks, while accomplishing this superior performance within a strict 12-hour time constraint-half the 24-hour limit used by previous baselines. These results demonstrate ML-Master's potential as a powerful tool for advancing AI4AI.
1 Introduction
AI4AI aims to automate AI-system development, but existing agents often separate exploration from analytical reasoning. ML-Master unifies them through adaptive memory and achieves stronger MLE-Bench performance under a shorter time limit.
- AI-for-AI (AI4AI) uses AI techniques to automate and optimize the design, training, and deployment of AI systems.
- Existing AI4AI studies often emphasize exploration without sufficiently leveraging advanced reasoning models, limiting adaptability in complex scenarios.
- ML-Master integrates exploration and analytical reasoning through an adaptive memory mechanism that selectively captures and summarizes exploration insights.
- 12 hours—half the earlier 24-hour limit—was sufficient for ML-Master to achieve its reported MLE-Bench performance.
2 Related Work
AutoML and LLM-based systems mark steps toward AI4AI, while self-evolving agents use generated experience to improve reasoning and adaptability. ML-Master is presented against this broader progression.
- Automated machine learning (AutoML): AutoML automates machine-learning engineering tasks such as data preprocessing, model selection, and parameter tuning.Earlier systems relied mainly on heuristic or learning-based approaches and often lacked adaptability or generalization.
- LLMs and multi-agent systems for AI4AI: LLMs extend earlier AutoML systems with complex reasoning, knowledge-based judgment, and code generation for more autonomous AI development.
- Self-evolving AI: Self-evolving AI autonomously acquires, refines, and learns from self-generated or personalized experiences to improve reasoning and adaptability.
- Self-evolving AI: Examples of self-evolving systems include iterative self-verification, self-training from generated outputs, and experience accumulation in autonomous agents.
3 Methodology
ML-Master integrates balanced parallel exploration with steerable reasoning through adaptive memory, creating a bidirectional iterative loop between empirical search outcomes and analytical guidance. Its tree-guided process manages diverse solution trajectories while selectively preserving insights and feedback for subsequent reasoning and exploration.
- 3 Methodology: ML-Master combines balanced multi-trajectory exploration with steerable reasoning through an adaptive memory mechanism.Exploration generates diverse empirical insights and execution feedback, while reasoning interprets these outcomes and guides subsequent search.
- 3.1 Balanced multi-trajectory exploration: The exploration module models AI development as MCTS, representing solution states as nodes and refinement actions as edges.This tree structure manages and prioritizes exploration across a complex solution space.
- 3.1 Balanced multi-trajectory exploration: Selection, expansion, verification, and backpropagation iteratively guide parallel solution trajectories through specialized actions and structured evaluation feedback.Multiple workers explore branches asynchronously, while verification rewards candidate quality and backpropagation updates the search tree.
- 3.1.1 Tree-guided exploration: UCT prioritizes promising yet under-explored nodes by combining accumulated reward, visit information, and an exploration-exploitation constant.Nodes with higher UCT values guide search toward potentially valuable regions of the solution space.
- 3.1.1 Tree-guided exploration: Improvement stagnation and excessive debugging terminate nodes, bounding attempts while preserving adaptive exploration.A node becomes terminal when failed improvements exceed τ_improve or consecutive debugging exceeds τ_debug.
- 3.2 Steerable Reasoning: Steerable reasoning embeds selectively scoped memory from the immediate parent and same-depth sibling nodes into the LLM’s reasoning component.The memory preserves reasoning insights and execution feedback, supporting continuity and alternative contexts without redundant trajectories or information overload.
4 Experiment
On MLE-Bench, ML-Master outperforms baselines across evaluation dimensions and task-complexity levels, while improving solutions over time within a 12-hour budget. Further ablations remain pending.
- Main results: 29.3% of tasks receive a medal, including 17.3% gold medals.Results are averaged over three runs with different random seeds.
- Main results: 93.3% of tasks receive valid submissions, and performance exceeds more than half of human submissions on 44.9% of tasks.
- Main results: ML-Master outperforms all baselines on every MLE-Bench evaluation dimension.The comparison uses Bronze+ and Silver+ thresholds, where plus denotes equal or better than the threshold.
- Complexity analysis: 20.2% medal rate is achieved on medium-complexity tasks, while high-complexity tasks reach 24.4%.Table 3 reports ML-Master as outperforming all baselines at each complexity level.
- Temporal analysis: Performance improves as iteration time increases, measured by the best solution's percentage improvement over its initial version.The figure tracks the best version up to each time point against the initial version.
- Limitations: Further ablation experiments are still being conducted and will be reported in updated versions.
5 Conclusions
The paper presents ML-Master as an AI4AI agent that unifies exploration and reasoning through adaptive memory. On MLE-Bench, it surpasses existing AI4AI methods, including with a 29.3% average medal rate and results achieved in 12 hours.
- ML-Master integrates parallel solution exploration and analytical reasoning through an adaptive memory mechanism.
- ML-Master achieves a 29.3% average medal rate on MLE-Bench and more than doubles the previous best result on medium-difficulty tasks.The medium-difficulty medal rate is described as achieved within 12 hours, half the time typically allocated in previous studies.
- The integration is presented as a step toward AI4AI systems that autonomously evolve, learn, and adapt to increasingly complex challenges.Future work targets scalability and adaptability in dynamic and multi-agent environments.