Source-linked AI summary
Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering
Xinyu Zhu, Yuzhu Cai, Zexi Liu, Bingyang Zheng, Cheng Wang, Rui Ye, Yuzhi Zhang, Linfeng Zhang, Weinan E, Siheng Chen, Yanfeng Wang
TL;DR
Ultra-long-horizon autonomy is a bottleneck for agentic science because LLM-based agents can be overwhelmed by accumulated execution details during delayed-feedback research. The paper introduces ML-Master 2.0 and Hierarchical Cognitive Caching to differentiate transient experience from stable knowledge and reusable wisdom, achieving a 56.44% medal rate on OpenAI’s MLE-Bench under 24-hour budgets.
Problem
Ultra-long-horizon autonomy remains difficult because research involves delayed feedback, high-dimensional exploration, and execution details that must be managed across extended experimental cycles.
Method
ML-Master 2.0 uses Hierarchical Cognitive Caching to dynamically promote, consolidate, or discard context across tiers as experience becomes more stable and reusable.
Results
56.44% medal rate was achieved on OpenAI’s MLE-Bench under a fixed 24-hour budget, with state-of-the-art performance reported across difficulty levels.
Takeaways & Limitations
The results support cognitive accumulation as a framework for sustaining strategic coherence and reusing experience during long-horizon machine learning engineering.
Takeaways & Limitations
Existing context-management research has limited unified policies for when raw interaction traces should be accumulated, promoted, or evicted across execution stages.
Abstract
from arXiv · showhide
The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles spanning days or weeks. While Large Language Models (LLMs) have demonstrated prowess in short-horizon reasoning, they are easily overwhelmed by execution details in the high-dimensional, delayed-feedback environments of real-world research, failing to consolidate sparse feedback into coherent long-term guidance. Here, we present ML-Master 2.0, an autonomous agent that masters ultra-long-horizon machine learning engineering (MLE) which is a representative microcosm of scientific discovery. By reframing context management as a process of cognitive accumulation, our approach introduces Hierarchical Cognitive Caching (HCC), a multi-tiered architecture inspired by computer systems that enables the structural differentiation of experience over time. By dynamically distilling transient execution traces into stable knowledge and cross-task wisdom, HCC allows agents to decouple immediate execution from long-term experimental strategy, effectively overcoming the scaling limits of static context windows. In evaluations on OpenAI's MLE-Bench under 24-hour budgets, ML-Master 2.0 achieves a state-of-the-art medal rate of 56.44%. Our findings demonstrate that ultra-long-horizon autonomy provides a scalable blueprint for AI capable of autonomous exploration beyond human-precedent complexities.
1. Introduction
Agentic science requires ultra-long-horizon autonomy to maintain strategic coherence and iterative correction across delayed-feedback experiments. ML-Master 2.0 addresses this challenge through cognitive accumulation and hierarchical context management, achieving strong MLE-Bench results.
- Ultra-long-horizon autonomy sustains strategic coherence and iterative correction across high-dimensional research cycles spanning days or weeks.
- AI-for-AI uses computational machine learning engineering tasks as a setting for prolonged trial-and-error and experience accumulation.MLE-Bench comprises 75 real-world Kaggle competitions.
- ML-Master 2.0 models autonomy as cognitive accumulation, distilling raw experience into reusable knowledge and cross-task wisdom.Short-term experience supports immediate decisions, stable knowledge preserves strategic consistency, and abstracted wisdom enables transfer.
- Hierarchical Cognitive Caching separates transient processing from stable, reusable state through a multi-level context architecture.The design is inspired by multi-level computer-system caches and is intended to migrate stable knowledge into persistent representations.
- 56.44% overall medal rate was achieved on OpenAI’s MLE-Bench under a fixed 24-hour budget, a 92.7% relative improvement over ML-Master.Performance increased from 48.48% to 75.76% on low-complexity tasks, from 20.18% to 50.88% on medium-complexity tasks, and from 24.44% to 42.22% on high-complexity tasks.
- The paper presents cognitive accumulation, HCC, and state-of-the-art MLE-Bench performance as its main contributions.The contribution summary characterizes HCC as dynamically coordinating promotion and consolidation to decouple execution feedback from strategic planning.
2. Related Work
Prior work studies hierarchical context organization, experiential abstraction, iterative MLE workflows, and search strategies, but these dimensions remain insufficiently unified. The paper identifies missing policies for differentiating and governing experience across long-horizon execution.
- 2.1. Context Management: Hierarchical memory systems organize context into multiple layers and retrieve abstractions while preserving access to lower-level details.Examples include MemGPT, HiAgent, G-Memory, and HippoRAG.
- 2.1. Context Management: Existing context methods largely emphasize storage and retrieval, while memory promotion or summarization is often heuristic.
- 2.1. Context Management: Experience-driven methods transform trajectories into reusable feedback, cases, strategy items, templates, or workflows.Examples include Reflexion, Memento, ReasoningBank, Buffer of Thoughts, and AWM.
- 2.1. Context Management: Existing work leaves limited investigation of unified control over working context, accumulated experience, and abstracted memory.The paper specifically identifies missing policies for when raw traces should be accumulated, promoted, or evicted.
- 2.2. Autonomous Machine Learning: MLE agents use execution feedback loops, iterative refinement, planning phases, and search or evolutionary strategies for extended tasks.Representative systems include MLAgentBench, OpenHands, AIDE, R&D-Agent, AIRA, AutoMLGen, and FM Agent.
- 2.2. Autonomous Machine Learning: Search and evolutionary approaches enable information transfer but generally treat knowledge as homogeneous, limiting cognitive differentiation during prolonged discovery.
- 2.2. Autonomous Machine Learning: HCC differs by separating processing from state and dynamically governing context promotion and consolidation to support long-term exploration without saturation.
3.1. Problem Formulation
The problem formulation represents agent–environment interaction as an alternating event sequence and defines context construction, policy actions, environment feedback, and final solution extraction. Because trial-and-error rapidly expands the history, context management must preserve immediate traces while distilling reusable long-horizon guidance.
- The agent–environment interaction is modeled as a chronological discrete event sequence containing environment-originated and agent-originated events.
- At each agent step, a context function maps interaction history to model input, the policy produces an action, and the environment returns feedback.
- For task τ_n, interaction continues until t_max, after which an extraction function obtains final solution code I* optimized by task metric F(·).
- Hierarchical research planning partitions interaction history into contiguous exploration phases bounded by successive plan time steps.
- Naively concatenating recent events causes context saturation, degrading strategic coherence and preventing reusable expertise accumulation over tens of hours.The formulation motivates preserving high-fidelity short-term traces while distilling stable knowledge and reusable wisdom.
3.2. Agent Overview
ML-Master 2.0 is an autonomous agent for long-horizon machine learning tasks that manages evolving cognitive state through Hierarchical Cognitive Caching. HCC separates transient experience, stable knowledge, and reusable wisdom to sustain coherent exploration.
- ML-Master 2.0 targets long-horizon machine learning tasks through an agent-centric loop that explicitly manages and evolves cognitive state.
- HCC combines hierarchical caching, which organizes context by temporal stability and reuse value, with context migration across cache tiers.
- The agent retrieves prior wisdom, constructs an initial context, generates initial code, proposes a hierarchical research plan, and executes suggestions in parallel.
- HCC structurally separates transient experience from stable knowledge and reusable wisdom, enabling coherent long-horizon exploration without overwhelming execution details.
3.3. Hierarchical Caching
Hierarchical caching uses three levels—Evolving Experience, Refined Knowledge, and Prior Wisdom—to separate immediate execution traces from increasingly stable and transferable cognitive state.
- Hierarchical Caching: The three-level cache maps L1, L2, and L3 to Evolving Experience, Refined Knowledge, and Prior Wisdom.
- Evolving Experience: Evolving Experience retains high-fidelity plans, code patches, terminal outputs, and metric logs needed for immediate debugging and execution-aware decisions.
- Evolving Experience: L1 retains raw traces for active execution while limiting older retained material to prevent context saturation during long-horizon exploration.
- Refined Knowledge: Refined Knowledge stores compact summaries of validated judgments, experimental insights, and progress rationales as mid-term strategic memory.
- Prior Wisdom: Prior Wisdom stores task-agnostic strategies such as model templates, preprocessing pipelines, and hyperparameter priors for cross-task transfer.
3.4. Context Migration
Context migration governs how ML-Master 2.0 retrieves, compresses, promotes, and removes information across its cognitive caches. It begins with prior-wisdom prefetching, uses cache-hit retrieval during execution, and promotes completed exploration into reusable knowledge and wisdom.
- Context Migration: Context migration comprises context prefetching, context hit, and context promotion within the hierarchical storage.
- Initialization via Context Prefetching: Prefetching embeds the current task descriptor and retrieves similar prior wisdom to construct the initial agent context before exploration.
- Retrieval via Context Hit: The context constructor retrieves available raw events from L1 and otherwise falls back to compact summaries in L2.
- Retrieval via Context Hit: This retrieval policy preserves raw plan and active-phase traces while representing completed phases with refined units, preventing context saturation without losing strategic continuity.
- Consolidation via Context Promotion: Phase-level promotion compresses parallel exploration trajectories into refined knowledge, writes the summary to L2, and removes the corresponding raw trajectories from L1.
- Consolidation via Context Promotion: Task-level promotion embeds completed-task wisdom and stores it in L3, crystallizing transient execution context into retrieval-ready cross-task knowledge.
4. Experiment
Experiments on MLE-Bench and MLE-Bench-Lite evaluate ML-Master 2.0 under fixed computational settings, benchmark comparisons, HCC ablations, context control, and improvement over iterations. The results report strong performance across task complexities and support the contributions of hierarchical cognitive caching and context migration.
- 4.1. Experiment Setup: MLE-Bench comprises 75 real-world Kaggle tasks, and experiments use a fixed 24-hour task budget with reported baseline results.The setup uses 36 AMD EPYC vCPUs and two NVIDIA GeForce RTX 4090 GPUs per agent.
- 4.2. Main Results: 75.8% in Low, 50.9% in Medium, and 42.2% in High complexity tasks give ML-Master 2.0 the highest medal rate across all difficulty levels.The reported pattern contrasts with agents that experience significant performance drops in complex scenarios.
- 4.2. Main Results: 95.6% average valid submission rate and outperforming 50% of human participants in 63.1% of tasks indicate a high performance floor and robustness.The valid submission rate is reported as comparable to other leading approaches.
- 4.3. Ablation Study and Analysis: Removing Evolving Experience reduces the valid submission rate to 54.5% and the medal rate to 22.7% in the MLE-Bench-Lite ablation.The ablation evaluates Experience, Knowledge, and Wisdom by excluding each hierarchy component separately.
- 4.3. Ablation Study and Analysis: HCC reduces peak context length from more than 200k to approximately 70k tokens while retaining insights from failed attempts and producing a medal by the fourth iteration.The comparison is between full context length and context length in HCC for random-actsof-pizza.
- 4.3. Ablation Study and Analysis: As iteration time increases, ML-Master 2.0 produces increasingly better solutions, while Refined Knowledge and Prior Wisdom support top-tier performance and reduce ineffective exploration.The reported ablations associate Refined Knowledge with complex-solution synthesis and Prior Wisdom with initialization.
5. Conclusions
The paper presents ML-Master 2.0 and its Hierarchical Cognitive Caching architecture for ultra-long-horizon autonomy in agentic science. On OpenAI’s MLE-Bench, it reports a 56.44% state-of-the-art medal rate and concludes that evolving context supports extended experimental workflows.
- 5. Conclusions: ML-Master 2.0 reframes context management as cognitive accumulation, distilling transient experiences into refined knowledge and reusable prior wisdom.This structural differentiation is intended to sustain strategic coherence without overwhelming the agent with interaction details.
- 5. Conclusions: 56.44% medal rate on OpenAI’s MLE-Bench is reported as state-of-the-art and significantly above existing open-source and closed-source baselines.The conclusion presents evolving context as essential for high-dimensional, delayed-feedback scientific research environments.
- 5. Conclusions: The paper positions ML-Master 2.0 as a scalable blueprint for autonomous agents orchestrating the full lifecycle of scientific discovery.This is the paper’s stated broader implication for agentic science.
A. Prompts for ML-Master 2.0
The appendix specifies prompts that guide ML-Master 2.0 through task summarization, coding, debugging, research planning, and strategic reflection. These prompts enforce concise formats, executable implementations, evaluation, submissions, and iterative use of prior solutions and creative ideas.
- A.1. Context Prefetching: Task summarization prompts require a single dense paragraph below 250 tokens covering task, data, outputs, evaluation, submissions, and constraints.They prohibit unsupported information, headings, lists, markdown, and explanatory analysis.
- A.2. Coding prompts: Initial coding prompts require implementation code that reads from ./input and writes predictions to submission.csv in the required output directory.They also recommend increasing DataLoader num_workers and permit installed machine-learning packages.
- A.2. Coding prompts: The coding environment provides installed packages including pandas, PyTorch, scikit-learn, transformers, XGBoost, LightGBM, NumPy, and torchvision.The prompt recommends PyTorch rather than TensorFlow for neural networks and states that packages need not be installed.
- A.2. Coding prompts: Implementation guidelines require printing a hold-out validation metric and saving test predictions as ./submission/submission.csv in a self-contained executable Python program.The submission file is explicitly required for grading and evaluation.
- A.2. Coding prompts: Research-plan prompts require a concise solution sketch, a reasonable evaluation metric, prepared data from ./input, and no EDA or unzipping.They also incorporate previous memory and solution context for subsequent planning.
- A.2. Coding prompts: Creative-idea prompts require implementing an improvement on top of the previous solution and reporting whether the plan worked, its performance, promising directions, and dead ends.The output combines execution summary with strategic insights and future direction.