PaperlayerView exact revision on arXiv

Source-linked AI summary

ASI-Evolve: AI Accelerates AI

Weixian Xu, Tiantian Mi, Yixiu Liu, Yang Nan, Zhimeng Zhou, Lyumanshan Ye, Lin Zhang, Yu Qiao, Pengfei Liu

arXiv:2603.29640v1cs.AI

TL;DR

AI research involves costly, long-horizon loops with broad hypothesis spaces and difficult-to-interpret feedback, leaving it unclear whether agents can accelerate AI development itself. ASI-EVOLVE closes this loop with cognition-guided evolution and analysis-driven iteration, reporting discoveries across architectures, data, and learning algorithms and initial transfer beyond AI tasks.

  • Problem

    AI research cycles are limited by costly experiments, broad search spaces, weakly supervised feedback, and human bottlenecks in exploration and knowledge transfer.

  • Method

    ASI-EVOLVE uses a learn–design–experiment–analyze loop with a cognition base, persistent experience database, candidate-program evolution, and a dedicated analyzer.

  • Results

    ASI-EVOLVE discovers and improves components across model architectures, pretraining data, and reinforcement-learning algorithms, with initial evidence of transfer to mathematics and biomedicine.

  • Takeaways & Limitations

    The results provide early evidence that closed-loop AI-for-AI research can support sustained improvement across foundational stages of AI development.

  • Takeaways & Limitations

    The paper identifies infrastructure as part of the broader AI development stack yet to be explored.

Abstract

from arXiv · show

Can AI accelerate the development of AI itself? While recent agentic systems have shown strong performance on well-scoped tasks with rapid feedback, it remains unclear whether they can tackle the costly, long-horizon, and weakly supervised research loops that drive real AI progress. We present ASI-Evolve, an agentic framework for AI-for-AI research that closes this loop through a learn-design-experiment-analyze cycle. ASI-Evolve augments standard evolutionary agents with two key components: a cognition base that injects accumulated human priors into each round of exploration, and a dedicated analyzer that distills complex experimental outcomes into reusable insights for future iterations. To our knowledge, ASI-Evolve is the first unified framework to demonstrate AI-driven discovery across three central components of AI development: data, architectures, and learning algorithms. In neural architecture design, it discovered 105 SOTA linear attention architectures, with the best discovered model surpassing DeltaNet by +0.97 points, nearly 3x the gain of recent human-designed improvements. In pretraining data curation, the evolved pipeline improves average benchmark performance by +3.96 points, with gains exceeding 18 points on MMLU. In reinforcement learning algorithm design, discovered algorithms outperform GRPO by up to +12.5 points on AMC32, +11.67 points on AIME24, and +5.04 points on OlympiadBench. We further provide initial evidence that this AI-for-AI paradigm can transfer beyond the AI stack through experiments in mathematics and biomedicine. Together, these results suggest that ASI-Evolve represents a promising step toward enabling AI to accelerate AI across the foundational stages of development, offering early evidence for the feasibility of closed-loop AI research.

1. Introduction

ASI-EVOLVE targets human bottlenecks in AI research by closing the loop from prior knowledge and hypothesis generation through experimentation, analysis, and refinement. Across architecture, data, and learning-algorithm design, it reports improvements and initial evidence of broader scientific transfer.

  • AI research is constrained by limited parallel hypothesis exploration, labor-intensive experiments, and poor preservation of insights across iterations.
  • ASI-EVOLVE combines a cognition base of accumulated human knowledge with an analyzer that converts complex experimental outcomes into reusable insights.
  • 105 discovered neural architectures surpassed DeltaNet, with the best model achieving a +0.97-point gain.
  • 3.96 points of average benchmark improvement came from evolved pretraining-data strategies, including gains exceeding 18 points on MMLU.
  • Up to +12.5 points on AMC32, +11.67 points on AIME24, and +5.04 points on OlympiadBench separated discovered reinforcement-learning algorithms from GRPO.
  • Experiments in circle packing and drug-target interaction prediction provide initial evidence that the framework can transfer beyond AI development tasks.

2. Preliminary

The paper frames scientific automation by task length across execution cost, search-space complexity, and feedback difficulty. Existing systems span question answering, structured execution, and lightweight discovery, while large-scale AI research occupies a more demanding region.

  • Scientific Task Length is characterized as Ltask = ⟨Cexec, Sspace, Dfeedback⟩.The dimensions capture execution cost, search-space complexity, and feedback difficulty.
  • Scientific question answering: Scientific question answering has virtually zero Cexec because it requires little or no experimental execution.
  • Structured task execution: Structured task-execution systems run experiments under clearly defined objectives but target predefined tasks.
  • Lightweight scientific discovery: Lightweight discovery frameworks demonstrate open-ended algorithmic discovery across mathematical, optimization, and systems problems.

3. ASI-Evolve

ASI-EVOLVE is an end-to-end evolutionary research pipeline for costly, open-ended AI tasks. It combines human priors, persistent experimental memory, candidate generation, evaluation, and analysis to support iterative improvement.

  • Each round samples database context, retrieves cognition items, generates a candidate, evaluates it under timeouts, and stores an analysis report for future rounds.
  • Large-scale AI research requires complete training runs, broad exploration, and interpretation of multiple benchmarks, loss dynamics, and efficiency metrics.
  • Cognition Base: The cognition base injects task-relevant heuristics, known pitfalls, and design principles to reduce blind exploration and repeated failure modes.
  • Researcher: The Researcher generates complete programs or localized edits from task descriptions, sampled nodes, and retrieved cognition items.
  • Engineer: The Engineer runs experiment-specific evaluations that return structured metrics and a primary scalar fitness score, with optional early rejection and judging.
  • Analyzer: The Analyzer receives full logs and metrics, performs causal analysis, and distills them into compact reports that preserve analytical depth for future retrieval.
  • Database: The database stores past motivations, programs, results, analyses, scores, and metadata, then supplies historical context for later rounds.
  • Database: Sampling policies including UCB1, random, greedy, and MAP-Elites produce distinct evolution trajectories and affect sustained improvement.

4 Main Tasks

ASI-EVOLVE applies closed-loop evolutionary research to architecture design, data curation, and reinforcement-learning algorithm design. Across these tasks, it combines domain priors, iterative experimentation, and analyzed feedback to discover improved designs.

  • Scope: ASI-EVOLVE evaluates three demanding AI research tasks: linear-attention architecture design, pretraining data curation, and reinforcement-learning algorithm design.These tasks involve large search spaces, costly experiments, implementation complexity, and limited task-specific prior knowledge.
  • Model Architecture Design: Architecture search produced 105 architectures surpassing DeltaNet across 1,773 exploration rounds.The search used a cognition repository of approximately 150 literature-derived entries and multi-stage evaluation from small-scale exploration to large-scale validation.
  • Model Architecture Design: +0.97 points: the best discovered architecture’s gain over DeltaNet was nearly three times Mamba2’s +0.34-point human-designed improvement.The discovered architectures also improved on both development and out-of-distribution generalization benchmarks.
  • Model Architecture Design: Adaptive, multi-scale routing was the common design pattern across the top five discovered architectures.The architectures dynamically allocate computation across local, contextual, long-range, and delta-rule pathways rather than relying on fixed allocation schemes.
  • Pretraining Data Curation: +3.96 points: Nemotron-CCASI+ raised average benchmark performance over raw data, including +18.64 points on MMLU.The evolved strategies combined targeted noise removal, format normalization, and domain-aware preservation rules; optimized and suboptimal strategies differed by 2.93 points.
  • Reinforcement Learning Algorithm Design: +12.5 points on AMC32, +11.67 points on AIME24, and +5.04 points on OlympiadBench: evolved reinforcement-learning variants outperformed GRPO.The search ran for 300 evolutionary rounds; representative algorithms introduced pairwise advantage estimation, asymmetric clipping, gradient dropout, and budget-constrained updates.

5. Empirical Analysis

ASI-Evolve is evaluated on circle packing as a controlled benchmark for comparing evolutionary frameworks and design choices. It reaches a strong score quickly and shows compatibility across base models.

  • 5.1 Benchmarking ASI-Evolve on Circle Packing: The circle packing task provides aligned conditions for comparing evolutionary frameworks because it has low verification cost but still requires iterative algorithm design.The task places 26 circles within a 1 × 1 square while maximizing their summed radii.
  • 5.1 Benchmarking ASI-Evolve on Circle Packing: ASI-Evolve reaches 2.63597 in 17 steps, the fastest among compared systems, with a best score of 2.635983.The benchmark places 26 circles in a 1 × 1 square and maximizes the sum of radii.
  • 5.2 Comparison Experiments: ASI-Evolve, GEPA, and OpenEvolve exhibit different evolution dynamics under the same Qwen3-32B base model and aligned prompt setup.The comparison examines how the frameworks improve over repeated evolution rounds.
  • 5.2 Comparison Experiments: ASI-Evolve starts above both baselines, improves steadily, and is the only framework reported to reliably reach SOTA-level performance.GEPA converges around 2.630, while OpenEvolve shows high variance and limited improvement.
  • 5.2 Comparison Experiments: ASI-Evolve with GPT-5-mini and Qwen3-32B converges to a similar range with consistent mid-to-late-stage improvement, although early progress differs.The results support compatibility across the two tested base models.

5.2 Comparison Experiments

Comparison experiments examine framework dynamics, base-model compatibility, and sampling strategies. The reported results favor ASI-Evolve and show that cognition-guided exploitation can accelerate later-stage discovery.

  • 5.2.3 Algorithm Comparison: Random sampling starts higher than MAP-Elites but later slows and falls behind MAP-Elites and UCB1.The passage attributes the early advantage to unrestricted parent selection and the later slowdown to lacking exploration–exploitation balancing.
  • 5.2.3 Algorithm Comparison: UCB1 reaches high-score regions faster than MAP-Elites and has lower variance across runs when cognition and Analyzer feedback provide directional guidance.The reported explanation is that strong guidance reduces the risks of exploitation-oriented sampling.

5.3 Ablation Study: Validating Component Effectiveness

Ablations separate the effects of the Analyzer and Cognition repository. Cognition improves cold-start performance, while the Analyzer supports sustained and reliable evolution.

  • 5.3 Ablation Study: The ablation compares the full four-stage method against variants without the Analyzer or Cognition repository.The full method includes Analyzer, Cognition, and the Learn–Design–Experiment–Analyze loop.
  • 5.3.1 Impact of removing Analyzer: Removing the Analyzer still yields high initial scores because literature-derived Cognition priors guide promising directions from the outset.This provides a cold-start advantage independently of Analyzer-driven feedback.
  • 5.3.2 Impact of removing Analyzer: Without the Analyzer, evolution enters a prolonged plateau with marginal gains, slower sustained improvement, and lower reliability than the full method.Some runs eventually reach SOTA-level scores, partly because raw evaluation scores still provide limited feedback and circle packing is relatively straightforward.
  • 5.3.3 Impact of removing Cognition: Removing Cognition causes slower, less stable early improvement and a prolonged low-score phase before the system enters a more productive exploration regime.The passage characterizes this as a more pronounced cold-start cost.
  • 5.3.3 Impact of removing Cognition: The No Cognition variant still evolves effectively through self-guided trial-and-error, but requires longer exploration when useful priors are unavailable.Its core learning mechanisms remain usable in entirely novel domains.

5.4 Validating Real-World Applicability: Drug–Target Interaction Discovery

ASI-Evolve is applied to drug-target interaction prediction to test whether AI-designed architectures transfer beyond AI development. The discovered architecture improves both standard and cold-start evaluations.

  • 5.4.1 Task and Methodology: ASI-Evolve targets drug-target interaction prediction, where models must represent drugs, proteins, and their complex interactions across a large discrete design space.The design space includes feature extraction, fusion mechanisms, and interaction modeling, with limited theoretical guidance.
  • 5.4.1 Task and Methodology: The DTI cognition repository is initialized from approximately 80 papers on graph neural networks, attention mechanisms, and DTI modeling.The papers capture limitations including shallow cross-attention and insufficient higher-order interaction modeling.
  • 5.4.2 Results: The discovered architecture improves over DrugBAN on most evaluation settings, including +1.91 AUROC and +2.95 F1 points on BindingDB development.On BindingDB-Random, it achieves +1.05 AUROC and +1.39 F1 points.
  • 5.4.3 Generalization Analysis: Cold-start gains reach +6.94 AUROC points for unseen drugs, +3.56 points for unseen proteins, and +4.36 points for doubly cold-start evaluation.These improvements exceed the reported in-distribution gains.
  • 5.4.4 Analysis: The best architecture introduces Sinkhorn Attention, domain-specific marginalization, and additional interaction-design innovations over DrugBAN.Sinkhorn iterations impose doubly-stochastic constraints intended to balance attention allocation and prevent attention collapse.
  • 5.4.4 Analysis: Evolution tracking reports that the Sinkhorn mechanism emerged by combining optimal transport theory with bipartite matching concepts from computational biology.The fitness curve improves steadily overall, with notable jumps when cross-domain ideas are synthesized.
  • 5.4.2 Results: The autonomously discovered architecture achieves competitive or superior performance across benchmarks against expert-designed DTI methods.The comparison includes methods using molecular encoders, protein language models, or specialized graph convolutions.

6 Conclusion

ASI-EVOLVE is presented as an agentic evolution framework for end-to-end autonomous scientific research across model architecture, training data, and training algorithms. The authors report effective framework design, rapid cold-start, continuous improvement, and SOTA-level results, while identifying broader AI-stack coverage as future work.

  • ASI-EVOLVE enables end-to-end autonomous scientific research through a closed learn–design–experiment–analyze loop.The framework targets multiple stages of the AI research pipeline rather than a single model component.
  • Controlled comparisons and ablation studies support the effectiveness of the framework design, including its structured cognition base and dedicated analyzer.
  • ASI-EVOLVE achieves rapid cold-start, sustains continuous improvement, and reliably reaches SOTA-level results.
  • The framework demonstrates breakthroughs across model architecture, training data, and training algorithms despite their implementation complexity, iteration cost, and indirect feedback.
  • The drug-target interaction experiment indicates that AI-discovered model designs can be deployed in real-world tasks beyond the core AI pipeline.
  • Architecture, data, algorithms, and infrastructure across the full AI development stack remain areas for future exploration.

A Analysis Configuration Details

The appendix describes circle-packing analysis settings, comparison protocols, ablation configurations, and the cognition repository used in the experiments. The setup aligns framework comparisons and varies selected system components while documenting geometric and optimization details.

  • The appendix summarizes concrete experiment settings for the circle-packing analysis and identifies framework and model comparisons as a configuration section.
  • Framework and model comparison: Framework comparisons align OpenEvolve-style prompting, full-file evolution, MAP-Elites/island databases, and population settings, while comparing GPT-5-mini and Qwen3-32B under the same system design.
  • Sampling algorithm comparison: Sampling comparisons hold the prompt, cognition, evolution mode, engineer, parallelism, and database settings fixed while changing only MAP-Elites versus UCB1 sampling.
  • Ablation study: The ablation study disables either the Analyzer or cognition usage while retaining the rest of the Researcher–Engineer–Analyzer loop as the comparison baseline.
  • Cognition repository: The shared cognition repository is initialized before evolution and contains 12 items spanning geometric priors, including hexagonal close packing.
  • Circle-packing experiment: Circle-packing settings account for boundary effects, variable radii, an n = 26 target, constrained SLSQP optimization, warm starts, multi-start configurations, and tight iteration limits.The target sum of radii is approximately 2.635, and optimization maximizes the sum subject to no-overlap and in-bounds constraints.

A. Analysis Configuration Details SII-GAIR

The SII-GAIR configuration details describe optimization priors, engineering guidance, numerical safeguards, and a controlled framework comparison. The comparison uses matched backbone models while acknowledging possible bias from simplifying OpenEvolve’s auxiliary-model setup.

  • Optimization methodology: Differential evolution is used for global refinement when local optimization plateaus, followed by optional radius-only refinement and SLSQP polishing.
  • Related optimization priors: AlphaEvolve’s reference result of 2.635 is associated with geometric initialization followed by strict-constraint numerical refinement.
  • Engineering guidance: Engineering guidance favors targeted optimizer, constraint, multi-start, or refinement changes over rewriting the full program, with construction and optimization kept separate.
  • Numerical stability: A small epsilon such as 10^-8 is used to avoid numerical tangency, and reported solutions are checked for constraint violations.
  • Plateau breaking: When optimization stalls near 2.3–2.4, the checklist recommends stronger initialization, maxiter 500–1000, explicit inequalities, 3–5 starts, or differential-evolution refinement.
  • Framework comparison: The aligned framework comparison uses the same backbone model on both sides to focus on framework design, but may bias results relative to an optimally configured OpenEvolve.
Loading 2603.29640v1…