Source-linked AI summary
Aster: Autonomous Scientific Discovery over 20x Faster Than Existing Methods
Emmett Bicker
TL;DR
Existing autonomous-discovery frameworks often need hundreds to thousands of iterations, making long-evaluation problems impractical. Aster iteratively refines programs using an initial program, evaluator, and prompt, achieving over 20x speedup and strong results across multiple tasks.
Problem
Existing state-of-the-art frameworks often require hundreds to thousands of iterations, restricting autonomous discovery to problems with short evaluation times.
Method
Aster is an autonomous agent that takes an initial program, evaluator, and prompt, then iteratively refines the solution.
Results
Aster achieved an over 20x speedup over OpenEvolve and improved or matched reported benchmarks across mathematics, biology, GPU kernel optimization, language-model training, and neuroscience tasks.
Takeaways & Limitations
Aster’s reduced iteration count enables computationally intensive discovery tasks that were previously out of reach.
Takeaways & Limitations
On ZAP-Bench, Aster matched the best human performance but remained above the 0.0176 MAE achieved by another autonomous discovery system.
Abstract
from arXiv · showhide
We introduce Aster, an AI agent for autonomous scientific discovery capable of operating over 20 times faster than existing frameworks. Given a task, an initial program, and a script to evaluate the performance of the program, Aster iteratively improves the program, often leading to new state-of-the-art performances. Aster's significant reduction in the number of iterations required for novel discovery expands the domain of tractable problems to include tasks with long evaluation durations, such as multi-hour machine learning training runs. We applied Aster to problems in mathematics, GPU kernel engineering, biology, neuroscience, and language model training. More specifically: the Erdos minimum overlap problem, optimizing the TriMul kernel, a single-cell analysis denoising problem, training a neural activity prediction model to perform well on ZAPBench, and the NanoGPT Speedrun Competition. Aster attains SOTA results in every task, except for ZAPBench, where it matches the performance of the best human solution with less than 1/190th of the compute. Aster is accessible via a web interface and API at asterlab.ai.
1. Introduction
Autonomous discovery systems use LLMs to iteratively improve programs, but current frameworks often need hundreds or thousands of iterations. Aster addresses this bottleneck with an agent that refines programs using an initial program, evaluator, and prompt, achieving over 20x speedup versus OpenEvolve.
- LLM-based autonomous discovery has produced breakthroughs in matrix multiplication, machine-learning kernels, and mathematical constructions.
- Current frameworks often require hundreds to thousands of iterations, limiting their use on tasks with long evaluation times.Training large machine learning models is given as an example of a task where repeated evaluations become prohibitive.
- Aster takes an initial program, evaluator, and prompt, then iteratively refines the solution as an autonomous agent.
- Over 20x speedup against OpenEvolve enables Aster to tackle computationally intensive discovery tasks previously out of reach.
2. Speedup Analysis
Aster is benchmarked against OpenEvolve on packing 26 circles, using the same underlying model distribution for a fair comparison. It reaches a score of 2.6353 in 5 iterations and the known SOTA score of 2.635983 in 6 iterations.
- Aster and OpenEvolve were compared on the circle packing problem, with Aster configured against a prominent open-source autonomous discovery framework.The benchmark packs 26 circles.
- 5 iterations yielded Aster’s score of 2.6353, surpassing OpenEvolve’s 2.634 score reached in 115 iterations.Both systems used the same underlying model distribution: 80% Gemini 2.0 Flash and 20% Claude 3.7 Sonnet.
- Figure 3 provides a visual comparison of the circle packing solutions.
3. New Discoveries
Aster produced state-of-the-art or highly competitive results across mathematical construction, single-cell denoising, GPU kernel optimization, language-model training, and neural-activity forecasting. On ZAP-Bench, it matched the best human performance with substantially less compute, while other tasks improved prior records or benchmarks.
- Erdős Minimum Overlap: 0.380874 surpassed the TTT-Discover record of 0.380876 for the Erdős Minimum Overlap problem after 40 iterations.Aster’s construction used 8192 pieces, compared with 600 in the previous state-of-the-art solution.
- Single-Cell Denoising: 0.711 improved on the TTT-Discover single-cell denoising result of 0.709 in 30 iterations.The evaluation minimized MSE while keeping Poisson below a specified threshold; the table caption describes a higher overall mean score from reduced MSE with competitive Poisson.
- GPU Kernel Optimization: 1114 µs reduced TriMul kernel runtime below the TTT-Discover benchmark of 1161 µs on an NVIDIA H100 GPU.Aster optimized the kernel over 70 iterations after restarting with the correct competition image.
- NanoGPT Speedrun: 95.2 seconds lowered the NanoGPT Speedrun record by 1.6 seconds in 8 iterations.The resulting program refined Triton kernels, optimized memory loads, and avoided unnecessary recomputation.
- ZAP-Bench: 0.0182 MAE matched the best human ZAP-Bench performance with 190x less compute after 34 iterations.The human model trained for 36 hours on 16 A100s, whereas Aster’s evaluation used an NVIDIA T4 and took approximately two and a half days of work.
- ZAP-Bench: 0.0176 MAE remained better than Aster’s 0.0182 on ZAP-Bench under similar runtime constraints.Aster was still improving when its run was stopped, so the reported result does not establish the best autonomous-discovery performance on this task.
A Programs
Aster-generated programs span mathematical optimization, single-cell denoising, and GPU-kernel engineering, combining specialized objectives with iterative optimization and data-processing pipelines.
- Erdős Minimum Overlap Program: Aster’s Erdős minimum-overlap program optimizes a bounded step function using overlap correlation, regularization, symmetry enforcement, and penalties.The objective uses FFT-based correlation, LogSumExp, total-variation regularization, and symmetry loss.
- Erdős Minimum Overlap Program: The Erdős optimizer uses sigmoid parameterization, integral normalization, centered-bump initialization, temperature scheduling, gradient updates, and an L-BFGS-B refinement phase.The final solution is checked with a full FFT and reported as a C5 upper bound.
- Single-Cell Denoising Program: The single-cell denoising program converts counts to arrays, applies variance stabilization, library-size normalization, log stabilization, graph construction, and multi-scale diffusion.Diffusion states are combined with harmonic weights before adaptive blending and inverse transformation.
- Single-Cell Denoising Program: The denoising pipeline preserves cell-specific signals through adaptive residual blending and applies inverse transformations followed by a Poisson-targeted nonlinear noise-floor contraction.The contraction uses depth-aware exponents and smoothly contracts values below a threshold rather than hard-zeroing them.
- TriMul Kernel Optimization Program: The TriMul kernel program uses fused Triton heads and tails, packed tensor layouts, cached fp16 weights, and LayerNorm-related operations for GPU execution.The implementation combines pointwise projections, sigmoid gates, optional masking, tensor-core matrix multiplication, and final projection.