An AI system to help scientists write expert-level empirical software

• Empirical software is central to scientific endeavors because it supports models of complex systems and has enabled major advances, including Nobel Prize-winning work.Examples span atmospheric weather modeling, turbulent-fluid stress response, social-system prediction, density functional theory, molecular dynamics, and protein structure prediction.Source: Introduction · p. 2
• Creating domain-specific empirical software is slow and difficult, limiting the range of scientific solutions that can be productively explored.Design choices are often driven by intuition or expediency rather than systematic searches and exhaustive experimentation.Source: Introduction · p. 2
• ERA uses an LLM to rewrite software for higher quality scores while Tree Search selects candidate solutions for further exploration.The system creates multiple software candidates and uses Tree Search to navigate which candidates merit continued exploration.Source: Introduction · p. 2
• ERA augments code mutation with research ideas from highly cited papers, specialized textbooks, and search engine results, injected by users or automatically.The LLM uses this injected guidance when writing code.Source: Introduction · p. 2
• ERA produces software that outperforms the state of the art across scorable tasks spanning broad scientific disciplines.The paper attributes this expert-level performance to exhaustive, tireless searches that identify high-quality solutions at unprecedented scale.Source: Introduction · p. 2

Overview of Scorable Tasks

• ERA evaluated scorable tasks spanning batch integration, COVID-19 forecasting, time-series prediction, geospatial segmentation, neural activity forecasting, and difficult numerical integration.The tasks were selected as relevant scientific or engineering problems and scored through Kaggle or domain-specific benchmarks.Source: Overview of Scorable Tasks · p. 2

Kaggle Playground Benchmark

• Across 16 Kaggle Playground competitions, ERA substantially outperformed a single LLM call, best-of-1000 LLM calls, and AIDE13 by maintaining a diverse, backtrackable tree of candidates.Problem-specific expert advice and boosted-decision-tree instructions further improved performance.Source: Figure 1 · p. 3, Kaggle Playground Benchmark · p. 4

Genomics: Batch Integration of Single-Cell RNA Sequencing Data

• ERA’s BBKNN (TS) improved overall performance by 14% over ComBat33, matched or exceeded published BBKNN across every dataset and 11/13 metrics, and eight of nine ERA implementations beat their published counterparts.The improvement came partly from using ComBat-corrected PCA embeddings before constructing cross-batch nearest-neighbor graphs, demonstrating the value of recombining ideas.Source: Genomics: Batch Integration of… · p. 5, Figure 2 · p. 6

Public Health: Prediction of U.S. COVID-19 Hospitalizations

• 14 ERA-generated COVID-19 forecasting strategies outperformed the official CovidHub-ensemble, including 10 recombinations, two Deep Research31 strategies, one AI co-scientist32 strategy, and one replicated baseline.The retrospective Google model achieved average WIS 26 versus 29 for the official ensemble.Source: Public Health: Prediction of U.S.… · pp. 7, 9, Figure 3 · p. 8

Time Series Forecasting: GIFT-Eval

• ERA outperformed the entire May 18, 2025 GIFT-Eval leaderboard with per-dataset solutions, while its unified adaptive forecasting library remained highly competitive.The unified model used up to eight dataset-specific configurations and sequentially modeled level, trend, seasonality, datetime features, and residual corrections.Source: Time Series Forecasting: GIFT-Eval · p. 10

Other Problems: Geospatial Analysis, Neuroscience and Numerical Analysis

• ERA achieved expert-level performance on geospatial semantic segmentation, vertebrate neural activity forecasting, and numerical analysis.These evaluations covered three additional domains beyond the Kaggle, genomics, public-health, and time-series benchmarks.Source: Other Problems: Geospatial Analysis,… · p. 10

System contribution

• ERA combines Tree Search with LLM-based semantic code rewriting, converting scientific software creation into a scorable search for programs that maximize a quality score.The rewriting approach integrates domain knowledge and external research ideas while exploring a large solution space.Source: Discussion · pp. 10–11

Positioning

• ERA differs from one-shot code-generation systems and fixed-scope AutoML by iteratively rewriting existing candidates and optimizing software beyond standard machine-learning pipelines.Its scope includes preprocessing, simulations, mathematical heuristics, and other software domains.Source: Discussion · p. 11

Novelty

• ERA is presented as the first system known to beat human performance across a wide range of relevant, well-studied scientific problems.When prompted with cutting-edge paper method descriptions, it often produced implementations with superior results.Source: Discussion · p. 11

Limitations

• The evaluated problems primarily measure advanced empirical software engineering rather than genuine scientific discovery, which requires theories, causal mechanisms, and mathematical frameworks.The authors state that ERA nevertheless extends toward scientific discovery beyond the core evaluated tasks.Source: Discussion · p. 12

Empirical results

• ERA created 40 methods that beat the best known scRNA-seq batch-integration method and 14 methods that outperformed the CDC ensemble for epidemiological prediction.It also achieved expert-level performance in geospatial reasoning, neural activity prediction, computational mathematics, and time-series prediction.Source: Discussion · p. 12

Implications and risks

• ERA reduces exploration of research ideas from weeks or months to hours or days, but autonomous expert-level software generation also lowers barriers to deploying advanced models in sensitive or dangerous domains.The authors describe this as a systemic risk arising from large-scale inference-time compute and improving foundation models.Source: Discussion · p. 12

Code mutation system

• ERA prompts an LLM to generate Python code, executes and scores candidates in a sandbox, and uses PUCT-based tree search to select and mutate solutions.The globally flat tree balances exploitation and exploration, enables branching from historical nodes, and backpropagates visit counts after each execution.Source: Code mutation system · p. 13

Adding research ideas to the code mutation system

• Research ideas were injected into code-mutation prompts for single-cell integration, COVID-19 prediction, remote-sensing segmentation, and whole-brain neural activity prediction.Ideas came from literature-based methods and LLM-driven searches, including Gemini Deep Research and AI co-scientist.Source: Adding research ideas to the code… · p. 14

ERA versus best-of-N

• ERA outperformed Best-of-N for every model and problem except GPT-5 on batch integration, where GPT-5’s one-shot performance was already sufficiently strong.The comparison used 128 search nodes and reported best performance over three experiments for each approach.Source: ERA versus best-of-N · p. 15

Recombination experiments

• ERA recombined expert approaches by asking Gemini to compare parent methods and prompting tree search to combine their best technical components.This produced 55 recombined scRNA-seq methods and 28 COVID-19 methods, operating in conceptual space through prompts rather than only distilled model space.Source: Recombination experiments · p. 15

Gemini embeddings

• Each tree-search implementation was represented by a 3,072-dimensional vector generated with the Gemini text embedding model.These vectors served as semantic representations of the implementations.Source: Gemini embeddings · p. 15

scRNA-seq batch integration

• For scRNA-seq integration, ERA optimized nine existing research methods after excluding poorly performing foundation models and data-leaking scANVI.Method summaries were extracted from manuscripts and added to tree-search prompts to guide improved code solutions.Source: scRNA-seq batch integration · pp. 16–17

COVID-19 prediction

• For COVID-19 forecasting, ERA used historical CDC hospitalization data, population values, and 2,000-node searches to replicate eight eligible CovidHub models.Models were selected using reproducibility, forecast-horizon coverage, and submission-duration criteria; retrospective comparisons used data available by 2025-05-01.Source: COVID-19 prediction · p. 17

GIFT-Eval benchmark

• 0.734 MASE was achieved on GIFT-Eval after iterative refinement from 0.82 to 0.77 and a final eight-configuration solution.The unified solution was developed with tree search and selected configurations using validation data before producing test forecasts.Source: GIFT-Eval benchmark · pp. 18–19

Code Availability

• The ERA implementation, six best candidate solutions, and representative tree-search data are publicly available for inspection through an accompanying interface.The interface shows solution progression, breakthrough plots, and code differences as tree search proceeds.Source: Code Availability · p. 25

• Code Mutation System: E.A., A.B., G.C., M.C., H.C., P.N., D.S., J.T., S.V., M.P., J.K., P.R., J.W., L. W., S.M., and M.P.B.Source: Author Contributions Statement · p. 25
• Application contributions covered Single Cell RNA-seq Batch Integration, COVID Forecasting, and Geospatial Analysis, with the listed authors assigned to each area.Source: Author Contributions Statement · p. 25
• Benchmark and scientific-software contributions included ZAPBench, GIFT-Eval, and Integrals, with B.P.W., J-M.L., Q.Z.; J.G., M.P.B.; and A.K., R.K., M.P.B., respectively.Source: Author Contributions Statement · p. 25
• Additional roles included User Interfaces, Graphical Design, Program Management, and Leadership, assigned to the authors listed in the statement.Source: Author Contributions Statement · p. 25

• The authors affiliated with Google are employees of Google Inc. and hold Alphabet stock.Source: Competing Interest Declaration · p. 25

Extended Data Figure Legends

• Extended Data Fig. 1 compares replicate scores on holdout OpenProblems datasets and reports missing-metric computation for some tree-search replicates.The figure also averages methods over method–dataset–metric combinations with non-NaN values across base methods and replicates.Source: Figure 1 · p. 26
• Breakthrough plots and solution trees trace score improvements and search structure for BBKNN, COVID-19 prediction, and GIFT-Eval time-series prediction.Green nodes mark abrupt score increases caused by code improvements, while diamond nodes denote the highest score.Source: Figure 2 · p. 27, Figure 6 · p. 31, Figure 8 · p. 33
• Extended Data Fig. 3 ablates scaling, standardization, ComBat+PCA, and ERA’s BBKNN implementation while comparing written and package BBKNN versions.Bars show overall OpenProblems performance, with dots representing three replicates per method.Source: Figure 3 · p. 28
• Extended Data Fig. 4 evaluates recombination against both parent methods across all 55 pairwise combinations of 11 base methods on shared successfully computed metrics.Color coding indicates whether recombination outperforms both, one, or neither parent method.Source: Figure 4 · p. 29
• Extended Data Fig. 5 tests whether models selected on six weeks of validation dates generalize to the next two unseen reference weeks.It reports WIS by reference date for the best-performing replicate from each validation window.Source: Figure 5 · p. 30
• Extended Data Fig. 7 compares hybrid COVID-19 forecasting models with their constituent CovidHub baselines using average WIS.Green, dark-blue, and red bars indicate that recombination outperformed both, one, or neither parent model.Source: Figure 7 · p. 32

Extended Data Tables

• Extended Data Table 1 provides expert manual inspection of whether ERA’s implementation adheres to the scRNA-seq batch-integration method.The supplied table description identifies adherence inspection but does not report its outcomes.Source: Table 1 · p. 34

A.1. Geospatial Analysis: Segmentation of Remote Sensing Images

• The task is difficult because imagery varies with time of day, season, and weather, while objects within a class differ substantially in size, shape, height, function, and lighting.Semantic segmentation assigns a class label to every image pixel and supports applications including land-use monitoring, environmental assessment, and disaster management.Source: §A.1 · p. 35
• ERA trained models to assign land-cover classes to pixels using a reproducible 80/20 DLRSD train/test split, with performance measured on the held-out test set.DLRSD is a densely labeled version of the UC Merced Land Use Dataset, whose pixels carry 17 class labels [20, 71].Source: §A.1 · p. 35
• mIoU exceeded 0.80 for all three top tree-search solutions, significantly outperforming reported results on the DLRSD benchmark (Supplementary Table S14; Supplementary Fig. S14).The solutions were evaluated on a held-out test set of 420 images using mean intersection over union.Source: §A.1 · p. 35
• Solutions 1 and 3 used UNet++ or U-Net with ImageNet-pretrained EfficientNet-B7 or SE-ResNeXt101-32x4d encoders, while Solution 2 used the SegFormer Transformer architecture.The models differed in data augmentation and prediction strategies; the U-Net-based solutions used extensive Albumentations augmentation, whereas SegFormer used more basic transforms.Source: §A.1 · p. 35
• All three solutions used test-time augmentation, averaging reverse-transformed predictions from flips and rotations to smooth errors and improve mask robustness [73].A representative tree and breakthrough plot for Solution 3 is shown in Supplementary Fig. S15.Source: §A.1 · p. 35

A.2. Neuroscience: Whole-Brain Neural Activity Prediction

• ERA predicted activity for 71,721 neurons up to 32 steps ahead from four-step contexts, using stimulus-specific 70%/10%/20% train-validation-test splits and MAE for model selection.The benchmark data came from two-hour light-sheet recordings across 7,879 time steps.Source: §A.2 · pp. 35–36
• ERA’s tree-searched model outperformed all evaluated baselines, including the best-performing video model, except for 1-step-ahead predictions.It combined temporal convolutions, a learned global brain state, neuron-specific embeddings, weight-shared residual blocks, and one-shot multi-step prediction.Source: §A.2 · p. 36
• The separately tree-searched model achieved leading performance for 1-step-ahead predictions using dynamic attention for global context and FiLM-like interactive conditioning.Its architecture also generated a learned global context vector to inform per-feature predictions.Source: §A.2 · p. 36
• Both ERA solutions trained in less than two hours on one T4 GPU, compared with 36 hours on 16 A100 GPUs for the best-performing video model.They also used cross-neuron information to generate predictions.Source: §A.2 · p. 36
• An exploratory Jaxley-based model simulated neurons with dynamically modulated Hodgkin-Huxley parameters and a latent autoencoder, remaining competitive with time-series baselines but below the top video model.The autoencoder supplied a system-wide corrective signal that approximated a functional connectome without direct synaptic simulation.Source: §A.2 · p. 36

A.3. Numerical Analysis: Library for Numerical Evaluation of Difficult Integrals

• The benchmark contained 38 oscillatory integrals with infinite upper limits for which quad() returned incorrect answers despite known analytic solutions.These cases targeted failures involving nonconvergence, missed integrand features, and precision loss from cancellations.Source: §A.3 · p. 37
• Manual examination of Gemini-generated SymPy expressions and constrained parameter values limited the benchmark’s scale.The dataset was constructed from LaTeX integrals and solutions sourced from Gradshteyn and Ryzhik82.Source: §A.3 · p. 37
• ERA was initialized with quad() and improved through a 1000-node tree search scored by the logarithm of absolute fractional error.The logarithmic score reduced the search’s tendency to overweight outliers.Source: §A.3 · p. 37
• The method partitions the infinite domain into finite subintervals, computes each segment with quad(), and applies Euler’s transformation to accelerate slowly converging series.Geometrically increasing segment lengths improve coverage of the domain tail, while repeated averaging extrapolates the series limit.Source: §A.3 · p. 37
• The evolved code correctly evaluated 17 of 19 held-out evaluation integrals within a fractional error of less than 3 percent.The integrals were randomly split into training and evaluation datasets of 19 integrals each.Source: §A.3 · pp. 37–38
• The evolved code falls back to specialized methods only when quad() has a large error estimate, returns NaN or Inf, or raises an exception, preserving quad() accuracy otherwise.This conditional structure supports use as a drop-in replacement for less pathological integrals.Source: §A.3 · p. 38

A.4. Towards Genuine Discovery

• ERA’s deployments illustrate the potential of its core tree-search algorithm to support genuine discovery across symbolic mathematics and computational neuroscience.These applications extend beyond routine predictive modeling into analytical derivation and recovery of underlying neural mechanisms.Source: §A.4 · p. 38
• Six novel analytical derivations and a closed-form exact solution were discovered for the previously unsolved cosmic-string radiation integral.The system also discovered a novel asymptotic expansion without empirical data or predictive modeling.Source: §A.4 · p. 38
• In larval zebrafish neural modeling, structural priors from wiring diagrams enabled recovery of causal effective connectivity that unconstrained search failed to identify.Unconstrained search instead produced predictive models relying on statistical shortcuts.Source: §A.4 · p. 38
• Mathematical discovery used a scoring metric comparing proposed analytical formulas with high-precision numerical solutions across random physical parameter values.The metric minimized residual differences against numerical integration, guiding the search toward exact solutions.Source: §A.4 · p. 38
• For open-ended scientific domains, designing an appropriate scoring metric requires domain expertise, scientific creativity, and iterative refinement by the user.Unlike standardized predictive-modeling scores, these metrics encode hypothesized scientific goals and are revised when those goals are not reached.Source: §A.4 · p. 38

A.5. Computational Cost and Resource Utilization

• Search-based LLM optimization is inherently more resource-intensive than single-shot or zero-shot prompting because it repeatedly generates and empirically evaluates candidate solutions.Representative computational costs per search node are reported for evaluated benchmark runs in Supplementary Table S1.Source: §A.5 · p. 38

Supplementary Figures

• The single-cell integration evaluation sampled 20,000 training and 20,000 validation cells, then selected each method’s best validation implementation for final OpenProblems.bio holdout testing.Three independent tree-search replicas were run per base method, with training used for hill climbing and validation used for selection.Source: Supplementary Figures · p. 39, Figure S1 · p. 39
• ERA-generated single-cell methods formed distinct non-deep-learning and deep-learning clusters in text-embedding space, with pairwise similarities above 0.85.Gemini classification confirmed the two-cluster organization; the cross-benchmark lower-bound similarity was 0.74.Source: Figure S4 · p. 41, Figure S5 · p. 42
• Across COVID-19 rolling validation dates, replicate forecasts showed consistent WIS trends, supporting reproducible generation of high-performing probabilistic forecasts.The supplementary visual reports average WIS by reference date for individual model replicates.Source: Figure S6 · p. 43
• ERA’s best COVID-19 recombination model achieved average WIS 11.63, versus 12.85 for CovidHub-ensemble and 14.39 for Google Retrospective.The tree-search model also produced narrower confidence bands while consistently containing the ground truth.Source: Figure S12 · p. 49
• Supplementary figures visualize breakthrough plots and solution trees for geospatial segmentation, ZAPBench, and numerical integration, highlighting score improvements as search nodes accumulate.The highest-scoring node is marked by a diamond, while green dots identify abrupt score increases caused by code improvements.Source: Figure S15 · p. 52, Figure S17 · p. 53, Figure S19 · p. 55
• The ZAPBench comparison evaluates three ERA solutions against mean and stimulus baselines using average MAE across conditions and three random seeds.It includes a general tree-search solution, a 1-step-ahead specialized search, and a Jaxley-prompted solution, with 95% confidence intervals.Source: Figure S16 · p. 53
• 17 of 19 held-out oscillatory integrals were solved within 3% by ERA, whereas scipy.integrate.quad() failed on all cases.The evaluation used a held-out set of 19 integrals, with zero representing a perfect score.Source: Figure S20 · p. 56

B. Supplementary Tables

• Supplementary tables document ERA’s computational budget, Kaggle Playground competition set and prompts, expert-advice and boosted-tree variants, and example BBKNN (TS) code.Execution cost scales linearly with explored nodes; integral prompt token counts are inflated by image tokenization, and the study used NVIDIA Tesla T4 GPUs.Source: Table S1 · p. 57, B. Supplementary Tables · p. 57, Table S2 · p. 57

Single-cell batch integration

• ERA’s single-cell example normalizes, log-transforms, and scales count data before applying ComBat batch correction and PCA.The corrected PCA embedding is assigned as the integrated output evaluated by ASW, LISI, and PCR metrics.Source: Example code generated by ERA · p. 60
• The single-cell implementation handles small datasets by limiting PCA dimensions and returning a trivial embedding with empty graph structures when PCA or graph construction is infeasible.This prevents failures when observations or dimensions are insufficient for meaningful computation.Source: Example code generated by ERA · p. 60
• The code constructs a custom batch-aware nearest-neighbor graph by querying neighbors within batches, merging candidates, retaining closest connections, and symmetrizing distances.It stores sparse distance and connectivity matrices with metadata identifying the strategy as custom_batch_aware_combat_pca.Source: Example code generated by ERA · pp. 60–61

COVID-19 forecasting

• ERA’s forecasting examples include a three-level COVID-19 ensemble combining state-level Holt-Winters, regional LSTM, and all-state LSTM predictions weighted by recent MAE.States are grouped into regions using two years of COVID-19 activity trends identified with Louvain clustering.Source: Example code generated by ERA · p. 64
• Another forecasting example combines a TCN encoder with an extended N-BEATS decoder, variance-producing residual blocks, predictor variables, and an ensemble median across lookback windows and initializations.The architecture generates quantile forecasts using a parametric gamma error distribution.Source: Example code generated by ERA · p. 65