PaperlayerView exact revision on arXiv

Source-linked AI summary

Foundation Models for Atomistic Simulation of Chemistry and Materials

Eric C. -Y. Yuan, Yunsheng Liu, Junmin Chen, Peichen Zhong, Sanjeev Raja, Tobias Kreiman, Santiago Vargas, Wenbin Xu, Martin Head-Gordon, Chao Yang, Samuel M. Blau, Bingqing Cheng, Aditi Krishnapriyan, Teresa Head-Gordon

arXiv:2503.10538v3physics.chem-ph

TL;DR

This Perspective asks whether scaling data, model parameters, and pre-training can support foundation models for atomistic simulation. It defines criteria for MLIP foundation models and surveys architecture, data, and training strategies; current models show strong benchmark performance, while important testing and uncertainty challenges remain.

  • Problem

    Whether MLIP foundation models can scale data, parameters, and pre-training strategies for learned simulations remains an open question.

    Source: §1 · p. 2

  • Method

    The Perspective defines MLIP foundation-model criteria and examines architectures, data needs, and fine-tuning strategies, focusing primarily on energy and force data.

    Source: §1 · pp. 2–3

  • Results

    UMA and eSEN-OMol top Rowan’s NNP Arena leaderboard for neutral, closed-shell organic molecular energies, with GMTKN55 errors against coupled cluster nearly 5 kcal/mol below the previous best model [183].

    Source: §3.5 · p. 17

  • Takeaways & Limitations

    The Perspective identifies scalable testing platforms as essential for evaluating MLIP foundation-model capabilities and deriving scientific insights.

    Source: §3.4 · p. 15

    Ensemble-based uncertainty estimates are computationally demanding, may not suit larger MLIP foundation models, and have not been included in previous leaderboards.

    Source: §3.4 · p. 16

Abstract

from arXiv · show

Given the power of large language and large vision models, it is of profound and fundamental interest to ask if a foundational model based on data and parameter scaling laws and pre-training strategies is possible for learned simulations of chemistry and materials. The scaling of large and diverse datasets and highly expressive architectures for chemical and materials sciences should result in a foundation model that is more efficient and broadly transferable, robust to out-of-distribution challenges, and easily fine-tuned to a variety of downstream observables, when compared to specific training from scratch on targeted applications in atomistic simulation. In this Perspective we aim to cover the rapidly advancing field of machine learned interatomic potentials (MLIP), and to illustrate a path to create chemistry and materials MLIP foundation models at larger scale.

1 Introduction

Accurate atomistic prediction requires quantum-mechanical accuracy and statistical-mechanical sampling, yet DFT-based ab initio molecular dynamics is inaccessible at many relevant scales. This perspective asks whether foundation-model strategies can make MLIPs broadly useful for atomistic simulation and outlines the concepts, data, training, and evaluation questions involved.

1 Introduction

  • DFT-based ab initio molecular dynamics remains inaccessible for the long timescales, large length scales, and system sizes of many important chemistry and materials problems.Statistical-mechanical observables also require integration over Boltzmann-weighted configurations, usually sampled along trajectories.Source: §1 · p. 2
  • The perspective examines whether foundation models can capture potential energy surfaces through machine-learned interatomic potentials.Its stated end goal is atomistic simulation and analysis across molecular and periodic systems, with little or no system-specific fine-tuning.Source: §1 · p. 2
  • The perspective distinguishes atomistic foundation models from universal potentials and transfer learning, using scaling laws and pre-training strategies as key criteria.It also examines model architecture, data requirements, training strategies, and physical infusion at scale.Source: §1 · pp. 2–3
  • It considers evaluation criteria and benchmarks for testing scaling laws and fine-tuning across diverse downstream applications and observables.The discussion focuses primarily on models trained on energy and force data, while recognizing other data modalities.Source: §1 · p. 3

2 Characteristics of Foundation Models

The perspective frames foundation models around scaling model size, data, and compute alongside pre-training on diverse data. It distinguishes these models from transferable or universal MLIPs and proposes capabilities that a true atomistic foundation model should demonstrate.

2 Characteristics of Foundation Models

  • Foundation-model scaling laws describe performance changes with model size, training data, and computational resources.The perspective presents them as a quantitative framework for studying how architecture, data, and pre-training contribute to performance.Source: §2 · pp. 3–4
  • Pre-training uses large amounts of data to build a model that can perform efficiently across benchmarks and be fine-tuned for downstream tasks.The passage describes language-model pre-training objectives such as predicting missing or next words, with minimal human supervision in earlier training stages.Source: §2 · p. 3
  • MACE-MP-0 is characterized as a universal PBE potential rather than a foundation model because it is supervised for energy and force prediction in one chemical domain at one theory level.The passage also notes that it does not distinguish systems with different total charge and spin.Source: §2 · p. 4
  • A foundation model should demonstrate broad downstream performance, scaling-law compliance, and emergent capabilities beyond its original training data and properties.The proposed criteria include performance superior to task-specific models trained from scratch, as well as capabilities such as predicting higher-quality CCSD(T) data or magnetic-field-dependent properties.Source: §2 · p. 4

3 Foundation Models for Atomistic Simulations

This section surveys MLIP foundation-model development, including how architectures encode physical constraints and the role of scalable data and parameters, data gaps, and model-agnostic training strategies. It also considers broad downstream capability, credible evaluation, and emerging models as steps toward that goal.

3.1 Machine Learning Architectures for Chemistry and Materials

  • Most MLIPs assume local neighborhoods, enforce physical properties such as energy conservation and invariances, and represent interactions through body-order terms.Message passing extends information to more distant neighbors; higher-body terms offer diminishing returns as computational cost rises.Source: §3.1 · p. 5
  • Scalable, expressive MLIPs can learn physical constraints from data while outperforming and using less compute than models that build those constraints in.The authors argue this supports prioritizing data and parameter scaling over constraint-heavy architectures.Source: §3.1 · pp. 5–6
  • Transferability beyond training data remains uncertain, while non-conservative force predictions can distort dynamics and short-range models miss long-range interactions between distant charged molecules.Current evaluations of universal MLIPs largely concern crystalline structure relaxation; the cited text also flags energy drift and artificial heat generation in molecular dynamics.Source: §3.1 · pp. 6–7

3.2 Data Requirements, Pitfalls, and Challenges

  • Robust pre-training requires larger, diverse datasets, but major gaps remain in chemical elements, charge and spin states, reactivity, interfaces, and long-range interactions.Existing MLIP datasets typically contain 1 to 200 million configurations, far fewer than image and language datasets; the authors also call for datasets with richer quantum information.Source: §3.2 · pp. 8, 10, 11
  • Fine-tuning can address reference-data errors from electronic-structure methods, basis sets, and calculation practices, while inconsistent or flawed datasets hinder fusion and learning.The text describes GGA RMS errors as typically 2–4 times those of leading hybrid functionals and hybrid-functional errors as at least 10 times those of CCSD(T).Source: §3.2 · pp. 11–12

3.3 Model-Agnostic Training Strategies

  • Pre-training options include geometric denoising, synthetic MLIP labels, and objectives that combine data from multiple levels of theory.Denoising has improved fine-tuned QM9 property accuracy over random initialization, while synthetic-label benefits depend on a reasonably reliable source model.Source: §3.3 · p. 13
  • Training can encourage physical laws through loss functions and data augmentation rather than encoding each constraint in the architecture.Examples include rotational augmentation and minimizing predicted-force curl to encourage energy conservation; these soft constraints do not guarantee compliance.Source: §3.3 · p. 13
  • Post-training uses supervised fine-tuning for downstream energy and force tasks, while distillation transfers a foundation model’s knowledge to smaller, faster models.The smaller models can specialize for particular tasks while retaining general-purpose representations learned by the foundation model.Source: §3.3 · p. 14, Figure 3 · p. 14
  • Hessian-matching distillation produces specialized student MLIPs up to 50 × faster than large-scale teachers while enabling gradient-based forces and energy conservation.The approach is architecture-agnostic and can transfer general-purpose representations into physically consistent students without requiring costly force constraints during teacher training.Source: §3.3 · pp. 14–15
  • Experimental observables can fine-tune MLIPs initialized from analytical priors or simulated observables, but differentiable simulation remains challenging, especially for dynamical observables.Reweighting, implicit differentiation, and adjoint methods make gradients through molecular-dynamics simulations more tractable.Source: §3.3 · p. 15

3.4 Criteria for a Successful Foundation Model

  • Robust MLIP foundation-model evaluation needs broad downstream tasks, costly but carefully validated testing platforms, and uncertainty tests beyond simple accuracy measures.Ensemble uncertainty estimates are computationally demanding and have not been included in previous leaderboards.Source: §3.4 · pp. 15–16
  • Current leaderboards do not establish foundation-model properties, and OC20 and MatBench Discovery emphasize relaxations and energy accuracy while omitting scaling tests and broader simulation behavior.Supplemental training data may risk test-set leakage, and energy-focused ranking may reward overfitting that degrades scientific utility; NNP Arena adds inference speed, while MLIP Arena assesses physical compliance without extra labels.Source: §3.4 · pp. 15–16

3.5 Toward Foundation Models in Chemistry and Materials

  • OMol25 provides over 100 million high-level DFT snapshots spanning 83 elements, up to 350 atoms, charges from -10e to +10e, and diverse molecular systems and sampling methods.It includes explicit out-of-distribution test sets and model evaluations intended to populate a public leaderboard.Source: §3.5 · p. 17
  • UMA trains on nearly 500 million examples and demonstrates scaling laws for parameters and compute; its MoLE enables multi-task training and fewer active parameters at inference.UMA combines OMol25, OMat24, ODAC23, OC20, and an unpublished Open Molecular Crystal dataset; its architecture is based on eSEN. EScAIP and Uni-Mol2 also show decreasing validation error with increased model size, data, and compute.Source: §3.5 · pp. 16–17
  • Nearly 5 kcal/mol lower GMTKN55 errors against coupled-cluster references put UMA and eSEN-OMol atop Rowan’s NNP Arena leaderboard for neutral, closed-shell organic molecular energies.UMA also has lower energy error than ωB97M-D3BJ/def2-QZVP DFT for variable-charge, variable-spin, and metal-containing large systems and barrier heights, but OMol25 tests still show >100 meV errors on charge, spin, and long-range interactions, likely due to naive charge and spin handling and a 6Å graph cutoff.Source: §3.5 · p. 17

4 Discussion and Future Directions

The discussion frames MLIP foundation models as a route to broader atomistic investigation, while emphasizing unresolved data, evaluation, efficiency, and access challenges.

4 Discussion and Future Directions

  • A successful foundation model could support investigation from metal-organic frameworks to enzymes and electrocatalysis without rebuilding each model from scratch.The paper presents this as a promise of MLIP foundation models.Source: §4 · p. 17
  • Whether physical laws can be learned at scale rather than built into architectures remains an open question; distillation could convert non-conservative models into conservative MLIPs for deployment.The authors argue architectural inductive biases are not the only route to sound MLIP foundation models.Source: §4 · p. 18
  • Pre-training may draw on larger low-level datasets, unlabeled structures, chemoinformatics, relevant text, and experimental multimodal data, though curation and scalable training strategies remain necessary.The authors do not restrict possible pre-training to DFT-labeled data.Source: §4 · p. 18
  • Current leaderboards do not explicitly assess scaling with data or parameters or standardized fine-tuning across domains, and lower force errors do not always correlate with MD performance.The authors also identify efficient data retrieval, graph-size load balancing, and communication-hiding techniques as important for rapid training.Source: §4 · pp. 18–19
  • Progress also depends on access to open data, software, and models, alongside substantial computing resources and cooperation across government, industry, and countries.The paper notes that some commercial DFT datasets have not been released for public use.Source: §4 · p. 19

6 AUTHOR CONTRIBUTIONS STATEMENT

The statement assigns responsibility for the perspective’s scope, figures, writing, and manuscript review.

6 AUTHOR CONTRIBUTIONS STATEMENT

  • S.M.B., B.C., A.K., and T.H.-G. defined the topics and scope of the perspective.Source: §6 · p. 35
  • E.C.-Y.Y., Y.L., J.C., S.R., T.K., S.V., and W.X. made the figures.Source: §6 · p. 35
  • The listed authors wrote the paper, and all authors discussed the results and edited the manuscript.Source: §6 · p. 35

9 Short Summary

The short summary reviews foundation models in language and vision and proposes scaling physics-infused machine learned interaction potentials for chemistry and materials science.

9 Short Summary

  • The paper examines the historical development and underlying principles of foundation models in language and vision.Source: §9 · p. 36
  • It proposes scaling physics-infused machine learned interaction potentials to create foundation models for chemistry and materials science.Source: §9 · p. 36
  • The proposed models are intended to transform chemistry and materials science at scale.Source: §9 · p. 36

Related papers

Loading 2503.10538v3…