Source-linked AI summary
Orb: A Fast, Scalable Neural Network Potential
Mark Neumann, James Gin, Benjamin Rhodes, Steven Bennett, Zhiyi Li, Hitarth Choubisa, Arthur Hussey, Jonathan Godwin
TL;DR
Materials design needs methods that retain useful accuracy while scaling beyond the limits of slow ab initio calculations. This paper introduces Orb, universal interatomic potentials built with scalable graph learning and diffusion pretraining, and evaluates them across optimization and simulation tasks. Orb achieves strong benchmark performance, faster execution, and stable simulations across diverse and out-of-distribution systems, while some applications still require additional data or larger evaluations.
Problem
Ab initio materials design is slow and poorly scalable, while molecular-dynamics instability can remain weakly correlated with force and energy errors.
Method
Orb combines a scalable graph neural network with diffusion pretraining, supervised energy-force-stress finetuning, and inference-time force corrections.
Results
Orb set a new state of the art on Matbench Discovery, is 2-6 times faster than close competitors depending on system size, and remains stable across diverse systems including out-of-distribution small molecules.
Takeaways & Limitations
Orb provides a general-purpose potential for geometry optimization and simulations that can scale to systems and domains challenging for existing neural network potentials.
Takeaways & Limitations
MOF adsorption results suggest that additional fine-tuning data may be needed for quantitatively correct heats of adsorption, and the MD17 comparison is based on only four simulations.
Abstract
from arXiv · showhide
We introduce Orb, a family of universal interatomic potentials for atomistic modelling of materials. Orb models are 3-6 times faster than existing universal potentials, stable under simulation for a range of out of distribution materials and, upon release, represented a 31% reduction in error over other methods on the Matbench Discovery benchmark. We explore several aspects of foundation model development for materials, with a focus on diffusion pretraining. We evaluate Orb as a model for geometry optimization, Monte Carlo and molecular dynamics simulations.
1 Introduction
Orb addresses the slow, poor scaling of ab initio materials design with a scalable graph neural network approach that learns atomic interactions and invariances from data. The resulting universal potentials combine broad applicability with faster computation and strong benchmark performance.
- Motivation: Ab initio methods for inorganic materials design are accurate but slow and poorly scalable for realistically sized systems.Deep learning is presented as a route toward ab initio accuracy with greater speed and scalability.
- Approach: Orb adapts scalable deep learning to materials modeling through a graph neural network that learns atomic interactions and invariances from data.This avoids imposing architectural constraints based on rotational equivariance or particular group symmetries.
- Contribution: Orb is a suite of pretrained machine-learning force fields designed as general-purpose universal interatomic potentials.The models are released under the Apache 2.0 license for research and commercial use.
- Results: 3-6 times faster across commodity hardware, Orb models outperformed existing methods on the Matbench Discovery Leaderboard upon release.Orb also includes a learned-dispersion model for materials where van der Waals forces are important.
2 Model
Orb combines graph-based atomic representations, a three-stage neural architecture, diffusion pretraining, and supervised potential finetuning. Its direct force prediction enables a single inference pass, while force corrections impose zero-net-force and zero-net-torque constraints for suitable systems.
- Model Architecture: The backbone comprises an Encoder, a message-passing Processor with smoothed graph attention, and a Decoder using independent MLPs for energy, forces, and stress.The Processor aggregates edge residuals through scalar cutoff gates before the Decoder projects node representations to predicted system quantities.
- Graph Construction: Orb represents atomic systems as graphs whose node embeddings encode atomic types and whose directed edges encode displacement vectors within a periodicity-aware cutoff.Node embeddings omit absolute positions to preserve translational invariance, while the unit cell is implicit in edge construction.
- Graph Construction: Gaussian radial basis functions encode interatomic distances, with centers and widths controlling the basis functions’ locations and scales.Cutoff functions preserve continuity under small perturbations of displacement vectors.
- Model Architecture: A single forward pass predicts energy, forces, and stress because Orb is not implemented as a conservative vector field.Inference-time mean and torque adjustments enforce zero net force and, for non-periodic systems, zero net torque.
- Training: Orb is pretrained as a denoising diffusion model and then finetuned as a neural network potential on energies, forces, and stresses from DFT optimization trajectories.The diffusion stage adds progressively increasing Gaussian noise and learns score functions, while finetuning uses weighted energy, force, and stress losses.
- Training: Pretraining combines minimum-energy configurations across datasets to broaden coverage, whereas finetuning prioritizes data quality and consistency across DFT settings.Orb-D3 models amortize dispersion-correction costs by training on datasets corrected with D3; standard comparisons generally omit Orb-D3 for comparability.
- Training: Up to 5000 atoms, Orb’s diffusion pretraining includes systems substantially larger than those used by the comparable MatterGen model.The figure describes element and system-size distributions for the full diffusion pretraining dataset.
3 Results
Orb achieves strong accuracy across geometry optimization and molecular-dynamics evaluations while improving computational speed and scalability. Diffusion pretraining improves force-field metrics, and zero-shot simulations remain competitive despite limited evidence from only four molecules.
- Matbench-Discovery: Orb models set a new state of the art on Matbench Discovery’s primary F1 metric and are generally superior across other metrics.Orb also has especially high precision, which is relevant when false-positive stability predictions are costly.
- Matbench-Discovery: 17% to 70% improvements from diffusion pretraining appear across Alexandria and MPtraj energy and force metrics.The gains occur even on Alexandria, which is an order of magnitude larger than MPtraj.
- Speed Benchmark: 3 to 6 times faster than MACE at large system sizes, Orb improves scaling for simulations involving dopants or sparse statistics such as diffusivity.The speed comparison measures model forward passes on a single NVIDIA A100 GPU.
- MD17-10k: molecule specific results: 300 ps stability was achieved for all four MD17 molecules, matching NequIP and exceeding the other baseline models.Orb’s simulated trajectories also closely match reference data through low h(r).
- MD17-10k: zero-shot results: In zero-shot MD17, Orb trained on MPtraj has the lowest h(r) across the evaluated models and outperforms many molecule-specific baselines.The evaluation extrapolates from zero-kelvin periodic crystals to unseen high-temperature non-periodic molecules without further finetuning.
- MD17-10k: zero-shot results: Net Torque Removal substantially improves molecular-dynamics quality while having minimal effect on Force MAE.The correction is applied at inference time to non-periodic systems because Orb directly predicts forces rather than deriving them from an energy gradient.
3.5 Molecular Dynamics of Crystalline Materials
Orb is evaluated on thermally stressed MOF-5 simulations and produces stable trajectories across increasing temperatures. The framework remains structurally stable until about 3,000 ps at 800 K, while linker dynamics remain consistent with reported behavior.
- Thermal stability: MOF-5 remains stable across temperatures from 300 K to 800 K, with RMSD relatively constant until approximately 3,000 ps.Beyond 3,000 ps, significant framework degradation occurs into gaseous constituents.
- Linker dynamics: Orb reproduces the experimentally observed restriction against complete phenylene-ring rotation between 300 K and 800 K.Higher temperatures broaden the dihedral-angle distribution, indicating greater rotational flexibility from increased thermal energy.
- Generalization: The MOF-5 evaluation extends Orb beyond small-molecule molecular dynamics to bulk crystalline materials despite no explicit training on MOFs.The observed linker dynamics agree with literature observations and suggest use for investigating mechanical properties of larger crystalline materials.
3.6 Adsorption
Orb-D3 is evaluated for low-pressure CO2 adsorption in Mg-MOF-74 using Widom insertion. It identifies favorable adsorption regions and produces adsorption geometries and energies that are qualitatively consistent with experiment, while quantitative heat predictions may require further fine-tuning.
- Method: Widom insertion with Orb-D3 computes the heat of adsorption and free-energy landscape of CO2 in Mg-MOF-74.The evaluation targets a framework with accessible open metal sites along one-dimensional hexagonal channels.
- Free-energy landscape: Both Orb-D3 and MACE-D3 predict favorable adsorption regions near the open metal centers, but MACE produces noticeably deeper potential wells.MACE’s deeper wells yield a heat of adsorption of -61.7 kJ/mol versus the experimental value of -44 kJ/mol, while Orb-D3’s wells are closer to experiment.
- Adsorption geometry: Orb identifies a tilted CO2 configuration with a Mg-O-C bond angle of 134.7° and a Mg-O(CO2) bond length of 2.43 Å.These values are close to the experimental angle of 131° and distance of 2.27 Å.
- Generalization and scope: Neither Orb nor MACE was trained on MOF structures, yet both produce qualitatively correct energy landscapes and adsorption dynamics.The authors state that further fine-tuning may be required for quantitatively correct heats of adsorption in Monte Carlo sampling.
4 Related Work
Related work frames Orb against interatomic potentials that often have limited system domains, instability in molecular dynamics, and costly symmetry-aware architectures. The paper positions diffusion pretraining and scalable non-equivariant modeling as responses to these challenges.
- Generalization limits: Earlier interatomic potentials were often confined to small systems and struggled to generalize across material classes.MACE is identified as an early universal interatomic potential with significant generalization attributes.
- Architectural trade-offs: Equivariant architectures encode rotational symmetries but require specialized design rules and tensor operations that are difficult to implement efficiently.Orb instead adopts a non-equivariant graph neural network architecture.
- Alternative symmetry strategies: Data augmentation and auxiliary-network approaches have been explored to introduce rotational equivariance or invariance at inference time.These approaches complement work investigating unconstrained models for three-dimensional point-cloud data.
- Simulation stability: Molecular-dynamics instability can cause nonphysical states and simulation collapse, while force and energy MAE correlate weakly with this failure mode.Active-learning remedies require expensive cycles of simulations, fine-tuning, and quantum-mechanical calculations.
- Pretraining: Prior atomistic pretraining commonly focused narrowly on materials or molecules and used architectures with limited data scaling as training data increased.The paper attributes one reason to the scarcity of compatible DFT data and motivates combining available datasets through generative pretraining.
5 Conclusion
The paper presents Orb as a universal interatomic potential that combines benchmark performance, computational speed, simulation stability, and generalization across chemistries and domains. It also evaluates relaxed symmetry assumptions while developing tests for out-of-distribution simulation behavior.
- Overall performance: Orb set a new state of the art on Matbench Discovery when released and remains 2–6 times faster than its closest competitors depending on system size.The paper also reports stability under simulation for diverse systems, including out-of-distribution small molecules.
- Model design: The study examines approximations to conservative force fields and equivariant models while exploiting the benefits of relaxing those assumptions.The authors note that theoretically preferable equivariant conservative models do not guarantee the best learned interatomic potentials.
- Generalization evaluation: Orb generalizes to new domains with varied chemistries through zero-shot evaluation of molecular-dynamics stability and descriptive statistics.The evaluation modifies MD-17 to test generalization for force-field models.
- Practical scope: The authors aim for Orb models to support computational-chemistry simulations that were difficult or impossible to run at the scale required for useful analysis.This stated goal follows the paper’s demonstrations of speed, stability, and cross-domain generalization.
6 Appendix
Orb combines force-prediction models with inference-time corrections designed to improve physical stability, especially in molecular dynamics. The implementation and release details include open licensing, training procedures, and graph construction choices.
- Orb models are released under an Apache 2.0 license, with code and weights available on GitHub.
- Models use diffusion pretraining and supervised finetuning, with random rotations and neighbor graphs capped at 20 neighbors within 10 Å.
- Orb directly predicts force vectors rather than obtaining them as energy gradients, which can produce unstable molecular dynamics trajectories.Unconstrained predictions may create physically impossible net translational or rotational forces.
- Mean-subtraction removes net force, while constrained torque removal eliminates net rotation in non-periodic systems.Torque removal is formulated as an efficient constrained optimization problem using Lagrange multipliers.
- Inference-time torque correction substantially improves molecular-dynamics quality with minimal effect on force MAE.The correction is inexpensive and does not require retraining.
Problem Statement
The force-adjustment problem seeks the smallest additive changes that preserve zero average adjustment while canceling the original forces’ net torque. Positions are measured relative to the system center of mass.
- Given N atomic positions and predicted forces, the method seeks additive force adjustments with minimal L2 norm.
- The average net adjustment must be zero.
- The adjustment’s net torque must cancel the net torque of the original predicted forces.
- Each relative position is defined from the atom’s position and the system center of mass.The predicted net torque is computed from these relative positions and the predicted forces.
Solving the Constrained Minimization Problem
The constrained minimization is solved with Lagrange multipliers: first enforce zero average adjustment, then solve the torque constraint using a small linear system. A pseudoinverse handles singular cases.
- Lagrange multipliers are introduced to solve the constrained force-adjustment problem.
- The zero-average-adjustment constraint determines the multiplier associated with the net-force condition.
- The torque multiplier is solved from the torque constraint after applying a triple-vector-product identity.
- The resulting linear system uses a 3×3 identity matrix and assumes invertibility of S − sI.When the matrix is singular or nearly singular, the Moore-Penrose pseudoinverse is used.
Final expression for adjusted forces
The paper applies the adjusted-force framework in non-periodic molecular-dynamics simulations and evaluates Orb across adsorption, free-energy, thermal-stability, and related-work comparisons. The reported methods include Widom insertion and MOF-5 simulations.
- Final expression for adjusted forces: Adjusted forces with zero net torque are used for non-periodic systems and are computed by minimizing a Lagrangian under force and torque constraints.
- Applications: MOF-5 thermal stability is evaluated with constant-volume NPT molecular dynamics from 300 K to 1,000 K using a 1 fs timestep.
- Applications: Widom insertion estimates adsorption energy by averaging energy changes from random CO2 insertions.An exclusion sphere avoids close-range interactions, and D3 corrections are included for MACE and Orb.
- Applications: Free energies are computed on a 20 x 20 grid with linear interpolation, using calculations conducted at 298 K.
- Related work: EquiformerV2 models trained on OMAT 24 surpass Orb-v2 on Matbench Discovery, while comparable models without DeNS perform similarly to Orb-v2 with fewer training resources.