E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials

Simon Batzner; Albert Musaelian; Lixin Sun; Mario Geiger; Jonathan P. Mailoa; Mordechai Kornbluth; Nicola Molinari; Tess E. Smidt; Boris Kozinsky

Source-linked AI summary

E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials

Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, Boris Kozinsky

arXiv:2101.03164v3Jan 8, 2021physics.comp-phcond-mat.mtrl-scics.LG

TL;DR

Molecular dynamics is limited by the computational cost of first-principles forces, motivating accurate interatomic potentials that are practical to evaluate. NequIP uses E(3)-equivariant convolutions and tensor features to learn such potentials, achieving state-of-the-art accuracy and exceptional data efficiency across molecules and materials, including with fewer than 1,000 or as few as 100 reference calculations.

Problem
The computational cost of first-principles forces limits molecular dynamics simulations to short time scales and small systems, restricting accessible physical phenomena.
Method
NequIP learns interatomic potentials with E(3)-equivariant graph convolutions over tensor features including scalars, vectors, and higher-order tensors.
Results
NequIP achieves state-of-the-art accuracy and exceptional data efficiency across small molecules and periodic materials, using fewer than 1,000 or as few as 100 reference calculations.
Takeaways & Limitations
NequIP's data efficiency facilitates developing accurate interatomic potentials from expensive high-order quantum chemical reference calculations.
Takeaways & Limitations
The resulting dynamics for formate dehydrogenation catalysis are left for a separate study.

Abstract

from arXiv · show

This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations. While most contemporary symmetry-aware models use invariant convolutions and only act on scalars, NequIP employs E(3)-equivariant convolutions for interactions of geometric tensors, resulting in a more information-rich and faithful representation of atomic environments. The method achieves state-of-the-art accuracy on a challenging and diverse set of molecules and materials while exhibiting remarkable data efficiency. NequIP outperforms existing models with up to three orders of magnitude fewer training data, challenging the widely held belief that deep neural networks require massive training sets. The high data efficiency of the method allows for the construction of accurate potentials using high-order quantum chemical level of theory as reference and enables high-fidelity molecular dynamics simulations over long time scales.

INTRODUCTION · Related Work · RESULTS

NequIP is an E(3)-equivariant, energy-conserving neural interatomic potential that uses geometric tensors to achieve accurate and highly data-efficient modeling for molecules and materials. The approach addresses the computational and data demands limiting first-principles molecular dynamics by enabling accurate potentials from fewer than 1,000, and sometimes as few as 100, reference calculations.

INTRODUCTION: Molecular dynamics is valuable across energy storage, catalysis, and biology, but first-principles force calculations restrict simulations to short timescales and small systems.The limitation arises from the unfavorable computational scaling of quantum-mechanical methods such as density functional theory.
INTRODUCTION: Machine-learned interatomic potentials seek to combine high-fidelity ab-initio reference calculations with favorable computational efficiency, but neural-network potentials often require large training sets.This training-data requirement is identified as a limiting factor of NN-IPs.
INTRODUCTION: Fewer than 1,000 or even as little as 100 reference ab-initio calculations can suffice for NequIP to construct accurate interatomic potentials, whereas other methods require orders of magnitude more.The reported data efficiency is presented alongside high accuracy relative to existing approaches.
INTRODUCTION: On small molecular data sets, NequIP outperforms other neural networks and is competitive with kernel-based approaches that usually provide better predictive accuracy at greater training and prediction cost.The comparison concerns predictive accuracy and computational cost on small data sets.
Related Work: Unlike descriptor-based and scalar-focused approaches, NequIP uses relative position vectors and features containing higher-order geometric tensors.These design choices provide a representation beyond distances and scalar descriptors.
Related Work: NequIP’s internal features are equivariant to rotation, allowing angular information to enter rotationally equivariant filters, while convolutions can remain local within cutoff distance r_c.The local interaction subset includes atoms closer to the central atom than the chosen cutoff distance.
Related Work: DimeNet incorporates angular three-body interactions but retains scalar features, whereas Cormorant applies equivariant networks to property prediction and demonstrates potential energies for small molecules.The comparison distinguishes DimeNet’s invariant features from the vector-based features used in NequIP.
Related Work: The proposed potential uses E(3)-equivariant convolutions over geometric tensors to provide state-of-the-art accuracy, outstanding data efficiency, and high-fidelity reproduction of structural and kinetic molecular-dynamics properties.The method is described as a deep learning, energy-conserving interatomic potential for both molecules and materials.

Equivariance · Neural Equivariant Interatomic Potentials

NequIP models atomic environments with E(3)-equivariant tensor features and convolutions, while predicting an invariant total potential energy. Its construction preserves physical transformation properties, guarantees energy-conserving forces, and uses local atomic interactions for efficient evaluation.

Equivariance: Equivariant neural networks preserve known physical transformation properties under coordinate changes and represent tensor properties and operations such as vector addition, dot products, and cross products.NequIP focuses on E(3), comprising rotations, reflections, and translations in three-dimensional space.
Neural Equivariant Interatomic Potentials: NequIP maps atomic positions and chemical species to total potential energy and forces by summing predicted atomic energies and differentiating the total energy with respect to positions.This construction guarantees energy conservation.
Neural Equivariant Interatomic Potentials: Unlike scalar-valued invariant GNN-IPs, NequIP predicts an invariant potential energy using internal geometric tensor features that are equivariant to rotation and reflection.The atomic local energies are scalar node attributes predicted by the graph neural network.
Neural Equivariant Interatomic Potentials: NequIP builds on Tensor-Field Network layers and associates each atom with scalar, vector, and higher-order tensor features that transform equivariantly to translation, parity, and rotation.Feature rotation order l = 0, 1, 2, ... and parity p ∈ (1, −1) label irreducible representations of O(3).
Neural Equivariant Interatomic Potentials: The convolution filters use relative interatomic vectors, making convolutions translation invariant and permutation invariant through summation over neighboring atoms, while equivariant operations preserve tensor structure.Input features and filters are combined through geometric tensor products using Clebsch-Gordan coefficients.
Neural Equivariant Interatomic Potentials: Local cutoffs restrict interactions to nearby atoms, causing evaluation cost to scale linearly with the number of atoms.Periodic systems use ASE neighbor lists to identify appropriate atomic neighbors.
Neural Equivariant Interatomic Potentials: The architecture comprises an atomic embedding, interaction blocks for neighboring-atom interactions, and an output block that processes final l = 0 features.Interaction blocks mix matching rotation-parity channels and use a ResNet-style update; the output produces one scalar atomic energy per atom.

Experiments

The experiments evaluate NequIP across diverse and challenging datasets, including small organic molecules, CCSD(T)-level force learning, and applications beyond isolated molecules. The method improves state-of-the-art accuracy on MD-17 and accurately learns forces at the CCSD(T) level.

Experimental validation: NequIP is validated on a diverse series of challenging datasets spanning small organic molecules, high-level quantum chemistry, and systems beyond isolated molecules.The experiments proceed from MD-17 to CCSD(T)-level forces and then broaden applicability beyond small isolated molecules.
Experimental validation: NequIP improves upon state-of-the-art accuracy on MD-17, a benchmark dataset of small organic molecules.MD-17 is widely used for benchmarking machine-learning interatomic potentials.
Experimental validation: NequIP accurately learns forces for small molecules obtained at the quantum chemical CCSD(T) level of theory.This experiment tests learning from high-level quantum chemical reference forces.

MD-17 small molecule dynamics

On the revised MD-17 benchmark of small organic molecules with DFT reference energies and forces from ab-initio molecular dynamics, NequIP outperforms all other evaluated methods. Increasing tensor rank beyond l = 1 consistently improves accuracy, with a significant gain from l = 0 to l = 1.

Benchmark: NequIP is evaluated on MD-17, a small-organic-molecule data set with DFT reference energies and forces generated by ab-initio molecular dynamics.A recomputed version with higher numerical accuracy is termed the revised MD-17 data set.
Results: NequIP outperforms all other methods on the MD-17 benchmark.Its consistent accuracy improvements over sGDML and FCHL19/GPR are particularly notable because those kernel methods typically perform better than deep neural networks on small training sets.
Architecture ablation: Increasing tensor rank beyond l = 1 consistently improves accuracy, with a significant improvement from l = 0 to l = 1.The convergence scan covers rotation orders l ∈{0, 1, 2, 3}.

Force training at quantum chemical accuracy

NequIP’s data efficiency facilitates accurate machine-learned interatomic potentials from expensive, high-order quantum-chemical reference calculations. The study evaluates this capability on molecules computed at CCSD or CCSD(T) accuracy and compares it with sGDML and GemNet.

Motivation: NequIP’s high data efficiency enables development of interatomic potentials using expensive, high-order ab-initio quantum-chemical methods.The passage identifies coupled cluster calculations, including CCSD(T), as an example of such methods.
Experimental setup: The evaluation uses a molecular dataset computed at quantum-chemical accuracy, with aspirin at CCSD and all other molecules at CCSD(T).These reference calculations are used to assess NequIP’s performance.
Experimental setup: The results are compared with previously reported results for sGDML and GemNet in table III.The comparison concerns the same quantum-chemical-accuracy molecular evaluation.

Liquid Water and Ice Dynamics

NequIP was evaluated on periodic liquid-water and ice systems and significantly outperformed DeepMD on force errors despite using 1,000-fold fewer training structures. Its liquid-phase performance remained competitive, while increasing energy weighting improved energy accuracy at a small force-error cost.

Benchmark systems: The benchmark jointly covers liquid water and three ice structures computed at the PBE0-TS level with periodic boundary conditions.The liquid-water and ice data include conditions such as 1 bar and temperatures of 273–330 K.
Data efficiency: 1,000x fewer training data enabled NequIP to significantly outperform DeepMD on force-component errors across all four parts of the water-and-ice data set.NequIP used 133 structures, whereas DeepMD used 133,500.
Data efficiency: NequIP’s results on the liquid phase were surprisingly competitive despite DeepMD’s much larger training set.The passage notes that the difference was even stronger on energies than on force components because each frame provides 3N force targets but only one energy target.
Loss weighting: Increasing the energy weighting in the loss function significantly improved energy errors while causing only a small increase in force errors.The reported models vary the relative energy and force weights, including λF = 1, λE = 0; λF = 100, λE = 1; and λF = 100,000, λE = 1.

Heterogeneous catalysis of formate dehydrogenation · Lithium Phosphate Amorphous Glass Formation

NequIP is applied to heterogeneous formate dehydrogenation on Cu(110) and to structural dynamics in amorphous Li4P2O7. It reproduces amorphous-glass structural distributions compared with AIMD after training on only 1,000 structures.

Heterogeneous catalysis of formate dehydrogenation: NequIP investigates formate dehydrogenation, HCOO∗→H∗+ CO2, on a Cu(110) surface.The study examines the dynamics of formate undergoing dehydrogenation decomposition.
Heterogeneous catalysis of formate dehydrogenation: The Cu(110) reaction is challenging because it combines metallic and covalent bonding with charge transfer between the metal and molecule.Different molecular states also produce dissimilar C-O interactions.
Heterogeneous catalysis of formate dehydrogenation: A more detailed analysis of the resulting formate-dehydrogenation dynamics is deferred to a separate study.The supplied passage identifies this analysis as outside the present report.
Lithium Phosphate Amorphous Glass Formation: NequIP describes structural dynamics in amorphous lithium phosphate, Li4P2O7, a solid-electrolyte material with non-trivial Li-ion transport and phase-transformation behaviors.The dataset includes two 50ps-long AIMD simulations: molten structure at T=3000 K and quenched glass structure.
Lithium Phosphate Amorphous Glass Formation: The analysis evaluates radial distribution functions and P-O-O and O-P-P angular distribution functions averaged over ten runs against ab-initio trajectories at the same temperature.P-O-O is the tetrahedral bond angle, while O-P-P is a bridging angle between corner-sharing phosphate tetrahedra.
Lithium Phosphate Amorphous Glass Formation: 1,000 structures suffice for NequIP to accurately reproduce the RDF and two ADFs compared with AIMD.The result demonstrates accurate reproduction of these structural distributions after limited training.

Lithium Thiophosphate Superionic Transport

NequIP is evaluated on Li-ion diffusivity in crystalline superionic LiPS, testing whether small training sets can support accurate kinetic-transport modeling. Models trained on as few as 10–2,500 structures are assessed through molecular-dynamics simulations, including a 50 ps NVT run with the 2,500-structure model.

Study system: LiPS (Li6.75P3S11) is a crystalline superionic lithium conductor used to study Li-ion diffusivity in an 83-atom simulation cell.The study targets kinetic transport properties, for which prior machine-learning potentials had required large training sets.
Training-set efficiency: Training sets of 10, 100, 1,000, and 2,500 structures are used to evaluate whether very small datasets yield highly accurate models.The results are presented in Table VI.
Simulation protocol: 50 ps NVT molecular-dynamics simulations at the AIMD temperature use a 0.25 fs timestep with the NequIP potential trained on 2,500 structures.The timestep was selected to improve the reliability and stability of long simulations.

Data Efficiency

NequIP demonstrates exceptional data efficiency, with equivariant networks outperforming invariant networks across training-set sizes and achieving competitive force accuracy with far fewer structures than BPNN. Its learning curves also exhibit behavior linked to the use of equivariant tensor features.

Data Efficiency: For M calculations on N-atom structures, energies and force components provide M(3N + 1) training labels.Each configuration contributes one energy label and 3N force-component labels.
Data Efficiency: Across 10–1,000 bulk-water structures, equivariant networks with l ∈ 1, 2, 3 significantly outperform invariant l = 0 networks in force-component MAE.The result supports a connection between data efficiency and tensor features or equivariance.
Learning Curves: Equivariant-network learning curves have a different log-log slope, whereas prior results typically find a fixed power-law exponent across algorithms for a given data set.The learning-curve exponent determines how quickly generalization error decreases as training data increase.
Comparison with Existing Methods: With 100 and 250 data points, NequIP l = 2 reaches force RMSEs of 129.8 meV/˚A and 103.4 meV/˚A, compared with BPNN’s ≈120 meV/˚A from 1303 structures.The BPNN result was evaluated on 290 remaining structures, while Figure 7 reports NequIP force MAE rather than RMSE.

DISCUSSION

NequIP uses E(3)-equivariant convolutions to achieve state-of-the-art accuracy and exceptional data efficiency on small-molecule and periodic-material datasets. The improvements are attributed to equivariant rather than invariant representations.

Method: NequIP is a graph neural network built on E(3)-equivariant convolutions.The method is introduced as a new type of neural equivariant interatomic potential.
Performance: NequIP exhibits state-of-the-art accuracy on datasets of small molecules and periodic materials.The reported accuracy spans both molecular and periodic-material datasets.
Performance: NequIP exhibits exceptional data efficiency on datasets of small molecules and periodic materials.The discussion identifies data efficiency as a second central performance outcome alongside accuracy.
Interpretation: The improvements are attributed to equivariant representations replacing more widely used invariant representations.The discussion isolates the introduction of equivariant representations as the source of the improvements.

METHODS

The methods evaluate NequIP across molecular, aqueous, reactive, and solid-state datasets, training networks with a size-invariant weighted energy–force loss and tailored normalization. Molecular dynamics integrates NequIP forces through ASE with a custom Nosé–Hoover thermostat.

Datasets: NequIP is evaluated on MD-17, revised MD-17, CCSD/CCSD(T) molecules, water and ice, formate decomposition on Cu, Li4P2O7 glass, and LiPS materials.The datasets span organic molecules, aqueous phases, a surface reaction, and lithium-containing solid-state systems.
Molecular Dynamics: NequIP force outputs are integrated with ASE using a custom Nosé–Hoover thermostat for Li4P2O7 and LiPS molecular-dynamics simulations.The thermostat parameter is chosen to match temperature fluctuations observed in the corresponding ab-initio molecular-dynamics run.
Training: Training uses a weighted sum of energy and force loss terms, with a default energy-to-force relative weighting of 1 to N^2_atoms.The N^2_atoms factor accounts for global energies, local forces, and the use of mean-squared-error loss, making the loss size invariant.
Normalization: Target energies are mean-centered, while energies and force components are scaled by the training-set root mean square of force components.Predicted atomic energies additionally use learnable per-species scale and shift parameters initialized to 1 and 0.
Normalization: For joint water-and-ice training, target energies and forces are not scaled or shifted because the structures contain different numbers of atoms.Learnable per-species shifts initialize to mean per-atom energy, and scales initialize to the average force-component standard deviation.

APPENDIX A: LONG MOLECULAR DYNAMICS SIMULATION OF LI4P2O7 · APPENDIX B: LEARNING CURVES · APPENDIX C: REVISED MD-17 DATA SET

The appendices document a 500 ps NequIP molecular-dynamics comparison with 50 ps AIMD, learning curves for energy and force errors, and label distributions for revised MD-17 aspirin data. These materials cover simulation evaluation, training-set scaling, controlled network comparisons, and dataset characterization.

APPENDIX A: LONG MOLECULAR DYNAMICS SIMULATION OF LI4P2O7: The Li4P2O7 RDF comparison evaluates a 500 ps NequIP simulation against a shorter 50 ps AIMD trajectory.Both simulations omit their first 10 ps when computing the RDF.
APPENDIX A: LONG MOLECULAR DYNAMICS SIMULATION OF LI4P2O7: 500 ps of Li4P2O7 dynamics driven by NequIP is compared with a 50 ps AIMD simulation using radial distribution functions.The first 10 ps of both simulations were excluded from RDF computation.
APPENDIX B: LEARNING CURVES: Energy errors are plotted against training-set size for the water data set from [50], measured using energy MAE for NequIP with l ∈{0, 1, 2, 3}.The learning-curve plot uses a log-log presentation of predictive error.
APPENDIX B: LEARNING CURVES: Force errors are plotted against data-set size for aspirin in MD-17, comparing NequIP with l ∈{0, 1, 2, 3} against weight- and feature-controlled versions.The force learning curves use force MAE and are presented on a log-log scale.
APPENDIX C: REVISED MD-17 DATA SET: The revised MD-17 appendix presents an energy-label histogram for all aspirin structures used for training and validation.The mean energy was subtracted before plotting.
APPENDIX C: REVISED MD-17 DATA SET: A separate histogram shows force components for all aspirin structures used for training and validation in the revised MD-17 data set.The figure characterizes the distribution of force labels without reporting an additional numerical summary.