PaperlayerView exact revision on arXiv

Source-linked AI summary

NewtonNet: A Newtonian message passing network for deep learning of interatomic potentials and forces

Mojtaba Haghighatlari, Jie Li, Xingyi Guan, Oufan Zhang, Akshaya Das, Christopher J. Stein, Farnaz Heidar-Zadeh, Meili Liu, Martin Head-Gordon, Luke Bertels, Hongxia Hao, Itai Leven, Teresa Head-Gordon

arXiv:2108.02913v1physics.chem-ph

TL;DR

Predicting molecular energies and forces with equivariant models raises challenges in representing directional information while keeping computation and data demands manageable. NewtonNet uses Newtonian-inspired message passing with latent force vectors and reports competitive or better performance than state-of-the-art models across molecular and reactive chemistry tasks.

  • Problem

    Predicting energies and force vectors requires representations that handle rotational transformations, while equivariant models can carry computational, feature-availability, and interpretability costs.

    Source: INTRODUCTION · p. 2

  • Method

    NewtonNet is a geometric message-passing network inspired by Newton’s equations, using latent force vectors and physics-infused operators while maintaining rotational equivariance.

    Source: INTRODUCTION · p. 3, THEORY · p. 4

  • Results

    NewtonNet achieves more accurate or competitive performance than state-of-the-art invariant and equivariant models, including on reactive combustion trajectories.

    Source: CONCLUSIONS · p. 12

  • Takeaways & Limitations

    The authors report accuracy without much computation or memory overhead and describe a more interpretable latent space for predictions.

    Source: CONCLUSIONS · p. 12

Abstract

from arXiv · show

We report a new deep learning message passing network that takes inspiration from Newton's equations of motion to learn interatomic potentials and forces. With the advantage of directional information from trainable latent force vectors, and physics-infused operators that are inspired by the Newtonian physics, the entire model remains rotationally equivariant, and many-body interactions are inferred by more interpretable physical features. We test NewtonNet on the prediction of several reactive and non-reactive high quality ab initio data sets including single small molecule dynamics, a large set of chemically diverse molecules, and methane and hydrogen combustion reactions, achieving state-of-the-art test performance on energies and forces with far greater data and computational efficiency than other deep learning models.

INTRODUCTION

NewtonNet introduces a geometric message-passing network inspired by Newton’s laws, using latent force vectors to preserve rotational equivariance. It targets limitations of invariant-only representations and aims to reduce computational cost and data needs while maintaining strong performance.

INTRODUCTION

  • Earlier atom-centric models predict molecular energies and forces using hand-crafted symmetry functions or trainable message-passing operators.The passage describes ANI-1 as containing more than 22 million DFT-evaluated configurations.Source: INTRODUCTION · p. 2
  • Invariant-only models can fail to distinguish systems even with up to four-body information, while force-vector prediction requires outputs to transform equivariantly.Rotational transformations are especially challenging for learning global orientations across molecular dynamics trajectories.Source: INTRODUCTION · p. 2
  • Equivariant models can improve predictive performance and reduce reference-data needs, but computational cost, precomputed features, and limited interpretability remain concerns.The paper identifies reducing the rank of equivariant features while retaining test accuracy as a design need.Source: INTRODUCTION · p. 3
  • NewtonNet uses Newtonian-inspired latent force vectors to preserve rotational equivariance, with more favorable computational cost and performance on reactive and non-reactive chemistry.The authors report training-data requirements reduced to 1–10% of those for invariant-only models.Source: INTRODUCTION · pp. 3–4

THEORY

NewtonNet iteratively updates atomic features alongside latent force and displacement features through modules inspired by Newton’s equations. Its energy module contracts directional features into rotationally invariant atomic features.

THEORY

  • Each layer updates atomic features and constructs force and displacement feature vectors, which are contracted into feature arrays for energy prediction.The model uses rotationally equivariant message-passing operations while keeping atomic features invariant to input rotations.Source: THEORY · p. 4
  • The atomic feature aggregator uses symmetric pair messages, sums neighbor messages, and updates each atom’s features.Atomic features are initialized from atomic numbers, and the edge operation uses radial basis functions and a continuous cutoff.Source: THEORY · pp. 4–5
  • The force module predicts symmetric force magnitudes, combines them with unit interatomic directions to obtain antisymmetric forces, and sums neighbor contributions.The final latent force vectors can be included in the loss to encourage the latent space to mimic physical rules.Source: THEORY · pp. 5–6
  • The displacement module learns atomic displacement vectors and updates displacement features using weighted sums of prior-layer displacements.The weights depend on messages between atoms, and the force and displacement features are initialized to zero.Source: THEORY · p. 6
  • The energy module contracts directional feature vectors into invariant features, representing atomic potential-energy changes.The dot product contracts the equivariant features, consistent with predicting rotation-invariant energies.Source: THEORY · p. 6

RESULTS

NewtonNet is evaluated on small-molecule dynamics, high-level electronic-structure data, chemically diverse molecules, and combustion reactions. Across these tests, it reports strong energy and force prediction performance, including comparisons with fewer training samples than competing models.

Single Small Molecules

  • On MD17, NewtonNet outperforms invariant deep-learning models and, for some molecules, equivariant models, while retaining chemical accuracy and computational efficiency.Training uses 950 samples per molecule; chemical accuracy is defined here as below 0.5 kcal/mol.Source: RESULTS · p. 7, Table 1 · p. 7

CCSD/CCSD(T) small molecules

  • NewtonNet outperforms the best reported prediction for three of five CCSD/CCSD(T) molecules and is competitive within uncertainty for the other two.It is also reported as robustly improved over the opponent kernel methods.Source: RESULTS · p. 8, Table 2 · p. 8

Small Molecules with large Chemical Variations

  • Using 2M ANI-1 samples, NewtonNet achieves 0.65 kcal/mol energy MAE, halving the error of ANNs trained on all 20M samples.The dataset provides energies but not atomic forces.Source: RESULTS · pp. 8–9, Table 3 · p. 8

Methane Combustion Reaction

  • With all methane-combustion training data, NewtonNet reaches 0.50 kcal/mol/atom energy MAE and 1.20 kcal/mol/Å force MAE, reducing errors by 85% and 57% versus the original model.With 1% of the data, it still outperforms the original DeepMD model trained on the full dataset.Source: RESULTS · p. 9, Table 4 · p. 9

Hydrogen Combustion Reaction

  • On hydrogen combustion, NewtonNet outperforms SchNet with 500 rather than 5,000 training samples per reaction and reaches chemical accuracy with 200 samples per reaction.The learning-curve figure averages energy and force MAEs across 16 independent reactions.Source: RESULTS · p. 10, Figure 2 · p. 10

DISCUSSION

NewtonNet’s latent force vectors increasingly align with reference forces across message-passing layers, while the model also reports lower training and inference times than NequIP in the stated comparisons.

DISCUSSION

  • NewtonNet constructs latent atomic forces to collect directional information for each atom.The paper attributes the vectors’ agreement with reference forces to appropriate loss functions and a latent-space design based on Newton’s second and third laws.Source: DISCUSSION · pp. 10–11
  • Average cosine distance between latent and reference forces falls from 1.17 to 0.05 across layers one to three, with the final vectors indicating learned force magnitudes and signs for pairwise interactions.The visualization shows the layerwise alignment for ethanol; the reported average cosine distances are for the test set.Source: DISCUSSION · p. 11, Figure 3 · p. 11
  • NewtonNet reaches state-of-the-art aspirin performance in 12 hours on a GeForce RTX 2080Ti, versus up to 8 days for NequIP on a faster Tesla V100.The comparison uses a straightforward comparison of similar-rank contributed tensors.Source: DISCUSSION · p. 11
  • For a 20-atom molecule, NewtonNet processes a trajectory snapshot in 4 milliseconds, compared with 16 milliseconds for a 15-atom molecule using NequIP.NewtonNet takes about 3 milliseconds on a Tesla V100; both reported prediction times are far below the minutes-to-hours scale of ab initio calculations.Source: DISCUSSION · p. 12

CONCLUSIONS

NewtonNet uses Newtonian-inspired geometric message passing to predict molecular energies and forces with competitive or better accuracy and computational efficiency. The reported results include data-efficient performance on CCSD(T) data and strong performance on combustion trajectories.

CONCLUSIONS

  • NewtonNet predicts molecular dynamics energies and forces more accurately than, or competitively with, state-of-the-art invariant and equivariant models.The conclusion identifies accurate prediction at an efficient timescale as important for studying chemical and biochemical systems.Source: CONCLUSIONS · p. 12
  • NewtonNet’s geometric message passing and rotationally equivariant latent space scale linearly with system size, supporting accuracy without much computation or memory overhead.The paper also notes that deep learning models such as NewtonNet can scale better than kernel methods and discusses active learning as a route to expanding training data.Source: CONCLUSIONS · p. 12
  • On CCSD(T) data for small organic molecules, NewtonNet is competitive or at least 10% better than state-of-the-art equivariant models while using significantly less training data.The authors frame this data efficiency as relevant to obtaining high accuracy levels associated with CCSD(T)/CBS using affordable data generation.Source: CONCLUSIONS · p. 12
  • NewtonNet performs with good chemical accuracy on combustion-reaction molecular dynamics trajectories, including the challenge of chemical reactivity.The authors propose higher-order tensor operations as a possible future extension for more distinguishable many-body features.Source: CONCLUSIONS · p. 12

APPENDIX A

The appendix argues that NewtonNet’s operators preserve rotational equivariance for vector features and invariance for atomic-energy predictions. Its proof relies on invariant distances and the equivariance of linear combinations of vectors or their products with invariant features.

APPENDIX A

  • Euclidean distance is invariant under rotation, so the feature arrays and functions acting on them yield invariant outputs.The proof then assumes linear combinations of vectors, and their products with invariant features, remain rotationally equivariant.Source: Appendix A · p. 13
  • Under the stated assumption, the rotation matrix propagates through equation 9, preserving equivariance of the force operator.The passage gives the relation for the transformed message and force vector.Source: Appendix A · p. 13
  • The dot-product operator in equation 16 makes the atomic-energy change invariant, and adding the resulting invariant arrays to atomic features preserves invariance of total-energy prediction.The proof describes this as contracting equivariant features into invariant arrays.Source: Appendix A · p. 13

METHODS

NewtonNet predicts total potential energy by summing atomic energies and obtains atomic forces as energy gradients. Its training objective combines energy, force, and latent-force-direction losses, with shared architectural settings and experiment-specific hyperparameters.

METHODS

  • The total potential energy is the sum of atomic energies mapped from the final updated atomic features.A differentiable function maps each atom’s features to its atomic potential energy.Source: METHODS · p. 13
  • Atomic forces are gradients of potential energy with respect to atomic positions, which the authors say guarantees energy conservation.The resulting forces also provide supervision for training atomic environments.Source: METHODS · p. 14
  • Training penalizes energy and force prediction errors plus deviations between latent-force and reference-force directions.The direction term uses cosine similarity, minimizing 1 − cos(α); λE, λF, and λD weight the three loss terms.Source: METHODS · p. 14
  • NewtonNet uses three message-passing layers, 128 features, and 20 basis sets; the ANI model also uses feature normalization at every layer.The authors report selecting other hyperparameters by system type; Table 5 lists settings for the reported experiments.Source: Table 5 · p. 14, METHODS · p. 15
  • Mini-batch gradient descent with Adam minimizes the loss; the hydrogen-combustion SchNet comparison uses five interaction layers and λF = 10.The SchNet setup otherwise uses the same hyperparameters as NewtonNet, according to the passage.Source: METHODS · pp. 14–15

Related papers

Loading 2108.02913v1…