PaperlayerView exact revision on arXiv

Source-linked AI summary

A Performance and Cost Assessment of Machine Learning Interatomic Potentials

Yunxing Zuo, Chi Chen, Xiangguo Li, Zhi Deng, Yiming Chen, Jörg Behler, Gábor Csányi, Alexander V. Shapeev, Aidan P. Thompson, Mitchell A. Wood, Shyue Ping Ong

arXiv:1906.08888v4physics.comp-phcond-mat.mtrl-sci

TL;DR

The paper addresses the need for rigorous, standardized evidence on the relative performance and cost of machine-learning interatomic potentials. It evaluates four descriptor-based ML-IAPs on a diverse DFT data set and finds near-DFT accuracy for energies, forces, and material properties, alongside a trade-off between accuracy, degrees of freedom, and computational cost.

  • Problem

    A critical gap is rigorous assessment of ML-IAP strengths and weaknesses across a standardized data set.

  • Method

    The study comprehensively evaluates GAP, MTP, NNP, and SNAP using common data and compares accuracy, material properties, and computational cost.

  • Results

    All ML-IAPs achieve near-DFT accuracy for energies, forces, and material properties, substantially outperforming traditional IAPs.

  • Takeaways & Limitations

    Accuracy, degrees of freedom, and computational cost are linked, creating a trade-off relevant to model selection for molecular dynamics and other applications.

  • Takeaways & Limitations

    The study does not attempt to assess ML-IAP transferability.

Abstract

from arXiv · show

Machine learning of the quantitative relationship between local environment descriptors and the potential energy surface of a system of atoms has emerged as a new frontier in the development of interatomic potentials (IAPs). Here, we present a comprehensive evaluation of ML-IAPs based on four local environment descriptors --- Behler-Parrinello symmetry functions, smooth overlap of atomic positions (SOAP), the Spectral Neighbor Analysis Potential (SNAP) bispectrum components, and moment tensors --- using a diverse data set generated using high-throughput density functional theory (DFT) calculations. The data set comprising bcc (Li, Mo) and fcc (Cu, Ni) metals and diamond group IV semiconductors (Si, Ge) is chosen to span a range of crystal structures and bonding. All descriptors studied show excellent performance in predicting energies and forces far surpassing that of classical IAPs, as well as predicting properties such as elastic constants and phonon dispersion curves. We observe a general trade-off between accuracy and the degrees of freedom of each model, and consequently computational cost. We will discuss these trade-offs in the context of model selection for molecular dynamics and other applications.

I. INTRODUCTION

The paper motivates ML interatomic potentials as a way to combine near-DFT accuracy with the scalability needed for larger atomistic simulations, while addressing the lack of standardized comparisons across models. It compares four ML-IAPs on common data spanning diverse elements, structures, properties, training requirements, and computational costs.

  • Motivation and approach: ML-IAPs describe the potential-energy surface using local environment descriptors invariant to translation, rotation, and permutation of homonuclear atoms.Examples include neural-network, Gaussian-process, SNAP, and moment-tensor potentials.
  • Motivation and approach: Near-DFT accuracy in energies and forces has been reported across diverse chemistries and atomic configurations, improving on traditional IAPs.
  • Motivation and approach: A remaining gap is rigorous assessment of ML-IAP relative strengths and weaknesses on a standardized data set.The paper frames this gap as analogous to prior standardized assessments of classical IAPs.
  • Study design: The study compares GAP, MTP, NNP, and SNAP for DFT energies, forces, equations of state, lattice parameters, and elastic constants.
  • Study design: The comparison uses matched DFT training and test sampling for Li, Mo, Cu, Ni, Si, and Ge, spanning bcc, fcc, and diamond structures and diverse bonding.
  • Model representations: The evaluated models differ in descriptor construction, including symmetry functions, SOAP kernels, SNAP bispectrum components, and tensor-based representations.The study also investigates quadratic SNAP, which adds distinct pairwise products of bispectrum components.

4. Moment Tensor Potential (MTP). The MTP[19] devises rotationally-covariant tensors

MTP represents local atomic environments with rotationally covariant tensors, contracts them into rotationally invariant basis functions, and fits energies by linear regression. Its polynomial power-like metric controls the tensor contractions and model complexity.

  • 4. Moment Tensor Potential (MTP): MTP encodes atomic environments with tensor products of neighbor-position vectors and radial functions.The tensor rank ν can be increased to approximate arbitrary interactions.
  • 4. Moment Tensor Potential (MTP): Tensor contractions produce rotationally invariant basis functions that are mapped to energies using linear regression.
  • 4. Moment Tensor Potential (MTP): The polynomial power-like metric determines which tensors are used and how many times they are contracted, controlling MTP performance.
  • 4. Moment Tensor Potential (MTP): MTP has been applied to metals, boron, binary and ternary alloys, and gas-phase chemical reactions.

B. DFT Data Sets

The study builds a diverse six-element DFT data set covering multiple crystal structures, strains, surfaces, thermal states, and vacancies. A nested training and hyperparameter-optimization workflow uses DFT energies, forces, stresses, and material properties.

  • B. DFT Data Sets: The data set contains Li, Mo, Ni, Cu, Si, and Ge, spanning main-group and transition metals, semiconductors, bcc, fcc, and diamond structures, and metallic and covalent bonding.
  • B. DFT Data Sets: Structures include ground-state crystals, strains from −10% to 10%, slabs with Miller indices up to three, and bulk AIMD snapshots.
  • B. DFT Data Sets: AIMD data cover 300 K and 0.5×, 0.9×, 1.5×, and 2.0× the melting point, with additional vacancy simulations at 300 K and 2.0× the melting point.
  • B. DFT Data Sets: DFT calculations use VASP with PAW, the PBE GGA functional, a 520 eV cutoff, and element-specific k-point meshes.
  • B. DFT Data Sets: Hyperparameters are optimized by combining grid search and differential evolution, using discrepancies in predicted and reference properties.

D. Data and code availability

The paper releases its code, data, and optimized models, and analyzes how ML-IAP accuracy varies with model complexity and computational cost. The results show a general accuracy–cost trade-off, with substantial cost differences among models and overfitting risks for high-complexity qSNAP.

  • D. Data and code availability: The code, data, and optimized ML models are published open-source, with Python interfaces for ML-IAP development and LAMMPS property calculators.
  • A. Optimized model parameters: Optimized cutoff radii are similar across ML-IAPs for a given element and fall between the 2NN and 3NN distances for fcc elements.
  • A. Optimized model parameters: Degrees of freedom strongly affect both ML-IAP accuracy and computational cost, motivating test-error versus cost analysis across model sizes.
  • A. Optimized model parameters: The optimized MTP, NNP, SNAP, and qSNAP models are two orders of magnitude less computationally expensive than the optimized GAP model.
  • A. Optimized model parameters: MTP models generally lie near the Pareto frontier, indicating a strong balance between accuracy and computational efficiency.
  • A. Optimized model parameters: For SNAP and qSNAP, bispectrum calculation is rate-limiting, while qSNAP quadratic terms add only a small computational-cost effect.
  • A. Optimized model parameters: qSNAP’s expanded coefficient count increases the likelihood of over-fitting, especially for Jmax > 3.

B. Accuracy in energies and forces

The ML-IAPs achieve near-DFT accuracy for energies and forces across the studied elements, with differences between models generally small relative to DFT error. Accuracy improves with more training data, especially for NNP and qSNAP models.

  • All ML-IAPs show extremely good performance relative to classical IAPs, with energy MAEs lowest for fcc systems, followed by bcc and diamond systems.
  • Training and test errors are similar, indicating no over-fitting for the optimized ML-IAPs; differences between models are on the scale of meV atom^-1 in energies and 0.1 eV Å^-1 in forces.
  • The GAP and MTP models generally have the lowest MAEs in predicted energies and forces, while SNAP and NNP models have the highest energy MAEs.
  • qSNAP models have moderately lower MAEs than linear SNAP, at the expense of a large expansion in the number of parameters.
  • Force MAEs are very low for Cu, Ni, and Li but higher for Mo and the diamond semiconductors, consistently across the studied ML-IAPs.
  • Increasing training structures decreases prediction errors for all models, with especially substantial energy improvements for NNP and qSNAP.SNAP Mo appears converged at approximately 600 energy-training structures and 400 force-training structures, whereas NNP energy accuracy continues improving at approximately 800 structures.

C. Accuracy in material properties

The ML-IAPs reproduce several material properties and equations of state close to DFT references. Their accuracy is generally strong, although diffusion-property errors depend on chemistry and model family.

  • All ML-IAPs predict lattice parameters within 0.1–2.0% of DFT and elastic constants typically within 10% of DFT values.
  • MTP, SNAP, and qSNAP perform well for elastic constants in fcc and bcc systems but show slightly higher errors for diamond systems.
  • GAP and MTP predict diffusion properties well across chemistries, with most errors within 10% of DFT values, but moderately underestimate migration energies for diamond systems.
  • SNAP and qSNAP accurately predict diffusion properties for fcc systems but considerably underestimate vacancy formation energies and activation barriers for diamond systems.
  • All ML-IAPs overestimate Mo migration energy by more than 20%.
  • All ML-IAPs predict equations of state within 2 meV/atom of DFT, while predicted phonon dispersion curves are in excellent agreement with DFT references.

E. Accuracy in molecular dynamics (MD) trajectories

The models provide stable MD-trajectory predictions and generally reproduce sampled-structure energies and forces well. However, extrapolation to unseen polymorphs reveals model- and chemistry-dependent differences not captured by aggregate MAEs.

  • GAP and MTP generally produce smaller energy and force errors than NNP, SNAP, and qSNAP on sampled MD structures.
  • The GAP model has both the lowest median error and the smallest interquartile range among the sampled MD error distributions.
  • Additional training data would likely improve NNP and qSNAP performance, particularly because their convergence was not established with the approximately 100-structure comparison.
  • All ML-IAPs qualitatively reproduce polymorph energy differences except for the especially small Li fcc–bcc difference, with training limited to ground-state structures.
  • The GAP model gives the largest error for Si and Ge wurtzite–diamond energy differences despite relatively low energy MAEs, possibly reflecting sensitivity to missing reference configurations.
  • Linear SNAP reproduces polymorphic energy differences among the best across systems, outperforming GAP and MTP for Mo, Si, and Ge despite larger energy and force MAEs.

IV. CONCLUSIONS

The study evaluates ML-IAPs across accuracy, training-data requirements, and computational cost, finding near-DFT accuracy alongside model-specific trade-offs. Diverse training data and Pareto-frontier analysis help identify configurations balancing accuracy and efficiency.

  • Evaluation scope: The evaluation compares GAP, MTP, NNP, SNAP, and qSNAP across six elemental systems spanning fcc, bcc, and diamond structures, metallic and covalent bonding.The systems include main-group and transition metals and semiconductors.
  • Trade-offs: Increasing degrees of freedom and training structures generally improves accuracy, but Pareto-frontier analysis identifies configurations where further degrees of freedom provide little improvement at higher computational cost.The study frames model selection around accuracy, training-data requirements, and computational cost.
  • Accuracy: All ML-IAPs achieve near-DFT accuracy for energies, forces, and material properties, substantially outperforming traditional IAPs.The evaluation targets properties for both seen and unseen structures.
  • Accuracy and cost: GAP and MTP exhibit the smallest energy and force MAEs, while GAP is among the most computationally expensive for a given accuracy.MTP models generally lie close to the Pareto frontier, balancing accuracy and computational efficiency.
  • Extrapolation: Linear SNAP has relatively high energy and force MAEs but shows the best extrapolability to higher-energy diamond polymorphs and reproduces their equation of state.This contrasts predictive accuracy on typical configurations with extrapolation behavior.
  • Training data: With approximately 100–200 structures, GAP, MTP, and SNAP reach meV atom−1 energy and 0.01 eV Å−1 force accuracy, while NNP and qSNAP can improve with more data.The training procedure samples diverse structures from ground-state and multi-temperature AIMD simulations.
  • Scope boundary: The study does not combine the descriptors with different ML frameworks, so alternative descriptor–framework pairings may offer better accuracy–cost trade-offs for particular applications.Descriptor choice affects local-environment encoding, while framework choice determines mapping flexibility.

A Performance and Cost Assessment of Machine Learning

The paper introduces several local-environment descriptor frameworks for representing atomic configurations and fitting their energies and forces. These representations encode invariant local structure, with model and hyperparameter choices balancing accuracy and computational complexity.

  • A. Descriptor and model formulations: ACSF converts each local atomic environment into rotationally and translationally invariant numeric vectors.Radial functions encode pair correlations, while angular symmetry functions incorporate neighbor angles and cutoff smoothing.
  • A. Descriptor and model formulations: NNP, GAP, SNAP, qSNAP, and MTP implementations use fitted descriptor-based energy models with optimized system-specific parameters.The supplied passages describe neural-network, Gaussian-process, and descriptor-based formulations, with optimized parameters tabulated for the elemental systems.
  • A. Descriptor and model formulations: GAP represents neighboring atoms with a Gaussian neighbor density and uses the SOAP kernel to compare local environments.The density is expanded in spherical harmonics and radial functions before forming a rotationally invariant power spectrum.

C. Spectral neighbor analysis potential

SNAP represents local atomic neighbor densities through invariant bispectrum coefficients and relates these descriptors to energy and force predictions. Its fitting procedure optimizes model hyperparameters against reference data and material properties.

  • C. Spectral neighbor analysis potential: SNAP projects the three-dimensional local neighbor density onto invariant coefficients using a spherical-harmonics expansion.Neighbor weights distinguish atom types, and the cutoff function smoothly reduces the density to zero at the cutoff radius.
  • C. Spectral neighbor analysis potential: SNAP expresses energy and force contributions as linear functions of bispectrum components.The coefficients are obtained by fitting against the database of quantum electronic-structure calculations.
  • C. Spectral neighbor analysis potential: A two-step fitting process optimizes SNAP hyperparameters by comparing predicted and reference material properties.The outer loop uses differential-evolution optimization after fitting a linear model in each inner iteration.

D. Quadratic Spectral neighbor analysis potential

qSNAP extends the linear SNAP energy model with quadratic bispectrum contributions and an embedding-energy interpretation. The study compares descriptor complexity and predictive-property reproduction while selecting model hyperparameters.

  • D. Quadratic Spectral neighbor analysis potential: qSNAP adds quadratic contributions to the SNAP energy, interpretable as an embedding-energy term.The extension augments the original bispectrum-based formulation rather than replacing its descriptors.
  • D. Quadratic Spectral neighbor analysis potential: The quadratic extension includes pairwise products of bispectrum components and expands energy-function complexity to seven-body effects.The pairwise terms are collected in a symmetric K × K matrix α.
  • D. Quadratic Spectral neighbor analysis potential: MTP constructs contracted rotationally invariant moment-tensor representations and linearly correlates them with potential energy.The choices of tensor parameters and polynomial powers control the balance between computational complexity and efficiency.
  • D. Quadratic Spectral neighbor analysis potential: Model hyperparameters were evaluated using elastic constants and phonon spectra, while weighted parameters were considered less relevant to predicted properties.The fitting procedure used different iteration limits for different polynomial-power ranges.

II. PHONON DISPERSION CURVES COMPUTED DFT, GAP, MTP, NNP,

The study compares phonon predictions and computational scaling for several ML-IAPs across elemental metals and semiconductors. The supplementary figures organize phonon curves by material, while molecular-dynamics tests examine system-size scaling.

  • II. Phonon dispersion curves: Phonon dispersion curves are compared against DFT for bcc Mo and Li, fcc Ni and Cu, and diamond Si and Ge.The supplementary figures show DFT, GAP, MTP, NNP, SNAP, and qSNAP results where indicated by the figure labels.
  • II. Phonon dispersion curves: Test error is plotted against computational cost for fcc Ni, fcc Cu, bcc Li, diamond Si, and diamond Ge.The plots include Pareto-frontier boundaries and comparisons with the number of degrees of freedom.
  • II. Phonon dispersion curves: All ML-IAPs show a greatly linear scaling relationship between computational cost and the number of atoms.The scaling study uses 300 K NPT molecular-dynamics simulations on Mo supercells containing 432, 2000, 5488, and 11664 atoms.
Loading 1906.08888v4…