Active learning of linearly parametrized interatomic potentials

Evgeny V. Podryabinkin; Alexander V. Shapeev

Source-linked AI summary

Active learning of linearly parametrized interatomic potentials

Evgeny V. Podryabinkin, Alexander V. Shapeev

arXiv:1611.09346v3Nov 28, 2016physics.comp-phcond-mat.mtrl-sci

TL;DR

Atomistic simulations face a trade-off between accurate but expensive quantum-mechanical models and efficient but less accurate empirical potentials, while MLIPs remain vulnerable to extrapolation. This paper applies D-optimality-based active learning to select new configurations and retrain linearly parametrized potentials on the fly, reporting reliable simulations without essentially reducing accuracy. The approach also extends beyond molecular dynamics in principle, although relaxation requires care because online potential changes can create false minima.

Problem
MLIPs are difficult to make reliable because their offline training datasets may not prevent extrapolation during online evaluation.
Method
The paper uses D-optimality-based active learning to detect extrapolative configurations and retrain linearly parametrized potentials on selected configurations.
Results
The proposed learning-on-the-fly algorithms keep molecular-dynamics trajectories within the physical region without essentially reducing interatomic-potential accuracy.
Takeaways & Limitations
The approach can in principle support molecular dynamics, structure relaxation, Monte Carlo, nudged elastic band, and accelerated molecular-dynamics simulations.
Takeaways & Limitations
During relaxation, online potential changes can falsely identify a configuration as a local minimum, requiring rerunning relaxation or using a force-only stopping criterion.

Abstract

from arXiv · show

This paper introduces an active learning approach to the fitting of machine learning interatomic potentials. Our approach is based on the D-optimality criterion for selecting atomic configurations on which the potential is fitted. It is shown that the proposed active learning approach is highly efficient in training potentials on the fly, ensuring that no extrapolation is attempted and leading to a completely reliable atomistic simulation without any significant decrease in accuracy. We apply our approach to molecular dynamics and structure relaxation, and we argue that it can be applied, in principle, to any other type of atomistic simulation. The software, test cases, and examples of usage are published at http://gitlab.skoltech.ru/shapeev/mlip/.

1. Introduction

Machine-learning interatomic potentials aim to combine quantum-mechanical accuracy with empirical-potential efficiency, but their reliability depends critically on training-set coverage. The paper therefore develops active learning to detect extrapolation and select configurations during or before simulation.

Motivation: Empirical potentials are efficient for large, long simulations but typically provide qualitative accuracy, whereas quantum-mechanical models are accurate but computationally expensive.DFT applicability is typically limited to hundreds of atoms and hundreds of picoseconds.
Machine-learning interatomic potentials: MLIPs seek to combine quantum-mechanical accuracy with the computational efficiency of empirical interatomic potentials.They fit flexible functions of neighboring-atom positions to quantum-mechanical energies, forces, and/or stresses.
Training-set challenge: MLIP reliability is difficult because interpolative models can fail outside their training domain, while generating a gap-free training set is nontrivial.The challenge becomes harder as the model is made more general and accurate.
Training-set challenge: Common sampling procedures do not ensure full coverage of the configuration-space region required for reliable MLIP training.Examples include random perturbations, ab-initio molecular dynamics, and classical molecular dynamics with an empirical potential or existing MLIP.
Active-learning approach: The paper proposes D-optimality-based active learning to detect extrapolative configurations and query quantum-mechanical data only when needed.The approach is intended for linearly parametrized potentials and can also optimize a training set offline.
Active-learning approach: Unlike earlier fixed-frequency learning-on-the-fly schemes, the proposed query strategy uses geometrical information from atomic positions and supercell vectors.Earlier learning-on-the-fly methods selected configurations every fixed number of steps.

2. Machine Learning Interatomic Potentials

The paper formulates local interatomic potentials whose energy is a linear combination of fixed basis functions and fits their parameters to quantum-mechanical observables. Energies, forces, and stresses become linear equations solved through least squares.

Problem setup: A periodic configuration contains N atoms in a supercell, and quantum-mechanical energy, forces, and stresses are treated as expensive black-box quantities.These observables are the reference data used for fitting.
Local potential: Each atom interacts with a local neighborhood containing atoms and periodic extensions within a cutoff distance Rcut.The neighborhood size n may vary between atoms.
Local potential: A linearly parametrized local potential has energy contributions expressed as a linear combination of fitting parameters θj and fixed basis functions Bj.The concrete basis construction is not needed for the general formulation; MTPs are used in this work.
Fitting: The fitting parameters are determined from quantum-mechanical energies, forces, and stresses evaluated on a designated training set of configurations.The training set is denoted XTS = {x^(1), ..., x^(K)}.
Fitting: Energy matching expands into a linear system Aθ = R, with the parameter vector obtained through the pseudoinverse for an overdetermined system.The stated solution is θ := (A⊤A)−1A⊤R.
Fitting: Force- and stress-fitting equations are combined with energy equations in a least-squares problem using coefficients Cf and Cs to set their relative importance.These coefficients weight force and stress fitting relative to energy fitting.

3. Active Learning

The active-learning procedure selects configurations using unlabeled data, so quantum-mechanical calculations are requested only for sufficiently new configurations. Its query strategy uses D-optimality and the maxvol algorithm to reduce parameter uncertainty and avoid extrapolation.

Active-learning principle: Active learning selects training configurations from a configuration set or stream without requiring quantum-mechanical labels for the selection decision.In learning on the fly, quantum-mechanical calculations are performed only when a configuration is sufficiently new.
Query strategy: The paper uses a variance-reduction query strategy based on D-optimality together with the fast maxvol algorithm.D-optimality can be derived by selecting equations that minimize noise in the fitted solution under Gaussian noise.
Query strategy: The equations used for fitting need not coincide with those used for selecting configurations.The paper fits using all equations while considering different query-strategy variants.

Query strategy QS1: Selection by the energy-fitting equation

QS1 selects configurations using only the energy-fitting equation, measuring how strongly a candidate changes the training matrix and replacing the most influential existing configuration when needed.

QS1 forms matrix A from energy-fitting equations for configurations already in the training set.
Replacing the k-th row of A with a candidate row changes |det A| by a factor of |Ck|.This factor follows from Cramer's formula.
When γ(x∗) exceeds γth, QS1 adds x∗ and removes x(k) with the largest |Ck|; otherwise, it keeps the training set unchanged.The replacement increases |det A| by at least γth.
The extrapolation grade γ(x∗) is the largest coefficient magnitude, γ(x∗) := max 1≤k≤m |Ck|.The coefficients express the candidate energy through the training-configuration energies.
The extrapolation grade is geometric: it depends on the candidate and training configurations, not on quantum-mechanical data.

Query strategy QS2: Selection by all equations

QS2 selects configurations using energy, force, and stress equations together, applying maxvol to enlarge the determinant-based representation while controlling extrapolation.

QS2 forms A from m equations selected from the energy, force, and stress fitting equations.
Each configuration contributes 1 + 3N + 6 rows instead of one energy row.
Maxvol greedily replaces rows of A with rows of B until every coefficient satisfies maxk,j |Ck j| ≤ γth.Because configurations can contribute multiple rows, fewer than m configurations may be selected.
The QS2 algorithm has complexity O(Nm2).

Query strategy QS3: Selection by neighborhoods

QS3 selects by neighborhood-level equations, while the broader active-learning workflow queries quantum-mechanical data only for sufficiently extrapolative configurations and then updates the potential.

Query strategy QS3: Selection by neighborhoods: QS3 constructs A from site-energy equations and applies the same determinant-based replacement strategy as QS2.
Query strategy QS3: Selection by neighborhoods: QS3 replaces rows when maxk,j |Ck j| exceeds γth, and its training set may contain fewer than m configurations.
Query strategy QS3: Selection by neighborhoods: QS3 does not require site energies from quantum-mechanical data and has complexity O(Nm2).
Learning on the fly: In learning on the fly, each unlabeled configuration is graded for extrapolation before quantum-mechanical energy, force, and stress calculations are requested.
Learning on the fly: If a configuration is sufficiently new, it updates the training set and potential before the workflow returns energy, forces, and stresses.The threshold γth balances quantum-mechanical calculation frequency against accuracy loss from extrapolation.
Learning on the fly: Offline active learning can reduce similar configurations while improving transferability and reducing maximal errors relative to the full database.

4. Testing

The tests show that active learning trades a small increase in fitting error for reliable atomistic simulations and substantially fewer quantum-mechanical calculations. Across one-dimensional fitting, molecular dynamics, temperature changes, and relaxation, the method identifies configurations requiring additional training and avoids observed extrapolative failures.

4.1. A one-dimensional illustration: The active-learning approach therefore offers reliability at a cost of an accuracy trade-off, while realistic molecular-dynamics tests show only a marginal error increase.The paper contrasts passive and active sampling on the basis of reliability and fitting accuracy.
4.2. Accuracy of learning molecular dynamics: For Li MTPs, fitting errors systematically converge as the number of basis functions increases, but adding more than 100 basis functions essentially does not reduce error.Subsequent tests use 100 basis functions.
4.4. Learning on the Fly: Increasing γth reduces QM calculations without materially increasing errors up to γth = 2; only at γth = 11 does the maximum error slightly increase.The authors report γth between 2 and 11 as a practical range, with calculations only a few times above the theoretical minimum.
4.4. Learning on the Fly: Active learning on the fly ran 0.5 µs without failure using about 50 additional QM calculations, whereas classical learning increased failure time to 150 ps at a cost of 1500 calculations.The comparison uses trajectories in which failure means escape into an unphysical region.

5. Discussion

The proposed active-learning algorithms preserve potential accuracy while keeping molecular-dynamics trajectories within the physical region. Their scope extends beyond MD, and D-optimality also provides geometric interpretations for selecting configurations and local environments.

5. Discussion: The learning-on-the-fly algorithms essentially preserve interatomic-potential accuracy while keeping MD trajectories within the physical region.The active-learning overhead and retraining cost were less than the energy, force, and stress calculations in the test examples.
5. Discussion: The active-learning algorithms can, in principle, support structure relaxation, Monte-Carlo sampling, nudged elastic band, and accelerated-MD simulations.They can also learn configurations with varying numbers of atoms, which is relevant when the target structure size is unknown.
5. Discussion: D-optimality can be interpreted as reducing uncertainty in potential parameters or maximizing the volume spanned by training configurations.These interpretations explain its use for avoiding extrapolation.
5. Discussion: QS3 selects configurations with the most different atomic neighborhoods according to the D-optimality criterion.This property may support learning-on-the-fly MD for systems with millions or more atoms by training on completed local-environment configurations.

6. Conclusion

The paper addresses MLIP reliability by applying active learning to detect and respond to extrapolation. It proposes a D-optimality-based scheme that is empirically accurate, efficient, and reliable, using linearly parametrized potentials such as MTPs.

6. Conclusion: MLIP reliability is difficult because increasingly general and accurate potentials require offline datasets that prevent extrapolation during online evaluation.The conclusion identifies reliability as the weak point of machine-learning interatomic potentials.
6. Conclusion: The paper proposes active learning based on D-optimality to detect extrapolation and fit machine-learning interatomic potentials.The approach is presented as a way to solve the reliability problem through active learning.
6. Conclusion: The proposed scheme yields an accurate, computationally efficient, and reliable interatomic interaction model.In learning-on-the-fly simulations, it detects attempted extrapolation and retrains on those configurations; offline, it controls the degree of extrapolation.
6. Conclusion: The method is demonstrated with Moment Tensor Potentials, whose energy depends linearly on model parameters and whose basis functions use finite-cutoff radial functions.The basis construction uses invariant tensorial descriptors of atomic environments, with a degree parameter limiting basis-function complexity.
6. Conclusion: The tensor-valued descriptors used in the potential can be interpreted as moments of inertia of an atom’s neighborhood.This interpretation assigns radial-basis weights to neighboring atoms and relates descriptor order to neighborhood moments.

Appendix B: Maxvol algorithm

The maxvol algorithm greedily selects rows forming a high-volume m × m submatrix from an k × m matrix. Its threshold controls the trade-off between determinant size and computational effort.

Appendix B: Maxvol algorithm: QS2 and QS3 find the m × m submatrix A with maximal |det A| in an k × m matrix B using the maxvol algorithm.Maxvol selects rows greedily, and each iteration has O(mk) complexity.
Appendix B: Maxvol algorithm: The algorithm initializes A, computes C = BA^-1, and identifies the largest absolute element Cij.This matrix determines whether a row swap is required.
Appendix B: Maxvol algorithm: When |Cij| > γth, maxvol swaps the corresponding row and updates C using a Sherman–Morrison rank-one update.The procedure then repeats the search for the largest absolute element.
Appendix B: Maxvol algorithm: A smaller threshold γth > 1 produces a larger |det A| at the cost of more iterations.Thus, γth directly controls the determinant-quality versus iteration-cost trade-off.
Appendix B: Maxvol algorithm: Table 4 reports fitting errors for different active-learning approaches.The supplied table caption identifies the comparison but does not specify the reported values or layout.

Appendix C: Active Learning from a Database

The appendix examines the advantages of active learning when fitting from a given database rather than using passive learning.

Appendix C: Active Learning from a Database: Active learning is evaluated as an approach offering advantages over passive learning when training from a fixed database.The supplied passage states the appendix’s comparison focus without specifying the measured advantages.

Errors of fitting

Active learning preserves fitting accuracy while reducing maximal error and improving the reliability and transferability of offline-trained potentials. The evaluation compares fitting strategies and tests failure times across molecular-dynamics conditions.

All active-learning methods marginally increase RMS error versus passive learning but significantly reduce maximal error.
Active-learning methods produce smaller errors than random configuration selection, indicating greater selection efficiency.
Offline active learning can reduce the training database size while keeping accuracy essentially at the same level as passive learning.
Reliability was evaluated using 100 molecular-dynamics runs at 300 K and 450 K, measuring time until the minimal interatomic distance fell below 1.5Å.The potentials were trained offline on 300 K molecular-dynamics trajectories, and average failure times were calculated for different MTPs.
Actively learned MTPs were more stable and transferable than passively learned ones, with QS3 showing by far the best results among the three active-learning approaches.The paper connects this to selecting highly different configurations, which reduces extrapolation on different evaluation sets; it speculates that QS3 benefits from local failure behavior.