PaperlayerView exact revision on arXiv

Source-linked AI summary

Hyperactive Learning (HAL) for Data-Driven Interatomic Potentials

Cas van der Oord, Matthias Sachs, Dávid Péter Kovács, Christoph Ortner, Gábor Csányi

arXiv:2210.04225v2physics.comp-phstat.ML

TL;DR

Unreliable predictions can create unphysical energy minima that trap or destabilize molecular dynamics. HAL biases MD toward high-uncertainty configurations, and in a PEG test reached million-step stability with an order of magnitude fewer exploratory timesteps than standard active learning.

  • Problem

    Unphysical energy minima caused by missing training data can trap or destabilize molecular-dynamics trajectories.

    Source: §I · p. 2

  • Method

    HAL balances the physical MD force with a bias toward high-uncertainty regions, with τ controlling exploration.

    Source: §I · pp. 2–3

  • Results

    An order of magnitude fewer exploratory MD timesteps achieved minimum 1 million step stability with HAL than with standard active learning.

    Source: §1 · p. 11

  • Takeaways & Limitations

    HAL readily finds rare low-energy PEG configurations with double hydrogen bonding that regular MD struggles to find, even at elevated temperatures.

    Source: §2 · p. 12

Abstract

from arXiv · show

Data-driven interatomic potentials have emerged as a powerful class of surrogate models for {\it ab initio} potential energy surfaces that are able to reliably predict macroscopic properties with experimental accuracy. In generating accurate and transferable potentials the most time-consuming and arguably most important task is generating the training set, which still requires significant expert user input. To accelerate this process, this work presents \text{\it hyperactive learning} (HAL), a framework for formulating an accelerated sampling algorithm specifically for the task of training database generation. The key idea is to start from a physically motivated sampler (e.g., molecular dynamics) and add a biasing term that drives the system towards high uncertainty and thus to unseen training configurations. Building on this framework, general protocols for building training databases for alloys and polymers leveraging the HAL framework will be presented. For alloys, ACE potentials for AlSi10 are created by fitting to a minimal HAL-generated database containing 88 configurations (32 atoms each) with fast evaluation times of <100 microsecond/atom/cpu-core. These potentials are demonstrated to predict the melting temperature with excellent accuracy. For polymers, a HAL database is built using ACE, able to determine the density of a long polyethylene glycol (PEG) polymer formed of 200 monomer units with experimental accuracy by only fitting to small isolated PEG polymers with sizes ranging from 2 to 32.

I. INTRODUCTION

Training databases are costly to build, and ordinary Boltzmann sampling may undersample defects and transition states. Hyperactive learning (HAL) biases molecular dynamics toward uncertain configurations while retaining a focus on low-energy regions.

I. INTRODUCTION

  • Boltzmann sampling may severely undersample defects and transition states, although these configurations can strongly affect material properties.The paper identifies training-database generation as a time-consuming challenge.Source: §I · p. 2
  • Active learning detects unreliable predictions, triggers ground-truth evaluation, extends the database, and refits the model to reduce risks from unphysical energy minima.The paper notes that such minima can trap MD trajectories or cause numerical instability.Source: §I · p. 2
  • HAL continually biases molecular dynamics toward high-uncertainty regions while balancing physical forces to find unreliable configurations and retain focus on low-energy regions.The biasing strength τ controls exploration and must be tuned to keep the trajectory energetically sensible.Source: §I · pp. 2–3
  • The framework is demonstrated with ACE and is also applicable to linear and Gaussian-process models, with potential extension to other uncertainty-equipped potentials, including neural networks.For ACE, the paper uses a Bayesian view of linear regression and a committee estimator to make uncertainty evaluation efficient during MD.Source: §I · p. 3
  • Relative force uncertainty provides a stopping criterion for extracting configurations, matching the similar 3–10% relative force accuracy cited for liquid and phonon properties.A configuration triggers a new ab initio calculation when its force uncertainty exceeds a predefined tolerance; ε regularizes the measure.Source: §I · p. 4

A. Filtering an existing training set

The silicon-diamond filtering study tests whether relative force uncertainty identifies errors and accelerates learning. Greedy selection by that uncertainty correlates with true error and reaches train–test generalization using a small fraction of the original environments.

A. Filtering an existing training set

  • The filtering protocol starts from one of 489 silicon-diamond configurations, then repeatedly adds the remaining configuration with the largest maximum relative force uncertainty.The ACE models use basis functions up to correlation order ν=3, polynomial degree 20, and committee size K=32.Source: A. Filtering an existing training set · p. 4, §1 · p. 5

1. Si diamond: error correlation and convergence

  • Filtering by maximum relative force uncertainty significantly accelerates learning versus random selection and achieves good train–test generalization with around 5% of the original environments.The comparison begins from the same initial configuration; Fig. 1b presents the learning-rate comparison.Source: Figure 1 · p. 5, §1 · p. 5
  • Maximum relative force uncertainty correlates well with maximum true relative force error for training sets of 4 and 10 silicon-diamond configurations.The test set comprises the remaining configurations among 489 silicon-diamond structures.Source: §1 · p. 5, Figure 1 · p. 5

2. Si diamond: property convergence

The silicon-diamond ACE models recover several properties using only a small fraction of the original database. For AlSi10, HAL-generated ACE models predict the melting temperature accurately, while model sparsity and errors vary with the ARD tolerance.

B. AlSi10

  • The AlSi10 HAL protocol iteratively builds solid and liquid databases from manually chosen random crystal structures, then combines them to fit a model for both phases.The initial random-alloy set uses 32-atom FCC cells with 29 Al and 3 Si atoms; solid-alloy sampling includes cell-volume adjustments and element swaps.Source: B. AlSi10 · pp. 6–8

2. Si diamond: property convergence

  • 5% of the original silicon-diamond database suffices for accurate energy-volume, phonon-spectrum, and thermal-property predictions relative to DFT.The phonon spectrum converges to a few-percent relative error even for forces near 0.01 eV/Å.Source: §2 · pp. 5–6, Figure 2 · p. 6

B. AlSi10

  • The ARD fits assign large coefficients to pair interactions, which contain most of the binding energy; α′ = 300k also includes low-relevance coefficients around 10^-4.The auxiliary pair potential V2 partly captures the pair-interaction contribution.Source: Figure 4 · p. 8, B. AlSi10 · p. 9
  • Increasing α′ reduces sparsity and training error, but test error rises at α′ = 300k, indicating overfitting.The reported model evaluations are all within 100 μs/atom/core per MD step.Source: Table II · p. 7, B. AlSi10 · pp. 8–9
  • 867 K is predicted by the AlSi10 ACE models with α′ = 300k and α′ = 80k, matching the Thermo-Calc reference.All models also predict the expected FCC ground structure.Source: Figure 5 · p. 9, B. AlSi10 · p. 10
  • Nested Sampling estimates the AlSi10 melting temperature from a heat-capacity peak while exploring decreasing enthalpy levels from gas through liquid to the ground structure.The reported comparison uses three independent runs for each ARD-fitted model.Source: B. AlSi10 · pp. 9–10

C. Polyethylene glycol (PEG)

PEG chains are challenging to model because they range from coiled to stretched configurations and can form hydrogen bonds. The section applies HAL to polymer database construction, including a density prediction for bulk PEG(200).

C. Polyethylene glycol (PEG)

  • PEG chains range from tightly coiled to fully stretched structures, and their end-group hydrogen bonding complicates simulation in vacuum.The OH groups can form hydrogen bonds.Source: C. Polyethylene glycol (PEG) · p. 10
  • The section benchmarks HAL against active learning, examines its discovery of high-error configurations, and applies it to predict bulk PEG(200) density.Source: C. Polyethylene glycol (PEG) · p. 10

1. PEG(n=2): HAL vs AL

For PEG(n=2), HAL biasing accelerates the search for ACE models with stable MD, but excessive biasing produces unstable dynamics. For PEG(n=4), HAL finds rare hydrogen-bonded configurations whose errors are reduced when they are included in training.

1. PEG(n=2): HAL vs AL

  • The PEG(n=2) benchmark began from 20 polymers sampled at 500 K; AL and HAL used ACE models, and configurations were evaluated by ORCA DFT once stol=0.5.The stability test ran the saved models for 1 million MD steps across 100 runs.Source: §1 · p. 10
  • At τr=0.25, excessive biasing generated very high-energy configurations far from the desired PES and resulted in unstable dynamics.Including these unphysical configurations in the training database led to a poorly performing model.Source: §1 · p. 11
  • An order of magnitude fewer exploratory MD timesteps were needed for τr=0.20 HAL than for standard AL to reach minimum 1 million-step stability.The benchmark evaluates stability across 100 seeds.Source: §1 · p. 11, Figure 6 · p. 11

2. PEG(n=4): rare events

  • HAL-generated configurations were low in energy, and adding them to the training database reduced errors on those configurations; most involved hydrogen bonding, which regular MD struggled to find even at elevated temperatures.Source: §2 · p. 12

3. PEG(n=200): bulk density

The PEG(200) density estimate uses a HAL-trained ACE model fitted to small isolated polymers, extending the training data to a periodic bulk simulation. The section also describes HAL as biasing MD toward uncertain configurations, with the bias strength requiring control.

3. PEG(n=200): bulk density

  • The model was fitted to isolated PEG polymers from n=2 to n=32, including 64 HAL configurations at n=16 and 91 at n=32, then used for periodic PEG(200).The periodic bulk simulation therefore extrapolates beyond the small polymer segments in vacuum used for training.Source: §3 · p. 12
  • DFT evaluation of the periodic 1400-atom PEG(200) simulation box was practically impossible because the training DFT code does not support periodic boundary conditions.Source: §3 · p. 12
  • 1.238 g/cm3 was the PEG(200) density from the last 500 ps at 300 K, around 3% above the experimental value of 1.2 g/cm3.The linear ACE model ran at 220 µs/atom/core per MD step.Source: §3 · p. 12

A. Hyperactive Learning (HAL)

  • The PEG(4) energy scatter plots compare models fitted to 500 K, 500 K+800 K, and 500 K+HAL data; HAL-found configurations often contain rare hydrogen bonding.Source: Figure 7 · p. 13
  • HAL adds a biasing force to MD to accelerate generation of high-uncertainty configurations, while τ=0 recovers standard MD dynamics.The uncertainty-gradient quantity is described as a computationally cheap conservative biasing force.Source: A. Hyperactive Learning (HAL) · p. 13
  • The bias strength τ must be controlled: strong bias can quickly produce unphysical configurations, whereas weak bias generates valuable configurations slowly.Adaptive biasing tunes τ to gently drive MD toward high uncertainty.Source: A. Hyperactive Learning (HAL) · pp. 13–14

B. Atomic Cluster Expansion (ACE)

ACE represents a configuration’s total energy as a sum of parameterized atomic energies. Its basis construction incorporates many-body interactions and rotational and reflection symmetry, while the linear model’s site-energy cost scales linearly with the number of neighbors and with body order.

B. Atomic Cluster Expansion (ACE)

  • ACE decomposes a configuration’s total energy into a sum of parameterized atomic energies.Each atomic energy is modeled using ACE basis functions.Source: B. Atomic Cluster Expansion (ACE) · p. 14
  • Taking a ν-order tensor product of pooled features yields many-body correlation functions incorporating (ν+1)-body interactions.The features are denoted the atomic basis in ACE.Source: B. Atomic Cluster Expansion (ACE) · pp. 14–15
  • Averaging the A-basis over rotations and reflections constructs an isometry-invariant basis through a sparse linear operation.The A-basis is permutation-invariant but does not itself incorporate rotation or reflection symmetry; the construction uses orthogonal-group representation theory, with further details in.Source: B. Atomic Cluster Expansion (ACE) · p. 15
  • Evaluating a linear ACE site energy costs linearly in the number of neighboring atoms and in body order ν + 1.Source: B. Atomic Cluster Expansion (ACE) · p. 15

C. (Bayesian) Linear Regression

The linear-regression fit minimizes a loss against DFT observations, with energy and force terms and, for materials, a virial-stress term. A Gaussian Bayesian formulation provides closed-form posterior quantities used for sampling during HAL, while BRR is used during data generation and ARD typically for the final fit.

C. (Bayesian) Linear Regression

  • The fitting loss uses DFT energy and force observations, with a virial-stress term added when fitting materials.The relative importance of the energy and force observations is controlled by weights wE and wF.Source: C. (Bayesian) Linear Regression · p. 15
  • The regression problem uses a design matrix and observations, with Tychonov regularization controlled by η.The parameter η is commonly chosen using a model-selection criterion such as cross-validation.Source: C. (Bayesian) Linear Regression · p. 15
  • A Gaussian prior and Gaussian error model yield a Gaussian posterior with closed-form mean and covariance.Those expressions support independent posterior samples and parameterizing the fitted model with the exact posterior mean.Source: C. (Bayesian) Linear Regression · p. 15
  • BRR is used to produce Bayesian fits during HAL data generation, while ARD is typically used for the final fit afterward.Source: C. (Bayesian) Linear Regression · pp. 15–16

D. Bayesian Ridge Regression (BRR)

BRR assumes an isotropic prior covariance, yielding a ridge-regularized least-squares solution. Its hyperparameters are determined by maximizing the marginal log likelihood, which marginalizes over the coefficients.

D. Bayesian Ridge Regression (BRR)

  • BRR assumes an isotropic prior covariance with hyperparameter α, the prior precision.Source: D. Bayesian Ridge Regression (BRR) · p. 16
  • Maximizing the posterior under this prior is equivalent to regularized least squares with ridge penalty η = α/λ.The prior therefore produces a regularized solution that keeps coefficient parameters small.Source: D. Bayesian Ridge Regression (BRR) · p. 16
  • BRR determines α and λ by maximizing the marginal log likelihood, or evidence, after marginalizing over coefficients.The evidence describes the likelihood of the data given α and λ; maximizing it yields the statistically most probable solution given the basis and data.Source: D. Bayesian Ridge Regression (BRR) · p. 16

E. Automatic Relevance Determination (ARD)

ARD assigns an independent regularization hyperparameter to each coefficient, enabling basis-function relevance ranking and pruning. Its threshold controls the balance between model accuracy and sparsity.

E. Automatic Relevance Determination (ARD)

  • ARD relaxes BRR’s isotropic prior by assigning each coefficient an independent regularization hyperparameter.The resulting prior determines the relevance of each parameter or basis function.Source: E. Automatic Relevance Determination… · p. 16
  • ARD ranks and prunes irrelevant basis functions to produce sparse models, which often generalize better than BRR.The threshold α′ for including basis functions controls the balance between accuracy and sparsity.Source: E. Automatic Relevance Determination… · p. 16

F. Posterior Predictive Distribution

The Bayesian posterior-predictive distribution quantifies uncertainty from both parameter uncertainty and observation error.

F. Posterior Predictive Distribution

  • The posterior-predictive distribution accounts for parameter uncertainty and uncertainty due to observation error.The parameter uncertainty is given by the posterior distribution.Source: F. Posterior Predictive Distribution · p. 16

IV. DATA AVAILABILITY

  • The data will be made available at publication.Source: §IV · p. 17

VII. AUTHOR CONTRIBUTIONS

  • CvdO developed efficient ACE uncertainties and led HAL implementation; MS conceived uncertainty-biasing for active-learning exploration.DK helped generate data; CvdO wrote the first manuscript version, and all authors discussed the theory and results and edited the manuscript.Source: §VII · p. 17
Loading 2210.04225v2…