PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimizing many-body atomic descriptors for enhanced computational performance of machine learning based interatomic potentials

Miguel A. Caro

arXiv:1905.02142v4physics.comp-phcond-mat.mtrl-sci

TL;DR

SOAP-based GAP models are accurate, but descriptor evaluation is a computational bottleneck. The paper introduces a separable density representation and analytical expansion methods, reporting approximately tenfold speedups and improved model accuracy.

  • Problem

    SOAP descriptor evaluation is the computational bottleneck in SOAP-based GAP models, limiting access to larger systems and longer molecular-dynamics simulations.

    Source: §I · p. 1

  • Method

    The authors simplify SOAP by using a separable density representation and analytical expansion methods for its coefficients.

    Source: C. Angular expansion coefficients · p. 6, A. Cohesive energy model · p. 9

  • Results

    The new implementation achieves approximately tenfold speedup over QUIP-SOAP in the regions of interest, while the new SOAP yields 10% to 30% greater accuracy for amorphous-carbon GAPs with similar hyperparameters.

    Source: D. Speed · p. 8, A. Cohesive energy model · p. 8

  • Takeaways & Limitations

    With optimized hyperparameters, the new SOAP reduces fitting error by nearly half, which the authors attribute to improved radial basis functions and independently chosen radial and angular widths.

    Source: A. Cohesive energy model · p. 9

    Sophisticated Bayesian optimization strategies for interatomic-potential hyperparameters fall outside the scope of this work.

    Source: C. Hyperparameter optimization for… · p. 10

Abstract

from arXiv · show

We explore different ways to simplify the evaluation of the smooth overlap of atomic positions (SOAP) many-body atomic descriptor [Bart\'{o}k et al., Phys. Rev. B 87, 184115 (2013)]. Our aim is to improve the computational efficiency of SOAP-based similarity kernel construction. While these improved atomic descriptors can be used for general characterization and interpolation of atomic properties, their main target application is accelerated evaluation of machine-learning-based interatomic potentials within the Gaussian approximation potential (GAP) framework [Bart\'{o}k et al., Phys. Rev. Lett. 104, 136403 (2010)]. We achieve this objective by expressing the atomic densities in an approximate separable form, which decouples the radial and angular channels. We then express the elements of the SOAP descriptor (i.e., the expansion coefficients for the atomic densities) in analytical form given a particular choice of radial basis set. Finally, we derive recursion formulas for the expansion coefficients. This new SOAP-based descriptor allows for tenfold speedups compared to previous implementations, while improving the stability of the radial expansion for distant atomic neighbors, without degradation of the interpolation power of GAP models.

I. SMOOTH OVERLAP OF ATOMIC POSITIONS

SOAP encodes atomic environments as rotationally invariant descriptors for comparing neighborhoods and supporting kernel-based learning. Its descriptor evaluation is a computational bottleneck for SOAP-based GAP models.

I. SMOOTH OVERLAP OF ATOMIC POSITIONS

  • SOAP represents an atom’s neighborhood as a rotationally invariant vector derived from a smoothed atomic density within a cutoff sphere.Neighbors beyond the cutoff are excluded.Source: §I · p. 1, Figure 1 · p. 2
  • SOAP-based GAPs can accurately interpolate potential-energy surfaces, but descriptor evaluation remains CPU-expensive and limits larger or longer molecular-dynamics simulations.The proposed simplifications target this computational bottleneck.Source: §I · pp. 1–2
  • The SOAP kernel compares atomic environments through a rotationally averaged density overlap, but evaluating that integral directly is computationally impractical.A basis expansion reformulates the calculation using radial functions and spherical harmonics; products of expansion coefficients form the descriptor.Source: §I · pp. 1–2
  • Normalized power-spectrum vectors define the SOAP many-body descriptors used to measure similarity between atomic environments.The kernel raises the overlap to a power greater than one to retain angular information.Source: §I · p. 2

II. NEW SOAP BASED ON PSEUDOGAUSSIAN FUNCTIONS

The new SOAP formulation approximates atomic densities with separable radial and angular factors, simplifying their expansion. It also permits independent channel widths and radial downscaling of distant neighbors.

II. NEW SOAP BASED ON PSEUDOGAUSSIAN FUNCTIONS

  • The new density representation separates radial and angular dependence, turning radial expansion into an effectively one-dimensional problem.Its spherical-cap geometry approximates a plane more accurately farther from the origin.Source: §II · p. 3
  • Independent radial and angular widths allow the density representation to reflect different characteristic strengths of length- and angle-preserving interactions.The formulation also permits radial dependence in the widths and downscales distant-neighbor contributions.Source: §II · p. 3
  • The radial contribution can be expanded analytically for suitable bases, and polynomial bases support recursion formulas for radial expansion coefficients.The radial and angular factors are recombined into the total expansion coefficients and power spectrum.Source: §II · p. 4
  • Symmetry in the angular coefficients and exchange symmetry of the power spectrum reduce the terms and coefficients that must be evaluated.The m < 0 angular coefficients need not be computed, and exchanging n and n′ avoids further evaluations.Source: §II · p. 4

A. Radial expansion coefficients

Polynomial radial bases enable analytical overlaps and recursion for the radial expansion coefficients. The chosen basis is numerically unstable when nmax exceeds 12.

A. Radial expansion coefficients

  • Polynomial basis functions are orthonormalized using their overlap matrix, and a central Gaussian is added to resolve the central atom exactly.The overlap-matrix-derived transformation need only be computed once for a given nmax.Source: A. Radial expansion coefficients · p. 4
  • Analytical overlaps with the polynomial functions provide coefficients from which the radial expansion coefficients are obtained by recursion.The sequence can be initialized without directly evaluating the first coefficient, and the recursion is valid for α < nmax.Source: A. Radial expansion coefficients · pp. 4–5
  • With this radial basis, linear dependencies in the overlap matrix develop for nmax > 12, making the implementation unstable.Source: A. Radial expansion coefficients · p. 5

B. Density smoothing at the cutoff

Density smoothing prevents sharp changes as atoms cross the SOAP cutoff by tapering contributions through a buffer zone. A Gaussian transition preserves the recursion-based expansion procedure.

B. Density smoothing at the cutoff

  • A soft cutoff retains full atomic densities, a hard cutoff excludes them, and a buffer zone smoothly tapers densities between the two radii.This smoothing avoids sharp kernel discontinuities as atoms move into or out of the cutoff sphere.Source: B. Density smoothing at the cutoff · p. 5
  • With nf = 4, the smoothing function is approximately 3.35 × 10^-4 at the hard cutoff, independent of the cutoff choices.The Gaussian smoothing function preserves the recursion relations because its product with the atomic Gaussian is also Gaussian.Source: B. Density smoothing at the cutoff · p. 6

C. Angular expansion coefficients

The angular expansion is optimized with recursions for the modified spherical Bessel-function product, complex exponentials, and associated Legendre polynomials. A near-endpoint condition handles divergence in the Legendre recursions.

C. Angular expansion coefficients

  • Recursion relations generate ilexp(x; l) values from l = 0 to lmax at the cost of obtaining ilexp(x; lmax), with a Taylor expansion used near zero.The function combines the exponential and modified spherical Bessel terms; the Taylor expansion avoids a singularity at x close to zero.Source: C. Angular expansion coefficients · p. 6
  • The spherical-harmonic conjugate combines factorial normalization, an exponential phase, and associated Legendre polynomials.These are the angular components used in the expansion.Source: C. Angular expansion coefficients · p. 6
  • Computing angular terms keeps only m ≥ 0 and uses recursions for prefactors and complex exponentials, requiring just two intrinsic trigonometric-function calls.Chebyshev recurrences generate the remaining sine and cosine values using sums and multiplications.Source: C. Angular expansion coefficients · pp. 6–7
  • Associated Legendre polynomials are computed by recursions over l and m, initialized with six polynomials.The sequence first obtains P_l0 and P_l1 by recursion on l, then obtains higher-m terms by recursion on m.Source: C. Angular expansion coefficients · p. 7
  • When ||x|−1| < 10^-5, the implementation sets P_lm to zero for l ≥ 3 and m ≥ 2 because the recursions diverge near ±1.The actual associated Legendre polynomials remain finite in this region.Source: C. Angular expansion coefficients · p. 7

D. Speed

The Fortran implementation is benchmarked against QUIP for SOAP density expansion and vector construction. It is approximately ten times faster in the regions of interest, with task costs distributed across radial expansion, angular expansion, and vector construction.

D. Speed

  • The comparison times density expansion and SOAP-vector construction for 10,000 randomly placed atoms, using numerically safe ifort timings across 3 ≤ nmax,lmax ≤ 12.Aggressive ifort optimization was about 40% faster but numerically unstable; numerically safe ifort was about 20% faster than gfortran.Source: D. Speed · p. 7
  • Approximately tenfold speedup over QUIP is achieved in the regions of interest.The figure defines speedup as QUIP timing divided by the new ifort timing.Source: Figure 3 · p. 7, D. Speed · p. 8
  • All 10,000 test-structure descriptors are computed in under one second on a single core; at nmax = lmax = 10, each of the three tasks takes about one third of execution time.Radial expansion is the bottleneck for small bases, while angular expansion becomes the bottleneck as lmax grows.Source: D. Speed · pp. 7–8

IV. GAP MODEL PERFORMANCE

This section tests whether changes to the SOAP density representation and basis expansion preserve machine-learning model accuracy. The authors train GAP models for amorphous carbon and examine hyperparameter effects.

IV. GAP MODEL PERFORMANCE

  • The evaluation asks whether modified SOAP kernels can retain at least the original descriptor's achievable average error per atom for interatomic interactions.The stated objective is to improve SOAP computational efficiency without degrading machine-learning model performance.Source: §IV · p. 8
  • The authors test the modified descriptors with an amorphous-carbon interatomic potential and an adsorption-energy model for the same material.Both models are generated within the GAP framework.Source: §IV · p. 8
  • The section also examines how hyperparameters affect model accuracy and how their values can be optimized.Source: §IV · p. 8

A. Cohesive energy model

For amorphous-carbon cohesive-energy GAPs, the new SOAP descriptor improves accuracy over the old descriptor, with optimized hyperparameters reducing fitting error by nearly half. The section also examines sparse-set size and the old SOAP model’s sensitivity to added data noise.

A. Cohesive energy model

  • The benchmark uses SOAP-only GAPs on an a-C database of about 4k supercells and tests sparse sets of 100–1000 environments.Models were evaluated on 50 held-out 64-atom a-C structures; the new implementation lacked kernel derivatives, so only old-SOAP models were trained with forces.Source: A. Cohesive energy model · p. 8
  • With similar hyperparameters, the new SOAP is 10%–30% more accurate than the old SOAP, depending on sparse-set size.Figure 4 compares RMSE against sparse-set size for cohesive-energy GAPs.Source: A. Cohesive energy model · p. 8, Figure 4 · p. 9
  • For old SOAP without forces, fit quality can worsen as sparse-set size grows, while forces, a less noisy structural kernel, or increased regularization can mitigate it.The authors suggest extra configurations may add data noise; at 1000 sparse samples, doubling regularization appears to improve the fit.Source: A. Cohesive energy model · p. 9, Figure 4 · p. 9
  • Optimized hyperparameters with the new SOAP reduce fitting error by nearly half.The authors attribute the improvement to improved radial basis functions and independently chosen radial and angular sigmas.Source: A. Cohesive energy model · p. 9

B. Adsorption energy model and role of hyperparameter on model performance

The authors use adsorption-energy GAPs for hydrogen on amorphous carbon to explore how hyperparameters affect model performance. Their results show that the cutoff radius has a pronounced effect, the new SOAP handles more distant neighbors without significant degradation, and some hyperparameters may transfer across models using the same structural information.

B. Adsorption energy model and role of hyperparameter on model performance

  • For hydrogen adsorption on a-C, the new SOAP’s performance does not degrade significantly when the cutoff radius increases beyond its optimum.Compared with the old SOAP analysis, the new descriptor allows more distant-neighbor information to improve accuracy.Source: B. Adsorption energy model and role of… · p. 9
  • The cutoff radius has a pronounced effect on performance, while the regularization term σ has a modest impact.The authors explored hundreds of thousands of models through Monte Carlo sampling of hyperparameter space.Source: B. Adsorption energy model and role of… · p. 9
  • The optimal values of σ_r and σ_⊥ differ, and the authors suggest hyperparameters may transfer across models that use the same atomic structural information.The adsorption-energy analysis informed hyperparameter choices that improved cohesive-energy GAP accuracy in the preceding section.Source: B. Adsorption energy model and role of… · p. 9

C. Hyperparameter optimization for interatomic potentials

Because cohesive-energy GAP training is costly, the authors use Sobol sequences to search hyperparameter space and assess the resulting models. The search yields a roughly 10% final improvement and similar optimal hyperparameters to those found with adsorption-energy GAPs, while more sophisticated optimization strategies remain outside the paper’s scope.

C. Hyperparameter optimization for interatomic potentials

  • Bayesian optimization and other sophisticated optimization strategies are outside the scope of this manuscript.The authors note that Bayesian optimization has its own hyperparameters and that acquisition decisions depend on previous observations, complicating parallelization.Source: C. Hyperparameter optimization for… · p. 10
  • Sobol-based optimization yields a final model improvement of ∼10%, with optimal hyperparameters similar to those from adsorption-energy GAPs.Figure 6 compares cohesive-energy GAP RMSE across hyperparameters and sparse-set sizes, with the latter results shown beside Figure 4.Source: C. Hyperparameter optimization for… · p. 10, Figure 6 · p. 11
  • The authors search with 1000 Sobol vectors, training 10 models per hyperparameter combination using randomly selected sparse sets of 100 configurations.They hold n_max = l_max = 8 fixed to keep computational effort comparable.Source: C. Hyperparameter optimization for… · p. 10
  • Directly optimizing hyperparameters on cohesive-energy GAPs improves performance only marginally, and training these models can take hundreds of CPU hours.The authors therefore consider whether a cheaper adsorption-energy GAP could serve as a surrogate for hyperparameter optimization.Source: C. Hyperparameter optimization for… · pp. 9–10

V. CONCLUSIONS AND OUTLOOK

The paper presents a modified SOAP descriptor and recipes for efficient evaluation, with improved accuracy and an expected 10–20× speedup at fixed accuracy.

V. CONCLUSIONS AND OUTLOOK

  • At fixed accuracy, practical applications are expected to gain 10–20× speedup, while the modified SOAP formulation also boosts accuracy.The authors estimate ML-based atomistic simulations could become one order of magnitude cheaper.Source: §V · p. 11
  • The work presents a modified SOAP descriptor and mathematical and computational recipes for efficient evaluation.The descriptor is used as an essential input for GAPs and other ML models for solids and molecules.Source: §V · p. 11
  • The authors suggest simulations of systems with up to one million atoms may now, or soon, become feasible.Source: §V · p. 11
Loading 1905.02142v4…