Automatic Selection of Atomic Fingerprints and Reference Configurations for Machine-Learning Potentials

Giulio Imbalzano; Andrea Anelli; Daniele Giofr é; Sinja Klees; J örg Behler; Michele Ceriotti

Source-linked AI summary

Automatic Selection of Atomic Fingerprints and Reference Configurations for Machine-Learning Potentials

Giulio Imbalzano, Andrea Anelli, Daniele Giofr é, Sinja Klees, J örg Behler, Michele Ceriotti

arXiv:1804.02150v1Apr 6, 2018physics.comp-phcond-mat.mtrl-sciphysics.chem-ph

TL;DR

Machine-learning potentials need representations that balance physical information with computational cost, but common fingerprint families can be highly redundant or enormous. This paper develops unsupervised selection protocols based on training geometries, and applies them to neural-network and Gaussian-process models, finding that compact selections can preserve accuracy while reducing cost.

Problem
Fingerprint choice strongly affects the accuracy, speed, and reliability of machine-learning potentials, while candidate representations can be computationally costly and redundant.
Method
The paper uses unsupervised linear-algebra and correlation-based protocols to select a compact subset of fingerprints from large candidate pools using reference geometries.
Results
The selected fingerprints achieve the same prediction error as physically motivated water symmetry functions at lower evaluation cost, while compact SOAP subsets can match or outperform full-kernel Gaussian-process regression.
Takeaways & Limitations
Automatic feature selection can simplify and greatly accelerate machine-learning models while retaining sufficient structural or chemical information for the studied systems.

Abstract

from arXiv · show

Machine learning of atomic-scale properties is revolutionizing molecular modelling, making it possible to evaluate inter-atomic potentials with first-principles accuracy, at a fraction of the costs. The accuracy, speed and reliability of machine-learning potentials, however, depends strongly on the way atomic configurations are represented, i.e. the choice of descriptors used as input for the machine learning method. The raw Cartesian coordinates are typically transformed in "fingerprints", or "symmetry functions", that are designed to encode, in addition to the structure, important properties of the potential-energy surface like its invariances with respect to rotation, translation and permutation of like atoms. Here we discuss automatic protocols to select a number of fingerprints out of a large pool of candidates, based on the correlations that are intrinsic to the training data. This procedure can greatly simplify the construction of neural network potentials that strike the best balance between accuracy and computational efficiency, and has the potential to accelerate by orders of magnitude the evaluation of Gaussian Approximation Potentials based on the Smooth Overlap of Atomic Positions kernel. We present applications to the construction of neural network potentials for water and for an Al-Mg-Si alloy, and to the prediction of the formation energies of small organic molecules using Gaussian process regression.

I. INTRODUCTION

The paper motivates careful fingerprint design because machine-learning accuracy and efficiency depend strongly on the representation of atomic environments. It introduces radial and angular symmetry functions and discusses generating broad candidate pools whose redundancy and cost require later selection.

Machine-learning representations must encode physically relevant structure and invariances while supporting efficient interpolation and transferability.
Behler-Parrinello symmetry functions describe different correlations between particles through parameterized families, making parameter selection crucial.
SOAP offers systematic convergence but can require tens of thousands of fingerprint elements, substantially increasing computational cost.
G2 functions encode pair correlations, whereas G3 functions encode angular correlations within a central atom’s neighborhood.
The candidate pool spans radial G2 functions centered at Rs = 0 or shifted along neighbor paths, with parameters varied heuristically across meaningful correlations.
Increasing cutoff radius and candidate count improves completeness but raises evaluation cost, while strongly correlated functions can cause overfitting and regression difficulties.

B. Smooth Overlap of Atomic Positions

SOAP represents local atomic environments with smooth species-resolved densities and rotationally averaged overlaps. Its systematically convergent power-spectrum representation is expressive but computationally expensive, motivating reduced fingerprint subsets.

SOAP constructs each local environment from smooth atomic probability amplitudes formed by Gaussian densities around neighboring atoms.
Rotational averaging of density overlaps makes the SOAP kernel invariant to relative rotations, complementing translation and within-species permutation invariance.
For n = 2, the SOAP overlap becomes a scalar product between power spectra that encode correlations between chemical species.
Each power-spectrum element is rotationally invariant, with the representation size controlled by nmax and lmax.
Increasing radial and angular cutoffs yields systematic kernel convergence but requires computing and processing tens of thousands of power-spectrum elements.

III. FINGERPRINT SELECTION

The paper formulates fingerprint selection as unsupervised dimensionality reduction over feature matrices built from representative atomic environments. The goal is to retain essential structural information with far fewer original features and little or no prediction loss.

Both symmetry-function and SOAP representations can produce arbitrarily high-dimensional feature spaces.
Unlike supervised trial-and-error selection, the proposed approaches use only reference-structure geometries, not energies, forces, or model performance.
The feature matrix X contains fingerprint values for representative structures, allowing standard linear-algebra methods to identify informative columns.
The same selected features can support regression schemes including linear fits, Gaussian-process regression, and neural networks.
The selected matrix X′ has N′ much smaller than N while preserving a satisfactory representation and reducing machine-learning computational load.

A. CUR Decomposition

CUR decomposition selects actual fingerprint columns that approximate the full feature matrix, avoiding replacement by linear combinations of descriptors. A deterministic iterative procedure ranks features while suppressing near-duplicate selections.

CUR decomposition represents the low-rank approximation using actual rows and columns of the original feature matrix.
Selecting columns corresponds to reducing fingerprints, while the same framework can also reduce rows representing reference structures.
Each candidate fingerprint receives an importance score derived from coordinates of right singular vectors.
CUR(k = 1) repeatedly selects the highest-scoring feature, improving numerical stability and speed while producing a ranking independent of the target feature count.
After each selection, remaining columns are orthogonalized and the singular decomposition and weights are recomputed to avoid multiple nearly identical features.
The number of selected features may be fixed beforehand or increased until the approximation error falls below a prescribed threshold.

B. Farthest Point Sampling

Farthest-point sampling selects fingerprints by treating feature columns as points and iteratively choosing diverse candidates.

B. Farthest Point Sampling: FPS applies a reference-point strategy to the columns of the feature matrix X.It selects fingerprints rather than reference structures in this procedure.
B. Farthest Point Sampling: After an arbitrary first selection, each subsequent fingerprint maximizes its Euclidean distance from previously selected fingerprints.The procedure repeats until N′ features have been chosen.

C. Pearson Correlation Method

The Pearson-correlation method selects candidate fingerprints sequentially to minimize redundancy in their values across the available atomic data.

C. Pearson Correlation Method: The PC method chooses candidate features one after another to minimize correlations among their values in the data set.The Pearson correlation is computed for features associated with atoms of the same element.
C. Pearson Correlation Method: Pearson correlation is computed from feature values, their arithmetic means, and standard deviations over atoms of the respective element.These quantities define the normalized correlation between two features.
C. Pearson Correlation Method: The procedure starts with a broad radial G2 function, then adds the least-correlated function and subsequently minimizes average correlation to selected functions.Selection is performed separately for each element in the system.

D. Global Fingerprints and Train Set Selection

The framework can also sparsify training structures rather than fingerprints by constructing global structure-level fingerprints from local atomic-environment descriptors.

D. Global Fingerprints and Train Set Selection: CUR, FPS, or PC can reduce the number of reference structures instead of the number of fingerprints.This can reduce ridge-regression evaluation costs or the number of property evaluations needed for training.
D. Global Fingerprints and Train Set Selection: A global fingerprint for structure A is defined as the average of local fingerprints over the environments composing that structure.This converts atomic-environment descriptors into a structure-level representation for train-set selection.
D. Global Fingerprints and Train Set Selection: For Behler-Parrinello symmetry functions, the global fingerprint concatenates sections corresponding to each chemical element.The representation is sparse because sections unrelated to the relevant central-atom identity are padded with zeros.

A. A Potential for Liquid Water

The liquid-water study benchmarks automatically selected symmetry-function sets of varying sizes against default and manually optimized references using neural-network potentials.

A. A Potential for Liquid Water: 768 candidate symmetry functions were generated for water, with duplicate and sub-0.75 Å length-scale functions removed before selection.The selection and training data comprised 1000 structures selected by FPS.
A. A Potential for Liquid Water: The selection scores can be weighted by estimated evaluation cost, while not entering the final symmetry-function form used in fitting.The water setup applies density- and cutoff-dependent weighting to G3 importance scores.
A. A Potential for Liquid Water: Water potentials were evaluated with CUR, FPS, and PC selections containing N′ = 16, 32, or 64 functions per element, alongside default and Ref. sets.Each set was trained in four neural networks using a 3:1 random train:test split.
A. A Potential for Liquid Water: All automatic selection strategies progressively improve test accuracy as the selection becomes more inclusive.CUR performs best for both feature-matrix approximation and energy and force test RMSE, followed by PC and FPS.
A. A Potential for Liquid Water: All automatic protocols outperform the default symmetry-function set, with the improvement described as dramatic for CUR.The table reports energy and force RMSE averaged over four independently initialized neural networks.
A. A Potential for Liquid Water: The manually optimized Ref. set achieves the same accuracy with 57 functions as CUR achieves with 128.Automatic selection nevertheless requires lower evaluation effort because estimated symmetry-function cost is included during selection.

B. A Potential for Aluminum Alloys

Automatic fingerprint selection is applied to an Al-Mg-Si alloy using representative structures and candidate G2/G3 symmetry functions. The resulting potentials improve interpolative accuracy while reproducing vacancy-migration behavior qualitatively and quantitatively within a DFT-comparable error.

Dataset: The alloy dataset contains 11,160 structures, from which 2,000 FPS-selected structures are used for fingerprint selection and training/testing.The source combines 10,551 structures with 609 β′′-phase precipitate and interface structures.
Fingerprint pool: The candidate pool includes six G2 sets and two G3 sets spanning multiple cutoffs, widths, and angular parameters.Duplicate and overly narrow symmetry functions are removed before selection.
Extrapolative test: The study evaluates potential stability on vacancy-assisted Al, Mg, and Si migration pathways not explicitly included in training.Energies are computed for both linear transition paths and corresponding relaxed positions.
Results: Increasing from 16 to 32 symmetry functions per species substantially improves the fit, whereas adding more functions yields smaller gains and can worsen Si diffusion.This contrasts systematic interpolative improvement with non-systematic extrapolative improvement.
Results: Except with 16 functions per element, relaxed migration-barrier errors remain below 0.1 eV, comparable to typical DFT error.Selections including three-body terms outperform the G2-only choice from Ref. 51, although the latter also predicts barriers with small error.

C. Learning Molecular Energies

The molecular-energy application reduces the very large SOAP feature space before Gaussian-process regression. CUR and FPS selection preserve or improve prediction accuracy on QM7b while substantially reducing the number of power-spectrum elements used.

SOAP representation: SOAP with rc = 3.0 Å, nmax = 9, and lmax = 9 produces 14,852 rotationally invariant symmetry functions.The feature count is much larger than the handful of symmetry functions commonly used in BPNN potentials.
Kernel reduction: Feature selection reduces the SOAP power spectrum while retaining a kernel formulation through a transformed reduced feature set.The transformation uses X′ = CA, and matrix A must be stored and applied during training and prediction.
QM7b results: Using 100 CUR-selected power-spectrum elements matches the full-kernel accuracy on 1,442 QM7b test structures trained with 500 FPS structures.The full SOAP kernel uses more than 14,000 features, while 400 selected features produce lower test error for this small training set.
QM7b results: At 400 selected elements, FPS yields a test RMSE 5% lower than the baseline SOAP result.Both CUR and FPS are evaluated using the same 500 FPS training structures.
Learning-curve behavior: With all 5,769 training structures, the baseline kernel becomes marginally more accurate, while the reduced model discards 97% of SOAP power-spectrum elements.With fewer training structures, 400 features achieve errors comparable to or lower than the reference value.

V. CONCLUSIONS

The paper presents an automatic, unsupervised scheme for selecting informative fingerprints, demonstrating accuracy retention or improvement across neural-network and Gaussian-process applications while reducing computational cost.

Automatic feature selection preserves the most informative descriptors from large candidate pools using relatively standard linear algebra methods.The scheme is designed to be transferable across systems and descriptor families.
For water, a small automatically selected set of Behler-Parrinello symmetry functions matches the reference prediction error while reducing evaluation cost.The reference uses a physically motivated symmetry-function set.
For an Al-Si-Mg alloy, neural-network potentials using CUR-selected symmetry functions outperform a previous potential in test RMSE and vacancy-assisted atom migration barriers.
3% of the SOAP power-spectrum components match or outperform Gaussian-process regression using the entire kernel for small-organic-molecule atomization energies.The reduced representation retains sufficient chemical information for substantially lower-cost SOAP-based prediction.
The results suggest that successful regression models for interatomic potentials may exploit a relatively low-dimensional manifold of energetically accessible molecular motifs.The authors identify manifold analysis and adaptation to accumulating simulation structures as future directions.