Alchemical and structural distribution based representation for improved QML

Felix A. Faber; Anders S. Christensen; Bing Huang; O. Anatole von Lilienfeld

Source-linked AI summary

Alchemical and structural distribution based representation for improved QML

Felix A. Faber, Anders S. Christensen, Bing Huang, O. Anatole von Lilienfeld

arXiv:1712.08417v1Dec 22, 2017physics.chem-ph

TL;DR

Accurate electronic-structure methods are computationally expensive, and QML performance depends on representations that capture chemically meaningful similarities. The paper introduces scaled distribution functions for atomic environments that encode both structural and elemental degrees of freedom. The resulting models show favorable learning curves across diverse systems and can predict covalent bonding involving element pairs absent from training.

Problem
Accurate ground-state property calculations are computationally expensive, while direct system-variable similarities provide disappointing QML predictive power.
Method
The paper represents each atom with Gaussian M-body distributions over structural variables and periodic-table elemental parameters, using scaling functions and QML models.
Results
The models show favorable learning curves for diverse out-of-sample systems and semi-qualitatively accurate single, double, and triple covalent-bond potentials involving untrained element pairs.
Takeaways & Limitations
Explicit elemental components lower learning curves and enable alchemical interpolation across the periodic table for chemically diverse QML applications.

Abstract

from arXiv · show

We introduce a representation of any atom in any chemical environment for the generation of efficient quantum machine learning (QML) models of common electronic ground-state properties. The representation is based on scaled distribution functions explicitly accounting for elemental and structural degrees of freedom. Resulting QML models afford very favorable learning curves for properties of out-of-sample systems including organic molecules, non-covalently bonded protein side-chains, (H$_2$O)$_{40}$-clusters, as well as diverse crystals. The elemental components help to lower the learning curves, and, through interpolation across the periodic table, even enable "alchemical extrapolation" to covalent bonding between elements not part of training, as evinced for single, double, and triple bonds among main-group elements.

I. INTRODUCTION

Ground-state property calculations can be accurate but expensive, motivating QML models whose performance depends critically on chemical representations. The paper introduces an atomic representation that combines structural information with elemental, or alchemical, information rather than separating atom types.

Ab initio quantum chemistry and DFT can estimate ground-state properties accurately, but their computational expense limits applicability especially for larger systems.
Directly comparing external potentials gives disappointing QML predictive power, while learned neural-network representations entangle representation design with the supervised model.
Distribution-based representations systematically encode internal coordinates, complementing alternatives such as interatomic potentials, basis-function fingerprints, and topology fingerprints.
The new representation encodes composition directly into distributions instead of binning by atomic type, enabling structural and alchemical differences between environments.
Kernel ridge regression is used because of its robustness, simple convergence, and computational efficiency, although other properly converged regressors may produce similar results.
The KRR model predicts a query property as a weighted sum of kernel functions centered on training compounds, with weights obtained by regularized linear regression.

B. Representation

The representation describes each atom through Gaussian distributions over elemental and structural degrees of freedom, including composition, pairwise distances, and angles. Scaling functions weight these terms, while fourth-body contributions were found to add negligible predictive benefit relative to their cost.

B. Representation: M-body expansions represent an atom’s chemical and structural environment using Gaussian basis functions over elemental parameters and structural variables, weighted by scaling functions.
B. Representation: Four-body terms were tested but produced negligible predictive improvements over the three-body expansion while substantially increasing kernel-generation cost.
B. Representation: A1 encodes stoichiometry as a Gaussian over an atom’s periodic-table period and group, with σP and σG controlling elemental smearing.
B. Representation: A2 multiplies the composition term by neighbor-dependent Gaussians over interatomic distance, period, and group, with ξ2 applying distance-dependent scaling.
B. Representation: Setting elemental widths toward zero recovers separate distributions for each chemical element tuple, whereas finite widths permit interpolation across elements.
B. Representation: A3 extends A2 with a second neighboring atom and represents pair distances, bond angle, and elemental parameters using an additional scaling function.
B. Representation: The chosen scaling functions are modified power laws inspired by London 1/r^6 and Axilrod–Teller–Muto 1/r^9 dispersion laws, with exponents selected for predictive performance.

C. Distances and scalar products

Atomic-environment distances are constructed by combining the distances between corresponding body-order distributions. These distribution distances use normalized Euclidean norms, with analytical Gaussian-product integrals and Fourier methods helping manage higher-order computational cost.

C. Distances and scalar products: The distance between two atomic environments is a weighted sum of the distances between their corresponding m-body expansions.
C. Distances and scalar products: Each distribution-term distance is evaluated with an L2 norm, while normalization factors ensure individual Gaussian basis functions have unit L2 norm.
C. Distances and scalar products: The required integrals can be solved analytically because the distributions consist of sums of Gaussian products.
C. Distances and scalar products: Third- and fourth-order terms are prohibitively expensive to calculate directly, but modified distributions and Fourier-space angular integration can substantially reduce that burden.

D. Comparison to other distribution based representations

The A3 representation distinguishes water geometries more uniquely and models nonlinear three-body energy surfaces more accurately than RDF, ADF, or their combination.

Geometric comparison: A3 produces a single well-defined distance minimum around equilibrium, whereas RDF and ADF show broad near-zero-distance configurational domains for oxygen and hydrogen.The comparison uses normalized L2 distances while one water molecule is stretched and bent relative to a fixed equilibrium molecule.
Energy modeling: RDF, ADF, and their linear combination decouple distances and angles, preventing them from representing general three-body surfaces E(d, θ).A3 retains coupled distance-angle information and can, by construction, model any three-body potential.
Interpretation: The linear mapping between A_n and an n-body potential-energy surface helps explain why the new representation can improve performance with nonlinear kernels.

1. Hyper-parameters

The model combines several hyperparameters, but only a small subset strongly affects out-of-sample errors; some low-composition datasets omit alchemical smearing.

Hyper-parameters: Only a small subset of the representation and KRR hyperparameters is sensitive, while many exact choices have little effect on out-of-sample errors.The authors report that models typically perform well across noncritical parameter choices.
Dataset-specific settings: The water-cluster and SSI datasets have little chemical-composition variation, so no alchemical smearing is used for them.
Evaluation setup: The water-energy comparison evaluates A3, RDF, ADF, and RDF + ADF using linear-kernel KRR errors over a grid of bond lengths and angles.

2. Scaling powerlaw parameters

The authors screen radial scaling exponents on QM9 and select n2 = 4 for A2 and n3 = 4 for A3 for subsequent calculations.

Exponent selection: n2 = 4 gives the lowest offset when ξ2 is screened with A2 on a QM9 subset.
Exponent selection: n3 = 4 performs best when ξ3 is screened after fixing ξ2 at its selected exponent.These selected exponents are used throughout the study unless otherwise specified.
Exponent selection: Figure 4 reports out-of-sample MAE against training-set size for A2 and A3, with separate curves for candidate exponents n2 and n3.

3. Alchemical smearing

Elemental-smearing widths affect prediction accuracy most in chemically diverse datasets, with optimized widths substantially reducing OQMD errors but offering weaker gains on QM9.

Alchemical smearing: The optimal alchemical Gaussian widths vary only slightly between OQMD and QM9 despite their different chemical diversity.OQMD has relatively high elemental diversity, whereas QM9 has relatively low diversity.
Alchemical smearing: ∼9.9% lower MAE occurs on OQMD at 100 training samples, increasing to ∼34% at 1k samples, when optimized widths replace σP = σG = 0.1.The baseline effectively has zero overlap between different atomic types.
Alchemical smearing: QM9 shows similar but much less pronounced error reductions, and its 1000-molecule optimization well is very shallow.The shallow minimum is consistent with QM9’s limited compositional diversity.
Scope boundary: Elemental overlap is not always beneficial for low-diversity datasets because including it is computationally more expensive.
Evaluation scope: The benchmark spans organic molecules, crystals, biomolecular dimers, water clusters, and main-group diatomics.

C. Biomolecular dimers: SSI

The benchmark spans biomolecular dimers, water clusters, crystals, and main-group diatomics to assess out-of-sample energy prediction and alchemical extrapolation. Learning curves provide the central evaluation framework across these datasets.

C. Biomolecular dimers: SSI: The SSI benchmark uses 2,356 neutral protein-sidechain dimers representing interacting amino-acid sidechains from protein crystal structures.Their reference energies correspond to the DW-CCSD(T**)-F12 level of theory.
C. Biomolecular dimers: SSI: The water-cluster benchmark contains 4,000 snapshots of a 40-molecule water cluster generated from a 300 K molecular-dynamics trajectory.Energies were computed with PBEh-3c for each geometry.
C. Biomolecular dimers: SSI: The OQMD benchmark uses 3,000 crystal structures with fewer than 40 unit-cell atoms and formation energies calculated using high-throughput DFT.The underlying database contains approximately 30,000 crystal structures.
C. Biomolecular dimers: SSI: The evaluation also includes published single, double, and triple main-group diatomic bond data to test alchemical interpolation.Learning curves report out-of-sample errors as training-set size varies across molecular, biomolecular, water, and crystalline datasets.

A. Energies of molecules, clusters, and solids

The representation delivers strong energy-prediction learning curves across molecules, clusters, and solids, while its elemental encoding supports semiquantitative extrapolation to unseen main-group bonds. Atomic representations learn particularly rapidly for non-equilibrium water clusters, and performance varies across crystal benchmarks.

A. Energies of molecules, clusters, and solids: QM9 reaches 1 kcal/mol chemical accuracy with 2k training points, whereas earlier QML models required an order of magnitude more molecules.The threshold is approximately 0.043 eV for enthalpy of formation.
A. Energies of molecules, clusters, and solids: The new QML model has the lowest MAE for all datasets at large training sizes, with errors consistently lower or similar to previously published models.The comparison covers QM9, QM7b, SSI, water, Elpasolites, and OQMD.
A. Energies of molecules, clusters, and solids: Atomic representations improve rapidly with increasing training data for non-equilibrium water clusters, unlike global representations that learn little initially.The authors associate this contrast with sorting and crowding in global representations, which complicates geometrical energy variations.
A. Energies of molecules, clusters, and solids: On OQMD, the new model has lower MAE than the sine matrix at every training size and surpasses the Voronoi model because its learning rate is steeper.The model starts with a larger offset than Voronoi but learns more steeply.
A. Energies of molecules, clusters, and solids: On compositionally diverse Elpasolites with identical crystal structures, the new model starts below FLLA but approaches the same learning-curve slope at larger training sizes.FLLA encodes periodic row and column rather than explicit coordinates.
B. Alchemical predictions: Alchemical extrapolation produces semiquantitative, physically adequate covalent-bond potentials for unseen single, double, and triple main-group bonds across periods 2 to 4.Chemical accuracy is not reached; predictions for central H2SiS are more accurate, while second-row elements show the largest deviations.

C. Other ground state properties of molecules

Across nine QM9 electronic ground-state properties, models using the new atomic representation improve systematically as training data increases. Higher-order structural contributions generally lower learning curves, while the representation also supports strong energy learning across diverse systems and element pairs absent from training.

QM9 ground-state properties: Systematic improvement with training-set size occurs for all nine properties, including atomization energy, orbital energies, gaps, dipole moment, polarizability, ZPVE, heat capacity, and highest vibrational frequency.The evaluation compares out-of-sample MAE and RMSE learning curves for models using atomic kernels with the new representation and molecular-kernel BAML models.
QM9 ground-state properties: RMSE learning curves improve with training-set size at slopes similar to MAE, indicating that predictions for outliers also improve.RMSE has the expected slightly higher offset than MAE.
Structural contributions: Adding higher-order structural terms systematically lowers learning curves by a significant amount for every property except ω.The comparison distinguishes two- and three-body contributions, with four-body terms additionally included for BAML.
Representation: The representation models atoms as sums of multidimensional Gaussians encoding elemental, atom-pairwise, and angular distributions with power-law scaling.For the compounds and properties studied, four-body contributions were found to be insignificant.
Energy learning: For organic molecules, water clusters, amino-acid side chains, and crystalline solids, the resulting energy models achieve low-offset, steep learning curves.For compositionally diverse systems, chemical accuracy of approximately 1 kcal/mol can be reached with only thousands of training instances.
Alchemical extrapolation and outlook: The model produces semi-qualitatively accurate single-, double-, and triple-bonding potentials for element pairs absent from training, while steep learning curves suggest high-level reference data could support high-throughput screening.The paper also identifies future extensions combining the representation with the amon approach and modeling forces and other properties.