PaperlayerView exact revision on arXiv

Source-linked AI summary

Operators in Machine Learning: Response Properties in Chemical Space

Anders S. Christensen, Felix A. Faber, O. Anatole von Lilienfeld

arXiv:1807.08811v3physics.chem-ph

TL;DR

Quantum machine-learning models of molecular response properties can converge slowly, motivating the question of whether response operators improve their predictions. The paper applies operators to kernel-based energy models, finding improved predictions for forces and dipole moments and demonstrating vibrational and infrared predictions for small molecules.

  • Problem

    Conventional QML models of response properties can have relatively slow error convergence, even when energy predictions are highly accurate.

    Source: §I · p. 1

  • Method

    The paper applies response operators to a kernel-based energy model to predict responses to nuclear displacement and external electric fields.

    Source: §I · p. 1

  • Results

    The model achieves state-of-the-art force and energy accuracy on MD17 and ISO17, on par with or better than GDML and SchNet, and its dipole learning curve has a lower offset and steeper slope than conventional KRR.

    Source: B. Learning dipole moments of QM9 · p. 6, §V · p. 11

  • Takeaways & Limitations

    Vibrational analysis and an infrared spectrum systematically approach the corresponding ab initio reference as training cases are added.

    Source: §V · p. 11

Abstract

from arXiv · show

The role of response operators is well established in quantum mechanics. We investigate their use for universal quantum machine learning models of response properties in molecules. After introducing a theoretical basis, we present and discuss numerical evidence based on measuring the potential energy's response with respect to atomic displacement and to electric fields. Prediction errors for corresponding properties, atomic forces and dipole moments, improve in a systematic fashion with training set size and reach high accuracy for small training sets. Prediction of normal modes and IR-spectra of some small molecules demonstrates the usefulness of this approach for chemistry.

I. INTRODUCTION

The paper asks whether applying response operators can improve quantum machine-learning models of molecular response properties. It develops a kernel-based approach and applies it to energy responses to nuclear displacements and electric fields.

I. INTRODUCTION

  • The study tests whether response operators yield QML learning curves with lower offsets and better learning rates for molecular response properties.It focuses on energy responses to nuclear coordinates and external electric fields, including forces and dipole moments.Source: §I · p. 1

A. Machine learning model

  • The proposed model predicts response properties by applying the corresponding differential operator to a kernel-based energy model.The introduction describes forces and dipole moments as energy responses to nuclear displacement and an external electric field, respectively.Source: §I · p. 1, A. Machine learning model · p. 2
  • The model represents molecular potential energy as a sum of local atomic contributions, with one regression weight per atomic environment.The kernel matrix is not symmetric because its basis uses atomic kernel functions.Source: A. Machine learning model · p. 2
  • Regression coefficients are obtained from normal equations or orthogonal factorizations, while solving normal equations can be numerically unstable because it squares the condition number.The implementation uses SVD, which remains numerically stable for rank-deficient kernels; underdetermined cases ignore singular values below a threshold.Source: A. Machine learning model · p. 2

B. Operators

The operators act on kernels to model energy responses, including forces, electric-field responses, and second-order properties. The framework specifies the domain for operator-based minimization and derives solutions for first- and second-order responses.

B. Operators

  • The formalism derives solutions for first-order differential operators acting on the energy and kernel, integrating over rotational degrees of freedom.The operator maps a scalar field to a three-dimensional vector field; the domain of minimization depends on the operator.Source: B. Operators · p. 3
  • Second-order response equations can be solved for two different perturbations, extending the operator treatment beyond first-order properties.The supplementary materials provide a step-by-step derivation.Source: B. Operators · p. 3
  • The study defines kernel elements for energy, forces, electric-field responses, the nuclear Hessian, and mixed derivatives used for infrared intensities.The force kernel applies the negative nuclear gradient, while the mixed second-order derivative yields dipole derivatives needed for infrared intensities.Source: B. Operators · p. 3
  • The authors report that they know of no other QML model that accounts for these effects simultaneously.Source: B. Operators · p. 3

C. Comparison to Gaussian Process Regression

The paper contrasts its operator formalism with conventional Gaussian-process regression, where training on function derivatives adds kernel functions for each operator component. Its basis instead enforces derivatives in regression while retaining one coefficient per atom.

C. Comparison to Gaussian Process Regression

  • The formalism enforces kernel derivatives in regression without adding basis kernels, assigning one regression coefficient per atom regardless of perturbation dimensionality.This basis choice has similarities to sparsification by Bartók and Csányi, though the mathematical origins differ.Source: C. Comparison to Gaussian Process… · p. 3
  • For the examples studied, this approach reduces memory requirements by about 10-fold and training times by about 100-fold relative to conventional GPR with the same training data.The number of kernel function evaluations needed for training is also reduced drastically.Source: C. Comparison to Gaussian Process… · p. 3
  • Training errors can become substantial in over-determined cases and were sometimes as large as test-set errors.Source: C. Comparison to Gaussian Process… · p. 3

D. Representation

The paper extends FCHL to include external electric-field dependence through modified scaling functions. This field-aware representation is intended to give kernel elements the physical dependence needed for response properties such as dipole moments.

D. Representation

  • The added field dependence is crucial for learning dipole moments, while the representation's fictitious dipoles and scaling parameter encode the field term.The fictitious dipoles arise from fictitious partial charges; the scaling parameter balances the terms in the scaling function.Source: D. Representation · pp. 3–4
  • The modified FCHL* representation adds an external-field-dependent term to the scaling function to adjust feature weighting and emulate field dependence.It retains the original element-type, interatomic-distance, and interatomic-angle features.Source: D. Representation · p. 4
  • At zero external field, FCHL* kernel elements match the original FCHL elements, but their derivative with respect to the perturbing field is nonzero.Source: D. Representation · p. 4
  • With a linear kernel, this representation is non-polarizable because its second derivative with respect to the field is zero.The paper notes that this could be amended using on-site multipole moments with polarizability tensors.Source: D. Representation · p. 4

III. RESULTS

The H-F toy models test response to nuclear-coordinate and electric-field perturbations, while benchmark sets assess forces and energies across chemical space. Force and dipole training improve the corresponding response predictions, and the operator approach achieves state-of-the-art force accuracy.

1. Toy model for forces

  • With four training points, adding interatomic forces reproduces the H-F potential energy surface almost quantitatively, unlike energy-only training.The energy-only model poorly describes dissociation and predicts a spurious transition state near the minimum-energy distances.Source: §1 · p. 4, Figure 1 · p. 4

2. Toy model for electric field

  • With one training point, including dipole components reproduces the H-F energy curve in a 0.001 a.u. field almost quantitatively as the molecule rotates.Without dipole training, the predicted field-dependent energy change is close to zero.Source: §2 · p. 5, Figure 2 · p. 5

A. Force and energy learning

  • On MD17, energy MAEs were similar across FCHL*, GDML, and SchNet, but dataset-specific winners varied; benzene force learning was weak and GDML modeled uracil best.Source: A. Force and energy learning · p. 5
  • On ISO17, FCHL* learning rates were qualitatively comparable to SchNet across known- and unknown-connectivity tests, but nonoverlapping curves prevent quantitative comparison.Source: A. Force and energy learning · p. 5
  • The operator approach predicts forces with state-of-the-art accuracy, on par with two of the most accurate published models.Source: A. Force and energy learning · p. 5

B. Learning dipole moments of QM9

The QM9 dipole comparison reports lower and steeper error learning when dipoles are predicted as energy derivatives rather than learned as scalars. For normal modes, larger training sets generally reduce failures, though low-frequency, non-local modes require more data than local distortions.

B. Learning dipole moments of QM9

  • At 5,000 QM9 training molecules, the dipole-norm MAE falls 54%, from 0.67 Debye to 0.31 Debye, with the energy-derivative approach.Its learning curve also has a lower offset and is steeper than conventional KRR.Source: B. Learning dipole moments of QM9 · pp. 5–6

C. Learning normal modes

  • For the illustrated molecule, 11 modes fail at N = 1 sample per fragment, two at N = 2, and all have minima inside the scan range at N = 4.At N = 8, two low modes involving very non-local conformational changes fall outside the range again; at N = 32, the QM energy curves are almost quantitatively reproduced.Source: C. Learning normal modes · pp. 6–7
  • Higher-frequency, mostly local distortions are almost always well reproduced, whereas lower-frequency, flatter and often non-local modes require larger training sets.Source: C. Learning normal modes · p. 7
  • Across the 83 nine-heavy-atom QM9 molecules, larger training sets reduce the fraction of normal-mode predictions that fail to find a nearby, well-defined minimum.The study trains on distorted sub-fragment geometries and tests energy scans along reference normal modes.Source: C. Learning normal modes · pp. 6, 8

D. Infrared spectrum for Dichloromethane

The dichloromethane IR-spectrum test combines learned energies, forces, and dipoles with vibrational analysis. The learned spectrum approaches the MP2 reference as training data increase, reaching an MAE of 2.5 cm−1 with 100 samples; IR intensities are mixed second derivatives, despite training on energy and first derivatives.

D. Infrared spectrum for Dichloromethane

  • The learned model supplies energies and derivatives for vibrational analysis after training on distorted geometries with MP2 energies, forces, and dipole moments.The training set contains 100 configurations generated by normal-mode sampling.Source: D. Infrared spectrum for Dichloromethane · p. 10
  • At 100 training samples, the IR-spectrum vibrational-frequency MAE is 2.5 cm−1, and the three most intense peaks are at 743, 793, and 1318 cm−1 versus MP2 values of 740, 793, and 1315 cm−1.With 50 samples, the MAE is 5.7 cm−1; five samples do not yield a meaningful spectrum.Source: D. Infrared spectrum for Dichloromethane · p. 10, Figure 9 · p. 10
  • IR intensities are mixed second derivatives, so the reported result demonstrates higher-order response from training that includes only energy and first-order derivatives.The authors suggest that systematically adding higher-order effects could further improve performance.Source: D. Infrared spectrum for Dichloromethane · p. 10

A. Used software

The paper reports state-of-the-art force and energy accuracy on MD17 and ISO17, and a 54% improvement in learning QM9 dipole norms over scalar learning with the same representation. It also demonstrates vibrational analyses and discusses extensions to other molecular properties.

A. Used software

  • Calculations used ORCA for HF and QM9 dipoles, Gaussian09 for QM9 vibrational analyses and dichloromethane properties, and Gaussian09 for ML vibrational analyses.The HF calculations used MP2/aug-cc-pVTZ; QM9 dipoles were recalculated at the original dataset's B3LYP/6-31G(2df,p) level, and QM9 vibrational calculations used ωB97xD/6-31G(d).Source: A. Used software · pp. 10–11

B. Hyperparameters

  • The FCHL* and Gaussian-kernel hyperparameters were held fixed, and the new scaling parameter was set to ϵ = 0.0005 Hartree−1.The parameters were not rigorously fitted to any dataset, so more optimal values may exist.Source: B. Hyperparameters · p. 11

V. CONCLUSION

  • Force and energy predictions reach state-of-the-art accuracy on MD17 and ISO17, on par with or better than GDML and SchNet.The operator formalism improves QM9 dipole-norm learning by 54% compared with learning the same quantity as a scalar using the same representation.Source: §V · p. 11
  • The predicted infrared spectrum systematically approaches the ab initio reference as more training cases are added.The spectrum comes from a vibrational analysis using the resulting model.Source: §V · p. 11
  • The authors envision extending the formalism to higher-order and magnetic response properties, alchemical derivatives, and properties beyond response operators.Examples include NMR chemical shifts, spin-spin coupling constants, electronic density, and kinetic energy.Source: §V · p. 11

VI. SUPPLEMENTARY INFORMATION FOR “OPERATORS IN MACHINE LEARNING: RESPONSE PROPERTIES IN CHEMICAL SPACE”: DERIVATION OF OPERATORS

The supplementary derivation builds operator-based regression from local kernel contributions to molecular energy, then obtains response predictions and regression coefficients through an operator-applied objective.

Derivation of operators

  • The query molecule's total potential energy is decomposed into local atomic energy contributions calculated as weighted sums of kernels.The sum runs over the molecule's atomic environments.Source: §VI · p. 12
  • A response property is calculated by applying its corresponding response operator to the energy.Source: §VI · p. 12
  • The regression coefficients are obtained by minimizing a Lagrangian over training values of the operator-applied reference energy, yielding the normal-equation solution.The derivation expands the objective and differentiates it with respect to the coefficients.Source: §VI · p. 12

First-Order Differential Operators

The first-order treatment applies differential operators to energy and kernels, with rotational degrees of freedom handled through a spherical integration domain.

First-Order Differential Operators

  • The treatment gives a solution for any first-order differential operator acting on the energy and kernel.The operators correspond to gradients with respect to three-dimensional variables, including nuclear coordinates and external fields.Source: First-Order Differential Operators · p. 12
  • The gradient is projected onto a spherical coordinate basis, and the associated integrals account for rotational degrees of freedom.Source: First-Order Differential Operators · p. 13

Second-Order Differential Operators

The second-order treatment defines an operator such as the energy Hessian, projects it onto spherical coordinates, and gives the corresponding rotational integrals.

Second-Order Differential Operators

Related papers

Loading 1807.08811v3…