PaperlayerView exact revision on arXiv

Source-linked AI summary

Boosting quantum machine learning models with multi-level combination technique: Pople diagrams revisited

Peter Zaspel, Bing Huang, Helmut Harbrecht, O. Anatole von Lilienfeld

arXiv:1808.02799v2physics.chem-ph

TL;DR

High-accuracy quantum machine-learning predictions require costly reference data, making large training sets at demanding theory levels difficult to generate. The paper combines machine-learning models across theory, basis-set, and training-molecule levels; its 3D results indicate that substantially fewer highest-level training instances can reach chemical accuracy. The authors conclude that exploiting correlations across these hierarchies can reduce the need for costly high-level reference numbers.

  • Problem

    Generating thousands of training values at CCSD(T) with large basis sets remains prohibitive, although such costly theories are needed to reach chemical accuracy for thermochemistry.

    Source: §I · p. 1

  • Method

    CQML combines independently trained kernel ridge regression models across theory, basis-set, and training-molecule levels using a multi-level combination approach.

    Source: §I · p. 2

  • Results

    Adding levels and spaces systematically reduces required highest-level training data; 3D CQML extrapolates chemical accuracy with 100 CCSD(T)/cc-pvdz training instances at s = 2.

    Source: C. 3D results for QM7b · p. 10, §V · p. 11

  • Takeaways & Limitations

    The results support using smaller-basis and more approximate-correlation training data to reduce the need for high-level reference numbers.

    Source: C. 3D results for QM7b · p. 10

Abstract

from arXiv · show

Inspired by Pople diagrams popular in quantum chemistry, we introduce a hierarchical scheme, based on the multi-level combination (C) technique, to combine various levels of approximations made when calculating molecular energies within quantum chemistry. When combined with quantum machine learning (QML) models, the resulting CQML model is a generalized unified recursive kernel ridge regression which exploits correlations implicitly encoded in training data comprised of multiple levels in multiple dimensions. Here, we have investigated up to three dimensions: Chemical space, basis set, and electron correlation treatment. Numerical results have been obtained for atomization energies of a set of $\sim$7'000 organic molecules with up to 7 atoms (not counting hydrogens) containing CHONFClS, as well as for $\sim$6'000 constitutional isomers of C$_7$H$_{10}$O$_2$. CQML learning curves for atomization energies suggest a dramatic reduction in necessary training samples calculated with the most accurate and costly method. In order to generate milli-second estimates of CCSD(T)/cc-pvdz atomization energies with prediction errors reaching chemical accuracy ($\sim$1 kcal/mol), the CQML model requires only $\sim$100 training instances at CCSD(T)/cc-pvdz level, rather than thousands within conventional QML, while more training molecules are required at lower levels. Our results suggest a possibly favourable trade-off between various hierarchical approximations whose computational cost scales differently with electron number.

I. INTRODUCTION

Quantum machine learning can predict molecular properties from training data, but high-accuracy predictions still depend on costly reference calculations. The paper introduces CQML and evaluates it on two molecular datasets using multiple theory levels and molecular representations.

I. INTRODUCTION

  • CCSD(T) calculations in large basis sets remain prohibitive for generating thousands of high-accuracy training values, despite QML prediction errors below DFT accuracy.The target is chemical accuracy of about 1 kcal/mol for thermochemistry.Source: §I · p. 1
  • The study combines electron-correlation levels, basis sets, and training-molecule counts, finding that adding levels and basis sets improves learning results at target accuracy.The results include multi-level and multi-space models for QM7b molecules.Source: §I · p. 2

A. Datasets

  • QM7b contains 7,211 molecules with up to seven heavy atoms, while CI9 contains 6,095 constitutional isomers of C7H10O2.QM7b molecules span HF, MP2, and CCSD(T) calculations with three basis sets; CI9 uses PM7, B3LYP/6-31g(2df,p), and G4MP2.Source: A. Datasets · p. 2

B. QML details

  • QM7b models use SLATM, while CI9 models use sorted Coulomb matrix or SLATM; the study reports out-of-sample MAE averaged over 20 training runs.The reported errors compare CQML predictions with the target theory level on 200 randomly selected test molecules.Source: B. QML details · p. 2

III. THEORY

Quantum chemistry composite methods use systematic error cancellation to approximate accurate energies at reduced computational cost. The paper develops CQML as a machine-learning generalization of this strategy, combining levels of theory and training data.

A. From Pople diagrams to CQML

  • The Pople-diagram adaptation places theory levels on the horizontal axis and molecular spaces on the vertical axis, illustrating CQML’s combination of increasingly accurate methods and fewer training molecules.Its bold diagonals represent qualitative constant cost-benefit ratios and Pareto-optimal solutions.Source: Figure 1 · p. 3
  • Composite methods combine quantum-chemical approximations to reach high accuracy, often chemical accuracy, at lower computational burden than brute-force convergence.They commonly combine a high theory level with a small basis and lower theory levels with larger bases.Source: A. From Pople diagrams to CQML · p. 3
  • CQML generalizes Δ-ML by estimating correction terms efficiently, extending the composite-method idea of combining approximations.The passage describes composite methods as a sophisticated extension of telescoping error-cancellation series.Source: A. From Pople diagrams to CQML · p. 4

B. The CQML approach

CQML combines machine-learning models trained on selected subspaces spanning theory levels, basis sets, and training-set sizes. Its sparse-grid combination strategy assigns few samples to costly high-accuracy subspaces and more to cheaper ones.

B. The CQML approach

  • CQML is a unified kernel ridge regression model that incorporates training data from multiple information spaces and levels.The approach generalizes Δ-ML to theories, basis sets, and training molecules.Source: B. The CQML approach · pp. 4–5
  • A subspace is a specific choice of training information, and its hierarchy spans theory accuracy, basis-set size, and training-molecule count.The paper treats larger randomly drawn training sets as higher levels in chemical space because predictive accuracy generally increases with training-set size.Source: B. The CQML approach · p. 4
  • Sparse-grid combination uses few expensive target-level samples alongside more numerous, cheaper subspaces to construct QML models.The method combines models from different subspaces rather than relying only on the most expensive training data.Source: B. The CQML approach · pp. 4–5
  • Unlike the described existing multi-fidelity models, CQML generalizes across multiple dimensions and combines independently trained kernels rather than one shared kernel matrix.The study restricts its application to theory, basis-set, and training-molecule spaces.Source: B. The CQML approach · p. 5

C. Kernel ridge regression and the ∆-ML approach

The paper reviews kernel ridge regression and Δ-ML before describing CQML’s recursive multi-level extension. In its two-dimensional form, models learn a baseline and successive corrections, with sparse-grid choices guiding training-set sizes across levels.

C. Kernel ridge regression and the ∆-ML approach

  • Kernel ridge regression trains on molecular representations and reference energies to predict an out-of-sample molecule’s property at a specified theory level.The model uses a kernel function and obtains its coefficients by solving a regularized kernel-matrix system.Source: C. Kernel ridge regression and the… · p. 5
  • Δ-ML learns the difference between baseline and target properties, but still requires evaluating the potentially costly baseline for each query molecule.CQML instead models the baseline with machine learning to recover conventional QML prediction speed.Source: C. Kernel ridge regression and the… · p. 5

D. Two-dimensional multi-level learning

  • Two-dimensional CQML generalizes Δ-ML across theory and training-molecule levels, recursively combining models for successive levels.The resulting higher-order models extend beyond the conventional two-level Δ-ML comparison described in the passage.Source: D. Two-dimensional multi-level learning · pp. 5–6
  • CQML can use different training amounts at different levels, while identical training molecules and no regularization make the two-level model equivalent to direct QML.If only a difference model is built with identical training molecules, the passage states that the two-level model and Δ-ML are identical.Source: D. Two-dimensional multi-level learning · p. 6
  • Sparse-grid balancing generally assigns many training samples to lower theory levels and few to higher levels, with relative ratios left as an optimization choice.The study evaluates balancing choices; more systematic tailoring of ratios is identified as future work.Source: D. Two-dimensional multi-level learning · p. 6

E. Three-dimensional multi-level learning

The three-dimensional CQML scheme combines levels of basis set, electron-correlation treatment, and molecular training-set size. Each subspace is identified by an ordered triplet of level indices.

E. Three-dimensional multi-level learning

  • Three dimensions—basis set (B), electron correlation (C), and molecular training set (M)—are represented by corresponding levels ℓB, ℓC, and ℓM.Table I exemplifies the levels used in the three-dimensional scheme.Source: E. Three-dimensional multi-level… · p. 6, Table I · p. 6
  • A subspace is encoded by the ordered triplet ℓ = (ℓC, ℓB, ℓM); CCSD(T)/cc-pvdz with N2 is (2, 0, 1).The corresponding CQML model is denoted E(2,2,2).Source: E. Three-dimensional multi-level… · p. 6
  • For this example, the model expression includes terms indexed by (1,1,0), (0,1,1), (1,0,1), and (0,0,2).The cited expression also contains the E(2,2,2) model; the indices range from 0 to 2 in each dimension.Source: E. Three-dimensional multi-level… · p. 6

F. n-dimensional multi-level learning

The n-dimensional extension combines QML models across a selected set of subspaces using the sparse-grid combination technique. It generalizes the lower-dimensional approach to d approximation spaces.

F. n-dimensional multi-level learning

  • The method generalizes the combination principle to d spaces, collecting their levels in a multi-index ℓ = (ℓ1, . . . , ℓd).For the three-space example, the indices refer to electron correlation, basis set, and molecular training-set size.Source: F. n-dimensional multi-level learning · p. 6
  • The generalized CQML model combines QML models from subspaces in the index set I.The index set is specified by a sparse-grid choice of levels and coefficients.Source: F. n-dimensional multi-level learning · p. 7
  • The summation uses vectors z containing all possible combinations of zeros and ones, and the method defines a characteristic function χI for the index set.Source: F. n-dimensional multi-level learning · p. 7
  • For d = 3, the initial subspace choice omits target-accuracy training data, so the index set is shifted to include those subspaces with at least a small training set.This adds CCSD(T) calculations with a cc-pvdz basis set to the relevant training subspaces.Source: F. n-dimensional multi-level learning · p. 7

IV. RESULTS AND DISCUSSION

The results are assessed through learning curves, which track out-of-sample prediction error as training data grow. For kernel ridge regression, the leading error term follows a power law in training-set size.

IV. RESULTS AND DISCUSSION

  • Kernel ridge regression’s leading prediction-error term is proportional to a/N^b, where a and b are the proportionality constant and power-law exponent.The passage presents learning curves as a way to compare model performance as training-set size changes.Source: §IV · p. 7
  • A single out-of-sample error is insufficient to characterize a model when its prediction error changes with training-set size.Source: §IV · p. 7
  • On log-log scales, power-law error decay appears linear, with log(Error) = log(a) − b log(N).The passage describes smaller offsets and steeper slopes as preferable, while error saturation indicates failure to learn.Source: §IV · p. 7

A. Data

The QM7b analysis examines how molecular size and structure relate to energy differences across basis sets and electron-correlation methods, then evaluates CQML learning curves. Combining training data across correlation and basis-set levels reduces the number of target-level training molecules needed for chemical accuracy.

A. Data

  • Across about 7,000 QM7b molecules, the energy-difference distributions change substantially from HF to MP2 but are barely changed from MP2 to CCSD(T).The molecular set includes HF, MP2, and CCSD(T) energies at sto-3g, 6-31g, and cc-pvdz.Source: A. Data · p. 7, Figure 3 · p. 8
  • S-containing molecules form a distinct cluster, and the authors report that sto-3g and 6-31g are too small to describe them.Among the 200 molecules at each extreme of the CCSD(T) − HF correlation contribution, highly saturated molecules have the largest contributions, while molecules with multiple unsaturated bonds or aromatic groups have the least.Source: A. Data · p. 7, Figure 4 · p. 9

B. 2D results for QM7b

  • The 2D study fixes the basis at 6-31g, varies HF, MP2, and CCSD(T) levels, and compares two training-set size ratios, s = 1 and s = 2.The analysis uses kernel ridge regression with a Laplacian kernel and SLATM representation.Source: B. 2D results for QM7b · p. 8, Table II · p. 8
  • Figure 5 plots CCSD(T) atomization-energy prediction errors against the number of CCSD(T)-energy training molecules, comparing CQML models with level ratios s = 1 and s = 2, including 3D-CQML.Source: Figure 5 · p. 9
  • Conventional QML reaches chemical accuracy with about 4,000 CCSD(T)/6-31g training molecules; adding MP2 data lowers that requirement to about 2,000 and 1,000 for s = 1 and s = 2.The MP2 training-set size is 4,000 molecules for both s values; replacing MP2 with HF gives slightly worse results.Source: B. 2D results for QM7b · pp. 9–10
  • Adding HF training data to the MP2-and-CCSD(T) model reaches chemical accuracy with about 1,000 and 300 CCSD(T) molecules for s = 1 and s = 2.The corresponding MP2 and HF training-set sizes are 2,000 and 4,000 for s = 1, and 1,200 and 2,400 for s = 2.Source: B. 2D results for QM7b · p. 10

C. 3D results for QM7b

The 3D-CQML model combines basis-set and electron-correlation levels, and its learning curves indicate that adding dimensions reduces the high-level training data needed. For CI9, adding levels also improves learning, with the best reported SLATM model reaching chemical accuracy using progressively more reference molecules at lower levels.

C. 3D results for QM7b

  • With CCSD(T) across basis sets, the error approaches chemical accuracy with about 1,000 examples at cc-pvdz, and increasing s from 1 to 2 lowers the learning curve.The basis sets are sto-3g, 6-31g, and cc-pvdz; the authors assume conclusions hold despite their relatively modest size.Source: C. 3D results for QM7b · p. 10
  • 500 and 100 CCSD(T)/cc-pvdz training instances are estimated to reach chemical accuracy for s = 1 and s = 2, respectively, with the combined 3D-CQML model.The curves end at small training sizes because lower-level requirements reach QM7b's 7,211-molecule limit; at s = 2, 100 highest-level instances require 25,600 lowest-level instances.Source: C. 3D results for QM7b · p. 10

D. 2D results for CI9

  • For CI9, the best 2D-CQML SLATM model reaches chemical accuracy with about 1,000 G4MP2, 2,000 B3LYP/6-31g(D), and 4,000 PM7 reference results.Increasing the included levels from two to three improves learning versus conventional QML; SLATM has a substantially lower offset than the Coulomb matrix representation.Source: D. 2D results for CI9 · p. 11

V. CONCLUSIONS

  • The conclusions report that adding levels and spaces systematically reduces the high-level training data required, allowing expensive training molecules to be replaced by multiple cheaper ones.The paper presents this as a possible route to more accurate QML models at constant training-data compute budget, given the scaling and prefactors of expensive quantum approximations.Source: §V · p. 11
  • The study's planned extensions include more intermediate theory levels, continuous basis-set changes, other physical dimensions, properties beyond atomization energies, and locally optimized training-level ratios.The current work focuses on atomization energies and sets the relative amount of training data across levels globally through parameter s.Source: §V · p. 11

Appendix A: Derivation of the combination technique for quantum machine learning

The appendix transfers the sparse-grid combination technique to quantum machine learning in two and three dimensions. It builds the approximation recursively from lower-dimensional spaces, avoiding the full tensor-product construction associated with exponential growth in work as dimension increases.

Appendix A: Derivation of the combination technique for quantum machine learning

  • The sparse-grid combination technique offers a cheaper alternative to full tensor-product approximation, which incurs exponentially growing computational work with dimension.The construction recursively introduces a sparse approximation space.Source: Appendix A · p. 11
  • The appendix derives the combination technique for QML in two and three dimensions, transferring the sparse-grid approach to this setting.It frames subspace models using chemical, basis-set, and training-set dimensions.Source: Appendix A · pp. 11–12
  • The 2D construction combines QML models over theory and training-set-size levels while holding the basis-set level fixed.Negative level indices are assigned a model value of exactly zero.Source: Appendix A · p. 12
  • The 3D construction recursively applies the 2D combination technique to incorporate basis-set size, and the appendix expands the case j3 = 2.The expanded terms are combined to recover the stated E(2,2,2) model.Source: Appendix A · pp. 12–13

SUPPLEMENTARY MATERIAL

The supplementary material describes the available QM7b energy data across three basis sets and three electronic-structure methods.

SUPPLEMENTARY MATERIAL

  • QM7b provides total and effective averaged atomization energies for sto-3g, 6-31g, and cc-pvdz, with HF, MP2, and CCSD(T) values in three-column text files.Effective averaged atomization energy is defined as total energy minus the sum of element counts times fitted effective atomic energies; free-atom energies are also included.Source: SUPPLEMENTARY MATERIAL · p. 13
Loading 1808.02799v2…