Source-linked AI summary
Transferability in Machine Learning for Electronic Structure via the Molecular Orbital Basis
Matthew Welborn, Lixue Cheng, Thomas F. Miller
TL;DR
The paper addresses limited chemical transferability in electronic-structure ML models that rely on atom- or geometry-specific features. It uses localized molecular-orbital features from Hartree–Fock calculations with Gaussian-process regression to predict MP2 and CCSD correlation energies, achieving accurate predictions across molecules and elements beyond the training set. The method uses at most 26 matrix elements and reports relative mean errors no larger than 0.24% in the studied applications.
Problem
Atom- and geometry-specific electronic-structure ML features can require rapidly expanding feature sets and extensive reference data, while prior methods had not demonstrated predictions for molecules containing elements absent from training data.
Method
The method uses Hartree–Fock molecular-orbital features and Gaussian-process regression to predict diagonal and off-diagonal contributions to post-Hartree–Fock correlation energies.
Results
Across the studied applications, relative mean error is at most 0.24% of the CCSD correlation energy and Pearson correlation coefficients are consistently greater than 0.9, except for butane and isobutane.
Takeaways & Limitations
The method demonstrates accurate MP2 and CCSD predictions across different bonding connectivities and chemical elements, using compact feature sets without atom-type, bond-type, geometry, or chemical-environment metadata.
Abstract
from arXiv · showhide
We present a machine learning (ML) method for predicting electronic structure correlation energies using Hartree-Fock input.The total correlation energy is expressed in terms of individual and pair contributions from occupied molecular orbitals, and Gaussian process regression is used to predict these contributions from a feature set that is based on molecular orbital properties, such as Fock, Coulomb, and exchange matrix elements. With the aim of maximizing transferability across chemical systems and compactness of the feature set, we avoid the usual specification of ML features in terms of atom- or geometry-specific information, such atom/element-types, bond-types, or local molecular structure. ML predictions of MP2 and CCSD energies are presented for a range of systems, demonstrating that the method maintains accuracy while providing transferability both within and across chemical families; this includes predictions for molecules with atom-types and elements that are not included in the training set. The method holds promise both in its current form and as a proof-of-principle for the use of ML in the design of generalized density-matrix functionals.
I. INTRODUCTION
Existing electronic-structure ML models often rely on atom- and geometry-specific features, limiting compactness and demonstrated chemical transferability. This work instead predicts post-Hartree–Fock correlation energies from Hartree–Fock molecular-orbital features, including diagonal and pair contributions.
- Atom- and geometry-specific features can require rapidly expanding feature sets and extensive reference data for diverse elements and chemistries.These requirements have hindered demonstrated chemical transferability.
- The method targets post-Hartree–Fock correlation energies while assuming the cost of a Hartree–Fock self-consistent-field calculation.
- Molecular-orbital features avoid explicit dependence on atom types and are intended to provide greater chemical transferability.
- The correlation energy is represented as a sum over occupied molecular orbitals, with ML describing diagonal and off-diagonal contributions.
- Using localized molecular orbitals yields a model designed to be compact in feature count and transferable across chemical systems.
II. FEATURE DESIGN AND SELECTION
The feature design represents Hartree–Fock matrix information in a localized molecular-orbital basis and organizes occupied-pair features to preserve symmetry and prioritize chemically relevant virtual orbitals. The resulting vectors combine Fock, Coulomb, and exchange elements while limiting the virtual-orbital information included.
- Features are selected from Fock, Coulomb, and exchange matrices represented in the localized molecular-orbital basis.Core occupied orbitals are excluded under the frozen-core approximation, while valence virtual orbitals are defined by projection onto a minimal basis.
- Pair feature vectors combine occupied-occupied, occupied-virtual, and virtual-virtual matrix blocks.
- Sorting occupied-pair terms by F_ii < F_jj guarantees that the predicted pair contribution satisfies ϵ_ij = ϵ_ji.
- Valence virtual orbitals are selected using the largest off-diagonal Coulomb elements, with separate selections for the relevant occupied orbitals.
- Virtual-virtual elements are sorted by combined Coulomb contributions and retained from the upper diagonal because they may inform MP2 and CCSD pair energies.
- Virtual-orbital sorting supports transferability by prioritizing spatially close orbitals, which are likely to be important for near-sighted dynamical correlation.
- The localized-orbital features are invariant to system rotation and translation and smooth with respect to molecular geometry, subject to the localization method.
- For diagonal contributions, the feature vector retains only unique terms from the corresponding same-orbital matrix elements.
III. CALCULATION DETAILS
Calculations use Hartree–Fock, MP2, and CCSD references with localized valence orbitals and Gaussian-process regression for the diagonal and off-diagonal energy contributions. Training geometries are sampled from finite-temperature molecular-dynamics trajectories, with feature reduction used to limit overfitting.
- Hartree–Fock, MP2, and CCSD calculations use Molpro 2018.0, generally with the cc-pVTZ basis and frozen-core correlated calculations.
- Intrinsic Bond Orbitals generate localized valence occupied and virtual orbitals using a 10^-12 localization threshold, while core orbitals remain excluded.
- Gaussian-process regression separately predicts diagonal and off-diagonal contributions using a Matérn 5/2 kernel with white-noise regularization.
- Training and test geometries come from ab initio molecular-dynamics trajectories at 350 K, sampled every 50 fs.
- Feature reduction prioritizes terms involving two occupied orbitals, then occupied–virtual terms, then virtual–virtual terms to reduce overfitting.
- Feature-set variants are summarized by the numbers of diagonal and off-diagonal features.
A. Transferability among geometries
For water and other small molecules, localized-orbital ML predictions reproduce MP2 and CCSD correlation contributions and total energies across sampled geometries. The reported errors are generally within chemical accuracy, with relative mean errors at or below 0.1% for the small-molecule cases summarized in Table II.
- 200 training geometries support predictions for 1000 water geometries at both MP2 and CCSD levels, including diagonal, off-diagonal, and total correlation energies.Figure 1 reports mean, maximum, relative mean errors, and Pearson correlation coefficients in mH.
- Water’s diagonal and off-diagonal contributions show excellent agreement with reference values, and total correlation energies are within chemical accuracy.
- For small molecules ranging from H2 to benzene, the correlation-energy Mean Error is generally much less than 1 mH and the Max Error remains in the chemical-accuracy range.
- The Rel. Mean Error is 0.1% or less for every small-molecule case summarized in Table II.
- Increasing training geometries improves accuracy until performance becomes limited by the GPR self-training error.
- Water predictions are not sensitive to the employed basis set across double-zeta through quintuple-zeta calculations.
B. Transferability within a molecular family
The MO-based ML model transfers within molecular families, predicting larger water clusters and longer or branched alkanes from smaller training molecules. Transfer improves when training data better represent the chemical environments in the targets, while water-cluster errors approach the GPR baseline.
- Water clusters: Water tetramer, pentamer, and hexamer predictions achieved Rel. Mean Errors of 0.060.07% and Pearson coefficients above 0.95 after training on monomers and dimers.The model used 200 monomer and 300 dimer geometries with a smaller feature set to avoid overfitting.
- Water clusters: Water-cluster prediction errors were similar to GPR self-training errors, indicating that regression self-training error, rather than limited transferability, dominated the observed error.The authors suggest more sophisticated regression methods could reduce this baseline.
- Alkanes: Training on methane and ethane yielded butane and isobutane Mean Errors of 1.2 and 1.4 mH, with reduced correlation coefficients of -0.05 and -0.31.These errors exceeded the corresponding GPR baselines, suggesting that additional training data would improve predictions.
- Alkanes: Adding 50 propane geometries improved butane and isobutane predictions, raising correlation coefficients to 0.77 and 0.32 versus a GPR baseline of 0.79.Butane errors reached the GPR baseline, whereas isobutane remained above it because tertiary-carbon examples were still absent.
- Alkanes: The model predicted molecules containing carbon atom-types absent from training, while adding chemically representative propane data produced systematic improvement.Propane supplied secondary-carbon information that particularly benefited butane; tertiary-carbon information remained unavailable for isobutane.
C. Transferability across molecules and elements
The model transfers across chemically distinct molecules and, in an extreme test, across elements absent from training. It retains comparable accuracy for methanol and accurately predicts ammonia, methane, and hydrogen fluoride from water-only training.
- Across molecules: A model trained on water, methane, and formic acid described those three chemically distinct molecules simultaneously with errors similar to separately trained models.This indicates sufficient flexibility for the represented molecular set.
- Across molecules: Methanol, absent from training, had Mean and Max Errors comparable to training molecules and only about twice the errors from training methanol on itself.Pearson correlation coefficients were high in all cases.
- Across elements: Training exclusively on water produced NH3, CH4, and HF Mean Errors within 0.5 mH, Rel. Mean Errors below 0.24%, and Pearson coefficients above 0.94.The results support transfer of information about lone pairs and sigma bonds across molecules composed of different elements.
V. DISCUSSION AND CONCLUSIONS
The method predicts correlated electronic-structure energies using compact, localized-molecular-orbital features rather than atom- or element-specific information. Its transferability is demonstrated across bonding connectivities and chemical elements, while future work targets baseline errors, lower-cost inputs, and gradients.
- The ML method predicts correlated electronic-structure energies from molecular-orbital features rather than atom-type or element-specific features.The approach is designed to provide compact features and transferability across chemical systems.
- Accurate MP2 and CCSD predictions transfer across molecules with different bonding connectivities and chemical elements from the training set.Across applications, relative mean error is at most 0.24% of the CCSD correlation energy, with Pearson coefficients generally above 0.9.
- At most 26 Hartree-Fock matrix elements are needed to predict each occupied-valence-orbital-pair contribution without atom, bond, geometry, or chemical-environment metadata.The calculations use between 11 and 26 unique features.
- Future work includes reducing baseline self-training errors, using smaller-basis or lower-cost SCF inputs, and implementing a gradient theory.The proposed gradient implementation would employ coupled perturbed SCF and localization.
SUPPORTING INFORMATION
The supporting information expands small-molecule predictions, provides MP2 results corresponding to the main-text CCSD results, and includes additional error plots.
- The supporting information contains expanded small-molecule predictions corresponding to Table II.
- It provides MP2 results corresponding to all CCSD results presented in the main text.
- It includes plots of ML prediction error versus total CCSD energy corresponding to Figures 2–5.