PaperlayerView exact revision on arXiv

Source-linked AI summary

OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features

Zhuoran Qiao, Matthew Welborn, Animashree Anandkumar, Frederick R. Manby, Thomas F. Miller

arXiv:2007.08026v3physics.chem-phcs.LG

TL;DR

The paper addresses the cost of predicting molecular potential-energy surfaces at DFT-level accuracy. It introduces ORBNET, which combines symmetry-adapted atomic-orbital features from low-cost electronic-structure calculations with graph neural networks. Across diverse molecular benchmarks, ORBNET achieves DFT-comparable accuracy while substantially reducing computational cost.

  • Problem

    DFT-level molecular energy calculations are accurate but costly, limiting large-scale molecular applications and conformational sampling.

  • Method

    ORBNET uses symmetry-adapted atomic-orbital features from low-cost mean-field calculations with a graph neural-network architecture to predict quantum-mechanical energies.

  • Results

    ORBNET predicts drug-like molecular and conformer energies with accuracy comparable to DFT while reducing computational cost by at least three orders of magnitude relative to DFT.

  • Takeaways & Limitations

    ORBNET provides transferable, training-efficient prediction of DFT-quality energies across diverse molecular systems using low-cost input features.

Abstract

from arXiv · show

We introduce a machine learning method in which energy solutions from the Schrodinger equation are predicted using symmetry adapted atomic orbitals features and a graph neural-network architecture. \textsc{OrbNet} is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that are obtained from semi-empirical electronic structure calculations. For applications to datasets of drug-like molecules, including QM7b-T, QM9, GDB-13-T, DrugBank, and the conformer benchmark dataset of Folmsbee and Hutchison, \textsc{OrbNet} predicts energies within chemical accuracy of DFT at a computational cost that is thousand-fold or more reduced.

I. INTRODUCTION

ORBNET targets efficient and transferable prediction of molecular quantum-mechanical energies by combining symmetry-adapted atomic-orbital features with graph neural networks. It uses low-cost electronic-structure features and an extensive energy-decoding scheme to reduce the computational burden of DFT-level predictions.

  • DFT accurately describes many molecular and materials energies but remains costlier than force-field and semi-empirical methods, motivating more efficient quantum-chemistry prediction.
  • ORBNET represents molecules with symmetry-adapted atomic orbitals and applies graph neural-network methods to learn quantum-mechanical properties.
  • The method targets transferable prediction of total and relative conformer energies across organic and drug-like molecular datasets.
  • Its features include SAAO-basis quantum-operator tensors, including Fock, Coulomb, exchange, and density-matrix quantities.
  • The workflow builds an attributed graph from diagonal and off-diagonal SAAO tensor elements, processes it with message passing, and decodes node contributions into an extensive energy.
  • ORBNET constructs localized, rotation-consistent SAAOs by diagonalizing small density-matrix blocks rather than using iterative orbital-localization procedures.

B. Approximated Coulomb and exchange SAAO features

ORBNET approximates Coulomb and exchange features for semi-empirical calculations using damped interaction kernels, avoiding the full cost of four-index electron-repulsion integrals. For the studied molecules, the resulting feature-generation bottleneck is not dominant.

  • Semi-empirical SAAO feature generation is bottlenecked by Coulomb and exchange terms because they require four-index electron-repulsion integrals.
  • The method introduces a generalized Mataga–Nishimoto–Ohno–Klopman formula, as used in sTDA-xTB, to approximate these interactions.
  • The damped interaction kernels depend on interatomic distance, average chemical hardness, and empirical decay parameters for Coulomb and exchange terms.
  • The approximated Coulomb and exchange matrices are constructed from transition densities obtained through Löwdin population analysis.
  • 31: Naive evaluation scales as O(N4), while a tight-binding approximation can reduce scaling to O(N2) with negligible accuracy loss.

C. OrbNet

ORBNET represents molecules as graphs whose node and edge attributes are symmetry-adapted atomic-orbital features, then uses attention-based message passing and extensive energy decoding. Its representation and architecture incorporate operator interactions, orbital-pair weighting, and size-consistent energy aggregation.

  • Graph representation: ORBNET encodes molecular systems as attributed graphs with nodes, edges, node attributes, and edge attributes.Node and edge attributes represent orbital-centered information and pairwise interactions, respectively.
  • SAAO features: Node attributes contain diagonal Fock, Coulomb, exchange, density, and core-Hamiltonian features, while edge attributes contain off-diagonal operator features.Edge features additionally include distance and overlap information.
  • Feature embedding: Nonlinear radial-basis embeddings transform normalized node and edge features before graph message passing.Sine basis functions are used for node embedding, while spherical Bessel functions are used for edge embedding.
  • Message passing: Each message-passing layer combines neighboring node and edge attributes into messages and multi-head attention scores, then updates node features through attention mixing and a dense layer.The resulting update is added to the original node attributes, while edge attributes are updated with trainable transformations and normalized activation functions.
  • Energy decoding: ORBNET predicts extensive energies by decoding node-resolved contributions from the embedding layer and every message-passing layer, summing over layers and then over orbitals.An edge cutoff encodes infinitely separated noninteracting systems as disconnected graphs, satisfying size-consistency.

D. Comparison with other methods that use quantum mechanical features

ORBNET differs from NeuralXC and DeePHF in both its quantum-mechanical feature representation and its graph-based regression architecture. Its SAAO features preserve rotational invariance and information content while representing broad off-diagonal interactions.

  • Feature generation: ORBNET uses SAAO-based features, whereas NeuralXC and DeePHF use atomic-orbital features derived from larger-basis mean-field calculations.NeuralXC uses low-level DFT features, while DeePHF uses Hartree–Fock features.
  • Feature information: ORBNET avoids shell-averaging and includes all off-diagonal operator matrix elements, preserving information and enabling long-range contributions.The included elements span intra- and inter-atom and intra- and inter-shell interactions.
  • Feature information: ORBNET includes interactions between different shells on the same atom, unlike DeePHF.The comparison distinguishes ORBNET’s richer representation from DeePHF’s shell-based treatment.
  • Rotational invariance: SAAOs provide rotationally invariant features without the information loss associated with using the trace or eigenvalues of local density matrices.This contrasts with the rotational-invariance strategies described for NeuralXC and DeePHF.
  • Regression architecture: ORBNET uses a multi-head graph-attention model with residual blocks rather than the dense neural networks used by NeuralXC and DeePHF.The architecture is presented as a flexible framework for learning orbital interactions and downstream transfer.

III. COMPUTATIONAL DETAILS

The study evaluates ORBNET across molecular and drug-like datasets using specified dataset splits, feature transformations, regularized losses, and common training settings. Conformer-focused training adds a relative-energy penalty to emphasize configurational energy variations.

  • Datasets: The evaluation covers QM7b-T, QM9, GDB-13-T, DrugBank-T, and related molecular datasets with varying molecule sizes and conformational sampling.DrugBank-T geometries are sampled from ab initio molecular-dynamics trajectories at 350 K.
  • Dataset splits: QM9 uses 110000 training molecules and 10831 test molecules after removing 3054 molecules that failed a geometric consistency check.Smaller QM9 training sets are subsampled from the 110000-molecule training pool.
  • Feature preprocessing: Input features are normalized separately for diagonal and off-diagonal SAAO operator values before model training.Diagonal values are mapped to [0,1), while most off-diagonal values use −ln(|Xuv|).
  • Model settings: The same hyperparameter set is used throughout the work to provide a universal model.Cutoffs are selected by examining feature-distribution overlap between QM7b-T and GDB13-T.
  • Training objective: For conformer training, a paired relative conformer loss penalizes errors in predicted energy differences between conformers of the same molecule.The paired conformer is randomly sampled within each minibatch.
  • Optimization: All models are trained with Adam using minibatches of 64 and a cyclical learning-rate schedule on one Nvidia Tesla V100-SXM2-32GB GPU.The schedule increases, decreases, and then exponentially decays the learning rate across successive training phases.

IV. RESULTS

ORBNET uses low-cost GFN1-xTB electronic-structure features to learn DFT-quality total energies. This delta-learning setup predicts total DFT energies from a GFN1-xTB calculation.

  • ORBNET learns DFT-energy differences from GFN1-xTB features, addressing the limited accuracy of semi-empirical methods.The approach targets the difference between high-level DFT and GFN1 energies.
  • The model includes a correction based on isolated-atom DFT–GFN1 energy differences determined by a linear model.
  • The resulting approach directly predicts total DFT energies from the results of a GFN1-xTB calculation.

A. The QM9 dataset

On QM9 total-energy prediction, ORBNET is evaluated against graph-based neural-network methods, with and without ensemble averaging. It achieves greater accuracy and learning efficiency than previous deep-learning methods without additional transferability strategies or ensembling.

  • QM9 comparisons evaluate total energies for organic molecules with up to nine heavy atoms at locally optimized geometries.
  • Ensembling five independently trained ORBNET models reduces prediction error by approximately 10–20%.
  • ORBNET is compared with SchNet, PhysNet, DimeNet, and DeepMoleNet, which use graph representations of atom-based features.
  • Even without model ensembling or additional physical-information strategies, ORBNET provides greater accuracy and learning efficiency than previous deep-learning methods.

B. Transferability and Conformer Energy Predictions

The transferability experiments test ORBNET across organic and drug-like molecular datasets, including larger molecules and conformer benchmarks. Models improve with more data and ensembling, while achieving DFT-comparable conformer-energy accuracy at substantially lower cost.

  • B. Transferability and Conformer Energy Predictions: ORBNET is tested by training on smaller-molecule datasets and evaluating transferability to larger and more diverse molecules.
  • B. Transferability and Conformer Energy Predictions: Median and mean absolute errors generally decrease from Model 1 to Model 4 as training data and ensemble modeling increase.DrugBank-T MAE is a stated exception to monotonic improvement.
  • B. Transferability and Conformer Energy Predictions: Model 1, trained only on QM7b-T, predicts DrugBank-T and Hutchison relative conformer energies comparably to more heavily trained models.These datasets include molecules with up to 50 heavy atoms.
  • B. Transferability and Conformer Energy Predictions: All ORBNET models predict relative conformer energies within the 1 kcal/mol threshold of chemical accuracy across four test datasets.
  • B. Transferability and Conformer Energy Predictions: Model 4 achieves a median R2 of 0.81 against DLPNO-CCSD(T) reference data and 0.90 against the ωB97X-D/Def2-TZVP training reference.
  • B. Transferability and Conformer Energy Predictions: ORBNET predicts drug-like-molecule relative conformer energies with DFT-comparable accuracy at a computational cost 1000-fold lower than DFT.

V. CONCLUSIONS

ORBNET uses low-cost electronic-structure features with a graph neural network to predict high-quality energies while targeting the accuracy–cost trade-off in quantum chemistry. It improves prediction accuracy over existing methods and achieves DFT-like conformer-energy accuracy at substantially lower computational cost.

  • ORBNET uses low-cost mean-field electronic-structure features and a graph neural network to predict high-quality electronic-structure energies.The demonstrated mapping uses GFN1-xTB features to predict ωB97X-D/Def2-TZVP energies.
  • 33% improvement in QM9 total-energy prediction accuracy is reported relative to DeepMoleNet using the same amount of data.
  • The method is presented as general with respect to both the high-level reference method and the mean-field method used to generate features.
  • ORBNET achieves accuracy similar to DFT for relative conformer energies while reducing computational cost by at least three orders of magnitude relative to DFT.The comparison uses a realistic and diverse drug-like conformer dataset compiled by Folmsbee and Hutchison.

VI. SUPPLEMENTAL MATERIAL

The supplemental material provides DrugBank-T structures and identifies molecules shared across training-test splits. It also documents an accuracy–computational-cost comparison on the Hutchison conformer benchmark using ORBNET Model 4.

  • The supplement includes structures for the DrugBank-T dataset.
  • The supplement specifies molecules that appear in all training-test splits for the trained models.

DATA AVAILABILITY STATEMENT

The study states that data supporting its findings are available within the article and supplementary material. The document also includes its cited research literature.

  • Data supporting the study's findings are available within the article and its supplementary material.
  • The availability statement covers the data supporting the reported findings.
  • The document includes a bibliography of prior quantum-chemistry and machine-learning research.
Loading 2007.08026v3…