PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantum-Chemical Insights from Deep Tensor Neural Networks

Kristof T. Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R. Müller, Alexandre Tkatchenko

arXiv:1609.08259v4physics.chem-ph

TL;DR

The paper addresses whether machine learning can reveal quantum-chemical structure beyond properties directly included in training data. It develops a deep tensor neural network based on many-body Hamiltonian concepts and learned atom-centered representations. The model achieves uniformly accurate 1 kcal/mol predictions across compositional and configurational chemical space for intermediate-sized molecules and supports analyses such as aromatic-ring stability and local chemical potentials.

  • Problem

    The paper asks whether machine learning can enable breakthroughs in understanding quantum many-body systems and provide insights beyond directly trained molecular properties.

  • Method

    The authors develop DTNNs that learn atom-centered chemical environments from molecular charges and distances while using recursive pairwise refinements and symmetry-preserving energy decomposition.

  • Results

    DTNN achieves uniformly accurate 1 kcal/mol predictions across compositional and configurational chemical space for molecules of intermediate size.

  • Takeaways & Limitations

    DTNN reveals aromatic-ring stability classifications, local chemical potentials, reliable isomer energies, and other quantum-chemical insights not explicitly contained in the training targets.

  • Takeaways & Limitations

    Extending DTNN predictive power to larger molecules requires a reliable model for long-range interatomic van der Waals interactions.

Abstract

from arXiv · show

Learning from data has led to paradigm shifts in a multitude of disciplines, including web, text, and image search, speech recognition, as well as bioinformatics. Can machine learning enable similar breakthroughs in understanding quantum many-body systems? Here we develop an efficient deep learning approach that enables spatially and chemically resolved insights into quantum-mechanical observables of molecular systems. We unify concepts from many-body Hamiltonians with purpose-designed deep tensor neural networks (DTNN), which leads to size-extensive and uniformly accurate (1 kcal/mol) predictions in compositional and configurational chemical space for molecules of intermediate size. As an example of chemical relevance, the DTNN model reveals a classification of aromatic rings with respect to their stability -- a useful property that is not contained as such in the training dataset. Further applications of DTNN for predicting atomic energies and local chemical potentials in molecules, reliable isomer energies, and molecules with peculiar electronic structure demonstrate the high potential of machine learning for revealing novel insights into complex quantum-chemical systems.

MOLECULAR DEEP TENSOR NEURAL NETWORKS

DTNN represents molecules through atom types and interatomic distances, then recursively refines atom-centered features through pairwise interactions. Atomic contributions are summed to predict total energy while preserving molecular symmetries and extensivity.

  • Molecular representation: DTNN represents molecular structures using nuclear charges and interatomic distances, with distances expanded in a Gaussian basis.This construction ensures rotational and translational invariance and captures interactions across different distance regimes.
  • Energy decomposition: The molecular energy is expressed as a sum of atomic energy contributions, preserving permutational invariance and scaling with molecular size.Each atom begins with an atom-type-specific coefficient vector that is repeatedly refined by pairwise interactions.
  • Interaction refinement: Each refinement correlates atomic neighborhoods with increasing many-body complexity through walks of increasing length in the molecular graph.The model recursively embeds atoms into their chemical environments by reducing overlaps between basis functions.
  • Tensor interactions: A tensor layer nonlinearly couples atomic features with interatomic distances, while low-rank factorization reduces parameters and overfitting risk.The factorized representation combines transformed atom features and distance features through element-wise multiplication.
  • Energy prediction: Two fully connected layers convert the final atomic coefficient vectors into energy contributions whose sum gives the molecular energy.The architecture is trained by stochastic gradient descent from randomly initialized weights, biases, and atom descriptors.

LEARNING MOLECULAR ENERGIES

DTNN models were evaluated across compositional molecular structures and configurational molecular-dynamics trajectories. The results show accurate energies across chemical space and support applications to chemical potentials, stability classification, and isomer energies.

  • Compositional space: DTNN models trained on GDB datasets achieve a mean absolute error of 1.0 kcal/mol.The reported training sets contain 5.8k GDB-7 and 25k GDB-9 reference calculations.
  • Configurational space: DTNN reproduces molecular-dynamics energies with mean absolute errors of 0.05, 0.18, 0.17, and 0.39 kcal/mol for benzene, toluene, malonaldehyde, and salicylic acid, respectively.The toluene trajectory and energy distribution agree closely with the corresponding DFT results.
  • Generalization: Accurate equilibrium-structure and molecular-dynamics predictions demonstrate feasibility across compositional and configurational degrees of freedom.The architecture is universal, although learned coefficients differ between GDB-9 and single-molecule MD trajectories.

APPLICATIONS

DTNN extracts spatially and chemically resolved information from molecular energy models, supporting stability classification and energy prediction across complex chemical spaces.

  • Quantum-chemical insights: DTNN-derived local chemical potentials reveal bond saturation, aromaticity differences, and atom-type-specific chemical-potential distributions despite training only on total molecular energies.The model uses a one-way probe-atom setup to assess the environment-dependent energy of a specified atom type.
  • Quantum-chemical insights: C6O3H6 is predicted to have the most stable aromatic carbon ring among GDB-9 molecules, consistent with its higher ring stability than benzene and toluene.The classification is based on DTNN atomic-energy distributions and a full stability ranking.
  • Quantum-chemical insights: DTNN classifies molecules by the stability of structural building blocks, including aromatic rings and methyl groups, using learned atomic energies.The resulting partitioning is learned globally from a large molecular dataset, although atomic energy partitioning is intrinsically nonunique.
  • Quantum-chemical insights: Analytical gradients of local chemical potentials could aid exploration of chemical compound space.The passage identifies gradients with respect to chemical composition or the local chemical potential as a possible tool.
  • Energy predictions for isomers: DTNN size extensivity and quantitative accuracy enable configurational and conformational energy differences to be mapped for molecular isomers.The demonstration used 6095 C7O2H10 isomers from the GDB-9 dataset.
  • Energy predictions for isomers: 1.7 kcal/mol mean absolute error demonstrates proof-of-principle prediction across equilibrium structures and molecular-dynamics trajectories for 113 C7O2H10 isomers.The model was trained on equilibrium isomers and half of each trajectory, with remaining molecular-dynamics calculations used for validation and testing.

DISCUSSION

DTNN learns transferable atom-centered chemical representations recursively from pairwise distances while guaranteeing key molecular invariances. Its current predictive scope is bounded by the need for long-range interaction modeling and uncertainty around unusual electronic structures.

  • DISCUSSION: DTNN learns an implicit atom-centered basis from ab initio reference calculations and embeds atoms in chemical environments through recursive refinement of pairwise distances.The architecture guarantees translation, rotation, and permutation invariance.
  • DISCUSSION: Unlike many-body neural-network expansions requiring separate networks for each non-equivalent term, DTNN uses a common learned basis to obtain higher-order interactions more efficiently.The shared basis is the central distinction identified relative to the cited many-body approaches.
  • DISCUSSION: DTNN shares size-extensive atomic-energy prediction with SOAP and symmetry-function methods but learns its basis rather than using a fixed set of environmental basis functions.The comparison concerns representation flexibility, while all cited approaches achieve size extensivity through atomic contributions.
  • DISCUSSION: Larger molecules require DTNN to be combined with a reliable model of long-range van der Waals interactions.This is identified as necessary future work for extending predictive power beyond the demonstrated molecular-size regime.
  • DISCUSSION: Large DTNN errors occur mainly for molecules with unconventional bonding and potentially multireference electronic structure, possibly because such cases are underrepresented in training.Higher-level quantum-chemical calculations are required to test whether smooth interpolation may nevertheless improve some predictions.

OUTLOOK

The DTNN provides a scalable framework for extracting quantum-chemical insights, while several extensions remain open for larger molecules and additional properties.

  • OUTLOOK: The DTNN achieves uniform 1 kcal/mol accuracy across compositional and configurational space for molecules of intermediate size.
  • OUTLOOK: The framework reveals aromatic-ring stability, local molecular chemical potentials, isomer energies, and molecules with peculiar electronic structure.
  • OUTLOOK: Future work includes extending the model to larger molecules, predicting atomic forces and frequencies, and handling non-extensive electronic and optical properties.
  • OUTLOOK: The model represents atoms by type-specific vectors and refines them through pairwise interaction passes based on inter-atomic distances.
  • OUTLOOK: The reported training procedure uses 3000 epochs, squared-error minimization, stochastic gradient descent with 0.9 momentum, and early stopping by validation error.

Computational cost of training and prediction

DTNN training cost increases with reference calculations, interaction passes, and molecular size, whereas prediction is rapid but scales quadratically with atom count without a cutoff.

  • Computational cost of training and prediction: Training times range from 6 hours for 5.768 GDB-7 reference calculations with one interaction pass to 162 hours for 100.000 GDB-9 calculations with three passes.
  • Computational cost of training and prediction: Predictions take less than 1 ms for employed datasets and less than 5 ms for a 100-atom molecule with three interaction layers.
  • Computational cost of training and prediction: Prediction and training scale linearly with interaction passes and quadratically with atom count because interactions require pairwise atomic distances.A distance cutoff could make the DTNN scale linearly with the number of atoms for large molecules.

Computing the local potentials of the DTNN

The DTNN can extract atom-specific representations and use a one-way probe-atom calculation to visualize learned molecular interaction potentials.

  • Computing the local potentials of the DTNN: The trained network provides coefficient vectors for each atom and interaction pass, from which each atom’s energy contribution can be obtained.
  • Computing the local potentials of the DTNN: A probe atom is represented by its charge and distances to the molecule’s atoms, while the molecule remains unaffected by the probe.
  • Computing the local potentials of the DTNN: The probe atom’s predicted energy generates spatial energy surfaces that visualize the interactions learned by the neural network.

Computing an alchemical path with the DTNN

The DTNN constructs alchemical paths by continuously changing atom positions and interpolating atom-type representations, while fading factors allow atoms to be added or removed.

  • Computing an alchemical path with the DTNN: Alchemical paths gradually move atoms and interpolate their initial coefficient vectors when atom types change.
  • Computing an alchemical path with the DTNN: Fading factors β1, . . . , βn ∈[0, 1] control the influence and energy contributions of individual atoms when atoms are added or removed.

Supplementary Text

DTNN models reach chemical accuracy across the evaluated datasets with two or three interaction passes. Increasing training data sharply improves performance initially, while multiple interaction passes improve predictions for previously unseen molecules and configurations.

  • Chemical accuracy is achieved on all employed datasets using models with two or three interaction passes.Evaluation reports mean absolute and root mean squared errors with standard errors over five randomly drawn training sets.
  • From 1,000 to 10,000 training examples, prediction error decreases drastically, whereas expanding to 100,000 yields comparatively small improvement.
  • 95% of unknown GDB-9 molecules have errors of 3.0 kcal/mol or lower when models with two or three interaction passes use at least 25,000 reference calculations.
  • 95% of unknown benzene configurations have maximum errors below 1.3 kcal/mol for models trained on at least 25,000 molecular-dynamics reference calculations.
  • Beyond a certain training-set size, one-interaction-pass models perform significantly worse, indicating that multiple passes enrich representations of unseen structures.

Relations to other deep neural networks

The paper relates DTNN interactions to continuous convolutions while using learned local potentials and atom-centered energy contributions to analyze molecular systems. Its figures examine learning behavior, atomic energies, aromatic-ring stability, alchemical paths, and prediction cost.

  • Relations to other deep neural networks: A factorized tensor layer couples atom representations with interatomic distances, providing efficient computation and regularization through a low-rank tensor-product approximation.
  • Relations to other deep neural networks: DTNN atom interactions are a nonlinear generalization of convolution adapted to molecules, whose atoms are not arranged on a grid.
  • Relations to other deep neural networks: The interaction equations become equivalent to a discrete convolution when the transformation h is the identity.
  • Relations to other deep neural networks: Surface plots are obtained by interpreting the trained deep network as a local potential ΩM.
  • Relations to other deep neural networks: The figures assess errors versus training-set size and interaction passes, atomic energy-contribution distributions, aromatic-ring ordering, alchemical transformations, and prediction time versus molecule size and passes.

Tables

The tables organize the network configurations, training duration, and prediction errors across datasets and training-set sizes. They report models with up to three interaction passes and include MAE, RMSE, standard errors, and maximum errors.

  • Tables: The table header compares models with one, two, and three interaction passes across datasets and training-example counts.
  • Tables: Training duration is reported for networks with T = 1, 2, and 3 after 3,000 stochastic-gradient-descent epochs on an NVIDIA Tesla K40 GPU.
  • Tables: The error table reports MAE, RMSE, their standard errors, and maximum error for multiple datasets and reference-calculation counts.
Loading 1609.08259v4…