By-passing the Kohn-Sham equations with machine learning

Felix Brockherde; Leslie Vogt; Li Li; Mark E. Tuckerman; Kieron Burke; Klaus-Robert Müller

Source-linked AI summary

By-passing the Kohn-Sham equations with machine learning

Felix Brockherde, Leslie Vogt, Li Li, Mark E. Tuckerman, Kieron Burke, Klaus-Robert Müller

arXiv:1609.02815v3Sep 9, 2016physics.comp-phcs.LGphysics.chem-phstat.ML

TL;DR

Machine-learning DFT functionals have been hindered by the difficulty of learning their derivatives. This paper directly learns the Hohenberg-Kohn density-potential map, improving accuracy and reducing computational cost for molecular energy calculations.

Problem
Machine-learning DFT functionals remain limited by unusable derivatives, while solving the Kohn-Sham equations is a computational bottleneck in electronic-structure calculations.
Method
The study directly learns the Hohenberg-Kohn map from one-body potentials to interacting ground-state densities for molecular calculations.
Results
The ML-HK map increases accuracy consistently across 1-D examples and 3-D molecules and outperforms energy models trained on more data.
Takeaways & Limitations
The first 3-D demonstration with real molecules and production-level codes supports machine-learned functionals as a potential lower-cost approach to routine DFT calculations.
Takeaways & Limitations
Training separate model weights for each density-grid point becomes intractable in three dimensions because grid-point counts grow cubically.

Abstract

from arXiv · show

Last year, at least 30,000 scientific papers used the Kohn-Sham scheme of density functional theory to solve electronic structure problems in a wide variety of scientific fields, ranging from materials science to biochemistry to astrophysics. Machine learning holds the promise of learning the kinetic energy functional via examples, by-passing the need to solve the Kohn-Sham equations. This should yield substantial savings in computer time, allowing either larger systems or longer time-scales to be tackled, but attempts to machine-learn this functional have been limited by the need to find its derivative. The present work overcomes this difficulty by directly learning the density-potential and energy-density maps for test systems and various molecules. Both improved accuracy and lower computational cost with this method are demonstrated by reproducing DFT energies for a range of molecular geometries generated during molecular dynamics simulations. Moreover, the methodology could be applied directly to quantum chemical calculations, allowing construction of density functionals of quantum-chemical accuracy.

INTRODUCTION

Kohn-Sham density functional theory is widely used, but machine-learning density functionals has been hindered by noisy kinetic-energy functional derivatives. The paper instead learns the Hohenberg-Kohn potential-to-density map, offering a more versatile route to electronic-structure calculations and functional development.

Background: Kohn-Sham density functional theory is widely used across fields including materials science, with standard exchange-correlation approximations providing useful accuracy.Applications include the materials genome initiative, particularly for weakly correlated materials.
Motivation: Most machine-learning applications predict molecular or materials properties from KS-DFT databases, while fewer learn DFT functionals to avoid solving the KS equations.Some applications also learn potential energy surfaces during molecular-dynamics simulations.
Problem: The key challenge is obtaining a usable functional derivative of the non-interacting kinetic-energy functional for the self-consistent KS cycle.The same kinetic-energy functional also contributes to the ground-state energy through an orbital-free mapping.
Contribution: The proposed ML approach directly learns the Hohenberg-Kohn map v(r) →n(r) from the one-body potential to the interacting ground-state density.This replaces solving the Euler equation with an ML-HK map.
Novelty: The ML-HK map is not restricted to a specific electronic-structure method and can support multiple density-dependent molecular properties beyond energy.The authors also report that learned densities can distinguish different DFT functionals, enabling future functional development.

RESULTS

The results present theoretical findings centered on the ML-HK map and illustrate the approach through simulations of one-dimensional systems and three-dimensional molecules.

The results first outline theoretical findings, prominently including the ML-HK map.
The approach is illustrated with simulations of one-dimensional systems and three-dimensional molecules.

ML-Hohenberg-Kohn map

The ML-Hohenberg-Kohn map directly predicts density from potential, avoiding difficult kinetic-energy gradients and their associated accuracy loss. A basis-function representation makes the approach tractable in three dimensions and supports regularization and independently solvable regressions.

Motivation and formulation: Kinetic-energy gradients are challenging because data-driven models lack information outside the data manifold, making gradients unusable without processing and reducing accuracy relative to T_s[n].The gradient problem arises when minimizing total energy by gradient descent.
Motivation and formulation: The ML-HK map directly trains a multivariate model to predict density as a functional of the potential, circumventing the kinetic-energy gradient.The approach is motivated by accurate semiclassical approximations to the Hohenberg-Kohn density-potential map that require no differential-equation solution.
Basis representation: 3-D training becomes tractable by predicting density basis-function coefficients instead of associating separate model weights with every grid point.Grid-point optimization grows cubically with the number of grid points, whereas basis representations avoid this intractability.
Basis representation: Basis coefficients enable density smoothing by removing high-frequency basis functions and allow regularization of model complexity for selected basis functions.The basis representation also supports extending the model to three dimensions.
Solution and regularization: For orthogonal bases, independent regression models admit an analytical solution analogous to Kernel Ridge Regression, with λ^(l) and σ^(l) selected independently by cross-validation.The ML-HK model avoids gradient descent and gradient de-noising; independence across l gives favorable scaling.

Functional and Density driven error

The paper evaluates ML-HK accuracy by separating total-energy error into functional-driven and density-driven components. Typically, functional-driven error dominates, although unusually large density errors can sometimes control the total error.

Functional and Density driven error: The total-energy error is decomposed into functional-driven error from the approximate functional and density-driven error from the approximate ground-state density.The definitions use ΔE_F = F̃[n] − F[n] and ΔE_D = Ẽ[ñ] − Ẽ[n].
Functional and Density driven error: In most DFT calculations, the total error ΔE is dominated by the functional-driven component ΔE_F.This establishes the usual error hierarchy used to assess the ML-HK map.
Functional and Density driven error: Abnormally large density errors can dominate the total error in specific cases, where using a more accurate density can greatly improve the result.These definitions are used to measure the accuracy of the ML-HK map.

1-D potentials

The 1-D tests compare machine-learned kinetic-energy functionals optimized through functional derivatives with direct ML-HK maps. Direct ML-HK prediction is more accurate, while functional-derivative errors become density-dominated as training data increase.

1-D potentials: Up to 200 cases of three-Gaussian-dip potentials trained a Kernel Ridge Regression model for the non-interacting kinetic-energy functional.The one-electron Schrödinger equation was solved precisely in a hard-wall box of length 1 atomic unit.
1-D potentials: Poor ML functional derivatives make the density-driven error comparable to or greater than the functional-driven error, eventually dominating as M grows.The calculation can improve substantially with a more accurate density from a finer grid.
1-D potentials: ML-HK is always more accurate than ML-OF, with its relative performance improving as M increases.Its density-driven error is an order-of-magnitude smaller than for ML-OF.
1-D potentials: The density-driven-error proxy overestimates the true error by a factor of 3, so it is used to estimate HK-ML energy errors when the kinetic functional is unavailable.Alternative ML-HK representations included 500 grid points and 200 Fourier basis functions; the grid variant is restricted to 1-D.

Molecules

In molecular tests, the ML-HK approach uses Gaussian external-potential representations and learns density–energy relationships to predict DFT energies and densities. Across H2, H2O, and larger-molecule simulations, ML-HK consistently improves accuracy over ML-KS while preserving DFT-level fidelity.

Method: Gaussian external potentials replace divergent Coulomb potentials as the molecular machine-learning representation.The representation uses width γ = 0.2 Å and grid spacing Δ = 0.08 Å.
H2: For H2, the ML-HK map has significantly lower energy MAE than ML-KS, showing that learning potential-to-density-to-energy is easier than direct potential-to-energy learning.The H2 dataset spans 150 geometries with R between 0.5 and 1.5 Å, using 50 test geometries.
H2: 2.3 kcal/mol is the PBE approximation MAE relative to exact CI for H2, exceeding the machine-learning errors and indicating negligible additional ML-HK error for DFT accuracy.The problematic short-bond region is R < R0 = 0.74 Å, where the ML-HK error remains smoother and smaller than ML-KS.
H2O: 1.2 kcal/mol is the ML-HK MAE for H2O PBE energies relative to CCSD(T), while ML-HK remains consistently more precise than ML-KS and improves the potential energy surface.H2O conformers vary two bond lengths and one bond angle around the PBE-optimized structure R0 = 0.97 Å and θ0 = 104.2°.
Molecular dynamics: 0.23 kcal/mol is the ML-HK MAE for ethane conformer energies on an independent 300 K molecular-dynamics trajectory.The test includes sparsely sampled eclipsed configurations, and training geometries are selected from 300 K and 350 K trajectories using K-means.
Molecular dynamics: 0.77 kcal/mol is the mean absolute energy error for an ML-HK-generated trajectory, although out-of-plane fluctuations reach a maximum error of 5.7 kcal/mol.The generated trajectory samples the same molecular configurations as the ab initio simulation, but typically underestimates energies at the extremes of the classical training set.

DISCUSSION

The work demonstrates machine-learned density functionals in three dimensions using real molecules and production-level codes, while addressing key computational bottlenecks. Its basis-function formulation and direct density-potential learning support integration into DFT codes and potential extension to quantum-chemical simulations.

Advances and scope: This is the first 3-D demonstration of machine-learned functionals using real molecules and production-level codes.Training conformers can be generated through informed scans or classical molecular-dynamics simulations.
Implications: The approach could greatly reduce the computational cost of routine DFT calculations.The authors describe functional approximation by machine learning as a potentially new approach to this problem.
Methodological advances: Directly learning the Hohenberg-Kohn density-potential map avoids the 3-D bottleneck of solving an intermediate gradient-descent problem.The approach uses transductive inference to predict the ground-state density directly.
Methodological advances: Basis functions make the method computationally feasible and easier to integrate into existing DFT codes.The approach also preserves spatial correlations by using low-frequency basis functions.
Limitations and extensions: The 3-D calculations machine-learned E[n], the entire energy including an exchange-correlation density-functional approximation, rather than only the kinetic energy.Using a quantum chemical code would enable training on more accurate quantum-chemical densities and energies.
Limitations and extensions: Quantum-chemical training could in principle yield nearly exact molecular density functionals and reduce the cost of quantum-chemistry molecular-dynamics simulations.The authors identify these as directions for extending the reported results.

METHODS

The methods combine KS-DFT reference calculations with kernel ridge regression to learn density-based maps, then evaluate models through cross-validation, accurate quantum-chemical energies, and molecular-dynamics simulations. Calculations use specified plane-wave, pseudopotential, molecular-box, and trajectory protocols.

Methodological basis: KS-DFT determines many-body properties from electron-density functionals, while solving the Kohn-Sham equations is the computational bottleneck.The Hohenberg-Kohn theorem establishes a one-to-one relationship between ground-state density and potential.
DFT calculations: The 3-D ML calculations use Quantum ESPRESSO with PBE, projector augmented waves, Troullier-Martin pseudization, a 20-bohr cubic box, and a 90-Ry wave-function cutoff.The 1-D dataset is taken from Snyder et al.
Machine-learning model: Kernel Ridge Regression minimizes least-squares error with ℓ2 (Tikhonov) regularization using a kernel matrix and an analytical solution.The Gaussian radial basis function kernel is described as a smooth nonlinear model in input space.
Model evaluation: The ML-HK map uses the L2 distance between predicted and true densities as its canonical error, with coefficients optimized independently for each basis coefficient.Model parameters and hyperparameters are estimated on the training set, and the fixed model is applied unchanged out-of-sample.
Exact calculations: Relative ML-model energy errors are assessed against Molpro reference energies from Full Configuration Interaction for H2 and CCSD(T) for H2O.These comparisons provide accurate quantum-chemical energy references for the KS-DFT-trained models.
Molecular dynamics: Molecular-dynamics datasets cover benzene, ethane, and malonaldehyde using GAFF-based classical simulations, while malonaldehyde also receives a PBE Born-Oppenheimer DFT trajectory.Classical trajectories use PINY MD with massive Nosé-Hoover chain thermostats, and snapshots are collected after equilibration for ML-model DFT calculations.