PaperlayerView exact revision on arXiv

Source-linked AI summary

Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks

David Pfau, James S. Spencer, Alexander G. de G. Matthews, W. M. C. Foulkes

arXiv:1909.02487v3physics.chem-phcs.LGphysics.comp-ph

TL;DR

Accurate many-electron wavefunctions require Ansätze that balance computational efficiency with fermionic antisymmetry and expressive correlation modeling. The paper introduces FermiNet, a determinant-based deep neural-network Ansatz optimized with VMC, and reports accuracy beyond conventional VMC Ansätze and coupled-cluster methods on challenging systems. The results support direct wavefunction optimization without finite basis sets for the systems examined.

  • Problem

    Accurate wavefunction Ansätze must balance efficiency and accuracy while obeying Fermi-Dirac antisymmetry, but exact solutions are generally inaccessible and existing neural-network approaches have been limited for electronic systems.

  • Method

    FermiNet uses permutation-equivariant neural networks whose electron-dependent features feed weighted sums of determinants, optimized within variational quantum Monte Carlo.

  • Results

    FermiNet achieves higher accuracy than other VMC Ansätze on atoms and small molecules and outperforms coupled cluster on stretched systems examined.

  • Takeaways & Limitations

    The results show that deep neural networks can make VMC competitive with DMC, AFQMC, and CCSD(T), while avoiding basis-set selection and extrapolation.

Abstract

from arXiv · show

Given access to accurate solutions of the many-electron Schrödinger equation, nearly all chemistry could be derived from first principles. Exact wavefunctions of interesting chemical systems are out of reach because they are NP-hard to compute in general, but approximations can be found using polynomially-scaling algorithms. The key challenge for many of these algorithms is the choice of wavefunction approximation, or Ansatz, which must trade off between efficiency and accuracy. Neural networks have shown impressive power as accurate practical function approximators and promise as a compact wavefunction Ansatz for spin systems, but problems in electronic structure require wavefunctions that obey Fermi-Dirac statistics. Here we introduce a novel deep learning architecture, the Fermionic Neural Network, as a powerful wavefunction Ansatz for many-electron systems. The Fermionic Neural Network is able to achieve accuracy beyond other variational quantum Monte Carlo Ansätze on a variety of atoms and small molecules. Using no data other than atomic positions and charges, we predict the dissociation curves of the nitrogen molecule and hydrogen chain, two challenging strongly-correlated systems, to significantly higher accuracy than the coupled cluster method, widely considered the most accurate scalable method for quantum chemistry at equilibrium geometry. This demonstrates that deep neural networks can improve the accuracy of variational quantum Monte Carlo to the point where it outperforms other ab-initio quantum chemistry methods, opening the possibility of accurate direct optimization of wavefunctions for previously intractable many-electron systems.

I. INTRODUCTION

The paper introduces FermiNet as a flexible neural-network Ansatz for fermionic many-electron wavefunctions, addressing the accuracy–efficiency limits of existing representations. It extends VMC beyond conventional Slater-Jastrow-backflow forms while preserving antisymmetry through determinant-based construction.

  • Motivation: Exact many-electron wavefunctions are generally inaccessible, making compact and accurate wavefunction Ansätze essential for practical quantum chemistry.The number of Slater determinants grows exponentially with system size, restricting full configuration interaction to tiny molecules.
  • Existing Ansätze: The default Slater-Jastrow Ansatz combines truncated Slater determinants with a Jastrow factor for close-range pair and triplet correlations.Backflow transformations can further shift electron coordinates, but the Slater-Jastrow-backflow form remains standard for continuous-space many-electron problems.
  • FermiNet: FermiNet uses a neural network as a more flexible VMC Ansatz, avoiding finite basis sets and compactly modeling higher-order electron-electron interactions.Its inputs include atomic positions and charges, together with electron-level features used to construct the wavefunction.
  • FermiNet: Unlike prior neural-network approaches limited by fermionic statistics, FermiNet generalizes orbitals to depend symmetrically on all electron positions and spins.The architecture starts from Slater determinants and replaces single-electron orbitals with exchangeable nonlinear functions of the full electronic configuration.
  • FermiNet: FermiNet uses permutation-equivariant intermediate representations so determinant outputs satisfy the antisymmetry requirement of fermionic wavefunctions.Features from one- and two-electron streams are processed through network layers before determinants produce the wavefunction.
  • Results: The authors report that FermiNet is competitive with projector methods and outperforms coupled cluster on stretched systems.The comparison is presented as evidence that VMC accuracy can exceed conventional expectations for challenging geometries.

II. FERMIONIC NEURAL NETWORKS

FermiNet constructs antisymmetric wavefunctions from permutation-equivariant neural-network features and weighted sums of determinants. Its architecture incorporates electron–nucleus and electron–electron features, spin-resolved streams, nonlinear mixing, and decay envelopes.

  • Architecture: FermiNet replaces conventional single-electron orbitals with permutation-equivariant functions that depend on one electron and the invariant set of all others.A small linear combination of determinants is used in practice, although a single such determinant is theoretically sufficient to represent any antisymmetric function.
  • Inputs: The network accepts electron–nucleus and electron–electron position differences and distances as inputs, allowing it to represent wavefunction cusps without a separate Jastrow factor.Direct distance inputs help express non-smooth behavior when particles coincide, which is important for estimating energies and other properties.
  • Layers: Permutation-equivariant layers average activations across streams, concatenate the resulting features, and feed them through linear layers with nonlinearities.One-electron streams receive information from one- and two-electron streams, while two-electron streams remain parallel to reduce computational overhead.
  • Output: The final network applies spin-dependent linear transformations and exponentially decaying envelopes before combining multiple determinants into the wavefunction.The envelopes enforce the boundary condition that the wavefunction vanishes far from the nuclei.
  • Empirical comparison: FermiNet substantially exceeds Slater-Jastrow-network accuracy for CO and N2 across increasing determinant counts against CCSD(T)/CBS baselines.The figure caption states that the FermiNet remains far more accurate regardless of the number of determinants used.
  • Spin structure: Spin-up and spin-down determinant factors can be separated through block-diagonal structure, while same-spin antisymmetry still yields correct spin-independent expectation values.The fully antisymmetric wavefunction can be reconstructed if required.
  • Properties: The authors report size-consistent solutions for N2 and the hydrogen chain, despite concerns motivated by similarities to truncated configuration-interaction Ansätze.They attribute the practical need for multiple determinants to finite neural-network representational limits, not to a formal insufficiency of the Ansatz.

B. Wavefunction optimization

Wavefunction optimization minimizes the variational energy using stochastic estimates and a modified KFAC second-order method. The reported calculations include atomic and molecular energy, ionization-potential, and electron-affinity benchmarks.

  • Energy minimization: The optimization objective is the energy expectation value of the wavefunction Ansatz.For an unrestricted real normalizable function, the minimum occurs at the ground-state eigenfunction; a restricted Ansatz yields an energy above the ground-state eigenvalue.
  • Stochastic optimization: Stochastic samples from p(X) ∝ ψθ^2(X) provide unbiased estimates of the energy gradient with respect to Ansatz parameters.The local energy appears in the gradient expression used for variational Monte Carlo optimization.
  • KFAC optimization: KFAC approximates natural-gradient updates by using a block-diagonal Fisher Information Matrix and Kronecker-factorized layer inverses.These approximations make solving Fδθ = ∇θL practical for neural networks with thousands to millions of parameters.
  • KFAC optimization: The modified KFAC procedure is equivalent to stochastic reconfiguration for unnormalized probability densities and is closely related to the linear method.The paper extends KFAC to the unnormalized wavefunctions used in this setting.
  • Benchmark evaluation: All reported ground-state energies are within chemical accuracy of exact numerical solutions, while ionization potentials and electron affinities are within chemical accuracy of experiment except neon.The atomic benchmarks include ground-state energies, ionization potentials, and electron affinities.

III. RESULTS

The paper evaluates FermiNet across problems in chemistry and electronic structure, with detailed architectures and training procedures provided separately.

  • Evaluation scope: FermiNet performance is evaluated on a variety of chemistry and electronic-structure problems.The exact architectures and training procedures for FermiNet and baseline methods are described in Appendix A.

A. Slater-Jastrow versus FermiNet Ans¨atze

FermiNet replaces conventional orbital, Jastrow, and backflow components with a more flexible neural-network Ansatz. It improves accuracy over Slater-Jastrow baselines and approaches coupled-cluster performance with relatively few determinants.

  • Compared Ansätze: The comparisons include conventional Slater-Jastrow and Slater-Jastrow-backflow Ansätze with varying numbers of determinants.Neural-network versions of these Ansätze are also evaluated.
  • Compared Ansätze: Neural-network Slater-Jastrow-backflow uses residual networks to represent orbitals, the Jastrow factor, and backflow transformations.This makes the baseline more flexible than versions using Hartree-Fock orbitals, closed-form Jastrow factors, and few-parameter backflow.
  • Optimization: KFAC shows a qualitative optimization advantage over ADAM for first-row atoms, H2, LiH, and the hydrogen chain.The plotted statistic is the median energy over the last 10% of iterations; a small KFAC overshoot reflects slow MCMC equilibration.
  • Atomic accuracy: The FermiNet reduces first-row atomic error relative to Slater-Jastrow without backflow by almost an order of magnitude and relative to Slater-Jastrow-backflow by more than a factor of two.A single FermiNet determinant comes within a few mEh of chemical accuracy, defined as 1.594 mEh.
  • Molecular accuracy: At 64 determinants, FermiNet comes within a few mEh of CCSD(T) for nitrogen and carbon monoxide, while Slater-Jastrow remains far from CCSD(T) accuracy.The FermiNet cuts the Slater-Jastrow-backflow error roughly in half, and its energies begin to plateau as determinant count increases.

B. Equilibrium Geometries

Across atoms and small molecules, the FermiNet achieves high accuracy, including chemical accuracy for several systems and strong performance relative to established quantum chemistry baselines. Its accuracy remains high for larger molecules, although correlation recovery declines gradually with system size.

  • Generalization: The same 16-determinant FermiNet generalizes across atoms, diatomic molecules, and small organic molecules using shared training hyperparameters.The evaluation uses experimental or exact computational results where available, and all-electron CBS CCSD(T) otherwise.
  • Accuracy: Chemical accuracy is achieved for all atoms, LiH, Li2, methane, and ammonia, outperforming conventional Slater-Jastrow-backflow VMC with more determinants.FermiNet energies are reported as more comparable in quality to diffusion Monte Carlo, which is typically more accurate than VMC.
  • Strong correlation: In the H4 rectangle, coupled cluster predicts a cusp and energy minimum at Θ = 90°, whereas FermiNet agrees with exact FCI results.The geometry is specified by R = 3.2843a0.
  • Larger molecules: Over 99% of correlation energy is recovered for ethene, declining gradually to approximately 97% for methylamine, ozone, ethanol, and bicyclobutane.Bicyclobutane is especially challenging because of its high ring strain and large number of electrons.
  • Comparison with CCSD(T): FermiNet is more accurate than CCSD(T) in the largest practically tractable finite basis sets, suggesting comparable extrapolation could match or exceed CCSD(T) accuracy.The reported basis sets are quintuple ζ for most systems and quadruple ζ for larger systems.
  • Charged species: Mean absolute errors are 0.09 mEh for first-row ionization potentials and 0.66 mEh for electron affinities, showing similar accuracy for charged and neutral species.These quantities are compared with experimental data after removing relativistic effects.

C. The H4 Rectangle

The paper evaluates FermiNet on hydrogen-chain and broader scaling and observable analyses, finding strong performance on challenging systems and favorable iteration-time behavior. It also examines densities and architectural scaling beyond total-energy accuracy.

  • Hydrogen chain: FermiNet remains comparable to AFQMC across all H10 separations, while coupled-cluster and conventional VMC errors are large at medium separations.The comparison uses differences from highly accurate MRCI+Q+F12 results, with shaded basis-set extrapolation uncertainty.
  • Hydrogen chain: Hydrogen-chain electron density dimerization becomes stronger as nuclear separation decreases, clustering density around pairs of nuclei.The dimerization pattern is relevant to electronic structure in insulating systems and metal-insulator transitions.
  • Observables: The neon exchange-correlation hole behaves as expected near coincident electron positions, indicating smooth and well-behaved first- and second-order statistics.The analysis separates same-spin, different-spin, and all-electron contributions across radial positions near the nucleus.
  • Scaling: Single-step wavefunction optimization has overall quartic asymptotic scaling with system size, but achieving constant accuracy may require system-dependent network sizes and sampling parameters.The stated scaling treats system size, network size, and sampling steps independently.
  • Scaling: Runtime measurements on atoms from lithium to zinc support a cubic fit, with little difference between cubic and quartic fits at this scale.The timing covers one iteration consisting of 10 MCMC steps, local-energy and gradient evaluation, and a KFAC update.

C. Feature Ablation and Network Architectures

Feature and architecture ablations show that distance features are important for stable, accurate training, while widening the one-electron stream improves hydrogen-chain accuracy more consistently than widening the two-electron stream.

  • Feature ablation: All oxygen-atom configurations without electron-nuclear distances became numerically unstable and diverged.The ablation therefore links these distance features to training stability as well as accuracy.
  • Feature ablation: Including interelectron distances produced most of the accuracy gain from interelectron features, while interelectron coordinates provided a smaller additional improvement.All tested networks retained nuclear coordinates.
  • Feature ablation: Distance inputs make the wavefunction significantly easier to approximate because they directly represent the non-smooth behavior associated with cusp conditions.A smooth neural network can approximate cusps closely, but not perfectly.
  • Network architecture: H10 accuracy improves with added layers, but the gain from three to four layers is only about 1 mEh.Increasing one-electron-stream width improves accuracy uniformly, whereas two-electron-stream improvements quickly saturate.
  • Network architecture: The one-electron-stream error scales as O(N^-0.395±0.067), implying approximately 760 hidden units would be needed for chemical accuracy on H10.Increasing the two-electron stream from 16 to 32 units appears slightly noisier.
  • Overall finding: Across the studied systems, one FermiNet architecture and one training-parameter set attained high accuracy on every system examined.The discussion presents this as evidence that antisymmetric neural networks can support high-accuracy calculations on challenging systems.

Appendix A: Experimental Setup

The experiments use a fixed FermiNet configuration with pretraining, MCMC sampling, and tuned optimization procedures, while computational resources and stability adjustments vary for larger systems.

  • Configuration: The default FermiNet uses four hidden layers, 256 one-electron-stream units, 32 two-electron-stream units per layer, tanh activations, and 16 determinants.The final linear orbital-output layer is not counted among the four layers.
  • Pretraining: Hartree-Fock orbital pretraining improves subsequent local-energy optimization stability and prevents determinant underflow failures on large systems.The pretraining uses a minimal STO-3G basis only to provide a stable initialization near the mean-field solution.
  • Sampling: Training samples combine Hartree-Fock-orbital and neural-network distributions, with MCMC used to estimate the pretraining loss and expectation values.The neural-network component increases sampling in regions where the wavefunction may be spuriously large.
  • Scaling setup: Bicyclobutane training used a halved batch size and tenfold more pretraining iterations, while larger systems generally required more GPUs and careful hyperparameter tuning.Bicyclobutane took roughly one month on 16 GPUs; KFAC damping and norm constraints critically affect stability and convergence.
  • Optimization: Optimization updates clip extreme local energies for gradient computation, using a five-times-total-variation threshold around each batch median.The total variation is preferred over standard deviation because it is more robust to outliers.
  • Sampling: All-electron moves are sampled simultaneously with Metropolis-Hastings because orbitals depend on all electrons, preventing fast one-electron determinant updates.The moves use a fixed isotropic Gaussian proposal covariance.

2. Slater-Jastrow networks

The paper contrasts conventional Slater-Jastrow-backflow constructions with FermiNet’s generalized determinant Ansatz. FermiNet uses permutation-equivariant orbitals and explicit distance features to represent antisymmetric many-electron wavefunctions.

  • Conventional Ansätze: The baseline Slater-Jastrow-backflow model uses standard Jastrow and backflow forms implemented with multilayer perceptrons and a shared multiplicative envelope.Its backflow network is initialized near zero from an optimized Slater-Jastrow network because random initialization was extremely unstable.
  • Reference methods: The paper’s CCSD(T) comparisons use all-electron calculations, with software and basis choices varying across atoms, dimers, and molecules.FCI calculations are reported for H4, while Cholesky decomposition reduces bicyclobutane memory requirements with an estimated O(10^-5) hartree energy error.
  • Universality: A single determinant is theoretically sufficient for any antisymmetric function when its entries are suitable permutation-equivariant functions.The paper notes that practical accuracy still improves empirically as the number of determinants grows.
  • Universality: The universal one-determinant construction is not directly learnable by FermiNet because its indicator-based functions are discontinuous.This may partly explain why multiple determinants remain necessary for high practical accuracy.

Appendix C: Equivalence of Natural Gradient Descent and Stochastic Reconfiguration

The appendix derives that stochastic reconfiguration is equivalent to natural gradient descent for unnormalized distributions. It also describes numerically stable derivative handling for singular matrices and determinant-based wavefunction outputs.

  • Equivalence: Stochastic reconfiguration is equivalent to natural gradient descent for unnormalized distributions.The appendix presents this connection as one that has not been widely appreciated across machine learning and quantum chemistry.
  • Equivalence: The wavefunction density is represented by p(X), with normalizing factor Z(θ) and unnormalized density ˜p(X) = ψ2(X).These definitions provide the quantities used to derive the stochastic-reconfiguration preconditioner.
  • Metric derivation: The stochastic-reconfiguration preconditioner M is a parameter-space metric that, up to a constant, equals the Fisher information metric for p(X).The derivation rewrites expectations involving the logarithm of the unnormalized density in terms of the normalizing factor before identifying the metric.
  • Numerical stability: The FermiNet outputs the logarithm of the wavefunction magnitude with its sign and uses log determinants for numerical stability.A log-sum-exp procedure also prevents numerical underflow when determinant magnitudes are poorly scaled.
  • Singular matrices: Closed-form reverse-mode derivatives keep determinant and cofactor derivatives well defined when matrices are singular.The appendix uses singular-value decompositions and cancellation of problematic terms, then fuses the computation into a custom TensorFlow operation with gradient-of-gradient support.

Appendix E: Non-interacting hydrogen chains

At sufficiently large separations, hydrogen atoms become non-interacting, so the combined energy should equal the sum of individual atomic energies. FermiNet tests this behavior on chains containing up to 10 hydrogen atoms.

  • Non-interacting limit: At sufficiently large separations, two systems become non-interacting and their combined energy should equal the sum of their individual energies.
  • Evaluation: FermiNet demonstrates this non-interacting behavior on well-separated hydrogen-atom chains.
  • Evaluation: The test covers hydrogen chains of up to 10 atoms.

Appendix F: Electron-Electron and Electron-Nuclear Cusps

Electron-electron and electron-nuclear coincidences produce wavefunction cusps that must be represented correctly. The appendix evaluates FermiNet’s cusp description for helium using spherical-surface Monte Carlo integration.

  • Cusp conditions: Wavefunction derivatives are discontinuous when two electrons or an electron and nucleus coincide, canceling corresponding Hamiltonian singularities.These relationships are expressed through electron-electron and electron-nuclear cusp conditions.
  • Cusp conditions: Correctly capturing cusps, especially the electron-nuclear cusp, is critical for accurately capturing correlation energy.
  • Cusp conditions: The cusp conditions relate the wavefunction to its derivative using electron-nuclear distance riI, electron-electron distance rij, and nuclear charge ZI.The derivative relation is obtained under the assumption that the wavefunction is non-zero at the coincidence points, with spherical averaging where indicated.
  • Helium evaluation: FermiNet correctly describes the cusps for the helium atom in the evaluation shown in Fig. 11.The estimates use Monte Carlo integration over spherical surfaces of radius 10^-5a0 centered on the helium nucleus and second electron.
Loading 1909.02487v3…