PaperlayerView exact revision on arXiv

Source-linked AI summary

Recurrent Neural Network Wave Functions

Mohamed Hibat-Allah, Martin Ganahl, Lauren E. Hayward, Roger G. Melko, Juan Carrasquilla

arXiv:2002.02973v4cond-mat.dis-nncond-mat.str-elphysics.comp-phquant-ph

TL;DR

Quantum many-body calculations need compact variational wave functions that can represent ground states and support efficient estimator evaluation. This paper trains autoregressive RNN wave functions stochastically across one- and two-dimensional spin models, finding accurate and competitive results with compact parameterizations. The approach also addresses complex sign structures through cRNNs and benefits from independent autoregressive samples.

  • Problem

    Approximating ground states and observables of strongly correlated many-body Hamiltonians requires effective variational wave functions, while conventional sampling methods can hinder estimation and optimization.

  • Method

    The paper uses stochastically optimized autoregressive RNN wave functions, with complex RNNs modeling sign structures and independent samples generated directly from the wave-function probability.

  • Results

    RNN wave functions reproduce ground-state energies, correlations, and entanglement across one- and two-dimensional spin models, agreeing competitively with established methods while using compact parameterizations.

  • Takeaways & Limitations

    Autoregressive sampling and weight sharing make RNN wave functions compact, expressive representations for variational studies of quantum many-body systems.

Abstract

from arXiv · show

A core technology that has emerged from the artificial intelligence revolution is the recurrent neural network (RNN). Its unique sequence-based architecture provides a tractable likelihood estimate with stable training paradigms, a combination that has precipitated many spectacular advances in natural language processing and neural machine translation. This architecture also makes a good candidate for a variational wave function, where the RNN parameters are tuned to learn the approximate ground state of a quantum Hamiltonian. In this paper, we demonstrate the ability of RNNs to represent several many-body wave functions, optimizing the variational parameters using a stochastic approach. Among other attractive features of these variational wave functions, their autoregressive nature allows for the efficient calculation of physical estimators by providing independent samples. We demonstrate the effectiveness of RNN wave functions by calculating ground state energies, correlation functions, and entanglement entropies for several quantum spin models of interest to condensed matter physicists in one and two spatial dimensions.

I. INTRODUCTION

The paper brings recurrent neural networks into variational quantum many-body physics, using their autoregressive structure to approximate ground states efficiently. It evaluates these wave functions across strongly correlated systems in one and two dimensions.

  • Neural-network methods are being applied to variational calculations for interacting many-particle systems, an active problem in quantum information and condensed matter physics.
  • RNNs had represented complex quantum systems and captured entanglement properties, but their use with natural-language-processing advances for many-body variational calculations remained limited.
  • The paper combines the variational principle with RNNs to construct autoregressive ansatz wave functions whose parameters approximate ground states of strongly correlated local Hamiltonians.
  • RNN wave functions provide accurate local correlation functions and entanglement entropies while using only a fraction of the variational parameters required by established approaches.
  • Scaling studies show that increasing model capacity systematically reduces the ansatz bias and improves ground-state approximations for large quantum systems.

A. RNNs for classical probability distributions

RNNs represent discrete correlated probability distributions by factorizing them into sequential conditional probabilities. Hidden states carry prior configuration information, enabling normalized likelihood evaluation, autoregressive sampling, and compact modeling of correlations.

  • A configuration is an ordered list of N discrete variables, each taking one of dv possible values.
  • The probability P(σ) factorizes into conditionals P(σi|σ<i), with each conditional depending on the preceding variables.
  • The full conditional representation can grow exponentially with system size, motivating approximate descriptions that use fewer resources.
  • Sequentially computing the conditionals yields a normalized probability, while autoregressive sampling generates configurations one variable at a time.
  • RNN hidden states encode previous configurations and pass that history forward, allowing the model to represent strongly correlated distributions.
  • Vanilla RNN training can produce exploding or vanishing gradients for long-distance correlations, so the paper uses GRU cells unless stated otherwise.

B. RNN wave functions

The paper converts RNN probability models into variational wave functions by modeling amplitude probabilities and, when necessary, phase information separately. This yields positive and complex ansätze suited to different ground-state sign structures.

  • Positive recurrent neural network wave functions model the probability P(σ)=|ψ(σ)|^2 with a conventional RNN for states having positive amplitudes.
  • The complex construction splits the wave function into amplitude and phase, extending the probability-distribution representation to complex-valued quantum amplitudes.
  • The complex RNN wave function combines a Softmax layer for the modulus with a Softsign layer for the phase of the complex amplitude.
  • In the complex architecture, sampling from |ψ(σ)|^2 uses only the Softmax layer and is unaffected by the phase layer.
  • The resulting term RNN wave function refers collectively to both positive and complex recurrent neural network wave functions.

III. GROUND STATES WITH RNN WAVE FUNCTIONS

The study applies RNN wave functions to transverse-field Ising and J1-J2 spin models, selecting positive or complex ansätze according to the ground-state sign structure. Variational parameters are optimized by minimizing the energy expectation value.

  • The benchmark systems are one- and two-dimensional transverse-field Ising models and the one-dimensional J1-J2 model with open boundary conditions.
  • Variational training minimizes Eλ=⟨Ψλ|H|Ψλ⟩, which is bounded below by the exact ground-state energy E0, using gradient-descent variants.
  • The positive RNN ansatz is used for the stoquastic transverse-field Ising model, whereas the complex ansatz is used for the J1-J2 model's sign structure.
  • One-dimensional RNN wave functions are used for both one-dimensional models, while one- and two-dimensional positive RNN wave functions are used for the two-dimensional Ising model.

A. 1D transverse field Ising model

The pRNN wave function accurately approximates the 1D TFIM ground state at criticality, matching DMRG across energies, correlations, and Rényi entropies. Symmetry enforcement further improves entropy estimates for the larger system.

  • Ground-state optimization: For N = 1000 spins at h = 1, a single-layer pRNN with 50 memory units obtains very accurate ground-state results using approximately 8000 parameters.The optimization uses 200 samples per gradient step, sufficient for convergence, and has parameter count comparable to an RBM with eight hidden units.
  • Correlation functions: The optimized pRNN correlation functions along the x and z axes are consistent with DMRG.The comparison uses 10^6 samples for sites n > 40.
  • Correlation functions: RNN wave functions provide physical estimators beyond energies and variances, including correlation functions relevant to condensed-matter measurements.Examples cited include density-density correlations and Green’s functions.
  • Entanglement entropy: For N = 20 and 80, pRNN estimates of the Rényi entropy S2 agree excellently with DMRG.The entropy is computed using the replica trick for reduced density matrices of spin-chain subregions.
  • Entanglement entropy: The parity-enforced “Symmetric RNN” gives a more accurate S2 estimate for N = 80 sites.The improvement is reported for the larger of the two tested system sizes.

B. 1D J1 −J2 model

The cRNN wave function is applied to the sign-structured ground states of the 1D J1-J2 model. A prior Marshall sign substantially improves energies at small J2, while the cRNN can also recover part of the unknown sign structure without that prior.

  • Model and ansatz: A three-layer cRNN with 100 memory units per GRU layer targets the J1-J2 ground state at J2 = 0.0, 0.2, 0.5, and 0.8.The ansatz imposes U(1) spin symmetry and is designed for ground states with computational-basis sign structure.
  • Model and ansatz: The J1-J2 model transitions near J2 = 0.241167 ± 0.000005 from a critical Luttinger liquid to a spontaneously dimerized gapped valence-bond phase.The phase boundary is reported from prior DMRG studies.
  • Sign structure: For small J2, applying a Marshall sign considerably improves cRNN ground-state energies relative to omitting a prior sign.Figure 4 compares relative errors to DMRG for N = 100 spins.
  • Sign structure: Without a prior sign, cRNN wave functions still achieve accurate ground-state energy estimates, indicating recovery of part of the unknown sign structure.The benefit of the Marshall sign is less pronounced at J2 = 0.8.
  • Special cases: At J2 = 0.5, the model is the Majumdar-Ghosh model, and the cRNN agrees with the exact ground-state energy within error bars.The ground state at this point is a product state of spin singlets.

C. 2D transverse field Ising model

The paper extends RNN wave functions to two-dimensional geometries and evaluates them on the 2D transverse-field Ising model. The 2D architecture achieves close agreement with established methods while using substantially fewer variational parameters.

  • Understanding strongly correlated quantum systems in two dimensions remains a challenging condensed-matter problem.
  • A 2D RNN passes hidden vectors vertically as well as horizontally, better matching the lattice geometry.The model uses a single-layer 2D vanilla RNN with 100 memory units and approximately 21,000 variational parameters.
  • The authors compare 2D and deep 1D pRNN wave functions with 2D DMRG and PixelCNN on the 2D Ising model.The comparisons use magnetic fields h = 2, 3, 4.
  • For large bond dimensions, all four methods show excellent agreement across the displayed magnetic fields.The comparison concerns the obtained ground-state energies.
  • The 2D pRNN uses about 0.03% of the parameters of DMRG with χ = 512, 2.6% of PixelCNN, and 14% of the 1D pRNN.

D. Scaling of resources

The resource study examines how RNN wave-function quality varies with memory units, samples, and layers. Increasing memory systematically improves the wave function, while sample count and depth show weak accuracy dependence.

  • Energy variance per spin σ2 measures wave-function quality, with σ2 = 0 for exact eigenstates.For variational approaches, the variance is expected to decrease as wave-function expressivity increases.
  • Increasing the number of memory units systematically decreases σ2 for critical 1D and 2D transverse-field Ising models.The study covers 1D sizes N = 20, 40, 80 and 2D lattices of 4 × 4, 5 × 5, and 6 × 6.
  • The dependence of σ2 on the number of optimization samples and recurrent layers is weak for the critical 1D model.The optimization used noisy gradient estimates, and the sample-count study found little sensitivity.
  • In 2D, the autoregressive sampling paths differ between raster-scanned 1D RNNs and zigzag 2D RNNs with horizontal and vertical hidden-state transfer.
  • Shallow RNNs with sufficient memory units appear sufficiently expressive, while deeper architectures do not seem accuracy-beneficial in this study.The authors note that deeper networks could still affect memory usage and training speed.

IV. CONCLUSIONS AND OUTLOOK

The paper presents recurrent neural network wave functions as a variational ansatz for estimating properties of quantum many-body ground states. The approach performs competitively while supporting systematic accuracy improvements, sign structures, and independent sampling.

  • RNN wave functions approximate ground-state energies, correlation functions, and entanglement in one- and two-dimensional quantum spin models.
  • RNN wave functions are competitive with DMRG and PixelCNN, particularly for the two-dimensional transverse-field Ising model.
  • Increasing the number of memory units systematically reduces the error in the results.
  • Complex RNN wave functions can model ground states with sign structure, with accuracy improved by an ansatz sign structure and U(1) symmetry enforcement.
  • Autoregressive RNN wave functions directly generate independent samples, avoiding long autocorrelation times associated with Markov-chain sampling.
  • Weight sharing yields compact, expressive states while allowing training with millions of variational parameters.
  • The authors identify attention and higher-order optimization as future techniques for making RNN wave functions more competitive.

OPEN-SOURCE CODE

The paper implements RNN wave functions with gated recurrent units, autoregressive sampling, variational Monte Carlo optimization, symmetry constraints, and extensions to two-dimensional lattices.

  • Appendix A: Gated Recurrent Neural Networks: The publicly available implementation uses GRU cells whose hidden states encode prior spin configurations and whose parameters are optimized by energy minimization.The GRU update interpolates between the previous and candidate hidden states through update and reset gates.
  • RNN wave functions: The RNN wave function generates conditional probabilities and phases from hidden states, enabling autoregressive sampling of spin configurations.The Softmax output represents conditional probabilities, while the Softsign output represents phases.
  • Appendix B: Two-dimensional Recurrent Neural Network wave functions: The two-dimensional RNN passes hidden states horizontally and vertically, preserving lattice geometry while retaining quadratic hidden-state computation and unchanged gradient cost.This construction permits training with a relatively large number of memory units.
  • Appendix C: Variational Monte Carlo and Variance Reduction: Variational Monte Carlo minimizes the energy expectation using independently sampled configurations and stochastic gradient estimates.The authors introduce a variance-reduction term because noisy gradient estimates can slow convergence; near convergence, the gradient variance approaches zero without bias.
  • Symmetry constraints: The framework can impose discrete symmetries and zero magnetization while preserving normalization, autoregressive sampling, and parallelization.The zero-magnetization scheme generates equal numbers of up and down spins and can be extended to fixed non-zero magnetization.

Appendix F: Tables of Results

The reported studies evaluate RNN wave functions on one- and two-dimensional spin models, examining sign structures, geometry, sampling, depth, and training settings.

  • 2D TFIM: The 2D TFIM comparison evaluates 1D and 2D pRNNs alongside PixelCNN and DMRG on a 12 × 12 lattice across transverse fields h.The critical point is reported at h ≈ 3.
  • Sampling study: Increasing the number of gradient-estimation samples produces only a weak change in energy variance for the studied 1D TFIM systems.The corresponding figure indicates that the variance saturates rather than improving substantially with larger sample counts.
  • Depth and parameter scaling: With approximately 31000 variational parameters, energy variance depends only weakly on pRNN depth across architectures of up to four layers.The authors conclude that shallow RNNs with sufficient memory units have enough expressivity for accuracy, while deeper networks may affect memory use and training speed.
  • Training settings: The reported experiments use varied RNN architectures, sample counts, training iterations, learning rates, and multiple seeds documented in the hyperparameter table.The table notes that sample count denotes training batch size and that multiple seeds provide error bars for resource-scaling studies.
Loading 2002.02973v4…