Source-linked AI summary
Deep autoregressive models for the efficient variational simulation of many-body quantum systems
Or Sharir, Yoav Levine, Noam Wies, Giuseppe Carleo, Amnon Shashua
TL;DR
Deep convolutional neural-network quantum states have been limited by costly stochastic sampling and optimization bottlenecks. This paper introduces autoregressive quantum states for efficient, exact sampling. For 10×10 systems, it reports a lower ground-state-energy relative error than the cited PEPS result.
Problem
Deep convolutional neural-network quantum states have remained relatively shallow because MCMC sampling is computationally expensive and large parameter counts create an optimization bottleneck.
Source: p. 1Method
The authors extend NADE into Neural Autoregressive Quantum States (NAQS), a convolutional architecture that supports efficient, exact, independent sampling.
Source: pp. 1–2Results
For 10×10, NAQS reached relative error 3.5×10^-5±0.4×10^-5 versus PEPS's 8.7×10^-5±0.6×10^-5 reported in Liu et al..
Source: p. 5Takeaways & Limitations
The approach allows training 20-layer convolutional networks with ∼1M parameters on a 21×21 lattice.
Source: p. 5The autoregressive construction requires its conditional factors to satisfy a normalization condition for each fixed preceding configuration.
Source: p. 2
Abstract
from arXiv · showhide
Artificial Neural Networks were recently shown to be an efficient representation of highly-entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in Variational Monte Carlo, most notably the use of Markov-Chain Monte-Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) Its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized Neural Network architecture that supports efficient and exact sampling, completely circumventing the need for Markov Chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.
Appendix A: Proof of Claim 1
Claim 1 is proved by induction on N, using the normalized first component as the base case and normalized conditional wave functions in the inductive step.
Appendix A: Proof of Claim 1
- The proof establishes normalization by induction: Ψ(s1) ≡ Ψ1(s1) for N = 1, then uses a normalized conditional wave function and the induction assumption for N = k + 1.The two steps are the base case and the inductive step, respectively.Source: Appendix A · p. 5
Appendix B: Technical Details
Appendix B covers technical details of the models and their optimization.
- Appendix B presents essential technical details of the models and how they are optimized.Source: Appendix B · p. 5
1. Architecture
The implementation is a PixelCNN-inspired row-wise autoregressive network with interacting vertical and horizontal convolutional branches; it represents complex log-amplitudes using two real coordinates.
Architecture
- The network uses PixelCNN-inspired row-wise ordering and interacting vertical and horizontal branches, with real-valued parameters encoding complex log-amplitudes as two real coordinates.The vertical branch represents dependencies on particles above a given particle, while the horizontal branch represents dependencies on particles to its left.Source: §1 · p. 5
- Causal padding and masking ensure the vertical branch depends only on rows above and the horizontal branch only on particles earlier in row-wise order.Both branches use 3 × 3 convolutions with 32 channels; the vertical branch pads two rows at the top, while the horizontal branch pads two rows and columns at the top and left and zeros its (3, 3) parameters.Source: §1 · p. 6
- Vertical features are shifted down and concatenated into the horizontal branch, whose two-channel output encodes log-amplitudes; separating branches addresses PixelCNN’s blind spot.The network typically uses 10–40 blocks, depending on the experiment.Source: §1 · p. 6
2. Handling Symmetries
The model uses problem symmetries such as rotations and flips to improve accuracy through a self-ensemble. Its aggregation must preserve probabilistic meaning so that efficient sampling remains possible.
- A self-ensemble transforms each spin configuration by the problem’s symmetries, evaluates each transformed input, and aggregates the resulting log-amplitude outputs.The authors report that leveraging inherent symmetries can dramatically improve model accuracy.Source: §2 · p. 6
- To preserve probabilistic meaning, the model samples a symmetry transformation uniformly, samples from the model, then applies the transformation, yielding a mixture over squared amplitude magnitudes.Equation B1 realizes the mixture in log-space, where the output’s real part represents log-magnitude.Source: §2 · p. 6
- For the imaginary part, the authors found that averaging circular phase quantities worked best in their experiments.The authors note that the phase aggregation has fewer restrictions than the magnitude aggregation.Source: §2 · p. 6
3. Optimization
Optimization uses a three-stage schedule that progressively increases batch size and switches from Adam to momentum SGD in the final stage.
3. Optimization
- Training uses Adam at batch size 100 for about 10K updates, batch size 1000 for another 10K, then a larger batch with momentum SGD for 5K.The first two stages use a learning rate in the order of 10^-3; experiments vary batch size, learning rate, and stage lengths, reporting the best-performing models.Source: §3 · p. 6
Appendix A: Proof of Claim ??
The appendix proves normalization of Ψ by induction, using the normalized base case and a step based on a normalized conditional wave function and the induction assumption.
Appendix A: Proof of Claim ??
- The proof establishes normalization by induction: Ψ is normalized for N = 1 because Ψ1 is normalized, and the N = k + 1 step uses a normalized conditional wave function and the induction assumption.The proof identifies these as the reasons for the two equalities in the induction step.Source: Appendix A · p. 8
Appendix B: Technical Details
Appendix B covers essential technical details of the models and their optimization.
Appendix B: Technical Details
- The appendix describes essential technical details of the models and how they are optimized.Source: Appendix B · p. 8
1. Architecture
The Neural Autoregressive Quantum State uses a PixelCNN-inspired row-wise ordering, with real-valued network operations representing complex conditional log-amplitudes. Its vertical and horizontal convolutional branches encode dependencies on preceding lattice sites.
Architecture
- The architecture follows a PixelCNN-inspired row-wise particle ordering and uses real-valued parameters and operations to represent complex conditional log-amplitudes as two real numbers.Source: §1 · p. 8
- A vertical branch captures dependencies on particles above each site, while a masked horizontal branch captures dependencies on particles to its left and prevents dependence on sites ahead in the ordering.The branches use 3 × 3 convolutional filters with 32 channels; padding and zeroed filter parameters enforce the dependency constraints.Source: §1 · p. 8
- Shifted vertical-branch activations are concatenated into the horizontal branch, whose final layer outputs two channels for the real and imaginary parts of each conditional log-amplitude.The network typically uses 10 to 40 blocks; the two-branch separation addresses PixelCNN’s blind-spot problem.Source: §1 · p. 8
2. Handling Symmetries
Leveraging problem symmetries through a self-ensemble can improve model accuracy, while the aggregation must preserve probabilistic meaning for efficient sampling.
2. Handling Symmetries
- The self-ensemble transforms each spin configuration by the problem’s symmetries, runs the transformed inputs through the model, and aggregates their log-amplitude outputs.The authors report that exploiting symmetries such as rotations and flips can dramatically improve accuracy.Source: §2 · pp. 8–9
- To retain efficient sampling, the model samples a symmetry transformation uniformly, draws a model sample, then transforms it, yielding a mixture over squared amplitudes.The mixture is represented in log-space by the proposed symmetrization equation.Source: §2 · p. 9
- For the imaginary output, the authors found that averaging circular phase quantities worked best experimentally, while permitting a broader choice of symmetric operators.The real output represents log-magnitude; the phase aggregation is expressed as a mean of circular quantities.Source: §2 · p. 9
3. Optimization
Optimization proceeds through staged changes in batch size and optimizer, while experiments vary the default settings and report the best-performing models.
3. Optimization
- Training starts with Adam, using batches of 100 samples and a learning rate on the order of 10^-3, then uses batches of 1,000 for 10K additional updates.The first stage lasts about 10K gradient updates; the second keeps the same learning rate and optimizer.Source: §3 · p. 9
- The final stage increases batch size, switches to momentum SGD, and runs for 5K additional updates.Source: §3 · p. 9
- Experiments test variations in batch size, learning rate, and update steps per stage, reporting results for the best-performing models.Source: §3 · p. 9