Source-linked AI summary
Deep autoregressive models for the efficient variational simulation of many-body quantum systems
Or Sharir, Yoav Levine, Noam Wies, Giuseppe Carleo, Amnon Shashua
TL;DR
The paper develops a Neural Autoregressive Quantum State architecture and details its convolutional implementation, symmetry handling, and staged optimization. Its supplied passages emphasize preserving exact sampling while incorporating symmetries into the generative model.
Problem
The paper focuses on the technical construction and optimization of neural-network quantum-state models.
Method
The method uses a PixelCNN-inspired autoregressive convolutional architecture with vertical and horizontal branches, plus symmetry-aware sampling.
Results
The symmetry procedure preserves probabilistic meaning while retaining efficient sampling from the generative model.
Takeaways & Limitations
Symmetry handling must be incorporated as a probabilistically valid mixture so efficient sampling remains possible.
Abstract
from arXiv · showhide
Artificial Neural Networks were recently shown to be an efficient representation of highly-entangled many-body quantum states. In practical applications, neural-network states inherit numerical schemes used in Variational Monte Carlo, most notably the use of Markov-Chain Monte-Carlo (MCMC) sampling to estimate quantum expectations. The local stochastic sampling in MCMC caps the potential advantages of neural networks in two ways: (i) Its intrinsic computational cost sets stringent practical limits on the width and depth of the networks, and therefore limits their expressive capacity; (ii) Its difficulty in generating precise and uncorrelated samples can result in estimations of observables that are very far from their true value. Inspired by the state-of-the-art generative models used in machine learning, we propose a specialized Neural Network architecture that supports efficient and exact sampling, completely circumventing the need for Markov Chain sampling. We demonstrate our approach for two-dimensional interacting spin models, showcasing the ability to obtain accurate results on larger system sizes than those currently accessible to neural-network quantum states.
Appendix A: Proof of Claim 1
The claim that the autoregressive product defines a normalized wave-function is established by induction on the number of particles.
- For N = 1, normalization follows because Ψ1 is normalized with respect to s1.
- The induction step assumes normalization for N = k and proves it for N = k + 1.
- The proof uses normalization of Ψk+1 together with the induction assumption to establish the result.
Appendix B: Technical Details
This appendix section introduces the technical details of the models and their optimization procedure.
- The section covers essential technical details of the models.
- The section also covers how the models are optimized.
- Its scope is methodological rather than a separate empirical result.
1. Architecture
The NAQS implementation uses a PixelCNN-inspired row-wise architecture with interacting vertical and horizontal convolutional branches that enforce autoregressive dependencies on a 2D lattice.
- 1. Architecture: The architecture uses a row-wise ordering of particles for the conditional wave-functions.
- 1. Architecture: Figure 4 illustrates a single block composing the architecture.
- 1. Architecture: The vertical branch models dependencies on particles above, while the horizontal branch models dependencies on particles to the left.
- 1. Architecture: The two branches are connected by shifting vertical features downward and concatenating them with horizontal-branch channels.
2. Handling Symmetries
The model is symmetrized by evaluating transformed spin configurations and aggregating their log-amplitudes, while preserving a probabilistic interpretation needed for efficient sampling.
- 2. Handling Symmetries: The self-ensemble scheme applies each problem symmetry to the input, runs the transformed configurations through the model, and aggregates log-amplitude outputs.
- 2. Handling Symmetries: The aggregation must preserve probabilistic meaning so that efficient sampling remains possible.
- 2. Handling Symmetries: Sampling first selects a symmetry transformation uniformly, then samples from the model and transforms the resulting configuration.
3. Optimization
The optimization strategy uses staged stochastic-gradient training, progressively increasing the batch size while retaining the Adam optimizer and an initially small learning rate.
- Training begins with Adam using batches of 100 samples and a learning rate of approximately 10^-3.
- After about 10K gradient updates, the batch size increases to 1000 for another 10K updates.
- The final optimization stage further increases the batch size and changes the optimizer.
Deep autoregressive models for the efficient variational simulation of many-body quantum systems – Supplementary material
The supplementary material identifies the paper’s authors.
- The paper is authored by Or Sharir, Yoav Levine, Noam Wies, Giuseppe Carleo, and Amnon Shashua.
Appendix A: Proof of Claim ??
The appendix proves the normalization claim by induction over the number of quantum variables, using normalized conditional wave-functions at each step.
- The proof starts with N = 1, where Ψ(s1) is identified with the normalized conditional wave-function Ψ1(s1).
- The induction assumes the normalization claim for N = k and constructs the case N = k + 1.
- The final equalities rely on Ψk+1 being normalized conditionally and on the induction assumption.
Appendix B: Technical Details
This section introduces the technical details of the models and their optimization.
- The section covers essential technical details of the models and how they are optimized.
1. Architecture
The implementation uses a row-wise autoregressive ConvNet with real parameters, representing each complex conditional log-amplitude through its log-magnitude and phase. Its vertical and horizontal branches encode dependencies from particles above and to the left in the two-dimensional lattice.
- The architecture uses real parameters, with each complex conditional log-amplitude represented by two real values: log-magnitude and phase.
- A vertical branch represents conditional dependencies between each particle and all particles above it in the 2D lattice.
- A horizontal branch represents conditional dependencies between each particle and all particles to its left.
- The two branches each comprise sequences of convolutional layers.
2. Handling Symmetries
The model incorporates problem symmetries through a self-ensemble that transforms configurations, evaluates them with the model, and aggregates the resulting log-amplitudes. The aggregation must preserve probabilistic meaning so efficient sampling remains possible.
- The self-ensemble symmetrizes model f(s) by applying the problem’s transformations to each input spin configuration and aggregating the resulting outputs.
- The generative symmetrization samples a transformation uniformly, draws a model sample, and then applies that transformation to the sample.
- This procedure yields a mixture model over the squared magnitudes of the network’s predicted amplitudes.
3. Optimization
Optimization proceeds in three stages, beginning with Adam and small batches, then increasing the batch size, and finally switching to momentum SGD with a larger batch.
- After about 10K gradient updates, the batch size increases to 1000 while retaining Adam and the same learning rate for another 10K updates.
- The final stage increases the batch size again and switches to standard SGD with momentum for an additional 5K updates.
- Experiments test multiple variations around the default batch sizes, learning rate, and update counts.