Source-linked AI summary
Solving Statistical Mechanics Using Variational Autoregressive Networks
Dian Wu, Lei Wang, Pan Zhang
TL;DR
Finite-size statistical-mechanics calculations need tractable alternatives to conventional mean-field estimation, especially when exact inverse-Ising observables are computationally intractable. The paper uses autoregressive variational networks trained with policy gradients, showing convergence toward true free energy while directly sampling configurations.
Problem
Exact magnetizations and correlations in inverse Ising reconstruction are computationally intractable, motivating efficient estimation from trained models.
Source: Inverse Ising problem and conventional… · p. 8Method
Autoregressive variational networks compute normalized configuration probabilities and use REINFORCE policy gradients to optimize variational free energy.
Source: Autoregressive networks · p. 7, Gradient estimator and variance… · p. 7Results
During SK-model training, decreasing variance coincides with variational free energy converging to the true free energy, while larger networks attain the lowest reported errors.
Source: Zero variance condition and exact free… · p. 8, Details on network structure and… · p. 11Takeaways & Limitations
The framework supports direct sampling for estimating magnetizations and correlations and can outperform previous mean-field methods with smaller, faster-training networks.
Source: Inverse Ising problem and conventional… · p. 8, Details on network structure and… · p. 11Mode collapse can prevent the learned normalization eβC from equaling the original Boltzmann normalization 1/Z.
Source: Zero variance condition and exact free… · p. 8
Abstract
from arXiv · showhide
We propose a general framework for solving statistical mechanics of systems with finite size. The approach extends the celebrated variational mean-field approaches using autoregressive neural networks, which support direct sampling and exact calculation of normalized probability of configurations. It computes variational free energy, estimates physical quantities such as entropy, magnetizations and correlations, and generates uncorrelated samples all at once. Training of the network employs the policy gradient approach in reinforcement learning, which unbiasedly estimates the gradient of variational parameters. We apply our approach to several classic systems, including 2D Ising models, the Hopfield model, the Sherrington-Kirkpatrick model, and the inverse Ising model, for demonstrating its advantages over existing variational mean-field methods. Our approach sheds light on solving statistical physics problems using modern deep generative neural networks.
Autoregressive networks
Autoregressive networks factorize a configuration’s joint probability into conditional probabilities, enabling exact normalized probabilities and direct sampling. Neural networks approximate these conditionals with polynomially many parameters, supporting likelihood and variational free-energy calculations.
Autoregressive networks
- The model evaluates a spin configuration’s normalized probability by factorizing its joint distribution into conditional probabilities.For four variables, qθ(s) = p(s4 | s1, s2, s3)p(s3 | s1, s2)p(s2 | s1)p(s1).Source: Autoregressive networks · p. 7
- Neural networks approximate all conditional probabilities using a polynomial number of parameters, whereas exact storage can require exponentially many.The exponential requirement applies in the worst case.Source: Autoregressive networks · p. 7
- The exact log probability ln qθ(s) enables variational free-energy computation from unbiased samples of the model.The configuration probability qθ(s) is defined by the autoregressive network.Source: Autoregressive networks · p. 7
- Direct sampling proceeds sequentially from stored conditional probabilities, assigning each variable in turn according to its conditional distribution.For four variables, s1 is sampled from p(s1), followed by s2 from p(s2 | s1), then s3 and s4.Source: Autoregressive networks · p. 7
Gradient estimator and variance reduction
The method derives an unbiased score-function gradient estimator because the variational free-energy gradient cannot be computed directly. It uses REINFORCE weighting and an s-independent baseline to reduce gradient variance while preserving the objective.
Gradient estimator
- The variational free-energy gradient with respect to model parameters θ cannot be computed directly, so a proper gradient estimator is derived.The estimator is constructed from samples drawn from qθ(s) and the variational free energy computed from those samples.Source: Gradient estimator and variance… · p. 7
- Each sample contributes ∇θ ln qθ(s) weighted by reward R(s), which is the REINFORCE score-function gradient estimator.Large rewards cause the optimizer to reduce the probability of generating those configurations, thereby reducing variational free energy.Source: Gradient estimator and variance… · p. 7
Variance reduction
- Subtracting any s-independent constant from the estimator leaves its expectation unchanged and enables variance reduction.This baseline-based modification does not alter the expected gradient.Source: Gradient estimator and variance… · p. 7
- The baseline is set to b = E_s∼qθ(s)R(s), computed at each iteration as an estimate of the log partition function.The work uses this simplest baseline strategy.Source: Gradient estimator and variance… · p. 7
- Optimizing L is equivalent to optimizing variational free energy, while reducing variance caused by the magnitude gap between E(s) and ln qθ(s).The estimated log partition function is represented by ln ˜Z = b.Source: Gradient estimator and variance… · p. 8
Zero variance condition and exact free energy
The exact Boltzmann distribution is characterized by zero variance of the variational energy expression, while zero variance conversely identifies a Boltzmann distribution up to normalization. Mode collapse can invalidate this inference, so temperature annealing is used to make variance a practical indicator of closeness to the exact distribution and free energy.
Zero variance condition
- If qθ(s) = pBoltzmann(s), then E(s) + 1/β ln qθ(s) has zero variance.The variational free energy is estimated over the variational distribution.Source: Zero variance condition and exact free… · p. 8
- Conversely, zero variance implies that qθ is a Boltzmann distribution because the quantity in brackets must be constant.Solving the resulting constant-value equation yields the Boltzmann form.Source: Zero variance condition and exact free… · p. 8
Exact free energy
- Mode collapse can make e^βC differ from 1/Z because the network may fail to capture all pure-state modes.Thus, zero variance alone does not necessarily establish the original Boltzmann normalization.Source: Zero variance condition and exact free… · p. 8
- Temperature annealing is proposed to avoid mode collapse, enabling variance to indicate closeness between qθ and the exact distribution without knowing the latter.The paper cites results in Fig. 3 as evidence that mode collapse does not occur and uses Fig. 5 as an illustrative example.Source: Zero variance condition and exact free… · p. 8
Inverse Ising problem and conventional mean-field methods
The inverse Ising problem reconstructs fields and couplings from prescribed magnetizations and correlations, but exact moment calculation is computationally intractable. The proposed method instead estimates these quantities from configurations directly sampled by an autoregressive network and compares reconstruction against established mean-field approaches.
Problem formulation
- The inverse Ising problem reconstructs external fields {hi} and couplings {Jij} from given magnetizations {mi} and correlations {Cij}.The Ising model is the maximum-entropy model when the first and second moments are constrained.Source: Inverse Ising problem and conventional… · p. 8
- Maximum-likelihood learning adjusts fields and couplings using discrepancies between network-generated and target magnetizations and correlations.Matching both quantities provides the learning condition for the inferred parameters.Source: Inverse Ising problem and conventional… · p. 8
Variational autoregressive method
- Exact magnetizations and correlations are computationally intractable, so the method estimates them from configurations sampled directly by the trained autoregressive network.Direct sampling provides an efficient alternative to exact moment calculation.Source: Inverse Ising problem and conventional… · p. 8
Experimental setup
- For zero-field models, the task reduces to reconstructing couplings from correlations, while Z2 symmetry makes magnetizations exactly zero.The evaluation uses small systems with exact correlations obtained by enumerating all configurations to avoid measurement noise.Source: Inverse Ising problem and conventional… · p. 8
Baselines and evaluation
- Performance is characterized by reconstruction error between inferred couplings Jinfer and true couplings Jtrue, alongside comparisons with naïve mean-field, Sessak–Monasson, independent-pair, and Bethe methods.Bethe correlations can be computed using susceptibility propagation applied to belief propagation.Source: Inverse Ising problem and conventional… · pp. 8–9
Details on network structure and training for Ising models
For 16×16 Ising models, VAN performance depends on network depth and temperature, while training uses staged inverse-temperature annealing and stabilization techniques. The reported architectures achieve accurate free energies and observables, with shallow and deep networks requiring 1.8 and 5.4 seconds per training step, respectively.
Network structure
- The study compares shallow depth = 3 networks with deep depth = 6 residue-block networks, selecting the lower-free-energy result.Shallow networks work better at high temperature, whereas deep networks work better otherwise.Source: Details on network structure and… · p. 9
- Convolutional networks use width = 64, identified as the elbow point in relative free-energy error versus parameter count.The 2D Ising lattice size is 16×16, and convolutional and densely connected layers are tested.Source: Details on network structure and… · p. 9
Training
- Training minimizes variational free energy with Adam, beginning at β = 0 and gradually increasing β to avoid mode collapse.Gradient-norm clipping improves stability; training runs for 10,000 steps and reports the lowest 100-step moving average of free energy.Source: Details on network structure and… · p. 10
- Numerical experiments show that the variational free-energy result is insensitive to many hyperparameters across tested ranges.The reported hyperparameters and their insensitive ranges are listed in Table II.Source: Details on network structure and… · pp. 10–11
- 1.8 sec. and 5.4 sec. are the typical one-step training times for shallow and deep 16×16 Ising networks on one NVIDIA Titan V GPU.These larger networks produce the lowest relative errors for free energy and physical observables, including energy and heat capacity.Source: Details on network structure and… · p. 11
Details on network structure and training for SK model
The section specifies network architectures and training hyperparameters for the SK and inverse SK models, including batch size, learning rates, and layer widths.
SK model
- For the SK model, VAN uses 20-neuron input and output layers, batch size 10, 000, and learning rate 0.001.These settings are reported as the SK-model training configuration.Source: Details on network structure and… · p. 11
Inverse SK model
- The inverse SK model uses either 20-neuron input and output layers or an additional 100-neuron hidden layer.The two VAN architectures differ by the inclusion of the 100-neuron hidden layer.Source: Details on network structure and… · p. 11