Source-linked AI summary
Symmetries and many-body excited states with neural-network quantum states
Kenny Choo, Giuseppe Carleo, Nicolas Regnault, Titus Neupert
TL;DR
The paper addresses optimization challenges for excited-state wavefunctions, where Monte Carlo estimates can be noisy and unstable. It applies stochastic reconfiguration with regularization and parallel tempering, finding faster convergence and polynomial scaling relative to exact diagonalization.
Problem
Monte Carlo estimates and poorly conditioned matrices can make excited-state optimization noisy and produce large errors.
Method
The approach uses stochastic reconfiguration, regularization, conjugate gradients, and parallel tempering with multiple temperature chains for Monte Carlo sampling.
Results
Stochastic reconfiguration is more stable and may converge in fewer iterations, while RBM computational cost scales polynomially rather than exponentially with system size as in exact diagonalization.
Takeaways & Limitations
Neural-network wavefunctions can avoid the exponential scaling of exact diagonalization, although their computational cost is higher for the system sizes examined.
Takeaways & Limitations
Standard Metropolis-Hastings can yield energies that differ significantly from parallel tempering or exact results; sufficient replicas have produced close energies.
Abstract
from arXiv · showhide
Artificial neural networks have been recently introduced as a general ansatz to compactly represent many- body wave functions. In conjunction with Variational Monte Carlo, this ansatz has been applied to find Hamil- tonian ground states and their energies. Here we provide extensions of this method to study properties of ex- cited states, a central task in several many-body quantum calculations. First, we give a prescription that allows to target eigenstates of a (nonlocal) symmetry of the Hamiltonian. Second, we give an algorithm that allows to compute low-lying excited states without symmetries. We demonstrate our approach with both Restricted Boltzmann machines states and feedforward neural networks as variational wave-functions. Results are shown for the one-dimensional spin-1/2 Heisenberg model, and for the one-dimensional Bose-Hubbard model. When comparing to available exact results, we obtain good agreement for a large range of excited-states energies. Interestingly, we also find that deep networks typically outperform shallow architectures for high-energy states.
Stochastic reconfiguration
Stochastic reconfiguration optimizes variational wavefunctions by approximating imaginary-time evolution, making it especially useful for excited-state searches. It is more computationally expensive per iteration than gradient descent but can converge in fewer iterations.
- Stochastic reconfiguration: SR optimizes variational parameters by approximating imaginary-time evolution toward a lower-energy wavefunction.The method expands parameter changes, uses logarithmic derivatives, and selects the update minimizing the distance to the imaginary-time-evolved state.
- Stochastic reconfiguration: The SR update is obtained by solving a linear system for the parameter displacement δα, then iterating until convergence.The parameters are updated as αk = α0_k + δαk after solving the linear equation.
- Stochastic reconfiguration: Each SR iteration costs O(Nw^3), compared with O(Nw) for gradient descent, where Nw is the number of variational parameters.The higher per-step cost comes from matrix inversion.
- Stochastic reconfiguration: SR is more stable than standard gradient descent and may require substantially fewer iterations to converge.The authors suggest that faster convergence can offset the larger computational cost per iteration.
Regularisation
Regularisation stabilizes the noisy SR linear solve by modifying its matrix before inversion. The paper uses an identity shift and conjugate gradients, while large regularisation approaches gradient descent.
- Regularisation: Monte Carlo estimates make the SR matrices noisy, and a large condition number can amplify small noise into large errors.This motivates regularisation when solving the inverse problem.
- Regularisation: Regularisation replaces A with à = A + λI and solves the modified system using conjugate gradients.The procedure is described as similar to Tikhonov or Ridge regularisation.
- Regularisation: The simulations typically choose λ ∈ [0.001, 0.01].This is the reported regularisation range for the article's simulations.
- Regularisation: For λ much larger than A, the SR step reduces to standard stochastic gradient descent with learning rate ϵ.Large regularisation changes the update into the gradient-descent limit.
Monte Carlo Sampling
The simulations use Metropolis-Hastings sampling, with parallel tempering added because excited-state wavefunctions are generally not smooth. Replica exchanges improve energy estimates when sufficient replicas are used, although temperature selection affects efficiency.
- Monte Carlo Sampling: Metropolis-Hastings generates a Markov chain by proposing configurations and accepting them according to a wavefunction-probability ratio.Accepted configurations form a chain that approaches samples from the target distribution.
- Monte Carlo Sampling: Samples are formed by taking configurations separated by a spacing ideally larger than the chain's correlation time.The simulations use between 1000 and 10000 samples, at least as many as the network's free parameters.
- Monte Carlo Sampling: Parallel tempering runs N chains at different inverse temperatures and exchanges neighboring configurations during sampling.The target distribution corresponds to β = 1, while other temperatures help sample excited-state distributions.
- Monte Carlo Sampling: The authors use parallel tempering because excited-state wavefunctions are generally not smooth.The method replaces the usual acceptance factor with a temperature-dependent factor and adds replica exchanges.
- Monte Carlo Sampling: Standard Metropolis-Hastings can produce energies that differ significantly from parallel tempering or exact values, whereas sufficient replicas yield energies close to exact results.The simulations typically use 100 to 200 replicas when parallel tempering is necessary.
- Monte Carlo Sampling: Equally spaced temperature points are known to be far from optimal, so temperature-set choice remains an efficiency limitation.Adaptive methods for optimizing the temperature set are noted but not used in the implementation described.
Summary of Optimisation Parameters
The simulations tune regularisation, learning rate, parallel-tempering replicas, Monte Carlo sample count, and initialization scale across the reported studies.
- Summary of Optimisation Parameters: The main optimization parameters are λ, ϵ, the number of parallel-tempering replicas, the number of Monte Carlo samples, and initialization standard deviation σ.The parameter choices for individual simulations are provided in a table.
Computational Time: ANN vs ED
For the tested system sizes, RBM calculations take more time and resources than exact diagonalization, but their scaling is polynomial rather than exponential.
- Computational Time: ANN vs ED: RBM generally requires more computational time and resources than ED for the presented system sizes.The comparison uses whole-process wall time under the specified sampling, iteration, and multicore settings.
- Computational Time: ANN vs ED: ED time scales exponentially with system size, whereas RBM time scales polynomially.
- Computational Time: ANN vs ED: An FFNN with a similar number of parameters to the RBM generally takes a similar amount of time.
Convergence
The paper illustrates convergence of its optimization schemes using representative excited-state calculations for the Heisenberg and Bose-Hubbard models.
- Convergence: Figure 4 shows typical convergence properties for the two optimization schemes introduced in the main text.
- Convergence: For the Heisenberg model, the example targets the first excited state at L = 28 using a two-step method with the sum of two RBMs.
- Convergence: For the Bose-Hubbard model, the example uses N = 10 bosons on L = 10 sites at momentum k = 8π/10 with a two-layer FFNN.
- Convergence: The red curve averages energy over the previous 100 iterations, while dashed lines mark exact values.