PaperlayerView exact revision on arXiv

Source-linked AI summary

Flow-based generative models for Markov chain Monte Carlo in lattice field theory

M. S. Albergo, G. Kanwar, P. E. Shanahan

arXiv:1904.12072v3hep-latcond-mat.dis-nncond-mat.stat-mechcs.LG

TL;DR

The paper addresses correlated MCMC sampling in lattice field theories, where critical slowing down increases autocorrelation times. It trains a flow-based generative model to approximate the target distribution, then uses independent proposals with Metropolis-Hastings correction. Training systematically improves autocorrelation properties, and the model can be trained from its own samples without existing target-distribution data.

  • Problem

    MCMC sampling in lattice field theory suffers from correlations and critical slowing down, which increases computational cost near critical points.

  • Method

    A flow-based generative model is trained to approximate the target distribution and supplies independent proposals for a Metropolis-Hastings Markov chain.

  • Results

    Training systematically improves autocorrelation properties, including in parameter regions where standard algorithms exhibit critical slowing down.

  • Takeaways & Limitations

    The approach supports exact corrected sampling with decorrelation determined by flow accuracy and can train without existing samples from the desired distribution.

Abstract

from arXiv · show

A Markov chain update scheme using a machine-learned flow-based generative model is proposed for Monte Carlo sampling in lattice field theories. The generative model may be optimized (trained) to produce samples from a distribution approximating the desired Boltzmann distribution determined by the lattice action of the theory being studied. Training the model systematically improves autocorrelation times in the Markov chain, even in regions of parameter space where standard Markov chain Monte Carlo algorithms exhibit critical slowing down in producing decorrelated updates. Moreover, the model may be trained without existing samples from the desired distribution. The algorithm is compared with HMC and local Metropolis sampling for $φ^4$ theory in two dimensions.

I. INTRODUCTION

The paper proposes flow-based MCMC for lattice field generation to address autocorrelations and critical slowing down. The approach trains a generative model without requiring existing target-distribution samples and uses independent proposals in an exacting Markov chain.

  • Motivation: Path-integral calculations commonly use MCMC, but correlations between configurations cause autocorrelation times to diverge near critical points.This critical slowing down substantially increases simulation costs in those parameter regions.
  • Approach: The proposed method transforms samples from a simple distribution into field configurations from a trained approximate target distribution.The flow is optimized within a variational family to make its effective distribution close to the desired one.
  • Approach: Metropolis-Hastings correction makes the flow proposals asymptotically exact while their independence from previous states couples acceptance rate to autocorrelation time.As acceptance approaches 1, the autocorrelation time approaches zero under the stated construction.
  • Benefits: Training can systematically decrease the Markov-chain autocorrelation time.The paper identifies model training as a way to improve sampling quality before chain generation.
  • Benefits: Independent proposals can be generated in parallel, while each chain step requires model evaluation and action computation.These properties support efficient composition of proposals into a Markov chain.
  • Benefits: The model trains on samples it produces itself, avoiding the need for existing samples from the desired probability distribution.The paper positions this self-training alongside prior machine-learning approaches to MCMC and lattice systems.

A. Metropolis-Hastings with generative models

The algorithm uses a normalizing flow to construct an independently sampled proposal distribution and corrects it with Metropolis-Hastings. Tractable likelihoods and self-training make the proposal both evaluable and improvable without pre-existing target samples.

  • Metropolis-Hastings with generative models: The independence Metropolis sampler proposes each new configuration from a known distribution independently of the previous chain state.Accepted proposals replace the current configuration; rejected proposals leave it unchanged.
  • Metropolis-Hastings with generative models: Metropolis-Hastings preserves balance for arbitrary proposal schemes, while nonzero proposal and target densities ensure ergodic convergence.Under these conditions, the chain converges to the desired distribution.
  • Normalizing flows: A normalizing flow maps samples from a simple prior r(z) to an approximate target distribution ˜pf(φ) through a smooth bijection.The flow's exact density is obtained by the change-of-variables formula.
  • Normalizing flows: Real NVP constructs the flow from affine coupling layers that transform alternating halves of the input while retaining invertibility and a tractable Jacobian.Composing more expressive layers systematically increases the flexibility of the output distribution.
  • Training: The shifted KL loss trains the flow distribution toward p(φ)=e^-S(φ)/Z without requiring computation of the partition function.Its minimum is achieved exactly when ˜pf=p.
  • Algorithm: Self-training draws batches from the flow itself, after which generated proposals are corrected with Metropolis-Hastings using their known likelihoods.If target samples exist, they can instead be used for pre-training.

C. Autocorrelation time for generative Metropolis-Hastings

For independence Metropolis sampling, rejection streaks determine autocorrelation at each chain separation, while average acceptance provides a bound and practical training metric.

  • Autocorrelation at separation τ equals the probability of τ consecutive rejections for generative Metropolis-Hastings chains.
  • A finite-sample estimator based on rejection indicators approximates the rejection-run probability for near-equilibrium chains.
  • The rejection-based autocorrelation measure is consistent with the usual observable-specific definition in the infinite-chain limit.
  • Increasing average acceptance is necessary to reduce autocorrelations, and matching proposal and target distributions yields zero autocorrelation.
  • The paper recommends evaluating autocorrelations alongside training loss because improving distributional distance is expected to reduce autocorrelations only on average.

D. Critical slowing down

Critical slowing down is characterized by divergent autocorrelation times near criticality. In the proposed approach, sampling CSD can be removed through training, but its difficulty shifts to proposal optimization.

  • Critical slowing down is the divergence of Markov-chain autocorrelation time as a critical point is approached.
  • Standard local-update algorithms typically exhibit power-law autocorrelation growth with lattice size, described by the dynamical critical exponent z_O.
  • An update algorithm with z_O = 0 is unaffected by critical slowing down.
  • Training to a target integrated autocorrelation time can trivially remove sampling CSD, at the expense of upfront model-training costs.
  • The resulting difficulty is shifted from sampling to optimizing the proposal distribution, whose cost and scaling are discussed for φ4 theory and future applications.

III. APPLICATION OF FLOW-BASED MCMC TO φ4 THEORY

The paper applies flow-based MCMC to two-dimensional lattice φ4 theory, a simple interacting field theory with symmetric and broken-symmetry phases, and studies standard observables and rejection behavior.

  • Two-dimensional φ4 theory is used as a testing ground for the proposed flow-based MCMC algorithm.
  • The discretized theory is defined on a finite periodic lattice with bare mass squared m2 and bare coupling λ.
  • Observables are estimated by expectation values over the Boltzmann distribution p(φ) = e^-S(φ)/Z.
  • The study considers connected two-point Green’s functions, their momentum-space representation, the corresponding pole mass, and the two-point susceptibility.
  • In the λ → ∞ limit with m2/λ < 0 fixed, scalar φ4 theory reduces to an Ising model, motivating Ising energy density as an additional observable.
  • The φ4 action has a discrete φ(x) → −φ(x) symmetry and supports symmetric and broken-symmetry phases depending on m2 and λ.

A. Model definition and training

The numerical study trains flow models for two-dimensional symmetric-phase φ4 ensembles and compares them with local Metropolis and HMC sampling at matched parameters.

  • The proof-of-principle study uses L = {6, 8, 10, 12, 14} two-dimensional lattices and chooses parameters to maintain m_pL ≈ 4.
  • All studied parameters lie in the symmetric phase; training models for the broken-symmetry phase remains to be shown.
  • Real NVP models use 8–12 affine coupling layers with alternating checkerboard updates of lattice sites.
  • Models are trained with gradient-based Adam updates to minimize shifted KL loss between the flow output distribution and the desired Boltzmann distribution.
  • For parameter set E5, Green’s functions and effective pole masses from the three methods agree within statistical errors using 10^6 configurations.
  • The proof-of-principle study does not exhaustively optimize the prior, depth, architecture, initialization, or coupling-layer choices, and training cost affects the optimal acceptance rate.
  • Training targets both 50% and 70% mean Metropolis acceptance; higher-target models produce shorter consecutive rejection runs.
  • At matched parameters, ensembles of 10^6 configurations are generated with flow-based MCMC, local Metropolis, and HMC for comparison.

B. Tests: physical observables and error scaling

The flow-based MCMC ensembles reproduce physical observables obtained with HMC and local Metropolis within statistical uncertainties. Their observable uncertainties decrease as 1/N, consistent with decorrelated sampling.

  • Physical observables: Flow-based MCMC observables agree with HMC and local Metropolis results within statistical uncertainties.The comparison covers physical observables computed from ensembles generated by all three methods.
  • Physical observables: Susceptibility χ2 and Ising energy E from 10^6 configurations are consistent across HMC, local Metropolis, and ML ensembles.The uncertainties are 68% confidence intervals estimated by bootstrap resampling with bins of size 100.
  • Error scaling: Observable statistical uncertainties scale as 1/N with sample number N, as expected for decorrelated samples.This behavior is shown for the candidate observables χ2 and E.

C. Critical slowing down

The study examines critical slowing down by comparing autocorrelation scaling for standard samplers and flow-based MCMC. Flow-based MCMC maintains lattice-size-independent autocorrelation at fixed acceptance, while training and acceptance-related variation remain relevant considerations.

  • Autocorrelation scaling: HMC and local Metropolis exhibit power-law growth of integrated autocorrelation time with lattice size.The study uses leading-order power-law fits to determine dynamical critical exponents for physical observables.
  • Autocorrelation scaling: Flow-based MCMC has a critical exponent consistent with zero at fixed acceptance, with observable-independent autocorrelation time.The result agrees with the acceptance-based estimator defined in the method section.
  • Comparison caveat: The absolute integrated autocorrelation times are not directly comparable across methods because their update costs differ.Lattice-size scaling, rather than absolute values, indicates each method’s sensitivity to lattice size.
  • Acceptance-rate dependence: Higher mean acceptance rates reduce flow-based MCMC integrated autocorrelation times and produce lower values in the lattice-size scaling plot.Figure 7 compares models trained to 50% and 70% mean Metropolis acceptance.

D. Training costs

Flow-based MCMC shifts computational effort into model training, whose cost is amortized over sufficiently large ensembles. The proof-of-principle architecture has unfavorable volume scaling, while convolutional and multi-scale designs are identified as routes toward improved scaling.

  • Amortization: Training adds an up-front cost, making computational advantage depend on training time for finite target ensembles.The cost is amortized over the ensemble and becomes advantageous naturally when generating many samples.
  • Measured cost: All models required one to two GPU-weeks of training, with larger lattices incurring greater computational cost.The largest lattice also used roughly 10× as many training epochs as the smallest.
  • Architecture scaling: The fully connected architecture scales as O(V^2) in both sampling and training time because of dense matrix-vector multiplications.This asymptotic behavior is attributed to the simple architecture used in the proof-of-principle study.
  • Potential improvements: Convolutional and multi-scale architectures reduced costs in related image-generation methods and improved scaling to O(V).These architectures exploit locality, translational invariance, and coarse-to-fine structure relevant to lattice field theories.
  • Parameter reuse: Models trained at one parameter set can initialize or serve as priors for models targeting nearby parameter values, further amortizing training.This addresses applications requiring ensembles across multiple parameter choices and parameter tuning.

IV. SUMMARY

The flow-based MCMC algorithm trains a real NVP model to propose samples, then uses Metropolis-Hastings to retain asymptotic exactness. Its sampling-stage autocorrelation is governed by model accuracy rather than critical slowing down, while scaling to QCD remains an open challenge.

  • IV. SUMMARY: The algorithm trains a real NVP flow to approximate the desired distribution, proposes samples from it, and accepts or rejects them with Metropolis-Hastings.The resulting chain is ergodic and balanced, guaranteeing convergence to the desired distribution in the long-chain limit.
  • IV. SUMMARY: The method combines neural-network normalizing flows with the theoretical guarantees of Markov chains for continuous, real-valued field configurations.This combination provides a trainable and asymptotically correct sampler.
  • IV. SUMMARY: Training systematically decreases the Markov-chain autocorrelation time, with perfect training corresponding to decorrelated samples and 100% acceptance.The autocorrelation time for all observables is dictated by the accuracy of the trained flow model.
  • IV. SUMMARY: Models trained to a fixed acceptance rate do not experience critical slowing down during the sampling stage.Independent proposals make the autocorrelation time fall toward zero as the acceptance rate approaches one.
  • IV. SUMMARY: Applying the approach to larger-dimensional theories and QCD requires efficient scaling and flow models expressive on compact gauge manifolds.The prior must also be extended to a distribution over the gauge-configuration manifold; a uniform prior is a candidate requiring testing.
  • IV. SUMMARY: If implemented for QCD, independent proposals could support parallel generation of arbitrarily large ensembles at minimal cost.A trained model could also be retrained or reused as a prior for nearby action parameters.

Appendix A: Acceptance rate estimator for autocorrelation

The appendix shows that the standard autocorrelation estimator converges to the proposal rejection probability under a thermalized chain. It also describes the MAE training loss, including its dependence on the partition function and its use before shifted-KL training.

  • Appendix A: Acceptance rate estimator for autocorrelation: The standard estimator for an observable's autocorrelation converges to pτrej as the path length approaches infinity.This establishes the acceptance-rate-based autocorrelation result claimed in Section II C.
  • Appendix A: Acceptance rate estimator for autocorrelation: The estimator is defined from products of centered observable values separated by lag τ, normalized by the centered-observable variance.The expectation is taken in the infinite-path-length limit.
  • Appendix A: Acceptance rate estimator for autocorrelation: The derivation assumes the Markov chain starts from the stationary distribution after sufficient iterations have been discarded for thermalization.The expectation is then split according to whether proposals in the interval are all rejected or at least one is accepted.
  • Appendix A: Acceptance rate estimator for autocorrelation: Taking N →∞ removes biases caused by conditioning on behavior within the interval [i, i + τ].This limiting step supports the asymptotic estimator relation.
  • Appendix A: Acceptance rate estimator for autocorrelation: The MAE loss is bounded below by D_KL(˜p_f||p) and shares its global minima when ˜p_f = p.In practice, the loss is estimated from batches of samples generated by the model.
  • Appendix A: Acceptance rate estimator for autocorrelation: The MAE loss requires an estimated or trainable partition function, and this study fixed its value using a multistage estimation method.Errors in log Z can shift the loss minimum away from the desired distribution, so the loss was used only before shifted-KL training.
Loading 1904.12072v3…