PaperlayerView exact revision on arXiv

Source-linked AI summary

Accurate Sampling from Diffusion Models

Dénes Sexty

arXiv:2609.29902v1hep-latcond-mat.dis-nncs.LG

TL;DR

MCMC can be costly for lattice simulations, and diffusion models trained from target samples still approximate rather than reproduce the target distribution. DM-SMC combines backward diffusion with sequential Monte Carlo weighting to sample accurately despite approximate scores and finite stepsizes, while an improved update offers a stepsize advantage with a large-lattice tradeoff.

  • Problem

    MCMC can be costly near phase transitions, and neural-network score approximations make diffusion-generated configurations only approximately distributed as the target measure.

    Source: §1 · p. 1, §4 · p. 4

  • Method

    DM-SMC uses sequential Monte Carlo to track weighted particles through known distributions built into backward diffusion, ending at the target measure.

    Source: §4 · p. 4, §7 · p. 10

  • Results

    DM-SMC supports accurate sampling despite approximate diffusion models and finite stepsizes, with results consistent with exact results for better fits or smaller stepsizes.

    Source: §6 · p. 9, §7 · p. 10

  • Takeaways & Limitations

    The improved update permits larger denoising steps, but large lattices require approximate weight updates and extrapolation to zero stepsize.

    Source: §7 · p. 10

    For large lattice models, approximate weight updates for the improved method require extrapolation to zero stepsize, and the cheaper update choice is application-dependent.

    Source: §5 · p. 5

Abstract

from arXiv · show

A new proposal called DM-SMC (Diffusion Model - Sequential Monte Carlo) is investigated, which samples ensembles defined in terms of an action, using diffusion models trained on samples from the ensemble. The SMC setup allows for accurate sampling in spite of an approximate diffusion model and the finite stepsize used in the numerical solution of the stochastic process. Improved update strategies are also investigated. Results are presented for a $Z_2$ symmetric scalar field theory in 2 dimensions near its 2nd order phase transition.

1 Introduction

Lattice Monte Carlo estimates observables when analytical treatment is impractical, but simulations can be costly near phase transitions. The introduction reviews generative models and diffusion models as approaches to lattice field theory sampling.

1 Introduction

  • Lattice discretisation reduces the path integral to finitely many integrals, and Monte Carlo simulations estimate averages of observables when analytical treatment is impractical.Source: §1 · p. 1
  • MCMC can be computationally demanding near the continuum limit and physical quark masses, while critical slowing down also affects lattice simulations around phase transitions.The introduction motivates machine learning as a way to reduce lattice-simulation costs.Source: §1 · p. 1
  • Generative models have been explored for field-theory sampling, but training costs can rise rapidly with system size and models can suffer from mode collapse.The passage also notes GAN applications and flow-based methods for lattice theories.Source: §1 · p. 1
  • Diffusion models are presented as an alternative, with prior applications to lattice QFTs, gauge theories, and the Complex Langevin equation.The cited work also studied their connection to path integrals and cumulants.Source: §1 · p. 1

2 Diffusion models

Diffusion models connect a target ensemble to a tractable one through a forward noising process and a reverse process. The reverse process can generate target samples by using a neural network to approximate its score.

2 Diffusion models

  • The forward process adds noise to target configurations, with field evolution described by a Langevin equation and its time-dependent probability density by a Fokker–Planck equation.Source: §2 · pp. 1–2
  • With vanishing drift, field variance grows until the distribution is well approximated by Gaussians, while the backward evolution is itself a diffusion process.Source: §2 · p. 2
  • Target samples can be generated by starting from the easy distribution and solving the backward process with a neural-network-fitted score in place of the exact score.The score fit is trained using samples from the target ensemble and the forward diffusion process.Source: §2 · p. 2

3 Sequential Monte Carlo and diffusion processes

Sequential Monte Carlo estimates averages by moving weighted particles through a sequence of distributions from an easy initial measure to a target. Applied to discretised diffusion, it tracks weights that preserve accurate averages even when finite-step updates reduce effective sample size.

3 Sequential Monte Carlo and diffusion processes

  • SMC moves a cloud of particles from an easy initial distribution through intermediate measures toward the target, using particle weights to estimate averages.Source: §3 · p. 2
  • Weight-only updates preserve correct averages but can produce high variance when a few particles dominate, so effective sample size (ESS) monitors update quality.ESS is M when all weights are equal and approaches 1 when one weight dominates.Source: §3 · pp. 2–3
  • The backward kernel can be chosen freely subject to the SMC condition, and the practical choice should minimize weight variance or maximize ESS.Source: §3 · p. 3
  • Resampling when ESS falls selects particles according to their weights and resets weights equally, but can introduce correlations that MCMC rejuvenation may remove.Source: §3 · p. 3
  • Random normalized weights introduce an O(1/M) bias in the average estimator.The passage expresses this as calculating ⟨A/B⟩ instead of ⟨A⟩/⟨B⟩.Source: §3 · p. 3
  • A discretised diffusion process supplies the SMC sequence through its time-dependent distributions.Source: §3 · p. 3
  • Using the backward diffusion kernel makes the weight update optimal and maximizes ESS, while finite-step weights still keep averages accurate.The weight identity is exact only as the stepsize tends to zero; the finite-step result assumes a linear or zero drift.Source: §3 · pp. 3–4
  • Euler–Maruyama makes forward and backward kernels easy to calculate, whereas improved updates can require numerically expensive Jacobian determinants.Source: §3 · pp. 3–4

4 SMC aided Diffusion models

DM-SMC combines backward diffusion with sequential Monte Carlo to target exact action-weighted averages despite an approximate learned drift and finite numerical steps. It tracks weights across a sequence ending at the target measure, correcting these errors at typically negligible cost relative to field updates.

4 SMC aided Diffusion models

  • DM-SMC uses sequential Monte Carlo over distributions along backward diffusion, ending with the target measure e^-S(φ), to estimate exact action-weighted averages.The intermediate distributions are represented by an energy learned by the neural network.Source: §4 · p. 4
  • The initial distribution G(φ) is Gaussian for non-compact manifolds and uniform for compact ones; finite noise, stepsize effects, and model error increase weight variance and reduce ESS.More noise, smaller stepsizes or improved updates, and better network fits can reduce these effects; otherwise result variance increases.Source: §4 · p. 4
  • Weight tracking corrects finite-stepsize effects and an inexact neural-network approximation, keeping the target distribution exact.The associated weight-evolution costs are typically negligible compared with updating fields.Source: §4 · p. 5
  • DM-SMC requires parallel generation of configurations, weight tracking, ESS monitoring, and resampling when ESS falls too far.Apart from ESS calculation and resampling, generation threads do not communicate.Source: §4 · p. 5
  • A MALA accept-reject step after each Langevin update preserves exact distributions despite finite Langevin steps.The rejuvenation process uses backward diffusion with its drift halved and the time argument held fixed.Source: §4 · p. 5

5 Simulation setup

The simulations test DM-SMC for a two-dimensional scalar field near its Z2-symmetry-breaking transition. The setup compares Langevin updates and uses an energy-based neural network to represent distributions during denoising.

Simulation setup

  • The first-order Euler–Maruyama update is correct to first order in Δτ; a weak order-2 alternative is implicit and requires iterations to solve.Its kernel involves the drift Jacobian, whose determinant is prohibitively expensive for lattice models except at small volumes.Source: §5 · p. 5
  • The second-order update’s trace approximation reintroduces Δt dependence and requires extrapolation to Δt → 0.Which update is cheaper—the first-order method with exact weights or second-order method with approximate weights—depends on the application.Source: §5 · p. 5
  • Simulations use a two-dimensional scalar field at κ = 0.27 and λ = 0.022, near the second-order transition where magnetization has a double-peaked histogram.The authors select this region because phase-transition regions are typically difficult for MCMC and generative methods.Source: §5 · p. 6
  • The energy-based network outputs R(φ, τ), with field-configuration and time inputs, so the denoising distribution is represented as exp(−R(φ, τ)).The convolutional architecture respects lattice translations and rotations or mirrorings in its output; invariance of its kernels is not enforced.Source: §5 · p. 6
  • Training combines a score-fitting loss with a term that makes the denoising energy at τ = 0 coincide with the target action.Training examples pair target-ensemble configurations with random times and Gaussian noise used to construct noised fields.Source: §5 · pp. 6–7

6 Results

DM-SMC histograms agree with HMC benchmarks within statistical errors on both the 4x4 and larger lattice. Improved denoising updates slow ESS decay, while denoising without SMC remains inexact.

6 Results

  • On 4x2 lattices without resampling, reducing the backward-process stepsize increases ESS only up to a limit set by neural-network fit quality.Even a nearly flat ESS curve requires a quite small stepsize.Source: §6 · p. 7
  • Without SMC, both denoising models produce inexact magnetization histograms, although the better-fitted model with improved updates gives a better approximation.The comparison uses Nτ = 200 steps for each model.Source: §6 · p. 7, Figure 2 · p. 8
  • On the 4x4 lattice, DM-SMC agrees with HMC within statistical errors despite using an inaccurately fitted model and 200 first-order Langevin steps.Only one resampling is used at the end; ESS/M averages 0.0716 ± 0.0006.Source: §6 · pp. 7, 9, Figure 3 · p. 8
  • With an improved update, ESS decays much more slowly; the exact-Jacobian method stays above ESS/M = 0.5 at Δτ = 0.1, while the four-noise approximation also keeps decay slow.The approximated update is biased at finite stepsizes and requires extrapolation to zero stepsize.Source: §6 · p. 9, Figure 4 · p. 9, Figure 5 · p. 9
  • On the larger lattice, DM-SMC magnetization and action-density histograms agree with HMC within statistical errors.The reported setup uses 10–11 resamplings and averages 140 independent runs, each with M = 8192.Source: Figure 6 · p. 10, §6 · p. 10

7 Conclusions

DM-SMC produces histograms and averages consistent with exact results near the critical point, even when the diffusion model is a relatively poor approximation. A second-order denoising update permits larger stepsizes, with an approximation and zero-stepsize extrapolation needed for large lattices.

7 Conclusions

  • Histograms and averages agree with exact results within statistical errors near the critical point, even with a relatively poor diffusion-process approximation.The results are exact up to O(1/M) corrections, which are overwhelmed by O(1/sqrt(M)) statistical errors.Source: §7 · p. 10
  • DM-SMC tracks a weighted particle cloud through denoising, using effective sample size to monitor quality and resampling to rejuvenate the sample when needed.Source: §7 · p. 10
  • The improved denoising update is correct to second order and permits larger stepsizes, but large lattices require approximate weight updates and extrapolation to zero stepsize.Source: §7 · p. 10
  • The method adds relatively low simulation costs and targets distributions where action calculations are cheap but constructing useful MCMC proposals is costly.The paper gives QCD as an example of this longer-term goal.Source: §7 · p. 10
Loading 2609.29902v1…