PaperlayerView exact revision on arXiv

Source-linked AI summary

Exact Global MCMC with Denoising Diffusion

Mitch Hill

arXiv:2609.00279v1stat.MLcs.LG

TL;DR

Efficient exact sampling from complex, high-dimensional unnormalized densities remains difficult because local samplers struggle to move between modes and exact training samples are often unavailable. The paper trains standard denoising diffusion models on locally converged MALA samples, applies path-based MH correction, and composes the resulting global proposals with MALA in DDMC. Across diverse targets, DDMC achieves high-acceptance global proposals while adhering closely to target distributions, including a 550-dimensional posterior.

  • Problem

    Sampling unnormalized densities efficiently across multimodal state spaces is difficult because local samplers mix within modes but struggle to cross low-density barriers, while exact training samples are often unavailable.

  • Method

    DDMC trains a standard denoising diffusion model on locally converged MALA samples, then combines its global path proposals with local MALA and path-based MH correction.

  • Results

    Experiments across Gaussian mixtures, particle potentials, and a 550-dimensional Bayesian neural network posterior show high-acceptance global proposals that closely adhere to target distributions.

  • Takeaways & Limitations

    Denoising diffusion can serve as an exact global MCMC sampler while MH correction accommodates reverse-model error and mismatched MALA-corpus mode weights.

  • Takeaways & Limitations

    Learning fails if the MALA initialization cannot access a region containing concentrated target probability mass; addressing this remains future work.

Abstract

from arXiv · show

This work shows that diffusion models learned with standard denoising loss can provide effective global MCMC proposals for complex high-dimensional target densities. The method is motivated by the observation that sequentially applying a forward and reverse diffusion process defines a Markov chain with a target stationary distribution for an ideal denoiser trained on samples of the target distribution. This observation can be made exact for any denoiser by applying a Metropolis-Hastings step whose acceptance ratio includes the density of the forward and reverse paths of a discrete time SDE approximation. We therefore propose to train denoising diffusion models on locally convergent MALA samples to learn global MCMC proposals. We call the composition of the global denoiser-based path sampler and a local MALA sampler Denoising Diffusion Monte Carlo (DDMC). Experiments show that DDMC can provide global proposals with high acceptance across a variety of complex target densities. Our results offer preliminary evidence that the established scaling behavior of standard diffusion training transfers directly to exact sampling from high-dimensional unnormalized densities.

1 Introduction

The paper develops DDMC, combining denoising-diffusion global proposals with local MALA and an MH correction to obtain exact sampling despite model and training-corpus errors. Experiments support high-acceptance global sampling across complex targets, including a 550-dimensional posterior.

  • Method: Metropolis–Hastings correction makes the chain exact for any reverse-path model, although inaccurate models reduce acceptance.The correction uses the joint density of the discrete diffusion path and absorbs reverse-model and discretization error.
  • Method: DDMC composes a denoising diffusion path sampler with local MALA steps to propose global moves while retaining local correction.The path sampler can cross basins that local MALA cannot, while interleaved MALA updates address the training distribution.
  • Training rationale: Standard denoising regression avoids trajectory-gradient and memory limits of variational path-space objectives and uses clean–noisy sample pairs.Locally converged MALA chains provide inexpensive, parallel training data even when exact target samples are unavailable.
  • Experiments: Experiments cover Gaussian mixtures, particle potentials, and a 550-dimensional Bayesian neural network posterior.The evaluation includes exactness at measured ground-truth floors, recovery from biased training weights, and accurate modeling of the LJ-55 equilibrium.
  • Results: 0.13 to 0.28 acceptance rates were obtained for global proposals on the 550-dimensional posterior.A cold-started chain reached the equilibrium energy region within one or two proposal rounds.

2 Background

The background frames sampling from unnormalized densities as an MCMC problem, emphasizing MALA’s local-mixing limitation and denoising diffusion’s tractable forward and reverse path densities. These properties support exact MH-corrected diffusion proposals whose trajectory probabilities can be evaluated without marginal target densities.

  • Unnormalized densities: The target density is known only up to an intractable normalizing constant, while its unnormalized value and score are available.This setting includes Bayesian posteriors and Boltzmann distributions.
  • Metropolis–Hastings: Metropolis–Hastings corrects any proposal using only the unnormalized density and preserves the target as stationary.The proposal determines mixing speed, while the correction enforces detailed balance.
  • Local samplers: MALA combines one Euler–Maruyama Langevin step with MH correction, producing small moves suited to within-mode exploration.Moving between well-separated modes requires crossing low-density barriers through many local steps.
  • Denoising diffusion: Denoising diffusion trains a denoiser on noisy samples across a noise ladder, estimating the score of each smoothed density.The training procedure is simulation-free regression with one noise level per update.
  • Diffusion paths: A discrete SDE approximation defines forward and reverse Gaussian path densities over states x_0 through x_T.Continuous-time denoising training permits arbitrary noise schedules and numbers of steps.
  • Diffusion paths: Both path densities are exactly evaluable as products of Gaussian kernels using T denoiser evaluations, without computing q_σk marginals.This tractability enables MH correction on the joint diffusion path.

3 Method

The method turns standard denoising diffusion into an exact MCMC sampler by correcting discrete reverse paths with Metropolis–Hastings, then combines global diffusion moves with local MALA steps. DDMC trains the denoiser on locally converged MALA samples and extends path proposals with i-SIR for improved movement.

  • Diffusion as a global sampler: An ideal diffusion model defines an ergodic Markov chain with stationary distribution q, while large noise levels enable transitions across energy barriers between modes.The forward pass has full support, and the ideal reverse process transports qσ samples back to q.
  • Exact path correction: Metropolis–Hastings correction of the joint discrete diffusion path makes the sampler exact for any learned reverse model, although inaccurate models reduce acceptance.The correction preserves the forward path density and therefore the target marginal q at the path endpoint.
  • Training corpus: The denoiser is trained with standard denoising loss on locally converged MALA samples generated from the unnormalized density, without direct access to the target density.Misweighted modes can be corrected by MH, at the cost of acceptance, provided the misweighting is not too extreme.
  • Denoising Diffusion Monte Carlo: DDMC composes path-space MH moves driven by a denoiser with local MALA steps, allowing local refinement when global proposals are rejected.Both component kernels leave q invariant, so their composition also leaves q invariant; with zero path acceptance, DDMC reduces to plain MALA.
  • Training corpus: Training-corpus construction tunes MALA and producer procedures, but fails when initialization cannot reach regions containing target probability mass.The experiments initialize MALA from normal or uniform distributions spanning high-probability regions; mapping basin structure is suggested as future work.
  • Diagnostics: A held-out single-step path-proposal acceptance estimate diagnoses denoiser quality without requiring long MCMC rollouts.The diagnostic is intended to guide data-size, architecture, model-size, and training-step decisions.
  • Parallel proposals: i-SIR draws independent proposals from the shared top point, and its probability of leaving the current state is nondecreasing in K without changing the stationary distribution.This multiple-proposal extension exploits the proposal kernel’s independence from the current path.

4 Related Work

DDMC belongs to exact learned-proposal methods that correct diffusion or global moves with MCMC, while differing from variational and inner-Monte-Carlo approaches through denoising training on MALA samples.

  • Related exact path-space corrections: MAD-Path and SPS concurrently proposed related Metropolis–Hastings corrections, but train score models variationally from the target density rather than from MALA samples.The paper positions sample-based denoising training as its central distinction from these methods.
  • Variational diffusion samplers: Variational diffusion samplers derive training signals from their own trajectories, leaving unvisited regions without gradients; many also require memory linear in the number of noise levels.Matching-based objectives remove the memory cost but retain the on-policy training signal.
  • Denoising-based training: Denoising regression instead uses simulation-free per-level regression with data generated without limit from the unnormalized density.In this work, the data are MALA samples rather than exact target samples.
  • Diffusion samplers with inner Monte Carlo: Inner-Monte-Carlo diffusion samplers estimate scores or denoising quantities through replay buffers, inner sampling, or observation-ladder chains rather than applying exact corrections.The cited methods can therefore retain bias on complex targets; DiGS is identified as an exception with an exact correction.
  • Exact MCMC with learned global proposals: DDMC keeps MCMC exact while learning global moves, combining a diffusion path with the local–global pattern established by flow-based and i-SIR methods.It replaces a one-shot flow proposal with a diffusion path.

5 Experiments

Experiments evaluate DDMC on Gaussian mixtures, particle potentials, and a 550-dimensional BNN posterior, finding accurate target statistics and effective global proposals, while exposing corpus-initialization and method-comparison boundaries.

  • Experimental protocol: DDMC experiments use 10,000-step chains across four seeds, discard 400 burn-in steps, thin every fourth sample, and report acceptance, IACT, and Wasserstein-2 metrics.Evaluations use exact, literature, or parallel-tempering reference samples depending on the target.
  • 5.1 Mixture of Gaussians: In the 2D 40-component mixture, DDMC tests whether MH correction can recover correct mode weights from a corpus whose uniform initialization produces biased weights.Local MALA cannot traverse the modes in a realistic timeframe, making this a proof of concept for globally corrected proposals.
  • 5.1 Mixture of Gaussians: DDMC reaches the exact-draw E(·) W2 floor on both Gaussian-mixture targets and recovers ground-truth mode weighting within statistical uncertainty.All Table 1 entries lie within 2 sd of exact-draw controls at matched effective sample size.
  • 5.2 Particle Potentials: On particle potentials, DDMC energy distances sit at or within uncertainty of the published ground-truth floor across targets.For DW-4, sample distance is 0.31 ± 0.01 against an independent parallel-tempering reference, despite not matching the published reference.
  • 5.2 Particle Potentials: The MAD-Path comparison on LJ-55 reports log q̃ values in the thousands below the reference range around 325, consistent with poor performance.The comparison is used to examine diffusion denoisers versus variational objectives for path proposals.
  • 5.3 Bayesian Neural Network: The 550-dimensional BNN posterior tests trainability and exactness rather than mixing because local MALA effectively covers the posterior.DDMC is therefore evaluated as a high-dimensional global-proposal method in a setting where local coverage is already available.
  • 5.3 Bayesian Neural Network: DDMC reaches the BNN equilibrium energy band in one to two proposal rounds, 2.8–5.5× faster than tuned MALA and 9–18× faster than DiGS from a cold start.The first DDMC proposal is accepted with probability near one and enters the equilibrium band directly.
  • 5.3 Bayesian Neural Network: DDMC offers accepted global proposals above 10% in 550 dimensions, while efficiency depends on workload: few chains favor DDMC, whereas hundreds of batched chains favor MALA.The paper emphasizes proposal existence rather than throughput as the central BNN accomplishment.

6 Conclusion

DDMC uses denoising diffusion as an exact global MCMC sampler, combining global path proposals with local MALA updates. Experiments show high-acceptance global proposals while noting computational and coverage limitations.

  • DDMC composes denoising-diffusion global proposals with local MALA steps to form an exact MCMC sampler.
  • MH correction preserves exactness even when MALA training samples have incorrect mode weights.
  • DDMC achieves global proposals with high acceptance across varied target densities while adhering closely to the target distribution.
  • DDMC is limited by reverse-diffusion cost and modeling failures when MALA samples do not cover or greatly underrepresent regions.

LLM Tool Disclosure

The project used several large language model assistants for writing, literature search, coding, derivation checks, and experimental bookkeeping. The author states that research ideas, method design, and experimental decisions originated with them and that all content was reviewed and verified.

  • ChatGPT 5.5 and several Claude assistants supported writing, literature search, coding, derivation checks, and experimental bookkeeping.
  • The author states that they originated the research ideas, method design, and experimental decisions and verified the paper’s content.

A Ergodicity of the Noise-to-Denoise Transition

The ergodicity argument combines invariant components with positive transition support. Gaussian forward and reverse kernels, together with positive MH acceptance, establish ergodicity for the DDMC chain under the stated conditions.

  • A q-invariant Markov kernel with positive transition density across the relevant support is q-irreducible, aperiodic, and ergodic.
  • SDE case: The forward SDE step and reverse Gaussian kernels have everywhere-positive transition densities because their variances are strictly positive.
  • ODE case: For the ODE case, smoothing by Gaussian convolution and sub-Gaussian tails yield a diffeomorphic reverse flow that preserves full support.
  • DDMC transition: Each DDMC component leaves q invariant, while positive Gaussian transitions and MH acceptance establish the positivity condition and ergodicity.

B Experimental Details

The paper states that full training configurations, corpus-generation schedules, and evaluation scripts will be released with the code.

  • Full training configurations, corpus-generation schedules, and evaluation scripts will be released with the code.

B.2 Metrics

Metrics are computed per chain using released evaluation code, with uncertainty across independent chains and reference-based ground-truth floors at n = 2000.

  • n = 2000 samples per chain are used for the energy Wasserstein-2 distance E(·) W2 and sample Wasserstein-2 distance W2.Both metrics use Guo et al.'s released evaluation code without modification.
  • Metrics are computed separately for each chain rather than pooled across chains.Reported uncertainty is the standard deviation across independent chains.
  • Ground-truth floors evaluate each metric between two independent reference sample sets at the same sample count.The reference sets are disjoint splits of a stored reference bank.

B.3 Reference Samples

Reference samples combine exact, published, held-out MALA, and independently constructed parallel-tempering sources, while the BNN NLL evaluation follows a fixed multi-seed protocol.

  • MoG-40 and GMM-256 use exact draws from analytically samplable targets.
  • Particle-target references use the published iDEM/PDNS ground-truth test split for each system.The DW-4 split distributed with PDNS was verified byte-identical to iDEM's split.
  • The BNN energy marginal is evaluated against 4,096 held-out MALA states disjoint from training and calibration banks.
  • Independent parallel-tempering references were constructed for DW-4, LJ-55, and BNN with reported ladder sizes, state counts, traversals, and swap acceptances.The BNN reference pools seven runs totaling 2,748,992 states and 9,632 hot-to-cold traversals.
  • BNN test NLL uses two chains, 75 recorded samples, and 5,000 gradient steps between recordings, repeated across five methods and four seeds.No pairwise Welch test separates methods after multiplicity correction.

C Diagnostic Terms During Training

Training diagnostics track endpoint mismatch, path-density balance, log acceptance ratios, and acceptance across targets; healthy training keeps path mismatch near zero while negative correlation tightens log-ratio spread.

  • C Diagnostic Terms During Training: δpath tends to remain clustered around zero during healthy training, indicating early balance between forward and reverse path densities.Improvement in log r is attributed to δq as diffusion proposals move closer to the target.
  • C Diagnostic Terms During Training: δq and δpath have a strong negative correlation that makes log r substantially tighter than either component.
Loading 2609.00279v1…