Variance Reduction using Nonreversible Langevin Samplers

1.1 Motivation

• Metastability and concentration along a low-dimensional manifold can make reversible Langevin sampling explore slowly and produce high variance for observables that vary across those regions.The paper frames choosing a process that preserves the target distribution but improves convergence or estimator variance as an optimization question.Source: §1.1 · p. 2

1.2 Nonreversible Langevin Dynamics

• Prior work cited in the introduction reports faster convergence from invariant-preserving nonreversible drifts, while studies of their effect on asymptotic variance are also reviewed.The cited convergence result applies to a drift that is divergence-free with respect to the target distribution.Source: §1.2 · p. 3

1.3 Objectives of this paper

• The paper studies how adding a nonreversible drift to overdamped Langevin dynamics changes the asymptotic variance of time-averaging estimators.Its analysis derives a spectral representation and examines how variance depends on perturbation strength.Source: §1.3 · p. 4
• For linear-drift Gaussian diffusions, the paper seeks perturbations that minimize asymptotic variance and finds variance reduction can differ from maximizing the spectral gap.For linear and quadratic observables, it gives a complete solution and describes cases with large variance reduction but no improvement in convergence rate.Source: §1.3 · p. 5
• Toy-model experiments examine variance reduction and computational cost, reporting that a well-chosen nonreversible perturbation can outperform MALA.The paper also investigates a higher-order split integrator for improved stability and discretization error.Source: §1.3 · p. 5

• The stated sufficient density condition holds for Gaussian targets, whereas the section notes it fails for Student’s t targets and that the corresponding process is not exponentially ergodic.Source: §2 · p. 7
• The Poisson equation’s well-posedness is necessary for finite asymptotic variance, and the section studies how the solution and variance depend on the nonreversible perturbation.The asymptotic variance is the Dirichlet form of the dynamics evaluated at the Poisson solution.Source: §2 · pp. 7–8
• Under the stated Lyapunov assumptions, the process is exponentially ergodic and the Poisson equation has a unique zero-mean solution for observables satisfying the given growth bound.The solution is in L2(π) and is bounded in terms of the Lyapunov function.Source: §2 · p. 7
• The section gives a central limit theorem for time averages of the diffusion, including processes started from an arbitrary initial distribution.The stated result applies to observables whose squared magnitude is bounded by the Lyapunov function.Source: §2 · p. 8

3.1 Mathematical Setting

• L2_0(π) denotes the square-integrable functions with zero mean under π.Source: §3.1 · p. 8
• The spaces Hk are completed in their stated norms, yielding Hilbert spaces for k ≥ 1.Source: §3.1 · p. 8
• The reversible operator S is symmetric and extends to a selfadjoint operator on L2_0(π), with domain H2.The section also assumes a spectral gap for L in L2_0(π).Source: §3.1 · p. 8

Assumption C

• The compact embedding H1⊂L2(π) yields a Poincaré inequality with λ independent of the perturbation, and the associated semigroup converges exponentially to equilibrium.Source: Assumption C · p. 9
• The level-preserving flow leaves barrier crossing to the reversible dynamics, retaining its Arrhenius law; space-dependent diffusion tensors are not considered.Source: Assumption C · p. 10

3.2 An Expression for the Asymptotic Variance

• A nonreversible perturbation never increases asymptotic variance; the paper expresses that variance through a symmetric quadratic form explicit in A and S.Source: §3.2 · p. 11

3.3 Quantitative estimates for the Asymptotic Variance

• The spectral analysis represents G as iΓ for a compact selfadjoint Γ, whose eigenfunctions form a complete orthonormal basis in H1.Source: §3.3 · pp. 11–12
• The decrease in asymptotic variance and increase in spectral gap arise from different mechanisms; increasing perturbation strength can even reduce convergence speed.Source: §3.3 · p. 13
• A perturbation minimizing variance across observables need not reduce variance for a fixed observable when the relevant nullspace is nontrivial.Source: §3.3 · p. 13

3.4 A Two Dimensional Example

• As |α|→∞, the two-dimensional example's asymptotic variance converges to 4, while motion in the angular direction is averaged out.Source: §3.4 · pp. 14–15
• For γ=J∇logπ, fast deterministic motion explores target level curves; a trivial nullspace yields variance converging to 0, while a nontrivial one leaves slow diffusion between components.Source: §3.4 · p. 15

4 Nonreversible Perturbations of Gaussian Diffusions

• The Gaussian stationary distribution N(0, I) is independent of α and J, and certain observables can nonetheless have dramatically reduced asymptotic variance.Source: §4 · p. 15

4.1 Explicit formula for the asymptotic variance

• For centered quadratic-plus-linear observables, the Poisson equation is solved analytically, yielding a closed-form asymptotic-variance expression.Source: §4.1 · pp. 15–16
• For linear-drift diffusions, asymptotic variance cannot always be decreased arbitrarily.Source: §4.1 · p. 17

4.2 Finding the Optimal Perturbation

• For the plotted linear observables, variance tends to zero when l is perpendicular to J’s nullspace, but remains 2 when l lies in that nullspace.Source: §4.2 · p. 18, Figure 2 · p. 19
• Pairing the smallest and largest eigenvalues, then the next-smallest and next-largest, minimizes the quadratic-observable bound, which is tight only when M = m I.Source: §4.2 · p. 19

5.1 Periodic Distribution

• About two orders of magnitude lower variance is observed as α increases from 0 to 10 for the periodic-distribution example, without noticeable bias increase.Source: §5.1 · p. 20, Figure 3 · p. 21

5.2 Warped Gaussian Distribution

• A factor-of-80 variance decrease as α rises from 0 to 10 comes with increased discretization bias for the warped Gaussian example.Source: §5.2 · p. 21, Figure 5 · p. 22, §5.3 · p. 22

5.3 Introducing a Metropolis–Hastings accept-reject step

• With Metropolis–Hastings, discretization bias is removed, but variance increases with α as larger-α proposals are more likely to be rejected.Source: §5.3 · pp. 22–23

5.4 Dimer in a Solvent

• For the dimer observable, variance falls 66% with J1 and 50% with J2 as α rises from 0 to 10, at increased computational cost.Source: §5.4 · p. 24

6 The Computational Cost of Nonreversible Langevin Samplers

• Increasing α is guaranteed to lower asymptotic variance, but it also increases stiffness and discretization error, requiring smaller timesteps and potentially greater computational cost.Source: §6 · p. 24
• For fixed computational budget, the MSE bound balances discretization and sampling error, while the error constant is expected to increase with α and its scaling remains open.Source: §6 · p. 25
• An 8.8-fold MSE advantage over MALA occurs at α=5; the lowest MSE occurs at α=10, with no further gain beyond that point.Increasing α trades lower variance against greater discretization error.Source: §6 · p. 25
• At α=25, Gaussian-model MSE gains are marginal; the discretization fails above 10^-3, while α=0 and MALA are accurate at timesteps of order 1.Source: §6 · pp. 25–26
• The splitting implementation combines a MALA step for reversible dynamics with a fourth-order Runge–Kutta step for the nonreversible flow.The comparison uses six gradient evaluations per splitting step against one per MALA step.Source: §6 · p. 26
• The splitting sampler beats MALA by factors of 13 (warped Gaussian) and 20 (standard Gaussian), with MSE an order of magnitude below Euler–Maruyama.The comparison accounts for six gradient evaluations per splitting step versus one per MALA step.Source: §6 · p. 26
• The authors leave rigorous analysis of the splitting scheme for future work, despite its improved stability and observed MSE performance.Source: §6 · p. 26

7 Conclusions and Further Work

• Theoretical analysis for periodic-coefficient and linear-drift diffusions, alongside toy-model experiments, finds that judicious perturbations can substantially reduce asymptotic variance, but arbitrary choices may not help.Source: §7 · p. 26
• A nonreversible Langevin sampler with a close-to-optimal drift significantly outperforms the reversible Metropolis-Hastings sampler in the numerical experiments.Source: §7 · p. 27
• The computational-cost study tracks the competing effects of reduced asymptotic variance and increased stiffness from adding a nonreversible drift.Source: §7 · p. 27
• Further work includes analyzing numerical schemes designed for nonreversible Langevin diffusions, hypoelliptic sampling, and nonreversible Metropolis–Hastings algorithms.Source: §7 · pp. 27–28