Source-linked AI summary
On skew-symmetric distributions and their use in Monte Carlo sampling algorithms: coordinate-free, Gibbs-style and manifold versions of the Barker proposal
Minh Vu, Samuel Livingstone, Pantelis Samartsidis
TL;DR
Skew-symmetric proposals offer a way to use gradient information in MCMC, including when other gradient-based methods fail. The paper develops coordinate-free and Gibbs-style Barker variants and a simplified manifold version; for coordinate-free variants, raw ESS is comparable to standard Barker, while added costs limit practical advantage.
Problem
Method
The paper reviews the preconditioned skew-symmetric Barker proposal and develops coordinate-free, Gibbs-style, and simplified manifold extensions.
Source: §1 · pp. 1–2, §7 · p. 16Results
Raw ESSs for standard Barker and coordinate-free alternatives are consistently comparable, while the added costs of new bases limit the coordinate-free variants' practical advantage.
Source: §7 · p. 16Takeaways & Limitations
The Barker construction uses both gradient magnitude and direction without making proposal-increment size depend on gradient magnitude, a stability property noted for irregular gradients.
Source: §2 · p. 3Gibbs-style Barker has the same derivative-evaluation count as standard Barker, but iterative partial-state evaluations can add implementation-dependent overhead.
Source: §4 · p. 8
Abstract
from arXiv · showhide
Skew-symmetric probability distributions provide a principled mechanism for incorporating gradient information into Markov chain Monte Carlo algorithms. Here we review the (preconditioned) Barker proposal, a Metropolis--Hastings algorithm built on skew-symmetric distributions, and motivate its design. We then introduce three natural extensions. First, we propose coordinate-free variants of the Barker algorithm. Second, we introduce a Gibbs-style Barker algorithm that re-evaluates the gradient at each partially updated coordinate. Third, we derive a simplified manifold Barker algorithm, producing a manifold sampler with enhanced robustness compared to natural comparators. Numerical experiments demonstrate that the Gibbs-style variant improves raw sampling efficiency on correlated targets, that the coordinate-free variants offer limited practical advantage over the standard Barker proposal once computational costs are accounted for, and that the simplified manifold Barker algorithm can achieve significant advantages over simplified manifold MALA when the local geometric structure of the target is irregular or unreliable.
1 Introduction
The introduction motivates skew-symmetric proposals as a way to incorporate gradient information and presents coordinate-free, Gibbs-style, and manifold Barker extensions. It previews benchmark comparisons of these variants with the standard Barker proposal and simplified manifold MALA.
1 Introduction
- Skew-symmetric proposals provide a principled way to incorporate gradients and can work well when other gradient-based methods fail.Their design is motivated by approximations to stochastic differential equations and locally balanced Markov jump processes.Source: §1 · pp. 1–2
- The paper introduces coordinate-free Barker variants, a Gibbs-style proposal that re-evaluates gradients during partial updates, and a simplified manifold Barker algorithm.The manifold version ignores third-derivative terms to reduce computational costs.Source: §1 · p. 2
- The authors compare coordinate-free variants with standard Barker and report limited practical advantage after accounting for computational costs.For simplified manifold Barker, they examine robustness relative to simplified manifold MALA when target geometry is irregular or unreliable.Source: §1 · p. 2
2 The Barker proposal
The preconditioned Barker proposal uses skew-symmetric component distributions whose skewness is determined by the preconditioned log-density gradient. Its design is motivated by approximations to Langevin diffusion and locally balanced jump processes.
2 The Barker proposal
- Each proposed component is skew-symmetric, centered at the current state, scaled by the preconditioner, and skewed according to the corresponding component of the preconditioned gradient.The original choice uses a Gaussian base density and logistic CDF; a bimodal base density can increase efficiency.Source: §2 · p. 3
- The proposal increment's size is determined by the symmetric base draw, not gradient magnitude, while skewness uses both gradient magnitude and direction.The authors identify this as a stability advantage when gradients are large or irregular.Source: §2 · p. 3
2.1 Motivation from Langevin dynamics
- Approximating a Langevin transition density with a tractable proposal requires a Metropolis–Hastings filter to ensure the resulting chain has limiting distribution π.Euler–Maruyama's Gaussian approximation yields MALA.Source: §2.1 · p. 4
- Skew-symmetric numerical schemes approximate diffusion transitions by generating increments skewed according to the volatility-scaled drift.For the described scheme, the true solution is recovered weakly as the time step tends to zero, and the scheme has weak order 1 under mild assumptions.Source: §2.1 · p. 4
2.2 Motivation from locally-balanced Markov processes
- Using Barker's balancing function in a first-order approximation to a locally balanced jump process yields the Barker proposal, with a normalizing constant of 1/2.In this approximation, jumps no longer occur at a state-dependent rate.Source: §2.2 · p. 5
3 A coordinate-free Barker algorithm
The section motivates coordinate-free proposals by sensitivity to the coordinate basis and develops Barker variants that randomize that basis. It compares Haar-uniform and structured Walsh–Hadamard approaches to generating orthogonal transformations.
3.1 Problems caused by over-reliance on the coordinate system
- The coordinate basis can strongly affect systematic-scan Gibbs sampling, whose relaxation time can be made arbitrarily large but becomes one iteration after rotating to independent coordinates.The authors say Barker is less affected than Gibbs in their experience, though its performance can still depend on the basis.Source: §3.1 · p. 6
3.2 Coordinate-free Metropolis–Hastings proposals
- A coordinate-free proposal is designed so orthogonal changes of coordinates do not change where its transition density places probability mass.The standard Barker proposal does not satisfy this property; a scalar summary of the gradient in a proposed multivariate alternative was found inferior to the standard construction.Source: §3.2 · p. 6
3.3 A randomized basis Barker proposal
- Randomizing the coordinate basis yields a mixture of skew-symmetric proposals, while sampling a basis and using its conditional proposal provides a tractable alternative when the mixture is intractable.The conditional approach bases acceptance on qQ rather than the marginalized qµ; its additional cost depends on generating Q and multiplying by Qᵀ.Source: §3.3 · pp. 6–7
3.4 Two different strategies for sampling orthogonal matrices
- The Haar strategy samples continuous orthogonal matrices at O(d^3) per draw, whereas the Walsh–Hadamard strategy uses sign-flipped structured matrices with O(d) sampling cost.The implementation applies the structured matrix by dense matrix-vector multiplication at O(d^2); dimensions that are not powers of two are zero-padded.Source: §3.4 · p. 7
4 A Gibbs-style Barker algorithm
Gibbs-style Barker updates coordinates in sequence, reevaluating the gradient at each partially updated state. This changes proposals when coordinates are dependent and can better align moves with the target’s local geometry, with possible implementation overhead.
4 A Gibbs-style Barker algorithm
- Gibbs-style Barker sweeps coordinates in a fixed order and evaluates the gradient at the partially updated state before each coordinate proposal.The standard coordinate-wise proposal instead freezes the gradient at the current state for all coordinate increments.Source: §4 · p. 7
- On a correlated Gaussian with ρ = 0.95, reevaluating the gradient at the partial state produces a second-coordinate increment better aligned with the target’s local elliptical geometry.For factorized targets, the standard and Gibbs-style proposals are identical.Source: §4 · p. 8, Figure 1 · p. 8
- Both methods require d partial derivatives for proposing and another d for the Metropolis–Hastings filter, but iterative evaluation can add implementation-dependent overhead.The overhead may depend on the model, programming language, and implementation style.Source: §4 · p. 8
5 A Manifold Barker algorithm
The manifold Barker algorithm uses position-dependent preconditioning to accommodate targets with varying local geometry. Its simplified form adapts the Barker proposal to a Riemannian setting, replacing a drift term with coordinate-wise sign flips in the preconditioned basis.
5 A Manifold Barker algorithm
- When local geometry, scaling, or correlation structures vary across the state space, a position-dependent preconditioner offers an alternative to a constant global one.The preconditioner A(x) must be positive definite; an inverse-Hessian choice may fail this condition when the target is not log-concave.Source: §5 · p. 9, §5.1 · p. 9
5.1 Riemannian Langevin Diffusion
- Riemannian Langevin diffusion adjusts the drift for state-dependent preconditioning, while its Euler–Maruyama discretization yields the manifold MALA proposal.Simplified manifold MALA drops a correction requiring derivatives of A(x), which can require third derivatives of log π(x); the Metropolis–Hastings correction still recovers π as the chain’s invariant distribution.Source: §5.1 · pp. 9–10
5.2 Simplified Manifold Barker
- Simplified manifold Barker replaces the SMMALA drift with coordinate-wise sign flips in the preconditioned basis, with probabilities determined by the proposal’s skewing rule.Proposals are generated as y = x + L(x)(b ⊙ z), where L(x)L(x)⊤ = A(x).Source: §5.2 · pp. 10–11
- Because L(x) depends on the current state, the Metropolis–Hastings ratio includes an additional factor |L(x)|/|L(y)|.The per-iteration cost is typically dominated by Hessian evaluation and eigendecomposition when using SoftAbs.Source: §5.2 · p. 10
6 Numerical Study
Across four benchmark settings, the study compares Barker variants and manifold samplers using sampling efficiency or marginal-distribution distances. Gibbs-style Barker often improves raw ESS but can incur substantial computational cost, while SMBarker performs especially well on the hybrid Rosenbrock target.
6 Numerical Study
- The study evaluates Barker variants on correlated Gaussian and Sonar targets, and compares SMBarker with MALA, Barker, and SMMALA on nonlinear-correlation targets.Metrics include minimum ESS, ESS per iteration and CPU second, and maximum marginal KS and AD distances; for FitzHugh–Nagumo, chains start at the true mode.Source: §6 · p. 11, §6.4 · p. 15
6.1 High-dimensional Gaussian
- Gibbs-style Barker had the highest raw ESS at 6 of 8 Gaussian dimensions, but its CPU cost was 3–21 times standard Barker’s; standard Barker led ESS/s across dimensions.Hadamard Barker offered competitive ESS with lower CPU cost than Gibbs-style or Rotated Barker, while Rotated Barker had the lowest ESS/s at d ≥ 59.Source: §6.1 · pp. 11–12, Table 1 · p. 12
6.2 Logistic regression with the Sonar dataset
- On Sonar logistic regression, Gibbs-style Barker achieved roughly 1.6 times the raw ESS of the next-best method, while Hadamard Barker delivered 13% more raw ESS than standard Barker but slightly lower ESS/s.Rotated Barker was effectively tied with standard Barker on raw ESS but had an order-of-magnitude higher CPU cost and ranked last in ESS/s.Source: §6.2 · p. 13, Table 2 · p. 13
6.3 Hybrid Rosenbrock distribution
- SMBarker achieved the lowest KS and AD distances for every tested value of b on the hybrid Rosenbrock target.None of the samplers clearly converged within the compute budget, so the comparison uses maximum marginal KS and AD distances rather than ESS.Source: §6.3 · pp. 13–15, Table 3 · p. 14
6.4 Fitzhugh–Nagumo model
- On FitzHugh–Nagumo, locally preconditioned methods yielded roughly four to over ten times the raw ESS of constant-preconditioner methods across the three parameters.SMMALA had the highest raw ESS; SMBarker reached about 80% of SMMALA’s value on parameter b.Source: §6.4 · pp. 15–16, Table 4 · p. 15
7 Discussion
The discussion finds that the Barker extensions have distinct trade-offs: coordinate-free variants offer no apparent practical benefit once costs are counted, while Gibbs-style sampling can improve raw ESS and simplified manifold Barker can improve robustness. These findings build on the Barker proposal’s gradient-guided skewing and its stochastic-process motivations.
7 Discussion
- The Barker proposal’s justification draws on approximations to diffusion transition densities and Markov jump-process kernels, both targeting the desired limiting distribution.Its name also connects to Barker’s alternative Metropolis–Hastings algorithm.Source: §7 · p. 16
- The paper presents coordinate-free, iterative Gibbs-style, and local-geometry manifold modifications to the preconditioned Barker proposal.The Gibbs-style variant makes each coordinate’s skew depend on innovations in previously updated coordinates.Source: §7 · p. 16
- Coordinate-free variants have raw ESS comparable to standard Barker, but offer no apparent benefit after accounting for the cost of generating new bases.The standard Barker proposal is not coordinate-invariant, though performance dependence on the coordinate basis appears mild compared with the Gibbs sampler.Source: §7 · p. 16
- Gibbs-style Barker showed about 1.8 times higher raw ESS on the Sonar logistic-regression example, but iterative gradient evaluation can limit time-normalized performance.Repeatedly recomputing partial derivatives may be especially costly when each derivative loops through the full dataset.Source: §7 · pp. 16–17
- Simplified manifold Barker appears easier to tune and more robust than simplified manifold MALA, sometimes at the cost of reduced efficiency.The discussion notes that mismatch in the simplified manifold Barker acceptance rate may limit efficiency gains like those seen with bimodal noise in standard Barker.Source: §7 · p. 17
Appendix: Speed tests for Gibbs-style Barker
The speed tests compare full updates resembling standard Barker with staged updates resembling Gibbs-style Barker. Staged Gaussian updates were comparable in compiled C++ but slower in interpreted languages, and remained slower for the logistic-regression example even in C++.
Appendix: Speed tests for Gibbs-style Barker
- For a 1000-dimensional Gaussian, compiled C++ runtimes were comparable for full and staged updates, while staged updates took significantly longer in Python and R.Table 5 compares runtimes over 1000 repetitions; the full update mimics standard Barker and the staged update mimics Gibbs-style Barker.Source: Appendix: Speed tests for Gibbs-style… · p. 17, Table 5 · p. 17
- For logistic regression using the Sonar dataset, the Gibbs-style update still took significantly longer in C++, unlike the comparable compiled-language Gaussian runtimes.The experiment used staged partial-derivative updates versus a full gradient update.Source: Appendix: Speed tests for Gibbs-style… · p. 17