Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis

Yuansi Chen; Francesco Pedrotti; Peter A. Whalley

Source-linked AI summary

Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis

Yuansi Chen, Francesco Pedrotti, Peter A. Whalley

arXiv:2610.31431v1Oct 22, 2026stat.MLcs.LGmath.PRmath.STstat.COstat.TH

TL;DR

Classical Wasserstein-2 guarantees require dimension-dependent stepsizes, while ULA’s bias matters for estimating expectations under high-dimensional targets. The paper analyzes this bias in subspace robust Wasserstein distance and reports a bell-shaped mean-bias profile across sample-size regimes, with dimension-free guarantees failing near N≍d.

Problem
High-dimensional inference needs expectations under π, but ULA is biased and classical W2 guarantees require dimension-scaled stepsizes, limiting their account of practice.
Source: §1 · pp. 1–3
Method
The paper bounds ULA bias in subspace robust Wasserstein distance and treats large-sample, proportional, and overparameterized ridge-separable regimes separately.
Source: Subspace robust Wasserstein bias for… · p. 5, §3 · p. 10
Results
Mean bias is small when N≪d and N≫d but larger near N≍d; dimension-free guarantees fail in general in the proportional regime.
Source: §1 · p. 2, Subspace robust Wasserstein bias for… · p. 5
Takeaways & Limitations
For Gaussian targets, SRW-bias is dimension-free, unlike standard Wasserstein-2 bias.
Source: §2 · p. 6
Classical W2 error control requires dimension-scaled stepsizes and iteration complexity, a dependence that can persist even for Gaussian targets.
Source: §1 · p. 3

Abstract

from arXiv · show

The Unadjusted Langevin Algorithm (ULA) is widely used for high-dimensional Bayesian inference. Classical analyses in Wasserstein-2 distance predict an asymptotic bias and a mixing time that grow with dimension, but this metric is stronger than necessary for many statistical tasks. We instead analyze the bias in subspace robust Wasserstein distance, which controls the mean and arbitrary low-dimensional marginals and is therefore well suited to observables typically estimated in practice. For ridge-separable potentials formed with i.i.d. data, we prove high-probability bounds showing a bell-shaped mixing profile as the number of data points $N$ varies relative to the dimension $d$. In the overparameterized regime, the effective dimension is the rank $N$ of the data-dependent perturbation, leading to complexity depending on $N$ rather than $d$. In the large-sample regime, Gaussian-perturbation estimates yield dimension-free complexity once $N$ is sufficiently large. In contrast, near the proportional regime $\asymp d$, we prove that dimension-free guarantees are impossible in general.

1 Introduction

The introduction motivates measuring ULA bias with a metric suited to means and arbitrary low-dimensional observables, then develops a three-regime account of its mixing complexity for ridge-separable models. The resulting profile is favorable when N is much smaller or larger than d, but not generally near N ≍ d.

1 Introduction

The mean error has a bell-shaped profile: it is small for N ≪ d and N ≫ d, and larger near N ≍ d.The paper uses the mean error as a proxy for SRW bias because that distance controls differences between means.Source: §1 · p. 2, §1.1 · p. 4
Classical Wasserstein-2 guarantees require a step size scaling inversely with dimension, so their iteration complexity grows with dimension and does not explain the reported practice.The cited limitation persists even for Gaussian targets.Source: §1 · p. 3

1.1 Our contributions

SRW is proposed because it controls the mean and arbitrary low-dimensional observables, unlike a guarantee limited to random low-dimensional directions.The paper presents it as a way to analyze observables practitioners commonly estimate.Source: §1.1 · pp. 3–4

Subspace robust Wasserstein bias for Gaussian perturbations

For Gaussian perturbations, the paper establishes SRW-bias bounds that recover a dimension-free inequality as the perturbation approaches zero.The Gaussian case itself has dimension-free SRW bias, unlike standard Wasserstein-2 bias.Source: §1.1 · p. 3, Subspace robust Wasserstein bias for… · p. 5

1.1 Our contributions

For ridge-separable potentials with i.i.d. sub-Gaussian data, the paper gives dimension-free SRW iteration complexity up to logarithmic factors in the overparameterized and sufficiently large-sample regimes.When N ≪ d, the data-dependent perturbation has effective rank N; when N ≫ d, posterior concentration permits Gaussian-perturbation estimates.Source: §1.1 · pp. 4–5, Subspace robust Wasserstein bias for… · p. 5
Near N ≍ d, dimension-free guarantees fail in general: the SRW bias and even the mean bias can grow with dimension for typical data.The paper gives an explicit example for this regime.Source: §1.1 · p. 4, Subspace robust Wasserstein bias for… · p. 5

2 Subspace robust Wasserstein bias for Gaussian perturbations

The section bounds ULA’s asymptotic bias for Gaussian perturbations in subspace robust Wasserstein distance and gives a separate mean-estimation improvement as the perturbation parameter S tends to zero. Gaussian targets have dimension-free SRW bias, unlike the standard Wasserstein-2 bias.

2 Subspace robust Wasserstein bias for Gaussian perturbations

For Gaussian targets, the SRW asymptotic bias is dimension-free, unlike the bias in the standard Wasserstein-2 distance.The section presents this as the motivation for studying Gaussian perturbations.Source: §2 · p. 6
Gaussian proximity is relevant to ridge-separable distributions when N≪d through the Gaussian prior and when N≫d through asymptotic results such as the Bernstein–Von Mises Theorem [Vaa98].Removing dimension dependence in the first term is crucial for deriving dimension-free iteration complexity in the rescaled metric for these applications.Source: §2 · pp. 6–7
Under Assumptions 1.1 and 2.1 with h≤1/β, Theorem 2 bounds ULA’s SRW asymptotic bias, improves the mean-estimation error as S→0, and also gives a W2 bias bound.The W2 bound is closely related to an estimate in [DK19, Theorem 5].Source: §2 · p. 7
The parameters M and S quantify closeness to Gaussianity under Assumption 2.1, with M=S=0 corresponding to a Gaussian measure.The parameter is introduced as scale-invariant and in the spirit of a condition number.Source: §2 · pp. 6–7

3 Main results for ridge-separable models

For ridge-separable models built from i.i.d. data, the section derives iteration-complexity bounds in subspace robust Wasserstein accuracy across asymptotic, proportional, and overparameterized regimes. The bounds become dimension-free for sufficiently large samples, while dimension-free rates are impossible in general near N ≍ d.

3 Main results for ridge-separable models

The analysis treats ridge-separable potentials formed from i.i.d. datapoints and distinguishes asymptotic, proportional, and overparameterized sample-size regimes.The section develops high-probability complexity bounds in subspace robust Wasserstein accuracy under its stated assumptions.Source: §3 · p. 10

3.1 Asymptotic regime: N ≫d

In the asymptotic regime, the iteration-complexity upper bound decreases with N and becomes dimension-free once N is sufficiently large.The proof uses that the potential becomes approximately quadratic as N grows, so the target approaches a Gaussian distribution.Source: §3.1 · pp. 11–12

3.2 Proportional regime: N ≍d

Near N ≍ d, a constructed ridge-separable potential rules out dimension-free complexity bounds for sampling in SRW-accuracy in general.For fixed accuracy, the argument requires a step size that shrinks with dimension, leading to an iteration requirement that grows with d.Source: §3.2 · pp. 13, 15

3.3 Overparameterized regime: N ≪d

In the overparameterized regime, the data-dependent perturbation has rank at most N, and Theorem 6 provides a high-probability iteration-complexity bound.The theorem applies when N ≤ d − 1 under its stated assumptions and permits a sufficiently small step size.Source: §3.3 · p. 16

A Additional simulation details

The simulations compare posterior-mean estimates from ULA with MALA references across sample sizes. The SUSY experiment uses repeated subsets of the dataset and shows bell-shaped error profiles alongside step-size scaling.

A Additional simulation details

For Figure 1, results are averaged over 20 independent simulated datasets for each N.The MALA estimate uses 106 samples, while the ULA estimate uses 5 × 105 samples.Source: A Additional simulation details · p. 21
For SUSY, the authors average over 10 uniformly sampled subsets of the first 50,000 observations for each N.The covariates are transformed so their coordinate variances are of order 1/d; the MALA reference uses two chains after tuning iterations, and ULA uses a step size scaled from MALA's.Source: A Additional simulation details · p. 21
Figure 4 reports bell-shaped profiles for estimated and rescaled mean error, with step-size scaling shown as N varies.The experiment is 18-dimensional Bayesian logistic regression on UCI SUSY with a standard Gaussian prior on transformed coefficients.Source: Figure 4 · p. 21

B Subspace robust Wasserstein and related metrics

This section contrasts subspace robust Wasserstein distance with averaged sliced distances and explains what each can control. SRW supports bounds for arbitrary low-dimensional observables, while typical-direction bounds also have implications for predictive inference.

Subspace robust Wasserstein distance

SRW bounds support estimation of Lipschitz test functions on low-dimensional projections and also control the means of the distributions.The section presents these guarantees as analogous to those obtained from W2.Source: B Subspace robust Wasserstein and… · p. 21

Related metrics

Sliced Wasserstein distances compare Wasserstein distances between projections, with variants changing projection dimension, Wasserstein order, or averaging norm.The standard squared 2-sliced distance averages projected W2 over a uniformly random direction.Source: B Subspace robust Wasserstein and… · pp. 21–22
Averaged sliced distances control typical random projections, not every direction, and therefore do not explain the observed bell-shaped iteration profile.The text notes that even a stated bound can fail when its right-hand side is replaced by SW2.Source: B Subspace robust Wasserstein and… · p. 22

Sliced Wasserstein distances and predictive inference

For typical datapoints, Lemma B.1 yields a dimension-free error, unlike the error for a worst-case test direction.The text connects this typical-direction observation to predictive inference and states that sliced-Wasserstein bias is dimension-free.Source: B Subspace robust Wasserstein and… · p. 23

C Random tensors in high dimension

This appendix develops high-probability technical bounds used in the main results. Its arguments include concentration after isotropic normalization and generic chaining for a matrix-indexed process.

C Random tensors in high dimension

The appendix develops high-probability bounds and technical results for the main analysis, using the sub-Gaussian norm convention cited in the text.The stated convention is attributed to.Source: C Random tensors in high dimension · p. 24
Lemma C.1 gives a high-probability bound under Assumption 3.1 when N ≥ d + u.The stated probability is at least 1 − 2e^−u.Source: C Random tensors in high dimension · p. 24
The proof first places the datapoint vectors in isotropic position and then applies concentration bounds to obtain covariance comparisons.The passage states the resulting comparison uses the relation between N and d + u.Source: C Random tensors in high dimension · p. 24
A sub-Gaussian process indexed by a matrix set is bounded using generic chaining, with its Gaussian width and radius estimated separately.The process is defined as X_U := ⟨A, U⟩_F, and the resulting probability bound is at least 1 − 2e^−t².Source: C Random tensors in high dimension · pp. 25–26
The proof also bounds the Hessian and third derivative of the ridge-separable potential before combining estimates by a union bound.The assembled event has probability at least 1 − 4e^−u.Source: C Random tensors in high dimension · pp. 26–27

D.1 Invariance under orthogonal transformations

The section establishes that Langevin dynamics and ULA are invariant under orthogonal changes of coordinates. It uses rotational invariance of Brownian motion to relate the transformed dynamics and biased limits.

D.1 Invariance under orthogonal transformations

Both Langevin dynamics and ULA are independent of the coordinate system.The text motivates this by the steepest-descent direction and the spherical symmetry of Brownian motion.Source: §D.1 · p. 27
An orthogonal matrix is used to define the pushforward measure for the transformed coordinates.The passage introduces the orthogonal transformation and its corresponding measure.Source: §D.1 · p. 27
Rotational invariance of Brownian motion supports the transformed dynamics, and the biased limits obey a corresponding relation.Source: §D.1 · p. 27

D.2 Classical bounds

The proof of classical bounds cites prior work for the first bound and derives the second and third from score subgaussianity under π.

The first bound in (13) is referred to [DKR22, Lemma 2].Source: §D.2 · p. 28
The second and third bounds in (13) follow from score subgaussianity under π, citing [Che25, Lemma 6.2.7] and.Source: §D.2 · p. 28

E Remaining omitted proofs

The omitted proofs use product-Gaussian structure and couplings for Wasserstein bounds, alongside a separate construction involving a centered probability measure.

Equal means and diagonalization of A make π and π̂ product measures, reducing the lower bound on W2 to one-dimensional Gaussians with the same mean.Source: E Remaining omitted proofs · p. 28
The upper bound on SRW(π, π̂) is approached by constructing a coupling, with subsequent bounds using 0 ≤ h_a ≤ 1 and 1 − h_a ≤ e^-h_a.Source: E Remaining omitted proofs · p. 28
Dominated convergence establishes that F is C1 in a neighborhood of (0, 0).Source: E Remaining omitted proofs · p. 29
An implicit-function construction gives a C1 map b near zero; for sufficiently small positive δ̄, I(δ̄) > 0, enabling the stated choice of constants and centered probability measure ρ.Source: E Remaining omitted proofs · p. 29
The last condition in the lemma is established using [Che+24, Proposition 3.1] with f(x) = x.Source: E Remaining omitted proofs · p. 29
Independent optimal couplings for the two factor pairs combine into a coupling of the product measures; the proof then takes a supremum using the SRW definition.Source: E Remaining omitted proofs · p. 30