Accelerating Non-Smooth and Heavy-Tailed Sampling

• The construction extends anchored dynamics, which can sample non-smooth targets without differentiating the target potential and can converge exponentially fast for heavy-tailed targets.The paper motivates this setting in part by the expense of Metropolis steps in some high-dimensional or large-data applications.Source: §1 · p. 3
• Earlier reflected anchored Langevin dynamics addressed possibly non-differentiable targets on constrained domains, while prior constrained non-reversible approaches imposed choices of skew field that can be challenging in practice.The paper positions NRALD as a response to the question of whether reversibility breaking accelerates anchored methods.Source: §1 · pp. 3–4
• NALD and NRALD are proposed for unconstrained and constrained sampling, respectively, with target invariance and a random-time-change representation.The construction includes divergence-free state-dependent skew tensors and, on constrained domains, mechanisms that retain ordinary normal reflection.Source: §1 · p. 4
• Breaking reversibility can accelerate convergence in finite-time total-variation bounds, and the level-two large-deviation rate comparison supports greater concentration around the target.The paper also establishes asymptotic-variance reduction under its stated assumptions.Source: §1 · pp. 4–5
• For the q-parametrized heavy-tailed family, the paper verifies conditions yielding both large-deviation and asymptotic-variance acceleration for a Student-t specialization.It also gives an explicitly solvable quadratic-clock example.Source: §1 · p. 5
• The experiments report about 1.8 times fewer iterates for NALD on a heavy-tailed target and up to a 1.85 speedup for NRALD on constrained Laplace targets.In constrained Bayesian logistic regression, NRALD reaches a given accuracy faster on both synthetic and real data.Source: §1 · p. 5

2 Main Results

• The proposed circulation uses a possibly state-dependent skew-symmetric field J and stream potential ψ, with divergence-free J; on bounded domains, ψ is chosen so the circulation is tangential at the boundary.A constant skew matrix is a useful special case, but it becomes reversible when the reference potential is constant in the constrained setting.Source: §2 · pp. 5–6
• The resulting generator is Lα = S + αA, where S is symmetric and A is anti-symmetric on their common core in L2(π).This form structure links the unconstrained and reflected dynamics.Source: §2 · p. 6

2.1 Unconstrained dynamics

• NALD is proposed on Rd under local regularity and well-posedness assumptions, using the reference-potential gradient and a state-dependent clock.The dynamics does not require the target potential U to have an everywhere-existing gradient.Source: §2.1 · p. 6
• Choosing the stream potential to make c(x) = −a(x)J(x)∇U0(x) gives the most direct anti-symmetric perturbation of anchored Langevin dynamics.The construction is presented as a useful special case of the more general circulation approach.Source: §2 · p. 5, §2.1 · p. 6

• For a state-dependent skew field satisfying the stated assumptions, the current c = e^U J∇ψ has zero weighted divergence, ∇·(πc) = 0.This is the stream-current construction used to extend beyond constant skew matrices.Source: §2.2 · p. 7
• When the diffusion is well posed and, in the reflected case, c·n = 0, the general state-dependent model inherits the invariance and form anti-symmetry conclusions.The normal component condition ensures no current crosses the boundary.Source: §2.2 · p. 7
• The proof uses vanishing divergence terms and the orthogonality of a skew-symmetric matrix to the symmetric Hessian.The remaining conclusions follow by integration by parts.Source: §2.2 · p. 7

• NRALD is proposed on a bounded domain with reflection driven by a nondecreasing boundary process supported on times when the process is on ∂K.Its generator has a Neumann domain, and no oblique reflection is needed.Source: §2.3 · p. 7
• The zero-flux requirement c·n = 0 can be enforced either by a stream potential constant on the boundary or by a state-dependent skew tensor satisfying Jn = 0.These conditions make the circulation compatible with ordinary normal reflection.Source: §2.3 · pp. 7–8
• The tensor condition Jn = 0 has no nontrivial construction in two dimensions when combined with the stated divergence-free condition.The paper identifies natural nontrivial tensor constructions from dimension three onward.Source: §2.3 · p. 8
• On a ball, a stream potential constant on the boundary yields a cross-product construction, while higher-dimensional examples can be built by embedding three-dimensional blocks.The particular block construction annihilates radial ∇U0, so the constant-J Student construction and more general stream choices remain useful.Source: §2.3 · p. 8

2.4 Sublevel-set constraints and normal reflection

• Choosing ψ = ρ makes the circulation drift tangent to the boundary, so normal reflection preserves the target on K for any reference potential U0 meeting the regularity assumptions.K is defined by ρ(x) < 0 with ∇ρ nonzero on its boundary.Source: §2.4 · p. 8
• For U0 ≡ 0, c(x) = e^U(x)J(x)x is an angular drift, and the sampler uses U only through function values.The state-dependent noise corrects the target density, while the angular current breaks detailed balance without altering it.Source: §2.4 · p. 9

2.5 Constraint sets and skew tensors

• A shared block cross-product construction defines skew tensors for the centered ball and smoothed ℓ1 ball, leaving up to two coordinates unblocked when d is not divisible by three.The construction uses a potential constant on the boundary; the constraint sets follow Wang et al. [2026].Source: §2.5 · pp. 9–10

2.5.1 Ball constraint

• For the ball constraint, the boundary-constant potential yields block vectors proportional to each coordinate triple, with the d = 9 example comprising three 3 × 3 blocks.The block scales can be chosen independently.Source: §2.5.1 · p. 10

2.5.2 Smoothed ℓ1 ball constraint

• For the smoothed ℓ1 ball, ε > 0 ensures the tensor is smooth; the construction satisfies skew-symmetry, boundary tangency, and divergence-free conditions.The section also states that the domains have smooth boundaries and that the resulting drift meets Assumption 4.1.Source: §2.5 · p. 9, §2.5.2 · p. 11

2.6 Invariance and nonreversibility

• The target π is invariant when the dynamics are well posed, and the generator decomposes into a symmetric part S and an antisymmetric part A.The reflected case additionally assumes the stated boundary condition.Source: §2.6 · p. 11
• When α ≠ 0 and c is not identically zero, the process is nonreversible; strong Feller and topological irreducibility imply that π is its unique invariant probability measure.Source: §2.6 · p. 11
• Unlike the canonical stream choice, the general construction can retain a nonzero antisymmetric drift when U0 is constant, which is useful for zeroth-order anchored samplers.For the canonical choice, the antisymmetric drift vanishes when U0 is constant.Source: §2.6 · p. 12

2.7 Random time change

• The random-time change uses reference non-reversible Langevin dynamics on Rd or a normally reflected domain K, whose invariant measure is π0.The representation applies to the compact reflected domain and the Student family in Corollary 6.2.Source: §2.7 · pp. 12–13
• If Ts tends to infinity almost surely, the time-changed process has generator Lα and solves the anchored dynamics, with invariant measure given by the stated time-change relation.The generator is multiplied by a, and the invariant speed measure is multiplied by a^-1.Source: §2.7 · p. 12
• The clock-divergence condition is automatic in the common positive-recurrent case when Z0 is finite and the reference diffusion is ergodic.Source: §2.7 · p. 13

3 Non-Asymptotic Convergence Analysis

• The comparison assumes a positive spectral gap for the reversible anchored dynamics, λ0 > 0.This is Assumption 3.1.Source: §3 · p. 13

3.2 Constrained case

• On bounded domains, the convergence bound is uniform over the starting point x ∈ K.The paper attributes uniformity in the initial point to boundedness of K.Source: §3.2 · p. 15

4.2 The compact reflected case

• On a compact reflected domain, the empirical-measure rate function decomposes into the reversible rate plus a non-negative term, implying acceleration.The result assumes the bounded, connected, regular domain and coefficient conditions of Assumption 4.1.Source: §4.2 · p. 18
• The compact-domain ordering of level-two rate functions is preserved under contraction, improving the large-deviation rate for every continuous ergodic observable.For the occupation-measure contraction, strict inequality holds when every reversible minimizer satisfies Av ≠ 0 in H−1.Source: §4.2 · pp. 19–20

4.3 The unconstrained case

• For the unconstrained LDP, sufficient conditions are stated using the differentiable anchor U0 rather than the possibly non-differentiable target potential U.Dissipativity of U and boundedness of a alone do not control ∇U0; a suitable sublinear bound on ∇log a can transfer dissipativity to the stated condition.Source: §4.3 · pp. 21–22
• On Rd, the empirical-measure LDP and rate-function comparison hold under Assumption 4.6 in the weighted topology τκ.Unlike the compact case, exponential tightness is not automatic on Rd; Proposition 4.7 supplies sufficient conditions for the canonical perturbation.Source: §4.3 · pp. 20, 21, 23
• With the canonical perturbation ψ = e−U0, Witten–Lyapunov estimates based on the stated W are uniform in α.The corresponding compact-level-set condition and three inequalities hold for Lα exactly when they hold for the symmetric part, with the same constants.Source: §4.3 · p. 23

Asymptotic variance reduction

• Under the weighted Poincaré, skew-adjointness, and validity assumptions, Theorem 5.1 gives an asymptotic-variance comparison for every centered observable g ∈ L2_0(π).Equality holds precisely when the reversible Poisson solution is not moved by the circulation under the stated operator assumptions.Source: §5 · p. 24

Compact reflected variance reduction

• On compact reflected domains satisfying Assumption 4.1, the stationary diffusion has a central limit theorem and asymptotic variance cannot increase when tangential circulation is added.The corollary establishes the weighted Poincaré and strong sector conditions for every g ∈ L2_0(π).Source: §5 · p. 25

Compact acceleration with a state-dependent tensor

• For compact reflected domains, the variance and empirical-measure comparisons also apply to the state-dependent tensor construction when its stated regularity and well-posedness conditions hold.The result gives invariant measure π and the rate-function comparison; strictness follows from the cited rate-function and variance conditions.Source: §5 · p. 26

Asymptotic variance reduction

• Increasing |α| damps precisely the components of the reversible Poisson solution that the circulation moves.This is the explicit spectral interpretation when G is compact.Source: §5 · p. 27
• Continuous-time variance reduction does not guarantee lower computational cost per unit time, since large |α| can make explicit discretization stiff and require smaller stepsizes.The numerical comparisons use a fixed number of evaluations of U, ∇U0, and ∇ψ, and report discretization bias and effective sample size.Source: §5 · p. 27

6.1 The anchored heavy-tailed class

• Under the stated regularity and tangency conditions, the heavy-tailed diffusion is nonexplosive and has a unique invariant probability measure.Source: §6.1 · p. 28
• For r ≥ 0, the symmetric generator satisfies a weighted Poincaré inequality; when r > 0, the principal coercive condition also permits admissible weights κ = o(1 + |x|^r).With the auxiliary condition and coefficient assumptions, Theorem 4.9 gives rate-function acceleration.Source: §6.1 · p. 28

Then (6.4) becomes

• For superquadratic Student clocks, a spherical Poincaré inequality controls the purely angular current and supplies a strong sector bound despite an unavailable pointwise coefficient bound.Source: Then (6.4) becomes · pp. 30–31
• For Student-t targets with ζ > 1, the paper establishes both weighted-topology empirical-measure LDP acceleration and asymptotic variance reduction; at ζ = 1, its LDP proof does not apply.The quadratic-clock endpoint still supplies coercivity for the asymptotic-variance comparison.Source: Then (6.4) becomes · pp. 31–32

6.2 A polynomial-tailed example with explicit variance

• For the quadratic-clock polynomial-tailed example, the drift closes on linear observables, yielding an explicit variance comparison that is strict exactly when Ju ≠ 0.Variance is unchanged in directions in ker J.Source: §6.2 · pp. 32–33
• The polynomial-tailed example uses a weighted Poincaré inequality because its reversible diffusion need not have a standard spectral gap.The anchored Dirichlet form has the growing weight a(x) = 1 + |x|².Source: §6.2 · p. 33

7.1.1 A heavy-tailed target with a non-smooth regularizer

• For heavy-tailed, anisotropic, non-differentiable targets, anchored construction replaces vanishing or nonexistent target gradients with smooth-anchor gradient; constant skew-symmetric matrix accelerates its linear part, while state-dependent clock restores correct target.Source: §7.1.1 · p. 36

7.2.1 Truncated elliptical and ℓ1-Laplace distributions

• At 3×floor, NRALD iterations fell from 381 to 259 on target (A) and 469 to 253 on target (B) as s rose from 8 to 16.The gain increased with s; once converged, stationary W1 levels were statistically similar across schemes. The tangential tensor adds no normal component near the boundary, leaving projection error unchanged.Source: §7.2.1 · pp. 37, 39, Figure 2 · p. 38

7.2.2 Bayesian logistic regression: Synthetic data

• At 100 iterations, NRALD test accuracy was 0.72 versus 0.56 with the smoothed ℓ1 ball and 0.68 versus 0.58 with the ball constraint.Both algorithms reached the same accuracy level by 1000 iterations.Source: §7.2.2 · p. 40, Figure 5 · p. 41, Figure 6 · p. 41

7.2.3 Bayesian logistic regression: Real data

• NRALD consistently reached each classification-accuracy level in fewer iterations on Telescope and Titanic under both constraint sets.At 50 iterations, Telescope test accuracy was 0.71 versus 0.55 (ball) and 0.75 versus 0.59 (smoothed ℓ1); at 150 iterations, Titanic accuracy was 0.72 versus 0.58 and 0.73 versus 0.54, respectively.Source: §7.2.3 · p. 42, Figure 7 · p. 43, Figure 8 · p. 43
• By run end, both algorithms reach the Bayesian logistic-regression fit’s accuracy and agree within 0.005, consistent with their shared invariant distribution.Source: §7.2.3 · p. 45
• During the transient, the non-reversible algorithm exhibits a much smaller spread across replicates.Source: §7.2.3 · p. 45

8 Conclusion

• NALD for Euclidean space and NRALD for bounded domains use a divergence-free weighted current to preserve the target while allowing a non-differentiable target potential.The dynamics are also represented as random time changes of non-reversible reference diffusions.Source: §8 · p. 45
• The analysis establishes finite-time total-variation bounds, level-two empirical-measure large deviation principles, and reduced asymptotic variance under the anti-symmetric perturbation.The paper verifies unbounded-space assumptions for heavy-tailed anchored diffusions, including a superquadratic Student-t model, and illustrates variance reduction with a quadratic-clock example.Source: §8 · p. 45
• Non-asymptotic convergence guarantees for discretized NALD and NRALD are left for future work because state-dependent diffusion and, for NRALD, reflection complicate the analysis.Source: §8 · p. 45