Source-linked AI summary
Ornstein-Uhlenbeck Is Hard to Beat, Yet Superlinear Drift Ships Lower Transport Costs
Attila Lovas, Lóránt Nagy
TL;DR
Prior work characterizes Ornstein–Uhlenbeck diffusion as hard to beat in forward convergence under at-most-linear drift, leaving superlinear drift outside that stated setting. This paper tests superlinear Langevin diffusions for score-based image generation using conditional scores computed numerically from a Fokker–Planck equation. Across nearly all tested comparisons, the superlinear models have lower empirical Wasserstein–1 distance than the linear baseline, qualifying the “hard to beat” characterization.
Problem
Prior work's “hard to beat” characterization concerns forward convergence under at-most-linear drift, while the experiments here examine superlinear drift in image generation.
Source: §6 · p. 5Method
Results
Superlinear choices α = 2 or α = 3 have lower empirical Wasserstein–1 distance than the linear baseline in 23 of 24 fixed (T, N) comparisons, with less variation across T.
Source: §6 · p. 5Takeaways & Limitations
The findings qualify the “hard to beat” characterization of Ornstein–Uhlenbeck diffusion: it does not universally describe empirical generation error for the superlinear models tested here.
Source: §6 · p. 5The time-reversal argument assumes finite relative entropy; for bounded-support data, the paper says this can be enforced with arbitrarily small Gaussian regularization.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Bre\v{s}ar and Mijatovi\'c \cite{bresar2025} show that Ornstein--Uhlenbeck diffusion is hard to beat in forward convergence under assumptions that exclude superlinear drift. We instead test superlinear Langevin diffusions for score-based image generation, computing their conditional scores numerically from a Fokker--Planck equation. In our experiments, the superlinear models beat the Ornstein--Uhlenbeck baseline on empirical Wasserstein distance across nearly the entire tested grid and show less variation across diffusion horizons. The ``hard to beat'' verdict of \cite{bresar2025} thus fails to be universal.
1 Introduction
Diffusion generation corrupts data forward and transports a simple terminal law back toward it. This experiment replaces the usual linear mean-reverting drift with a superlinear Langevin drift and computes conditional scores numerically.
1 Introduction
- Diffusion generation progressively destroys data structure in a forward process, then transports a simple terminal law back toward the data distribution.The passage situates this principle in diffusion and score-based models [10], [7], [11], with the reverse-time formula classical.Source: §1 · p. 1
- The experiment replaces the linear mean-reverting drift with a coordinatewise superlinear Langevin drift and uses numerically computed transition scores as regression targets.The scores are computed from the associated Fokker–Planck equation.Source: §1 · p. 1
- The construction needs neither an analytical corrupted-data marginal nor an analytical conditional transition density.For sufficiently regular transition densities, conditional scores can instead be approximated by solving the Fokker–Planck equation.Source: §1 · p. 1
2 Forward diffusion
The forward process is a gradient Langevin diffusion, with coordinatewise scalar dynamics and a conditional transition density that factorizes across coordinates.
2 Forward diffusion
- The forward process is specified as a gradient Langevin diffusion evolving over the interval [0, T].Its displayed dynamics are dX_t = B_α(X_t) dt + dW_t, with X_0 distributed as µ⋆.Source: §2 · p. 2
- The forward process has an invariant probability measure.The supplied passage identifies it as the invariant measure of process (1), without giving its expression.Source: §2 · p. 2
- For a scalar initial value y0, q_y0(t, y) denotes the transition density of the scalar diffusion.Source: §2 · p. 2
- The conditional transition density of the full process factorizes across coordinates.The passage introduces this factorization after specifying the coordinatewise dynamics.Source: §2 · p. 2
3 Time reversal for the superlinear family
The section establishes time reversal using a stationary reference diffusion with the same Langevin dynamics and an invariant initial law. Under the stated conditions, the reference is reversible and the time-reversal theorem applies.
3 Time reversal for the superlinear family
- The reference process Y follows the same Langevin dynamics as X but starts from an invariant law.The path laws of X and Y are denoted P and R, respectively.Source: §3 · p. 2
- R is the reversible diffusion associated with the invariant measure; its drift is Bα(x) = −1/2 a∇Vα(x), where Vα(x) = 2Uα(x) + log Zα and ∇Vα(x) = 2∇Uα(x).Source: §3 · p. 2
- The time-reversal theorem applies when the reference diffusion satisfies nondegeneracy, reversibility, and growth conditions and P has finite relative entropy with respect to R.The text states that the finite-entropy assumption can be enforced by arbitrarily small Gaussian regularization of bounded-support data, including normalized images.Source: §3 · p. 3
4 Reverse diffusion and the score
The reverse process requires the marginal score, which is generally unavailable in closed form. A network is trained on conditional scores to approximate that full-dimensional marginal score.
4 Reverse diffusion and the score
- The marginal law at time t generally retains dependencies between image coordinates because the initial data distribution contains dependencies.Its density is obtained by integrating the conditional transition density over the initial data distribution.Source: §4 · p. 3
- The network approximates the full d-dimensional marginal score, using the conditional score as its regression target.The marginal score depends on the posterior distribution of the original image given the corrupted image.Source: §4 · p. 4
- Under the usual regularity assumptions, the population minimizer of objective (6) is the marginal score required by the reverse dynamics [11].The reverse-time equation's unknown quantity is the marginal score.Source: §4 · pp. 3–4
5 Score construction and numerical implementation
The method constructs conditional transition scores by numerically solving a scalar Fokker–Planck equation, then uses them to train and evaluate reverse diffusions.
5 Score construction and numerical implementation
- The conditional score is computed from the scalar transition density by solving its Fokker–Planck equation, rather than requiring a closed-form density for nonlinear drift choices.The transition density is approximated numerically for nonlinear α.Source: §5 · p. 4
- On a finite grid spanning starting values y0, finite differences of the log-density yield conditional scores stored in a (y0, t, y)-indexed table, with training scores obtained by trilinear interpolation.A narrow Gaussian approximates the initial point mass.Source: §5 · p. 4
- Experiments use 28 × 28 grayscale images and test α ∈ {1, 2, 3}, T ∈ {3, …, 10}, and N ∈ {100, 150, 200}.For α = 1, the forward process is an Ornstein–Uhlenbeck diffusion with invariant law N(0, I_d).Source: §5 · p. 4
- A U-Net is trained on one uniformly selected nonzero trajectory time per image, and reverse sampling uses Euler–Maruyama from an approximated invariant law.Explicit Euler can produce rare large excursions for superlinear drifts; numerical explosions are monitored and images with over ten percent non-finite coordinates are discarded.Source: §5 · p. 4
- The code for the numerical experiments is publicly available.The passage provides the project repository URL.Source: §5 · p. 4
6 Evaluation and results
Across the tested comparisons, superlinear models usually have lower empirical Wasserstein–1 distance than the linear baseline. The paper frames this as a qualification of the Ornstein–Uhlenbeck “hard to beat” characterization, whose cited result concerns forward convergence.
6 Evaluation and results
- Generated images are evaluated by empirical Wasserstein–1 distance with Euclidean ground cost, using 3,000 real images and a discrete optimal-transport computation.Real and generated 28 × 28 images are mapped to [0, 1]^d and flattened.Source: §6 · p. 5
- The experiments qualify the “hard to beat” characterization of the Ornstein–Uhlenbeck process in, which concerns forward convergence under at-most-linear drift rather than empirical generation error.The row summaries also show less variation across T; stronger mean reversion is conjectured to contribute to the improved scores and reduced sensitivity.Source: §6 · p. 5
- Superlinear choices (α = 2 or α = 3) have lower empirical Wasserstein–1 values in 23 of 24 fixed (T, N) comparisons, with the exception N = 100, T = 8.The best value in each table is also attained by a superlinear model.Source: §6 · p. 5, Table 1 · p. 6, Table 2 · p. 6