Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective

Erkan Turan; Nicolas Dufour; Maks Ovsjanikov

Source-linked AI summary

Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective

Erkan Turan, Nicolas Dufour, Maks Ovsjanikov

arXiv:2603.09936v2Mar 10, 2026cs.LG

TL;DR

The paper addresses underexplored mathematical questions about drifting, including identifiability, kernel choice, and stop-gradient stability. It analyzes the Gaussian drift through score matching and McKean–Vlasov dynamics, derives variational and spectral explanations, and reports an annealing schedule that improves ImageNet FID while preserving the Gaussian kernel’s theoretical structure.

Problem
Drifting’s mathematical structure remains underexplored, including whether stop-gradient is theoretically essential and how foundational properties should be established.
Method
The paper analyzes Gaussian drifting as a score difference, studies its prescribed-velocity McKean–Vlasov dynamics spectrally, and constructs a Sinkhorn-divergence drift from an entropy-regularized optimal-transport energy.
Results
The analysis establishes identifiability, explains Gaussian high-frequency suppression, derives the JKO-mandated frozen-field stop-gradient, and reduces convergence time from exp(O(Kmax^2)) to O(log Kmax) with exponential bandwidth annealing.
Takeaways & Limitations
Kernel choice controls both theoretical structure and convergence dynamics, while annealing empirically reconciles Gaussian analytical convenience with improved high-frequency behavior.
Takeaways & Limitations
The Landau-damping diagnosis is local to small perturbations around homogeneous equilibrium, and a fully nonlinear treatment is left for future work.

Abstract

from arXiv · show

Generative Modeling via Drifting~\citep{deng2026drifting} has recently achieved state-of-the-art one-step image generation through a kernel-based drift operator, yet its success is largely empirical and its theoretical foundations remain poorly understood. We observe that \emph{under a Gaussian kernel, the drift operator is exactly a score difference on smoothed distributions}. This answers three questions left open in the original work: (1) whether a vanishing drift guarantees equality of distributions ($V_{p,q}=0\Rightarrow p=q$), (2) how to choose between kernels, and (3) why the stop-gradient operator is indispensable for stable training. Our observations position drifting within the score-matching family. By linearizing the McKean-Vlasov dynamics and probing them in Fourier space, we reveal frequency-dependent convergence timescales comparable to \emph{Landau damping} in plasma kinetic theory: the Gaussian kernel suffers an exponential high-frequency bottleneck, potentially explaining the empirical preference for the Laplacian kernel. This suggests a fix: an exponential bandwidth annealing schedule $σ(t)=σ_0 e^{-rt}$ that reduces convergence time from $\exp(O(K_{\max}^2))$ to $O(\log K_{\max})$. Finally, by formalizing drifting as a Wasserstein gradient flow of the smoothed KL divergence, we prove that the stop-gradient operator is not a heuristic but is derived from the frozen-field discretization mandated by the Jordan-Kinderlehrer-Otto (JKO) scheme, and removing it severs training from any gradient-flow guarantee. This variational perspective further provides a general template for constructing novel drift operators, which we demonstrate with a Sinkhorn divergence drift. We validate our analysis on toy datasets and scale it up to ImageNet.

1 Introduction

The paper reframes drifting as score matching under Gaussian kernels and uses this identity to address identifiability, kernel selection, and stop-gradient stability. Spectral and variational analyses yield convergence insights, an annealing schedule, and a broader drift-construction template.

Motivation: Generative Modeling via Drifting achieves one-step image generation with a kernel-based drift, but its mathematical structure and three foundational questions remain underexplored.The open questions concern identifiability, kernel selection, and the theoretical necessity of stop-gradient.
Core identity: Under the Gaussian kernel, the drift operator is exactly a score difference on smoothed distributions, placing drifting within the score-matching family.The identity is derived by direct kernel substitution and is numerically confirmed in a 4-mode Gaussian mixture.
Theoretical analysis: Identifiability follows from Fourier injectivity, while Fourier-space analysis of linearized McKean–Vlasov dynamics gives principled criteria for kernel selection.The analysis finds exponential slowdown for the Gaussian kernel and polynomial slowdown for the Laplacian kernel.
Variational structure: Drifting is the JKO-discretized Wasserstein gradient flow of a smoothed KL divergence, making stop-gradient the required frozen-field structure rather than a heuristic.Removing stop-gradient severs the training procedure from the associated descent guarantee.
Algorithmic improvements: An exponential bandwidth schedule σ(t) = σ0e−rt reduces convergence time from exp(O(Kmax^2)) to O(log Kmax) while retaining Gaussian identifiability.The same variational template also yields new drift operators, including a Sinkhorn-divergence drift.

2 Background

Score matching underlies continuous generative modeling, while diffusion and flow models learn dynamics used during inference. Drifting instead shifts its dynamics to training.

Score matching and Energy-Based Models use score functions to avoid intractable normalization and support generation.
Denoising score matching replaces unavailable data scores with scores of Gaussian-perturbed distributions.
Diffusion models connect noise prediction to perturbed-data scores through Tweedie’s formula and reverse-time stochastic dynamics.
Flow models learn velocity fields and generate samples by integrating an ordinary differential equation.
Unlike diffusion and flow models, drifting moves its prescribed dynamics from inference to training.

3 Drifting Models

Drifting trains a generator with a kernel-based operator that attracts generated samples toward data and repels them from one another. Its original formulation left identifiability, kernel choice, and stop-gradient necessity unresolved.

The drift operator combines attraction toward nearby data with repulsion among generated samples to prevent mode collapse.
During training, generated samples are mapped to target states by adding their drift, while the data distribution enters through the operator.
The generator is trained with a stop-gradient loss to match its drifted outputs.
The original work established q = p implies V_p,q = 0 but left the converse, kernel selection, and the necessity of stop-gradient open.

4 Drifting is Score Matching

Under a Gaussian kernel, drifting’s operator is exactly a difference between scores of smoothed data and generated distributions. This identity places drifting within score matching and enables its subsequent identifiability, spectral, and variational analyses.

Under the Gaussian kernel, the drift operator has a closed-form representation as a score difference for smoothed distributions.
The identity positions drifting within the score-matching family and provides the basis for the paper’s later analyses.
Classical denoising score matching learns a smoothed data score, whereas drifting computes the score difference non-parametrically from current particles during training.
Drifting uses a generator pushforward for one-step sampling, so the score computation disappears at inference.
The prescribed Gaussian-kernel velocity yields a McKean–Vlasov equation that is amenable to stability analysis.

5 Theoretical Consequences of the Identity

The score identity resolves drifting’s theoretical questions through Gaussian-convolution identifiability, Fourier analysis of kernel-dependent convergence, and a Wasserstein gradient-flow interpretation. It also motivates bandwidth annealing and explains why stop-gradient is structurally required.

5.1 Identifiability: Gaussian convolution is injective, so a vanishing Gaussian drift implies equality of the data and generated distributions.
5.2 Kernel diagnosis: An exponential bandwidth schedule reduces total convergence time to logarithmic dependence on K_max and gives the fastest spectral error reduction across modes.
5.2 Kernel diagnosis: Fourier linearization gives mode-dependent convergence rates governed by the kernel’s spectral response.
5.2 Kernel diagnosis: The Gaussian spectral relation is analogous to Landau damping because mode decay is controlled by the medium’s spectral properties.
5.2 Kernel diagnosis: The Gaussian kernel has an exponential high-frequency slowdown, whereas the Laplacian kernel has polynomial scaling.
5.2 Kernel diagnosis: The Gaussian bottleneck is intrinsic to the kernelized particle dynamics and persists even with a perfect function approximator.
5.3 Stop-gradient necessity: Drifting is a JKO-discretized Wasserstein gradient flow of a smoothed KL functional, with well-posedness, monotone energy descent, and convergence as τ → 0.
5.3 Stop-gradient necessity: Stop-gradient implements the required frozen-field explicit Euler step; removing it permits drift collapse and severs the descent guarantee.

6 From Theory to Practice

The paper turns its theoretical analysis into practical choices: exponential bandwidth annealing addresses Gaussian high-frequency slowdown, while a variational template produces Sinkhorn drift operators. Experiments show these constructions converge on toy data and improve or match kernel baselines on ImageNet.

Spectral fixes: Exponential bandwidth annealing reduces convergence-time dependence on K_max from exponential for fixed-bandwidth Gaussian drift to logarithmic.The schedule continuously sweeps the optimal-rate window across frequencies; experiments confirm fastest spectral error reduction across modes.
Variational operators: A general variational template V = −∇(δF/δq) extends drift construction beyond the original kernel family.Any sufficiently regular nonnegative discrepancy satisfying the stated well-posedness and identifiability conditions inherits the JKO, frozen-field, and stop-gradient guarantees.
Variational operators: Sinkhorn divergence yields a closed-form drift whose attractive and repulsive terms arise from entropic optimal transport rather than hand-chosen kernel antisymmetry.Its operator retains the JKO–frozen-field–stop-gradient guarantees and converges comparably to Laplacian drift on checkerboard data.
ImageNet experiments: Fixed Gaussian drift remains competitive with Laplacian drift, with FID 8.69 versus 8.55 and IS 157.0 versus 148.0.The Gaussian combines a clean score-matching identity and identifiability guarantees with strong empirical performance.
ImageNet experiments: On ImageNet, exponentially annealed Gaussian drift achieves FID 8.36 versus 8.69 for fixed Gaussian and 8.55 for Laplacian baselines.All variants use one-step generation; the reported result supports transferring the spectral diagnosis beyond its linearized toy setting.
ImageNet experiments: Sinkhorn-divergence drift reaches FID 8.81 on ImageNet, comparable to the kernel-based operators.This result supports the constructive value of the gradient-flow template beyond kernel mean-shift.

7 Related Work

The related work situates drifting among score-matching, spectral-analysis, and optimal-transport approaches while distinguishing this paper’s mode-resolved kernel-dynamics analysis and variational construction.

Score matching: Score matching has unified denoising, diffusion, and flow-matching methods under score estimation.Prior work connects denoiser training, diffusion noise prediction, and flow matching through score-based formulations.
Spectral methods: This paper differs from network-level spectral-bias studies by identifying timescales intrinsic to kernelized particle dynamics.The rates persist with a perfect approximator and originate in the kernel dispersion relation.
Spectral methods: Unlike analyses yielding only a single global convergence rate, this work recovers mode-resolved timescales.The comparison is made with exponential KL convergence results for continuous SVGD.
Optimal transport: The paper uses the JKO framework to interpret drifting as an explicit-Euler discretization of Wasserstein gradient flow.Optimal transport is used here primarily as an analytical framework for understanding drifting.

8 Conclusion

The paper identifies Gaussian drifting with score differences on smoothed distributions and uses this foundation to address identifiability, spectral convergence, stop-gradient training, and operator design. It proposes annealing for Gaussian bottlenecks and a variational route to Sinkhorn drift.

Core conclusions: Under a Gaussian kernel, drifting computes a score difference on smoothed distributions.This identity places drifting within the score-matching family and supports the paper’s subsequent analyses.
Core conclusions: Fourier injectivity establishes identifiability, while Landau-damping analysis explains Gaussian high-frequency suppression and motivates exponential annealing.The annealing schedule reduces convergence-time dependence from exp(O(K_max^2)) to O(log K_max).
Core conclusions: The stop-gradient operator is the frozen-field discretization required by the JKO scheme rather than a heuristic.Removing it produces drift collapse and lowers the loss without transporting mass toward the data distribution.
Core conclusions: The template V = −∇(δF/δq) generalizes drift construction beyond the original kernel family through a Sinkhorn divergence drift.This provides a modular variational basis for constructing new operators.

9 Limitations and Future Work

The paper identifies limits in the scope of its spectral theory and outlines future directions involving kernels and alternative Wasserstein discretizations. Its spectral conclusions are local, while broader guarantees and faster schemes remain open.

Scope of spectral analysis: The Landau-damping diagnosis is local because it linearizes around equilibrium under a homogeneous-background approximation.Its predicted timescales hold strictly in the small-perturbation regime; a fully nonlinear treatment is left for future work.
Kernel design: The analysis exposes a trade-off between Gaussian analytical structure and Laplacian high-frequency resolution.Future work could design kernels with prescribed spectral profiles while retaining Gaussian-style analytical scaffolding.
Alternative discretizations: JKO supplies one first-order explicit-Euler discretization, leaving higher-order, splitting, and accelerated Wasserstein schemes unexplored.The paper proposes systematic study of these alternatives as a route to faster or more stable training.

B.3 Proof of Theorem 5.2

The proof diagonalizes the linearized drifting dynamics in Fourier space, showing independent exponential decay of each mode with a kernel-dependent convergence timescale. It then connects the variational construction to JKO discretization and establishes energy descent toward the gradient-flow PDE.

Fourier-space dynamics: Each Fourier mode evolves independently under a scalar decay equation determined by the effective kernel spectrum.The decay rate is λκ(ξ)=cκ|ξ|2κ̂(ξ), with timescale τκ(ξ)=1/[cκ|ξ|2κ̂(ξ)].
Kernel specialization: For the exponential kernel, the effective convolution kernel differs from the original kernel but still fits the theorem’s linearized form.The proof identifies the effective kernel and applies Theorem 5.2 with cκ=1 and κ=κτ.
Gaussian bottleneck: At high frequencies, the Gaussian kernel’s slowest mode occurs at |k|=Kmax, producing the bottleneck governing convergence time.The slowest-decaying mode in the band |k|≤Kmax determines the required damping time.
Variational structure: The smoothed KL functional has a first variation that generates the drift velocity, making the drifting PDE its Wasserstein gradient flow.The velocity is the negative spatial gradient of the functional derivative, and the resulting PDE evolves the density through that velocity field.
JKO discretization: The JKO scheme converges as τ→0 to the gradient-flow PDE, while its frozen-field explicit Euler approximation is implemented by stop-gradient training.The stop-gradient objective therefore corresponds to a specific discretization rather than an ad hoc optimization device.
Energy behavior: Under the stated density bounds, the smoothed KL energy is continuous under narrow convergence and nonincreasing along the limiting flow.The limiting solution satisfies Fσ[q(t)]≤Fσ[q(s)] for 0≤s<t≤T.

D.3 Proof of Consistency: ie, Equation 5.3 (Implicit–Explicit Consistency)

The consistency proof compares the implicit JKO update with a frozen-field explicit Euler step through pushforward representations and Wasserstein stability. It then shows that stop-gradient training realizes the explicit transport step, whereas differentiating through the drift changes the optimization problem.

Implicit–explicit consistency: The explicit approximation freezes the velocity field at the current iterate, converting the implicit JKO update into a tractable Euler step.The update is analyzed as a transport map applied to the current distribution.
Pushforward comparison: Both implicit and explicit updates are rewritten as pushforwards of a common base measure for direct Wasserstein comparison.The proof uses pushforward stability and approximation of the inverse map to control their discrepancy.
Consistency bound: Under the stated regularity condition that the limiting velocity is Lipschitz, a first-order expansion supports the consistency estimate.The proof combines Taylor expansion, JKO optimality, bounded energy, and a final Wasserstein estimate.
Stop-gradient correspondence: With stop-gradient and a realizable target, minimizing the regression loss produces the frozen-field map S_n(x)=x+τvσ[qθn](x).Pushing forward the base noise distribution yields exactly the frozen-field explicit Euler step.
Necessity of stop-gradient: Removing stop-gradient introduces a distribution-feedback gradient term, so the target velocity changes with the model parameters.The resulting coupled objective is not regression toward a fixed explicit Euler target and can admit drift-collapse stationary points.

E Schedule Ablation

The schedule ablation studies Gaussian, exponential, linear, and cosine bandwidth schedules alongside alternative drift constructions. The experiments support the Fourier analysis: fixed Gaussian smoothing has a high-frequency bottleneck, while exponential annealing accelerates convergence and Sinkhorn drift provides a viable variational alternative.

Schedule comparison: The exponential schedule converges fastest across tested modes when sweeping σ from 1.5 to 0.03.It reaches the activation condition σ2|k|2=2 earliest for each k, matching the activation-time analysis.
Spectral convergence: Fixed Gaussian smoothing exhibits exponential slowdown beyond k*=2/σ≈4.7, whereas the exponential kernel has polynomial scaling.The annealed Gaussian schedule eliminates the high-frequency bottleneck in the reported spectral experiment.
Toy validation: With stop-gradient, mean drift magnitude and sliced Wasserstein distance are strongly correlated and jointly approach zero on Swiss roll and checkerboard targets.The reported log-log correlation exceeds 0.95 across three seeds, with final SW=0.016.
Sinkhorn drift: Sinkhorn divergence satisfies the stated energy conditions and yields a drift with JKO, frozen-field, and stop-gradient guarantees.The construction uses the template V=−∇x(δF/δq) for the Sinkhorn energy.
Sinkhorn validation: On checkerboard, exponential-kernel and Sinkhorn drifts reach final sliced Wasserstein distances of 2.07 × 10−2 and 1.42 × 10−2, respectively.Both operators successfully transport the generator to the target.

G.6 Compute, memory, and implementation notes

The implementation study compares compute costs across drift variants, finding similar overhead for kernel methods and scale-dependent behavior for Sinkhorn drift.

All ablations use 16 NVIDIA H100 GPUs, and a full 30,000-step run takes roughly 10–11 wall-clock hours.The reported duration depends on the loss.
All drift variants share O(B^2d) pairwise-distance cost, while Sinkhorn additionally performs S iterations on two entropic coupling matrices.
Kernel variants remain within 2–4% of one another across shapes, so bandwidth scheduling and kernel choice add negligible overhead to shared pairwise-distance computation.
At (B, d)=(64, 256), Sinkhorn is approximately 32% slower than the softmax baseline because fixed Sinkhorn-sweep overhead dominates at small batch size.
At (B, d)=(1024, 256), Sinkhorn is approximately 1.65× faster than the kernel variants, after its overhead is amortized.At (256, 256), it is already on par: 1.04 vs. 1.21 ms.