PaperlayerView exact revision on arXiv

Source-linked AI summary

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

Tim Dockhorn, Arash Vahdat, Karsten Kreis

arXiv:2112.07068v4stat.MLcs.LG

TL;DR

Score-based generative models depend critically on their diffusion process, but existing choices can make denoising unnecessarily difficult. This paper introduces critically-damped Langevin diffusion with velocity variables, a tailored score-matching objective, and a specialized sampler, outperforming previous SGMs and Euler–Maruyama on image synthesis and sampling.

  • Problem

    The diffusion process determines score-learning complexity in SGMs, motivating alternatives to existing diffusion designs.

    Source: §1 · p. 1

  • Method

    The paper introduces CLD, which couples data with auxiliary velocity variables, trains via conditional velocity-data score matching, and uses a tailored SDE integrator.

    Source: §1 · p. 2

  • Results

    CLD-based models outperform previous SGMs in image synthesis at similar model capacity and sampling budgets, while SSCS outperforms Euler–Maruyama.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    CLD provides a diffusion framework with smoother, easier-to-denoise dynamics and efficient sampling for score-based image synthesis.

    Source: §6 · p. 9

    The empirical validation focuses on image synthesis benchmarks.

    Source: §7 · p. 10

Abstract

from arXiv · show

Score-based generative models (SGMs) have demonstrated remarkable synthesis quality. SGMs rely on a diffusion process that gradually perturbs the data towards a tractable distribution, while the generative model learns to denoise. The complexity of this denoising task is, apart from the data distribution itself, uniquely determined by the diffusion process. We argue that current SGMs employ overly simplistic diffusions, leading to unnecessarily complex denoising processes, which limit generative modeling performance. Based on connections to statistical mechanics, we propose a novel critically-damped Langevin diffusion (CLD) and show that CLD-based SGMs achieve superior performance. CLD can be interpreted as running a joint diffusion in an extended space, where the auxiliary variables can be considered "velocities" that are coupled to the data variables as in Hamiltonian dynamics. We derive a novel score matching objective for CLD and show that the model only needs to learn the score function of the conditional distribution of the velocity given data, an easier task than learning scores of the data directly. We also derive a new sampling scheme for efficient synthesis from CLD-based diffusion models. We find that CLD outperforms previous SGMs in synthesis quality for similar network architectures and sampling compute budgets. We show that our novel sampler for CLD significantly outperforms solvers such as Euler--Maruyama. Our framework provides new insights into score-based denoising diffusion models and can be readily used for high-resolution image synthesis. Project page and code: https://nv-tlabs.github.io/CLD-SGM.

1 INTRODUCTION

The paper argues that diffusion determines SGM learning complexity and introduces critically-damped Langevin diffusion (CLD), which augments data with coupled velocity variables and enables easier score learning and tailored sampling.

Motivation

  • The forward diffusion uniquely determines the score-learning problem’s complexity, making diffusion design central to improving synthesis quality and sampling speed.SGMs otherwise use diffusion to transform data into an analytically tractable prior and reverse it through iterative denoising.Source: §1 · p. 1

Method

  • CLD augments data with velocity, couples both through Hamiltonian-like dynamics while injecting noise only into velocity, and learns ∇v_t log p_t(v_t|x_t).This joint diffusion produces smoother data trajectories and an arguably easier conditional-score target than directly learning the score of diffused data.Source: §1 · p. 2, Figure 1 · p. 2

Technical contributions

  • The paper derives a CLD score-matching objective, a scalable denoising score-matching method, and a reverse-time SDE integrator tailored for efficient sampling.These contributions connect score-based modeling with techniques from statistical mechanics and molecular dynamics.Source: §1 · p. 2

Empirical validation

  • CLD-based models outperform previous diffusion models on CIFAR-10 for similar network architectures and sampling compute budgets, while the tailored sampler significantly outperforms Euler–Maruyama.The paper attributes the gains to CLD’s Hamiltonian component and easier velocity-conditioned score target.Source: §1 · p. 2
  • Ablations find that CLD does not have difficult-to-tune hyperparameters, and CLD-trained neural networks are smoother than those of previous SGMs.The framework is presented as applicable to high-quality synthesis without adversarial objectives, building on SGMs’ broad generative applications.Source: §1 · pp. 1–2

2 BACKGROUND

This section introduces forward SDEs that perturb data toward an approximately Normal equilibrium and reverse-time SDEs that use learned scores for generation. It also notes how affine dynamics enable analytic conditional distributions and expose quality–likelihood trade-offs.

Forward diffusion

  • A forward diffusion evolves u_t through drift f(u_t,t) and diffusion G(u_t,t) driven by a standard Wiener process over t ∈ [0,T].The dynamics are expressed as du_t = f(u_t,t) dt + G(u_t,t) dw_t.Source: §2 · p. 2

Reverse-time process

  • The corresponding reverse-time diffusion depends on the score ∇_{ū_t} log p_{T−t}(ū_t), the marginal score at reverse time T−t.The reverse process is obtained by defining ū_t := u_{T−t} (Song et al., 2021c).Source: §2 · p. 2

Score-based generation

  • Reverse-time SDEs serve as generative models: data begin from p(u_0)=p_data(x), while simple SDEs drive p(u_T) toward approximately N(0,I_d) and learn scores for synthesis.The forward process relaxes data from a far-from-equilibrium state toward equilibrium, and the learned score is used in the reverse-time SDE (Song et al., 2021c; Kim et al., 2021).Source: §2 · pp. 2–3

Score matching

  • When f and G are affine, p_t(x_t|x_0) is analytically available as a Normal distribution, while λ(t) controls synthesis-quality versus likelihood trade-offs.This setting supports denoising score matching with an analytically tractable conditional distribution (Vincent, 2011; S¨arkk¨a & Solin, 2019; Song et al., 2021b; Vahdat et al., 2021).Source: §2 · p. 3

3 CRITICALLY-DAMPED LANGEVIN DIFFUSION

The paper introduces CLD, an extended-space diffusion coupling data with auxiliary velocities and balancing Hamiltonian and noise dynamics at critical damping. It derives conditional-velocity score matching, a variance-reducing HSM objective, and a structure-aware sampler for CLD-based SGMs.

3 CRITICALLY-DAMPED LANGEVIN DIFFUSION

  • CLD augments data with auxiliary velocities and achieves fastest smooth convergence to equilibrium when Γ^2=4M, balancing Hamiltonian coupling against Ornstein–Uhlenbeck noise.Only velocities receive direct noise; data are perturbed indirectly through coupling, unlike overdamped diffusions that inject noise directly into data variables.Source: §3 · p. 3

3.1 SCORE MATCHING OBJECTIVE

  • The CLD score model learns only ∇_v_t log p_t(v_t|x_t), which is closer to a Normal score and yields simpler, smoother networks than VPSDE score modeling.At t=0, the conditional velocity distribution is the initialized Normal distribution, while the data score can be highly complex or unbounded.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 SCALABLE TRAINING

  • Hybrid score matching analytically marginalizes the Normal initial-velocity distribution, reducing gradient variance versus DSM while avoiding DSM’s unbounded-score instability.The CLD score is parameterized with a mixed Normal-score and residual formulation, predicting only velocity-injected noise during reparameterized sampling.Source: §3.2 · pp. 4–5

3.3 SAMPLING FROM CLD-BASED SGMS

  • The Symmetric Splitting CLD Sampler separates analytically tractable Hamiltonian and Ornstein–Uhlenbeck dynamics from the score term, applying Euler only to the latter.Because CLD’s learned conditional velocity score is near-Normal for much of the diffusion, the score dynamics are often small, making the split scheme potentially more accurate than Euler–Maruyama.Source: §3.3 · pp. 5–6

4 RELATED WORK

CLD connects score-based generative modeling to statistical mechanics, molecular-dynamics thermostats, and auxiliary-variable methods, while differentiating itself from related diffusion, latent-space, sampling, and likelihood approaches. Its SSCS sampler is derived from first principles and avoids data-specific assumptions.

Statistical Mechanics and Molecular Dynamics

  • CLD draws on statistical mechanics and molecular-dynamics thermostats to frame SGM diffusion as rapid, smooth equilibration, with SSCS development borrowing molecular-dynamics techniques.The framework is inspired by the Langevin thermostat and by methods used to derive molecular-dynamics algorithms.Source: §4 · p. 6

Further Related Work

  • CLD improves the diffusion process itself, unlike LSGM, which simplifies data through a two-stage latent embedding requiring additional encoder and decoder networks.Compared with non-Gaussian diffusion methods, CLD also introduces improved sampling schemes and training objectives rather than merely changing the injected noise distribution.Source: §4 · pp. 6–7
  • Existing SGM sampling accelerators generally do not support straightforward log-likelihood estimation and were developed primarily for discrete-step diffusion, unlike CLD’s continuous-time framework.The cited methods include San-Roman et al. (2021), Watson et al. (2021), Kong & Ping (2021), and Song et al. (2021a).Source: §4 · p. 7
  • Unlike GGF’s adaptive Euler-based solver, CLD’s SSCS is derived from first principles, makes no data-specific assumptions, and performs extremely well without adaptive step sizes.GGF refers to “Gotta Go Fast” (Jolicoeur-Martineau et al., 2021a), while CLD uses an SDE integrator designed for its diffusion.Source: §4 · p. 7
  • CLD’s auxiliary velocities relate to augmented normalizing flows, Hamiltonian Monte Carlo, and optimization momentum, accelerating diffusion mixing while making CLD a second-order Langevin method.The paper does not focus on training CLD models toward high likelihood and notes that higher-order schemes could potentially further improve SGMs.Source: §4 · p. 7

5 EXPERIMENTS

Experiments show that CLD-SGMs achieve strong CIFAR-10 image-generation quality and favorable sampling-quality trade-offs, including under limited NFE budgets. Ablations indicate that the method is robust to hyperparameter choices while HSM and mixed-score parametrization materially support training and synthesis.

5 EXPERIMENTS

  • Experiments implement CLD with NCSN++ and DDPM++ using six input channels, β=4, Γ=1, γ=0.04, HSM weighting, and RK4(5), EM, or SSCS sampling.Nonadaptive EM and SSCS use quadratic evaluation times, while probability-flow sampling uses an adaptive Runge–Kutta 4(5) method.Source: §5 · p. 7

5.1 IMAGE GENERATION

  • FID 2.25 with probability-flow ODEs and 2.23 with generative SDEs, while outperforming published results at similar capacity and NFE budgets ≤500.CLD uses approximately 100M parameters, significantly outperforms LSGM-100M, and exceeds other baselines except LSGM and NSCN++/VESDE with 2,000-step predictor-corrector sampling.Source: §5.1 · pp. 7–8, Table 1 · p. 8
  • The model reaches an NLL bound of 3.31, comparable to recent works, although the bound may be loose and the experiments did not optimize training for likelihood.Image quality was measured by FID using 50k CIFAR-10 samples, while NFE counts were recorded for sampling comparisons.Source: §5 · p. 7, §5.1 · p. 8
  • CelebA-HQ-256 samples appear diverse and high-quality despite limited compute and no careful hyperparameter tuning.This demonstrates suitability for high-resolution image synthesis, with additional samples provided in Appendix F.Source: §5.1 · p. 8, Figure 4 · p. 8

5.2 SAMPLING SPEED AND SYNTHESIS QUALITY TRADE-OFFS

  • Across adaptive and fixed-step solvers, CLD-SGM outperforms all baselines except VESDE with 2,000 NFEs, while SSCS surpasses EM under limited NFE budgets.At high NFE, SSCS and EM perform similarly; CLD also outperforms GGF and LSGM-100M in FID.Source: §5.2 · p. 8, Table 2 · p. 9, Table 3 · p. 9

5.3 ABLATION STUDIES

  • Varying the mass parameter and initial velocity variance has mostly small effects, with smaller γ slightly improving FID while modestly worsening the NLL bound.Smaller M^-1 slows noise transfer from velocity to data and motivates time rescaling; the chosen settings were reused without fine-tuning across CIFAR-10 and CelebA-HQ-256.Source: §5.3 · p. 9, Table 4 · p. 9, Table 5 · p. 9
  • Mixed-score parametrization improves FID from 3.56 without mixed score to 3.14 with it.This agrees with the benefit reported for mixed-score training in prior work (Vahdat et al., 2021).Source: §5.3 · p. 9
  • Replacing HSM with regular DSM often destabilizes training, likely because standard DSM produces unbounded scores near t=0, making HSM crucial for CLD-SGMs.The authors therefore regard HSM as a central training component rather than merely an implementation choice.Source: §5.3 · p. 9

6 CONCLUSIONS

The paper presents critically-damped Langevin diffusion (CLD) as a smoother, easier-to-denoise diffusion for score-based generative models, with improved synthesis and a tailored sampler. It also introduces CLD-specific training and integration methods and outlines extensions beyond image generation.

Contributions and findings

  • CLD outperforms previous SGMs for similar-capacity models and sampling compute budgets, while SSCS outperforms Euler–Maruyama in CLD-based SGMs.These results support CLD’s smoother denoising, fast synthesis, and improved expressivity.Source: §6 · p. 9

Technical contributions

  • The work derives HSM, a CLD-specific score-matching objective, and a tailored SDE integrator.HSM is a denoising score-matching variant suited to CLD.Source: §6 · p. 9

Future work

  • Future directions include applying CLD beyond images, accelerating its sampling, adapting it toward maximum likelihood, and exploring other statistical-mechanics thermostating methods.The authors suggest CLD could serve as the backbone diffusion process for next-generation SGMs.Source: §6 · p. 9

7 ETHICS AND REPRODUCIBILITY … A.1 DIFFERENT DAMPING RATIOS

The paper reports no immediate ethical concerns from CLD itself, while noting that image-generation applications can have beneficial or harmful uses. It supports reproducibility through public code and detailed training documentation, and motivates critical damping as the fastest, smoothest Langevin regime.

7 ETHICS AND REPRODUCIBILITY

  • CLD has no immediate ethical concerns, but image synthesis can support creative applications or nefarious uses, so impacts depend on the application domain and task.The paper also notes that SGMs may model data more faithfully than GANs, without mode dropping and problematic biases.Source: §7 · p. 10
  • The authors publicly release code and instructions for reproducing the main results, with training details and hyperparameters documented in Appendix E.The project page and code repository are provided as reproducibility resources.Source: §7 · p. 10

A LANGEVIN DYNAMICS

  • Langevin dynamics can be analyzed through different mass-to-friction ratios that determine the balance between Hamiltonian motion and noise.The appendix revisits the Langevin dynamics formulation from the main paper before comparing damping regimes.Source: A LANGEVIN DYNAMICS · p. 18, §A.1 · p. 18

A.1 DIFFERENT DAMPING RATIOS

  • Underdamping causes oscillatory dynamics that slow convergence, whereas overdamping suppresses Hamiltonian acceleration through strong noise and is also inefficient.These regimes correspond to Γ2 < 4M and Γ2 > 4M, respectively.Source: §A.1 · p. 18
  • Critical damping at Γ2 = 4M balances the competing effects, yielding smooth, non-oscillatory convergence to equilibrium.The critical regime is described as mixing and converging quickly.Source: §A.1 · p. 18
  • Critical damping mixes and converges quickly in the visualized trajectories, while underdamped and overdamped dynamics behave undesirably or inefficiently.The appendix states that critical damping can be shown to be optimal for convergence, citing McCall (2010).Source: §A.1 · p. 18

A.2 VERY HIGH FRICTION LIMIT AND CONNECTIONS TO PREVIOUS SDES IN SGMS … B.3 CLD OBJECTIVE

The appendix connects CLD to prior high-friction diffusions and details its perturbation kernel, equilibrium, and training objectives. It shows that CLD’s velocity-only score matching objective and critical-damping regime provide a shorter, less noise-intensive route to equilibrium than previous data-noise diffusions.

A.2 VERY HIGH FRICTION LIMIT AND CONNECTIONS TO PREVIOUS SDES IN SGMS

  • CLD’s critical-damping regime reaches equilibrium with less injected noise or a shorter diffusion time than previous high-friction diffusions, whose excessive noise makes denoising harder.The VPSDE is identified as the high-friction limit of the general Langevin diffusion, while prior diffusions injecting noise directly into data variables are likewise high-friction.Source: §A.2 · p. 20

B CRITICALLY-DAMPED LANGEVIN DIFFUSION

  • CLD augments data x_t with auxiliary velocities v_t and evolves them jointly through a critically-damped Langevin diffusion in the extended state space.Critical damping is obtained by setting M = Γ^2/4; the derivations also allow time-dependent β(t), although experiments use constant β.Source: §B.1 · p. 20

B.1 PERTURBATION KERNEL

  • Because the CLD drift and diffusion coefficients are affine, Gaussian initial states remain Gaussian, enabling analytical means, covariances, and perturbation kernels.The appendix derives these quantities through ordinary differential equations, including kernels conditioned on both initial data and velocity or on initial data alone.Source: §B.1 · pp. 20–21

B.2 CONVERGENCE AND EQUILIBRIUM

  • CLD converges to the analytically tractable equilibrium p_EQ(u) = N(x; 0_d, I_d) N(v; 0_d, M I_d).This is a special case of Langevin dynamics with equilibrium p_pot(x)N(v; 0_d, M I_d), where the potential is the harmonic-oscillator Gaussian.Source: §B.2 · pp. 21–22

B.3 CLD OBJECTIVE

  • The CLD objective requires gradients only with respect to the velocity variables v_t, yielding a score-matching formulation for the extended diffusion.The derivation starts from maximum-likelihood training, uses the Fokker–Planck equation and KL divergence, and permits a general weighting function λ(t).Source: §B.3 · p. 22
  • The training formulation supports both denoising score matching and hybrid score matching, with conditioning-dependent covariance factors and perturbation kernels.The scaling factor ℓ_t differs between DSM, based on p(u_t | u_0), and HSM, based on p(u_t | x_0), and the model parameterizes the velocity score as s_θ(u_t,t) = −ℓ_t α_θ(u_t,t).Source: §B.3 · pp. 23–24
  • For high-quality image synthesis, the objective can be modified by dropping its variance prefactor, while HSM may require adding ϵ_num = 10^-9 for numerical stability.The variance-dropping modification follows prior work, and the stabilization is sufficient when numerical computation of ℓ_t becomes unstable even in double precision.Source: §B.3 · p. 24

B.4 CLD-SPECIFIC IMPLEMENTATION DETAILS … C HSM: HYBRID SCORE MATCHING

The appendix details numerical stabilization for HSM, probability-flow sampling and likelihood bounds, and the motivation and derivation of hybrid score matching. It explains how Hamiltonian dynamics improves mixing while reducing reliance on difficult score estimation, and how HSM can denoise using only data samples by marginalizing velocities.

B.4 CLD-SPECIFIC IMPLEMENTATION DETAILS

  • A stabilization term ϵ_num = 10^-9 with double precision makes HSM numerically usable despite divergence and instability as t → 0.The stabilization adds ϵ_numI_2d to Σ_t before Cholesky decomposition, under the main setup M = 0.25, γ = 0.04, and T = 1.Source: §B.4 · p. 24

B.5 LOWER BOUNDS AND PROBABILITY FLOW ODE

  • CLD samples can be generated by solving the probability-flow ODE, whose dynamics combine Hamiltonian and score terms while omitting the Ornstein–Uhlenbeck term.The authors use an adaptive-step Runge–Kutta 4(5) integrator in practice because the equation is not perfectly Hamiltonian.Source: §B.5 · pp. 24–25
  • The probability-flow ODE supports a stochastic unbiased estimate of joint log p(x_0, v_0), yielding a lower bound on log-likelihood and an upper bound on NLL.The estimate uses Hutchinson’s trace estimator for the Jacobian log-determinant and includes the prior probability; the velocity entropy is H(p(v_0)) = 1/2 log(2πeγM).Source: §B.5 · p. 25
  • The likelihood bound can be quite loose because importance weighting over multiple velocity samples is unavailable, while a velocity encoder is left for future work.The limitation arises because only a stochastic estimate log p_ε(x_0, v_0) is accessible.Source: §B.5 · p. 25

B.6 ON INTRODUCING A HAMILTONIAN COMPONENT INTO THE DIFFUSION

  • Adding Hamiltonian dynamics promotes deterministic, analytically invertible mixing, reducing noise injection and the score-model complexity required for reverse-time generation.The authors attribute smoother networks and improved CIFAR-10 performance under matched capacity and limited compute to this balance between Hamiltonian mixing and score modeling.Source: §B.6 · p. 26
  • The MCMC perspective motivates Hamiltonian components because efficient mixing reaches the prior faster, shortening the reverse generation path and potentially easing SGM sampling.The discussion connects this rationale to Hamiltonian sampling in MCMC and to thermostating methods from statistical mechanics and molecular dynamics.Source: §B.6 · p. 26

C HSM: HYBRID SCORE MATCHING

  • HSM rewrites the CLD score-matching objective as an equivalent hybrid objective and can denoise using only p(x_0) while marginalizing over the Normal initial velocity distribution p(v_0).Because p(v_0) is Normal, the resulting perturbation kernel p_t(u_t | x_0) remains Normal; the derivation establishes equivalence up to an additive constant.Source: C HSM: HYBRID SCORE MATCHING · p. 27

C.1 GRADIENT VARIANCE REDUCTION VIA HSM · D SYMMETRIC SPLITTING CLD SAMPLER (SSCS) · D.1 BACKGROUND

HSM yields lower-variance Monte Carlo gradients than DSM, especially for large γ, while the paper develops SSCS using symmetric-splitting and operator techniques from statistical mechanics and molecular dynamics.

C.1 GRADIENT VARIANCE REDUCTION VIA HSM

  • HSM is advocated over DSM because it avoids unbounded scores as t →0 and also lowers the variance of practical Monte Carlo gradients.Although LHSM = LDSM + const, the objectives differ in practical estimation behavior.Source: §C.1 · p. 28
  • The HSM and DSM gradient-variance difference arises from their squared norm terms because the sampled ut and network-gradient factor are shared.The analysis compares λ(t)∇θLHSM and λ(t)∇θLDSM at discretized times with T = 1.Source: §C.1 · p. 28
  • The variance comparison evaluates empirical covariance traces of KHSM and KDSM on CIFAR-10 using a trained CLD ablation model.The model uses the standard setup M^-1 = β = 4, and covariance-matrix trace is treated as total variation.Source: §C.1 · p. 28
  • HSM clearly reduces gradient variance relative to DSM, particularly for large γ, while DSM exhibits very large small-t covariance traces and instability for small γ.HSM integrates out the initial velocity distribution, whereas DSM uses noisy initial-velocity samples.Source: §C.1 · p. 28, Figure 7 · p. 29

D SYMMETRIC SPLITTING CLD SAMPLER (SSCS)

D.1 BACKGROUND

  • SSCS derives from symmetric-splitting techniques and Fokker–Planck operator concepts drawn from statistical mechanics and molecular dynamics.The discussion points to Tuckerman (2010), Leimkuhler & Matthews (2015), and Leimkuhler & Matthews (2013) for background on these methods.Source: §D.1 · p. 29

D.2 DERIVATION AND ANALYSIS · E IMPLEMENTATION AND EXPERIMENT DETAILS

The SSCS sampler derives an intractable CLD propagator into analytically tractable and numerically approximated components, yielding a first-order integrator whose practical error can be lower than Euler–Maruyama. Its implementation uses symmetric splitting, Gaussian half-steps, a neural-network-centered full step, and final denoising, with support for variable step sizes and efficient data-only output.

D.2 DERIVATION AND ANALYSIS

  • The formal Fokker–Planck operator construction provides a transparent description of stochastic state evolution and a basis for deriving the SSCS integration method, but is not directly used in computation.The classical propagator cannot be evaluated analytically, motivating the splitting approximation.Source: §D.2 · p. 30
  • SSCS splits the intractable generative propagator into analytically tractable components and a numerically approximated velocity update, enabling integration through repeated small steps.The splitting error decreases with finer discretization, while individual tractable updates introduce no evaluation error.Source: §D.2 · pp. 30–31
  • The analytically tractable A component evolves the joint state through an affine SDE, so Normal inputs remain Normal and its mean and covariance can be computed exactly.The proposed mean and covariance satisfy their governing differential equations.Source: §D.2 · pp. 30–31
  • SSCS is a first-order integrator with local error ∼δt^2 and global error ∼δt, because its S component uses Euler integration.The method is therefore formally comparable to Euler–Maruyama in convergence order, despite exploiting exact substeps.Source: §D.2 · pp. 31–33
  • When the dynamics are dominated by analytically handled components, SSCS is expected to be more accurate than Euler–Maruyama, while both methods converge similarly for very small steps.This regime is motivated by the conditional velocity score being close to Normal for much of the diffusion, making the S component small; experiments validate the expected advantage and few-step suitability.Source: §D.2 · p. 32
  • SSCS integrates from t=0 only to t=T−ϵ, then applies a denoising step; the scheme also extends to time-dependent β(t), although the main paper uses constant β.Adaptive step sizes and higher-order integration are identified as future extensions.Source: §D.2 · p. 33
  • The algorithm initializes Gaussian position and velocity priors, applies symmetric Gaussian half-steps around a learned-score full step for N iterations, and returns synthesized data with an optional velocity sample.Variable step sizes are allowed, and the final denoising step can update only the data component to avoid a final neural-network call.Source: §D.2 · p. 33
  • Only the center full step requires a neural-network evaluation, so combining adjacent half-steps offers negligible computational benefit because the half-steps are virtually free.The sampler’s structure is analogous to Leapfrog integration, but merging half-steps is unnecessary in this setting.Source: §D.2 · p. 33

E.1 SCORE AND JACOBIAN EXPERIMENTS … F.1.1 ANALYTICAL SAMPLING

Analytical toy experiments show that CLD reduces score and neural-network complexity, while SSCS improves sampling over Euler–Maruyama; image experiments establish implementation choices, ablation effects, and a parameter-matched LSGM comparison.

E.1 SCORE AND JACOBIAN EXPERIMENTS

  • CLD’s learned conditional velocity score is closer to the Normal equilibrium score than VPSDE’s data score, with ξ_CLD(t) smaller for all t and especially early times.Both methods are compared at matched equilibrium distributions using M = γ = 1 for CLD.Source: §E.1 · pp. 34–35
  • Neural-network complexity is significantly lower for CLD than VPSDE, and mixed-score parameterization further reduces complexity for both diffusions.The experiments train small ResNets and measure complexity using squared Frobenius norms of network Jacobians.Source: §E.1 · p. 35

E.2 IMAGE MODELING EXPERIMENTS; E.2.1 TRAINING DETAILS AND MODEL ARCHITECTURES

  • The image experiments use CIFAR-10 and CelebA-HQ-256, adapting NCSN++ and DDPM++ with six input channels to accommodate velocity variables.The additional channels add negligible neural-network parameters.Source: §E.2 · p. 35, §E.2.1 · p. 36

E.2.3 QUADRATIC STRIDING

  • Quadratic striding concentrates function evaluations near the data, where the diffusion distribution is most complex, and is used during the synthesis experiments.The schedule uses quadratic evaluation times and linearly varying individual time steps.Source: §E.2.3 · p. 37

E.2.4 DENOISING

  • Denoising only slightly changes CLD FID with SSCS, unlike direct-data SDEs where it is crucial and probability-flow ODE integration can otherwise blow up near t →0.For CLD, the network denoises only velocity, while the primary interest is the data component.Source: §E.2.4 · p. 38, Table 7 · p. 38

E.2.5 SOLVER ERROR TOLERANCES FOR RUNGE–KUTTA 4(5); E.2.6 ABLATION EXPERIMENTS

  • CIFAR-10 ablations find that γ and M have small FID effects, whereas mixed-score parameterization significantly improves competitive FIDs; Runge–Kutta comparisons use error tolerances 10^-5 and 10^-3.These tolerances yield mean function evaluations of 312 and 137 for CLD, respectively.Source: §E.2.6 · p. 38, §E.2.5 · p. 38

E.2.7 LSGM-100M MODEL; E.2.2 CIFAR-10 RESULTS FOR VESDE AND VPSDE

  • The parameter-matched LSGM-100M achieves FID 4.60, NLL bound 2.96 bpd, and 131 average NFE, while alternative small configurations were unstable or significantly worse.LSGM-100M has ≈109M parameters versus ≈108M for the CLD-based SGM.Source: §E.2.7 · pp. 38–39

F.1.1 ANALYTICAL SAMPLING

  • For every tested step count n ∈{20, 50, 100, 200}, CLD with SSCS outperforms VPSDE and CLD with Euler–Maruyama in negative log-likelihood.Euler–Maruyama produces overly broad distributions and is poorly suited to CLD, especially with fewer steps and larger step sizes.Source: §F.1.1 · p. 39, Table 8 · p. 39

F.1.2 MAXIMUM LIKELIHOOD TRAINING · F.2 CIFAR-10 — EXTENDED RESULTS · F.3 CELEBA-HQ-256 — EXTENDED RESULTS

Extended experiments show that CLD improves maximum-likelihood toy modeling, outperforms competing samplers on CIFAR-10 across most budgets, and produces high-quality CelebA-HQ-256 samples while revealing interpretable data–velocity generation dynamics.

F.1.2 MAXIMUM LIKELIHOOD TRAINING

  • CLD requires only a bounded score at t = 0, unlike VPSDE-style overdamped Langevin models, which must learn an unbounded score as t →0.Image experiments nevertheless use a reweighted objective to improve visual sample quality; maximum-likelihood image training remains future work.Source: §F.1.2 · p. 39
  • For CLD maximum-likelihood training, larger γ generally yield smoother, less complex neural networks, yet even γ = 0.04 remains smoother than VPSDE.The comparison uses the maximum-likelihood objective with λ(t) = Γβ across three γ values; the VPSDE mixed-score model did not converge and is excluded from Fig. 12.Source: §F.1.2 · p. 41
  • In maximum-likelihood toy experiments, CLD significantly outperforms VPSDE and captures both coarse and fine multi-scale structure that VPSDE misses.VPSDE captures the main modes but not fine structure and undesirably connects modes, whereas CLD adequately reproduces the five rolls with few intermodal samples.Source: §F.1.2 · p. 41

F.2 CIFAR-10 — EXTENDED RESULTS

  • For CIFAR-10, CLD with SSCS and quadratic striding outperforms DDIM for every sampling budget n ≥150, while DDIM is better only at n = 50.Quadratic striding improves VPSDE and CLD sampling broadly, but significantly worsens VESDE FID scores; DDIM is specifically designed for few-step sampling.Source: §F.2 · p. 42
  • CIFAR-10 CLD samples achieve FID 2.23 with EM-QS using 2000 evaluations and FID 3.07 with SSCS-QS using only 150 evaluations.These are additional generated samples from the CLD-SGM model; the 2.23 setup is the reported best FID among these samples.Source: §F.2 · p. 42, Figure 14 · p. 43, Figure 15 · p. 44

F.3 CELEBA-HQ-256 — EXTENDED RESULTS

  • On CelebA-HQ-256, adaptive Runge–Kutta 4(5) sampling still produces very good samples at tolerance 10−3 using an average of 129 neural-network evaluations.Low-frequency artifacts appear at tolerance 10−3, while deterioration begins at 10−2 in the corresponding visual comparison.Source: §F.3 · p. 45, Figure 17 · p. 47
  • CelebA-HQ-256 generation paths show velocities encoding noisy versions of data at intermediate times, with equilibrium behavior at t = 1.0 and distinct Normal limits as t →0.The inverse colorization reveals negative velocities that pull pixels toward equilibrium, a consequence of Hamiltonian coupling and the negative x_t term.Source: §F.3 · p. 45, Figure 18 · p. 48, Figure 19 · p. 49

G PROOFS OF PERTURBATION KERNELS … G.2.2 PROOF OF CORRECTNESS OF THE COVARIANCE

The appendix proves the perturbation-kernel formulas for both the forward diffusion and SSCS’s analytical splitting term by substituting claimed mean and covariance solutions into their governing ODEs. The derivations cover general time-dependent β(t), with SSCS specialized in practice to constant β and small time steps.

G PROOFS OF PERTURBATION KERNELS

  • The perturbation-kernel proofs establish correctness for the forward diffusion and SSCS analytical splitting term under general time-dependent β(t).The section presents both derivations and their associated mean and covariance ODEs.Source: G PROOFS OF PERTURBATION KERNELS · p. 50

G.1 FORWARD DIFFUSION

  • The forward-diffusion evolution is formulated through ODEs for the mean and covariance matrix, followed by claimed closed-form solutions.The claimed solutions are introduced after the governing ODEs.Source: §G.1 · p. 50

G.1.1 PROOF OF CORRECTNESS OF THE MEAN

  • The forward-diffusion mean proof substitutes the claimed solution into its ODE, decomposes the result into component equations, and verifies the relevant equations.The appendix explicitly states that this completes the proof of the mean’s correctness.Source: §G.1.1 · pp. 50–51

G.1.2 PROOF OF CORRECTNESS OF THE COVARIANCE

  • The forward-diffusion covariance proof substitutes the claimed covariance solution into its ODE and verifies Eqs. (134)–(136) after simplification.The appendix concludes that these substitutions complete the covariance proof.Source: §G.1.2 · pp. 51–52

G.2 ANALYTICAL SPLITTING TERM OF SSCS

  • SSCS’s analytical splitting term has mean and covariance ODEs analogous to the forward diffusion, differing by flipped off-diagonal signs in f(T − t).Its claimed solutions use a general initial time t′ and initialize covariance at zero because updated samples have no uncertainty.Source: §G.2 · p. 52

G.2.1 PROOF OF CORRECTNESS OF THE MEAN

  • The SSCS mean proof substitutes its claimed solution into the governing ODE, decomposes the result, and verifies the resulting component equations.The appendix states that this completes the proof of the mean’s correctness.Source: §G.2.1 · p. 53

G.2.2 PROOF OF CORRECTNESS OF THE COVARIANCE

  • The SSCS covariance proof substitutes the claimed solution into its ODE and verifies Eqs. (164)–(166) after simplification.The appendix concludes that these substitutions complete the covariance proof.Source: §G.2.2 · pp. 53–54
  • For practical SSCS, constant β, T = 1, and small time steps δt yield B(t) = β δt^2/2 and the expressions given in App. D.2.This connects the general covariance derivation to the implementation used in SSCS.Source: §G.2.2 · p. 54

Related papers

Loading 2112.07068v4…