PaperlayerView exact revision on arXiv

Source-linked AI summary

gDDIM: Generalized denoising diffusion implicit models

Qinsheng Zhang, Molei Tao, Yongxin Chen

arXiv:2206.05564v2cs.LG

TL;DR

Diffusion models are difficult to sample efficiently, especially beyond isotropic diffusions, and the mechanism behind DDIM’s acceleration is not fully understood. The paper interprets DDIM through numerical score approximations and extends it as gDDIM with a modified score parameterization. gDDIM achieves strong low-NFE results on CLD and more than 20 times acceleration on BDM.

  • Problem

    Diffusion-model sampling is slow, DDIM’s acceleration mechanism is unclear, and extending DDIM beyond isotropic diffusions remains an open question.

  • Method

    The paper derives DDIM from score approximations for diffusion SDEs and introduces gDDIM by modifying score-network parameterization for general diffusion models.

  • Results

    gDDIM achieves FID 2.26 with 50 NFEs and FID 2.86 with 27 NFEs on CLD, while exceeding 20 times acceleration on BDM.

  • Takeaways & Limitations

    The interpretation explains why deterministic ODE-based sampling is advantageous at low NFE and supports efficient DDIM-style sampling across a broader class of diffusion models.

  • Takeaways & Limitations

    The paper’s theoretical propositions rely on a Dirac data-distribution assumption, although a manifold-based interpretation is offered for realistic datasets.

Abstract

from arXiv · show

Our goal is to extend the denoising diffusion implicit model (DDIM) to general diffusion models~(DMs) besides isotropic diffusions. Instead of constructing a non-Markov noising process as in the original DDIM, we examine the mechanism of DDIM from a numerical perspective. We discover that the DDIM can be obtained by using some specific approximations of the score when solving the corresponding stochastic differential equation. We present an interpretation of the accelerating effects of DDIM that also explains the advantages of a deterministic sampling scheme over the stochastic one for fast sampling. Building on this insight, we extend DDIM to general DMs, coined generalized DDIM (gDDIM), with a small but delicate modification in parameterizing the score network. We validate gDDIM in two non-isotropic DMs: Blurring diffusion model (BDM) and Critically-damped Langevin diffusion model (CLD). We observe more than 20 times acceleration in BDM. In the CLD, a diffusion model by augmenting the diffusion process with velocity, our algorithm achieves an FID score of 2.26, on CIFAR10, with only 50 number of score function evaluations~(NFEs) and an FID score of 2.86 with only 27 NFEs. Code is available at https://github.com/qsh-zh/gDDIM

1 INTRODUCTION

Diffusion models offer strong generation quality and training advantages, but their slow sampling limits broader application. The paper analyzes DDIM’s mechanism and generalizes it to accelerate other diffusion models.

  • Slow and expensive sampling remains a major obstacle to applying diffusion models to more complex and higher-dimensional tasks.
  • Probability-flow ODEs provide a promising acceleration route because they preserve diffusion-model marginals while enabling efficient numerical solvers.
  • The paper asks why probability-flow ODEs outperform SDEs at low step counts, how DDIM accelerates sampling, and whether DDIM generalizes beyond isotropic diffusion.
  • The authors interpret DDIM through score approximations, explain its deterministic acceleration, and introduce generalized DDIM for broader diffusion models.
  • gDDIM accelerates diffusion models deterministically and stochastically, achieving FID 2.86 with 27 steps and FID 2.26 with 50 steps on CLD, plus more than 20 times acceleration on BDM.

2 BACKGROUND

Diffusion models learn to reverse a continuous-time noising process, but conventional sampling can require hundreds or thousands of evaluations. DDIM reduces this cost, while non-isotropic models remain difficult to accelerate.

  • Diffusion models use a forward SDE to add noise and a backward denoising process to generate samples by removing it.
  • The forward diffusion is a linear SDE whose state u(t) evolves from the data distribution toward an approximately Gaussian distribution.
  • A learned score network sθ approximates ∇log p_t(u), commonly trained with denoising score matching.
  • Probability-flow ODEs and reverse-time SDEs form marginal-equivalent sampling choices, with λ=0 yielding the ODE and λ=1 yielding reverse-time diffusion.
  • DDPM sampling requires hundreds or thousands of score evaluations, whereas DDIM can generate reasonable samples within 50 NFEs.
  • CLD adds an auxiliary velocity state and injects noise through velocity coupling, while BDM uses frequency-dependent transformations; both are harder to accelerate than DDPM.

3 REVISIT DDIM: GAP BETWEEN THE EXACT SOLUTION AND NUMERICAL

The paper explains DDIM as a numerical method based on local score approximations. In a one-point toy setting these approximations can be exact, while a manifold-based interpretation motivates their use on realistic datasets.

  • Standard numerical solvers incur discretization error on a one-point dataset, but DDIM can recover the training point in one step.
  • ODE sampling: For deterministic DDIM, the noise prediction remains constant along exact probability-flow ODE trajectories under a Dirac data distribution.
  • ODE sampling: Approximating the noise prediction as locally constant yields an update that coincides with deterministic DDIM.
  • ODE sampling: With the ground-truth noise predictor, the deterministic DDIM update is precise and recovers the toy training data in one step.
  • SDE sampling: For stochastic sampling, the score must account for state dependence because injected noise changes the trajectory and must be canceled.
  • SDE sampling: A score approximation leads to a numerical scheme that coincides with stochastic DDIM, while DDIMs generally arise from local approximations.
  • Justification of Dirac approximation: The Dirac assumption is strong, but a manifold interpretation argues that locally dominant modes can approximate realistic-data scores and help explain DDIM effectiveness.

4 GENERALIZE AND IMPROVE DDIM

gDDIM generalizes DDIM to diffusion models with general drift and diffusion by using a carefully chosen score-network parameterization. The method includes deterministic and stochastic updates, plus multistep predictor-corrector variants for reducing function evaluations.

  • Generalized DDIM: gDDIM extends DDIM beyond DDPMs to diffusion models with general drift and diffusion through a carefully chosen K_t.The DDPM-specific choice is recovered when K_t = √(1 − α_t)I_d.
  • Generalized DDIM: For Gaussian initial data, the transformed ground-truth score quantity remains constant along probability-flow ODE solutions.This extends the earlier single-point result and supports accurate one-step sampling in the toy setting.
  • Score approximation: The realistic-data gDDIM approximator uses the new score parameterization to estimate the score across time from a score evaluation.The same approximation is used to construct the deterministic gDDIM update.
  • Multistep acceleration: Multistep predictor-corrector methods reduce the number of sampling steps while retaining accuracy.The authors report better performance with more predictor NFEs when the total NFE is small.
  • Stochastic variant: Stochastic gDDIM reduces to deterministic gDDIM when λ = 0.The two update forms therefore coincide at the deterministic setting.

5 EXPERIMENTS

Experiments evaluate gDDIM on CLD and BDM, focusing on parameterization choices, stochasticity, and acceleration across diffusion models. The results favor the proposed R_t parameterization and deterministic sampling at small NFE, with major acceleration for BDM.

  • Experimental setup: The experiments compare gDDIM and other samplers across CLD, BDM, and related diffusion models using similar network sizes and training settings.The evaluation includes quantitative CIFAR10 comparisons and tests of the propositions, K_t, and λ choices.
  • Choice of K_t: The proposed R_t parameterization makes the learned ϵθ output flatter along probability-flow ODE solutions than the alternative L_t parameterization.The comparison is performed on real datasets with a trained score model.
  • Choice of λ: Increasing λ deteriorates sample quality, supporting better deterministic than stochastic performance when NFE is small.Stochastic gDDIM nevertheless significantly outperforms EM in the reported comparison.
  • Acceleration across DMs: More than 20 times acceleration is achieved on BDM: gDDIM with 50 NFE outperforms the original ancestral sampler with 1000 NFE.The comparison uses the reformulated BDM SDE and corresponding SDE/ODE solvers.

6 CONCLUSIONS AND LIMITATIONS

The paper interprets DDIM through numerical structure and extends it to general diffusion models with gDDIM. It reports broad acceleration benefits while identifying limited validation and remaining opportunities to exploit diffusion structure.

  • Conclusions: gDDIM generalizes DDIM to general diffusion models through a modified score parameterization.The paper presents gDDIM as a deterministic and stochastic sampler for a large class of diffusion models.
  • Conclusions: The DDIM interpretation explains its discretization and why ODE-based samplers are preferred over SDE-based samplers when NFE is low.The analysis connects the method’s acceleration to numerical treatment of the diffusion dynamics.
  • Limitations: The paper validates gDDIM on only three diffusion models and calls for more efficient diffusion processes and further exploitation of structural knowledge.The stated scope boundary is limited empirical verification despite the method’s general design.
  • General diffusion models: BDM is formulated as a non-isotropic diffusion process with a linear SDE representation derived from its mean and covariance.The construction uses frequency-dependent diagonal matrices and a transformed data representation.
  • Numerical interpretation: The gDDIM update uses score approximations that produce a linear ODE with a closed-form solution based on the diffusion transition matrix.The same framework underlies the DDIM update and its multistep predictor-corrector extensions.

B.3.2 PROOF OF PROP 6

This proof shows that the approximate reverse-time dynamics remain analytically tractable. Their conditional transition is Gaussian, and the resulting mean update recovers the DDIM-related formula.

  • Derivation: The approximated reverse dynamics form a linear differential equation whose conditional transition distribution is Gaussian.The covariance and mean can therefore be derived in closed form using the transition matrix.
  • Derivation: The approximation preserves the transition structure needed to derive the mean update used by the method.The derivation identifies the resulting mean with the update in Eq. (22).
  • DDIM connection: The exact solution of the specialized reverse equation is the same as the DDIM update.This establishes the DDIM step as the closed-form solution for the chosen coefficient structure.
  • DDPM specialization: For the DDPM specialization, substituting the relevant time interval recovers the mean and covariance in the sampler update.The proof separately verifies both components of the transition.

B.3.4 PROOF OF PROP 7

When λ = 0, the generalized update and the earlier update are equivalent. The proof establishes this by showing that their transition matrices satisfy the same linear differential equation.

  • Equivalence: At λ = 0, the generalized update from s to t and the earlier update are the two expressions being compared.The equivalence is reduced to a lemma relating their transition representations.
  • Proof strategy: The auxiliary transition matrices satisfy the same linear differential equation as the matrix used in the earlier construction.This shared evolution equation is the key bridge between the two updates.
  • Proof strategy: Linear system theory gives ˆΨ(t, s) = R_tR_s^-1, allowing the two update forms to be identified.Substituting this transition identity completes the equivalence argument.
  • Empirical context: The practical experiments use repeated runs with different random seeds and include reproduced BDM training because pretrained BDM models were unavailable.This contextualizes the empirical evaluation accompanying the theoretical development.

C.2 CLD: TRAINING AND SAMPLING

For CLD, gDDIM changes the score parameterization from K_t = L_t to K_t = R_t and uses numerical coefficient calculations before online EI-based sampling. The altered parameterization requires both score channels but adds little training cost.

  • Score parameterization: Unlike the original CLD parameterization, gDDIM requires both x- and v-channel score outputs.The original setup learned only velocity-channel signals because of the special diffusion structure.
  • Score parameterization: gDDIM changes the CLD parameterization from K_t = L_t to K_t = R_t.The new choice is tied to the generalized sampler’s score representation.
  • Implementation: The new parameterization doubles the output-layer channels, while the added parameter count remains negligible relative to the rest of the diffusion model.This is the principal architectural modification reported for CLD.
  • Coefficient preparation: gDDIM coefficients are computed numerically through ODE solves and definite integrals, then reused across sampling runs.For CLD, the coefficients are 2 × 2 matrices and the full calculation takes within 1 min.
  • Sampling: Online execution uses high-order exponential-integrator ODE solvers with multistep predictor and corrector updates.The sampler executes the prepared coefficients through the EI-based solver pipeline.

C.5 MORE EXPERIMENTS ON THE CHOICE OF SCORE PARAMETERIZATION

Experiments compare score parameterizations in VPSDE and CLD, finding that the Rt choice improves sampling quality and enables larger steps, especially at low NFE.

  • Rt provides better sampling quality than Lt at the same small NFE budget.
  • In VPSDE, Rt, Lt, and √Σt coincide, so gDDIM reduces to DDIM.
  • In CLD, oscillations under Lt prevent numerical solvers from taking large step sizes, whereas Rt avoids this issue.
  • Experiments on CIFAR10 and CELEBA show significant Rt advantages, especially when NFE is small.
  • Large-q multistep methods can harm low-NFE sampling because outdated information worsens the nonlinear approximation.

C.6 MORE EXPERIMENTS ON THE CHOICE OF λ

Experiments vary λ to assess stochasticity and compare gDDIM with other samplers. Lower λ produces smoother trajectories and better low-NFE sampling, while SSCS remains stronger at λ = 1.

  • Smaller λ produces smoother trajectories, while larger λ introduces more randomness that is harder to predict with small NFE.
  • When NFE is small, increasing λ negatively affects gDDIM sampling quality because score evaluations cannot fully remove the injected noise.
  • At λ = 1, both evaluated methods perform worse than Symmetric Splitting CLD Sampler.
  • SSCS at λ = 1.0 performs much worse than gDDIM at λ = 0.
  • Among training-free acceleration methods, gDDIM achieves the best reported sampling acceleration but remains behind some distillation-based methods.

C.8 NEGATIVE LOG LIKELIHOOD EVALUATION

The paper evaluates a lower bound related to negative log likelihood for its joint CLD model, using probability ODE simulation and Monte Carlo estimation.

  • Probability ODE simulation is used to estimate the log likelihood of given data.
  • The model represents the joint distribution p(u0) = p(x0, v0), making the marginal p(x0) challenging to obtain because v0 must be integrated out.
  • A lower bound on log likelihood is estimated with a Monte Carlo approach.
  • 3.33 bits/dim is the trained model's NLL upper bound, compared with 3.31 bits/dim reported for the original CLD.
Loading 2206.05564v2…