PaperlayerView exact revision on arXiv

Source-linked AI summary

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

arXiv:2206.00927v3cs.LGstat.ML

TL;DR

DPMs require hundreds or thousands of sequential evaluations, while existing training-free samplers struggle in the few-step regime. DPM-Solver exploits diffusion ODE structure to generate high-quality samples in around 10 steps, achieving 4.70 FID with 10 evaluations on CIFAR-10.

  • Problem

    DPM sampling remains slow, and existing general-purpose ODE solvers lack satisfactory performance in the roughly 10-step regime.

  • Method

    DPM-Solver analytically handles the linear part of diffusion ODEs and approximates the resulting exponentially weighted neural-network integral with first-, second-, and third-order solvers.

  • Results

    Across continuous- and discrete-time DPMs and datasets, DPM-Solver produces reasonable or high-quality samples within around 10–12 evaluations; it achieves 4.70 FID with 10 evaluations on CIFAR-10.

  • Takeaways & Limitations

    DPM-Solver provides a training-free route to substantially faster diffusion-model sampling without further training, including for continuous-time, discrete-time, and classifier-guided models.

  • Takeaways & Limitations

    DPM-Solver targets fast sampling rather than likelihood evaluation and remains unsuitable for real-time applications compared with commonly used GANs.

Abstract

from arXiv · show

Diffusion probabilistic models (DPMs) are emerging powerful generative models. Despite their high-quality generation performance, DPMs still suffer from their slow sampling as they generally need hundreds or thousands of sequential function evaluations (steps) of large neural networks to draw a sample. Sampling from DPMs can be viewed alternatively as solving the corresponding diffusion ordinary differential equations (ODEs). In this work, we propose an exact formulation of the solution of diffusion ODEs. The formulation analytically computes the linear part of the solution, rather than leaving all terms to black-box ODE solvers as adopted in previous works. By applying change-of-variable, the solution can be equivalently simplified to an exponentially weighted integral of the neural network. Based on our formulation, we propose DPM-Solver, a fast dedicated high-order solver for diffusion ODEs with the convergence order guarantee. DPM-Solver is suitable for both discrete-time and continuous-time DPMs without any further training. Experimental results show that DPM-Solver can generate high-quality samples in only 10 to 20 function evaluations on various datasets. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset, and a $4\sim 16\times$ speedup compared with previous state-of-the-art training-free samplers on various datasets.

1 Introduction

DPM-Solver addresses the slow sampling of diffusion probabilistic models by treating sampling as diffusion-ODE solving and analytically exploiting their semi-linear structure. The resulting training-free solver targets high-quality few-step sampling across discrete-time, continuous-time, and conditionally guided DPMs.

  • Motivation: DPMs are powerful generative models used for image, video, text-to-image, speech, and lossless-compression tasks.They are defined by discrete-time random processes or continuous-time SDEs that gradually remove noise from data points.
  • Limitations of Existing Samplers: Existing fast samplers either require additional training or offer limited flexibility across models, datasets, and sampling-step counts.The training-based category includes knowledge distillation and noise-level or sample-trajectory learning.
  • Approach: The work targets few-step sampling by formulating DPM sampling as solving the corresponding diffusion ODEs.The stated regime uses around 10 sequential function evaluations, and the diffusion ODEs have a semi-linear structure.
  • Method: DPM-Solver approximates the ODE solution integral with first-, second-, and third-order solvers that have convergence-order guarantees.The method also includes an adaptive step-size schedule.
  • Applicability: DPM-Solver applies to continuous-time and discrete-time DPMs, including conditional sampling with classifier guidance.Figure 1 compares DPM-Solver using 10 function evaluations with DDIM using 10, 15, 20, and 100 function evaluations on ImageNet 256×256.

2 Diffusion Probabilistic Models

Diffusion probabilistic models define a noise-controlled forward process whose reverse dynamics can be modeled with a learned noise-prediction network. Sampling requires solving reverse SDEs or associated probability-flow ODEs, with ODEs enabling larger steps but existing methods still requiring many evaluations.

  • Forward diffusion: DPMs define a forward process for x0 with conditional distributions parameterized by differentiable noise schedules αt and σt.The schedule is chosen so the terminal marginal approaches an isotropic Gaussian and the signal-to-noise ratio decreases over time.
  • Reverse dynamics: A reverse-time SDE has the same transition structure, with the unknown score function estimated by a neural network ϵθ(xt, t).The network predicts the Gaussian noise added to xt and is trained using a weighted objective.
  • Sampling challenge: First-order ancestral SDE sampling typically requires hundreds or thousands of function evaluations to converge, making generation extremely slow.Large SDE step sizes can also cause non-convergence because of Wiener-process randomness, particularly in high-dimensional spaces.
  • Probability-flow ODE: The probability-flow ODE has the same marginal distribution as the SDE at each time and can be solved with larger steps because it has no randomness.Replacing the score with the noise-prediction model yields the parameterized diffusion ODE.
  • Probability-flow ODE: ∼60 function evaluations with RK45 reached quality comparable to a 1000-step SDE solver on CIFAR-10.This motivates using efficient numerical ODE solvers to accelerate diffusion-model sampling.

3 Customized Fast Solvers for Diffusion ODEs

This section develops DPM-Solver by exploiting the semi-linear structure of diffusion ODEs: it computes the linear component exactly and reduces the remaining problem to an exponentially weighted neural-network integral. Taylor-based approximations yield first-, second-, and third-order solvers with convergence guarantees and few function evaluations per step.

  • Motivation: Diffusion ODEs motivate a dedicated solver because black-box ODE solvers empirically fail to converge in few steps, despite ODEs being easier to solve than high-dimensional SDEs.The goal is fast, high-quality few-step sampling.
  • Exact solution formulation: The diffusion ODE is semi-linear: its linear term is solved exactly, eliminating that term’s discretization error while leaving the neural-network term as an integral to approximate.Previous black-box solvers treat both terms together and incur approximation errors in both.
  • Exact solution formulation: Changing variables to the log-SNR λt reduces the solution to an exponentially weighted integral of the transformed neural network, a formulation related to exponential integrators.This directly targets the nonlinear contribution while avoiding approximation error from the linear term.
  • Solver construction: DPM-Solver-k approximates the weighted integral with Taylor expansions and stiff order conditions, producing solvers for k = 1, 2, and 3.Higher-order variants use additional intermediate points between time steps.
  • Solver construction: k function evaluations per step are required for DPM-Solver-k with k = 1, 2, 3, while higher-order solvers are usually more efficient because they need fewer steps.The intermediate time points are obtained using the analytical inverse tλ(·) for the practical noise schedule.
  • Convergence and scope: O(h^k) approximation error at time 0 establishes DPM-Solver-k as a k-th-order solver for k = 1, 2, 3 under stated regularity conditions.Solvers with k ≥4 are omitted because they require substantially more intermediate points.

4 Comparison with Existing Fast Sampling Methods

DPM-Solver-1 is mathematically identical to DDIM, while its semi-linear ODE formulation enables principled higher-order solvers and convergence analysis. Compared with traditional Runge-Kutta methods and training-based samplers, DPM-Solver exploits the diffusion ODE structure without additional training.

  • DDIM: DPM-Solver-1 and DDIM have identical update rules despite being motivated from different perspectives.The equivalence follows by substituting the definitions of λ and the step size into the DPM-Solver formulation.
  • Higher-order ODE solvers: DPM-Solver’s semi-linear ODE formulation supports principled generalization to higher-order solvers and convergence-order analysis.This is the stated advantage of the formulation over merely identifying DDIM as a first-order method.
  • DDIM: DDIM exploits the semi-linearity of diffusion ODEs, explaining its superiority over traditional Euler discretization.Prior work identified DDIM as a first-order diffusion-ODE discretization but did not explain its difference from Euler methods.
  • Runge-Kutta methods: Traditional explicit Runge-Kutta methods can incur exponentially increasing linear-term error because they approximate both linear and nonlinear components numerically.Their approximation error depends on hθ, including the linear term f(τ)xτ and nonlinear noise prediction model ϵθ.
  • Training-free samplers: Training-based fast samplers require additional training or optimization, and progressive distillation can reach a fast sampler within 4 steps while losing information from the original DPM.Examples include knowledge distillation, learning noise levels or schedules, and learning sample trajectories.
  • Other DPM variants: DPM-Solver may also accelerate DPM variants with low-dimensional latents, bounded score functions, or GAN-assisted reverse processes, although this is left for future work.These approaches modify the DPM type rather than directly designing a fast sampler.

5 Experiments

Experiments evaluate DPM-Solver as a training-free sampler for continuous-time and discrete-time diffusion models across multiple noise schedules, datasets, and function-evaluation budgets. DPM-Solver achieves good sample quality in around 10 NFE and outperforms comparable solvers and methods in efficiency and sample quality.

  • Experimental setup: DPM-Solver is evaluated against existing samplers for continuous-time and discrete-time DPMs using linear and cosine noise schedules while varying the number of function evaluations.Each experiment draws 50K samples and measures sample quality against other methods.
  • Continuous-time models: Around 10 NFE, DPM-Solver generates good sample quality, while competing solvers show large discretization errors even at 50 NFE.The comparison uses Euler-Maruyama, adaptive SDE solving, and RK45 ODE solving on a continuous-time CIFAR-10 model.
  • Continuous-time models: ∼5 speedup, DPM-Solver improves efficiency over the previous best solver in the continuous-time CIFAR-10 comparison.The reported comparison varies solver tolerances and uses uniform time steps for Euler discretization.
  • Continuous-time models: At matched NFE and solver order, second- and third-order DPM-Solver consistently produce better sample quality than RK methods in time t and half-log-SNR λ.The ablation compares RK methods after applying the change of variable to half-log-SNR λ.
  • Discrete-time models: Discrete-time experiments compare DPM-Solver with DDPM, DDIM, Analytic-DDPM, Analytic-DDIM, PNDM, FastDPM, Itô-Taylor, and the additionally trained GGDM method from 10 to 1000 NFE.Evaluations cover CIFAR-10, CelebA 64x64, ImageNet 64x64, ImageNet 128x128 with classifier guidance, and LSUN bedroom 256x256 under linear or cosine schedules.

6 Conclusions

The paper proposes DPM-Solver, a fast training-free diffusion ODE solver enabling DPM sampling in around 10 function-evaluation steps. It also notes limitations for likelihood evaluation, real-time applications, and potential misuse of generated content.

  • 6 Conclusions: DPM-Solver enables fast, training-free sampling from diffusion probabilistic models in around 10 function-evaluation steps.It is a dedicated solver for diffusion ODEs.
  • 6 Conclusions: The solver exploits diffusion ODE semi-linearity by directly approximating an exact-solution formulation involving an exponentially weighted integral of the noise prediction model.This formulation analytically handles the linear component rather than treating all terms as black-box ODE-solver operations.
  • Limitations and broader impact: DPM-Solver may not accelerate likelihood evaluations, remains too slow for real-time applications compared with commonly used GANs, and may amplify potential misuse of generated content.These concerns are identified as limitations and broader impacts of the proposed approach.

Checklist … A.2 Choosing Time Steps for λ is Invariant to the Noise Schedule

The paper establishes noise-schedule-invariant formulations for diffusion-model training and sampling, enabling DPM-Solver to use λ-based time steps independently of intermediate schedules. It also documents theoretical, experimental, reproducibility, and ethics practices.

  • Checklist: The checklist confirms that the paper states its assumptions, provides complete proofs, supplies reproducibility code, and discusses limitations and societal impacts.The method is training-free, with evaluation hyperparameters reported; code and compute details are provided in supplemental materials and appendices.
  • A Sampling with Invariance to the Noise Schedule: The paper presents an exact diffusion-ODE solution in the half-logSNR variable λ and develops its implications for sampling.The formulation is introduced through Proposition 3.1 and is related to maximum-likelihood training.
  • A.1 Decoupling the Sampling Solution from the Noise Schedule: The formulation decouples diffusion-ODE solutions from the specific noise schedule between starting and ending λ values.Given λs, λt, an initial state, and a noise-prediction model, the resulting solution is schedule-invariant.
  • A.1 Decoupling the Sampling Solution from the Noise Schedule: Because the transformed integral depends only on endpoint λ values and the noise-prediction function, intermediate schedule choices do not affect the solution.The remaining coefficients likewise depend only on the endpoints.
  • A.1 Decoupling the Sampling Solution from the Noise Schedule: DPM-Solver analytically computes schedule-dependent coefficients, leaving only the neural network’s Taylor expansion in λ as its approximation.This preserves known information while enabling tailor-made samplers.
  • A.2 Choosing Time Steps for λ is Invariant to the Noise Schedule: Choosing λ time steps determines DPM-Solver’s solution independently of the intermediate noise schedule.The same invariance applies to the algorithm’s update equations after selecting {λi}M_i=0.
  • A.2 Choosing Time Steps for λ is Invariant to the Noise Schedule: The experiments uniformly split [λT, λϵ] to choose λ time steps, while more precise step-selection strategies remain future work.Uniform splitting is the simple setting used in the experiments.

A.3 Relationship with the Maximum Likelihood Training of Diffusion Models … B.2 General Expansion of the Exponentially Weighted Integral

The paper connects diffusion-model maximum-likelihood training and sampling through integrals with respect to λ, yielding schedule invariance, and develops Taylor expansions for the exponentially weighted integral under stated regularity assumptions.

  • A.3 Relationship with the Maximum Likelihood Training of Diffusion Models: The KL-divergence between q0 and p0 is bounded by a weighted score-matching loss for the forward and reverse diffusion distributions.The notation specifies xt = αtx0 + σtϵ and pT = N(0, I).
  • A.3 Relationship with the Maximum Likelihood Training of Diffusion Models: Maximum-likelihood training and diffusion-model sampling can both be expressed as integrals with respect to λ, making the formulation invariant to specific noise schedules.The training expression is related to importance sampling and the continuous-time diffusion loss.
  • B.1 Assumptions: The diffusion ODE solution xs is defined from xT, and DPM-Solver-k is analyzed under assumptions on derivatives, Lipschitz continuity, and step size.The assumptions concern derivatives dλj, the first argument of ϵθ(x, s), and hmax = O(1/M).
  • B.1 Assumptions: Taylor’s theorem motivates the derivative assumption, Lipschitz continuity permits replacing ϵθ(˜xs, s), and hmax = O(1/M) excludes significantly large step sizes.The replacement introduces an O(xs −˜xs) term before Taylor expansion with respect to λs.
  • B.2 General Expansion of the Exponentially Weighted Integral: For t < s, the exponentially weighted integral is expanded by setting h := λt − λs and taking a Taylor expansion of ˆϵθ(ˆxλ, λ) with respect to λ.The construction uses the k-th total derivative of ˆϵ.
  • B.2 General Expansion of the Exponentially Weighted Integral: The exponential integrator and the solution xt are rewritten using the Taylor expansion, with closed forms listed for ϕk for k = 1, 2, 3.The derivation first introduces additional notation before rewriting the integrator and solution.

B.3 Proof of Theorem 3.2 when k = 1 … D.2 Sampling from Discrete-Time DPMs

The appendices establish the convergence orders of DPM-Solver-1, 2, and 3, relate the method to exponential Runge-Kutta ideas while emphasizing its diffusion-specific formulation, and describe implementation and discrete-time adaptations for sampling. These adaptations include adaptive solvers, an end time ε>0, and time reparameterizations that enable continuous-time solver inputs.

  • B.3 Proof of Theorem 3.2 when k = 1: DPM-Solver-1 completes its proof with an O(hmax) approximation error under Assumption B.2 and Eq. (3.7).
  • B.4 Proof of Theorem 3.2 when k = 2: DPM-Solver-2 achieves ˜xti = xti + O(h3) by analyzing an update with the exact preceding solution and showing the auxiliary approximation has O(h3) error.The proof uses h := λt − λs and the fact that eh − 1 = O(h).
  • B.5 Proof of Theorem 3.2 when k = 3: DPM-Solver-3 achieves ¯xt = xt + O(h4) by selecting intermediate times with r1 and r2 and verifying the required conditions through Taylor expansion.The construction uses λs2 − λs = r2h with r2 = 2/3.
  • B.6 Connections to Explicit Exponential Runge-Kutta (expRK) Methods: DPM-Solver is inspired by exponential Runge-Kutta methods’ approximation of an exponentially weighted integral but differs in its linear term and diffusion-ODE-specific formulation.The comparison specifies α = 1 and N = ˜ϵθ for the shared integral-approximation technique.
  • C Algorithms of DPM-Solvers: The algorithm section provides DPM-Solver-1, the general DPM-Solver-2 with r1 ∈ (0, 1), and DPM-Solver-3, plus adaptive combinations DPM-Solver-12 and DPM-Solver-23.The usual DPM-Solver-2 setting is r1 = 0.5; adaptive implementations use tolerances to balance accuracy and NFE.
  • D Implementation Details of DPM-Solver; D.1 End Time of Sampling: Sampling practically solves the diffusion ODE from T to ε>0 without a final denoising trick, and discrete-time models are first converted to continuous time.The end-time cutoff avoids numerical issues near t = 0, while DPM-Solver-23 shares function values between its second- and third-order updates.
  • D.2 Sampling from Discrete-Time DPMs: Discrete-time DPMs commonly use N = 1000 or N = 4000 fixed steps, but their noise model cannot predict below t1 = T/N, requiring scaling into a continuous-time interval.Two scaling types map the discrete range toward either [ε, T] or [0, T], allowing DPM-Solver to use continuous-time inputs.
  • D.2 Sampling from Discrete-Time DPMs: For fixed K function evaluations, Type-1 with ε = 10^-3 performs better for small K, whereas Type-2 with ε = 10^-4 performs better for large K.In practice, T = 1 and the smallest discrete time is t1 = 10^-3.

D.3 DPM-Solver in 20 Function Evaluations … E.1 Diffusion ODEs w.r.t. λ

The paper specifies a function-evaluation budgeted DPM-Solver schedule, derives analytical time reparameterizations, extends sampling to classifier-guided conditioning, and addresses numerical stability and experimental ODE formulations.

  • D.3 DPM-Solver in 20 Function Evaluations: For a fixed budget K ≤20, DPM-Solver divides [λT, λϵ] into M = (⌊K/3⌋ + 1) segments and uses exactly K evaluations.The step allocation depends on the remainder R of K mod 3.
  • D.3 DPM-Solver in 20 Function Evaluations: The remainder-based schedule combines third-, second-, and first-order solver steps to match K evaluations for R = 0, 1, or 2.For R = 0, it uses M−2 DPM-Solver-3 steps followed by DPM-Solver-2 and DPM-Solver-1; for R = 1 or 2, it uses M−1 DPM-Solver-3 steps followed by DPM-Solver-1 or DPM-Solver-2.
  • D.4 Analytical Formulation of the function tλ(·) (the inverse function of λ(t)): DPM-Solver’s analytic inverse time mapping has negligible computational cost for linear and cosine noise schedules, with analogous derivations for other schedule types.The paper mainly considers variance-preserving DPMs because they are most widely used.
  • D.5 Conditional Sampling by DPM-Solver: DPM-Solver supports conditional sampling by replacing the noise predictor with a classifier-guided conditional model using guidance scale s.The default classifier guidance scale is 1.0.
  • D.6 Numerical Stability: Numerical stability is improved by computing expm1(hi) instead of exp(hi)−1 in the DPM-Solver algorithm.This substitution is used when evaluating e^hi − 1.
  • E Experiment Details: Experiments evaluate variance-preserving DPMs on NVIDIA A40 GPUs, while batch-size tuning permits other GPUs such as the NVIDIA GeForce RTX 2080Ti.The theoretical results and method are stated to be independent of the noise schedule choice.
  • E.1 Diffusion ODEs w.r.t. λ: The diffusion ODE can be reparameterized in the λ domain for variance-preserving models, with other noise types treated similarly.For a given λ, the transformed schedule functions are denoted ˆαλ and ˆσλ.
  • E.1 Diffusion ODEs w.r.t. λ: The λ-domain diffusion ODE can also be solved directly with Runge–Kutta methods, as used for RK2 (λ) and RK3 (λ) experiments in Table 1.This formulation supports the reported RK experiments.

E.3 Sample Quality Comparison with Continuous-Time Sampling Methods · E.4 Sample Quality Comparison with RK Methods

The experiments compare DPM-Solver with continuous-time SDE and ODE samplers under different tolerances, and with RK2/RK3 methods using uniform time or λ steps. DPM-Solver uses different solver variants and tolerances for ϵ = 10−3 and ϵ = 10−4, while RK baselines are evaluated under explicit midpoint and Heun’s third-order formulations.

  • E.3 Sample Quality Comparison with Continuous-Time Sampling Methods: Diffusion SDE samplers perform better at ϵ = 10−3, whereas diffusion ODE samplers perform better at ϵ = 10−4.This comparison is based on the detailed FID results corresponding to Table 3 and Fig. 2a.
  • E.3 Sample Quality Comparison with Continuous-Time Sampling Methods: The SDE Euler baseline uses the PC sampler with an Euler–Maruyama predictor, no corrector, uniform time steps, and final-step denoising.The denoising trick can greatly improve FID for ϵ = 10−3.
  • E.3 Sample Quality Comparison with Continuous-Time Sampling Methods: The SDE Improved Euler results follow the original paper and are reported only for ϵ = 10−3.The corresponding relative tolerances are 0.50, 0.10, and 0.05.
  • E.3 Sample Quality Comparison with Continuous-Time Sampling Methods: Table 3 reports CIFAR-10 sample quality using FID across continuous-time methods and varying numbers of function evaluations.The comparison includes diffusion SDE and ODE samplers for ϵ = 10−3 and ϵ = 10−4.
  • E.3 Sample Quality Comparison with Continuous-Time Sampling Methods: The ODE RK45 solver is evaluated by tuning atol and rtol across increasing NFE settings.For ϵ = 10−3, atol = rtol values are 0.1, 0.01, and 0.001; for ϵ = 10−4, they are 0.1, 0.05, 0.02, 0.01, and 0.001.
  • E.3 Sample Quality Comparison with Continuous-Time Sampling Methods: DPM-Solver uses the Appendix D.3 method for NFE ≤20 and an adaptive step-size solver otherwise.It uses DPM-Solver-12 with ϵrtol = 0.05 for ϵ = 10−3 and DPM-Solver-23 with ϵrtol = 0.05 for ϵ = 10−4.
  • E.4 Sample Quality Comparison with RK Methods: The RK comparison examines RK methods alongside DPM-Solver-2 and DPM-Solver-3, with detailed settings provided for the experiments.The section introduces RK2 as the explicit midpoint method and RK3 as Heun’s third-order method.
  • E.4 Sample Quality Comparison with RK Methods: The RK experiments use F (xt, t) for RK2 (t) and RK3 (t), F (ˆxλ, λ) for RK2 (λ) and RK3 (λ), and uniform step sizes in t or λ.The RK2 update is described as the explicit midpoint method, while RK3 is described as Heun’s third-order method.

E.5 Sample Quality Comparison with Discrete-Time Sampling Methods … E.8 Conditional Sampling on ImageNet 256x256

Across discrete-time benchmarks, DPM-Solver is evaluated against established samplers, with different solver orders showing the expected efficiency-quality tradeoffs. Its runtime is nearly identical to DDIM at equal NFE, while 15 NFE produces samples comparable to DDIM at 100 NFE for conditional ImageNet sampling.

  • E.5 Sample Quality Comparison with Discrete-Time Sampling Methods: Sample quality is compared using FID across CIFAR-10, CelebA 64×64, ImageNet 64×64, ImageNet 128×128 with classifier guidance, and LSUN bedroom 256×256.The comparisons vary the number of function evaluations and include DDPM, DDIM, Analytic-DDPM, Analytic-DDIM, GGDM, and DPM-Solver.
  • E.5 Sample Quality Comparison with Discrete-Time Sampling Methods: For CelebA 64x64, DPM-Solver uses Type-1 and Type-2 discrete conversions, Appendix D.3 for NFE ≤20, and the adaptive step size solver thereafter.DDPM and DDIM use quadratic time steps, while Analytic-DDPM and Analytic-DDIM use uniform time steps.
  • E.6 Comparing Different Orders of DPM-Solver: For NFE less than 20, DPM-Solver-fast is usually better than the single-order methods; for larger NFE, DPM-Solver-3 outperforms DPM-Solver-2, which outperforms DPM-Solver-1.The ordering matches the proposed convergence-rate analysis, with actual NFE sometimes below the given NFE for second- and third-order solvers.
  • E.7 Runtime Comparison between DPM-Solver and DDIM: DPM-Solver and DDIM have almost the same runtime at the same NFE because both are dominated by serial evaluations of the large neural network.The other coefficients in DPM-Solver are computed analytically at ignorable cost, and both runtimes scale linearly with NFE.
  • E.7 Runtime Comparison between DPM-Solver and DDIM: DPM-Solver is slightly faster than DDIM in the authors’ implementation under the same NFE because it reduces repetitive coefficient computation.Runtime measurements use discrete-time pretrained diffusion models across datasets on a single NVIDIA A40.
  • E.8 Conditional Sampling on ImageNet 256x256: 15 NFE with DPM-Solver produces conditional ImageNet samples comparable to DDIM with 100 NFE.The comparison uses classifier guidance with classifier scale 1.0, uniform DDIM time steps, and DPM-Solver-fast at 10, 15, 20, and 100 steps.

E.9 Additional Samples

Additional sampling results compare DPM-Solver with DDIM across CIFAR-10, CelebA 64x64, ImageNet 64x64 and 256x256, and LSUN bedroom 256x256. The figures evaluate random or class-conditional samples at 10–20 function evaluations using pretrained diffusion models and matched random seeds.

  • Dataset coverage: Figures 4–8 report additional samples on CIFAR-10, CelebA 64x64, ImageNet 64x64 and 256x256, and LSUN bedroom 256x256.These results are presented across five datasets or resolutions.
  • ImageNet 256x256: ImageNet 256x256 shows DDIM and DPM-Solver samples at 10, 15, 20, and 100 function evaluations with classifier guidance.The samples use pretrained DPMs and the same random seed.
  • CIFAR-10 and CelebA 64x64: CIFAR-10 and CelebA 64x64 compare DDIM with DPM-Solver at 10, 12, 15, and 20 function evaluations using the same random seed.The experiments use pretrained discrete-time DPMs; DDIM uses quadratic time steps.
  • ImageNet 64x64 and LSUN bedroom 256x256: ImageNet 64x64 and LSUN bedroom 256x256 compare DDIM with DPM-Solver at 10, 12, 15, and 20 function evaluations using uniform time steps.Both comparisons use the same random seed and pretrained discrete-time DPMs.
  • ImageNet 256x256: Class-conditional ImageNet 256x256 samples for class 90, lorikeet, compare DDIM with DPM-Solver at 10, 12, 15, and 20 function evaluations with classifier scale 1.0.The comparison uses uniform time steps, the same random seed, and pretrained discrete-time DPMs.
Loading 2206.00927v3…