Improved Denoising Diffusion Probabilistic Models

Alex Nichol; Prafulla Dhariwal

Source-linked AI summary

Improved Denoising Diffusion Probabilistic Models

Alex Nichol, Prafulla Dhariwal

arXiv:2102.09672v1Feb 18, 2021cs.LGcs.AIstat.ML

TL;DR

DDPMs produced high-quality samples but had not achieved competitive log-likelihoods, and their scaling to diverse datasets and fast image sampling remained uncertain. This paper learns reverse-process variances and combines training objectives, achieving better likelihoods, faster sampling with little impact on sample quality, stronger mode coverage than GANs, and smooth scaling with compute.

Problem
DDPMs had high-fidelity samples but lacked competitive log-likelihoods, while scaling to diverse datasets and fast image sampling remained unclear.
Method
The paper learns reverse-process variances through reparameterization, combines variational and simplified objectives, and uses importance sampling to reduce gradient noise.
Results
The modified DDPMs achieve better likelihoods, sample with fewer steps and little quality loss, match GAN sample quality with better recall, and scale smoothly with compute.
Takeaways & Limitations
DDPMs emerge as a promising generative-modeling direction combining good likelihoods, high-quality samples, reasonably fast sampling, and scalable training.
Takeaways & Limitations
The models do not generally achieve optimal log-likelihoods because they use the hybrid objective rather than directly optimizing the variational lower bound.

Abstract

from arXiv · show

Denoising diffusion probabilistic models (DDPM) are a class of generative models which have recently been shown to produce excellent samples. We show that with a few simple modifications, DDPMs can also achieve competitive log-likelihoods while maintaining high sample quality. Additionally, we find that learning variances of the reverse diffusion process allows sampling with an order of magnitude fewer forward passes with a negligible difference in sample quality, which is important for the practical deployment of these models. We additionally use precision and recall to compare how well DDPMs and GANs cover the target distribution. Finally, we show that the sample quality and likelihood of these models scale smoothly with model capacity and training compute, making them easily scalable. We release our code at https://github.com/openai/improved-diffusion

1. Introduction

This paper improves DDPMs to achieve competitive log-likelihoods on high-diversity datasets, reduce sampling to as few as 50 forward passes, and better characterize distribution coverage and scaling. It combines learned reverse-process variances with improved objectives and evaluates diffusion models against GANs and across training compute.

Motivation: DDPM scaling to high-diversity datasets such as ImageNet, and efficient image sampling with few steps, had remained unclear.Prior work showed strong results on CIFAR-10 and LSUN, while few-step generation had been demonstrated for audio rather than images.
Method: Learned reverse-process variances, a simple reparameterization, and a hybrid VLB objective make DDPM log-likelihoods competitive with other likelihood-based models.The hybrid objective combines the VLB with Ho et al.’s simplified objective.
Optimization: Importance sampling reduces the gradient noise of direct log-likelihood optimization and achieves better log-likelihoods than the hybrid objective.The paper reports that the hybrid objective itself outperforms direct optimization before importance sampling is applied.
Fast sampling: 50 forward passes suffice for good samples with very little change in sample quality, compared with the hundreds required by DDPM.Learned variances enable substantially faster sampling for practical applications.
Distribution coverage: Diffusion models achieve much higher recall than GANs at similar FID, suggesting broader coverage of the target distribution.The comparison uses improved precision and recall metrics.
Scaling: Increasing model size and training compute produces predictable improvements in performance.The evaluation examines model performance as both capacity and training compute increase.

2. Denoising Diffusion Probabilistic Models

DDPMs define a Gaussian forward noising process that transforms data into near-isotropic Gaussian latents, then learn an approximate reverse process to generate samples. Their variational lower bound decomposes into tractable Gaussian KL terms, while the reverse mean can be parameterized through predicted noise, data, or mean.

Forward and reverse processes: The forward process transforms x0 ∼ q(x0) into latents x1 through xT by adding Gaussian noise with variance βt ∈ (0, 1) at timestep t.The process uses a fixed noising schedule and diagonal Gaussian noise at each timestep.
Forward and reverse processes: With sufficiently large T and a well-behaved βt schedule, xT is nearly isotropic Gaussian, enabling reverse sampling from xT ∼ N(0, I).Because the exact reverse distribution depends on the full data distribution, DDPMs approximate it with a neural network.
Variational objective: The variational lower bound is decomposed as Lvlb := L0 + L1 + ... + LT−1 + LT.Aside from L0, each term is a Gaussian KL divergence evaluable in closed form; image likelihood for L0 uses 256 bins per color component.
Reverse-process parameterization: The reverse-process mean µθ(xt, t) can be parameterized by directly predicting the mean, predicting x0, or predicting noise ϵ.The predicted x0 or ϵ can be transformed into µθ(xt, t) using the forward-process equations.
Reverse-process parameterization: Predicting ϵ worked best for Ho et al. (2020), particularly with a reweighted loss that produced better sample quality than directly optimizing Lvlb.The reweighted objective can be viewed as a reweighted form of Lvlb without the terms affecting Σθ.

3. Improving the Log-likelihood

The authors improve DDPM log-likelihood by learning reverse-process variances, using a hybrid objective and cosine noise schedule, and show that these changes improve likelihood while preserving sample quality. Importance sampling makes direct variational-bound optimization less noisy, while optimizing it further improves likelihood at higher FID.

Baseline and diffusion length: 3.99 bits/dim is achieved by the baseline setup on ImageNet 64 × 64 after 200K iterations, improving to 3.77 when T increases from 1000 to 4000.The remainder of the section uses T = 4000.
Learned variances: Learning reverse-process variances is motivated because early diffusion steps contribute most to the variational lower bound, although β_t and ˜β_t remain close over most of the process.The authors parameterize variance as an interpolation between β_t and ˜β_t and use a hybrid objective with λ = 0.001.
Cosine noise schedule: Skipping up to 20% of reverse diffusion barely worsens FID for the linear-schedule model, motivating a cosine schedule that preserves information longer.The linear schedule destroys information more quickly than necessary for 64 × 64 and 32 × 32 images.
Training objectives: The hybrid objective achieves better training-set log-likelihood than direct L_vlb optimization given the same training time, because L_vlb has much noisier gradients.The authors measure gradient noise scales for both objectives and seek to reduce L_vlb variance.
Importance sampling: Importance sampling makes L_vlb considerably less noisy and yields the authors’ best log-likelihoods, whereas it does not help when optimizing the less-noisy L_hybrid objective.The importance-sampled L_vlb curve is shown as the strongest direct variational-bound result.
Ablation results: L_hybrid with the cosine schedule improves log-likelihood while maintaining similar FID to the Ho et al. baseline, whereas optimizing L_vlb further improves likelihood at higher FID.The authors generally prefer L_hybrid because it boosts likelihood without sacrificing sample quality, and their best likelihood models are competitive with conventional methods.

4. Improving Sampling Speed

Sampling can be substantially accelerated without fine-tuning: Lhybrid models trained for 4000 steps retain high sample quality with far fewer sampling steps, while fixed-sigma models degrade more. Compared with DDIM, Lhybrid achieves near-optimal FIDs at 100 steps, whereas DDIM is better below 50 steps but worse at 50 or more.

Sampling acceleration: 4000 training steps can be reduced to seconds-long sampling with high-quality outputs, without fine-tuning pre-trained Lhybrid models.The models otherwise take several minutes per sample on a modern GPU.
Sampling acceleration: 100 sampling steps suffice for near-optimal FIDs in fully trained Lhybrid models.This result holds for models trained with 4000 diffusion steps.
Sampling acceleration: Fixed-sigma Lsimple models suffer substantially larger sample-quality losses when sampling steps are reduced than Lhybrid models with learned sigmas.Lhybrid models maintain high sample quality under reduced-step sampling.
Comparison with DDIM: DDIM produces better samples with fewer than 50 sampling steps but worse samples with 50 or more, while performing worse early in training.DDIM closes its gap to other samplers as training continues.

5. Comparison to GANs

The section compares class-conditional diffusion models with BigGAN-deep using precision, recall, and FID rather than likelihood. BigGAN-deep achieves better FID than the smaller diffusion model, while diffusion appears stronger at covering distribution modes.

Method: The comparison uses precision and recall because likelihood is difficult to compare directly with GANs.Both models are trained class-conditionally, with class information added through the timestep-embedding pathway.
Experimental setup: The experiments train a 100M-parameter diffusion model for 1.7M steps, a 270M-parameter model for 250K iterations, and a 100M-parameter BigGAN-deep model.Diffusion models use the Lhybrid objective and 250 sampling steps.
Experimental setup: Metrics use 50K generated samples for direct comparison with other works, while FID reference features come from the full training set.This is the only reported ImageNet 64 × 64 FID computed using 50K samples.
Results: BigGAN-deep outperforms the smaller diffusion model in FID but struggles with recall.The result suggests diffusion models cover more distribution modes than comparable GANs.

6. Scaling Model Size

The section investigates how DDPM performance scales with training compute and model capacity. Preliminary results suggest FID follows a predictable power law, while validation NLL scales less favorably.

Motivation: These results are preliminary and examine how FID and NLL change with training compute as model size and training time increase.The motivation follows the broader observation that larger models and more training tend to improve performance.
Experimental setup: Four ImageNet 64 × 64 models varied capacity using first-layer widths of 64, 96, 128, or 192 channels under the Lhybrid objective.The models used depth multipliers across all layers to change capacity.
Scaling results: FID improves approximately linearly on a log-log compute plot, suggesting that it follows a power-law scaling relationship.Figure 10 compares FID and validation NLL against theoretical training compute.
Scaling results: Validation NLL does not fit a power law as cleanly as FID, indicating less-favorable scaling behavior.The section proposes unexpectedly high irreducible loss or overfitting as possible explanations.
Limitations: The models are not generally log-likelihood-optimal because they use Lhybrid rather than directly optimizing Lvlb, preserving both likelihood and sample quality.This training-objective choice limits how directly the scaling results characterize optimal log-likelihood.

7. Related Work

Related work applies DDPMs to conditioned audio generation, score matching, fast sampling, and energy-based reverse diffusion. These studies explore improved objectives, architectures, and alternative sampling processes for high-fidelity or faster generation.

Conditioned audio generation: Chen et al. (2020b) and Kong et al. (2020) use DDPMs to generate high-fidelity audio conditioned on mel-spectrograms.Chen et al. (2020b) combine an improved schedule with L1 loss to sample in fewer steps with very little reduction in sample quality.
Score matching: Jolicoeur-Martineau et al. (2020) use score matching and an adversarial objective to improve x0 predictions, but report that a better architecture removes its necessity.This suggests adversarial training is not required for powerful generative modeling.
Fast sampling: Song et al. (2020a) and Song et al. (2020b) propose fast sampling algorithms for DDPM-objective models using different sampling processes.Song et al. (2020a) derive an implicit model with DDPMs’ marginal noise distributions and deterministic noise-to-image mapping; Song et al. (2020b) discretize a continuous SDE and identify a corresponding ODE.
Energy-based reverse diffusion: Gao et al. (2020) model reverse diffusion steps with an energy-based model, potentially reducing the diffusion steps needed for good samples.The passage presents fewer required diffusion steps as a potential implication rather than a demonstrated result.

8. Conclusion · A. Hyperparameters · B. Fast Sampling on LSUN 256 × 256

The paper concludes that modified DDPMs improve likelihood, preserve sample quality, enable faster sampling, and achieve better mode coverage than GANs, while scaling with compute. It also specifies UNet-based hyperparameters and shows that sampling performance on LSUN depends on the number of diffusion steps and training configuration.

8. Conclusion: Learning Σθ with the proposed parameterization and Lhybrid objective improves likelihood with little impact on sample quality and enables sampling with many fewer steps.The authors report that this brings DDPM likelihoods closer to other likelihood-based models.
8. Conclusion: DDPMs match GAN sample quality while achieving much better mode coverage as measured by recall, and additional training compute improves both sample quality and log-likelihood.
8. Conclusion: The combined results make DDPMs attractive because they offer good likelihoods, high-quality samples, reasonably fast sampling, a stationary training objective, and scalable training compute.
A. Hyperparameters: The experiments use a UNet architecture with four-head multi-head attention at 16x16 and 8x8 resolutions, plus modified timestep conditioning.
A. Hyperparameters: ImageNet 64 × 64 uses four downsampling steps with three residual blocks each, mirrored upsampling, stage widths [C, 2C, 3C, 4C], and C = 128 in ablations.
A. Hyperparameters: CIFAR-10 uses three residual blocks per downsampling stage, widths [C, 2C, 2C, 2C] with C = 128, and dropout 0.1 for the linear schedule versus 0.3 for the cosine schedule.
A. Hyperparameters: The models use Adam; most experiments use batch size 128, learning rate 10^-4, and EMA rate 0.9999, while larger class-conditional ImageNet experiments use batch size 2048.

C. Sample Quality on ImageNet 256 × 256

The section compares direct and two-stage diffusion models for class-conditional ImageNet 256 × 256 generation. The two-stage model combines a pretrained 64 × 64 model with a diffusion upsampler, whose 256 × 256 samples have FID 12.3 versus 2.92 at 64 × 64.

Model configurations: Two class-conditional ImageNet 256 × 256 models were trained: one directly models 256 × 256 images, while the other uses two diffusion stages.The two-stage approach chains a pretrained 64 × 64 model with an upsampling diffusion model.
Model configurations: The upsampling model conditions its UNet on the downsampled 64 × 64 image x64 in addition to the class label y.It models p(x256|x64, y) after the pretrained p(x64|y) model.
Sample quality: FID 2.92 is achieved by the 64 × 64 model, compared with FID 12.3 after upsampling to 256 × 256.Both models use 250 sampling steps.

D. Combining Lhybrid and Lvlb Models · E. Log-likelihood with Fewer Diffusion Steps

The paper combines θvlb and θhybrid according to their timestep-specific strengths, improving the likelihood–sample-quality trade-off on ImageNet 64 × 64. It also improves log-likelihood under reduced-step sampling by modifying the strided timestep schedule.

D. Combining Lhybrid and Lvlb Models: θvlb performs better at the beginning and end of diffusion, whereas θhybrid performs better through the middle.Values below 1.0 in Figure 13 indicate that θhybrid is better than θvlb at that timestep.
D. Combining Lhybrid and Lvlb Models: The differing timestep strengths suggest that θvlb emphasizes imperceptible details, which corresponds to lower sample quality.
D. Combining Lhybrid and Lvlb Models: The ImageNet 64 × 64 ensemble uses θhybrid for t ∈[100, T −100) and θvlb elsewhere.The ensemble was constructed to combine the two models’ timestep-specific strengths.
D. Combining Lhybrid and Lvlb Models: 19.9 FID and 3.52 bits/dim NLL were achieved by the ImageNet 64 × 64 ensemble.This was only slightly worse than θhybrid in FID and better than both models in NLL.
E. Log-likelihood with Fewer Diffusion Steps: Constant striding did not significantly affect FID but drastically reduced log-likelihood.The comparison covers ImageNet 64 × 64 and CIFAR-10 models trained with 4000 diffusion steps.
E. Log-likelihood with Fewer Diffusion Steps: Including every t from 1 to T/K alongside a strided subset greatly improves log-likelihood over a uniformly strided schedule.The modification requires T/K extra evaluation steps and uses the same strided subset of timesteps as for FID.

F. Overfitting on CIFAR-10 · G. Early stopping for FID

On CIFAR-10, all models eventually overfit, with cosine-schedule models reaching optimal FID and then overfitting faster than linear-schedule models. The same FID degradation appeared on class-conditional ImageNet-64, where EMA settings of 0.9999 and 0.99995 performed best in a hyperparameter sweep.

F. Overfitting on CIFAR-10: Increasing FID reflected overfitting artifacts similar to those previously reported when CIFAR-10 overfitting became excessive.The linear schedule learned more slowly but did not overfit as quickly.
F. Overfitting on CIFAR-10: All CIFAR-10 models overfit but tended to reach similar optimal FID during training.The study reports that models eventually passed through a similar best FID before degrading.
F. Overfitting on CIFAR-10: Cosine-schedule models reached optimal performance and then overfit more quickly than linear-schedule models at fixed dropout.The authors compensated by using more dropout for cosine models.
G. Early stopping for FID: Figure 17 reports a sweep of dropout and EMA hyperparameters for class-conditional ImageNet-64.The sweep was used to tune EMA before further dropout experiments.
G. Early stopping for FID: Class-conditional ImageNet-64 also exhibited overfitting despite being larger and more diverse than CIFAR-10.The main observable effect was worsening FID over training.
G. Early stopping for FID: 0.9999 and 0.99995 worked best for the EMA hyperparameter in the ImageNet-64 sweep.These values were identified before testing runs with dropout 0.1 and 0.3.

H. Samples with Varying Steps and Objectives

The section visualizes unconditional ImageNet 64 × 64 and CIFAR-10 samples from Lhybrid models as sampling steps are reduced, and compares samples from Lhybrid and Lvlb objectives. These experiments use 4K diffusion-step models trained for 1.5M iterations on ImageNet and 500K on CIFAR-10.

Objectives: Using the exact same random noise, ImageNet 64 × 64 and CIFAR-10 samples compare Lhybrid and Lvlb models trained for 1.5M and 500K iterations, respectively.Lhybrid samples appear on top and Lvlb samples on the bottom in both comparisons.