PaperlayerView exact revision on arXiv

Source-linked AI summary

Classifier-Free Diffusion Guidance

Jonathan Ho, Tim Salimans

arXiv:2207.12598v1cs.LGcs.AI

TL;DR

Diffusion guidance traditionally uses a separately trained noisy-data classifier, raising whether guidance can be performed without classifier gradients. The paper introduces classifier-free guidance, which mixes conditional and unconditional score estimates, and shows it achieves similar FID/IS trade-offs while supporting high-fidelity pure-generative sampling.

  • Problem

    The paper asks whether classifier guidance can be performed without training an extra classifier and without relying on classifier gradients.

  • Method

    Classifier-free guidance jointly trains conditional and unconditional diffusion models, then mixes their score estimates during sampling.

  • Results

    Classifier-free guidance attains a FID/IS trade-off similar to classifier guidance, with best FID at w = 0.1 or w = 0.3 and best IS at w ≥4.

  • Takeaways & Limitations

    Pure generative diffusion models can boost classifier-based IS and FID metrics without classifier gradients or an extra trained classifier.

  • Takeaways & Limitations

    At the fair T = 128 sampling-speed comparison, the method underperforms ADM-G in FID because each step evaluates the denoising model twice.

Abstract

from arXiv · show

Classifier guidance is a recently introduced method to trade off mode coverage and sample fidelity in conditional diffusion models post training, in the same spirit as low temperature sampling or truncation in other types of generative models. Classifier guidance combines the score estimate of a diffusion model with the gradient of an image classifier and thereby requires training an image classifier separate from the diffusion model. It also raises the question of whether guidance can be performed without a classifier. We show that guidance can be indeed performed by a pure generative model without such a classifier: in what we call classifier-free guidance, we jointly train a conditional and an unconditional diffusion model, and we combine the resulting conditional and unconditional score estimates to attain a trade-off between sample quality and diversity similar to that obtained using classifier guidance.

1 INTRODUCTION

The paper asks whether diffusion guidance can avoid a separately trained classifier and introduces classifier-free guidance as a pure-generative alternative. It combines conditional and unconditional score estimates to trade off sample quality and diversity, with effects illustrated on ImageNet and a Gaussian mixture.

  • Illustration: The ImageNet illustration shows increasing classifier-free guidance on the malamute class from left to right, starting with non-guided samples.The example uses a 64x64 ImageNet diffusion model.
  • Illustration: In the Gaussian-mixture illustration, stronger guidance concentrates each class-conditional density into smaller regions farther from other classes.The figure presents increasing guidance from left to right, beginning with the non-guided marginal density.
  • Motivation: Classifier guidance improves diffusion sample quality but requires a separately trained classifier and can complicate the training pipeline.The classifier must be trained on noisy data, so a standard pretrained classifier generally cannot be reused.
  • Method: Classifier-free guidance combines conditional and unconditional diffusion-model score estimates instead of using classifier gradients.The method sweeps a mixing weight to control guidance strength.
  • Method: Increasing classifier-free guidance strength produces a controlled trade-off between sample quality and diversity, similar to classifier guidance.The paper frames this trade-off using FID/IS and precision/recall.

2 BACKGROUND

The paper formulates diffusion modeling as continuous-time denoising across noise scales, then defines conditional and reverse generative processes for sampling. The learned score estimates guide transitions along an increasing sequence of log signal-to-noise levels.

  • Forward process: The continuous-time forward process corrupts data as z_λ = α_λx + σ_λε with λ sampled across a bounded noise-scale interval.The process is variance-preserving, with ε drawn from a standard Gaussian.
  • Forward process: λ represents the log signal-to-noise ratio, and the forward process runs toward decreasing λ.The marginal notation p(z_λ) refers to the noisy-data distribution induced by the forward process.
  • Sampling: The reverse process starts from N(0, I) and applies transitions along λ_min = λ_1 < ··· < λ_T = λ_max.This is the discrete-time ancestral sampler; in the continuous-time limit, correct modeling yields samples distributed as p(z).
  • Sampling: The reverse-process mean uses an estimate x_θ(z_λ) of the clean data, parameterized through ε-prediction.The relation is x_θ(z_λ) = (z_λ − σ_λε_θ(z_λ))/α_λ.
  • Training: The model learns score estimates through denoising score matching over multiple noise scales.With uniform noise-scale sampling, the objective is proportional to a variational lower bound on marginal log likelihood, up to specified terms.
  • Conditional modeling: For conditional generation, the reverse-process approximator receives conditioning information c, such as a class label.The conditional model is written as ε_θ(z_λ, c).

3 GUIDANCE

Guidance adjusts diffusion sampling to trade sample quality against diversity. Classifier-free guidance achieves a similar tradeoff without a separate classifier by combining conditional and unconditional score estimates.

  • Classifier guidance: Classifier guidance modifies the diffusion score with an auxiliary classifier gradient controlled by guidance strength w.The modified score replaces the original score during sampling.
  • Classifier guidance: Increasing classifier guidance can improve Inception score while reducing sample diversity.The mechanism up-weights data assigned high likelihood by the classifier to the correct label.
  • Classifier guidance: In a three-class 2D example, stronger classifier guidance moves probability mass away from other classes and concentrates it in smaller, high-confidence regions.The guided conditionals become markedly non-Gaussian.
  • Classifier-free guidance: Classifier-free guidance empirically trades off FID and IS like classifier guidance, while avoiding a separately trained classifier.Its score combination is not generally the gradient of a classifier because the neural-network score estimates are non-conservative vector fields.
  • Classifier-free guidance: Classifier-free guidance jointly trains conditional and unconditional diffusion models, using a null class identifier for unconditional predictions.Conditioning is randomly discarded with probability p_uncond during joint training.
  • Classifier-free guidance: At sampling time, classifier-free guidance uses the linear combination ˜ϵθ(zλ, c) = (1 + w)ϵθ(zλ, c) −wϵθ(zλ).The resulting guided score is used in the diffusion sampling transition without classifier gradients.

4 EXPERIMENTS

Experiments test classifier-free guidance on class-conditional ImageNet at 64×64 and 128×128 resolution, examining guidance strength, unconditional-training probability, and sampling steps. Guidance produces the intended fidelity–diversity trade-off, while sampling-step and training-probability choices affect quality and efficiency.

  • Varying guidance strength: Sweeping guidance strength yields a clear trade-off: FID decreases monotonically while Inception score increases, with best FID at w = 0.1 or 0.3 and best IS at w ≥4.The experiments use 50,000 samples per guidance value and compare guidance strengths from w = 0 to 4.
  • Varying guidance strength: Increasing guidance strength decreases sample variety while increasing individual sample fidelity, consistent with the intended truncation-like behavior.Strongly guided 128×128 samples can also display saturated colors.
  • Varying unconditional training probability: With unconditional training probabilities p_uncond ∈ {0.1, 0.2, 0.5}, p_uncond = 0.5 performs worse across the IS/FID frontier, while 0.1 and 0.2 perform about equally.The result suggests that only a relatively small portion of model capacity must support unconditional generation for effective guided scores.
  • Varying the number of sampling steps: For 128×128 ImageNet, increasing sampling steps improves quality, and T = 256 provides a good balance between sample quality and sampling speed.The study compares T ∈ {128, 256, 1024}; fair speed comparison with ADM-G uses T = 128 because each classifier-free step evaluates the denoising model twice.

5 DISCUSSION

Classifier-free guidance simplifies guidance by mixing conditional and unconditional score estimates, while exposing trade-offs in speed, diversity, and implementation assumptions.

  • Practical advantages: Classifier-free guidance requires only conditioning dropout during training and score mixing during sampling, rather than a separately trained classifier.The classifier in classifier guidance must be trained on noisy zλ, preventing direct use of a standard pre-trained classifier.
  • Interpretation: Its guided sampler can trade off IS and FID like classifier guidance while using a pure generative model and no classifier gradients.Because unconstrained neural networks need not define conservative vector fields, the sampler’s directions need not resemble classifier gradients or adversarial attacks.
  • Practical advantages: The method can sometimes avoid unconditional-model training when the class distribution is known and has few classes, but this requires one forward pass per class.That alternative becomes inefficient for high-dimensional conditioning.
  • Limitations: Sampling may be slower than classifier guidance because classifier-free guidance runs two diffusion-model forward passes for conditional and unconditional scores.The paper leaves late conditioning injection as a possible mitigation for this cost.
  • Limitations: Increasing fidelity at the expense of diversity raises deployment concerns when underrepresented data require broad sample coverage.The paper identifies maintaining quality while preserving diversity as an open direction.

6 CONCLUSION

The paper presents classifier-free guidance as a way to improve diffusion-model sample quality while reducing diversity, without requiring classifier gradients. It concludes that pure generative models can optimize classifier-based sample-quality metrics through guidance.

  • Classifier-free guidance increases sample quality while decreasing sample diversity in diffusion models.

A SAMPLES

The samples illustrate classifier-free guidance on ImageNet at 64x64 and 128x128 resolutions, including random-class, single-class, and guided-versus-non-guided comparisons.

  • ImageNet 64x64 examples compare random classes on the left with a single malamute class on the right using the same random seed.
  • ImageNet 128x128 examples include comparisons between random classes and a single malamute class.
  • Additional 128x128 ImageNet examples compare non-guided samples with classifier-free guided samples using w = 3.0.
Loading 2207.12598v1…