PaperlayerView exact revision on arXiv

Source-linked AI summary

An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization

Minshuo Chen, Song Mei, Jianqing Fan, Mengdi Wang

arXiv:2404.07771v1cs.LGmath.STstat.MLstat.TH

TL;DR

Despite diffusion models’ broad empirical success, theory remains limited, particularly for conditional guidance and task-specific adaptation. This paper surveys applications and theory, progressing from unconditional to conditional models and connecting conditional diffusion to black-box optimization. It reviews theoretical results and open questions across these areas.

  • Problem

    Existing theory chiefly provides sampling and distribution-learning guarantees for unconditional models, while theoretical foundations for conditional guidance design and task-specific adaptation remain limited.

    Source: §1 · p. 3

  • Method

    The paper reviews diffusion applications and theory, moving from unconditional models to conditional counterparts and examining conditional diffusion for black-box optimization.

    Source: §6 · p. 20, §8 · p. 25

  • Results

    The review covers score approximation and estimation, conditional guidance, and sampling theory, including a first sampling theory for discrete diffusion under access to estimated ratios.

    Source: §4.2.3 · p. 14, §5.1 · p. 18, §5.3 · p. 19

  • Takeaways & Limitations

    The review connects conditional diffusion to high-dimensional structured optimization by formulating solution search as conditional sampling.

    Source: §6 · p. 20, §8 · p. 25

    For discrete diffusion, estimating ratios from finite samples and designing data-relevant transition kernels remain open; the cited sampling theory assumes estimated ratios are available.

    Source: §7.3 · p. 24

Abstract

from arXiv · show

Diffusion models, a powerful and universal generative AI technology, have achieved tremendous success in computer vision, audio, reinforcement learning, and computational biology. In these applications, diffusion models provide flexible high-dimensional data modeling, and act as a sampler for generating new samples under active guidance towards task-desired properties. Despite the significant empirical success, theory of diffusion models is very limited, potentially slowing down principled methodological innovations for further harnessing and improving diffusion models. In this paper, we review emerging applications of diffusion models, understanding their sample generation under various controls. Next, we overview the existing theories of diffusion models, covering their statistical properties and sampling capabilities. We adopt a progressive routine, beginning with unconditional diffusion models and connecting to conditional counterparts. Further, we review a new avenue in high-dimensional structured optimization through conditional diffusion models, where searching for solutions is reformulated as a conditional sampling problem and solved by diffusion models. Lastly, we discuss future directions about diffusion models. The purpose of this paper is to provide a well-rounded theoretical exposure for stimulating forward-looking theories and methods of diffusion models.

1 Introduction

Diffusion models support diverse generation and guided sampling, but theory has focused mainly on unconditional distribution learning, leaving conditional guidance and task adaptation less understood. This paper reviews applications and theory, then connects conditional diffusion to high-dimensional black-box optimization.

1 Introduction

  • Diffusion models have expanded beyond vision and audio into text, sequential modeling, reinforcement learning, control, and life science.The paper also notes methodological work on discrete generation, faster sampling, and fine-tuning toward desired properties.Source: §1 · p. 2
  • Conditional diffusion models are reviewed as samplers guided toward task-desired properties, with guidance signals steering generation.The paper calls diffusion models with guidance conditional diffusion models.Source: §1 · p. 3
  • The paper identifies a theory–practice gap: guarantees mainly address unconditional distribution estimation, while principled guidance design and task-specific adaptation lack theoretical foundations.It frames accurate, efficient distribution learning and guidance alignment and sample complexity as central questions.Source: §1 · p. 3
  • The review proceeds from diffusion applications and theory to conditional models, then considers future directions and interdisciplinary connections.Its theory coverage includes score learning, approximation and estimation, distribution learning, and sampling.Source: §1 · pp. 3–4
  • The review presents conditional diffusion as an approach to high-dimensional structured optimization, generating high-fidelity solutions that preserve data latent structure.It reports solution quality aligned with optimal off-policy bandit performance.Source: §1 · p. 4

2 Diffusion Model Preliminaries

The preliminaries describe diffusion as noise corruption followed by reverse-time generation, with the score function guiding the backward process. Conditional models keep guidance fixed during corruption and learn a conditional score for guided sampling.

2.1 Forward and Backward Processes

  • The forward process progressively corrupts data with Gaussian noise, while the backward process removes noise to generate new samples.In the continuous-time account, the forward process approaches standard Gaussian noise; the reverse process uses the score function.Source: §2 · p. 4, Figure 1 · p. 4, §2.1 · p. 5
  • In practice, the backward process substitutes a standard Gaussian initialization and an estimated neural-network score because the true score and terminal distribution are unknown.Sampling simulates a discretization; thousands of backward steps can be time-consuming, motivating acceleration methods.Source: §2.1 · p. 5

2.2 Conditional Diffusion Models

  • Conditional diffusion targets samples from P(·|y), corrupting x while keeping guidance y fixed and reversing the conditional forward process for generation.Its key distinction from unconditional diffusion is estimating the conditional score, which can be related to the unconditional score and motivates guidance and fine-tuning methods.Source: §2.2 · pp. 5–6

3 Emerging Applications of Diffusion Models

The applications review presents diffusion models as flexible conditional generators across vision, audio, control, reinforcement learning, and life science. It also describes reframing black-box optimization as conditional sampling, with target selection constrained by the coverage of collected data.

3.1 Vision and Audio Generation

  • Conditional diffusion models are used across image and audio generation, editing, restoration, and enhancement, with samples guided by labels, text prompts, or known image regions.Text-to-image methods can fine-tune a model using reward alignment with the desired prompt [78].Source: §3.1 · pp. 6–7, Figure 2 · p. 7

3.2 Control and Reinforcement Learning

  • In control and reinforcement learning, diffusion models represent state-conditioned action policies or generate high-reward state-action trajectories directly.Decision Diffuser conditions trajectories on reward, constraints, or skills [118].Source: §3.2 · pp. 7–8, Figure 3 · p. 8

3.3 Life-Science Applications

  • Life-science applications include protein and drug design, with diffusion models generating protein sequences conditioned on desired properties.The properties described include natural-sequence likelihood, binding affinity, and aggregation tendency.Source: §3.3 · pp. 8–9

3.4 Black-Box Optimization

  • Across control, reinforcement learning, and life science, guidance can be represented as a reward function, making conditional diffusion a sampler for optimizing that reward.The review characterizes this as generating samples from a conditional distribution aimed at reward optimization.Source: §3.4 · p. 9
  • Offline black-box optimization can be reframed as sampling solutions conditioned on a target objective value, but choosing an excessively high target may require extrapolation and degrade quality.The suitable target depends on coverage of the collected data; sample reweighting is one method used to encourage large-reward solutions.Source: §3.4 · p. 9

4 Theoretical Progress on Unconditional Diffusion Models

The review links diffusion performance to accurate score learning, covering network approximation, score estimation, and distribution estimation. It emphasizes structural assumptions as a way to understand score properties and high-dimensional statistical rates.

4 Theoretical Progress on Unconditional Diffusion Models

  • The review organizes unconditional diffusion theory around learning the score, whose accuracy is tied to diffusion-model performance.It considers neural-network approximation and estimation, followed by statistical rates for estimating the data distribution.Source: §4 · pp. 9–10
  • Neural-network choices for score approximation draw on universal and adaptive approximation capabilities, while data assumptions such as low-dimensional support and graphical structure induce score structure.The review also examines whether score estimation can avoid curse-of-dimensionality issues in high-dimensional settings.Source: §4 · p. 10

4.1 Learning Score Function

The section explains how score learning turns an uncomputable objective into a sample-based denoising objective, and why practical estimation stops before time zero. It also describes the U-Net structure used in score learning and generation.

4.1 Learning Score Function

  • Denoising score matching replaces the uncomputable score objective with an equivalent objective that can be approximated from finite samples.The forward Gaussian transition kernel has an analytical score, which is the noise added to the clean sample.Source: §4.1 · p. 10
  • As time approaches zero, the score’s orthogonal component grows without bound for nonzero x, motivating an early stopping time t0 > 0.The term enforces the orthogonal component to vanish, reproducing the low-dimensional subspace structure in generated samples.Source: §4.1 · pp. 10–11
  • In practice, the objective is approximated with n i.i.d. data points, Gaussian-corrupted samples, and times sampled from [t0, T].Source: §4.1 · p. 11
  • The U-Net compresses the image and time embedding before lifting them back to the original dimension, using convolution layers and shortcut connections.The architecture is used at each discretized backward-process step to transform samples.Source: §4.1 · p. 11, Figure 4 · p. 11

4.2 Score Approximation and Estimation

The section reviews score-network approximation and estimation theory across Euclidean, low-dimensional subspace, and graphical-model data. It highlights how data structure affects network design and sample-complexity results.

4.2 Score Approximation and Estimation

  • Score networks must be rich enough to approximate the true score yet sufficiently simple to learn from finite samples.Source: §4.2 · p. 11

4.2.1 Score Approximation Guarantees

  • Score approximation is harder than conventional function approximation because the score is defined on an unbounded high-dimensional domain and depends on time.The target error is often measured in L2.Source: §4.2.1 · p. 12
  • For low-dimensional subspace data, the simplified U-Net’s linear encoder and decoder support efficient score approximation without dependence on ambient dimension D.The network size scales as e O(ϵ^-γ), with γ depending only on subspace dimension when such structure exists.Source: §4.2.1 · pp. 12–13, Figure 5 · p. 12

4.2.2 Sample Complexity of Score Estimation

  • For Lipschitz scores on unknown d-dimensional subspaces, [91] gives an e O(ϵ^-(d+5)) sample complexity and shows the subspace can be estimated automatically.The result assumes Gaussian-tailed data; kernel methods attain e O(ϵ^-(d+4)) for Lipschitz scores.Source: §4.2.2 · p. 13

4.2.3 Score Estimation in Graphical Models

  • For Ising graphical models under certain high-temperature conditions, message passing can approximate scores and a residual network yields estimation error bounds without exponential dependence on dimension.The network uses O(D^2L) parameters, where L is the number of layers.Source: §4.2.3 · pp. 13–14

4.3 Sampling and Distribution Estimation

The section connects score-estimation accuracy to sampling error, then reviews end-to-end distribution-learning rates. Its results cover Euclidean and low-dimensional subspace settings, with conditions and data structure shaping the guarantees.

4.3.1 Sampling Theory

  • Sampling analyses bound the gap between generated and data distributions in terms of score-estimation error, with assumptions and error metrics varying across results.Some results require a log-Sobolev inequality, while others relax this to bounded moments.Source: §4.3.1 · p. 14
  • Later analyses improve sampling-error bounds and extend theory to DDIM and probabilistic ODE backward sampling.Discretization error depends on distribution regularity and step size, while forward error reflects terminating the forward process at finite T.Source: §4.3.1 · p. 14

4.3.2 Sample Complexity of Distribution Estimation

  • For smooth densities supported on a cube, diffusion models learn distributions with total-variation error guarantees under technical conditions.Higher smoothness yields more efficient estimation, and the reported rate matches the minimax-optimal Euclidean rate.Source: §4.3.2 · p. 15
  • For data on an unknown d-dimensional subspace, diffusion models can estimate the subspace and learn a distribution there, with rates depending on d rather than ambient dimension D.The result uses smoothness index s = 1 and compares the learned distribution with a slightly perturbed data distribution.Source: §4.3.2 · pp. 15–16

4.4 Alternative Formulation: Stochastic Localization

Stochastic localization provides an alternative generative formulation linked to diffusion models. The process converges in the long-time limit to a draw from the data distribution and is equivalent to the diffusion model’s backward SDE after reparameterization.

4.4 Alternative Formulation: Stochastic Localization

  • Stochastic localization defines a measure-valued process through an SDE initialized at zero, with drift given by a posterior expectation.Source: §4.4 · p. 16
  • As t tends to infinity, Yt/t converges to a random variable distributed according to Pdata.Source: §4.4 · p. 16
  • The stochastic localization process is equivalent to the diffusion model’s backward SDE up to time and scale reparameterization.For spin-glass and spiked-matrix sampling tasks, approximate posterior expectations support efficient sampling in the high-temperature regime.Source: §4.4 · p. 16

5 Theoretical Progress on Conditional Diffusion Models

Conditional diffusion theory extends the study of unconditional models to conditional score learning, statistical guarantees, and guidance effects. The review highlights both guidance’s control–diversity trade-off and unresolved questions about choosing its strength.

5.1 Learning Conditional Score

  • Classifier guidance combines the unconditional score with an external classifier’s gradient, but classifier training on highly corrupted data can limit its performance.Classifier-free guidance instead randomly masks conditioning information to train conditional and unconditional scores together, avoiding the external classifier.Source: §5.1 · p. 17
  • Bayes-rule decomposition motivates fine-tuning pretrained scores with gradients of scalar-valued functions, and universal guidance recovers classifier guidance for discrete labels.The universal-guidance example uses a sample-quality function and loss; classifier-free guidance is not tailored to adaptation because of computational overhead.Source: §5.1 · p. 18

5.2 Conditional Score and Distribution Estimation

  • Conditional score approximation theory extends diffused-basis methods to unbounded domains and conditional distributions, obtaining optimal statistical rates under a mild bounded Hölder norm assumption.The analysis approximates the conditional density and its gradient separately.Source: §5.2 · p. 19

5.3 Theoretical Insights on Strength of Guidance

  • Under mild conditions, stronger guidance increases classification confidence while reducing diversity and the differential entropy of generated conditional distributions in Gaussian mixture models.These theoretical results align with empirical observations.Source: §5.3 · pp. 19–20
  • In discretized backward sampling, large guidance can split a Gaussian mixture’s center component into two clusters, with the effect depending on component locations and step size.A smaller discretization step requires a much larger guidance strength for the split to emerge.Source: §5.3 · p. 20, Figure 7 · p. 20

5 Theoretical Progress on Conditional Diffusion Models

  • A principled rule for selecting guidance strength across tasks remains unavailable, despite theoretical insights into its effects.Conditional diffusion theory is highly limited, and identifying intrinsic smoothness with respect to the conditioning variable remains open for improving dimension dependence.Source: §5 · p. 16, §5.2 · p. 19, §5.3 · p. 20

6 Diffusion Model for Optimization

The section frames data-driven black-box optimization as reward-conditioned sampling, using conditional diffusion to seek high-reward solutions while retaining data structure. Its theory bounds reward quality and fidelity, with concrete settings tied to off-policy bandit sub-optimality.

6 Diffusion Model for Optimization

  • The setting assumes solutions come from pre-collected data because further objective interactions are prohibitive, and that data concentrates on a latent linear subspace.The labeled data can contain noisy real-valued rewards or pairwise human preferences.Source: §6 · pp. 20–21
  • [176] reformulates data-driven black-box optimization as sampling conditional on objective value, with the learned distribution implicitly capturing data latent structures.Figure 8 depicts this reformulation.Source: §6 · p. 21, Figure 8 · p. 21
  • The four-step procedure estimates rewards from labeled data, pseudo-labels unlabeled data, trains a conditional diffusion model, and samples at target reward a.The resulting distribution and reward are estimates rather than ground truth.Source: §6 · p. 21
  • Theorem 2 separates reward-estimation error, on-support diffusion error, and off-support penalty; labeled-sample size often governs the dominant reward-estimation term.The off-support penalty discourages samples from leaving the latent subspace, while higher target rewards can increase distribution shift and hurt sample quality.Source: §6 · p. 22
  • For linear rewards, [176] derives a sub-optimality bound aligned with optimal off-policy bandit performance and depending on latent dimension d; human preferences yield a similar bound.The human-preference analysis uses the Bradley-Terry-Luce choice model.Source: §6 · p. 22

7 Future Directions

The future-directions discussion connects diffusion models to stochastic control, robustness, and discrete data. It also identifies open theoretical questions, including end-to-end robustness analysis and efficient estimation for discrete diffusion.

7 Future Directions

  • The paper highlights stochastic control, distributional robustness, and discrete diffusion as directions for further study.These directions extend the discussion beyond continuous diffusion models.Source: §7 · p. 22

7.1 Connection to Stochastic Control

  • Viewing the score as a control that steers generated samples motivates principled methods and theories for diffusion models under task-specific objectives.For black-box optimization, the conditional score is the control and cost is negative reward.Source: §7.1 · p. 23

7.2 Adversarial Robustness and Distributionally Robust Optimization

  • Diffusion-based adversarial purification has empirical support, but an end-to-end analysis remains missing; diffusion may also cover uncertainty sets relevant to distributionally robust optimization.The robustness procedure repeatedly transforms adversarial examples and assigns a majority-vote label.Source: §7.2 · pp. 23–24

7.3 Discrete Diffusion Models

  • Discrete diffusion uses continuous-time Markov transitions on finite supports, fitting data such as tokens and molecular structures; ratio estimation and principled transition-kernel design remain open problems.The backward process requires estimating probability ratios, which can be sparse and vary widely; [216] proves a sampling theory assuming estimated ratios are available.Source: §7.3 · p. 24

8 Conclusion

The paper surveys diffusion-model applications, sample generation, and theory. It progresses from unconditional to conditional models, connects conditional diffusion to black-box optimization, and discusses future directions.

8 Conclusion

  • The survey covers diffusion-model applications, sample generation, and theoretical underpinnings, moving from unconditional to conditional models and then to black-box optimization and future directions.It discusses continuous-time forward and backward processes, training with guidance, and theories of score approximation, statistical estimation, and sampling.Source: §8 · p. 25
Loading 2404.07771v1…