Source-linked AI summary
A Survey on Generative Diffusion Model
Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, Stan Z. Li
TL;DR
Diffusion models have become a major generative approach spanning imagery, text, speech, biology, and healthcare, motivating a comprehensive account of their development. This survey reviews their formulations, algorithmic enhancements, and applications, and concludes that the field combines broad practical utility with unresolved efficiency and data-related challenges.
Problem
The growing diffusion-model literature spans formulations, algorithmic improvements, and applications, creating a need for a comprehensive and current synthesis.
Method
The survey organizes diffusion research around fundamental formulations, algorithmic enhancements, and wide-ranging applications, with monthly repository updates.
Results
The survey documents successful diffusion-model applications in image, video, 3D, medical, text, speech, time-series, molecule, and graph generation.
Takeaways & Limitations
The review provides a structured guide to diffusion-model advances and practical applications across diverse domains.
Takeaways & Limitations
Diffusion models remain limited by low inference speed, difficulty learning from low-quality data, large-scale computational demands, and biased or uneven sampling.
Abstract
from arXiv · showhide
Deep generative models have unlocked another profound realm of human creativity. By capturing and generalizing patterns within data, we have entered the epoch of all-encompassing Artificial Intelligence for General Creativity (AIGC). Notably, diffusion models, recognized as one of the paramount generative models, materialize human ideation into tangible instances across diverse domains, encompassing imagery, text, speech, biology, and healthcare. To provide advanced and comprehensive insights into diffusion, this survey comprehensively elucidates its developmental trajectory and future directions from three distinct angles: the fundamental formulation of diffusion, algorithmic enhancements, and the manifold applications of diffusion. Each layer is meticulously explored to offer a profound comprehension of its evolution. Structured and summarized approaches are presented in https://github.com/chq1155/A-Survey-on-Generative-Diffusion-Model.
1 INTRODUCTION
Diffusion models are presented as a stable and high-quality approach to generative modeling, built from forward noising and learned reverse denoising. The survey organizes their development around formulations, algorithmic improvements, and broad applications.
- 1 INTRODUCTION: Diffusion models map data to a simple prior, often Gaussian, then learn a reverse process that gradually reconstructs samples.The reverse process uses a neural network trained with denoising score matching.
- 1 INTRODUCTION: Compared with GANs, diffusion models provide a more stable training objective and demonstrate superior generation quality.
- 1 INTRODUCTION: The survey covers fundamental diffusion formulations, algorithmic enhancements, and applications across multiple generation domains.Its application scope includes imagery, video, 3D data, medicine, text, speech, time series, molecules, and graphs.
2 PRELIMINARIES
Diffusion models progressively transform data into Gaussian noise and reverse that process through discrete or continuous dynamics. The preliminaries introduce DDPM, SDE, and conditional formulations, along with score-based training and controllable sampling.
- Diffusion process: Diffusion begins from data distribution p0, incrementally adds noise through intermediate states, and approaches a Gaussian prior pT.The states may be arranged along either a discrete or continuous timeline.
- DDPM: DDPM uses forward and reverse transition kernels across a discrete timeline, with learnable Gaussian reverse kernels parameterized by θ.The reverse kernel predicts the preceding state using learnable mean and variance.
- Score SDE: Small perturbation kernels allow the discrete process to generalize to continuous-time SDEs, with a marginally equivalent reverse-time ODE also available.The continuous formulation provides theoretical support and connects diffusion to ODE/SDE techniques.
- DDPM: The DDPM objective minimizes a variational bound involving prior, reconstruction, and posterior-versus-reverse divergence terms.The simplified objective is linked to denoising score matching and is commonly used for training.
- Score SDE: Score-based training estimates the score function using a weighted denoising score-matching loss whose optimum equals the true score under the stated formulation.
- Score SDE: Probability-flow ODE sampling is deterministic and supports larger step sizes than SDE sampling, enabling faster solvers such as PNDMs and DPM-Solver.
- Conditional diffusion: Conditional diffusion incorporates a condition c into the score network during training to generate samples from conditional distributions.Conditions can include class labels or text linked to the data.
3 ALGORITHM IMPROVEMENT
The survey organizes diffusion-model improvements around faster sampling, improved forward processes, likelihood optimization, and integration with faster generative models. These approaches target slow iterative sampling and the limitations of pixel-space diffusion while also addressing bridges between arbitrary distributions.
- Sampling Acceleration: Slow iterative sampling is a central practical limitation motivating acceleration methods and other algorithmic improvements.The survey highlights distillation, schedule optimization, training-free acceleration, and integration with faster generative models.
- Knowledge Distillation: Knowledge distillation reduces sampling steps or model size by aligning simpler models with the original diffusion trajectory.It frames distillation as trajectory optimization across distributions for faster, controllable generation.
- Training Schedule Optimization: Training-schedule improvements modify diffusion schemes, noise schemes, truncation, and diffusion kernels to balance speed, fidelity, and learning performance.Examples include variable diffusion-step counts, one-step sampling from less-diffused data, blurring, and masking-based kernels.
- Training-Free Acceleration: Training-free acceleration applies advanced ODE/SDE samplers, analytical solutions, and dynamic programming to pretrained diffusion models without retraining.ODE solvers such as Heun’s second-order method and DPM-Solver reduce the number of function evaluations, while analytical methods have distribution-specific assumptions.
- New Forward Processes: Latent-space diffusion, alternative forward processes, and hybrid VAE/GAN pipelines simplify or accelerate generation beyond conventional pixel-space Brownian motion.Latent diffusion jointly or sequentially uses representation models and diffusion models, while VAE/GAN hybrids provide intermediate or direct predictions.
- Bridging Distributions: Bridging arbitrary distributions remains challenging because standard diffusion models primarily transform simple Gaussian distributions.Proposed solutions include α-blending, Rectified Flow, general interpolant ODEs, Schrödinger Bridges, and Gaussian junctions.
4 APPLICATION
Diffusion models are applied across image, 3D, video, medical, text, and time-series generation. Applications adapt conditioning, representations, temporal modeling, and guidance to the structure and data constraints of each domain.
- Image Generation: Image-generation systems support unconditional, class-conditioned, text-conditioned, and image-conditioned synthesis, including editing with encoded image features.Text-to-image methods use cross-attention to inject text embeddings, while image-to-image methods incorporate source-image features.
- 3D Generation: 3D diffusion follows direct training on 3D data or uses 2D diffusion priors because 3D data are limited.Representations include NeRFs, point clouds, voxels, and Gaussian splatting; DreamFusion distills a NeRF using score distillation sampling.
- Video Generation: Video diffusion extends image diffusion with a temporal axis and temporal layers that model cross-frame dependence.Representative systems include Video Diffusion Models, Make-A-Video, AnimatedDiff, RVD, FDM, MCVD, and RaMViD.
- Medical Analysis: Diffusion models address medical-analysis settings with scarce annotated data through in-distribution analysis and cross-distribution generation.Applications include super-resolution, classification, noise robustness, image translation, registration, anomaly detection, and synthetic-data enrichment.
- Text Generation: Text diffusion enables parallel generation, trading faster generation for relatively weaker semantic coherence than autoregressive models.Approaches operate in discrete word spaces or latent token spaces using categorical transitions, enhanced losses, and specialized architectures.
- Time Series Generation: Time-series diffusion combines masking-based conditioning, self-conditions, and temporal-spatial modules for inpainting, prediction, and probabilistic generation.Reported applications include continuous healthcare and environmental data generation and spatio-temporal learning.
5 CONCLUSIONS & DISCUSSIONS
The survey presents diffusion models through their formulations, algorithmic enhancements, and broad applications, while identifying generalization, computational, and sampling challenges. It positions multimodal generation with LLMs and semi-supervised learning as future directions.
- Conclusions: The survey synthesizes diffusion-model theories, algorithmic improvements, and applications into a comprehensive guide.Its scope is intended to explain model advancements and practical applications.
- Comparison to Existing Surveys: Compared with existing surveys, this work broadly covers algorithm enhancement and wide-ranging applications while tracking updates through a monthly GitHub repository.The comparison spans surveys focused on general diffusion, domains, and data structures.
- Challenges Under Data Limitations: Diffusion models face low inference speed, weak generalization from low-quality data, large-scale computational demands, and sensitivity to biased sampling.These constraints can involve extended training, high memory usage, failure to converge, and limited adaptability across domains or demographics.
- Challenges Under Data Limitations: Improving pattern and correlation identification, data sampling, model parameters, and model structures is presented as necessary for better generalization with limited data.The stated goal is generation that matches training data and meets specific requirements.
- Advanced Multi-modal Generation Leveraging LLMs: Integrating large language models is proposed to advance multimodal generation across text, images, and other modalities.The survey associates LLM integration with improved cross-modal understanding, prompt-based generation efficiency, and broader domains.
- Future Directions: Semi-supervised learning is identified as a way to use unlabeled data for conditional generation and generalization when labeled data are limited.The survey presents this as an opportunity for combining diffusion models with traditional machine-learning theories.
APPENDIX B EVALUATION METRIC
The Inception Score evaluates generated images through diversity and quality components, using ImageNet-based class information and sample–class similarity.
- The Inception Score evaluates generated-image diversity and resolution using the ImageNet dataset.
- Its diversity component uses the class entropy of generated samples, with larger entropy indicating greater diversity.
- Its quality component measures similarity between each sample and related class images through entropy-based evaluation.
B.2 Frechet Inception Distance (FID)
FID addresses reference-dataset bias by measuring the distance between real and generated data distributions through their means and covariances. Negative log-likelihood is another metric used to describe distribution matching.
- FID was proposed to reduce bias arising from evaluation against specific reference datasets.
- FID measures the distance between real-world and generated data distributions using their means and covariances.The generated and real distributions are represented by separate mean and covariance terms.
- Negative log-likelihood describes all modes of a data distribution and is used by some diffusion models as a distribution-matching measure.
APPENDIX C BENCHMARKS
The survey provides benchmarks for landmark diffusion models and improved techniques across multiple evaluation metrics and datasets, with results ordered by NFE for comparison.
- Benchmarks report FID, Inception Score, and NLL for diverse datasets including CIFAR-10, ImageNet, and CelebA-64.
- The selected performances are listed according to NFE in descending order to facilitate comparison.
- The benchmark set covers landmark models alongside techniques intended to improve their performance.
C.1 Benchmarks on CelebA-64
The appendix identifies a benchmark table for CelebA-64.
- Table 3 presents benchmarks on CelebA-64.
APPENDIX E TABLE OF NOTATION
The appendix tables organize diffusion-model benchmarks, improved-method details, and application information across several datasets and system notions.
- Tables 4–6 present diffusion-model benchmarks for ImageNet-64 and CIFAR-10, split by NFE thresholds for CIFAR-10.
- Table 7 records details for improved diffusion methods.
- Table 8 provides details for diffusion applications.
- Table 9 lists notions used in diffusion systems.