Source-linked AI summary
Fast-DDPM: Fast Denoising Diffusion Probabilistic Models for Medical Image-to-Image Generation
Hongxu Jiang, Muhammad Imran, Teng Zhang, Yuyin Zhou, Muxuan Liang, Kuang Gong, Wei Shao
TL;DR
Diffusion models are computationally expensive to train and sample, limiting their use in medical imaging. Fast-DDPM aligns training and sampling at 10 time steps and reports state-of-the-art performance across three medical image-generation tasks.
Problem
Diffusion models’ many time steps and large dataset requirements make training and sampling costly, with sampling a 3D volume potentially taking several hours.
Source: §I · p. 1Method
Fast-DDPM aligns training and sampling at 10 time steps and uses uniform or non-uniform noise schedulers for conditional image-to-image generation.
Source: §I · pp. 1–2, B. Fast Denoising Diffusion… · p. 4Results
Fast-DDPM achieved state-of-the-art performance across three medical image-to-image tasks while reducing the process from 1,000 time steps to 10.
Source: §VII · p. 9Takeaways & Limitations
The reported advance has the potential to further research on real-time medical imaging applications of diffusion models.
Source: §VII · p. 9The study focuses on 2D diffusion models, which may not perform optimally compared with 3D models.
Source: D. Limitations and Future Work · p. 9
Abstract
from arXiv · showhide
Denoising diffusion probabilistic models (DDPMs) have achieved unprecedented success in computer vision. However, they remain underutilized in medical imaging, a field crucial for disease diagnosis and treatment planning. This is primarily due to the high computational cost associated with (1) the use of large number of time steps (e.g., 1,000) in diffusion processes and (2) the increased dimensionality of medical images, which are often 3D or 4D. Training a diffusion model on medical images typically takes days to weeks, while sampling each image volume takes minutes to hours. To address this challenge, we introduce Fast-DDPM, a simple yet effective approach capable of improving training speed, sampling speed, and generation quality simultaneously. Unlike DDPM, which trains the image denoiser across 1,000 time steps, Fast-DDPM trains and samples using only 10 time steps. The key to our method lies in aligning the training and sampling procedures to optimize time-step utilization. Specifically, we introduced two efficient noise schedulers with 10 time steps: one with uniform time step sampling and another with non-uniform sampling. We evaluated Fast-DDPM across three medical image-to-image generation tasks: multi-image super-resolution, image denoising, and image-to-image translation. Fast-DDPM outperformed DDPM and current state-of-the-art methods based on convolutional networks and generative adversarial networks in all tasks. Additionally, Fast-DDPM reduced the training time to 0.2x and the sampling time to 0.01x compared to DDPM. Our code is publicly available at: https://github.com/mirthAI/Fast-DDPM.
I. INTRODUCTION
Fast-DDPM targets the computational burden of diffusion models by aligning training and sampling around a small set of time steps. The introduction presents its medical image-generation results and situates the approach among existing sampling-acceleration methods.
I. INTRODUCTION
- Diffusion training can take days on a single GPU, while sampling hundreds of 2D slices for a 3D volume can take several hours.The introduction attributes the computational burden to many discrete time steps and the need for large datasets to model complex image distributions.Source: §I · p. 1
A. Fast Sampling of Diffusion Models
- Fast-sampling research includes training-free solvers and quantization, while training-based distillation reduces steps further but requires additional student-model training.The cited approaches report reductions to 50–100 steps for some solvers, as few as 10 for distillation, and 1–4 for adversarial diffusion distillation.Source: §I · p. 1, A. Fast Sampling of Diffusion Models · p. 2
I. INTRODUCTION
- Training on 1,000 time steps while sampling with DDIM at 100 skips 900 trained time points, motivating Fast-DDPM’s matched 10-step training and sampling.The approach is framed as optimizing time-step utilization.Source: §I · pp. 1–2
- Fast-DDPM achieved state-of-the-art performance across three medical image-to-image tasks, with sampling about 100 times faster and training about 5 times faster than DDPM.The tasks were prostate MRI multi-image super-resolution, low-dose CT denoising, and brain MRI translation.Source: §I · p. 2
- Fast-DDPM introduces uniform and non-uniform noise schedulers with 5–10 time steps, tailored to specific tasks.Its training strategy aligns training and sampling procedures to enhance speed and accuracy.Source: §I · p. 2
B. Diffusion Models in Medical Imaging
Medical imaging uses diffusion models across a range of tasks, but their computational demands constrain broader use, especially for higher-dimensional applications. Diffusion models may be unconditional or conditional, and their noise schedule governs the forward corruption process and its reverse.
B. Diffusion Models in Medical Imaging
- Medical diffusion models have been applied to tasks including segmentation, denoising, reconstruction, super-resolution, and image-to-image translation, while 3D and 4D applications can take weeks to months to train.2D models predominate because of their ease of implementation and lower memory requirements.Source: B. Diffusion Models in Medical Imaging · p. 2
C. Unconditional and Conditional Diffusion Models
- Conditional diffusion models use additional inputs to guide generation, whereas unconditional models generate from the learned data distribution alone.Image-guided models support dense image-to-image tasks such as super-resolution, inpainting, and style transfer.Source: C. Unconditional and Conditional… · pp. 2–3
A. Background
- The forward process adds noise to a clean image until it becomes Gaussian noise, and the reverse process can generate images when the score function is known.A linear-β scheduler with T = 1000 is common because too few steps leave the endpoint non-Gaussian, while too many make it Gaussian early.Source: A. Background · pp. 3–4
B. Fast Denoising Diffusion Probabilistic Model (Fast-DDPM)
Fast-DDPM reduces training and sampling to task-specific schedules with 5–10 time steps. It trains a conditional denoiser on those selected steps and uses a DDIM reverse-time ODE to produce the output.
B. Fast Denoising Diffusion Probabilistic Model (Fast-DDPM)
- Fast-DDPM accelerates DDPM training and sampling using task-specific noise schedulers with only 5–10 time steps.The method aligns training steps with the reduced set used during sampling.Source: B. Fast Denoising Diffusion… · p. 4
- The two proposed α schedulers sample 10 time points uniformly or non-uniformly from [0,1].The non-uniform schedule places 60% of steps above t = 699 and 40% at or below t = 699, reflecting the stated heuristic that later steps contribute more to perceptual quality.Source: B. Fast Denoising Diffusion… · p. 4, Figure 3 · p. 5
- A conditional denoiser estimates the score at each time using the noisy target image and its associated condition image or images.The training loss assigns equal weights to different times.Source: B. Fast Denoising Diffusion… · p. 4
- Sampling starts from Gaussian noise and a condition image, then uses the DDIM sampler to solve the reverse-time ODE and produce the corresponding noise-free output.Noisy ground truth is used during training, not inference.Source: B. Fast Denoising Diffusion… · pp. 4–5
IV. EXPERIMENTS
Fast-DDPM was evaluated on three medical image-to-image generation tasks.
IV. EXPERIMENTS
- The evaluation covers multi-image super-resolution, image denoising, and image-to-image translation.Source: §IV · p. 5
A. Datasets
The experiments use public datasets for prostate MRI super-resolution, low-dose CT denoising, and brain MRI translation, with task-specific paired or triplet inputs and targets.
A. Datasets
- Prostate MRI super-resolution uses three consecutive T2w slices, with the first and third as inputs and the middle slice as ground truth.Training uses 6,979 triplets from 120 volumes; testing uses 580 triplets from 10 volumes.Source: A. Datasets · p. 5
- Low-dose CT denoising maps a scan reconstructed at 10% of routine dose to its corresponding normal-dose scan.The split contains 13,211 image pairs from 38 training patients and 3,501 pairs from 10 evaluation patients.Source: A. Datasets · p. 5
- Brain MRI translation converts registered T1w images to T2w images using paired scans from BraTS 2018.The dataset contains 5,760 training pairs and 768 testing pairs.Source: A. Datasets · pp. 5–6
B. Evaluation Metrics
PSNR measures distortion relative to the ground-truth image’s maximum possible value, while SSIM measures structural similarity in a way that better aligns with human perception.
B. Evaluation Metrics
- PSNR relates the ground-truth image’s maximum possible pixel value to the power of distorting noise.The mean squared error (MSE) denotes the distortion measure.Source: B. Evaluation Metrics · p. 6
- SSIM measures structural similarity between two images and aligns better with human perception of image quality.Its formula uses the images’ means, standard deviations, and covariance, with constants defined using the maximum pixel value.Source: B. Evaluation Metrics · p. 6
C. Training Details
Fast-DDPM used task-dependent noise sampling, a shared U-Net setup, and fewer training iterations than DDPM.
C. Training Details
- Fast-DDPM used non-uniform sampling for super-resolution and uniform sampling for denoising and translation.Both schedulers were generated from a linear β schedule with T = 1000, β1 = 10^-4, and βT = 0.02.Source: C. Training Details · p. 6
- Experiments used DDIM-following U-Nets, Adam at 2 × 10^-4, batch size 16, and 400,000 Fast-DDPM iterations versus 2 million for DDPM.Runs used four 80-GB NVIDIA A100 GPUs, Python 3.10.6, and PyTorch 1.12.1.Source: C. Training Details · p. 6
V. RESULTS
Fast-DDPM outperformed the compared methods in multi-image super-resolution and CT denoising, with reported quality and efficiency gains over DDPM. Qualitative results highlight reconstructions in both tasks.
V. RESULTS
- The evaluation compared Fast-DDPM with GAN- and CNN-based methods, DDPM ancestral sampling, DDIM, and DPM-Solver++, and examined time-step and scheduler ablations.These comparisons and ablations covered the three image-to-image tasks.Source: §V · p. 6
A. Multi-Image Super-Resolution
- PSNR rose from 25.3 to 27.1 and SSIM from 0.83 to 0.89, while training fell from 136 to 26 hours and per-volume inference from 3.7 minutes to 2.3 seconds.Fast-DDPM also outperformed miSRCNN, miSRGAN, DDPM, DDIM, and DPM-Solver++.Source: Table I · p. 6, A. Multi-Image Super-Resolution · p. 6
- Fast-DDPM most effectively reconstructed missing information between the previous and next MRI slices, including the shadowed area highlighted by arrows.The center slice served as ground truth; miSRCNN and miSRGAN failed to accurately reconstruct the shadowed area.Source: A. Multi-Image Super-Resolution · p. 6, Figure 4 · p. 7
B. Image Denoising
- Fast-DDPM preserved fine CT details, with the lung fissure clearest and sharpest; other methods except DDIM and DPM-Solver++ showed blurring and detail loss there.The comparison describes low-dose, normal-dose, and predicted normal-dose CT images for a representative subject.Source: B. Image Denoising · pp. 6–7
- Fast-DDPM outperformed DDPM, REDCNN, and DU-GAN on CT denoising, and exceeded DDIM and DPM-Solver++ in PSNR and SSIM.Against DDPM, inference was approximately 100-fold faster and training approximately 5-fold faster.Source: B. Image Denoising · p. 6, Table II · p. 7
C. Image-to-Image Translation
Fast-DDPM outperformed leading methods on brain MRI translation while substantially reducing training and sampling time. Ablations show that the best number of diffusion steps and noise scheduler depend on the image-to-image task.
C. Image-to-Image Translation
- Fast-DDPM outperformed Pix2Pix, RegGAN, DDPM, DDIM, and DPM-Solver++; training took 27 hours and sampling 13.2 seconds per batch, versus DDPM’s 135 hours and 22.2 minutes.Table III compares performance and average inference time per batch of 360 slices.Source: Table III · p. 7, C. Image-to-Image Translation · p. 7
- Fast-DDPM produced the sharpest, least noisy T2w MRI translations, while Pix2Pix, RegGAN, DDIM, and DPM-Solver++ produced blurry predictions.DDPM recovered some structural detail but yielded relatively low signal-to-noise ratio.Source: C. Image-to-Image Translation · pp. 7–8, Figure 6 · p. 7
D. Ablation Study
- The step-count ablation tested 3 to 1,000 steps, using uniform schedules for denoising and translation and a non-uniform schedule for super-resolution.The tested task-specific step counts are reported in the ablation setup.Source: D. Ablation Study · p. 8
- Super-resolution and denoising performance peaked at 10 steps, whereas image-to-image translation peaked at 100 before declining.The authors attribute the difference to the input’s structural prior in fidelity-enhancement tasks versus the transformations required for translation.Source: D. Ablation Study · p. 8
- Uniform scheduling performed better for denoising and translation, while non-uniform scheduling performed better for super-resolution.The authors conclude that scheduler choice should be tailored to the image-to-image task.Source: D. Ablation Study · p. 8, Table V · p. 8
VI. DISCUSSION
The discussion frames medical image generation around clinical fidelity and the practical cost of diffusion models. It contrasts unconditional generation with conditional tasks, where input guidance can make fewer, well-chosen steps effective.
A. Alignment with Clinical Standards
- Fast-DDPM targets clinically relevant tasks and evaluates outputs using PSNR and SSIM to assess intensity and structural fidelity.The discussion links these qualities to diagnostic accuracy and clinical workflows.Source: A. Alignment with Clinical Standards · p. 8
B. Speed-Accuracy Trade-Off
- Unlike unconditional generation from noise, conditional image generation refines existing structures, so fewer optimized steps can focus capacity on relevant noise levels.The authors hypothesize that prioritizing noise levels on the sampling path may reduce redundancy from training across a broad range of noise levels.Source: B. Speed-Accuracy Trade-Off · pp. 8–9
C. Diffusion Models vs. GANs
The paper concludes that Fast-DDPM achieves state-of-the-art performance across three medical image-to-image tasks while reducing diffusion steps. It also discusses claimed advantages over GANs, limitations of the current study, and potential clinical relevance.
C. Diffusion Models vs. GANs
- The authors report that Fast-DDPM produces higher-quality samples than GANs, avoids artifacts and mode collapse, and is more stable and robust to hyperparameter tuning.They attribute the stability and tuning contrast to avoiding the need to balance generator and discriminator losses.Source: C. Diffusion Models vs. GANs · p. 9
D. Limitations and Future Work
- The study uses manually selected noise schedulers and 2D diffusion models; 3D development was beyond the authors’ computational capacity.The authors propose exploring learned adaptive schedulers and 3D models in a smaller latent space using autoencoders.Source: D. Limitations and Future Work · p. 9
E. Significance and Implications
- The authors connect improved image quality and reduced training and inference time to potential gains in diagnostic accuracy and clinical workflow efficiency.They describe real-time image analysis and more timely interventions as potential implications.Source: E. Significance and Implications · p. 9
VII. CONCLUSION
- Fast-DDPM achieved state-of-the-art performance across three medical image-to-image tasks while reducing diffusion steps from 1,000 to 10.The conclusion says this could further research into real-time medical imaging applications.Source: §VII · p. 9