Source-linked AI summary
Diffusion Probabilistic Models beat GANs on Medical Images
Gustav Müller-Franzes, Jan Moritz Niehues, Firas Khader, Soroosh Tayebi Arasteh, Christoph Haarburger, Christiane Kuhl, Tianci Wang, Tianyu Han, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn
TL;DR
Limited and privacy-constrained medical training data motivates generative synthesis, while GAN diversity and fidelity remain limited. The study introduces Medfusion, a conditional latent DDPM, and compares it with GAN baselines across three medical imaging domains. Medfusion achieved better image-quality metrics across all three datasets, supporting DDPMs as an alternative to GANs for medical image synthesis.
Problem
Medical image generation needs to expand limited, privacy-constrained training data, but GANs have limited diversity and fidelity.
Method
Medfusion combines an image autoencoder with a conditional latent DDPM and is compared with GAN models across ophthalmologic, radiological, and histological datasets.
Results
Medfusion achieved lower FID and higher precision and recall than GANs across all three medical imaging datasets.
Takeaways & Limitations
DDPMs provide a promising alternative to GANs for medical image generation across multiple imaging domains.
Takeaways & Limitations
CheXpert and AIROGS images were trained and generated at lower, square resolutions, and the image-quality metrics were not developed specifically for medical images.
Abstract
from arXiv · showhide
The success of Deep Learning applications critically depends on the quality and scale of the underlying training data. Generative adversarial networks (GANs) can generate arbitrary large datasets, but diversity and fidelity are limited, which has recently been addressed by denoising diffusion probabilistic models (DDPMs) whose superiority has been demonstrated on natural images. In this study, we propose Medfusion, a conditional latent DDPM for medical images. We compare our DDPM-based model against GAN-based models, which constitute the current state-of-the-art in the medical domain. Medfusion was trained and compared with (i) StyleGan-3 on n=101,442 images from the AIROGS challenge dataset to generate fundoscopies with and without glaucoma, (ii) ProGAN on n=191,027 from the CheXpert dataset to generate radiographs with and without cardiomegaly and (iii) wGAN on n=19,557 images from the CRCMS dataset to generate histopathological images with and without microsatellite stability. In the AIROGS, CRMCS, and CheXpert datasets, Medfusion achieved lower (=better) FID than the GANs (11.63 versus 20.43, 30.03 versus 49.26, and 17.28 versus 84.31). Also, fidelity (precision) and diversity (recall) were higher (=better) for Medfusion in all three datasets. Our study shows that DDPM are a superior alternative to GANs for image synthesis in the medical domain.
1 Introduction
Medical image synthesis could expand limited training data, but GANs have constrained diversity and fidelity. Although DDPMs outperform GANs on natural images, broad direct comparisons in medical imaging remained limited.
- Medical deep learning depends critically on the size of available training datasets.
- Privacy issues often prevent pooling large medical datasets across sites, motivating generative models for data availability.Generative models may also support data sharing, modality translation, and improved deep learning performance.
- GANs have been used to generate additional medical data, but producing meaningful medical images remains difficult.
- DDPMs and latent DDPMs have achieved state-of-the-art results and outperformed GANs on natural images.
- A wide-scale direct comparison of latent DDPMs and GANs across multiple medical imaging domains had not yet been conducted.Prior direct comparisons included specific medical use cases such as latent-DDPM generation of 3D brain MRI images.
2 Material and Methods
The study compared Medfusion, a conditional latent DDPM, with established GANs across medical imaging datasets using reconstruction, generative-quality, and qualitative evaluations. Medfusion combines a compressed latent autoencoder with a DDPM and was assessed using FID, precision, recall, and MS-SSIM.
- Datasets: The study used three publicly available medical datasets spanning ophthalmologic, radiological, and histological images.The datasets included AIROGS fundus images, CRCDX histology images, and CheXpert chest radiographs.
- Baselines: GAN baselines were selected for strong quality on the respective datasets, including ProGAN for CheXpert and StyleGAN-3 for AIROGS.
- Medfusion architecture: Medfusion encodes images into an 8-times compressed latent space with an autoencoder, then diffuses the latent representations into Gaussian noise using a DDPM.The autoencoder and diffusion components were trained in two subsequent phases.
- Experimental design: The experiments tested whether the autoencoder preserved medically relevant details and compared real, GAN-generated, and Medfusion-generated images quantitatively and qualitatively.Qualitative comparisons were performed side-by-side.
- Evaluation: FID measured real–synthetic feature agreement, while improved precision and recall measured fidelity and diversity; MS-SSIM measured structural image similarity.
- Implementation: Experiments were implemented in Python using TorchMetrics and executed on an Nvidia RTX 3090.Source code for the compared models and Medfusion was made available through referenced repositories.
3 Results
Medfusion’s autoencoder achieved high numerical reconstruction quality, but qualitative review revealed dataset-specific detail loss that improved with eight latent channels. Across three medical imaging datasets, Medfusion generated more realistic and diverse images than GANs, although artifacts remained in some synthetic radiographs.
- Autoencoder reconstruction: MS-SSIM = 1 and MSE = 0 indicated nearly perfect reconstruction for AIROGS and CheXpert, while CRCDX reconstruction was good but lower.The authors attribute the CRCDX difference most likely to its four-times-higher resolution.
- Autoencoder reconstruction: Dataset-specific reconstruction errors included subtle fundus structural changes, histology color shifts, and reduced sharpness in thorax images.These qualitative deviations were consistent with the high numerical reconstruction results but showed that small medically relevant details could change.
- Autoencoder reconstruction: Eight channels produced a notable reconstruction-quality gain compared with four channels, at the cost of a lower compression ratio.Increasing parameter count was not preferred because the Stable Diffusion four-channel VAE already had about three times as many parameters as Medfusion’s four-channel VAE.
- Image generation comparison: Medfusion achieved higher fidelity and diversity than the corresponding GAN models across all three datasets according to Precision and Recall.The comparison included fundus, histopathology, and chest-radiograph synthesis tasks.
- Image generation comparison: Medfusion generated more realistic-looking images than the GAN models in all three datasets based on the FID metric.Qualitative comparisons were provided for glaucoma, microsatellite stability, and cardiomegaly conditioning.
- Image generation comparison: GAN images showed characteristic artifacts such as duplicate optical discs and artificial histology grids, whereas blurred letters and irregular medical-device borders appeared in both GAN and Medfusion radiographs.Some real images also contained acquisition artifacts, but these differed from the synthetic artifacts.
4 Discussion
The study evaluates Medfusion across ophthalmologic, radiological, and histological medical images, finding higher image quality than GAN-based models. It also identifies resolution, metric validity, and autoencoding as important boundaries for interpretation and future work.
- Medfusion generated higher-quality images than GAN models across eye, histology, and chest datasets, with lower FID scores in every domain.The reported FID scores were 11.63 versus 20.43 for eye images, 30.03 versus 49.26 for histology, and 17.28 versus 84.31 for chest images.
- Medfusion also achieved higher precision and recall than GANs across the three medical-image datasets.The study interprets higher precision as greater fidelity and higher recall as greater diversity.
- A prior 3D brain-MRI study likewise found lower FID for a latent DDPM than for two GANs, although its FIDs were 3 to 4 orders of magnitude lower.The authors suspect this difference relates to 3D rather than 2D data.
- The study used lower-resolution, square CheXpert and AIROGS images to match prior GAN results and accommodate StyleGAN-3 constraints.Future work should evaluate Medfusion against GANs at higher resolutions.
- FID, precision, and recall were not developed specifically for medical images, so their validity for judging medical-image quality remains uncertain.The authors call for medical-image metrics that better proxy human judgment.
5 Conclusion
The conclusion presents DDPMs as a promising alternative to GANs for medical image generation and identifies higher-resolution evaluation and improved quality metrics as priorities.
- DDPMs provide promising new ways to generate medical images besides GANs.
- Future work should examine DDPM behavior at high resolution for both 2D and 3D medical images.
- Future work should develop and assess quality metrics for 2D and 3D medical images.
Funding
The paper reports that this work received no funding.
- The authors state that this work has not received any funding.
- The funding disclosure reports no financial support for the study.
- No funding source is identified for this work.
Supplemental Material
Supplemental analyses document artifacts in real datasets, reconstruction behavior, evaluation-set composition, and sampling-step effects. They show that sampling quality improves substantially before plateauing around 150 steps.
- Visual artifacts: About 300 AIROGS, CheXpert, and CRCDX images were inspected, revealing several severe visual artifacts.The artifacts were described as unnatural, rare in clinical routine, or specific to the datasets.
- Autoencoder: The autoencoder loss combined embedding and reconstruction terms, including Kullback–Leibler divergence, L1 distance, LPIPS, SSIM, and a PatchGAN discriminator.
- Evaluation data: The independent evaluation sets included 200 frontal CheXpert chest X-rays and 32,361 CRCDX histology images, while AIROGS had no public test set.
- Sampling steps: 150 sampling steps appeared to balance increasing FID, precision, and recall quality against longer inference time.Quality increased notably during the first 150 steps and then reached a plateau.