Source-linked AI summary
Brain Imaging Generation with Latent Diffusion Models
Walter H. L. Pinaya, Petru-Daniel Tudosiu, Jessica Dafflon, Pedro F da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, M. Jorge Cardoso
TL;DR
Small and privacy-constrained medical imaging datasets limit the scale of deep-learning research, motivating synthetic-data generation. The paper trains latent diffusion models on 31,740 UK Biobank T1w brain MRIs, conditions generation on demographic and structural variables, and releases 100,000 synthetic images. The models generated realistic brain images, supported controllable conditioning, and outperformed GAN-based methods in the unconditioned scenario.
Problem
Medical imaging projects face limited datasets because collecting medical data is costly and privacy concerns restrict sharing, constraining large-scale research.
Method
The study combines compression models with diffusion models to generate high-resolution 3D brain MRI conditioned on age, sex, ventricular volume, and normalized brain volume.
Results
The diffusion models generated realistic brain images, enabled conditioning on demographic and structural covariates, and outperformed GAN-based methods in an unconditioned scenario.
Takeaways & Limitations
A publicly available dataset of 100,000 synthetic human brain images provides the scientific community with openly accessible generated data.
Abstract
from arXiv · showhide
Deep neural networks have brought remarkable breakthroughs in medical image analysis. However, due to their data-hungry nature, the modest dataset sizes in medical imaging projects might be hindering their full potential. Generating synthetic data provides a promising alternative, allowing to complement training datasets and conducting medical image research at a larger scale. Diffusion models recently have caught the attention of the computer vision community by producing photorealistic synthetic images. In this study, we explore using Latent Diffusion Models to generate synthetic images from high-resolution 3D brain images. We used T1w MRI images from the UK Biobank dataset (N=31,740) to train our models to learn about the probabilistic distribution of brain images, conditioned on covariables, such as age, sex, and brain structure volumes. We found that our models created realistic data, and we could use the conditioning variables to control the data generation effectively. Besides that, we created a synthetic dataset with 100,000 brain images and made it openly available to the scientific community.
1 Introduction
Medical imaging benefits from deep learning but is constrained by small, costly-to-collect datasets and privacy-related data-sharing challenges. This study investigates diffusion-based generation of high-resolution brain MRI, including controllable synthesis and public data release.
- Medical imaging datasets are often small because data collection is costly and laborious, while privacy concerns restrict public sharing.
- Limited medical imaging data can bottleneck model generalizability and slow deployment of advanced methods in clinical practice.
- Synthetic data could complement real data and enable medical image research and model training at larger scale.
- Generating meaningful synthetic data remains difficult for complex organs such as the brain.
- Earlier VAE-GAN work generated whole-brain volumes but resized images to 64 × 64 × 64 voxels, limiting replication of essential features.
- The study trains Latent Diffusion Models on high-resolution brain MRI, conditions generation on covariates, and releases 100,000 synthetic brain images.
2 Methods
The methods use UK Biobank T1w MRI data and a latent diffusion pipeline designed to handle high-resolution 3D brain images. Generation is conditioned on demographic and brain-volume variables.
- Datasets and Image Preprocessing: The models were trained on 31,740 UK Biobank participants with T1w images aged 44–82 years.The sample included 14,942 male subjects (47%), with average age 63.6 ± 7.5 years.
- Datasets and Image Preprocessing: Conditioning variables included age, sex, ventricular cerebrospinal-fluid volume, and brain volume normalized for head size.The conditioning variables were min-max normalized before entering the models.
- Datasets and Image Preprocessing: Images were rigidly registered to common MNI space, resampled to 1 mm^3 voxels, and cropped to 160 × 224 × 160 voxels.
- Generative models: Latent Diffusion Models compress brain images into lower-dimensional representations before diffusion modeling, enabling scaling to high-resolution medical images.The encoder maps each brain image to a latent representation of size 20 × 28 × 20.
3 Experiments
The experiments show that latent diffusion models generated realistic, high-quality 3D brain images, supported controllable conditioning, and extrapolated brain and ventricular volume representations. The authors also released 100,000 synthetic brain images openly.
- 3.1 Sampling Quality: LDMs produced high-quality brain images with sharp details and realistic textures, while GAN baselines showed mode collapse at high resolution.The authors report more stable and easier convergence for diffusion training than for the VAE-GAN and LSGAN baselines.
- 3.1 Sampling Quality: 50 DDIM timesteps reduced sampling time from 142.3±1.6s to 7.6±0.2s per sample with minimal performance loss.The models used 50 timesteps instead of 1000 for the remaining analyses.
- 3.2 Conditioning Evaluation: The model generated images conditioned on age, sex, ventricular volume, and brain volume relative to intracranial volume.Conditioned sampling varied ventricular and normalized brain volumes while keeping other variables constant.
- 3.2 Conditioning Evaluation: 0.972 was the Pearson correlation between input ventricular-volume conditions and SynthSeg-measured ventricular volumes across 1,000 synthetic brains.This correlation was used to evaluate the effectiveness of ventricular-volume conditioning.
- 3.2 Conditioning Evaluation: r=0.692 was the correlation between input age and predicted brain age in the synthetic dataset.Brain age was predicted using a 3D convolutional neural network trained on the same training set as the LDM.
- 3.3 Synthetic Dataset: The model extrapolated conditioning beyond the training range, generating enlarged or absent ventricles and altered brain volumes for out-of-range inputs.The experiments used normalized ventricular values of 1.5, 1.9, and -0.5, plus negative normalized brain-volume values.
- 3.3 Synthetic Dataset: 100,000 synthetic human brain images were made openly available with their conditioning information.The dataset was released through Academics Torrents, FigShare, and HDRUK Gateway.
4 Conclusions
The study shows that latent diffusion models can generate synthetic brain images that replicate training-image properties and respond to covariate conditioning. The openly released synthetic dataset addresses the challenge of obtaining large imaging datasets while preserving privacy.
- Diffusion models generated synthetic brain images that replicated properties of the training images.
- In unconditioned generation, the diffusion models outperformed alternative GAN-based methods.
- Conditioning on age, sex, and brain structure volumes produced the expected representations.
- The openly available synthetic dataset addresses the challenge of obtaining large imaging datasets without posing privacy-infringement threats.