Source-linked AI summary
GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
Christopher Bowles, Liang Chen, Ricardo Guerrero, Paul Bentley, Roger Gunn, Alexander Hammers, David Alexander Dickie, Maria Valdés Hernández, Joanna Wardlaw, Daniel Rueckert
TL;DR
Medical imaging often lacks large labelled datasets, while suitable augmentation procedures are unavailable for many sources of variance. This paper evaluates GAN-derived synthetic images for brain segmentation and finds modest, significant improvements, strongest when real training data is limited.
Problem
Medical imaging datasets are often small and expensive to annotate, while retaining non-pertinent variance can promote overfitting.
Method
The paper uses GANs to generate synthetic training images that augment two brain segmentation tasks and compares results across segmentation architectures.
Results
GAN augmentation provides modest but significant segmentation improvements in many cases, with the strongest gains when real training data is most limited.
Takeaways & Limitations
GAN augmentation is presented as a practical, low-overhead addition to traditional augmentation, particularly for limited-data or imbalanced training settings.
Takeaways & Limitations
GANs interpolate within the training distribution but do not extrapolate beyond its extremes, so appropriate traditional augmentation remains necessary.
Abstract
from arXiv · showhide
One of the biggest issues facing the use of machine learning in medical imaging is the lack of availability of large, labelled datasets. The annotation of medical images is not only expensive and time consuming but also highly dependent on the availability of expert observers. The limited amount of training data can inhibit the performance of supervised machine learning algorithms which often need very large quantities of data on which to train to avoid overfitting. So far, much effort has been directed at extracting as much information as possible from what data is available. Generative Adversarial Networks (GANs) offer a novel way to unlock additional information from a dataset by generating synthetic samples with the appearance of real images. This paper demonstrates the feasibility of introducing GAN derived synthetic data to the training datasets in two brain segmentation tasks, leading to improvements in Dice Similarity Coefficient (DSC) of between 1 and 5 percentage points under different conditions, with the strongest effects seen fewer than ten training image stacks are available.
1 Introduction
The paper investigates GAN-based augmentation as a way to generate synthetic medical images that expand limited training datasets. It focuses on whether this approach can address difficult sources of imaging variance while preserving useful anatomy and pathology.
- Motivation: Data augmentation can reduce overfitting and improve generalisation when training data are limited.Conventional augmentation increases samples through transformations such as rotation, reflection, cropping, translation, and scaling.
- Motivation: GANs learn the training-data distribution and generate synthetic samples visually similar to real images.A generator produces images that an adversarial discriminator cannot distinguish from real examples.
- Motivation: Medical-image augmentation must account for anatomical constraints because transformations suitable for one application may disregard brain symmetry, rigidity, or structure.The paper contrasts elastic deformation in microscopy with its potential unsuitability for brain images.
- Motivation: GANs can model multiple variance sources jointly, including difficult variations such as continuous changes in ventricle size.Unlike manually designed deformations, the GAN infers the variation model directly from available data.
- Contribution: The study tests GAN augmentation across multi-class CT cerebrospinal-fluid segmentation and single-class MR white-matter-hyperintensity segmentation.It also examines segmentation architecture, rotation augmentation, synthetic-data quantity, real-data quantity, and cross-dataset generalisation.
- Contribution: The paper examines whether generated images provide challenging combinations such as shared pathology with different unrelated anatomy, and vice versa.This analysis assesses whether GAN samples differ meaningfully from the original dataset.
2 Methods
The method trains a Progressive Growing GAN on image patches containing both scans and segmentation labels, then adds sampled synthetic patches before training segmentation networks. Experiments vary data availability, synthetic-data quantity, datasets, and network architectures.
- GAN training: A Progressive Growing GAN generates synthetic training data for the segmentation experiments.It was selected for training stability at large image sizes and apparent robustness to hyperparameter selection.
- GAN training: The PGGAN is trained on 80k multi-channel patches containing acquired images and manual segmentation labels.Learning the joint image-label distribution allows synthetic examples to be sampled from the learned manifold.
- GAN training: Synthetic patches are sampled from the trained generator and added to the same 80k real patches before segmentation-network training.For CT data, Gaussian noise was concatenated at the fourth 32x32 resolution level to produce a more realistic noise pattern.
- Evaluation: Segmentation performance is evaluated using Dice Similarity Coefficient on held-out test data selected through validation monitoring.The best validation model at training conclusion is applied to the test set.
- Experimental design: The experiments vary available real data, additional synthetic data, dataset, and segmentation network.Real-data availability is reduced from 10% to 90%, while synthetic patches are added rather than replacing real patches.
- Experimental design: UNet, Res-UNet, and DeepMedic represent the segmentation architectures evaluated.DeepMedic was modified to accept 128x128 two-dimensional patches, while the other network settings followed prior work.
3 Results
The results compare segmentation performance across real-data availability, synthetic-data quantities, network architectures, and augmentation techniques. They also evaluate generated MR images against nearest training-set neighbours to characterize added information.
- Quantitative evaluation: Results are reported as average DSC with standard deviations, with statistically significant differences from baseline shown in bold.Significance was assessed using a 2-tailed t-test at the 5% level.
- Quantitative evaluation: Table 2 varies the proportion of available training data and the amount of added synthetic data for UNet and UResNet architectures.
- Quantitative evaluation: Table 3 compares different augmentation techniques for UNet CSF segmentation across different proportions of available training data.
- Quantitative evaluation: Figure 2 compares class-specific and mean CSF DSC with and without +50% synthetic data, and examines performance as synthetic data increases at 100%, 50% and 10% real-data levels.Coloured dots represent experiments; black circles show means, with filled circles marking results significantly different from baseline.
- Qualitative evaluation: Figure 3 compares synthetic MR images with their nearest training-set neighbours for GANs trained on patches from 5, 25 and 50 real MR images.The comparison examines duplicated lesions across anatomy, changed lesions with fixed anatomy, and substantially different neighbours indicating potentially novel anatomy.
4 Discussion
GAN augmentation generally improves segmentation, especially when real training data are limited or classes are imbalanced, but its benefits depend on the data regime. It complements traditional augmentation by interpolating within the learned distribution, while traditional methods extend beyond its extremes.
- GAN augmentation provides modest but significant segmentation improvements in many cases, with the strongest gains when the available real training data are limited.At the very lowest data levels, improvement may drop because the GAN has too little data to train properly.
- A loss in DSC when all MR data are used suggests a data-dependent tipping point beyond which GAN augmentation can harm performance.
- Class-imbalanced ventricular and brain stem CSF benefited most from GAN augmentation, while ventricular CSF was already consistently well segmented.The training-set class ratio was 1.35:4.35:1 for ventricular, cortical, and brain stem CSF, respectively.
- GAN augmentation produced similar effects across segmentation networks and across a broad range of synthetic-data quantities, reducing the need for fine-grained tuning.The authors characterize the approach as practical because synthetic data rarely impaired performance in their results.
- Combining traditional and GAN augmentation yielded improvements consistently greater than the sum of their separate improvements, suggesting complementary information and potential synergy.GANs interpolate within the training distribution, whereas traditional transformations such as rotation can extrapolate beyond its extremes.
- With five training images, synthetic samples retained strong similarity to training images but varied lesion presence, anatomy, and pathology; with 25 or 50 images, they increasingly diverged from close training matches.The observed modifications appeared reasonable, with no obvious unrealistic lesions or anatomy synthesized.