PaperlayerView exact revision on arXiv

Source-linked AI summary

GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, Hayit Greenspan

arXiv:1803.01229v1cs.CVcs.LGstat.ML

TL;DR

Limited and costly medical annotations motivate methods for synthetic data augmentation. The paper combines GAN-based liver-lesion synthesis with CNN classification and evaluates generated examples on a small CT dataset. Adding synthetic augmentation improved reported classification performance over classic augmentation, while the study identifies extensions beyond its 2-D, class-specific GAN setup.

  • Problem

    Medical imaging lacks large annotated datasets, while collecting expert-labeled data is complex and expensive.

  • Method

    The study synthesizes labeled liver-lesion images with GANs and uses them alongside classic augmentation to train a CNN classifier.

  • Results

    78.6% to 85.7% total accuracy was observed when synthetic augmentation was added after classic augmentation reached saturation.

  • Takeaways & Limitations

    Synthetic lesion examples added variability to the input dataset and improved classification performance for the studied liver-lesion task.

  • Takeaways & Limitations

    The study used 2-D inputs and separate GANs for each lesion class, increasing training complexity and limiting the evaluated setup.

Abstract

from arXiv · show

Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods for generating synthetic medical images using recently presented deep learning Generative Adversarial Networks (GANs). Furthermore, we show that generated medical images can be used for synthetic data augmentation, and improve the performance of CNN for medical image classification. Our novel method is demonstrated on a limited dataset of computed tomography (CT) images of 182 liver lesions (53 cysts, 64 metastases and 65 hemangiomas). We first exploit GAN architectures for synthesizing high quality liver lesion ROIs. Then we present a novel scheme for liver lesion classification using CNN. Finally, we train the CNN using classic data augmentation and our synthetic data augmentation and compare performance. In addition, we explore the quality of our synthesized examples using visualization and expert assessment. The classification performance using only classic data augmentation yielded 78.6% sensitivity and 88.4% specificity. By adding the synthetic data augmentation the results increased to 85.7% sensitivity and 92.4% specificity. We believe that this approach to synthetic data augmentation can generalize to other medical classification applications and thus support radiologists' efforts to improve diagnosis.

I. INTRODUCTION

Medical imaging faces limited, costly annotated datasets, motivating synthetic augmentation. This work combines GAN-based lesion synthesis with CNN classification and evaluates the approach on CT liver lesions.

  • Motivation: Limited medical datasets and costly expert annotations constrain supervised medical imaging research.Precise organ or lesion annotations can require substantial radiologist time.
  • Motivation: Classic augmentation applies translations, rotations, flips and scaling, but small image modifications add limited information.Synthetic examples learned by generative models can add variability to the training data.
  • Generative approach: GANs generate images through adversarial training between a network producing fake images and a discriminator distinguishing real from fake images.The paper applies this framework to medical image synthesis.
  • Clinical task: The study targets automated CT-based classification of cysts, metastases and hemangiomas, a task intended to assist radiologists.The liver is a common site for metastatic cancer, and CT is widely used for lesion detection and diagnosis.
  • Contributions: The paper synthesizes high-quality liver lesion images, designs a CNN classifier, and uses generated images to augment CNN training.These are presented as the work's three main contributions.

II. LIVER LESION CLASSIFICATION

The classification system uses radiologist-annotated lesion ROIs from 182 portal-phase 2-D CT scans spanning three lesion categories. ROIs are standardized and processed by a CNN designed for small medical datasets.

  • Data: The dataset contains 182 portal-phase 2-D CT scans: 53 cysts, 64 metastases and 65 hemangiomas.An expert radiologist marked lesion margins, with diagnoses established by biopsy or clinical follow-up.
  • Data: Liver lesions vary in shape, contrast and size from 10 to 102 mm, with substantial within-category variability.Metastases and hemangiomas are noted as especially variable.
  • ROI preparation: The system crops lesion ROIs with surrounding tissue from CT scans using radiologist annotations.ROI size varies because lesion sizes vary.
  • CNN architecture: The CNN receives 64×64 ROIs with intensities rescaled to (0, 1) and predicts three lesion classes using convolutional, pooling and dense layers.The network has approximately 1.3M parameters and includes dropout with probability 0.5.
  • Training: Training uses mean subtraction, batch size 64, learning rate 0.001, 150 epochs and stochastic gradient descent with Nesterov momentum.These settings define the reported CNN training procedure.

III. GENERATING SYNTHETIC LIVER LESIONS

The paper enlarges the small training set through classic image transformations and GAN-generated lesion examples. Classic augmentation preserves lesion characteristics while standardizing all ROIs to 64×64 pixels.

  • Augmentation strategy: The training set is enlarged using classic image manipulation and synthetic examples learned by generative models.The method addresses the lack of a large labeled training dataset.
  • Classic data augmentation: Classic augmentation uses translation, rotation, scaling and flipping while avoiding shearing to preserve lesion characteristics.The ROI is kept centered around the lesion.
  • Classic data augmentation: Each lesion ROI is rotated, flipped, translated and scaled using randomized transformations tied partly to lesion diameter.The total augmentation count is N = Nrot × (1 + Nflip + Ntrans + Nscale).
  • Classic data augmentation: Figure 3 illustrates lesion ROIs alongside translation, rotation, flipping and scaling augmentations.All augmented ROIs are resized to a uniform 64×64-pixel size using bicubic interpolation.

B. Generative Adversarial Networks for Lesion Synthesis

The study explores GAN architectures for synthesizing labeled liver-lesion ROIs, focusing first on DCGAN and its adversarial generator–discriminator framework.

  • GAN framework: GANs learn image distributions to generate new samples, using separate generator and discriminator networks trained adversarially.The generator maps noise to image space, while the discriminator estimates whether samples are real.
  • DCGAN: DCGAN uses deep CNNs for both generator and discriminator, with the generator producing samples intended to appear real to the discriminator.Its training goal is to make the generated distribution approach the data distribution.
  • Adversarial training: The discriminator maximizes real-sample scores and minimizes scores for generated samples, while the generator improves by increasing discriminator acceptance of its outputs.This alternating optimization drives progressively more realistic synthesized images.
  • Generator architecture: The DCGAN generator maps 100 uniform random values to a 64 × 64 × 1 liver-lesion image through a reshaped fully connected layer and four up-sampling convolutional layers.The initial representation is reshaped to 4 × 4 × 1024 before spatial expansion.
  • Discriminator architecture: The discriminator receives a 64 × 64 × 1 lesion ROI and outputs a real-versus-fake decision using four convolutional layers and a fully connected layer.Strided convolutions reduce spatial dimensionality, and the output layer uses a sigmoid likelihood score.
  • Training procedure: DCGAN training synthesized liver-lesion ROIs separately for each lesion category using iterative generator–discriminator updates and mini-batches of 64 examples.Noise samples were used after the generator learned each class-specific distribution.

C. Conditional Lesion Synthesis

The second approach is ACGAN, which conditions lesion synthesis on class labels and adds class prediction to the discriminator alongside real-versus-fake discrimination.

  • ACGAN: ACGAN extends GANs by conditioning generation on external information to improve labeled-sample synthesis.The generator combines class embeddings with noise samples.
  • Conditional discrimination: Unlike a standard discriminator, the ACGAN discriminator reconstructs class information while also deciding whether an image is real or fake.Its outputs include both source and class predictions.
  • Conditional generation: ACGAN generates labeled examples for all three lesion types using class labels C and noise z as generator inputs.The corresponding generator expression is described as Xfake = G(c, z).
  • Training objective: The ACGAN discriminator maximizes Ls + Lc, whereas the generator maximizes Lc − Ls.The two loss components represent source and class objectives in the described formulation.

IV. EXPERIMENTS AND RESULTS

The experiments evaluate CNN liver-lesion classification and compare classification with and without synthetic lesion augmentation using patient-level cross-validation.

  • Experimental design: The study compares classical and synthetic data augmentation for classifying cysts, metastases, and hemangiomas with a CNN.The experiment uses DCGAN and ACGAN as the two synthetic-lesion generation methods.
  • Evaluation protocol: Three-fold cross-validation separated cases at the patient level, with folds containing 63, 63, and 62 balanced lesion ROIs.Performance was assessed using accuracy, confusion matrices, sensitivity, and specificity.
  • Evaluation metrics: Sensitivity and specificity were calculated for each lesion category, with positives defined as examples from that category and negatives as examples from the other two.This defines the category-wise evaluation used alongside total accuracy.
  • Implementation: The implementation used Keras for the classification CNN, TensorFlow for GAN architectures, and an NVIDIA GeForce GTX 980 Ti GPU.These tools supported all reported training and evaluation processes.
  • Synthetic examples: Synthetic DCGAN ROIs were visualized separately for cyst, metastasis, and hemangioma categories.Figure 6 organizes examples by lesion class.
  • Experiment workflow: The experiment flowchart evaluates synthetic augmentation in the liver-lesion ROI classification task.It represents the comparison between baseline augmentation and the addition of synthesized ROIs.

B. Evaluation of the Synthetic Data Augmentation

The synthetic-augmentation evaluation begins with a classical-augmentation CNN baseline, then adds generated lesion ROIs to the training data to test classification effects.

  • Evaluation procedure: The experiment first evaluates classical data augmentation as the baseline and then evaluates classification after adding synthesized ROIs to the training set.The same CNN classification task is used for both stages.
  • Classical augmentation: CNN-AUG denotes the baseline network trained with classical augmentation over increasing amounts of augmented data.The original images and additional classic augmented images form the evaluated training groups.
  • Data-group construction: Testing uses the same data examples across evaluations while training groups are formed additively to examine increasing augmentation volumes.This design isolates changes associated with adding more training examples.
  • Data-group construction: The first data group contains only the original ROIs.Subsequent groups add augmented examples to the original training data.
  • Augmentation volume: Each original ROI generated 480 classic augmentations from rotation, flipping, translation, and scaling, yielding approximately 30,000 examples per folder.Random sampling selected balanced augmentation volumes for each original lesion when forming fold data groups.

2) Synthetic data augmentation:

The study trains class-specific DCGANs to generate balanced synthetic liver-lesion ROIs and adds them to classically augmented training data. Synthetic augmentation raises total classification accuracy beyond the classical-augmentation baseline.

  • Synthetic ROI generation: Class-specific DCGANs learned lesion distributions from classically augmented data and generated new ROI examples from normally distributed noise.The GAN training used the same 3-fold partitions, with classic augmentation incorporated because the dataset was too small for effective GAN training.
  • Synthetic ROI generation: Synthetic samples were added equally across classes, with 100×3 through 4000×3 examples considered across the three folds.Equal sampling preserved class balance during synthetic augmentation.
  • Classification results: Figure 8 compares total accuracy as training-set size increases for classical augmentation versus synthetic augmentation.The red line denotes classical augmentation and the blue line denotes synthetic augmentation.
  • Classification results: 78.6% total accuracy was reached with classical augmentation before saturation at 5000 augmented samples per fold.Without augmentation, accuracy was 57%; adding more data after the saturation point did not improve the result.

C. Visualization using t-SNE

The t-SNE analysis visualizes CNN features learned with classical versus synthetic augmentation, while expert assessment evaluates lesion realism and class recognizability. Cysts appear more distinct, whereas metastases and hemangiomas overlap more substantially.

  • Visualization using t-SNE: t-SNE embeds last-layer CNN features into two dimensions for models trained with classical augmentation and with synthetic augmentation.The visualization compares CNN-AUG and CNN-AUG-GAN features for real cyst, metastasis, and hemangioma ROIs.
  • Visualization using t-SNE: Cysts occupy a more distinct t-SNE location, while metastases and hemangiomas show overlapping features.The overlap corresponds to the more confusing characteristics of metastases and hemangiomas relative to cysts.
  • Expert assessment: Two radiologists assessed whether synthesized ROIs appeared realistic and sufficiently distinct for three-class classification.They classified lesion categories and distinguished real from synthetic images using 182 real and 120 synthesized ROIs.
  • Expert assessment: Radiologists identified real versus synthetic lesions with overall accuracy around 60%, while lesion-category accuracy was 77.8% for Expert 1 and 69.2% for Expert 2.The experts agreed on lesion class for 222 of 302 lesions, with 185 correctly classified.
  • Expert assessment: Similar classification performance on real and synthesized lesions suggests that the generated lesions were meaningful in appearance.The authors note that the assessment task did not match regular clinical workflow.

E. Comparison with Other Classification Methods

The authors compare DCGAN and ACGAN synthetic augmentation and benchmark the CNN against BoVW-MI. DCGAN-based augmentation performs best among the evaluated generative-model options, while synthetic augmentation yields the strongest overall classification performance.

  • Generative-model comparison: ACGAN-based synthetic augmentation improved over classical augmentation but performed worse than DCGAN-based augmentation.The ACGAN discriminator jointly distinguished real from fake images and classified lesions into three categories.
  • Comparison with other methods: The CNN with classical augmentation performed on par with the BoVW-MI state-of-the-art liver-lesion classifier using the same ROI input.The comparison used the current dataset and matched lesion partitions under 3-fold cross-validation.
  • Comparison with other methods: Synthetic data augmentation in the proposed CNN architecture produced the best performance in the comparison.The study compared CNN-AUG and CNN-AUG-GAN against BoVW-MI.

V. DISCUSSION AND CONCLUSIONS

On a limited three-class liver-lesion dataset, synthetic GAN-generated ROIs improved CNN classification beyond classical augmentation and produced meaningful visual and feature representations. The authors identify extensions involving 3-D inputs, multiclass GANs, unlabeled data, and modified losses.

  • Discussion and conclusions: The study addresses small medical-imaging datasets by generating synthetic images with GANs for CNN data augmentation.The dataset size reflects the relatively small datasets commonly available in medical imaging.
  • Discussion and conclusions: Standard augmentation improved classification only up to a saturation point as the augmented training set increased.The experiment used translation, rotation, flipping, and scaling on three lesion categories.
  • Discussion and conclusions: 85.7% versus 78.6% total accuracy was obtained with synthetic versus classical augmentation, respectively.The authors associate the improvement with additional variability in the input data distribution and report increased sensitivity and specificity for metastases and hemangiomas.
  • Discussion and conclusions: Expert assessment found similar classification accuracy for real and synthesized lesions, supporting the validity of the lesion-generation process.The assessment was challenging and was not conducted in the regular radiologist working environment.
  • Discussion and conclusions: Synthetic lesions had meaningful visualizations and features and could be incorporated into a classification system.The study evaluated their ability to improve another system operating behind the scenes.
  • Discussion and conclusions: ACGAN synthetic augmentation did not improve results over CNN-AUG-GAN, whereas DCGAN performed better for this dataset.DCGAN trained separate models for each lesion class; ACGAN trained all three classes together with an auxiliary classifier.
  • Discussion and conclusions: Compared with BoVW-MI, classical CNN augmentation performed similarly, while synthetic augmentation achieved the best performance.The comparison used the same ROI input and evaluated the proposed CNN-based system against a non-neural state-of-the-art method.
  • Limitations and future work: The study is limited by 2-D inputs, separate GAN training for each lesion class, and the need to investigate unlabeled data and alternative regularized losses.The authors propose 3-D CNN analysis, multiclass generation, unlabeled-data incorporation, and L1- or L2-regularized losses as extensions.
Loading 1803.01229v1…