PaperlayerView exact revision on arXiv

Source-linked AI summary

An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection

Liyan Sun, Jiexiang Wang, Yue Huang, Xinghao Ding, Hayit Greenspan, John Paisley

arXiv:1810.10850v2cs.CV

TL;DR

Medical image analysis often relies on expert lesion labels, and paired normal-abnormal scans are unrealistic to obtain in practice. ANT-GAN predicts a lesion-free counterpart without paired training data; experiments report realistic outputs on brain MRI and liver CT, with potential to support diagnosis and data augmentation.

  • Problem

    Expert lesion labeling is laborious, and paired normal-abnormal images are unrealistic to obtain because only one can exist at a time.

    Source: §I · p. 1

  • Method

    ANT-GAN uses abnormal-to-normal and normal-to-abnormal generators with adversarial and cycle-consistency objectives to translate between lesion-containing and healthy images.

    Source: §I · p. 2, A. ANT-GAN architecture overview · p. 3, 2) Term 2: GAN · p. 3

  • Results

    Experiments on BratS brain MRI and LiTS liver CT report realistic healthy-looking images corresponding to lesion-containing inputs, while healthy MRI inputs are nearly replicated.

    Source: §I · p. 2, §VI · p. 9

  • Takeaways & Limitations

    Generated healthy counterparts can provide additional information for segmentation and diagnosis, while reverse synthesis has potential for data augmentation.

    Source: §I · pp. 1–2

Abstract

from arXiv · show

The identification of lesion within medical image data is necessary for diagnosis, treatment and prognosis. Segmentation and classification approaches are mainly based on supervised learning with well-paired image-level or voxel-level labels. However, labeling the lesion in medical images is laborious requiring highly specialized knowledge. We propose a medical image synthesis model named abnormal-to-normal translation generative adversarial network (ANT-GAN) to generate a normal-looking medical image based on its abnormal-looking counterpart without the need for paired training data. Unlike typical GANs, whose aim is to generate realistic samples with variations, our more restrictive model aims at producing a normal-looking image corresponding to one containing lesions, and thus requires a special design. Being able to provide a "normal" counterpart to a medical image can provide useful side information for medical imaging tasks like lesion segmentation or classification validated by our experiments. In the other aspect, the ANT-GAN model is also capable of producing highly realistic lesion-containing image corresponding to the healthy one, which shows the potential in data augmentation verified in our experiments.

I. INTRODUCTION

Lesion labeling is laborious, and normal-abnormal image pairs are unrealistic to obtain in practice. ANT-GAN predicts a lesion-free counterpart without paired training data, with experiments showing realistic outputs on brain MRI and liver CT.

I. INTRODUCTION

  • Labor-intensive expert labeling and scarce lesion-containing scans limit supervised analysis, while lesion-free images contain useful information about healthy brain structure.In practice, paired normal and abnormal images are unrealistic because only one can exist at a time.Source: §I · p. 1
  • ANT-GAN uses an abnormal-to-normal generator to predict a lesion-free counterpart and a discriminator to distinguish generated images from real healthy images.If no lesion is detected, the output should be indistinguishable from the input.Source: §I · p. 2
  • On BratS MRI and LiTS CT, ANT-GAN produced realistic healthy-looking images corresponding to lesion-containing inputs; BratS difference maps highlighted tumor regions.For healthy BratS inputs, the generated output showed little difference from the input.Source: Figure 1 · p. 1, §I · p. 2
  • Cycle consistency also provides a normal-to-abnormal generator, which can produce realistic lesion-containing images and has potential for data augmentation when training data are scarce.The authors state that this regularization can stabilize training and help convergence.Source: §I · p. 2

II. RELATED WORK

Prior medical-image synthesis largely translated between modalities or inferred normal tissue; ANT-GAN instead targets lesion-specific abnormal-to-normal translation using bidirectional adversarial and cycle-consistency training.

II. RELATED WORK

  • Prior work mostly synthesized across imaging modalities or inferred normal tissue; constrained adversarial auto-encoders for lesion detection were reported as difficult to scale to high-resolution synthesis.Examples of cross-modality synthesis include MRI-to-CT,,, MRI-to-PET, and multi-MRI-to-other-modality mapping.Source: §II · p. 2

III. METHODS

  • The BratS t-SNE embedding shows a small distance between normal and abnormal manifolds, which the authors assume differs only because of lesions.The figure depicts 100 real images from each class and mapped outputs from the respective generators.Source: §III · p. 2, Figure 2 · p. 2

A. ANT-GAN architecture overview

  • ANT-GAN combines adversarial and cycle-consistency objectives to produce a normal-looking counterpart while preserving the input’s healthy regions.The abnormal-to-normal generator should leave healthy inputs essentially unchanged, and the input-output difference can support lesion segmentation or healthy-versus-unhealthy classification.Source: A. ANT-GAN architecture overview · pp. 2–3
  • Training masks identify abnormal locations, and the anomaly penalty penalizes changes outside those regions during abnormal-to-normal translation.The masks are not available or needed at test time.Source: 1) Term 1: Anomaly mask · p. 3, A. ANT-GAN architecture overview · p. 3
  • Bidirectional generators share information through abnormality- and normal-synthesis cycle consistency; ablation favors both directions, while N2A can synthesize realistic lesions for augmentation.The paper reports that bidirectional cycle consistency learns a better model than either unidirectional consistency term alone.Source: 3) Term 3: Cycle consistency · pp. 3–4, 2) Term 2: GAN · p. 3

B. Implementation

The generators use an encoder–residual-block–decoder architecture, with a global shortcut intended to focus abnormal-to-normal translation on lesions. Training uses a fixed cycle-consistency and anomaly-mask weighting with ADAM.

B. Implementation

  • The global shortcut helps the generator isolate and remove lesions while preserving other image regions.The generator architecture comprises an encoder, residual blocks, and a decoder; the shortcut is intended to support lesion-focused mapping.Source: B. Implementation · p. 4
  • The generators share an encoder, residual blocks, and decoder architecture but have different parameters.Stride-2 convolutions downsample shallow-layer features, and stride-2 deconvolutions upsample them to the input size.Source: B. Implementation · p. 4
  • Training uses λCC = 10 and λAM = 10, with ADAM, batch size 1, and a learning rate of 2×10−4 for 20 epochs.The training run comprises 300K iterations.Source: B. Implementation · p. 4

A. Data

Experiments use brain MRI from BratS18 and liver CT from LiTS, with randomly selected training and test splits. The datasets differ in imaging modality, tissue, and image characteristics.

A. Data

  • The study evaluates lesion synthesis on BratS18 brain MRI and LiTS liver CT datasets.BratS18 includes 210 high-grade and 75 lower-grade glioma MRI volumes; LiTS contains 131 contrast-enhanced abdominal CT volumes.Source: A. Data · p. 4
  • BratS18 experiments use FLAIR MRI, while LiTS slices are resized from 512 × 512 to 256 × 256.Not every slice in either dataset contains a tumor or lesion, so those slices are treated as healthy images.Source: A. Data · p. 4
  • Each dataset is split randomly into 80% training data and 20% test data.The datasets differ in tissue and modality, and LiTS CT images include greater noise-like artifacts than MRI.Source: A. Data · p. 4

B. Qualitative analysis

The qualitative ablations show that the baseline GAN and GAN+NC can collapse, while cycle consistency reduces artifacts but leaves a grayscale shift. GAN+AC produces more meaningful structures, yet still has severe artifacts.

B. Qualitative analysis

  • The ablation starts from a baseline GAN without shortcut or anomaly-mask loss, retaining cycle consistency.The study evaluates the cycle-consistency, anomaly-mask, and shortcut components.Source: B. Qualitative analysis · p. 5

1) Evaluation of the cycle consistency:

  • The baseline GAN and GAN+NC exhibit model collapse, while GAN+AC produces more meaningful image structures.GAN+AC still has severe artifacts due to the lack of optimizing constraints.Source: 1) Evaluation of the cycle consistency · p. 5
  • Cycle consistency reduces artifacts in the image translation results, although a grayscale shift remains.The comparison uses CycleGAN as a state-of-the-art unsupervised image-to-image translation baseline.Source: 1) Evaluation of the cycle consistency · p. 5

2) Evaluation of the shortcut and anomaly mask:

The shortcut and anomaly-mask design targets lesion regions while preserving healthy tissue. ANT-GAN achieves the highest reported PSNR among compared models, and visualizations show the generator focusing on lesions.

2) Evaluation of the shortcut and anomaly mask:

  • The global shortcut helps detect, remove, and inpaint tumors without affecting non-tumor regions, while the anomaly-mask term enforces lesion-only modification.CycleGAN+shortcut remains less satisfactory than ANT-GAN on some healthy-region details.Source: 2) Evaluation of the shortcut and… · pp. 5–6
  • ANT-GAN is framed more as image restoration than image generation, unlike GANs aimed at generating realistic samples with variation.The comparison is described as involving less problem-tailored state-of-the-art GAN methods.Source: 2) Evaluation of the shortcut and… · p. 6, Figure 7 · p. 6
  • Generator feature maps show that the global shortcut directs GA2N to capture and work on lesion regions.The Figure 6 caption describes the tumor as gradually identified.Source: 2) Evaluation of the shortcut and… · p. 6, Figure 6 · p. 6
  • ANT-GAN achieves the highest PSNR among the compared models, supporting the benefit of the shortcut and anomaly-mask loss for non-lesion preservation.Lesion regions cannot be objectively assessed because their healthy counterparts are naturally unknown.Source: 2) Evaluation of the shortcut and… · p. 6, Table IV · p. 6

3) Comparison with other GAN formulations:

ANT-GAN is compared with other GAN formulations and evaluated as additional input for segmentation and classification. The authors report advantages over competing formulations and improved results for both downstream tasks.

3) Comparison with other GAN formulations:

  • ANT-GAN outperforms DualGAN in imaging quality; UNIT does not work for this problem, and CAAE struggles with high-resolution synthesis.The authors attribute ANT-GAN’s fidelity and learning support to its anomaly-mask regularization; CAAE evaluations cited here used 32 × 32 images.Source: 3) Comparison with other GAN… · p. 6

C. Practical Applications

  • ANT-GAN’s generated lesion-free image can supplement existing segmentation and classification models with additional information.The practical-application experiments compare those models using the original image alone against models also given ANT-GAN output.Source: C. Practical Applications · p. 6
  • The absolute difference between an input image and its generated lesion-free counterpart can be used to locate potential lesion regions.The segmentation approach uses the difference between x and GA2N(x); an example is binarized at a preset threshold.Source: C. Practical Applications · p. 6
  • ANT-UNet pairs each MRI slice with its generated lesion-free counterpart and improves Dice performance over Plain UNet on BratS18.Both models use matched parameter counts and are evaluated with 5-fold cross-validation; Dice is better when larger.Source: C. Practical Applications · p. 7, Table V · p. 7
  • ANT-VGG uses the input image and generated lesion-free image, and outperforms Plain VGG on all three classification metrics.The BratS18 comparison uses 5-fold cross-validation and matched parameter counts.Source: C. Practical Applications · p. 7, Table VI · p. 7

D. Results on more MRI modalities with brain tumor.

The authors report qualitative ANT-GAN results on additional brain MRI modalities and on liver CT. The LiTS results show successful abnormal-region modification alongside greater deformation than on BratS18.

D. Results on more MRI modalities with brain tumor.

E. Results on the LiTS Challenge dataset.

  • On LiTS liver CT, ANT-GAN modifies abnormal regions successfully, but produces more deformations than on BratS18.The authors note that liver lesions have much lower contrast than brain MRI lesions and describe LiTS as the more difficult task.Source: E. Results on the LiTS Challenge dataset · p. 7

V. DISCUSSIONS

The discussion reports a useful balance from the anomaly-mask parameter setting and explores ANT-GAN’s reverse generator for synthesizing abnormal-looking images. It also outlines a small-scale evaluation of synthetic-image augmentation.

A. Sensitivity to anomaly mask parameter λAM

B. Synthesizing abnormal-looking images

  • The reverse generator GN2A produces realistic lesion-containing MRI, with its discriminator encoding real abnormal-image patterns to avoid physiologically unreasonable lesion locations.The examples include a synthesized tumor near the brain boundary.Source: B. Synthesizing abnormal-looking images · pp. 7, 9, Figure 13 · p. 9
  • The authors assess augmentation by comparing AlexNet lesion classification trained on 50 healthy and 50 lesion-containing BratS18 images with training that adds 50 synthetic examples of each class.The augmented and original datasets are tested on remaining BratS18 data; the supplied table caption identifies the evaluation but gives no metric values.Source: B. Synthesizing abnormal-looking images · p. 9, Table VII · p. 9

VI. CONCLUSION

ANT-GAN translates lesion-containing medical images into lesion-removed counterparts through color correction. Its objective combines an anomaly-mask term and cycle consistency, and the authors report validation on BratS18 and LiTS.

VI. CONCLUSION

  • ANT-GAN translates a lesion-containing medical image into a corresponding image with the lesion removed via color correction.For healthy MRI without lesions, the generator outputs a near replication of the input.Source: §VI · p. 9
  • The objective combines a binary anomaly-mask term with cycle consistency; the mask is not needed for test images.The authors report that comparison with other GAN setups showed the mask was required for learning from the small dataset, and cite BratS18 and LiTS experiments validating the framework for segmentation and classification.Source: §VI · p. 10

Related papers

Loading 1810.10850v2…