MADGAN: unsupervised Medical Anomaly Detection GAN using multiple adjacent brain MRI slice reconstruction

Changhee Han; Leonardo Rundo; Kohei Murao; Tomoyuki Noguchi; Yuki Shimahara; Zoltan Adam Milacski; Saori Koshino; Evis Sala; Hideki Nakayama; Shinichi Satoh

Source-linked AI summary

MADGAN: unsupervised Medical Anomaly Detection GAN using multiple adjacent brain MRI slice reconstruction

Changhee Han, Leonardo Rundo, Kohei Murao, Tomoyuki Noguchi, Yuki Shimahara, Zoltan Adam Milacski, Saori Koshino, Evis Sala, Hideki Nakayama, Shinichi Satoh

arXiv:2007.13559v2Jul 24, 2020cs.CVcs.LGeess.IV

TL;DR

Existing unsupervised methods generally reconstruct single medical images, limiting detection of diseases expressed through subtle anatomical changes across adjacent slices and leaving disease-stage, disease-variety, and multisequence performance insufficiently established. MADGAN reconstructs multiple adjacent brain MRI slices from healthy training data and uses reconstruction discrepancies for diagnosis. It detected MCI, late-stage AD, and brain metastases with AUCs of 0.727, 0.894, and 0.921, respectively.

Problem
Single-image unsupervised reconstruction does not account for continuity across adjacent slices, while evidence across disease stages, multiple diseases, and MRI sequences is limited.
Method
MADGAN reconstructs the next three brain MRI slices from three healthy axial slices using GAN-based reconstruction, then diagnoses scans with average L2 reconstruction loss.
Results
AUC 0.727 for MCI, 0.894 for late-stage AD, and 0.921 for brain metastases were achieved using Self-Attention MADGAN.
Takeaways & Limitations
The approach discriminated AD across stages and detected diseases on T1 and T1c MRI scans in an unsupervised setting.
Takeaways & Limitations
The authors identify limited reconstruction and diagnosis generalizability and note that more healthy T1c scans are needed for reliable detection and localization of various diseases.

Abstract

from arXiv · show

Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's Disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with either disease stages, various (i.e., more than two types of) diseases, or multi-sequence Magnetic Resonance Imaging (MRI) scans. Therefore, we propose unsupervised Medical Anomaly Detection Generative Adversarial Network (MADGAN), a novel two-step method using GAN-based multiple adjacent brain MRI slice reconstruction to detect brain anomalies at different stages on multi-sequence structural MRI: (Reconstruction) Wasserstein loss with Gradient Penalty + 100 L1 loss-trained on 3 healthy brain axial MRI slices to reconstruct the next 3 ones-reconstructs unseen healthy/abnormal scans; (Diagnosis) Average L2 loss per scan discriminates them, comparing the ground truth/reconstructed slices. For training, we use two different datasets composed of 1,133 healthy T1-weighted (T1) and 135 healthy contrast-enhanced T1 (T1c) brain MRI scans for detecting AD and brain metastases/various diseases, respectively. Our Self-Attention MADGAN can detect AD on T1 scans at a very early stage, Mild Cognitive Impairment (MCI), with Area Under the Curve (AUC) 0.727, and AD at a late stage with AUC 0.894, while detecting brain metastases on T1c scans with AUC 0.921.

Background

The paper addresses limits of single-slice unsupervised reconstruction by introducing MADGAN, which reconstructs adjacent brain MRI slices and diagnoses anomalies across disease stages and MRI sequences.

Background: Single-image reconstruction detects outliers by comparing medical images with GAN- or autoencoder-based reconstructions.These methods primarily target diseases readily detectable from individual images.
Background: MADGAN reconstructs the next three brain MRI slices from three preceding healthy axial slices before computing scan-level anomaly scores.Its reconstruction combines WGAN-GP and weighted L1 loss, while diagnosis uses average L2 loss between ground-truth and reconstructed slices.
Background: The method targets diseases whose abnormalities accumulate subtly across adjacent slices, including Alzheimer’s disease, and supports multisequence structural MRI.The paper evaluates T1 scans for AD and T1c scans for brain metastases and other diseases.
Background: 1,133 healthy T1 and 135 healthy T1c scans were used for training the unsupervised detection experiments.The datasets support evaluation across AD stages and different disease types and MRI sequences.
Background: AUC 0.727 for MCI, 0.894 for late-stage AD, and 0.921 for brain metastases were reported with Self-Attention MADGAN.The results cover early and late AD detection on T1 scans and metastasis detection on T1c scans.

Alzheimer’s disease diagnosis

AD diagnosis seeks quantitative assessment of characteristic brain atrophy, while prior approaches have largely used supervised or semi-supervised learning with engineered or labeled features.

Alzheimer’s disease diagnosis: Early AD diagnosis and progression tracking focus on mesial temporal and temporo-parietal cortical atrophy.Relevant structures include the hippocampus, entorhinal cortex, and perirhinal cortex.
Alzheimer’s disease diagnosis: Traditional supervised methods used handcrafted MRI-derived features to support quantitative and reproducible AD assessment.These approaches sought alternatives to subjective physician assessment.
Alzheimer’s disease diagnosis: Deep learning studies reduced labeling requirements through semi-supervised CNNs, sparse regression models, and graph convolutional networks.Reported tasks included clinical decision-making and prediction of MCI-to-AD conversion.
Alzheimer’s disease diagnosis: Brain metastases, aneurysms, hemorrhage, and infarctions remain challenging targets that have mostly been addressed with supervised CNN-based methods.Unsupervised segmentation has also been applied to multiple sclerosis lesions and glioblastoma.

Unsupervised medical anomaly detection

Unsupervised medical anomaly detection estimates healthy anatomy from healthy examples and identifies pathological outliers without requiring abnormal-image labels.

Unsupervised medical anomaly detection: Unsupervised diagnosis estimates normative healthy-anatomy distributions from healthy examples and detects outliers in feature space or reconstruction loss.GANs and VAEs have enabled estimation of high-dimensional healthy distributions.

Self-Attention GANs (SAGANs)

SAGAN introduces self-attention into GAN generators and discriminators to model global and long-range dependencies for image generation.

Self-Attention GANs (SAGANs): SAGAN applies self-attention in GAN generators and discriminators to learn global and long-range dependencies.The approach was proposed for diverse image generation and further performance improvement on large feature maps.
Self-Attention GANs (SAGANs): SAGAN has been used across human pose estimation, image colorization, photo-realistic de-quantization, and large-scale image generation.These applications demonstrate the mechanism’s use across multiple image-generation tasks.

AD dataset: OASIS-3

The AD dataset uses longitudinal T1 MRI scans from OASIS-3, with healthy scans selected for unsupervised training and CDR-defined disease stages for testing.

Dataset source: The dataset contains longitudinal 3.0T T1 brain MRI scans from normal-aging subjects and AD patients in OASIS-3.Slices are zero-padded to 176 × 256 pixels.
Disease staging: CDR = 0 denotes cognitively healthy subjects, while CDR values of 0.5, 1, and 2 represent progressively later dementia stages.CDR = 0.5 corresponds approximately to Mild Cognitive Impairment.
Preprocessing and selection: Only scans with unchanged CDR = 0 are used to ensure healthy training examples, while selected slices include the hippocampus, amygdala, and ventricles.CDR labels are assigned using the assessment date closest to MRI acquisition.
Dataset split: The training set contains 1,133 healthy scans, while the test set includes healthy scans and CDR stages 0.5, 1, and 2.The test set contains 473 unchanged-CDR-0, 253 CDR-0.5, 135 CDR-1, and 10 CDR-2 scans.
Training design: Training uses only healthy structural MRI scans, and no validation set is used because the unsupervised diagnosis step is non-trainable.The datasets are strongly biased toward healthy scans, resembling clinical MRI inspection.

Brain metastasis and various disease dataset

The T1c dataset is a heterogeneous, non-longitudinal collection containing healthy subjects, brain metastases, and several other disease categories for evaluation.

Dataset characteristics: The T1c dataset consists of heterogeneous 1.5T and 3.0T axial MRI scans collected at Japan’s National Center for Global Health and Medicine.The dataset is not publicly available because of ethical restrictions.
Disease categories: The various-disease group includes small infarctions, aneurysms, benign tumors, hemorrhages, cysts, white matter lesions, and postoperative inflammations.T1c slices are selected to include the hippocampus, amygdala, and ventricles.
Dataset split: The split contains 135 normal training scans, plus test sets of 58 normal, 79 brain-metastasis, and 66 various-disease scans.Training reconstruction also uses healthy T1 and T1c slices simultaneously for knowledge transfer.

MADGAN-based multiple adjacent brain MRI slice reconstruction

MADGAN reconstructs three adjacent brain MRI slices from the preceding three and diagnoses abnormalities using reconstruction error across each scan.

Slice reconstruction: For each scan, MADGAN reconstructs the next three MRI slices from the previous three across all possible adjacent-slice windows.A 40-slice scan yields 35 reconstruction setups.
Model architecture: The model uses a U-Net-like generator with convolutional, deconvolutional, and batch-normalization layers, supplemented by self-attention modules.Self-attention supports feature recalibration between T1 and T1c slices.
Model comparisons: The compared reconstruction objectives are WGAN-GP plus 100 ℓ1 loss and WGAN-GP without ℓ1 loss.The experiments also compare models with zero, three, or seven self-attention modules.
Loss functions: The ℓ1 and ℓ2 losses aggregate pixelwise absolute and squared differences over P pixels between an input image x and reconstruction x′.Here, P denotes the number of pixels.
Anomaly diagnosis: Diagnosis classifies unseen scans using the average ℓ2 loss over all reconstructed three-slice windows.Squared error is used because it is sensitive to outliers and outperformed ℓ1, Dice, and Structural Similarity losses in preliminary work.
Evaluation: T1c evaluation reports ROC/AUC results for normal scans versus combined abnormalities, brain metastases, and various diseases.This evaluates both the metastasis-specific and broader disease-detection settings.

Reconstructed brain MRI slices

Reconstructions expose abnormalities through localized or elevated ℓ2 error, while self-attention changes sensitivity to anatomical anomalies and texture stability.

Lesion localization: Pixelwise ℓ2 loss tends to increase around lesions because their image distribution differs from healthy training samples.High-intensity heatmap regions therefore highlight lesion-associated reconstruction errors.
Reconstruction behavior: The combined ℓ1 and WGAN-GP objectives capture T1-specific appearance and anatomical changes from preceding slices.The ℓ1 term supports realism for similar distributions, while WGAN-GP captures recognizable structure.
Self-attention effects: The 7-SA MADGAN is less stable in texture preservation but more sensitive to abnormal anatomical changes, producing moderately higher average ℓ2 loss than MADGAN.This behavior is attributed to attention-driven long-range dependency modeling.
AD-related reconstruction: Higher-CDR scans tend to reconstruct poorly around the hippocampus, amygdala, cerebral cortex, and ventricles because the reconstructed atrophy is insufficient.These regions align with anatomical areas used by physicians for AD diagnosis.
Sequence-dependent behavior: T1c scans have lower average ℓ2 loss than T1 scans because their darker texture differs from the predominantly brighter T1 training images.Consequently, anomaly-related reconstruction failure contributes less to average loss for small local T1c lesions than for large global lesions.

Unsupervised anomaly detection results

MADGAN detects disease-related anomalies across stages, diseases, and MRI sequences, with performance shaped by self-attention, training duration, reconstruction stability, and lesion characteristics.

Training behavior: AUCs generally increase during training, while models with more self-attention modules require more steps to converge.Most models converge after 900k steps, but abundant-self-attention MADGAN might improve further on data-sparse T1c scans with continued training.
Model stability: Self-attention models provide most task-specific best results but perform unstably, while removing the L1 loss tends to reduce AUCs and increase fluctuations.The 7-SA model without L1 loss performs well on T1 scans but poorly on T1c scans.
Various diseases and MRI sequences: AUC 0.921 is achieved for brain metastases detection by 7-SA MADGAN, outperforming MADGAN on T1c scans.The paper attributes this result to self-attention feature recalibration.
Lesion characteristics: MADGAN generally detects both hard- and easy-to-detect diseases, but local small lesions are difficult whereas hyper-intense enhancing metastases are easier to detect.Difference visualizations between real and reconstructed slices may help understand and prevent various diseases, including rare diseases.
Limitations and future work: The authors identify limited generalizability of both reconstruction and diagnosis as two main limitations and plan additional attention, loss, data, and multimodal-imaging studies.Future work includes collecting more healthy T1c scans and integrating modalities such as PET.

Figures

The figures show MADGAN’s adjacent-slice reconstruction pipeline, loss-based anomaly scoring, architectural variants, and evaluations across disease stages and MRI disease groups.

Framework: MADGAN trains on three healthy axial MRI slices to reconstruct the next three, then classifies unseen scans using average ℓ2 loss per scan.The reconstruction uses WGAN-GP with ℓ1 loss.
Architecture: The architecture generates three 256 × 176 slices from three input slices, comparing MADGAN variants with different numbers of self-attention modules.The 3-SA model has three modules, while the 7-SA model has seven.
T1 examples: Example T1 figures compare input slices, ground-truth targets, reconstructions, and pixelwise ℓ2-loss heatmaps for CDR values 0, 0.5, 1, and 2.The heatmaps use a Jet colormap over [0, 0.2].
T1c examples: Example T1c figures compare healthy and metastatic scans, showing reconstructions and pixelwise ℓ2-loss maps with values displayed for each slice.The heatmaps use a Jet colormap over [0, 0.06].
T1c examples: Example T1c figures extend the visualization to four different brain diseases using the same ground-truth, reconstruction, and pixelwise-loss comparison.Both MADGAN and 7-SA MADGAN reconstructions are shown.
Loss distributions: Loss distributions are plotted for T1 scans across four CDR groups and for T1c scans across healthy, metastatic, and various-disease groups.Each distribution compares MADGAN with 7-SA MADGAN.
AUC evaluations: AUC figures evaluate T1 and T1c anomaly detection across training steps from 150k to 1.8M, using healthy groups as the comparison reference.The T1 evaluations separate dementia stages, while T1c evaluations separate metastases and various diseases.
AUC evaluations: The tables report AUC performance for T1 and T1c anomaly detection using average ℓ2 loss over whole slice sets or selected continuous slice sets.T1 comparisons cover CDR groups; T1c comparisons cover metastases and various diseases.