Source-linked AI summary
Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection
David Zimmerer, Simon A. A. Kohl, Jens Petersen, Fabian Isensee, Klaus H. Maier-Hein
TL;DR
Medical-image anomaly scores based only on reconstruction error omit latent-representation deviations and lack formal sample comparability. The ceVAE combines Context Encoder reconstruction with VAE density-based scoring, improving sample-wise detection and pixel-wise localization while achieving state-of-the-art unsupervised segmentation results on two public benchmarks.
Problem
Reconstruction-error anomaly scores ignore deviations in model-internal latent representations and lack formal assertions and comparability between samples.
Method
The ceVAE combines Context Encoders and Variational Autoencoders to fuse reconstruction-based and latent density-based anomaly scores for sample-wise and pixel-wise detection.
Results
The ceVAE outperformed the presented baselines and literature results, achieving state-of-the-art ROC-AUCs for unsupervised segmentation on public benchmark data.
Takeaways & Limitations
The approach supports unsupervised anomaly detection and localization in medical images without labeled data.
Takeaways & Limitations
Evaluation performance may be a lower bound because surrogate datasets can have unlabeled anomalies, and domain shifts between datasets can obstruct assessment.
Abstract
from arXiv · showhide
Unsupervised learning can leverage large-scale data sources without the need for annotations. In this context, deep learning-based auto encoders have shown great potential in detecting anomalies in medical images. However, state-of-the-art anomaly scores are still based on the reconstruction error, which lacks in two essential parts: it ignores the model-internal representation employed for reconstruction, and it lacks formal assertions and comparability between samples. We address these shortcomings by proposing the Context-encoding Variational Autoencoder (ceVAE) which combines reconstruction- with density-based anomaly scoring. This improves the sample- as well as pixel-wise results. In our experiments on the BraTS-2017 and ISLES-2015 segmentation benchmarks, the ceVAE achieves unsupervised ROC-AUCs of 0.95 and 0.89, respectively, thus outperforming state-of-the-art methods by a considerable margin.
1 Introduction
Medical imaging generates growing volumes of increasingly detailed data, while manual review is costly and can miss unexpected abnormalities. Unsupervised anomaly detection uses only normal samples to identify and localize abnormal regions without manual annotations.
- 1 Introduction: Manual medical-image inspection is time- and resource-consuming, does not scale well, and can miss unexpected anomalies.Machine-learning support systems commonly require large annotated datasets for each condition and modality, and may fail on unrepresented conditions.
- 1 Introduction: Unsupervised anomaly detection identifies abnormal data points from normal samples without requiring manual annotations.The approach can highlight interesting regions for further manual inspection while internalizing the appearance of normal tissue.
- 1 Introduction: The proposed ceVAE combines Context Encoders with Variational Autoencoders to improve sample-wise anomaly scoring and pixel-wise localization.Its contributions include using latent posterior deviations, fusing them with reconstruction error, and outperforming unsupervised state-of-the-art methods on two public segmentation challenges.
2 Related Work
Autoencoders learn representations that reconstruct input data, with reconstruction error serving as a basis for identifying abnormal samples. Deep convolutional encoders and decoders model this process, while mean-squared error is a common reconstruction objective.
- 2 Related Work: An autoencoder encodes input x into a representation z and decodes z to reconstruct the original input.Deep convolutional autoencoders implement the encoder and decoder as networks with learnable parameters.
- 2 Related Work: Mean-squared error is a common choice for the reconstruction error Lrec(x, ˆx).The error compares the input with its reconstruction using the squared norm ||x − ˆx||2.
Denoising Autoencoder
Denoising autoencoders reconstruct clean data from corrupted inputs to learn more robust representations. Context Encoders implement this corruption by masking local input patches, encouraging representations that capture semantic information.
- Denoising Autoencoder: A denoising autoencoder reconstructs an unperturbed sample from a noisy input, producing more robust and perturbation-invariant representations.Additive Gaussian noise is a commonly used corruption process.
- Denoising Autoencoder: Context Encoders are denoising autoencoders that mask local input patches instead of adding Gaussian noise.The masking resembles salt-and-pepper noise and was reported to produce representations capturing both appearance and semantic information.
- Denoising Autoencoder: Variational autoencoders assume a latent-variable model in which a latent variable z causes the observation x.This assumption facilitates a lower bound on the probability of a data sample.
Variational Autoencoders
Variational autoencoders model data through latent-variable distributions parameterized by neural networks. Maximizing the ELBO approximates the data distribution and enables probability estimates for samples.
- Variational Autoencoders: Maximizing the Evidence Lower Bound approximates the true data distribution and enables probability estimates for data samples.The ELBO uses a prior p(z), an approximate inference model q(z|x), and a generative model p(x|z).
- Variational Autoencoders: VAE encoders parameterize q(z|x), while decoders parameterize the generative model using neural networks.The prior and generative distributions are assumed to be diagonal Gaussian distributions.
- Variational Autoencoders: VAE training samples ˜z from the encoded Gaussian distribution using the reparameterization trick and commonly uses mean-squared error for reconstruction.The encoded distribution is defined by the mean and variance-related outputs of the encoder.
2.2 Anomaly detection
Classification-based unsupervised anomaly detection methods distinguish normal from abnormal data using learned feature-space boundaries.
- One-class methods separate normal and abnormal data in feature space.
- OC-SVM is a prominent classification-based method for unsupervised anomaly detection.
- The OC-SVM learns a decision boundary between data features and the origin.
Classification-based methods
The paper situates reconstruction-based autoencoders among unsupervised anomaly-detection approaches, alongside classification and density-based methods.
- Autoencoder methods reconstruct normal samples while producing high reconstruction errors for abnormal data.
- Compared with PCA-based reconstruction, autoencoders better handle nonlinear relations in data.
- Prior medical-imaging studies evaluated GANs, VAEs, and different autoencoders primarily through pixel-wise reconstruction errors.
- Density-based approaches assign anomaly scores from estimated data likelihoods, using either local or parametric distributions.
Problem Statement
Medical-image anomaly detection commonly relies on reconstruction error, but this score omits latent deviations and lacks formal comparability across samples.
- Reconstruction error ignores deviations of latent representations from their normal ranges, including anomalies that reconstruct perfectly.
- Figure 1 presents the ceVAE model structure associated with the proposed approach.
- Reconstruction error has no formal assertion or theory-backed validity as a calibrated, comparable anomaly score.
3 Methods
ceVAE combines context encoding and variational autoencoding to score anomalies using both calibrated reconstruction behavior and deviations in latent distributions at sample and pixel levels.
- ceVAE combines context encoding with a VAE to capture latent deviations and expressive reconstruction errors for sample- and pixel-level detection.
- The context-encoding branch masks random input regions and reconstructs the perturbed sample using the mean encoder and decoder.
- The denoising objective is intended to align reconstruction error with the input derivative of log-density and produce richer representations.
- The VAE branch uses encoder distributions, a decoder, and a standard diagonal Gaussian prior to analyze latent deviations through KL divergence.
- The combined objective adds VAE reconstruction, context-encoding reconstruction, and KL-loss terms while avoiding prior-normality constraints on perturbed inputs.
- Pixel-wise scores combine reconstruction error with back-traced latent deviations, with the derivative obtained by backpropagating the ELBO to the input.
4 Experiments
The experiments train on healthy HCP MRI data and test anomaly detection on BraTS-2017 and ISLES-2015, comparing ceVAE against OC-SVM and several autoencoder variants.
- Experimental Setup: The model was trained on 1,092 HCP patients, with 20 validation patients, then tested for anomalies on BraTS-2017 and ISLES-2015.HCP was the only training dataset, using 136,576 training slices and 2,496 validation slices.
- Model Architecture: The encoder and decoder are symmetric fully convolutional networks with five convolutional stages, CoordConv, and a latent variable size of 1024.The architecture uses Leaky-ReLU activations and shared encoder weights with separate mean and variance heads.
- Benchmark Methods: The benchmark compares ceVAE with OC-SVM, standard AE, denoising AE, context encoder, and VAE models using matched structure and training schemes.The ceVAE-Factor controls the ratio of context-encoding reconstruction loss to VAE loss; 0.0 is VAE-only and 1.0 is CE-only.
- Benchmark Methods: Figure 2 compares slice-wise anomaly detection performance across different models on the BraTS-2017 dataset.The supplied caption identifies the dataset and slice-wise comparison, but not individual plotted values.
- Evaluation: Slice-wise evaluation discriminates normal from abnormal slices using estimated sample probability p(x) and reports ROC-AUC.Pixel-wise evaluation uses anomaly score h and reports pixel-wise ROC-AUC plus mean patient-wise Dice score.
5 Results
The ceVAE improves anomaly detection by combining context encoding with variational modeling and by combining reconstruction error with the KL-loss gradient for pixel localization.
- Slice-wise Detection: The ceVAE outperformed all other methods on slice-wise anomaly detection in the BraTS-2017 comparison.Performance improved progressively from OC-SVM to AE variants, VAE, and finally ceVAE.
- Pixel-wise Detection: Figure 3 reports Dice score and pixel-wise ROC-AUC for different ceVAE-Factors on the BraTS-2017 and ISLES-2015 datasets.The supplied figure caption identifies the metrics and factor comparison, while the prose describes the combination as strongest overall.
- Pixel-wise Detection: Combining reconstruction error with the KL-loss gradient produced the best pixel-wise results across the experiments.The ceVAE outperformed the non-combined methods in all cases.
- Pixel-wise Detection: CE performed best with reconstruction error alone, whereas VAE performed best with the KL-loss gradient alone.The VAE exceeded the CE for the gradient-only pixel-wise score.
- Pixel-wise Detection: Absolute performance differed between datasets, possibly because of differences in dataset quality and the underlying data distributions.This qualification applies to the reported pixel-wise experiments.
6 Discussion & Conclusion
ceVAE combines reconstruction- and density-based anomaly detection for unsupervised medical-image detection and localization, outperforming the presented baselines and literature results. Evaluation remains constrained by dataset shifts, while several tested architectural and loss modifications provided no consistent benefit.
- 6 Discussion & Conclusion: ceVAE outperformed all presented baselines and literature results while achieving state-of-the-art unsupervised segmentation on public benchmark data.The method supports both anomaly detection and pixel-wise localization without labeled data.
- 6 Discussion & Conclusion: Evaluation is limited by surrogate datasets, incomplete anomaly labels, domain shifts, age differences, scanner variation, and image-quality differences that can cause miss-detections.The low image quality of ISLES-2015 may explain its low absolute scores.
- 6 Discussion & Conclusion: Adding layers, residual connections, normalization variants, pixel reshuffling, or 2.5D input produced no significant benefits, so the final model retained a simpler design.Extending the approach to 3D was identified as a possible next step.
- 6 Discussion & Conclusion: Guided backpropagation slightly outperformed vanilla backpropagation, but the authors found no model or output independence for guided backpropagation.This result qualifies the interpretability behavior of the localization pipeline.
- 6 Discussion & Conclusion: MMD- and adversarial-loss variants slightly improved reconstruction error but worsened overall performance because of higher-variance gradients.Alternative reconstruction losses also failed to provide significant performance benefits and introduced training costs or instability.