PaperlayerView exact revision on arXiv

Source-linked AI summary

Anomaly Detection in Medical Imaging with Deep Perceptual Autoencoders

Nina Shvetsova, Bart Bakker, Irina Fedulova, Heinrich Schulz, Dmitry V. Dylov

arXiv:2006.13265v3cs.CV

TL;DR

Medical anomalies can be barely different from normal images, and thorough comparisons of anomaly-detection methods in the medical domain are lacking. The paper proposes a perceptual autoencoder with progressive training and uses a small, confined set of anomalies for hyperparameter selection. It outperformed SOTA methods on both medical datasets, including a 2.8% absolute gain on Camelyon16, but did not outperform SOTA on natural-image baselines.

  • Problem

    Medical anomalies can closely resemble normal images, and a thorough comparison of SOTA anomaly-detection methods in the medical domain was missing.

    Source: §I · p. 1

  • Method

    The method trains an autoencoder with perceptual loss and progressive growing, using a small number of confined anomalies only to select hyperparameters.

    Source: §I · p. 2, §V · p. 10

  • Results

    The method excelled on both medical datasets, including a 2.8% absolute gain over SOTA on Camelyon16, but did not outperform SOTA on natural-image baselines.

    Source: §I · p. 3, D. RESULTS · p. 8, G. LIMITATIONS · p. 10

  • Takeaways & Limitations

    The authors present the approach as a strong anomaly-detection baseline for complex medical imaging tasks and report that limited anomaly examples can support hyperparameter selection.

    Source: §V · p. 10

    The authors hypothesize the method may suit low-to-medium-variability data better than high-variability datasets, where autoencoder overgeneralization may yield low error for abnormal data.

    Source: G. LIMITATIONS · p. 10

Abstract

from arXiv · show

Anomaly detection is the problem of recognizing abnormal inputs based on the seen examples of normal data. Despite recent advances of deep learning in recognizing image anomalies, these methods still prove incapable of handling complex medical images, such as barely visible abnormalities in chest X-rays and metastases in lymph nodes. To address this problem, we introduce a new powerful method of image anomaly detection. It relies on the classical autoencoder approach with a re-designed training pipeline to handle high-resolution, complex images and a robust way of computing an image abnormality score. We revisit the very problem statement of fully unsupervised anomaly detection, where no abnormal examples at all are provided during the model setup. We propose to relax this unrealistic assumption by using a very small number of anomalies of confined variability merely to initiate the search of hyperparameters of the model. We evaluate our solution on natural image datasets with a known benchmark, as well as on two medical datasets containing radiology and digital pathology images. The proposed approach suggests a new strong baseline for image anomaly detection and outperforms state-of-the-art approaches in complex medical image analysis tasks.

I. INTRODUCTION

The paper addresses the difficulty of detecting medical anomalies that closely resemble normal images and the lack of thorough comparisons of anomaly-detection methods in medical imaging. It proposes a perceptual autoencoder with progressive training and a small, confined set of anomalies for hyperparameter selection.

I. INTRODUCTION

  • Medical anomalies can closely resemble normal images, including subtle chest X-ray findings and metastases, while thorough comparisons of medical-domain SOTA methods were missing.The paper identifies barely different abnormal tissues as a challenge in both chest X-rays and lymph-node images.Source: §I · pp. 1–2
  • The approach combines autoencoders, perceptual loss, and progressive growth to address high-dimensional medical images.The autoencoder is trained only with perceptual loss to represent normal-image content rather than produce a realistic-looking reconstruction.Source: §I · p. 2
  • Progressive growing starts training at low resolution and adds layers to introduce detail, improving anomaly detection in high-resolution medical data.The method also gradually grows perceptual information in the loss function.Source: §I · p. 2
  • A small number of known anomalies with confined variability is used to select model hyperparameters, rather than requiring a fully anomaly-free setup.The authors frame this as a weakly supervised setup intended to reflect practical tasks and support reproducibility.Source: §I · p. 2
  • 93.4 ROC AUC on Camelyon16 and 92.6 on the NIH subset outperformed SOTA methods by 2.8% and 5.2% in absolute value, respectively.The NIH result applies to a subset of the dataset.Source: §I · pp. 2–3

A. CONTRIBUTIONS

The contributions include comparisons of strong anomaly-detection baselines on medical tasks and a perceptual-autoencoder approach with a limited-anomaly tuning setup. The related-work discussion situates this work among distribution-based, reconstruction-based, and transformation-based methods, noting gaps in medical benchmarks and comparisons.

A. CONTRIBUTIONS

  • The proposed solution outperformed SOTA methods on both medical datasets, and 0.5% of training images with anomalies was enough for hyperparameter selection to yield an outcome within 2% of optimum.The paper also reports comparisons with fine-tuned strong baselines in chest X-ray and H&E-stained histology tasks.Source: A. CONTRIBUTIONS · p. 3

II. RELATED WORK

  • Distribution-based methods flag samples in low-probability regions, using density models or boundaries around normal examples; deep representations can be vulnerable to source-to-target domain shift.Examples discussed include KDE, GMM, One-class SVM, Isolation Forest, and Deep IF.Source: §II · p. 3
  • Reconstruction-based methods rely on models trained only on normal data failing to reconstruct anomalies; choosing a dissimilarity metric and bottleneck size are key difficulties.The paper positions deep-feature perceptual dissimilarity as a promising metric for this method family.Source: §II · p. 3
  • Deep GEO scores anomaly through classification of geometric transformations, a strategy described as sub-optimal for biomedical images invariant to rotation and translation.The paper notes that these transformations are uninformative for homogeneously looking biomedical images.Source: §II · pp. 3–4
  • Medical anomaly studies had included only a few papers and lacked detailed comparisons with the latest strong models; the authors compare methods on two medical datasets with different abnormalities.The paper also identifies the absence of a standardized medical anomaly-detection benchmark.Source: §II · p. 4

III. METHOD

The method trains an autoencoder to preserve image content using perceptual feature differences rather than requiring a realistic reconstructed image. It uses the same reconstruction loss as the abnormality score and discards constraints used by PIAD.

A. DEEP PERCEPTUAL AUTOENCODER

  • Deep Perceptual Autoencoder measures content dissimilarity between input and reconstruction using deep features, without requiring pixel-level similarity or a realistic output image.The authors use perceptual loss both to train the autoencoder and to compute restoration error during evaluation.Source: A. DEEP PERCEPTUAL AUTOENCODER · p. 4
  • The relative perceptual-L1 loss is described as robust to noise and image-contrast changes, and the same reconstruction loss predicts abnormality during evaluation.The loss compares deep features of the input and reconstructed image.Source: A. DEEP PERCEPTUAL AUTOENCODER · p. 4
  • Unlike PIAD, Deep Perceptual Autoencoders discard normally distributed latent vectors and realistic-image reconstruction constraints, producing a tensor that stores input content.The authors propose this flexibility to capture features relevant to data normality.Source: A. DEEP PERCEPTUAL AUTOENCODER · p. 4

B. PROGRESSIVE GROWING

The training pipeline synchronizes progressive autoencoder growth with a gradual increase in perceptual-loss feature depth. It begins at low resolution and repeats resolution-doubling alongside increasing feature depth until the target resolution is reached.

B. PROGRESSIVE GROWING

  • Progressive training synchronizes added autoencoder layers with deeper perceptual-loss features, gradually raising the resolution represented by both.The loss begins with coarse features and incorporates deeper features as training advances.Source: B. PROGRESSIVE GROWING · p. 5, Figure 3 · p. 5
  • Training starts with 8 × 8 images and coarse-layer features, then incrementally adds autoencoder layers while increasing feature depth.The feature extractor computes distances between the input and reconstruction.Source: B. PROGRESSIVE GROWING · p. 5
  • During resolution doubling, α increases linearly from 0 to 1 as new layers are introduced smoothly and image inputs mix new and upsampled prior resolutions.The perceptual-loss information also transitions smoothly from features f1 to f2, with downsampling by a factor of two.Source: B. PROGRESSIVE GROWING · p. 5
  • Training alternates fixed-resolution learning with resolution doubling, repeating the cycle until the target image resolution is reached.Each cycle adds layers, scales up resolution, and increases feature depth in the perceptual loss.Source: B. PROGRESSIVE GROWING · p. 5

C. WEAKLY-SUPERVISED PARADIGM

The paper uses a small, limited-variation set of labeled anomalies to select model hyperparameters during setup, rather than to train for anomaly coverage. This addresses the difficulty of choosing hyperparameters using reconstruction error alone.

C. WEAKLY-SUPERVISED PARADIGM

  • Lower reconstruction loss does not necessarily indicate better anomaly detection, since a larger bottleneck can also reconstruct anomalous inputs accurately.The paper notes that reconstruction-error-based hyperparameter tuning may therefore select a model with weaker detection quality.Source: C. WEAKLY-SUPERVISED PARADIGM · p. 5
  • A small, limited-variation set of labeled anomalies serves as a validation or optimization set solely for selecting model hyperparameters during setup.The paradigm uses these examples for model selection rather than training to improve target performance.Source: C. WEAKLY-SUPERVISED PARADIGM · p. 5
  • 20 abnormal images, less than 0.5% of the training set, are enough to select a model within 2% of the optimum.The examples are used to initiate hyperparameter selection, not to cover every anomaly type.Source: C. WEAKLY-SUPERVISED PARADIGM · p. 6

IV. EXPERIMENTS

The experiments evaluate anomaly detection under a novelty-detection setup, with normal-only training data, across medical and natural-image tasks. They describe the datasets, evaluation protocol, metrics, and baselines used for comparison.

A. DATASETS AND EVALUATION PROTOCOL

  • All approaches are evaluated as novelty detectors with training data assumed to contain no anomalies; hyperparameters are selected using average ROC AUC over three folds.For the NIH subset, the experiments instead repeat the same train-validation split three times to replicate the DAOL settings.Source: A. DATASETS AND EVALUATION PROTOCOL · p. 6, 3) Metrics · p. 6, C. IMPLEMENTATION DETAILS · p. 7
  • Camelyon16 experiments train on healthy tissue to detect metastases, using tiles from four tumor slides for validation and 7,612 normal and 200 tumor images for setup.The test set contains 4,000 normal and 817 tumor images; tiles are sampled from whole-slide images and normalized.Source: 1) Medical Images · p. 6
  • The NIH chest X-ray task covers 14 findings, separates PA and AP projections, and uses Infiltration cases as abnormal examples for hyperparameter search.The evaluated clearer-case subset includes 4,261 normal training images and separate normal and abnormal validation and test sets.Source: 1) Medical Images · p. 6
  • CIFAR10 and SVHN use one-vs-all evaluation across ten experiments per dataset, with 32 × 32 images and one abnormal class sampled for validation.Table 1 reports ROC AUC averages across the ten experiments and three runs, showing default and cross-validated hyperparameter options.Source: 2) Natural Images · p. 6, Table 1 · p. 7

B. BASELINES

  • Medical-image comparisons include Deep GEO, Deep IF, and PIAD; the NIH experiments also compare against DAOL, while natural-image tests add several other baselines.The strongest baselines receive hyperparameter tuning for standardized comparison.Source: B. BASELINES · p. 7, C. IMPLEMENTATION DETAILS · p. 7

C. IMPLEMENTATION DETAILS

  • The proposed approach uses pre-activation residual-block autoencoders and ImageNet-pretrained VGG19 features for relative-perceptual-L1 loss.Training stops when hold-out normal-image loss stops decreasing, and hyperparameter search selects bottleneck size and feature layer.Source: C. IMPLEMENTATION DETAILS · p. 7

D. RESULTS

The approach significantly outperforms Deep GEO and Deep IF on both medical datasets, while on natural-image benchmarks it trails Deep GEO and closely approaches leaders on CIFAR10.

Natural Images

  • For 32×32 CIFAR10/SVHN images, progressive growth was not used; the method ranked second on SVHN and outperformed PIAD by ∼3% on both datasets, closely approaching reported CIFAR10 leaders Deep GEO and Deep IF.Source: D. RESULTS · p. 7
  • The authors hypothesize that highly variable normal images let autoencoders generalize better to unseen classes, making reconstruction-based methods inferior to Deep GEO on natural-image tasks.Their hyperparameter tuning seeks an autoencoder that reconstructs normal data well without generalizing to other data.Source: D. RESULTS · p. 7

Medical Images

  • Our approach significantly outperforms Deep GEO and Deep IF on both medical datasets; Deep GEO reaches only 52.4% ROC AUC on digital pathology.The authors attribute Deep GEO’s poor pathology performance to scans being invariant to geometric transformations.Source: Table 2 · p. 8, D. RESULTS · pp. 8–9
  • Figure 4 shows normal and anomalous pathology and chest X-ray examples with predicted scores, where higher scores indicate greater anomaly likelihood, including borderline cases.The examples are from Camelyon16 lymph-node images and the NIH chest X-ray dataset.Source: Figure 4 · p. 8
  • Deep IF achieves ROC AUC scores of 52.2% on NIH (PA proj.) and 54.3% on NIH (AP proj.), which the authors associate with ImageNet-to-X-ray domain shift.The authors identify feature representation as Deep IF’s weakest side and note no explicit algorithm for obtaining discriminative feature spaces across experimental setups.Source: D. RESULTS · p. 9
  • Progressive growth adds 1% in image quality (ROC AUC), and the authors report their method is easier and faster to train and least resource-hungry among the considered models.Table 3 reports average training times; its caption specifies minutes and experiments on a GeForce GTX 1080 Ti with PyTorch 1.4.0.Source: Table 3 · p. 8, D. RESULTS · p. 9

E. HYPERPARAMETERS TUNING ANALYSIS

In the two experiments, 20 abnormal examples of one type—less than 0.5% of the normal training examples—were sufficient to select hyperparameters within 2% of the optimum.

  • 20 abnormal examples of one type, less than 0.5% of normal training examples, selected hyperparameters within 2% of the optimal configuration in both experiments.The authors report that this small set was enough to reject inferior hyperparameter configurations.Source: E. HYPERPARAMETERS TUNING ANALYSIS · p. 9

F. ABLATION STUDY

The ablations compare perceptual and L1 losses, hyperparameter-selection criteria, additional reconstruction or adversarial losses, and progressive growing. Perceptual loss and limited-anomaly hyperparameter selection improved performance, while extra losses deteriorated it and progressive growing added improvement.

F. ABLATION STUDY

  • Ten ablation scenarios compared L1 and perceptual-loss training, hyperparameter criteria, added adversarial or L1 losses, and progressive growing.The adversarial-loss variants used a jointly trained discriminator with a Wasserstein GAN with Gradient Penalty objective.Source: F. ABLATION STUDY · p. 9
  • Hyperparameter selection using a confined-variability subset of anomalies noticeably benefited performance over unsupervised reconstruction-loss criteria.The compared scenarios used the best validation ROC AUC for the weakly supervised selection.Source: F. ABLATION STUDY · p. 9
  • Natural-image results use ROC AUC (%) with standard deviations across three runs, comparing default and weakly supervised cross-validated settings in one-vs-all CIFAR10 and SVHN tests.Each dataset has ten configurations, with one class treated as normal and the others as abnormal.Source: Table 5 · p. 10
  • Perceptual loss outperformed L1 by a large margin, additional losses deteriorated performance, and progressive growing further improved the proposed model.The authors conclude that perceptual loss alone gives the autoencoder more flexibility to interpret normality.Source: F. ABLATION STUDY · p. 10

G. LIMITATIONS

The proposed method excelled on the medical datasets but did not outperform the assessed state-of-the-art methods on natural-image baselines. Its hypothesized suitability for low- to medium-variability data remains to be tested through a controlled study of data diversity.

G. LIMITATIONS

  • The method excelled on medical datasets but did not outperform the assessed state-of-the-art approaches on natural-image baselines.The authors hypothesize it may suit low- to medium-variability data better than high-variability data; they leave controlled study of this dependence to future work.Source: G. LIMITATIONS · p. 10
  • The proposed preference for modality-tuned sensitivity over generalized methods is framed as a possibility for individual imaging niches.The passage presents this as an argument, not a demonstrated general preference.Source: G. LIMITATIONS · p. 10

V. CONCLUSIONS

The paper proposes a perceptual-loss autoencoder trained with progressive growing and uses a small, limited-variation set of anomalies to select hyperparameters. It reports strong results on metastasis and abnormal chest X-ray detection, establishing a baseline for medical-image anomaly detection.

V. CONCLUSIONS

  • The approach achieved 0.934 ROC AUC for metastasis detection and 0.926 for abnormal chest X-ray detection.Source: §V · p. 10
  • The method uses an autoencoder to model normal-data representations, optimized with perceptual loss under progressive-growing training.Source: §V · p. 10
  • A small set of anomalous examples with limited variation is used to select model hyperparameters on new data.Source: §V · p. 10
  • The authors conclude that the approach establishes a strong baseline for anomaly detection in medical imaging.Source: §V · p. 10

APPENDIX A

The appendix reports natural-image evaluation details and provides ROC curves for NIH dataset variants alongside training-performance curves for the proposed method.

APPENDIX A

  • Table 5 reports ROC AUC (%) and standard deviations over three runs for ten one-vs-all configurations per dataset on CIFAR10 and SVHN.Each configuration designates one class as normal and the remaining classes as abnormal; default and weakly supervised cross-validated hyperparameters are reported.Source: Appendix A · pp. 10–11
  • Figure 6 presents ROC curves for Deep GEO, PIAD, Deep IF, and the proposed model on three NIH dataset variants.The variants are an NIH subset, NIH (PA proj.), and NIH (AP proj.).Source: Figure 6 · p. 11, Appendix A · p. 11
  • Figure 7 shows training-performance curves for the proposed method on the same three NIH dataset variants.Source: Figure 7 · p. 11, Appendix A · p. 11
Loading 2006.13265v3…