PaperlayerView exact revision on arXiv

Source-linked AI summary

Attention Guided Anomaly Localization in Images

Shashanka Venkataramanan, Kuan-Chuan Peng, Rajat Vikram Singh, Abhijit Mahalanobis

arXiv:1911.08616v4cs.CVeess.IV

TL;DR

Anomaly localization is difficult because anomalies are scarce, small, and often unavailable during training, while prior methods may require anomalous images for threshold selection. CAVGA uses a spatial convolutional latent representation with guided attention losses, and it outperforms reported SOTA methods across localization and detection benchmarks, including weak supervision with only 2% anomalous training images.

  • Problem

    Anomaly localization must segment anomalous regions despite small anomaly pixel coverage and the limited availability of anomalous training images needed by many prior threshold-based methods.

  • Method

    CAVGA combines a convolutional latent variable with attention expansion and complementary guided attention losses for unsupervised and weakly supervised anomaly localization.

  • Results

    CAVGA outperforms SOTA anomaly localization methods on MVTAD, mSTC, and LAG, and SOTA anomaly detection methods across MVTAD, mSTC, LAG, MNIST, Fashion-MNIST, and CIFAR-10.

  • Takeaways & Limitations

    CAVGA provides anomaly localization without anomalous training images in the unsupervised setting and remains effective with only 2% extra anomalous images in the weakly supervised setting.

Abstract

from arXiv · show

Anomaly localization is an important problem in computer vision which involves localizing anomalous regions within images with applications in industrial inspection, surveillance, and medical imaging. This task is challenging due to the small sample size and pixel coverage of the anomaly in real-world scenarios. Most prior works need to use anomalous training images to compute a class-specific threshold to localize anomalies. Without the need of anomalous training images, we propose Convolutional Adversarial Variational autoencoder with Guided Attention (CAVGA), which localizes the anomaly with a convolutional latent variable to preserve the spatial information. In the unsupervised setting, we propose an attention expansion loss where we encourage CAVGA to focus on all normal regions in the image. Furthermore, in the weakly-supervised setting we propose a complementary guided attention loss, where we encourage the attention map to focus on all normal regions while minimizing the attention map corresponding to anomalous regions in the image. CAVGA outperforms the state-of-the-art (SOTA) anomaly localization methods on MVTec Anomaly Detection (MVTAD), modified ShanghaiTech Campus (mSTC) and Large-scale Attention based Glaucoma (LAG) datasets in the unsupervised setting and when using only 2% anomalous images in the weakly-supervised setting. CAVGA also outperforms SOTA anomaly detection methods on the MNIST, CIFAR-10, Fashion-MNIST, MVTAD, mSTC and LAG datasets.

1 Introduction

Anomaly localization is important but difficult when anomalous images are scarce and anomalies occupy few pixels. CAVGA addresses this with guided attention, avoiding anomalous training images in the unsupervised setting and using limited anomalous data weakly supervised.

  • Motivation: Existing methods often use thresholded reconstruction differences and require anomalous training images to determine class-specific localization thresholds.This requirement is problematic because anomalous images may be unavailable in real-world settings, while anomalies can have small pixel coverage.
  • Proposed approach: CAVGA uses a convolutional latent variable to preserve spatial relations while learning anomaly localization without anomalous training images.The method is presented as an end-to-end trainable anomaly localization framework with attention guidance.
  • Unsupervised setting: The attention expansion loss trains the network to focus on all normal regions in unsupervised training, so unfocused regions can indicate anomalies at test time.The loss is motivated by using attention supervision to alleviate the need for large training datasets.
  • Weakly supervised setting: The complementary guided attention loss expands normal attention while suppressing anomalous attention for correctly classified normal images in the weakly supervised setting.This extension introduces a classifier and uses a small set of anomalous training images without requiring segmented anomaly masks.
  • Reported contributions: CAVGA outperforms SOTA anomaly localization methods on MVTAD, mSTC, and LAG, and SOTA anomaly detection methods on six datasets.The reported anomaly localization gains cover IoU and mean AuROC on MVTAD and mSTC, and IoU on LAG; anomaly detection evaluations include MVTAD, mSTC, LAG, MNIST, Fashion-MNIST, and CIFAR-10.

2 Related Works

Prior work studies anomaly localization as pixel-accurate segmentation and commonly relies on reconstruction-based or patch-based deep-learning methods. CAVGA is distinguished in the supplied comparison by satisfying all listed unsupervised working properties.

  • Definitions and scope: Anomaly localization or segmentation concerns pixel-accurate segmentation of anomalous image regions, complementing anomaly detection's image-level normal-versus-anomalous decision.Applications include industrial inspection and medical imaging, while related novelty-detection methods also extend to video analysis and remote sensing.
  • Comparison: Table 1 reports that only CAVGA satisfies all listed working properties among the compared unsupervised anomaly localization methods.The supplied table caption identifies the comparison scope but does not enumerate the individual properties.
  • Anomaly localization methods: Deep anomaly localization methods use autoencoders, GANs, and inpainter-detector or regularized-autoencoder approaches to identify anomalous regions.Several methods localize anomalies through thresholded pixel-wise differences between inputs and reconstructions, while another uses patch-based localization.

3 Proposed Approach: CAVGA

CAVGA combines a convolutional latent variable with attention-guided losses to localize anomalies without anomalous training images, while extending the framework with classifier-guided supervision for weakly supervised training.

  • CAVGA framework: CAVGA uses a convolutional latent variable to preserve spatial relations between the input and latent representation, supporting spatial anomaly localization.Unlike traditional autoencoders, the latent variable is not flattened; the framework uses convolutional representations and adversarial reconstruction.
  • Unsupervised CAVGAu: The attention expansion loss Lae encourages Grad-CAM attention from z to cover all normal image regions, so anomalies are localized where attention is absent.Lae is added to the reconstruction and adversarial objectives because standard attention mechanisms typically focus only on the most salient regions.
  • Unsupervised CAVGAu: During testing, CAVGAu reconstructs the input, computes pixel-wise reconstruction error as an anomaly score, and inverts thresholded attention to obtain anomaly localization.The anomaly score is normalized and thresholded at 0.5, while the inverted attention map is likewise evaluated using a 0.5 threshold.
  • Weakly supervised CAVGAw: CAVGAw adds a binary classifier trained with binary cross entropy, using its prediction to generate anomalous-class attention during inference.The classifier is attached to a flattened copy of z, while z and the classifier branch share parameters.
  • Weakly supervised CAVGAw: The complementary guided attention loss Lcga suppresses anomalous attention and expands normal attention, but is applied only to correctly classified normal images.This condition avoids training on erroneous attention maps produced by incorrect classifier predictions.

4 Experimental Setup

CAVGA is evaluated on multiple anomaly-localization and anomaly-detection datasets against established baseline methods, using unsupervised training on normal images and weakly supervised training with limited anomalous data.

  • Benchmark datasets: CAVGA is evaluated for anomaly localization on MVTAD, mSTC, and LAG, and for anomaly detection on those datasets plus MNIST, CIFAR-10, and Fashion-MNIST.For mSTC, every fifth video frame is extracted without temporal information, producing the modified image dataset mSTC.
  • Baseline methods: Localization comparisons include AVID, AEL2, AESSIM, AnoGAN, CNNFD, TI, γ-VAEg, LSA, ADVAE, and VM-based approaches.The comparison includes methods using different autoencoder, GAN, feature-dictionary, and gradient-based approaches.
  • Architecture details: The architecture uses ResNet-18 convolution layers pretrained on ImageNet as the encoder and a modified residual generator as the residual decoder.Additional CAVGA variants use a Celeb-A-pretrained discriminator and generator, or DC-GAN components, for fair architectural comparisons.
  • Evaluation measures: Tables report category- or scene-level performance together with aggregate IoU, AuROC, or classification-accuracy measures for the evaluated datasets.Table 3 covers category-specific and mean localization metrics on MVTAD.
  • Training and evaluation: Networks are trained only on normal images in the unsupervised setting, while weakly supervised training uses 2% of anomalous images with all normal training images.The anomalous images are randomly selected because baseline methods do not report how many they use for threshold computation.

5 Experimental Results

CAVGA outperforms the compared methods across the reported anomaly-localization benchmarks and also achieves stronger anomaly-detection performance across image and video-derived datasets.

  • MVTAD localization: 5% higher mean IoU than γ-VAEg on MVTAD is achieved by CAVGA-Du without anomalous training images, while CAVGA-Dw improves over CAVGA-Du by 22% in mean IoU and 8% in mean AuROC.CAVGA-Du also exceeds CNNFD, TI, and VM by 9%, 12%, and 10% in mean AuROC, respectively, and performs comparably to the best baseline.
  • Qualitative results and caveat: Qualitative attention maps depict localized anomalies in red on MVTAD and mSTC, while low contrast between anomalies and background remains a localization challenge.CAVGA still outperforms the baselines in these difficult scenarios but does not localize the anomaly well.
  • mSTC and LAG localization: CAVGA outperforms the baseline methods in mean IoU and mean AuROC on mSTC and exceeds Patho-GAN by 16% in IoU on LAG.The reported LAG result concerns anomaly localization, while the mSTC comparison reports both mean localization metrics.
  • Dataset anomaly detection: CAVGA-Du exceeds the strongest MVTAD anomaly-detection baseline by 1.3% in mean classification accuracy and outperforms baselines on mSTC and LAG by 2.6% and 1.1%, respectively.The MVTAD metric is the mean accuracy of correctly classified anomalous and normal images.
  • Standard-image anomaly detection: On MNIST, CIFAR-10, and Fashion-MNIST, CAVGA-Du outperforms all listed baselines in mean AuROC in the unsupervised setting.On MNIST, it exceeds MemAE and β-VAE by 1.1% and 8%; on CIFAR-10, by 21% and 38%, respectively.

6 Ablation Study

Ablations show that the convolutional latent variable and attention losses each improve anomaly localization, with complementary guided attention benefiting weakly supervised models.

  • Ablation protocol: The ablation study evaluates five randomly chosen MVTAD categories and reports the mean across all 15 categories in Table 8.Qualitative ablation results are shown in Fig. 5.
  • Effect of convolutional latent variable z: Preserving spatial relations through convolutional z improves anomaly-localization IoU in both unsupervised and weakly supervised settings, with or without the attention losses.Convolutional z outperforms a flattened latent variable even when Lae or Lcga is added.
  • Effect of attention expansion loss Lae: Lae enhances IoU regardless of whether the latent variable is flattened or convolutional in the unsupervised setting.The comparison uses columns c1 versus c2 and c3 versus c4 in Table 8.
  • Effect of complementary guided attention loss Lcga: Lcga enhances IoU regardless of whether the latent variable is flattened or convolutional in the weakly supervised setting.The comparison uses columns c5 versus c6 and c7 versus c8 in Table 8.

7 Conclusion

CAVGA uses guided attention in an end-to-end convolutional adversarial variational autoencoder for anomaly localization. It outperforms SOTA localization and detection methods across multiple datasets and supervision settings.

  • 7 Conclusion: CAVGA outperforms SOTA anomaly localization methods on MVTAD, mSTC, and LAG in unsupervised and weakly supervised settings.The weakly supervised setting uses only 2% extra anomalous images for training.
  • 7 Conclusion: CAVGA also outperforms SOTA anomaly detection methods on MVTAD, mSTC, LAG, MNIST, Fashion-MNIST, and CIFAR-10.
  • 7 Conclusion: The method is an end-to-end convolutional adversarial variational autoencoder using guided attention for anomaly localization.
Loading 1911.08616v4…