PaperlayerView exact revision on arXiv

Source-linked AI summary

AnoViT: Unsupervised Anomaly Detection and Localization with Vision Transformer-based Encoder-Decoder

Yunseung Lee, Pilsung Kang

arXiv:2203.10808v1cs.CVcs.LG

TL;DR

Image anomaly detection must identify abnormal images and locate defects, but convolutional models have difficulty learning global image context. AnoViT uses a ViT-based encoder-decoder to model relationships between image patches and detect anomalies through reconstruction error. It improves over l2-CAE across the reported benchmark datasets, with qualitative localization results across anomalous-area types and sizes.

  • Problem

    Convolutional anomaly-detection methods mainly use local image information, while detecting abnormal images and locating defects matter in applications such as manufacturing and medical care.

    Source: §1 · pp. 1–2

  • Method

    AnoViT uses a ViT encoder to learn relationships between patches, preserves patch locations in a feature map, and applies pixelwise reconstruction error for anomaly detection and localization.

    Source: §1 · p. 2, §3.1 · p. 4, §3.2 · p. 4

  • Results

    AnoViT improved over l2-CAE in 7 of 10 MNIST classes and 7 of 10 CIFAR10 classes; on MVTecAD, it improved anomaly-detection results in 9 of 15 categories.

    Source: §5.1 · p. 6

  • Takeaways & Limitations

    Qualitative localization results were reported across anomalous-area sizes and shapes, with predicted areas similar to ground-truth masks.

    Source: §5.2 · pp. 7–8

    The evaluation compared AnoViT with l2-CAE; comparison with several other encoder-decoder models remains future work.

    Source: §6 · p. 8

Abstract

from arXiv · show

Image anomaly detection problems aim to determine whether an image is abnormal, and to detect anomalous areas. These methods are actively used in various fields such as manufacturing, medical care, and intelligent information. Encoder-decoder structures have been widely used in the field of anomaly detection because they can easily learn normal patterns in an unsupervised learning environment and calculate a score to identify abnormalities through a reconstruction error indicating the difference between input and reconstructed images. Therefore, current image anomaly detection methods have commonly used convolutional encoder-decoders to extract normal information through the local features of images. However, they are limited in that only local features of the image can be utilized when constructing a normal representation owing to the characteristics of convolution operations using a filter of fixed size. Therefore, we propose a vision transformer-based encoder-decoder model, named AnoViT, designed to reflect normal information by additionally learning the global relationship between image patches, which is capable of both image anomaly detection and localization. The proposed approach constructs a feature map that maintains the existing location information of individual patches by using the embeddings of all patches passed through multiple self-attention layers. The proposed AnoViT model performed better than the convolution-based model on three benchmark datasets. In MVTecAD, which is a representative benchmark dataset for anomaly localization, it showed improved results on 10 out of 15 classes compared with the baseline. Furthermore, the proposed method showed good performance regardless of the class and type of the anomalous area when localization results were evaluated qualitatively.

1. Introduction

Image anomaly detection identifies abnormal images, while localization finds defects within them; encoder-decoder methods use reconstruction error, but convolutional models have difficulty learning global context. AnoViT applies a ViT-based encoder-decoder to use relationships between patches for both tasks.

1. Introduction

  • Encoder-decoder methods reconstruct normal inputs and identify anomalies from differences between original and reconstructed images.The introduction contrasts these with generation-based methods, which learn a distribution of normal data and compare original and generated images.Source: §1 · pp. 1–2
  • Patch-level methods represent detailed image areas and can simplify relative-location prediction in self-supervised learning.The cited examples include patch-level inputs used in recent anomaly-detection work and pretext tasks.Source: §1 · p. 2
  • CNN-based methods mainly learn local information, whereas ViT uses patch-level images and attention to learn global image context.The paper presents this as a limitation of CNN-based representations and a motivation for ViT.Source: §1 · p. 2
  • AnoViT uses a ViT-based encoder-decoder and reconstruction error for image anomaly detection and localization, incorporating both local and global information.The paper reports improved detection and localization performance on MVTecAD.Source: §1 · p. 2

2. Related Work

Encoder-decoder anomaly detection learns to reconstruct normal images and uses reconstruction error to identify anomalies. ViT processes image patches with self-attention to learn relationships between them.

2.1. Encoder-Decoder-based Image Anomaly Detection

  • Encoder-decoder methods train on normal images and use reconstruction error to distinguish abnormal inputs.The encoder compresses a normal image and the decoder reconstructs it; the model learns the normal-image distribution during optimization.Source: §2.1 · p. 2
  • Convolutional autoencoders support image- and pixel-level anomaly detection, while prior MVTec comparisons found l2-CAE outperformed SSIM-CAE, AnoGAN, and CNN Feature Dictionary.CAE methods have used l1- or l2-loss, sometimes adding SSIM for texture defects; other approaches include multi-scale encoding, denoising, and memory modules.Source: §2.1 · pp. 2–3

2.2. Vision Transformer

  • The original ViT architecture illustrates a transformer model adapted for images.The cited figure is identified as the architecture of original ViT.Source: Figure 1 · p. 3
  • ViT divides images into patches, adds positional embeddings, and uses multi-head self-attention to learn relationships between patches.The original ViT uses the [cls] token with an MLP head for image classification; the paper also reports ViT outperformed CNN models on classification benchmarks.Source: §2.2 · p. 3

3. Proposed Method

AnoViT combines a ViT encoder with a convolutional decoder, using patch relationships to learn normal-image representations. It reconstructs inputs and uses pixelwise reconstruction errors for anomaly detection and localization.

3.1. ViT-based Encoder-Decoder

  • AnoViT combines a ViT encoder and convolutional decoder, learning patch relationships through multi-head self-attention while minimizing reconstruction error.The architecture is presented in Figure 2.Source: §3.1 · p. 3, Figure 2 · p. 4
  • Unlike classification ViTs, AnoViT uses patch embeddings rather than only the [cls] token, then rearranges patch embeddings into a three-dimensional feature map.The feature map goes directly to the decoder without a fully connected layer, preserving spatial information between patches.Source: §3.1 · p. 4
  • The decoder reconstructs an image at the original size from the feature map using transposed convolution layers.This design lets the feature map serve as decoder input while preserving patch spatial information.Source: §3.1 · p. 4

3.2. Anomaly Detection and Localization

  • AnoViT computes per-pixel l2 reconstruction error and averages across channels to form score map M.The input pixel is compared with its reconstructed counterpart.Source: §3.2 · p. 4
  • The maximum value in score map M supplies the image-level anomaly score, while M indicates abnormal pixels for localization.The method relies on anomalous areas being reconstructed poorly, increasing their reconstruction error.Source: §3.2 · p. 4

4. Experimental Setup

The experiments use MNIST, CIFAR10, and MVTecAD to evaluate anomaly detection, with localization evaluated on MVTecAD. They compare AnoViT with l2-CAE using AUROC and describe the model implementation and score-map postprocessing.

4.1. Data

  • Anomaly detection was evaluated on MNIST, CIFAR10, and MVTecAD, while anomaly localization was evaluated on MVTecAD.Source: §4.1 · pp. 4–5

4.1.1. MNIST, CIFAR10

  • For MNIST and CIFAR10, one of ten classes was treated as normal training data and the other nine as abnormal; both datasets were resized to (224, 224, 3).MNIST has approximately 6,000 training images per class, with 80% used for training and verification; CIFAR10 has 5,000 per class, with 4,500 used for training and 500 for verification.Source: §4.1.1 · p. 5

4.1.2. MVTEC ANOMALY DETECTION DATASET

  • MVTecAD contains 15 products—10 object and five texture classes—with 3,629 normal and 1,725 abnormal images resized to (384, 384, 3).It provides image-level labels and pixel-level anomaly-location ground truth for detection and localization evaluation.Source: §4.1.2 · p. 5

4.2. Baselines and Evaluation Metrics

  • The CNN-based l2-CAE, which also uses reconstruction error, served as the baseline because prior work reported its leading MVTecAD detection and localization performance.The authors re-implemented it using the two cited Bergmann papers.Source: §4.2 · p. 5
  • AUROC was used to evaluate anomaly detection and localization from the false-positive and true-positive rates, without deterministically setting an abnormality threshold.Source: §4.2 · p. 5

4.3. Implementation Details

  • AnoViT uses a pretrained ViT encoder with patch size 16, embedding dimension 768, and eight heads, plus a six-block transposed-convolution/ReLU decoder and final upsampling.Gaussian smoothing disperses pixel scores into a final score map; the same image augmentations were applied to AnoViT and l2-CAE.Source: §4.3 · pp. 5–6

5. Experimental Results

AnoViT outperformed l2-CAE on anomaly detection across MNIST, CIFAR10, and MVTecAD. Qualitative MVTecAD results show localization close to ground-truth masks across varied anomaly sizes and shapes.

5.1. Anomaly Detection & Localization Performance

  • AnoViT outperformed l2-CAE on MNIST, CIFAR10, and MVTecAD, improving 7/10 classes on each of the first two datasets and 9/15 MVTecAD categories.The reported MNIST average AUROC increased by 1.774% compared with l2-CAE.Source: §5.1 · p. 6, Table 2 · p. 6, Table 3 · p. 6
  • On MVTecAD, AnoViT detected anomalies relatively better than l2-CAE by over 5.624% on average and performed well on texture and object products.Table 4 reports anomaly detection results across MVTecAD categories.Source: §5.1 · p. 6, Table 4 · p. 6
  • Under the same training and preprocessing conditions, anomaly detection and localization improved over CNN, with self-attention reflecting global as well as local information in patch embeddings.The authors attribute this outcome to multiple self-attention operations in ViT.Source: §5.1 · p. 7

5.2. Qualitative Analysis

  • Figure 3 qualitatively shows localization across texture and object products, with AnoViT detecting anomalous areas regardless of their size and shape.Its predicted regions were very similar to the ground-truth masks.Source: §5.2 · pp. 7–8, Figure 3 · p. 7
  • For wood and grid images, l2-CAE assigned high anomaly scores to normal-region pixels, whereas AnoViT’s scores outside anomalous regions were low.The reported comparison notes that the l2-CAE abnormal area extended beyond the ground-truth mask.Source: §5.2 · p. 7

6. Conclusion

AnoViT uses a ViT encoder-decoder to detect and localize anomalies from reconstruction error, incorporating relationships between image patches while preserving spatial information. It outperformed the CNN-based l2-CAE in the reported evaluations, though the comparison was limited to that baseline.

6. Conclusion

  • Rich normal embeddings matter because encoder-decoder anomaly scores are calculated from reconstruction error.The paper situates anomaly detection and localization in applications including manufacturing and medical care.Source: §6 · p. 8
  • AnoViT performed better than the CNN-based l2-CAE on quantitative and qualitative evaluations.The study selected l2-CAE as its CNN-based comparison model; further evaluation against other encoder-decoder models is needed.Source: §6 · p. 8
  • The ViT encoder represents normal information using relationships between image patches, and its patch embeddings form a three-dimensional feature map that preserves spatial information for the decoder.Anomaly detection and localization use reconstruction error.Source: §6 · p. 8
  • The evaluation compared AnoViT with l2-CAE; the authors call for further comparisons with other encoder-decoder models.They suggest evaluating DeiT or CrossViT encoders in future work.Source: §6 · p. 8

Related papers

Loading 2203.10808v1…