PaperlayerView exact revision on arXiv

Source-linked AI summary

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization

Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, Tomas Pfister

arXiv:2104.04015v1cs.CV

TL;DR

The paper targets defect detection with unknown anomalies when only normal training data are available. It learns self-supervised representations by distinguishing normal images from CutPaste augmentations, then applies generative one-class modeling and patch representations. On MVTec, it reports 95.2 image-level AUC from scratch, 96.6 with ImageNet transfer learning, and 96.0 pixel-level localization AUC without anomaly data.

  • Problem

    Anomaly detection must identify unknown, often subtle defects despite limited access to anomalous training data.

  • Method

    The method learns self-supervised deep representations by classifying normal images against CutPaste augmentations, then builds a generative one-class classifier and extends representations to local patches.

  • Results

    96.6 image-level AUC is achieved with ImageNet transfer learning, while training from scratch reaches 95.2 AUC and patch-based localization reaches 96.0 pixel-level AUC on MVTec.

  • Takeaways & Limitations

    CutPaste-based representations generalize from synthetic spatial irregularities to varied real-world defects and support both image-level detection and pixel-wise localization.

  • Takeaways & Limitations

    The framework uses a simple parametric Gaussian density estimator because nonparametric KDE requires many examples and can be computationally expensive.

Abstract

from arXiv · show

We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data. To this end, we propose a two-stage framework for building anomaly detectors using normal training data only. We first learn self-supervised deep representations and then build a generative one-class classifier on learned representations. We learn representations by classifying normal data from the CutPaste, a simple data augmentation strategy that cuts an image patch and pastes at a random location of a large image. Our empirical study on MVTec anomaly detection dataset demonstrates the proposed algorithm is general to be able to detect various types of real-world defects. We bring the improvement upon previous arts by 3.1 AUCs when learning representations from scratch. By transfer learning on pretrained representations on ImageNet, we achieve a new state-of-theart 96.6 AUC. Lastly, we extend the framework to learn and extract representations from patches to allow localizing defective areas without annotations during training.

1. Introduction

The paper addresses anomaly detection when anomalous data are scarce and defects are subtle, using CutPaste-based self-supervised representations followed by one-class modeling. On MVTec, the approach improves image-level detection and extends to patch-based localization.

  • Anomaly detection must identify unknown defective patterns despite limited anomalous data and fine-grained defects in high-resolution images.
  • The proposed two-stage framework first learns self-supervised representations with CutPaste, then builds a generative one-class classifier on those representations.CutPaste cuts a rectangular image patch and pastes it at a random location to create augmented examples.
  • Whole-image representations support holistic anomaly detection and GradCAM localization, while patch representations produce dense anomaly maps for detection and localization.
  • CutPaste creates spatial irregularities as coarse approximations of unavailable real defects, yet representations trained to detect them generalize to real defect detection.The augmentation varies patch sizes, aspect ratios, and rotation angles.
  • 96.6 image-level AUC is achieved through ImageNet transfer learning, while training from scratch reaches 95.2 AUC and patch-based localization reaches 96.0 pixel-level AUC.The results are reported on the MVTec anomaly detection dataset without using anomaly data.

2. A Framework for Anomaly Detection

The framework learns self-supervised representations from normal images using CutPaste-based irregularity prediction, then fits generative one-class classifiers for image-level detection and patch-level localization.

  • The two-stage framework first learns deep representations from normal data, then constructs a one-class classifier using those representations.
  • Self-Supervised Learning with CutPaste: CutPaste creates local irregularities by cutting a variable rectangular patch, optionally rotating or jittering it, and pasting it at a random location.
  • Self-Supervised Learning with CutPaste: The self-supervised objective trains a binary classifier to distinguish normal images from CutPaste-augmented images using cross-entropy.
  • CutPaste Variants: CutPaste-Scar adds a long-thin, scar-like pasted patch, and a 3-way task separates normal, CutPaste, and CutPaste-Scar examples.
  • CutPaste Variants: Representations learned from CutPaste distinguish normal, augmented, and real defect examples even though CutPaste samples barely overlap with real defects.
  • Localization with Patch Representation: The detector uses generative density estimation on learned representations, while patch representations produce patch anomaly scores that are smoothed into pixel-level localization maps.

3. Related Work

The paper situates CutPaste among one-class anomaly detection and self-supervised methods, distinguishing its label-free structural patch augmentation from related approaches and combining it with transfer learning and patch models.

  • One-class anomaly detection commonly uses only normal training data, while self-supervised approaches include transformation prediction and contrastive learning.
  • CutPaste prediction is designed for defect detection and can combine with ImageNet transfer learning and patch-based models for improved performance and localization.
  • Unlike Cutout and RandomErasing, CutPaste fills a region with a structural patch from the image and learns representations discriminative to the augmentation.
  • CutPaste with a long-thin scar-scale patch improves upon representations trained by predicting Cutout.
  • Unlike CutMix, CutPaste is self-supervised without image labels and targets one-class classification rather than standard supervised tasks.
  • Compared with patch-swap denoising autoencoders and GAN-based local-augmentation methods, the paper describes CutPaste as simpler and practical.

4. Experiments

Experiments evaluate CutPaste-based one-class defect detection and localization on MVTec, including augmentation comparisons, patch-based localization, and transfer learning with EfficientNet.

  • Experimental Setup: MVTec AD contains 10 object and 5 texture categories, with normal training images and defective test images annotated at pixel level.Training sets contain 60 to 391 images per category.
  • Experimental Setup: The one-class protocol trains a separate classifier for each category using only its normal training examples.The detector uses Gaussian density estimation on top-pooled features.
  • Main Results: Rotation prediction reaches 73.1 AUC, whereas Scar prediction reaches 85.0 AUC for defect detection.The authors attribute some rotation failures to unaligned objects such as screws and report suboptimal performance on capsule.
  • Main Results: CutPaste, CutPaste-Scar, and 3-way CutPaste achieve 90.9, 93.5, and 95.2 AUCs, respectively; ensembling five 3-way models reaches 96.1 AUC.The 95.2 AUC result surpasses P-SVDD’s 92.1 AUC when learning from scratch.
  • Defect Localization: GradCAM with the image-level detector achieves 88.3 pixel-wise localization AUC, while the patch-based model achieves 96.0 AUC.The patch model uses 64×64 patches from 256×256 images, extracts scores densely with stride 4, and applies receptive-field upsampling with Gaussian smoothing.
  • Transfer Learning with Pretrained Models: Fine-tuning pretrained EfficientNet B4 with 3-way CutPaste reaches 96.6 AUC, compared with 94.5 AUC from its untuned representation.CutPaste also improves the level-7 EfficientNet feature from 96.8 to 97.1±0.0.

5. Ablation Study

The ablation studies examine CutPaste variants, augmentation combinations, generalization to synthetic unseen anomalies, and semantic anomaly detection. Increasing proxy-task difficulty and combining augmentations improve defect detection, while CutPaste remains behind rotation prediction for CIFAR-10 semantic anomalies.

  • CutPaste variants: 71.3 AUC from standard Cutout improves as proxy tasks introduce random colors or local structures, encouraging irregularity detection.The standard Cutout task may permit naive solutions, whereas harder variants better support generalization.
  • Augmentation combinations: 3-way classification of normal, CutPaste, and CutPaste-scar representations outperforms binary classification using their union.Using both augmentations improves performance, with the 3-way formulation performing better than the binary alternative.
  • Synthetic anomaly detection: CutPaste representations generalize to synthetic anomalies formed by patching diverse shapes and statistics unseen during training.The synthetic anomalies include digits, squares, ellipses, and hearts filled with random colors or natural images.
  • Semantic anomaly detection: 69.4 AUC from CutPaste exceeds Cutout’s 60.2 AUC but remains below rotation prediction’s 91.3 AUC on CIFAR-10 semantic anomaly detection.This comparison concerns semantic anomalies, where rotation prediction performs substantially better than the CutPaste variants tested.

6. Conclusion

The paper presents CutPaste as a data-driven approach for defect detection and localization. It reports strong image-level detection and state-of-the-art pixel-wise localization using self-supervised representations and patch-level features.

  • Conclusion: CutPaste is proposed as a simple augmentation that encourages representations to identify local irregularities.The approach targets defect detection and localization without requiring anomalous training data or localization annotations.
  • Conclusion: The method achieves superior image-level anomaly detection performance on a real-world dataset.
  • Conclusion: Patch-level representations yield state-of-the-art pixel-wise anomaly localization performance.

A.1. Experiment with ResNet-18

The ResNet-18 experiment trains on 256×256 images with momentum SGD and scheduled regularization. Random translation and color jitter are used to improve representation invariance.

  • Experiment with ResNet-18: The model is trained on 256×256 images for 65k steps using momentum SGD with learning rate 0.03, momentum 0.9, and batch size 64 or 96 for 3-way classification.
  • Experiment with ResNet-18: A cosine learning-rate decay cycle and L2 weight regularization with coefficient 0.00003 are applied during training.
  • Experiment with ResNet-18: Random translation and color jitters are used as data augmentation to enhance representation invariance.

A.2. Implementation details on CutPaste

CutPaste samples patches with varied area ratios and aspect ratios, then randomly cuts and pastes them while keeping the entire patch inside the image. CutPaste-Scar uses directly sampled narrow widths.

  • Implementation details on CutPaste: CutPaste samples the patch-to-image area ratio from (0.02, 0.15).
  • Implementation details on CutPaste: Patch aspect ratios are sampled from (0.3, 1) ∪ (1, 3.3), while cut and paste locations are selected randomly within the full image.The sampling ensures the entire pasted patch appears inside the image.
  • Implementation details on CutPaste: CutPaste-Scar directly samples patch widths between.

A.3. Ablation Study on Hyperparameters

The ablation study examines optimization and CutPaste augmentation hyperparameters, finding robustness across many settings but greater sensitivity to jitter for texture categories.

  • Optimization hyperparameters: The method is fairly robust across learning rates and numbers of epochs, although learning rates ≤0.003 lead to slow convergence.The authors suggest training longer when using very small learning rates.
  • CutPaste hyperparameters: Jitter intensity matters more for texture categories because it makes pasted patches easier to distinguish from repetitive surrounding patterns.Without jitter, CutPaste can be too difficult to distinguish from original texture images; jitter increases the contrast between the patch and its surroundings.

A.4. Experiment with EfficientNet

The EfficientNet experiment uses staged fine-tuning and describes patch-based localization through dense feature extraction, location-specific scoring, and upsampling, with visualizations spanning object and texture categories.

  • EfficientNet fine-tuning: EfficientNet B4 is fine-tuned in two stages: first the MLP head, then all layers while keeping batch-normalization layers frozen.The head is trained for 10 epochs at learning rate 0.03, followed by 64 epochs of full-layer fine-tuning at learning rate 0.0001.
  • Patch-based localization: Patch-based models densely extract 32×32 patch representations with stride 4 and produce a 57×57 anomaly score map.Each location has a 512-dimensional embedding vector whose anomaly score is computed before upsampling.
  • Patch-based localization: The patch-based score map is upsampled to full 256×256 resolution using receptive-field upsampling via Gaussian smoothing.The procedure uses a transposed convolution with stride 4, matching the dense feature-extraction stride.
  • Category-specific scoring: For aligned-object categories, separate one-class classifiers at each location help detect missing or dislocated components.The aligned categories listed are bottle, cable, capsule, metal nut, pill, toothbrush, transistor, and zipper.
  • Localization visualizations: Figures 7–21 visualize 24 localization examples for each of 10 object and 5 texture categories using GradCAM and patch heatmaps.The visualizations include both successful and failure cases.

B.1. Failure Case Analysis

The failure-case analysis presents representative localization errors across several MVTec categories, while the accompanying figures compare inputs, ground-truth masks, GradCAM, and patch heatmaps.

  • Scope of failure analysis: The listed failures are representative rather than a comprehensive enumeration of all localization failures.The cases were identified through visual inspection of localization visualizations.
  • Object categories: Cable localization includes a missing-components failure in row 2, columns 6–8.Figure 8 presents cable localization examples with inputs, ground-truth masks, GradCAM results, and patch-based heatmaps.
  • Object categories: Metal nut localization includes flipped components in row 1, columns 1–2.Figure 11 presents the corresponding comparison between ground truth, GradCAM, and patch heatmaps.
  • Object categories: Screw localization includes background speckle noise in row 3, columns 1 and 6.Figure 13 shows the screw-class localization comparison.
  • Object categories: Transistor localization includes dislocated or missing components in row 2.Figure 15 shows transistor inputs, red ground-truth masks, GradCAM outputs, and patch heatmaps.
  • Texture categories: Tile localization includes dyed tiles in row 2, columns 7–8, and row 3, columns 1–2.Figure 20 shows tile-class localization examples using the same four-row comparison structure.
Loading 2104.04015v1…