PaperlayerView exact revision on arXiv

Source-linked AI summary

SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation

Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, Onkar Dabeer

arXiv:2207.14315v1cs.CV

TL;DR

Industrial anomaly detection must find rare, often small defects across varied manufacturing settings, yet universal self-supervised pre-training for this task remains underdeveloped. The paper introduces the VisA dataset and SPD, which regularizes pre-training for local anomaly sensitivity; experiments show consistent improvements across anomaly detection and segmentation settings.

  • Problem

    Industrial inspection involves rare, small defects across diverse objects and tasks, while universal self-supervised representations for anomaly detection and segmentation remain underdeveloped.

  • Method

    The paper releases VisA and proposes SPD, which uses SmoothBlend local perturbations and weak global augmentations to regularize self-supervised and supervised ImageNet pre-training.

  • Results

    SPD consistently improves contrastive self-supervised baselines and supervised pre-training for anomaly detection and segmentation across VisA and MVTec-AD.

  • Takeaways & Limitations

    SPD improves local sensitivity in pre-trained representations and provides benefits across 1-class, 2-class, high-shot, and low-shot anomaly settings.

Abstract

from arXiv · show

Visual anomaly detection is commonly used in industrial quality inspection. In this paper, we present a new dataset as well as a new self-supervised learning method for ImageNet pre-training to improve anomaly detection and segmentation in 1-class and 2-class 5/10/high-shot training setups. We release the Visual Anomaly (VisA) Dataset consisting of 10,821 high-resolution color images (9,621 normal and 1,200 anomalous samples) covering 12 objects in 3 domains, making it the largest industrial anomaly detection dataset to date. Both image and pixel-level labels are provided. We also propose a new self-supervised framework - SPot-the-difference (SPD) - which can regularize contrastive self-supervised pre-training, such as SimSiam, MoCo and SimCLR, to be more suitable for anomaly detection tasks. Our experiments on VisA and MVTec-AD dataset show that SPD consistently improves these contrastive pre-training baselines and even the supervised pre-training. For example, SPD improves Area Under the Precision-Recall curve (AU-PR) for anomaly segmentation by 5.9% and 6.8% over SimSiam and supervised pre-training respectively in the 2-class high-shot regime. We open-source the project at http://github.com/amazon-research/spot-diff .

1 Introduction

Industrial anomaly inspection requires accurate detection and localization despite rare, small defects and diverse manufacturing settings. The paper addresses these challenges with the VisA dataset and SPD training for locally sensitive pre-trained representations.

  • Motivation: Industrial defects are rare, often small, and span diverse objects, domains, and inspection tasks.Manufacturing inspection also demands highly accurate models.
  • Motivation: Existing surface-anomaly models typically target individual objects, while universal self-supervised representations for surface anomaly detection remain underdeveloped.Prior CutPaste representations are learned per object and may generalize poorly across objects.
  • Motivation: MVTec-AD is saturating near 95% AU-ROC and is limited to 1-class evaluation, motivating more challenging datasets and broader benchmarks.The paper introduces VisA with complex structures, multiple instances, 12 objects across 3 domains, and multiple anomaly classes.
  • Approach: Standard contrastive methods encourage invariance to globally transformed local details, which may be suboptimal for detecting low-level anomalies.The paper frames local sensitivity as a central requirement for anomaly detection.
  • Approach: SPD uses SmoothBlend-generated local perturbations as negatives and weak global augmentations to promote local anomaly sensitivity while tolerating slight imaging variations.This reverses the usual treatment of globally augmented views as positives in SimCLR, MoCo, and related methods.
  • Contributions: The VisA dataset is 2× larger than MVTec-AD, includes image and pixel-level annotations, and supports 1-class and 5/10/high-shot 2-class benchmarks.SPD regularizes self-supervised ImageNet pre-training and improves anomaly detection and segmentation over strong self-supervised and supervised baselines.

2 Related Works

Prior work includes unsupervised industrial anomaly methods, supervised ImageNet initialization, and self-supervised representation learning. However, large-scale self-supervised pre-training remains unexplored for quality inspection.

  • Unsupervised Anomaly Detection: Unsupervised anomaly detection and segmentation commonly train using only normal samples.Methods such as SPADE, PatchCore, and PaDiM use patch features for anomaly scoring, while CutPaste learns from cut-and-pasted patches.
  • Anomaly Detection Methods: Existing industrial methods address low-level texture anomalies, whereas other approaches target high-level semantic anomalies with different challenges.The paper distinguishes texture-focused inspection from semantic anomaly detection.
  • Research Gap: Large-scale self-supervised pre-training had not been explored for quality-inspection applications before this work.This gap motivates adapting self-supervised representations to industrial anomaly detection and segmentation.
  • Self-Supervised Learning: Self-supervised methods including MoCo, SimCLR, and SimSiam have matched or exceeded supervised pre-training in several transfer-learning tasks.These methods learn representations through surrogate or multi-view objectives.

3 SPot-the-Difference (SPD) Regularization

SPD regularizes contrastive self-supervised learning by making representations invariant to weak global changes but sensitive to local deformations. It combines standard contrastive objectives with synthetic spot-the-difference examples.

  • 3.1 Background on Self-supervised Contrastive Learning: Standard contrastive learning maximizes similarity between augmented views of an image and minimizes similarity to other batch images.SimCLR and MoCo use strong global augmentations, while SimSiam can omit negatives.
  • 3.1 Background on Self-supervised Contrastive Learning: Strong global transformations can erase local details, motivating explicit local sensitivity for anomaly detection.The paper notes that views may retain global semantics while differing in local details.
  • 3.2 Augmentations for SPD: SmoothBlend creates local deformations by alpha-blending a blurred mask over a color-jittered patch pasted into the same image.The augmented image is computed as ¯x = (1 − α) ⊙ x + α ⊙ u.
  • 3.2 Augmentations for SPD: SPD uses locally deformed images as negatives instead of other batch images, while weak global augmentations model lighting, position, cropping, blur, flipping, and color variation.The design separates local anomaly-like changes from slight global manufacturing variations.
  • 3.2 Augmentations for SPD: SPD learning minimizes similarity to local negatives and maximizes similarity to weakly globally augmented positives.The SPD loss is cos(z_i, z̃_i^−) − cos(z_i, z̃_i^+).
  • SPD Training: For SimCLR, the combined objective adds the SPD loss to the standard InfoNCE loss, weighted by η.The same regularization is applied to MoCo, SimSiam, and supervised pre-training variants.

4 Visual Anomaly (VisA) Dataset

VisA is a high-resolution industrial anomaly dataset designed to provide challenging object-level and pixel-level evaluation across varied structures, instances, poses, and training regimes.

  • Dataset Description: VisA is twice the size of MVTec-AD, with 10,821 images compared with MVTec-AD’s 5,354.
  • Dataset Characteristics: VisA covers challenging cases involving complex object structures, multiple objects, and substantial variation in object location.
  • Evaluation Protocol: The evaluation protocols include 1-class training and 2-class high-shot and low-shot settings, including 5-shot and 10-shot benchmarks.High-shot uses 60%/40% normal and anomalous images for training/testing; low-shot samples k=5 or 10 images from both classes.
  • Evaluation Metrics: AU-PR is reported alongside AU-ROC because class imbalance can make AU-ROC appear inflated, especially for anomaly segmentation.The paper argues that AU-PR better reflects performance when anomalous samples or pixels are rare.

5 Experiments

Experiments evaluate SPD-regularized ImageNet pre-training across anomaly detection and segmentation tasks on VisA and MVTec-AD, showing broad improvements across training regimes and baselines.

  • Experimental Setup: Experiments use ImageNet 2012 for self-supervised and supervised pre-training, with VisA and MVTec-AD as downstream benchmarks.MVTec-AD is used as a 1-class benchmark, while VisA supports broader protocols.
  • 1-class Evaluation: SPD improves anomaly detection and segmentation across almost all pre-training baselines on both VisA and MVTec-AD in the 1-class setting.Self-supervised methods improve by up to 2.6% AU-PR, averaged over the 12 VisA objects.
  • High-shot Evaluation: 5.9% and 6.8% AU-PR gains are obtained for segmentation over SimSiam and supervised pre-training, respectively, in the VisA 2-class high-shot regime.AU-ROC saturates while AU-PR retains room for improvement, particularly for segmentation.
  • Low-shot Evaluation: 2.3% AU-PR is the maximum gain from SPD in 5-shot and 10-shot anomaly segmentation, while MoCo+SPD is the best few-shot segmentation method.MoCo+SPD even outperforms supervised pre-training in this setting.
  • Low-shot Evaluation: SPD improves SimSiam by 4.2% AU-PR in 5-shot detection and 2.6% in 10-shot detection, though both remain inferior to supervised pre-training.
  • Ablation Study: Ablations show SPD benefits multiple loss weights, outperforms CutPaste by 4.0% and 3.8% AU-PR in selected tasks, and generalizes across backbones and PatchCore.

6 Conclusions

The paper introduces VisA and SPD to improve anomaly detection and segmentation, with experiments supporting benefits across self-supervised and supervised pre-training settings.

  • Conclusions: SPD regularizes pre-trained representations toward local sensitivity to anomalous patterns.
  • Conclusions: VisA provides a large industrial anomaly detection dataset for evaluating these methods across varied setups.
  • Conclusions: SimSiam with SPD is superior or competitive in low-shot settings, while supervised learning with SPD performs better across various setups.

A Scatter Plots for AU-ROC

The AU-ROC scatter plots compare ImageNet pre-trained representations for classification and segmentation in 1-class VisA and MVTec-AD setups. SPD almost uniformly improves AU-ROC across the evaluated baselines and tasks.

  • The plots compare classification and segmentation AU-ROCs for various ImageNet pre-trained representations in the 1-class setup.
  • SPD almost improves AU-ROC for all baselines on both classification and segmentation tasks across VisA and MVTec-AD.The passage presents this pattern as evidence of SPD’s effectiveness.

B Further Discussion on AU-PR and AU-ROC

Because anomaly datasets are highly imbalanced, AU-PR can reveal positive-class performance that AU-ROC obscures. A toy comparison shows that a model with lower AU-ROC can achieve much better anomaly precision-recall performance.

  • AU-PR is more informative than AU-ROC when anomalies are rare and performance on the minor class matters.The paper notes that AU-ROC may provide an inflated view of minor-class performance in imbalanced datasets.
  • The toy test set contains 100 anomalous positives and 100,000 normal negatives, producing a negative-to-positive ratio of 1,000.The example uses threshold-dependent scores to construct ROC and PR curves for two models.
  • Model A achieves 99.5% AU-ROC but only 10.5% AU-PR and 18.2% Max F1.At its best threshold, Model A has 10% precision with 100% recall.
  • Model B has lower AU-ROC than Model A but reaches 90.6% AU-PR and 94.7% Max F1.Model B achieves 100% precision and 90% recall at its best operating point, compared with Model A’s 10% precision and 100% recall.
  • The comparison demonstrates that AU-ROC can provide an inflated and misleading view of positive-class performance.

C Implementation Details

The experiments use a fixed SPD loss weight and preserve each contrastive baseline’s default hyperparameters. Training schedules differ between high-shot and few-shot supervised setups.

  • The SPD loss weight is fixed at η = 0.1 unless otherwise specified.
  • SPD-augmented SimSiam, MoCo, and SimCLR experiments follow the corresponding baseline’s default hyperparameters.
  • Pre-training: SmoothBlend cuts patches covering 0.5%−1% of the full image, with aspect ratios ranging from 0.3 to 3.
  • High-shot two-class models are fine-tuned for 80 epochs with SGD and learnable backbone parameters.
  • Few-shot models use 1,000 iterations for classification and 500 iterations for segmentation with a fixed learning rate of 0.0001.

D Full results for each subset of VisA

The full VisA results are reported across object subsets and training regimes for several pre-training strategies. Difficulty varies by object structure: multiple-instance subsets are hardest, while single-instance subsets are easiest among the listed groups.

  • Results cover 1-class, two-class high-shot, two-class 5-shot, and two-class 10-shot classification and segmentation setups.Tables 7–14 report these regimes for SimSiam, SimSiam+SPD, supervised, and supervised+SPD pre-training.
  • Macaroni1, Macaroni2, Capsules, and Candles are the most difficult VisA subsets, with the lowest scores.
  • PCB1, PCB2, PCB3, and PCB4 are relatively easier than the multiple-instance cases.
  • Cashew, Chewing gum, Fryum, and Pipe Fryum are easier than the complex-structure cases because they contain single instances.

E Qualitative Results

Qualitative evaluations indicate that SPD improves anomaly localization and segmentation relative to its pre-training baselines. Attention maps and PaDiM outputs especially suggest greater sensitivity to defective regions and better segmentation quality.

  • Attention maps: SPD makes SimSiam attention maps more sensitive to defective regions than SimSiam alone.GradCAM uses negative cosine similarity to the nearest normal sample, with high-energy regions contributing most to feature distance.
  • Anomaly segmentation results: SPD produces better qualitative PaDiM segmentation results than both SimSiam and supervised pre-training baselines.The comparison includes SimSiam, SimSiam+SPD, supervised, and supervised+SPD ResNet-50 pre-training.
  • Attention maps: Figure 11 compares normal and anomalous images using GradCAM maps from SimSiam and SimSiam+SPD, highlighting defects and high-energy regions in red.The rows are organized as normal images, anomalous images, SimSiam attention maps, and SimSiam+SPD attention maps.
  • Anomaly segmentation results: Figure 12 presents PaDiM segmentation results across four pre-training settings: SimSiam, SimSiam+SPD, supervised, and supervised+SPD.The figure is used to compare qualitative segmentation outputs from these pre-trained ResNet-50 models.
Loading 2207.14315v1…