Source-linked AI summary
Patch SVDD: Patch-level SVDD for Anomaly Detection and Segmentation
Jihun Yi, Sungroh Yoon
TL;DR
The paper addresses image anomaly detection and pixel-level segmentation, extending Deep SVDD to patch-wise inspection with self-supervised learning. Patch SVDD enables localization and improves detection, achieving state-of-the-art performance on MVTec AD, while high patch variation limits direct single-center extensions.
Problem
Image anomaly detection and segmentation require deciding whether an image is anomalous and localizing anomalies at the pixel level, often with insufficient abnormal training examples.
Method
Patch SVDD extends Deep SVDD to patch-wise inspection and adds self-supervised learning to handle high intra-class patch variation.
Results
Patch SVDD achieves state-of-the-art MVTec AD performance, including 0.957 segmentation AUROC and a 55.6% improvement over Deep SVDD.
Takeaways & Limitations
Patch-wise inspection enables defect localization, while self-supervised learning improves detection performance on the MVTec AD industrial anomaly dataset.
Takeaways & Limitations
A direct single-center patch-wise Deep SVDD extension performs poorly on highly complex images because patches have high intra-class variation.
Abstract
from arXiv · showhide
In this paper, we address the problem of image anomaly detection and segmentation. Anomaly detection involves making a binary decision as to whether an input image contains an anomaly, and anomaly segmentation aims to locate the anomaly on the pixel level. Support vector data description (SVDD) is a long-standing algorithm used for an anomaly detection, and we extend its deep learning variant to the patch-based method using self-supervised learning. This extension enables anomaly segmentation and improves detection performance. As a result, anomaly detection and segmentation performances measured in AUROC on MVTec AD dataset increased by 9.8% and 7.0%, respectively, compared to the previous state-of-the-art methods. Our results indicate the efficacy of the proposed method and its potential for industrial application. Detailed analysis of the proposed method offers insights regarding its behavior, and the code is available online.
1 Introduction
Image anomaly detection decides whether an image contains an anomaly, while anomaly segmentation localizes anomalies at the pixel level. Patch SVDD extends Deep SVDD to patch-wise inspection using self-supervised learning, enabling segmentation and improving detection performance.
- Image anomaly detection is a binary decision problem, while anomaly segmentation localizes anomalies at the pixel level.
- SVDD is a classic one-class classification method that encloses normal data within a hypersphere in kernel space.
- Patch SVDD extends Deep SVDD to patch-wise detection, addressing patch variation with self-supervised learning.The extension enables anomaly segmentation and improves anomaly detection performance.
- Patch-wise inspection provides position-specific results, allowing defect localization and finer-grained detection.
2 Background
Anomaly detection scores whether inputs are abnormal, while segmentation assigns anomaly scores to pixels. Background approaches include reconstruction, discriminative, SVDD-based, and self-supervised representation-learning methods.
- Anomaly detection and segmentation: Anomaly detection assigns scores to inputs and uses AUROC to evaluate whether normal and abnormal examples are distinguished.
- Anomaly detection and segmentation: Anomaly segmentation generates an anomaly map by assigning an anomaly score to every pixel.
- Autoencoder-based methods: Autoencoder methods use reconstruction differences between inputs and reconstructed images to indicate abnormality.Variants incorporate structural similarity, adversarial training, negative mining, and iterative projection.
- Classifier-based methods: Discriminative methods train classifiers on synthetic transformations and flag inputs when the classifier lacks confidence or predicts incorrectly.
- SVDD-based methods: SVDD learns the smallest hypersphere enclosing normal training data, with anomalies expected outside the hypersphere.
- SVDD-based methods: Deep SVDD replaces the kernel function with a neural network that learns a data-dependent transformation for high-dimensional structured data.Bias terms are removed to avoid the trivial constant-output solution.
- Self-supervised representation learning: Self-supervised representation learning trains encoders with pretext tasks that provide synthetic learning signals without annotation.Examples include relative patch-location prediction, jigsaw solving, colorization, object counting, and rotation prediction.
3 Methods
Patch SVDD extends Deep SVDD from whole images to patches, combining nearest-normal-patch scoring with self-supervised and hierarchical encoding to support localization and improve detection.
- Patch-wise Deep SVDD: Deep SVDD trains an encoder to place normal data near a single feature-space center, using representation distance as the anomaly score.The center is calculated before training, and test-time distance to it measures abnormality.
- Patch-wise Deep SVDD: Patch-wise inspection encodes each patch separately, enabling position-level defect localization and finer-grained image examination.Patch scores are later distributed to pixels to form an anomaly map.
- Patch-wise Deep SVDD: Directly mapping diverse patches to one center performs poorly on complex images because background and object patches have high intra-class variation.The single-center assumption weakens the connection between feature representation and patch content, while multiple-center allocation is cumbersome.
- Self-supervised learning: Self-supervised learning gathers semantically similar patches by minimizing distances between adjacent-patch features and trains an encoder for relative-position prediction.The relative-position classifier is discarded after training, while the two encoders share weights.
- Hierarchical encoding: Hierarchical encoding addresses anomaly-size variation by dividing each patch into a 2 × 2 grid, independently encoding sub-patches, and aggregating their features.The large and small encoders use receptive-field sizes of 64 and 32, respectively.
- Generating anomaly maps: At test time, Patch SVDD assigns each patch the L2 distance to its nearest normal training patch, distributes scores to pixels, and multiplies maps from multiple encoders.The maximum pixel anomaly score provides the image-level detection score, while high values in the combined map indicate defects.
4 Results and Discussion
Patch SVDD achieves strong anomaly detection and segmentation results on MVTec AD, with patch-level representations, self-supervised losses, and hierarchical multi-scale inspection supporting performance and localization.
- Anomaly detection and segmentation results: 0.957 AUROC is achieved by Patch SVDD for anomaly segmentation on MVTec AD, exceeding the listed autoencoder and VAE baselines.The compared methods range from 0.804 to 0.893 AUROC.
- Anomaly detection and segmentation results: Patch SVDD provides state-of-the-art detection and segmentation performance and improves over Deep SVDD by 55.6%.The method also localizes defects regardless of their size.
- Effect of self-supervised learning: Modifying LSVDD to LSVDD’ and adding LSSL both improve anomaly detection and segmentation performance.The effects vary across classes: texture classes are less sensitive, while object classes such as cable and transistor benefit significantly from LSSL.
- Effect of self-supervised learning: LSSL produces semantically organized multi-modal clusters for object-class patches, separating dissimilar semantics and making anomaly inspection more deliberate and accurate.For transistor features, LSVDD and LSVDD’ produce uni-modal clusters, whereas LSSL produces semantic multi-modal clusters; the multi-modal structure is especially useful under high intra-class variation.
- Hierarchical encoding and hyperparameters: Hierarchical encoding and aggregated multi-scale inspection improve inspection performance, while larger embedding dimension D improves results and the average intrinsic dimension saturates at D = 64.The study therefore uses D = 64 throughout.
- Random encoder: Random encoders can distinguish anomalies for certain classes, with some results outperforming the trained L2-AE model; raw-patch nearest neighbors show a similar pattern.The paper relates this behavior to the strong relationship between random features and raw image patches.
5 Conclusion
Patch SVDD performs anomaly inspection at the patch level, enabling defect localization while self-supervised learning improves detection. The method achieves state-of-the-art performance on MVTec AD, and raw patches or random features can provide useful anomaly signals.
- Patch SVDD inspects images at patch level, enabling anomaly segmentation in addition to image-level detection.Unlike Deep SVDD, patch-wise inspection supports defect localization.
- Self-supervised learning improves Patch SVDD’s anomaly detection performance.
- The proposed method achieves state-of-the-art performance on the MVTec AD industrial anomaly detection dataset.
- A nearest-neighbor algorithm using raw patches can discriminate anomalies surprisingly well despite prior feature-extraction practice.
- Random features can provide distinguishable anomaly signals because distances in random feature space closely relate to raw-image-space distances.
A1 Pseudo code
Patch SVDD trains a hierarchical encoder with patch-level and self-supervised objectives, extracts normal patch features, and scores query patches by nearest-neighbor distances to produce anomaly maps and image scores.
- Patch SVDD training takes normal images, λ, an encoder, and a classifier as inputs.
- The encoder minimizes distances between randomly jittered versions of each patch while a classifier predicts the relative position of a neighboring patch.
- The training objective combines the patch SVDD loss and self-supervised cross-entropy loss as λLSVDD’ + LSSL.
- After training, normal features are collected separately from the trained multi-scale encoders.
- At test time, nearest-neighbor distances assign anomaly scores to patches, distribute them across pixels, and multiply multi-scale maps.
A2 Results
The appendix reports Patch SVDD’s per-class and average detection and segmentation AUROC results on MVTec AD, alongside analyses of hierarchical and multi-scale encoding. Example anomaly maps compare heatmap scores with ground-truth defect contours.
- Table A1 reports per-class and average detection and segmentation performance in AUROC on MVTec AD.
- Table A2 analyzes the effect of hierarchical encoding and reports that multi-scale aggregation boosts performance.
- Table A2 also reports that adopting a hierarchical encoder structure is helpful for performance.
- The anomaly-map figures show class names, defect types, ground-truth defects as red contours, and darker heatmaps for higher anomaly scores.
A3 Implementation details
The implementation evaluates Patch SVDD on the 15-class MVTec AD dataset using convolutional encoders, a two-layer MLP classifier, hierarchical patch scales, and specified hardware and software.
- MVTec AD contains 15 industrial image classes, with object and texture categories, normal training images, and pixel-annotated abnormal test images.
- All images are downsampled to 256 × 256, and grayscale images are converted to RGB by channel replication.
- The encoder uses convolutional layers, while the classifier is a two-layer MLP with 128 hidden units per layer.
- The hierarchical encoder uses receptive fields K = 64 and K = 32, with strides S = 16 and S = 4, respectively.
- Experiments use an Intel i7-5930K CPU, an NVIDIA GeForce RTX 2080 Ti GPU, Python 3.7, and PyTorch.