Source-linked AI summary
Sub-Image Anomaly Detection with Deep Pyramid Correspondences
Niv Cohen, Yedid Hoshen
TL;DR
Visual kNN anomaly detection performs strongly at the image level but lacks segmentation of the anomalous region. SPADE aligns an anomalous image with similar normal images using pretrained multi-resolution feature-pyramid correspondences, achieving stronger results than current methods on two realistic datasets with simple deployment.
Problem
Image-level kNN anomaly detection lacks a segmentation map showing where anomalies lie, while visual anomaly detection commonly has only normal training data.
Method
SPADE uses pretrained deep features, retrieves K nearest normal images, and scores pixels through dense correspondences across a multi-resolution feature pyramid.
Results
SPADE outperformed the strongest current methods on two realistic sub-image anomaly-detection datasets for image-level and pixel-level detection.
Takeaways & Limitations
The method offers a simple, easy-to-deploy approach for detecting and segmenting anomalies inside images.
Takeaways & Limitations
Runtime performance is significantly reliant on kNN, whose complexity scales linearly with the dataset size.
Abstract
from arXiv · showhide
Nearest neighbor (kNN) methods utilizing deep pre-trained features exhibit very strong anomaly detection performance when applied to entire images. A limitation of kNN methods is the lack of segmentation map describing where the anomaly lies inside the image. In this work we present a novel anomaly segmentation approach based on alignment between an anomalous image and a constant number of the similar normal images. Our method, Semantic Pyramid Anomaly Detection (SPADE) uses correspondences based on a multi-resolution feature pyramid. SPADE is shown to achieve state-of-the-art performance on unsupervised anomaly detection and localization while requiring virtually no training time.
1 Introduction
Visual anomaly detection matters for identifying novel events and faulty products, but anomalies are unexpected and often only normal data are available for training. The paper addresses both detection and pixel-level localization with a fast, training-light method.
- Anomaly detection is important because computers must identify images that deviate from previously observed patterns.
- Assembly-line inspection illustrates the value of promptly detecting occasional faults such as dents, wrong labels, or duplicated parts.
- Normal-only training is practical because normal data are usually easy to obtain, whereas anomaly distributions are not observed during training.
- Pixel-level localization is a distinct visual challenge because segmenting anomalous regions supports explainability, operator trust, and informed product decisions.
- The proposed method retrieves K normal neighbors, finds dense pixel correspondences, and labels unmatched target regions as anomalous without an extended training stage.
2 Previous Work
Previous anomaly-detection work includes reconstruction-, distribution-, and classification-based image-level methods, while sub-image segmentation has received less attention. Existing sub-image approaches use patch features, dimensionality reduction, adversarial models, or autoencoders.
- Image-level anomaly methods classify whole images and are commonly grouped as reconstruction-based, distribution-based, or classification-based.
- Reconstruction-based methods mark samples anomalous when basis functions cannot faithfully reconstruct them, using choices such as K-means, kNN, or PCA.
- Distribution-based methods model the normal-data probability density and flag test samples with low density, using parametric or non-parametric estimators.
- Classification-based work includes one-class SVM, SVDD, and self-supervised transformation-recognition approaches for image-level anomaly detection.
- Sub-image segmentation has received less attention and includes patch-level deep features with K-means, ADGAN, and autoencoder approaches.
3 Correspondence-based Sub-Image Anomaly Detection
The method extracts pretrained image and pixel-level features, retrieves similar normal neighbors, and scores target pixels by correspondence to those neighbors. A multi-resolution feature pyramid supplies both local detail and broader context for alignment-based segmentation.
- Method overview: The pipeline combines feature extraction, K-nearest-neighbor retrieval of normal images, and pixel alignment using deep feature-pyramid correspondences.
- Feature Extraction: ImageNet-pretrained ResNet features provide the global representations used for image-level retrieval and later pixel-level alignment.
- K Nearest Neighbor Normal Image Retrieval: For each test image, the method retrieves K nearest normal training images using Euclidean distance between global feature representations.
- Sub-image Anomaly Detection via Image Alignment: A multi-image gallery averages distances to κ nearest pixel features, reducing dependence on alignment to one normal image.
- Sub-image Anomaly Detection via Image Alignment: Pixels are declared anomalous when their feature distance exceeds threshold θ because no closely corresponding pixel is found among retrieved normal images.
- Feature Pyramid Matching: Concatenating features from multiple pyramid levels combines fine-grained local information with global context for correspondences to K ≥1 normal images.
- Implementation Details: The experiments use a Wide-ResNet50 × 2 pretrained on ImageNet, specified input resizing, and Gaussian smoothing with σ = 4.
- Implementation Details: Figure 1 illustrates retrieved normal neighbors, detected masks, and predicted anomalous pixels for insect, color, and spot anomalies.
4 Experiments
The experiments evaluate SPADE for image-level and pixel-level anomaly detection on MVTec and Shanghai Tech Campus, including qualitative examples and ablations. SPADE uses nearest normal neighbors and pyramid-based correspondences to localize anomalous regions, achieving strong reported performance without anomaly supervision.
- Evaluation setup: SPADE is evaluated extensively against state-of-the-art and established baselines for sub-image anomaly detection.The evaluation covers MVTec and Shanghai Tech Campus, with image-level and pixel-level metrics.
- Qualitative localization: The method identifies anomalous regions by matching a target image with retrieved normal neighbors and detecting regions without close correspondences.In the MVTec hazelnut example, the scratch lacks correspondence with the complete normal nut and is localized accurately.
- Metrics: ROCAUC measures pixel-level anomaly ranking, while PRO measures average connected-region coverage across false-positive rates from 0 to 0.3.PRO is introduced to reduce ROCAUC’s bias toward large anomalies.
- MVTec results: SPADE significantly outperforms prior methods on MVTec per-pixel ROCAUC and PRO, including the reported ensemble approach on PRO.The comparisons include autoencoder-based methods, CAVGA-Ru, and baselines from prior work.
- Shanghai Tech Campus results: On Shanghai Tech Campus, SPADE’s image-level performance is comparable to the top-performing method, while its pixel-level ROCAUC significantly exceeds the best reported unsupervised method.The comparison excludes methods using anomaly supervision because SPADE does not use anomalous training data.
- Ablation study: Using all feature-pyramid levels performs best, whereas very high-resolution activations hurt performance and higher levels alone lose performance through lower resolution.Selecting top-k nearest normal images also improves performance, especially for the variable Grid category.
5 Discussion
The method avoids feature training by aligning test images with retrieved normal images, while using feature pyramids and kNN in ways that balance localization quality with runtime considerations.
- Anomaly detection via alignment: Alignment-based anomaly detection avoids the large parametric autoencoding functions used by many existing sub-image methods.The approach requires no feature training and can operate with very small datasets.
- The role of retrieval: Retrieving the nearest normal images before alignment is evaluated against random selection, with stage 1 being more important for some classes than others.The comparison uses 10 nearest neighbours selected by stage 1 or randomly selected images.
- The role of context for anomaly detection: Deep feature pyramids combine shallow-layer local context for high-resolution segmentation with deeper features that capture broader context.Alignment quality depends strongly on the extracted features and on context at multiple scales.
- Optimizing runtime performance: kNN retrieval is central to the method, but its complexity scales linearly with the search dataset and can become problematic for large or high-dimensional data.The method mitigates this through 2048-dimensional global-pooled features and speedup techniques such as KDTrees.
- Pre-trained vs. learned features: Pre-trained features outperform self-learned or combined feature approaches in the authors’ numerical results for normal-only training on small datasets.The authors attribute this setting’s difficulty to limited supervision and dataset size, while identifying feature finetuning as future work.
6 Conclusion
The paper presents an alignment-based method for detecting and segmenting image anomalies using nearest neighbors of pixel-level feature pyramids from pre-trained deep features. It outperforms the strongest current methods on two realistic datasets while remaining simpler and easy to deploy.
- 6 Conclusion: The method detects and segments image anomalies by aligning pixel-level feature pyramids from K nearest normal neighbors.The feature pyramids are extracted using pre-trained deep features.
- 6 Conclusion: It outperforms the strongest current methods on two realistic sub-image anomaly detection datasets while being much simpler.
- 6 Conclusion: The method’s ease of deployment makes it a candidate for practitioners.