Source-linked AI summary
Towards Total Recall in Industrial Anomaly Detection
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, Peter Gehler
TL;DR
Cold-start industrial anomaly detection must identify and localize defects from nominal examples alone, despite limited nominal context and varied industrial defects. PatchCore addresses this with locally aware mid-level patch-feature memory banks and coreset subsampling. It delivers state-of-the-art detection and localization with low computational cost, including over 99% MVTec image AUROC and high sample efficiency.
Problem
Cold-start industrial anomaly detection requires detecting and segmenting defects using only nominal examples, while pretrained high-level features provide limited nominal context and may not transfer across domains.
Method
PatchCore stores locally aware mid-level nominal patch features in a memory bank and applies greedy coreset subsampling to preserve context while reducing redundancy and runtime.
Results
PatchCore achieves state-of-the-art cold-start detection and localization with low computational cost, including over 99% image anomaly detection AUROC on MVTec and high sample efficiency.
Takeaways & Limitations
PatchCore is presented as a practical industrial anomaly detection system that retains nominal context, runs quickly, and performs competitively with only a fraction of nominal training data.
Takeaways & Limitations
Applicability is limited by the transferability of the pretrained features, motivating future integration with feature adaptation.
Abstract
from arXiv · showhide
Being able to spot defective parts is a critical component in large-scale industrial manufacturing. A particular challenge that we address in this work is the cold-start problem: fit a model using nominal (non-defective) example images only. While handcrafted solutions per class are possible, the goal is to build systems that work well simultaneously on many different tasks automatically. The best performing approaches combine embeddings from ImageNet models with an outlier detection model. In this paper, we extend on this line of work and propose \textbf{PatchCore}, which uses a maximally representative memory bank of nominal patch-features. PatchCore offers competitive inference times while achieving state-of-the-art performance for both detection and localization. On the challenging, widely used MVTec AD benchmark PatchCore achieves an image-level anomaly detection AUROC score of up to $99.6\%$, more than halving the error compared to the next best competitor. We further report competitive results on two additional datasets and also find competitive results in the few samples regime.\freefootnote{$^*$ Work done during a research internship at Amazon AWS.} Code: github.com/amazon-research/patchcore-inspection.
1. Introduction
The paper addresses cold-start industrial anomaly detection using only nominal images, where defects range from subtle scratches to major structural changes. PatchCore combines locally aware patch features, representative memory banks, and coreset subsampling to achieve strong detection and localization with practical inference costs.
- Motivation: Cold-start industrial anomaly detection must distinguish nominal images from out-of-distribution defects using only normal examples.The setting matters because normal imagery is easier to acquire than a complete specification of possible defect variations.
- Motivation: Industrial defects span thin scratches and larger structural failures such as missing components.
- Existing approaches: Prior methods use autoencoding, GANs, unsupervised adaptation, or ImageNet features to model nominal data and match test features against it.Multiscale features support both fine-grained defect segmentation and higher-level structural anomaly detection.
- PatchCore: PatchCore uses locally aggregated mid-level patch features to maximize nominal context, reduce ImageNet bias, and preserve spatial information.Its memory bank is built from nominal patch representations with local-neighbourhood aggregation.
- PatchCore: Greedy coreset subsampling reduces redundancy, storage requirements, and inference time in the patch-feature memory bank.
- Results: 99.6% AUROC is achieved on MVTec AD, with more than half the prior detection error removed and state-of-the-art localization across MVTec AD and MTD.The reported results retain fast inference, require no training on the target dataset, and remain competitive when using only a fraction of nominal training data.
2. Related Works
Related work primarily learns nominal representations and then detects anomalies through reconstruction, nearest-neighbour distances, or one-class classification. Industrial methods increasingly reuse pretrained ImageNet features, while PatchCore is positioned relative to memory-bank, patch-level, and coreset approaches.
- Learned nominal representations: Nominal representations are learned using autoencoders, Gaussian mixtures, GANs, physical-augmentation invariance, structural objectives, and related methods.
- Anomaly scoring: Given nominal and test representations, anomaly detection can use reconstruction errors, k-nearest-neighbour distances, or one-class models such as OC-SVM and SVDD.These approaches commonly support localization through pixel-wise reconstruction or feature errors.
- Industrial anomaly detection: Industrial anomaly detection increasingly uses ImageNet-pretrained models without adapting them to the target data.
- Closest methods: SPADE and PaDiM are the closest components to PatchCore because they use pretrained-backbone memory banks or patch-level representations for anomaly detection.
- Closest methods: PatchCore adapts greedy coreset selection to approximate memory-bank feature-space coverage while pursuing sensitive patch-level detection and segmentation.
3. Method
PatchCore builds a searchable memory bank of locally aware, mid-level nominal patch features, then reduces redundancy with greedy coreset subsampling. Test-time nearest-neighbor patch comparisons support both image-level anomaly detection and pixel-level localization while retaining spatial resolution and inference efficiency.
- Locally aware patch features: PatchCore uses intermediate or mid-level feature representations to preserve localized nominal information and reduce bias toward ImageNet classification.The method targets feature representations that retain industrially relevant context without relying on highly abstract final-layer features.
- Locally aware patch features: Local neighbourhood aggregation combines feature vectors around each position into one representation, increasing receptive-field context and robustness to small spatial deviations.PatchCore uses adaptive average pooling while retaining the feature-map resolution.
- Locally aware patch features: PatchCore aggregates two intermediate feature hierarchies after bilinear rescaling so their patch collections share matching spatial dimensions.Using two hierarchies provides some empirical benefit while retaining generality and spatial resolution.
- Patch-feature memory bank: The memory bank contains locally aware patch features from all nominal training images and can become excessively large as the nominal dataset grows.The memory bank is constructed from the patch collections of nominal samples.
- Coreset-reduced patch-feature memory bank: Greedy coreset subsampling reduces the memory bank while approximately preserving feature-space coverage, unlike random subsampling that can lose nominal information.PatchCore uses minimax facility location and an iterative greedy approximation, with random projections reducing coreset-selection cost.
- Anomaly detection with PatchCore: At test time, PatchCore computes the maximum nearest-neighbor distance between test patch-features and the nominal memory bank as an image-level anomaly score.The method uses patch-based comparisons for anomaly detection and segmentation, with a scaling factor applied to account for the behavior of neighboring nominal patches.
4. Experiments
Experiments evaluate PatchCore on industrial anomaly detection benchmarks, ablations, inference efficiency, higher-resolution settings, and limited nominal training data. Results show strong detection and localization performance, with coreset subsampling preserving accuracy while reducing memory and inference costs.
- Experimental setup: MVTec AD contains 15 sub-datasets with 5,354 images, including nominal-only training data and anomalous test samples with ground-truth masks.Image-level performance uses class-average AUROC; segmentation uses pixel-wise AUROC and PRO.
- MVTec AD performance: 57% error reduction lowers PaDiM’s 2.1% error to 0.9% for PatchCore-25% on MVTec AD.PatchCore also reports 98.1 versus 97.5 pixel-wise AUROC and 93.5 versus 92.1 PRO against PaDiM.
- Higher-resolution configurations: Higher-resolution or larger-backbone PatchCore-1% configurations retain inference times below PatchCore-10% at default resolution while further reducing image-level AUROC error from 1% to 0.4%.Table 4 evaluates higher resolution, larger backbones, and ensembles enabled by coreset subsampling.
- Inference time: Coreset subsampling makes PatchCore faster than PaDiM while retaining state-of-the-art image-level detection and segmentation performance.Without subsampling, PatchCore-100% is faster than SPADE and performs better; approximate nearest-neighbor search reduces performance but still outperforms other methods.
- Locally aware patch-features and hierarchies: A neighbourhood size of p = 3 provides the clearest locality–global-context trade-off, while combining hierarchy levels 2+3 is selected as the default.Moving deeper in the hierarchy adds global context but reduces resolution and increases ImageNet class bias.
- Coreset subsampling: Coreset selection outperforms random subsampling and learned proxies, while a two-orders-smaller coreset performs comparably to using the full memory bank.Subsampling to 1% raises the fraction of memory-bank samples used at test time from under 30% to nearly 95%; increased striding degrades performance.
- Additional benchmarks: PatchCore matches or slightly outperforms the reported comparison on MTD image-level detection and achieves state-of-the-art segmentation on the mSTC benchmark.The additional benchmark results extend evaluation beyond MVTec AD.
5. Conclusion
PatchCore is a cold-start anomaly detection algorithm that uses nominal patch-level features while balancing nominal context and runtime. It achieves state-of-the-art detection and localization on industrial benchmarks, including over 99% image AUROC on MVTec.
- PatchCore detects and segments anomalous data at test time using only nominal examples.
- PatchCore balances extensive nominal context from locally aware patch-level memory banks with low runtime through coreset subsampling.
- PatchCore achieves state-of-the-art cold-start image anomaly detection and localization with low computational cost on industrial benchmarks.
- On MVTec, PatchCore achieves image anomaly detection AUROC above 99% and the highest sample efficiency in relevant small-training-set regimes.
A. Implementation Details
The implementation uses ImageNet-pretrained vision models and a WideResNet50 backbone for direct comparison with prior methods. Patch-level features come from aggregated outputs of network blocks 2 and 3, with FAISS used for retrieval and distance computation.
- PatchCore uses a WideResNet50 backbone by default to enable direct comparability with SPADE and PaDiM.
- Patch-level features are extracted from feature-map aggregation of the final outputs in blocks 2 and 3.
- The models use torchvision and PyTorch Image Models implementations of ImageNet-pretrained networks.
- FAISS performs nearest-neighbor retrieval and distance computations.
B. Full MVTec AD comparison
Across detailed MVTec AD comparisons, PatchCore performs strongly across datasets and working points, with coreset variants achieving leading results. Larger-image experiments also examine efficient inference with higher-resolution inputs.
- PatchCore−25% solves six of the 15 MVTec datasets and achieves the highest AUROC on most datasets and on average.
- PatchCore maintains consistently low classification errors at both 100% recall and an F1-optimal threshold.
- PatchCore shows near-optimal Precision-Recall and ROC curves across datasets, unlike the compared SPADE and PaDiM implementations.
- PatchCore−1% supports efficient anomaly detection at inference time with 280 × 280 images and a WideResNet-101 backbone.
C.1. Detailed Low-Shot experiments
Low-shot experiments report consistently higher detection and localization metrics, while backbone choice is mostly stable. Increasing image size slightly improves detection and consistently improves anomaly segmentation.
- Low-shot experiments find consistently higher numbers for detection and anomaly localization metrics.
- Results are mostly stable across different backbone choices, with WideResNet50 selected for comparability with SPADE and PaDiM.
- Larger image sizes produce slightly higher detection performance, which saturates for PatchCore.
- Anomaly segmentation improves consistently with larger image sizes, making resolution a factor to validate when localization matters.
C.4. Remaining Misclassifications
At the F1-optimal threshold, PatchCore retains 19 false-positive and 23 false-negative errors. These errors mainly reflect ambiguous labels, high nominal variance, insufficient anomaly weighting, fine-grained anomalies, and one preprocessing crop.
- Error overview: 19 false-positive and 23 false-negative errors remain at the F1-optimal threshold.The errors are visualized in Figures S1 and S2, with segmentation maps normalized to the threshold value.
- False positives: Most false positives arise from ambiguous labelling or high nominal variance resembling potential anomalies.The figure caption also identifies background variance as another false-positive source.
- False positives: Some nominal-variance false positives could be addressed through adaptation to nominal data, potentially alongside PatchCore.The text notes that labelling ambiguity is difficult for proposed methods to address and that PatchCore outperforms adaptive methods.
- False negatives: Many false negatives localize anomalies but assign them insufficient weight, while others involve fine-grained anomalies, nominal variance, or preprocessing that crops out the anomaly.The figure caption distinguishes weakly weighted, unrecovered, cropping-related, and fine-grained missed anomalies.