Source-linked AI summary
Deep Nearest Neighbor Anomaly Detection
Liron Bergman, Niv Cohen, Yedid Hoshen
TL;DR
Image anomaly detection needs methods that work with limited labels and difficult, variable anomalies. This paper evaluates kNN on ImageNet-pretrained image features and finds that the resulting DN2 method outperforms existing methods across accuracy, sample complexity, training time, robustness, and dataset type, while remaining simple and broadly applicable.
Problem
Image anomaly detection must identify difficult, previously unseen anomalies, while recent self-supervised methods may use weaker representations than ImageNet-pretrained features.
Method
DN2 embeds images with an ImageNet-pretrained ResNet, uses kNN distance to the training embeddings as an anomaly score, and extends the approach to contaminated data and image groups.
Results
DN2 outperformed state-of-the-art methods in accuracy, training time, robustness to input impurities and dataset type, and sample complexity.
Takeaways & Limitations
DN2 is presented as an excellent practitioner starting point and an important baseline for future anomaly-detection research.
Takeaways & Limitations
The approach depends on high-quality generic feature extractors, which do not exist for some important anomaly-detection domains such as tabular data and time series.
Abstract
from arXiv · showhide
Nearest neighbors is a successful and long-standing technique for anomaly detection. Significant progress has been recently achieved by self-supervised deep methods (e.g. RotNet). Self-supervised features however typically under-perform Imagenet pre-trained features. In this work, we investigate whether the recent progress can indeed outperform nearest-neighbor methods operating on an Imagenet pretrained feature space. The simple nearest-neighbor based-approach is experimentally shown to outperform self-supervised methods in: accuracy, few shot generalization, training time and noise robustness while making fewer assumptions on image distributions.
1. Introduction
Anomaly detection identifies unusual data to focus resources on potentially important observations, but its extreme-case, highly variable targets make the task difficult. This paper evaluates a simple kNN method using pretrained image features and reports broad advantages over existing methods.
- Anomaly detection labels data as normal or anomalous, specifically targeting extreme cases that are highly variable and difficult to predict.
- Semi-supervised detection trains on normal images only, whereas supervised detection requires labeled normal and anomalous examples.The supplied background also identifies unsupervised detection as a more difficult setting.
- The method embeds train and test images with an ImageNet-pretrained ResNet, scores each test image by its K-nearest-neighbor distance, and thresholds that score.
- The baseline is reported to achieve higher-than-state-of-the-art accuracy, extremely low sample complexity, and minimal-cost use of strong external feature extractors.
- The approach is plug-and-play, makes few assumptions about image size or rotation invariance, and can tolerate anomalies in the training set with a two-stage approach.
- The paper introduces an adaptation of kNN for image group anomaly detection, where sets rather than individual images are analyzed.
2. Previous Work
Earlier anomaly detection methods use reconstruction or distribution modeling, while newer deep approaches combine learned representations with classical detectors. Self-supervised methods learn features through auxiliary tasks, but these representations typically underperform features learned from large supervised datasets such as ImageNet.
- Classical anomaly detection follows reconstruction-based or distribution-based paradigms.
- Reconstruction methods learn basis functions for normal data and classify samples by thresholding reconstruction cost.
- Distribution-based methods model the probability density of normal data and evaluate the probability of new samples.
- Deep learning has been combined with classical methods through deep autoencoders, perceptual losses, generative adversarial networks, semantic embeddings, and deep SVDD.
- Self-supervised methods train networks on inexpensive auxiliary tasks such as frame prediction, colorization, or puzzle solving.
- Self-supervised representations typically underperform those learned from large supervised datasets such as ImageNet.
3. Deep Nearest-Neighbors for Image Anomaly Detection
DN2 applies kNN to pretrained deep image embeddings: it extracts features, compares samples with nearby training embeddings, and thresholds the resulting distance. Extensions support contaminated training data and unordered image groups.
- DN2 is a simple kNN-based method for image anomaly detection.
- In the semi-supervised setting, DN2 assumes all training images are normal and applies a pretrained feature extractor to them.
- The paper uses an ImageNet-pretrained ResNet, whose features are reported to work even when target images are not closely related to ImageNet.
- DN2 stores the extracted training embeddings, allowing the initial training-set inference to be amortized.
- For a new image, DN2 extracts its embedding, computes its kNN distance to training embeddings, and uses that distance as the anomaly score.
- Euclidean distance is used between the test embedding and its k nearest training embeddings, and a threshold determines whether the image is normal or anomalous.
- For fully unsupervised data, DN2 first cleans the input by identifying likely anomalies through kNN distances before proceeding as in the semi-supervised case.
- Group anomaly detection handles unordered image sets whose collective combination may be anomalous even when individual images appear normal.
4. Experiments
Experiments evaluate DN2 across unimodal, cross-dataset, real-world, multimodal, few-shot, and noisy settings. DN2 generally outperforms self-supervised and other anomaly-detection baselines, while requiring little tuning and showing robustness to difficult data conditions.
- Unimodal anomaly detection: DN2 significantly outperforms all other methods on CIFAR10 anomaly detection.The evaluation reports ROCAUC, and DN2 is deterministic for a fixed train/test split.
- Unimodal anomaly detection: DN2 outperforms OCSVM, Deep SVDD, Geometric, and GOAD on Fashion MNIST despite the dataset differing visually from Imagenet.The Fashion MNIST evaluation uses 6000 training images and 1000 test images per class.
- Comparisons against MHRot: DN2 significantly outperforms MHRot on Flowers, Birds, and CatsVsDogs.These comparisons are reported as average ROCAUC across the tested datasets.
- Effect of the number of neighbors: Two neighbors are usually best, although ROCAUC differences across neighbor counts are not particularly large.The number of neighbors is DN2’s only free parameter.
- Real-world data: DN2 outperforms MHRot by a significant margin on high-resolution, orientation-invariant white-blood-cell images.The WBC dataset contains only a few tens of images per class, and images are split 80% for training and 20% for testing.
- Group anomaly detection: DN2 achieves significantly stronger performance than Geometric on multimodal normal-image sets, particularly when the training data is sufficiently varied.The paper attributes Geometric’s reduced effectiveness to its requirement that the network not generalize on anomalous data.
- Generalization from small training datasets: DN2 detects anomalies accurately from only 10 training images, while Geometric deteriorates quickly as the training set shrinks.On FashionMNIST, Geometric is omitted for small datasets because of numerical issues.
5. Analysis
The analysis attributes DN2’s strong anomaly-detection performance to locality-preserving Imagenet-pretrained features, while comparing its accuracy and retrieval-speed trade-offs with alternatives.
- Pretrained vs. self-supervised features: Pretrained features preserve strong locality, explaining DN2’s strong performance when nearest-neighbor density distinguishes normal from anomalous images.The analysis links compact normal-class embeddings and strong locality to effective nearest-neighbor detection.
- Efficiency trade-offs: kNN inference scales linearly with the number of training samples, unlike constant-size hypersphere methods such as One-Class SVM or SVDD.The trade-off is that hypersphere methods typically have lower performance, while kNN retains higher retrieval cost.
- Pretrained vs. self-supervised features: Imagenet-pretrained features separate normal and anomalous CIFAR10 images more clearly than Geometric or Deep-SVDD features.The t-SNE plots show compact same-class regions for pretrained features, while Deep-SVDD is scattered and Geometric is less concentrated.
- Efficiency trade-offs: K-means approximations can substantially reduce retrieval time with only a small accuracy loss by applying kNN to cluster centers.The paper uses the sum of distances to the two nearest neighbors in its approximation comparison.
6. Discussion
The discussion presents pretrained generic features combined with standard anomaly detectors as a fast, broadly applicable alternative to self-supervised anomaly detection. It also identifies scaling and non-image modalities as important boundaries.
- General paradigm: A proposed anomaly-detection paradigm learns general features from vaguely related datasets and applies standard methods such as kNN or k-means.The paradigm uses available external supervision rather than training a task-specific self-supervised representation.
- Generality: DN2-like methods are expected to remain broadly applicable even when target images differ substantially from Imagenet.The paper explicitly frames this as an expectation based on its evaluation beyond Imagenet-like imagery.
- External supervision: Imagenet-pretrained extractors are presented as readily available, free, and inexpensive to deploy without new supervision for each dataset.The described process requires minimal time or storage costs and can be applied with a single PyTorch line.
- Scaling limitations: Nearest-neighbor retrieval becomes slower as datasets grow because runtime increases linearly with training-set size.The discussion mentions fast retrieval and recursive or k-means-based approximations as ways to address this scaling issue.
- Non-image data: The approach’s image-focused advantage does not directly extend to tabular data and time series, where generic feature extractors generally do not exist.The paper identifies these modalities as promising areas for self-supervised anomaly detection.
7. Conclusion
The paper positions kNN on deep image features as a simple but strong baseline for semi-supervised and unsupervised anomaly detection.
- kNN on deep image features outperformed state-of-the-art methods across accuracy, training time, robustness, and sample complexity.