Source-linked AI summary
Registration based Few-Shot Anomaly Detection
Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael Spratling, Yan-Feng Wang
TL;DR
Few-shot anomaly detection must operate with only a few normal images per category, while prior approaches generally use category-specific models. The paper introduces category-agnostic feature registration for a single model that transfers to new categories without fine-tuning, and reports AUC improvements over state-of-the-art FSAD methods on MVTec and MPDD.
Problem
FSAD has only a limited number of normal images for each category, while existing studies follow a one-model-per-category paradigm and do not explore inter-category commonality.
Method
RegAD trains a category-agnostic feature registration model on aggregated categories and compares registered test features with normal support features during testing.
Results
5.1%, 6.9%, and 8.0% AUC improvements are reported on MVTec for 2-shot, 4-shot, and 8-shot settings, respectively, with improvements also reported on MPDD.
Takeaways & Limitations
A single trained model is directly generalizable to new categories without re-training or parameter fine-tuning, supporting category-agnostic FSAD.
Takeaways & Limitations
The method assumes that the learned feature registration ability generalizes to the target category without parameter fine-tuning.
Abstract
from arXiv · showhide
This paper considers few-shot anomaly detection (FSAD), a practical yet under-studied setting for anomaly detection (AD), where only a limited number of normal images are provided for each category at training. So far, existing FSAD studies follow the one-model-per-category learning paradigm used for standard AD, and the inter-category commonality has not been explored. Inspired by how humans detect anomalies, i.e., comparing an image in question to normal images, we here leverage registration, an image alignment task that is inherently generalizable across categories, as the proxy task, to train a category-agnostic anomaly detection model. During testing, the anomalies are identified by comparing the registered features of the test image and its corresponding support (normal) images. As far as we know, this is the first FSAD method that trains a single generalizable model and requires no re-training or parameter fine-tuning for new categories. Experimental results have shown that the proposed method outperforms the state-of-the-art FSAD methods by 3%-8% in AUC on the MVTec and MPDD benchmarks.
1 Introduction
The paper proposes RegAD, a category-agnostic FSAD framework that uses feature registration to train one model across categories and apply it to novel categories without fine-tuning. It addresses limited normal data per category and reports consistent AUC gains over state-of-the-art FSAD methods on MVTec and MPDD.
- Approach: The trained model applies directly to novel categories without re-training or parameter fine-tuning, requiring only estimation of the normal feature distribution from the support set.At test time, anomalies are identified by comparing registered test features with corresponding normal support features.
- Motivation: FSAD provides only a limited number of normal images for each category, making per-category model training costly in applications with many products.Existing AD methods commonly train a dedicated model for each category, while FSAD reduces the available training samples per category.
- Approach: RegAD uses feature registration as a category-agnostic proxy task to learn a common model shared across multiple categories.Its registration network is a Siamese network with three spatial transformer blocks and a feature-level registration loss based on cosine similarity.
- Results: 5.1%, 6.9%, and 8.0% AUC improvements over state-of-the-art FSAD methods are reported on MVTec for 2-shot, 4-shot, and 8-shot settings.On MPDD, the corresponding improvements are 3.2%, 5.0%, and 3.4%.
- Results: RegAD outperforms state-of-the-art FSAD methods on both anomaly detection and anomaly localization tasks.
2 Related Work
Prior anomaly detection methods learn normality from normal-only data, while few-shot anomaly detection uses only a few normal support images for target categories. Related FSAD methods reduce sample demands through augmentation, generative modeling, density estimation, or auxiliary meta-training, whereas RegAD learns category-agnostic feature registration.
- Anomaly Detection: Anomaly detection commonly learns from normal-only data and identifies samples whose distributions differ from the learned normal distribution.
- Few-shot Learning: Few-shot learning methods adapt to novel classes using limited annotated examples through metric learning, generation, or optimization strategies.
- Few-shot Anomaly Detection: FSAD detects anomalies in target categories using only a few normal support images.Existing approaches include transformation-based generative modeling, normalizing-flow density estimation, and Metaformer with additional data and pixel-level annotations.
- Few-shot Anomaly Detection: RegAD differs from prior FSAD by learning category-agnostic feature registration to detect anomalies in new categories without fine-tuning.
3 Problem Setting
The FSAD problem is to train a category-agnostic anomaly detector from normal samples in multiple known categories and apply it to an unseen category using only a small normal support set. The detector must predict whether a target image is normal or anomalous without image-level or pixel-level annotations.
- Problem Definition: Training data contains only normal samples from multiple known categories, and the goal is to learn one category-agnostic anomaly detection model.
- Problem Definition: At test time, the model receives an image from an unseen target category and a support set containing k normal samples from that category.
- Challenges: The model must predict whether the target image is anomalous despite having no image-level or pixel-level training annotations and only a few target-category normal samples.
4 Method
RegAD learns category-agnostic feature registration from paired same-category images, then estimates target-category normal feature distributions from augmented support images. During inference, Mahalanobis distances form and realign an anomaly map whose maximum gives the image-level score.
- Feature Registration Network: Three spatial transformer modules transform convolutional feature maps, with affine matrices learning mappings between source and target feature coordinates.The first three ResNet convolutional blocks retain spatial information, and each block contains an STN feature-transformation module.
- Feature Registration Network: A Siamese encoder and predictor optimize a symmetrized feature-registration loss using cosine similarity between paired registered features.Stop-gradient on one branch prevents collapsing solutions, while spatial averaging preserves information useful for anomaly maps.
- Testing: RegAD applies a shared feature-registration model to support and test images, avoiding parameter fine-tuning for target categories.The model assumes registration generalizes to the target category and uses the learned model without parameter fine-tuning.
- Normal Distribution Estimation: At each feature-grid position, augmented registered support features are aggregated and modeled with a multivariate Gaussian distribution.The covariance includes ϵI so the estimated matrix is full rank and invertible.
- Normal Distribution Estimation: Support-image augmentations expand the support set and benefit normal-distribution estimation.The method applies augmentations including rotation, translation, and flipping.
- Inference: Mahalanobis distances at feature patches form an anomaly map, which is inverse-transformed to the original image coordinates for the final map.The image-level anomaly score is the maximum value in the final anomaly map.
5 Experiments
Experiments evaluate RegAD on MVTec and MPDD under cross-category and individual-training settings, showing strong performance, fast adaptation, and benefits from its registration components.
- Experimental Setup: Experiments use MVTec and MPDD industrial defect-detection benchmarks, evaluating aggregated multi-category training, unseen-category adaptation, and individual support-set training.MVTec contains 15 categories, while MPDD contains six metal-part classes with varied orientations, positions, distances, lighting, and backgrounds.
- Experimental Setup: RegAD is compared with TDG, DiffNet, their multi-category extensions, individual-training RegAD-L, and vanilla AD methods trained with full normal datasets.AUC is measured at image level for anomaly detection and pixel level for anomaly localization.
- Main Comparisons: 82.4% and 57.8% AUC are achieved with one shot on MVTec and MPDD, respectively, without parameter fine-tuning for new categories.RegAD outperforms all other baselines in nine of 15 MVTec categories and has the lowest standard deviation at k=8.
- Ablation Studies: Support augmentations improve AUC, feature-registration aggregated training improves MVTec AUC by 3.3%, 2.9%, and 2.6%, and STNs raise k=8 AUC from 89.3% to 91.2% on MVTec.STNs also increase MPDD AUC from 64.8% to 71.9%, while pixel-level localization remains similar with or without STNs.
- Ablation Studies: Rotation+scale STNs perform best on centered MVTec samples, whereas affine STNs perform best on less-centered MPDD samples.The transformation choice follows differences in spatial alignment across datasets.
- Qualitative Analysis: Aggregated training produces localization closer to ground truth and more compact within-category features than individual training.The learned features also separate different categories, supporting normal-distribution estimation.
6 Conclusion
The paper proposes category-agnostic feature registration for FSAD, training on aggregated category data and applying the model to new categories without retraining or parameter fine-tuning.
- Conclusion: RegAD uses registration as a category-agnostic proxy task to train a feature-registration network from aggregated few-shot data.At test time, anomalies are identified by comparing registered test features with corresponding support features.
- Conclusion: The resulting model generalizes directly to new categories without re-training or parameter fine-tuning.
A Main Contributions
The paper targets practical FSAD with one category-agnostic model that handles novel categories without fine-tuning, using registration and comparison rather than reconstruction or classification.
- RegAD addresses FSAD with a single model for all categories, few images per novel category, and only normal samples available.
- The method learns a category-agnostic model through registration, aligns images with STNs, and implements comparison using a Siamese network.
- RegAD is distinct from popular reconstruction-based and classification-based anomaly detection methods because it uses comparison-based detection.
- The reported state-of-the-art results on MVTec and MPDD show the effectiveness of the proposed method.
B.1 Experiments with a Large k.
With larger support sets, RegAD remains competitive without parameter fine-tuning, but performance gains saturate quickly as the shot number increases.
- RegAD adapts to unseen categories without parameter fine-tuning, reducing the training burden.
- Increasing the shot number produces performance gains that saturate very soon when support examples are used without fine-tuning.
- When k increases from 64 to 128, only a limited performance gain is observed.
- Despite using a shallow backbone, the results remain competitive with anomaly detection methods trained by full data.
B.2 Ablation Studies on Support Set Augmentations.
Support-set augmentations are important for RegAD’s detection and localization, with different augmentation types performing better on MVTec and MPDD.
- Support-set augmentations are essential for both anomaly detection and localization.
- All proposed augmentation methods improve the results in ablation studies using k = 2.
- Rotation and translation perform better on MVTec, whereas flipping and rotation perform better on MPDD.
- The ablation table reports macro-average AUC in % over all categories for 10 runs on MVTec and MPDD.
B.3 Comparisons with Metaformer [39].
On MVTec with k=8, RegAD reports higher AUC than Metaformer under the same test set and evaluation protocol, despite Metaformer’s additional training advantages.
- 91.2% AUC for RegAD exceeds Metaformer’s about 88% AUC on MVTec when k=8, an ≈3% improvement.
- The comparison uses the same test set and evaluation protocol, although the training settings differ.
- Metaformer uses MSRA10K, additional pixel-level annotations, novel-category fine-tuning, and a deeper transformer trained for more epochs with a larger batch size.