PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detection

Wei Luo, Yunkang Cao, Haiming Yao, Xiaotian Zhang, Jianan Lou, Yuqi Cheng, Weiming Shen, Wenyong Yu

arXiv:2503.02424v2cs.CV

TL;DR

Existing anomaly detection often compares test images with training-derived normality that may be poorly aligned because of appearance, position, or class variation. INP-Former extracts Intrinsic Normal Prototypes directly from each test image and uses them to guide normal reconstruction. It reports state-of-the-art or comparable performance across major anomaly-detection settings and datasets, while also showing zero-shot capability for unseen classes.

  • Problem

    Training-derived normal references can be misaligned with test images, limiting prototype-based anomaly detection when appearance, position, or class variation differs.

  • Method

    INP-Former dynamically extracts Intrinsic Normal Prototypes from a single test image and uses an INP-Guided Decoder to reconstruct normal tokens while suppressing anomalous regions.

  • Results

    INP-Former achieves state-of-the-art or comparable performance across single-class, multi-class, and few-shot tasks on MVTec-AD, VisA, and Real-IAD, with zero-shot capability on unseen classes.

  • Takeaways & Limitations

    A single image can provide concise, aligned normality for anomaly detection across multiple task settings, including some unseen-class cases.

  • Takeaways & Limitations

    INP-Former can miss logical anomalies that closely resemble the background, because the INP Extractor may incorrectly treat them as normal prototypes.

Abstract

from arXiv · show

Anomaly detection (AD) is essential for industrial inspection, yet existing methods typically rely on ``comparing'' test images to normal references from a training set. However, variations in appearance and positioning often complicate the alignment of these references with the test image, limiting detection accuracy. We observe that most anomalies manifest as local variations, meaning that even within anomalous images, valuable normal information remains. We argue that this information is useful and may be more aligned with the anomalies since both the anomalies and the normal information originate from the same image. Therefore, rather than relying on external normality from the training set, we propose INP-Former, a novel method that extracts Intrinsic Normal Prototypes (INPs) directly from the test image. Specifically, we introduce the INP Extractor, which linearly combines normal tokens to represent INPs. We further propose an INP Coherence Loss to ensure INPs can faithfully represent normality for the testing image. These INPs then guide the INP-Guided Decoder to reconstruct only normal tokens, with reconstruction errors serving as anomaly scores. Additionally, we propose a Soft Mining Loss to prioritize hard-to-optimize samples during training. INP-Former achieves state-of-the-art performance in single-class, multi-class, and few-shot AD tasks across MVTec-AD, VisA, and Real-IAD, positioning it as a versatile and universal solution for AD. Remarkably, INP-Former also demonstrates some zero-shot AD capability. Code is available at:https://github.com/luow23/INP-Former.

1. Introduction

Unsupervised anomaly detection commonly models normality from training data, but appearance, position, and cross-class variation can misalign stored normal references with test images. INP-Former instead extracts aligned normal prototypes from each test image and uses them to reconstruct normal content for anomaly detection.

  • Unsupervised anomaly detection learns normality from normal samples and identifies abnormal patterns and regions in images.
  • Training-derived prototypes can misalign with test images because of intra-class appearance and positional variation, while cross-class prototypes may resemble anomalies.Increasing few-shot sample size can improve coverage but adds prototypes and reduces inference efficiency.
  • Intrinsic Normal Prototypes use normal patches within an anomalous test image, providing concise prototypes that share the anomaly’s geometric context and appearance.
  • The INP Extractor, INP Coherence Loss, INP-Guided Decoder, and Soft Mining Loss respectively support prototype extraction, normality coherence, normal reconstruction, and hard-sample optimization.Reconstruction errors are used as anomaly scores, while combinations of discrete INPs help reconstruct weakly represented normal regions and suppress anomalous regions.
  • INP-Former learns INP extraction with normal training images, then dynamically derives INPs from a single test image for INP-guided reconstruction.
  • INP-Former reports superior or state-of-the-art performance across single-class, multi-class, and few-shot tasks on MVTec-AD, VisA, and Real-IAD, with zero-shot capability for unseen classes.The method can represent images using only six INPs and extracts INPs for unseen classes.

2. Related Works

Related anomaly-detection methods span single-class, few-shot, and multiclass settings, with different approaches for modeling normality. Prototype learning represents training normality through representative prototypes compared against test samples, while few-shot methods address limited coverage of normal variation.

  • Universal Anomaly Detection: Universal anomaly detection collectively covers single-class, few-shot, and multiclass anomaly-detection setups.
  • Single-Class Anomaly Detection: Single-class methods build distinct models for each class, using reconstruction, knowledge distillation, prototype learning, or embeddings after pretrained-encoder representation.Their reliance on class-specific models limits scalability across many classes.
  • Few-shot Anomaly Detection: Few-shot anomaly detection addresses cases where limited normal samples do not capture the full variability of normality.Prior approaches include spatial alignment, contrastive learning, and broad pretrained knowledge from vision-language models such as CLIP.
  • Prototype Learning: Prototype learning extracts representative training-set prototypes and classifies test samples by measuring distances in a metric space.PatchCore applies this idea to anomaly detection by comparing test features with the nearest normal prototypes.

3. Method: INP-Former

INP-Former dynamically extracts Intrinsic Normal Prototypes from each test image and uses them to reconstruct normal features for anomaly scoring. Its extractor, coherence objective, INP-guided decoder, and soft mining loss address prototype misalignment, anomalous reconstruction, and difficult regions.

  • 3.1. Overview: INP-Former dynamically extracts M Intrinsic Normal Prototypes from a single image and uses them to guide feature reconstruction.A fixed pre-trained Encoder produces multi-scale features; the INP Extractor, Bottleneck, and INP-Guided Decoder complete the pipeline.
  • 3.2. INP Extractor: The INP Extractor uses cross attention with learnable tokens as queries and pre-trained features as key-value pairs to linearly aggregate INPs.The extracted prototypes incorporate global semantic information rather than representative local features alone.
  • 3.2. INP Extractor: The INP Coherence Loss minimizes each normal feature’s cosine distance to its nearest INP, encouraging coherent normal representations while limiting anomalous capture.The distance for each query feature is defined against its nearest INP item.
  • 3.3. INP-Guided Decoder: Table 1 compares computational cost and memory usage, while Table 2 reports multi-class anomaly-detection performance across datasets.The supplied passages provide the table captions but not their numerical entries.
  • 3.3. INP-Guided Decoder: The INP-Guided Decoder combines discrete INPs to reconstruct low-representative normal regions while suppressing reconstruction of anomalous queries.Using INPs as key-value pairs makes decoder outputs linear combinations of normal INPs; reconstruction discrepancies provide token-wise anomaly signals.
  • 3.3. INP-Guided Decoder: INP-guided attention reduces complexity from O(N^2C) to O(NMC), and memory usage from O(N^2) to O(NM), approximating O(NC) and O(N) when M ≪ N.The reduction follows from using M INPs with M much smaller than the number of feature tokens N.
  • 3.4. Soft Mining Loss: Soft Mining Loss emphasizes regions with greater optimization difficulty by modifying feature gradients rather than applying direct reweighting.The difficulty indicator is the ratio of an individual normal region’s reconstruction error to the average normal-region error.
  • 3.4. Soft Mining Loss: The total training objective combines Soft Mining Loss and INP Coherence Loss as Ltotal = Lsm + λLc.The coherence term controls how INPs represent normal features, while soft mining focuses optimization on difficult regions.

4. Experiments

INP-Former is evaluated across multi-class, few-shot, and single-class anomaly detection on MVTec-AD, VisA, and Real-IAD, with ablations examining its components and INP count. The experiments also assess qualitative localization, semantic INP coverage, and zero-shot generalization.

  • Experimental settings: Experiments use MVTec-AD, VisA, and Real-IAD, comparing INP-Former with reconstruction-, embedding-, prototype-, and VLM-based anomaly detection methods.The evaluation covers multi-class, few-shot, and single-class settings.
  • Multi-class anomaly detection: 99.7/99.9/99.2 image-level and 98.5/71.0/69.7/94.9 pixel-level metrics are reported on MVTec-AD for multi-class anomaly detection.On VisA, image-level metrics are 98.9/99.0/96.6 with best or second-best pixel-level performance; on Real-IAD, they are 90.5/88.1/81.5 and 99.0/47.5/50.3/95.0.
  • Few-shot anomaly detection: INP-Former outperforms previous few-shot SOTAs across all three datasets, with image-level gains up to 10.1↑ and pixel-level gains up to 18.9↑ over second-best results.The reported improvements span 4-shot evaluation on MVTec-AD, VisA, and Real-IAD.
  • Single-class anomaly detection: INP-Former achieves new SOTA performance on MVTec-AD and Real-IAD and competitive performance on VisA for single-class anomaly detection.Per-category results are provided in Appendix Sec. J.
  • Ablation studies: The INP modules improve overall performance, INP coherence loss avoids anomalous-region capture, and soft mining loss focuses training on harder regions.The soft mining loss also reduces overlap between normal and abnormal pixel anomaly-score distributions.
  • INP analysis: Performance stabilizes when the INP count exceeds four, but excessive counts can include abnormal-token information; the study therefore uses six INPs.Learned INPs attend to object regions, edges, and background areas, capturing diverse semantic information.
  • Generalization capabilities: An INP Extractor trained on Real-IAD detects INPs on unseen MVTec-AD images, enabling distance-map-based zero-shot anomaly detection.Without specific zero-shot training, the method reports pixel-level AUROCs of 88.0 on MVTec-AD and 88.7 on VisA.

5. Conclusion

INP-Former learns Intrinsic Normal Prototypes by combining normal tokens and uses them to guide reconstruction of normal tokens for anomaly detection. Experiments support strong performance across several AD settings, while logical anomalies resembling background remain a limitation.

  • 5. Conclusion: INP-Former linearly combines normal tokens into INPs and uses them to guide reconstruction of normal tokens.The INP Coherence Loss refines prototype quality, while the Soft Mining Loss optimizes training.
  • 5. Conclusion: Extensive experiments on MVTec-AD, VisA, and Real-IAD show SOTA or comparable performance in single-class, multi-class, and few-shot AD.
  • 5. Conclusion: INPs can be extracted from unseen categories, enabling zero-shot anomaly detection.
  • 5. Conclusion: Logical anomalies closely resembling the background can be misidentified as INPs, limiting detection in cases such as misplaced Transistor anomalies.The authors propose combining INPs with pre-stored prototypes in future work.

A. More implementation details

The implementation uses grouped ViT features and regional cosine distances to compute anomaly scores, with few-shot augmentation following RegAD. Supplementary materials also describe efficiency and loss-weight analyses.

  • A. More implementation details: Features from ViT-Base layers 3–6 and 7–10 form two groups for group-to-group supervision.The anomaly map uses regional cosine distance between encoder and decoder groups.
  • A. More implementation details: The image-level anomaly score averages the top 1% of the regional cosine-distance map.
  • A. More implementation details: Few-shot experiments use data augmentation techniques similar to RegAD.On Real-IAD, “shot” denotes the number of images rather than views.
  • A. More implementation details: Table S1 compares model size and computational complexity, while Figure S1 examines loss-weight effects on MVTec-AD performance.

B. More details about comparison methods

The supplementary comparison section documents the baselines, evaluation settings, and efficiency analyses used to contextualize INP-Former across anomaly-detection tasks.

  • B. More details about comparison methods: Compared methods include feature-alignment, Gaussian-distribution, memory-bank, vision-language, knowledge-distillation, reconstruction, diffusion, and state-space approaches.Examples include SPADE, PaDiM, PatchCore, WinCLIP, RD4AD, UniAD, DiAD, and MambaAD.
  • B. More details about comparison methods: PromptAD targets few-shot one-class AD through automated prompt learning, semantic concatenation, and an explicit margin.
  • B. More details about comparison methods: INP-Former uses fewer FLOPs than DeSTSeg, DiAD, and Dinomaly while reporting higher detection performance than those methods.It has larger parameter size and FLOPs than SimpleNet, UniAD, and MambaAD but reports improved detection performance.
  • B. More details about comparison methods: The supplementary experiments examine super-multi-class performance, image size, ViT architecture, and loss-weight effects.Super-multi-class training combines MVTec-AD, VisA, and Real-IAD.

E. Influence of ViT Architectures.

INP-Former remains effective with ViT-Small and improves as the ViT grows, although ViT-Large is less practical because of its computational cost. The method also has a specific weakness for logical anomalies resembling background.

  • E. Influence of ViT Architectures.: INP-Former shows strong detection performance with ViT-Small, with performance improving as ViT size increases.
  • E. Influence of ViT Architectures.: ViT-Large achieves the best performance but has high FLOPs and parameter count, so ViT-Base is used by default.
  • E. Influence of ViT Architectures.: Logical anomalies similar to the background can be incorrectly extracted as INPs, as illustrated for misplaced anomalies in the Transistor class.
  • E. Influence of ViT Architectures.: Image-level performance is robust to loss-weight changes, whereas pixel-level performance rises and then falls as λ increases.Low λ may allow anomalous information into INPs; high λ can also reduce performance.
  • E. Influence of ViT Architectures.: Super-multi-class performance declines only slightly relative to multi-class performance, supporting a unified model across broader product ranges.The authors associate this setting with reduced memory consumption in industrial applications.

I. More Few-shot Anomaly Detection Results

INP-Former achieves state-of-the-art or competitive few-shot anomaly-detection results across MVTec-AD, VisA, and Real-IAD, with reported comparisons for 1-shot and 2-shot settings.

  • I. More Few-shot Anomaly Detection Results: INP-Former achieves state-of-the-art or competitive results across MVTec-AD, VisA, and Real-IAD in 1-shot and 2-shot settings.These comparisons are reported in Tables S6 and S7.
  • I. More Few-shot Anomaly Detection Results: Per-class performance is reported for INP-Former on MVTec-AD, VisA, and Real-IAD under single-class anomaly detection.The results appear in Tables S9, S10, and S11, respectively.

K. More Zero-shot Anomaly Detection Results

INP-Former shows some zero-shot anomaly-detection efficacy by extracting INPs from unseen-class images, although its image-level performance is weaker than that of the specified comparison method.

  • K. More Zero-shot Anomaly Detection Results: INP-Former extracts INPs from unseen-class images and compares all tokens with them for zero-shot anomaly detection.Table S8 compares its performance with WinCLIP.
  • K. More Zero-shot Anomaly Detection Results: Table S8 reports zero-shot anomaly-detection performance across different AD datasets.The table identifies the best result in bold.
  • K. More Zero-shot Anomaly Detection Results: 88.0 and 88.7 pixel-level AUROCs are achieved on MVTec-AD and VisA, respectively, in zero-shot anomaly detection.The method is not designed specifically for zero-shot anomaly detection.

L. More qualitative results

Qualitative multi-class results show that INP-Former accurately localizes anomalous regions across a wide range of categories on MVTec-AD, VisA, and Real-IAD.

  • L. More qualitative results: INP-Former accurately localizes anomalous regions across a wide range of categories in multi-class anomaly detection.Predicted anomaly maps are shown for MVTec-AD, VisA, and Real-IAD.

M. More detailed analysis of the limitations

The detailed analysis identifies a limitation for logical anomalies resembling their background, contrasts learnable INPs with handcrafted prototypes, and examines qualitative localization and attention behavior.

  • M. More detailed analysis of the limitations: INP-Former detects the misplaced logical anomaly in Cable but misses the visually similar misplaced anomaly in Transistor.The authors attribute the miss to the extractor mistakenly selecting the Transistor anomaly as an INP.
  • M. More detailed analysis of the limitations: The authors propose combining pre-stored prototypes with INPs to address logical anomalies that resemble the background.Pre-stored prototypes provide comprehensive semantic information, while INPs provide strong alignment.
  • N. Comparison of INP with handcrafted aggregated prototypes: Unlike handcrafted aggregated prototypes limited to zero-shot texture anomaly detection, the learnable INP extractor supports adaptable shapes and application to objects.The INP-guided decoder also reduces self-attention computational cost and supports detection across multiple settings.
  • O. Comparision of INP with MuSc: INP-Former requires only a single test image, unlike MuSc, which relies on many test images for mutual scoring.The authors therefore characterize direct performance comparisons between the two methods as unfair because their setups differ.
  • M. More detailed analysis of the limitations: Cross-attention maps show that INPs capture semantic information from object regions, boundaries, and background areas.This visualization connects INP representations with diverse image-patch regions.
Loading 2503.02424v2…