PaperlayerView exact revision on arXiv

Source-linked AI summary

Anchor and Adapt: Asymmetric Prompt Adaptation for Few-Shot Industrial Anomaly Detection

Mengyang Zhao, Teng Fu, Haiyang Yu, Ke Niu, Bin Li, Xiangyang Xue

arXiv:2610.07016v1cs.CVcs.AI

TL;DR

Few normal target images provide no direct defect supervision, while manually specified anomaly descriptions require product-specific effort and are sensitive to prompt selection. Anchor and Adapt learns transferable normal and abnormal anchors from auxiliary data, then adapts an additional normal branch while keeping the anchors fixed. Cross-dataset experiments report competitive detection and localization, and ablations assess the contributions of the learned anchors and adaptation mechanisms.

  • Problem

    Few normal target images provide no direct defect supervision for learning anomaly prompts, while manual anomaly descriptions require product-specific effort and prompt selection affects performance.

    Source: §I · p. 1

  • Method

    Anchor and Adapt learns normal and abnormal anchors from annotated auxiliary data, then keeps them fixed while adapting an additional normal branch from target normals.

    Source: §I · p. 1

  • Results

    Cross-dataset experiments on MVTec-AD and VisA, with additional MPDD evaluation, demonstrate competitive detection and localization.

    Source: §VI · p. 9

  • Takeaways & Limitations

    The framework reduces reliance on manually specified category-specific anomaly descriptions and combines learned anomaly semantics with target-specific normal evidence without synthetic anomaly generation.

    Source: §I · pp. 1–2

Abstract

from arXiv · show

In few-shot industrial anomaly detection, the few normal target images provide no direct defect supervision, making anomaly prompts difficult to learn from these samples alone. Some vision-language methods therefore use manually specified descriptions to supply explicit anomaly semantics. However, constructing these descriptions requires product-specific effort, and their effectiveness depends on prompt selection. We propose Anchor and Adapt, a two-stage prompt learning framework that separates the acquisition of anomaly semantics from adaptation to target normal appearance. Stage I learns transferable normal and abnormal anchors from annotated auxiliary data. Stage II keeps these anchors fixed and adapts an additional normal branch using the few target normal samples. The inherited and adapted normal branches jointly characterize target normality, with text-anchor regularization encouraging consistency with the generic normal prior and separation from the abnormal anchors. This design retains learned anomaly knowledge while reducing dependence on category-specific anomaly templates, without requiring synthetic anomaly generation. Cross-dataset experiments between MVTec-AD and VisA under 1-, 2-, and 4-shot settings demonstrate competitive detection and localization performance. Controlled ablations assess the roles of transferred anchors, asymmetric adaptation, dual-normal representations, and anchor regularization.

I. INTRODUCTION

Few-shot target normals describe product appearance but provide no direct defect supervision, making anomaly semantics difficult to learn without manually designed descriptions. Anchor and Adapt learns transferable semantic references from auxiliary data, then adapts normality to the target while retaining those references.

I. INTRODUCTION

  • Few normal target images provide no direct defect supervision, while manual anomaly descriptions require product-specific design and prompt selection can affect performance differently across products.Removing two manual prompts improves some PromptAD categories and degrades others.Source: §I · p. 1
  • Annotated auxiliary data provide defect supervision for transferable normal and abnormal references, while target normals characterize the new product’s appearance.The distinct supervision sources motivate a two-stage approach that learns references first and adapts to target normality second.Source: §I · p. 1
  • Anchor and Adapt learns generic normal and abnormal anchors from auxiliary data, then freezes them while adapting an additional class-specific normal branch from target samples.The inherited and adapted normal branches jointly represent normality; text-anchor regularization encourages consistency with the generic normal prior and separation from abnormal anchors.Source: §I · p. 1

A. Vision-Language Models

  • Vision-language models compare visual features with text descriptions, and prompt-learning methods use learnable tokens alongside manually specified descriptions.The introduction frames reduced dependence on category-specific anomaly templates as a motivation for the proposed approach.Source: A. Vision-Language Models · p. 2, Figure 1 · p. 2, §I · p. 2

B. Industrial Anomaly Detection

Prior industrial anomaly methods use text prompts, visual references, or auxiliary anomaly knowledge to model defects. Anchor and Adapt combines transferable normal and abnormal anchors with normal-only target adaptation.

B. Industrial Anomaly Detection

  • Industrial anomaly detection methods include text prompt ensembles, normal-target prompt learning, fine-grained alignment, simulated anomalies, and residual anomaly representations.The literature also explores normal reference images and video-based normality modeling.Source: B. Industrial Anomaly Detection · pp. 2–3
  • Anchor and Adapt transfers auxiliary-learned normal and abnormal anchors, keeps them fixed, and learns an additional class-specific normal branch from target normals.The inherited and adapted normal branches provide complementary evidence, with anchor regularization guiding adaptation relative to both reference sets.Source: B. Industrial Anomaly Detection · p. 3, Figure 2 · p. 3

A. CLIP and Vision-Language Transfer

  • CLIP transfers to unseen categories by matching image and text features, allowing category descriptions to be replaced without fixed category-specific classifier parameters.Its shared embedding space is learned through large-scale contrastive pretraining.Source: A. CLIP and Vision-Language Transfer · p. 3

B. Prompt Learning

  • Prompt learning replaces hand-crafted context words with learnable vectors, adapting a pretrained vision-language model by optimizing a small set of prompt parameters.CoOp is an example of this parameter-efficient adaptation approach.Source: B. Prompt Learning · p. 3

C. CLIP for Few-Shot Industrial Anomaly Detection

CLIP-based anomaly detection compares visual features with normal and abnormal text features for image detection and pixel localization. Anchor and Adapt learns generic anchors with auxiliary supervision, then retains them during target-normal adaptation.

C. CLIP for Few-Shot Industrial Anomaly Detection

  • CLIP-based anomaly detection estimates anomaly scores from visual-feature similarity differences to normal and abnormal text features, at image and patch levels.Intermediate-layer patch features produce anomaly maps for localization.Source: C. CLIP for Few-Shot Industrial… · p. 4

A. Overview

  • Anchor and Adapt separates transferable anomaly-semantic learning from target-normal adaptation, with inherited and adapted normal branches jointly representing normality.Text-anchor regularization guides adaptation relative to fixed references.Source: A. Overview · p. 4

B. Stage I: Auxiliary Anchor Pretraining

  • Stage I pretrains generic normal and abnormal prompt anchors on annotated auxiliary data using frozen encoders and global and local supervision.Focal and Dice losses supervise probability maps aligned with anomaly masks; the pretrained normal contexts initialize the Stage II class-specific normal branch.Source: B. Stage I: Auxiliary Anchor Pretraining · p. 4

C. Stage II: Asymmetric Target Prompt Adaptation

Target adaptation uses only a few normal images, preserving both auxiliary-learned anchor sets while adding a class-specific normal branch. The branch adapts transferred normal context through low-rank residuals and new tokens.

C. Stage II: Asymmetric Target Prompt Adaptation

  • Stage II uses only a few normal images from each unseen product, retains both auxiliary-learned anchor sets as fixed references, and learns an additional class-specific normal branch.The adapted and inherited generic normal branches jointly represent target normality.Source: C. Stage II: Asymmetric Target Prompt… · p. 4

1) Class-specific normal prompt construction:

2) Dual-normal evidence:

The method aggregates normal evidence from inherited generic and adapted class-specific prompts, while abnormal evidence comes from frozen abnormal prompts.

3) Text-anchor regularization:

Text-anchor regularization constrains adapted normal prompts relative to inherited normal and abnormal anchors. Image- and pixel-level adaptation use the same prompt formulation and regularization, but supervise different visual features.

3) Text-anchor regularization:

  • Text-anchor regularization encourages adapted prompts to match the generic normal reference and separate from fixed abnormal anchors.The normal- and abnormal-anchor margins are denoted mn and ma, respectively.Source: 3) Text-anchor regularization · p. 5

4) Image-level prompt adaptation:

  • Image-level adaptation applies prompt-based logits to global image features from normal target images, with λanchor controlling regularization strength.Source: 4) Image-level prompt adaptation · p. 5

5) Pixel-level prompt adaptation:

  • Pixel-level adaptation treats selected-layer patch features as local normal evidence and uses the same prompt formulation and text-anchor regularization as image-level adaptation.Source: 5) Pixel-level prompt adaptation · p. 5

D. Inference

Inference combines prompt-guided scores from dual-normal and fixed abnormal anchors with visual-reference evidence from a normal memory to produce anomaly maps.

D. Inference

  • Prompt-guided anomaly scores use the dual-normal representation and fixed abnormal anchors.Source: D. Inference · p. 5

1) Pixel-level anomaly scoring:

  • Text-guided patch scores compare abnormal and normal evidence across selected visual layers, then apply normalized log-sum-exp fusion.A sigmoid, spatial reshaping, and upsampling produce the text-guided anomaly map.Source: 1) Pixel-level anomaly scoring · p. 5
  • A visual normal memory is built from few-shot normal-image patches; larger nearest-neighbor normalized cosine distances indicate greater anomaly likelihood.Scores are averaged across selected layers and upsampled.Source: 1) Pixel-level anomaly scoring · p. 5
  • The text-guided and visual-reference maps are normalized before weighted averaging, with equal weights used for the two branches.Source: 1) Pixel-level anomaly scoring · pp. 5–6

2) Image-level anomaly scoring:

Image-level anomaly scoring combines text-based semantic evidence with visual-reference evidence, optionally including a score derived from the fused pixel map.

2) Image-level anomaly scoring:

  • The text-based image score measures affinity to the frozen abnormal anchor relative to dual-normal evidence.Source: 2) Image-level anomaly scoring · p. 6
  • Visual-reference image evidence is summarized by averaging the top-K spatial responses of the visual-reference map.Source: 2) Image-level anomaly scoring · p. 6
  • When the fused pixel map contributes to image-level scoring, its topK responses are averaged and the image-level scores are combined using PromptAD’s harmonic-fusion form.The final term is omitted when the pixel-level score is not used.Source: 2) Image-level anomaly scoring · p. 6

V. EXPERIMENTS

The experiments evaluate few-shot detection and localization on VisA and MVTec-AD, with MPDD added to assess transferability. They use fixed, disjoint support and query sets and report image-, pixel-, and region-level metrics.

V. EXPERIMENTS

  • Experiments use VisA and MVTec-AD as primary benchmarks and MPDD to assess transferability to metal-part inspection.The primary datasets contain 12 and 15 product categories, respectively; MPDD contains six metal-part categories.Source: §V · p. 6, 1) Datasets and Evaluation Metrics · p. 6

1) Datasets and Evaluation Metrics:

  • For each target category, 1, 2, or 4 normal training images form the support set, while the fixed full test split is the query set.Support and query sets are disjoint, and query images and labels are not used to optimize target prompts.Source: 1) Datasets and Evaluation Metrics · p. 6
  • Evaluation reports image- and pixel-level AUROC, image-level AUPR, and region-level PRO; MPDD reports AP and AUPRO in the corresponding columns.All results are percentages, with higher values indicating better performance.Source: 1) Datasets and Evaluation Metrics · p. 6

2) Implementation Details:

  • The method uses CLIP ViT-L/14@336px and cross-dataset Stage-I training, then adapts normal prompts using only target-category normal training images.Stage I trains for 15 epochs and Stage II for 150; target test data are excluded from both stages.Source: 2) Implementation Details · p. 6

B. Main Results

Anchor and Adapt reports competitive detection and localization across VisA, MVTec-AD, and MPDD. Its controlled VisA ablations favor learned anchors, fixed abnormal references, and combining inherited and target-specific normal representations.

1) Comparison methods:

  • Unlike FineGrainedAD’s category-specific anomaly descriptions and AnoPLe’s synthetic anomalies, Anchor and Adapt learns references from real auxiliary data without either.The comparison describes methods and requirements; the cited passage does not report their relative performance.Source: 1) Comparison methods · p. 6

2) Image-level anomaly detection:

3) Pixel-level anomaly localization:

4) Additional evaluation on MPDD:

  • On MPDD, pixel AUROC and PRO lead at all three shots, reaching 98.0 and 94.2 at 4 shots; image AUROC is 82.0, 85.8, and 87.2, tying PromptAD at 4 shots.Image AUROC exceeds PromptAD by 1.3 and 0.5 points at 1 and 2 shots, respectively.Source: 4) Additional evaluation on MPDD · p. 7, Table III · p. 8

C. Ablation Study

  • With the same normal memory, target prompt adaptation further improves I-AUC, I-AUPR, P-AUC, and PRO by 3.8, 3.4, 0.5, and 0.3 points; manual prompts reduce image AUROC from 91.4 to 87.3.The Stage-I-only zero-shot model scores 83.9 image AUROC; adding normal memory without parameter updates raises it to 87.6.Source: C. Ablation Study · pp. 7–8, Table IV · p. 8
  • Keeping abnormal anchors frozen improves all four metrics over constrained abnormal-anchor adaptation, supporting the normal-side adaptation strategy.The alternative updates abnormal anchors under source-anchor preservation and a bounded residual; its losses target normal features and high-violation patches.Source: C. Ablation Study · p. 8
  • Removing the generic normal branch lowers image AUROC by 0.8 and PRO by 0.4 points; removing residual adaptation or anchor regularization lowers AUROC by 0.7 or 1.1 points and also reduces the other metrics.These ablations support combining inherited and target-specific normal representations with residual adaptation and anchor regularization.Source: C. Ablation Study · pp. 8–9

D. Hyperparameter Sensitivity

The conclusion reports competitive cross-dataset detection and localization, while sensitivity tests identify a strong prompt-capacity setting and visualizations show anomaly maps generally aligning with ground-truth masks.

D. Hyperparameter Sensitivity

  • J = 2 and ρ = 8 yield the highest image-level AUROC and AUPR and pixel-level AUROC, with tied-best PRO on VisA under 2-shot.Both parameters control class-specific normal-prompt capacity; increasing either further does not improve scores.Source: D. Hyperparameter Sensitivity · p. 9

E. Visualization Results

  • On VisA under 2-shot, predicted anomaly maps generally highlight regions consistent with ground-truth masks across varied categories and defect patterns.The examples include small localized defects, surface contamination, and structural abnormalities.Source: E. Visualization Results · p. 9

VI. CONCLUSION

  • Cross-dataset experiments on MVTec-AD and VisA, with additional MPDD evaluation, demonstrate competitive detection and localization.The conclusion also reports controlled ablations supporting learned anchors and the proposed adaptation mechanisms, including gains beyond normal-memory retrieval.Source: §VI · p. 9
Loading 2610.07016v1…