PaperlayerView exact revision on arXiv

Source-linked AI summary

DriftAD: Visually-Guided Text Drift for Few-Shot Industrial Anomaly Detection

Wenyang Liu, Tianyi Liu, Dongshuo Zhang, Kejun Wu, Adams Wai-Kin Kong

arXiv:2608.23723v1cs.CV

TL;DR

Existing vision-language anomaly detectors use globally static text representations that cannot adapt to defect location and appearance. DriftAD uses visually guided text drift and spatial gating to create adaptive anomaly descriptors and improve text-visual alignment, yielding gains over an ASA-only baseline on MVTec-AD and VisA.

  • Problem

    Existing VLM-based methods use static category-level text embeddings shared across spatial locations and encoder depths, limiting precise defect referencing.

  • Method

    DriftAD amplifies defect cues, dynamically drifts frozen CLIP embeddings using local visual context, and uses abnormal descriptors to gate anomaly-relevant visual features.

  • Results

    +1.1% AUROC and +0.8% pAUROC on MVTec-AD, while +0.6% AUROC and +0.2% pAUROC on VisA, versus the ASA-only baseline.

  • Takeaways & Limitations

    Explicit alignment between drifted text descriptors and visual features is essential for fully exploiting visually guided text drift.

Abstract

from arXiv · show

Few-shot anomaly detection (FSAD) has recently benefited from vision-language models such as CLIP, which enable anomaly de?tection by aligning visual features with text descriptions of normal and abnormal states. However, existing methods typically rely on static text prompts that are applied uniformly across the entire feature hierarchy and spatial dimensions. This rigid global-to-local matching fails to capture the highly localized and scale-dependent physical variations of industrial defects. To address this, we propose DriftAD, a FSAD framework built on three key modules. First, an Anomaly Signal Amplification (ASA) module enhances subtle defect signals through spatial and frequency branches before text-visual matching. Second, Visually-Guided Text Drift (VGTD) dynamically transforms frozen CLIP text embeddings, steering them into layer?wise, spatially-adaptive anomaly descriptors conditioned on local visual context at each encoder depth. Third, Drift-Guided Spatial Gating (DGSG) uses the drifted abnormal descriptor as a spatial probe to selectively enhance anomaly-relevant visual features. Addi?tionally, a drift separation loss prevents representational collapse of the drifted descriptors, and a gate supervision loss enforces spatially discriminative gating in DGSG. Extensive experiments on MVTec?AD and VisA demonstrate state-of-the-art performance across all 1-, 2-, and 4-shot settings on both image-level and pixel-level metrics. Code is available at https://github.com/wenyang001/DriftAD.

1 Introduction

DriftAD addresses the limitation of static, category-level anomaly text descriptors in few-shot industrial anomaly detection. Its Visually-Guided Text Drift module converts frozen CLIP embeddings into layer-wise, spatially adaptive descriptors conditioned on local visual context.

  • Unsupervised anomaly detection methods model normality effectively but remain constrained by a one-model-per-category paradigm, limiting scalability across diverse product categories.
  • Prior approaches include large-language-model reasoning and learnable continuous prompts optimized on few-shot support images to produce category-adaptive descriptors.
  • Existing vision-language methods use category-level anomaly descriptors that are shared uniformly across spatial locations and encoder depths, preventing adaptation to individual test images.
  • DriftAD introduces Visually-Guided Text Drift to transform frozen CLIP embeddings into layer-wise, spatially adaptive anomaly descriptors conditioned on each input image’s local visual context.Normal and abnormal descriptors dynamically drift from their corresponding frozen embeddings according to visual features at each spatial location and encoder depth.

2 Related Work

Related work progresses from costly closed-set, category-specific anomaly models toward open-set and few-shot vision-language approaches. Yet existing methods mainly adapt visual features or use globally fixed text prompts, leaving text descriptors unable to respond to local query-image content.

  • Traditional Anomaly Detection: Traditional unsupervised anomaly detection requires a separate model and many normal samples for each object category, limiting scalability and efficiency.This closed-set design also incurs substantial computational and memory costs in dynamic industrial applications.
  • Vision-Language Anomaly Detection: CLIP-based open-set methods use semantic priors for anomaly detection, progressing from multi-scale prompt matching to learnable prompts and specialized projectors.WinCLIP pioneered zero-shot patch–text matching, while AnomalyCLIP and AdaCLIP improved adaptation but zero-shot methods lack visual reference for fine-grained domain-specific features.
  • Few-Shot Anomaly Detection: Few-shot methods exploit limited visual references for cross-category generalization through feature transformation, distribution regularization, guided learning, continuous prompts, and visual-textual decoding.Representative methods include RegAD, FastRecon, NAGL, IIPAD, and AnomalyGPT, with AnomalyGPT targeting pixel-level localization.
  • Text-Visual Alignment: Existing alignment methods either learn global category-level prompt tokens or strengthen visual features, but their text descriptors remain fixed and unaware of each query image’s local content.PromptAD and AnomalyCLIP use learnable continuous prompts, while APRIL-GAN, KAG-prompt, and FocusPatch-AD enhance or localize visual representations before matching.

3 Methodology

DriftAD combines anomaly-signal amplification, visually guided text drift, spatial gating, and multi-branch scoring to adapt frozen CLIP representations to localized, layer-dependent industrial defects. Its training objective additionally regularizes drift geometry and spatial gating.

  • Overall Pipeline: DriftAD extracts multi-level frozen CLIP features, projects them into scale-aware representations, and processes them through four components: ASA, VGTD, DGSG, and MBAS.The components respectively amplify anomaly signals, generate adaptive descriptors, refine visual features, and produce the final anomaly map and image-level score.
  • Anomaly Signal Amplification: ASA combines spatial residual analysis with frequency-spectrum reweighting to amplify structural and textural anomaly signals suppressed by dominant normal patterns.The spatial branch contrasts features with 3×3 and 9×9 smoothed backgrounds, while the frequency branch reweights low- and high-frequency components after 2D FFT decomposition.
  • Visually-Guided Text Drift: VGTD conditions layer-wise, spatially adaptive descriptors on amplified visual features, replacing globally uniform anomaly text embeddings with location- and depth-specific representations.It propagates deeper semantic context toward shallower layers, generates normalized drift fields with dedicated layer generators, and displaces frozen text embeddings within CLIP space.
  • Drift-Guided Spatial Gating: DGSG uses drifted abnormal descriptors as spatial probes to gate and selectively amplify anomaly-relevant regions while preserving low-level details through skip-connected refinement.Multi-head pixel-wise similarities are fused into a unified attention gate applied to the amplified visual features.
  • Multi-Branch Anomaly Scoring: MBAS fuses clean CLIP matching, drift-guided matching, and support-set nearest-neighbor evidence to produce the final anomaly map.The support-set branch uses a memory bank at inference time, while the clean branch preserves CLIP’s zero-shot generalization capability.
  • Training Objective: Training combines classification, segmentation, gate, and drift objectives, with drift regularization preserving CLIP’s discriminative geometry during spatially adaptive displacement.The method uses binary cross-entropy, focal, and dice losses alongside tailored VGTD and DGSG losses.

4 Experiments

DriftAD is evaluated on MVTec-AD and VisA under few-shot protocols, where it achieves the strongest average results across 1-, 2-, and 4-shot settings. Ablations and qualitative analyses show that visually guided text drift, spatial gating, and their supporting losses improve localization and text–visual alignment.

  • Few-shot comparisons: DriftAD achieves the best average performance across all 1-, 2-, and 4-shot settings on MVTec-AD and VisA.In the 1-shot setting, it improves average AUROC by up to 1.5% and average pAUROC by up to 0.5% over KAG-prompt.
  • Qualitative analysis: DriftAD produces clearer, sharper anomaly maps by suppressing background activations and precisely highlighting defect regions.This advantage is especially evident on VisA, where defects are small and scattered across structurally complex objects.
  • Ablation studies: VGTD alone yields marginal pAUROC gains, whereas combining VGTD with DGSG improves the ASA-only baseline by +1.1% AUROC and +0.8% pAUROC on MVTec-AD.On VisA, the combined modules improve the ASA-only baseline by +0.6% AUROC and +0.2% pAUROC, indicating that DGSG helps align visual features with drifted text descriptors.
  • Ablation studies: The full model performs best on both MVTec-AD and VisA when both L_gate and L_drift are included.The two losses are complementary: L_gate improves over the base model, while L_drift provides additional gains.
  • Ablation studies: The layer-adaptive VGTD grid configuration {4, 8, 8, 16} outperforms uniform and descending alternatives.Uniform-16 forces shallow layers to generate overly detailed drift fields, while deeper layers benefit from finer-grained fields.

5 Conclusion

DriftAD addresses the limitation of globally static text representations in VLM-based anomaly detection by adapting anomaly descriptors to defect-specific spatial and visual characteristics. It combines anomaly signal amplification with visually guided text drift of frozen CLIP embeddings.

  • Conclusion: DriftAD targets the inability of globally static text representations to adapt to defects’ spatial distribution and visual characteristics.This limitation is identified as a key weakness of existing VLM-based methods.
  • Conclusion: The Anomaly Signal Amplification module highlights subtle defect cues before subsequent visual-text adaptation.The module is described as the first component of DriftAD.
  • Conclusion: Visually-Guided Text Drift uses visual contexts to transform frozen CLIP embeddings into layer-wise, spatially-adaptive anomaly descriptors.The descriptors are conditioned by the amplified visual cues and adapt across encoder layers and spatial locations.

Supplementary Material

The supplementary material details the Anomaly Signal Amplification (ASA) module, which combines spatial and frequency branches to enhance defect-related residuals. The branches extract multi-scale spatial backgrounds and adaptively reweight frequency components before fusing their outputs.

  • Anomaly Signal Amplification: ASA’s spatial branch extracts multi-scale backgrounds using 3×3 and 9×9 average pooling, then applies attention-guided modulation to the spatial residual.The branch uses two pooling scales to model spatial backgrounds before modulation.
  • Anomaly Signal Amplification: ASA’s frequency branch applies a 2D FFT to V_l, separates low-frequency components, high-frequency components, and phase, and adaptively reweights the spectrum.The decomposed components are denoted E_low, E_high, and Φ.
  • Anomaly Signal Amplification: The spatial and frequency branch outputs are fused to produce the amplified ASA representation.

A Static Text Prompt Construction

The method constructs static normal and abnormal CLIP text embeddings by ensembling hand-crafted state and sentence templates. These input-agnostic embeddings serve as base anchors for VGTD rather than being used directly for text–visual matching.

  • Prompt Construction: Normal and abnormal embeddings are built by combining state descriptors with category names inside sentence templates, then averaging prompt outputs from the frozen CLIP text encoder.The resulting class embeddings are t+ and t− ∈ R^D.
  • State-Level Descriptors: Normal descriptors include flawless, perfect, and unblemished states, whereas abnormal descriptors include damaged, broken, flaw, defect, and damage states.Each descriptor uses the category name as [c].
  • Template-Level Sentences: Basic templates use “a photo of a [s]” and “a photo of the [s]”, supplemented by cropped, close-up, and black-and-white variants for broader coverage.The state descriptor [s] is inserted into each sentence template.
  • Prompt Ensemble: The ensembled embeddings are computed once per category and remain fixed during inference, carrying no query-image spatial or structural information.Their static, input-agnostic nature motivates transforming them before matching.
  • Prompt Ensemble: DriftAD uses t+ and t− as base anchors for VGTD, which transforms them into layer-wise, spatially adaptive anomaly descriptors Tℓ,±.The anchors are therefore not used directly for text–visual matching.

B Anomaly synthesis for Training.

Training anomalies are synthesized from normal images using NSA, with automatically generated masks providing supervision and static text embeddings anchoring VGTD during inference.

  • Anomaly synthesis for Training: NSA blends patches from different normal images to generate pseudo-anomalies, while paired binary masks supervise Lseg and Lgate and embeddings t+ and t− anchor VGTD inference.Synthesis and masks follow NSA’s default implementation settings; text prompts follow Section A.

C Details of Anomaly Signal Amplification

The Anomaly Signal Amplification module enhances anomaly-relevant features through parallel spatial and frequency branches before text-visual matching. Their learnable fusion balances structural and textural signals, producing enriched visual contexts for Visually-Guided Text Drift.

  • Overview: ASA processes scale-aware features through parallel spatial and frequency branches to amplify anomaly-relevant signals before text-visual matching.The amplified features are subsequently passed to VGTD for drift generation.
  • Spatial branch: The spatial branch uses 3×3 and 9×9 average pooling to estimate local and regional backgrounds, then forms a local contrast residual highlighting structural deviations.A two-layer convolutional attention network processes the residual for spatial amplification.
  • Frequency branch: The frequency branch separates FFT magnitude features into low- and high-frequency components, reweights them learnably, and reconstructs the feature domain while preserving the original phase.The resulting textural contrast residual highlights abnormal spectral patterns and is fused with the original features through convolution.
  • Adaptive fusion: A learnable weight adaptively fuses the spatial and frequency outputs, shifting emphasis between structural and textural anomaly signals according to the defect category.The fused amplified features serve as enriched visual contexts for drift generation in VGTD.

D Breakdown Performance

DriftAD delivers the strongest subset-wise performance across MVTec-AD and VisA over 1-, 2-, and 4-shot settings, with especially large advantages on fine-grained or spatially localized defects. Its gains are most pronounced in 1-shot evaluation, where VGTD-based text adaptation is particularly beneficial.

  • Evaluation scope: The evaluation covers 24 tables spanning MVTec-AD and VisA, three shot settings, four metrics, and six baseline methods.The metrics are AUROC, pAUROC, AUPR, and PRO; baselines include PatchCore, WinCLIP, AnomalyGPT, PromptAD, KAG-prompt, and FocusPatch-AD.
  • MVTec-AD: DriftAD consistently achieves the highest mean performance on MVTec-AD across all shot settings and metrics.Its gains are particularly notable on grid, transistor, and screw, where static global text embeddings can produce diffuse or misaligned anomaly maps.
  • MVTec-AD: 100.0% AUROC is achieved by DriftAD on grid under the 1-shot setting.This result is reported in Table 7 as an example of its performance on a structurally challenging category.
  • MVTec-AD: 83.4% AUROC is achieved by DriftAD on screw, exceeding KAG-prompt at 73.1% and FocusPatch-AD at 78.4%.The advantage is attributed to VGTD’s effectiveness on fine-grained, spatially localized defects and is most pronounced in the 1-shot setting.
  • VisA: DriftAD surpasses all baselines across metrics and shot settings on VisA, with consistent gains on macaroni1, pcb1, and pcb2.These categories contain subtle structural anomalies requiring precise spatial localization.
  • VisA: 94.7% AUROC is achieved by DriftAD on pcb1 under 1-shot, versus 86.2% for FocusPatch-AD and 84.4% for KAG-prompt.On pcb2, DriftAD reaches 87.4% AUROC versus 70.9% for FocusPatch-AD and 83.3% for KAG-prompt, a reported gain of 16.5%.

E More Qualitative Results

The section presents qualitative anomaly-localization visualizations for DriftAD on representative categories from MVTec-AD and VisA. Each example contrasts normal and anomalous images with defect annotations, ground-truth masks, and predicted anomaly maps.

  • Qualitative anomaly localization: DriftAD’s qualitative results cover representative categories and diverse defect types from both MVTec-AD and VisA.Three samples are shown per selected category.
  • Visualization layout: Each visualization group contains a normal reference, an anomalous input with a red defect bounding box, a ground-truth binary mask, and DriftAD’s predicted anomaly map.The four-row layout enables direct comparison between the input defect, annotation, and prediction.
  • Dataset-specific examples: Figure 7 reports qualitative localization examples on selected MVTec-AD categories, while Figure 8 reports corresponding examples on selected VisA categories.Both figures present three defect samples per category.
Loading 2608.23723v1…