PaperlayerView exact revision on arXiv

Source-linked AI summary

PL-SCEA: Reconfiguring Pretrained Attention for Few-Shot Industrial Anomaly Detection

Xiaoyu Yang, Qixing Wu, Huixian Zhao, Changlong Jin

arXiv:2609.03655v1cs.CV

TL;DR

Few-shot anomaly detection has limited normal references, and pretrained attention may not adequately expose localized defects. PL-SCEA reconfigures frozen-VFM attention and pairs the resulting features with a lightweight VAE; the framework achieves strong localization across evaluated settings, with a reported MVTec AD improvement over AnomalyDINO-S.

  • Problem

    Few-shot industrial anomaly detection must learn category-specific normality from limited normal references while remaining sensitive to local defects, and pretrained semantic attention may be mismatched with localized anomaly structure.

    Source: Introduction · p. 1

  • Method

    PL-SCEA retains QK semantic conditioning while reweighting self-correlations over contextualized value features, and a lightweight VAE models the resulting relation-enhanced normal features.

    Source: Introduction · pp. 1–2

  • Results

    4.1–4.8 percentage points higher pixel-level F1-max than AnomalyDINO-S on MVTec AD; across MVTec AD and VisA, the framework has competitive image-level detection and strong localization across 1–8-shot settings.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    Controlled MVTec AD 1-shot comparisons support attention reconfiguration as useful with either a VAE or a memory bank, rather than only one normality model.

    Source: Introduction · p. 2

Abstract

from arXiv · show

Vision Foundation Models (VFMs) provide transferable patch representations for few-shot industrial anomaly detection, but their attention computation is typically inherited from pretraining objectives centered on semantic aggregation. This creates a potential mismatch: token relations that support semantic recognition may not adequately expose the localized texture and structural deviations required for anomaly localization. We therefore investigate the hypothesis that the attention computation of a frozen VFM can be reconfigured as a task-relevant component of anomaly detection. We instantiate this idea with Power-Law Self-Correlation Enhanced Attention (PL-SCEA), which retains the semantic context of pretrained query-key attention while constructing token-adaptive self-correlations over contextualized value features. Positive-correlation filtering and power-law reweighting then emphasize relations that are salient relative to each token's relational background, without introducing additional trainable attention projections. The resulting features are modeled by a lightweight variational autoencoder that provides a fixed-size reconstruction-based representation of category-specific normality. The two stages serve complementary roles: attention reconfiguration shapes how local relational deviations are represented, while reconstruction-based modeling converts deviations from learned normality into anomaly scores. Across MVTec AD and VisA, the complete framework achieves competitive image-level detection and consistently strong pixel-level localization across the evaluated few-shot settings. Ablations further show that PL-SCEA improves localization with either the VAE or a memory bank under the tested setting. These results support the view that task-aligned attention reconfiguration can improve the anomaly-localization capability of frozen pretrained representations.

Introduction

Few-shot anomaly detection must learn category-specific normality from scarce references, while pretrained attention may not expose localized defects well. The paper reconfigures frozen-VFM attention with PL-SCEA and pairs the resulting features with reconstruction-based normality modeling.

Introduction

  • Few-shot detection must establish category-specific normality from only K normal references while remaining sensitive to local defects.Limited normal data may not cover category-specific variation in texture, pose, and structure, making reference collection costly.Source: Introduction · p. 1
  • Pretrained attention aggregates tokens for semantic representation, which may mismatch the localized texture and structural deviations needed for anomaly localization.The passage contrasts fragmented QK responses with more continuous but sometimes region-merging QQ, KK, and VV responses.Source: Introduction · p. 1
  • A lightweight VAE models relation-enhanced normal patch features, complementing PL-SCEA’s role in shaping how local relational deviations are represented.The reconstruction model converts deviations from learned normality into patch-level anomaly scores.Source: Introduction · p. 2
  • 4.1–4.8 percentage points higher pixel-level F1-max than AnomalyDINO-S on MVTec AD; across MVTec AD and VisA, the framework has competitive image-level detection and strong localization across 1–8-shot settings.Controlled comparisons also show PL-SCEA improves localization with either a VAE or memory bank in MVTec AD 1-shot.Source: Introduction · p. 2
  • PL-SCEA retains pretrained QK semantic conditioning while applying token-adaptive, power-law-reweighted self-correlations without additional trainable attention projections.It computes self-correlations over contextualized value features and emphasizes relations salient relative to each token’s relational background.Source: Introduction · pp. 1–2

Related Work

Prior work addresses few-shot anomaly detection through feature matching, cross-category transfer, vision-language methods, and attention reshaping. The paper identifies a remaining question: how to combine pretrained semantic conditioning with anomaly-sensitive token relations.

Related Work

  • Cross-category methods use meta-learning, hierarchical generation, or registration-based alignment, but generally require auxiliary category data or target-category adaptation.Source: Related Work · p. 2
  • Feature-matching methods score query patches against stored normal features, while FastRecon reconstructs query features from supports; both typically depend on reference storage or retrieval.PatchCore uses coreset sampling, GraphCore models support-feature relations, and AnomalyDINO uses nearest-neighbor retrieval.Source: Related Work · p. 2
  • Prior feature-matching and reconstruction approaches operate on pretrained patch representations, leaving open whether the backbone’s token interaction rule suits anomaly localization.Attention reshaping methods modify attention for dense local representations, but direct same-type relations do not explain how to combine semantic conditioning with anomaly-sensitive structure.Source: Related Work · pp. 2–3
  • Vision-language anomaly methods use prompts, multi-scale windows, or normal-image context, but localization still depends on dense visual-feature quality.AnomalyGPT extends this paradigm to instruction-guided analysis.Source: Related Work · p. 2

Normality Modeling

The paper separates few-shot anomaly detection into representation formation and normality estimation. PL-SCEA reconfigures final-layer token interactions, while reconstruction discrepancies provide evidence of deviations from learned normality.

Normality Modeling

  • The framework separates relation-aware feature formation from normality estimation, enabling tests of whether attention reconfiguration contributes beyond a particular downstream model.The second stage reconstructs the features and uses directional reconstruction discrepancies as anomaly evidence.Source: Normality Modeling · p. 3

Token Correlations for Anomaly Detection

  • Original QK attention retains semantic context but can produce fragmented responses, while direct QQ, KK, and VV correlations are more continuous yet may merge distinct local patterns.This example motivates retaining semantic conditioning while selectively recalibrating token relations.Source: Token Correlations for Anomaly Detection · p. 3, Figure 2 · p. 4

Power-Law Self-Correlation Enhanced Attention

  • PL-SCEA first obtains semantically conditioned values, then constructs token-adaptive self-correlations and emphasizes above-average relations rather than indiscriminately increasing similarity.Correlations are computed over contextualized tokens; the method uses the final attention layer of a frozen VFM.Source: Power-Law Self-Correlation Enhanced… · p. 4
  • With γ = 2 by default, power-law reweighting increases relative contrast; rows with no positively salient relation retain the token itself before value reaggregation.The reweighted correlations are normalized row-wise to produce the final attention weights.Source: Power-Law Self-Correlation Enhanced… · p. 4

O = APL-SCEAV. (10)

The category-specific normality model compresses normal patch features into fixed-size encoder and decoder parameters, unlike a memory bank whose state grows with stored patches. A lightweight VAE reconstructs enhanced patch features, and standardized reconstruction discrepancy supplies patch-level anomaly scores.

Reconstruction-Based Latent Normality Modeling

  • Training LNM uses normalized enhanced patches as reconstruction targets and adds Gaussian noise before normalization to form encoder inputs.The passage denotes the enhanced patch as x_i and the clean target as x̄_i.Source: Reconstruction-Based Latent Normality… · p. 5
  • The encoder parameterizes a Gaussian posterior, a sampled latent code feeds the decoder, and the objective combines cosine reconstruction loss with KL regularization.A small β limits the risk that excessive regularization suppresses fine-grained variation among normal patches.Source: Reconstruction-Based Latent Normality… · p. 5
  • At inference, the encoder processes normalized test features without sampling, and the decoder reconstructs from the posterior mean.The reconstruction scores are standardized using the mean and standard deviation computed from normal training patches.Source: Reconstruction-Based Latent Normality… · p. 5
  • Standardized reconstruction discrepancy is used as the patch-level anomaly score, under the assumption that normal features have smaller directional discrepancies than anomalous features.Only the resulting spatial-token features are passed to LNM and patch-level anomaly scoring; the residual, LayerNorm, and MLP sublayers remain unchanged.Source: Reconstruction-Based Latent Normality… · p. 5, O = APL-SCEAV. (10) · p. 5
  • LNM stores category-specific normality in fixed-size parameters Θ, with O(|Θ|) storage independent of reference count, unlike a memory bank; the shared backbone is excluded.Θ comprises the encoder and decoder parameters.Source: Reconstruction-Based Latent Normality… · p. 5

Experiments

Across MVTec AD and VisA, PL-SCEA provides competitive image-level detection and strong pixel-level localization; controlled ablations examine attention variants, insertion layer, normality modeling, memory, and power exponent.

Experiments

  • Evaluation uses three support-set seeds for main category macro-averages, while ablations use a fixed MVTec AD 1-shot set and single-run macro-averages over 15 categories.Comparisons with cited baselines are descriptive because support-set selection and implementation details may differ; ablation values are not numerically interchangeable with Table 1 means.Source: Experiments · p. 5, Ablation Studies · p. 7

Quantitative Results

  • PL-SCEA retains the highest or tied-highest reported means across MVTec AD metrics at 1–2 shots and most metrics at 4–8 shots, with pixel-level F1-max gains of 4.1–4.8 points over AnomalyDINO-S.On VisA, it leads all classification and segmentation metrics at 2, 4, and 8 shots; at 1 shot, it leads classification F1-max and all pixel metrics, while image AUROC and AUPR trail the highest means by 0.1 and 1.0 points.Source: Quantitative Results · pp. 5–6, Table 1 · p. 6, Table 2 · p. 7

Qualitative Analysis

  • Figure 3 shows examples of fine scratches, local damage, and larger structural defects, with predicted responses overlapping annotated regions across defect shapes and scales.The displayed 4-shot maps appear more concentrated than their 1-shot counterparts, though these examples alone do not establish performance across all categories.Source: Figure 3 · p. 6, Qualitative Analysis · p. 7

Ablation Studies

  • Under the fixed MVTec AD 1-shot setting, PL-SCEA leads tested attention variants on image AUROC, image AUPR, pixel AUROC, and pixel F1-max, improving pixel F1-max over QK by 3.78 points.Direct QQ, KK, and VV replacements do not reproduce the same overall gains; QQ has the highest PRO at 93.77%, with PL-SCEA at 93.74%.Source: Table 3 · p. 7, Ablation Studies · p. 7
  • Applying PL-SCEA to the final RADIOv3 layer yields the highest reported values on selected metrics: 97.3% image AUROC, 97.6% pixel AUROC, 65.3% pixel F1-max, and 93.7% PRO.Relative to layer 2, the final layer improves image AUROC by 1.3 points and pixel F1-max by 2.8 points; the framework therefore uses the final Transformer block.Source: Table 4 · p. 7, Ablation Studies · p. 8
  • PL-SCEA improves pixel-level F1-max over QK by 3.8 points with the VAE and 3.5 points with the memory bank, and improves pixel AUROC and PRO with both models.The VAE pairing leads on pixel-level AUROC, F1-max, and PRO, while the memory-bank variant has 0.2-point higher image AUROC; the comparison does not establish synergy.Source: Table 5 · p. 8, Ablation Studies · p. 8
  • The VAE inference state remains 14.0 MB across 1–8 shots, whereas the memory bank grows from 34.8 to 278.4 MB; at 8 shots, the VAE uses about 5.0% of its memory.That is a 95.0% reduction, excluding the frozen backbone shared by both alternatives.Source: Figure 4 · p. 8, Ablation Studies · p. 8
  • Image-level AUROC and AUPR rise as γ increases from 0.5 to 2, reaching 97.25% and 98.66%, then fall at larger tested values; at γ = 6 they are 97.12% and 98.43%.The results favor a moderate exponent, and the remaining experiments use γ = 2.Source: Figure 5 · p. 8, Ablation Studies · p. 8

Conclusion

The conclusion presents task-aligned attention reconfiguration as a useful design direction for improving few-shot industrial anomaly localization. Across MVTec AD and VisA, the framework’s results support its effectiveness, with MVTec AD 1-shot ablations showing localization gains with either a VAE or a memory bank.

Conclusion

  • Results on MVTec AD and VisA support the effectiveness of the complete framework.The conclusion reports that PL-SCEA improves localization with either a VAE or a memory bank in MVTec AD 1-shot ablations.Source: Conclusion · p. 8
  • PL-SCEA retains pretrained QK conditioning while standardizing and power-law reweighting self-correlations of contextualized tokens.Source: Conclusion · p. 8
  • Reconstruction-based latent normality modeling complements the representation stage by summarizing normal features and converting reconstruction discrepancies into anomaly scores.Source: Conclusion · p. 8
  • Within the tested datasets and backbone, the findings support attention reconfiguration as a useful direction for adapting pretrained representations to localized anomaly detection.Source: Conclusion · pp. 8–9
Loading 2609.03655v1…