Source-linked AI summary
CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization
Sungwook Lee, Seunghyun Lee, Byung Cheol Song
TL;DR
Anomaly localization needs to distinguish normal from abnormal features despite biased pre-trained CNN features and memory-bank limitations. CFA adapts features to the target dataset with a learnable patch descriptor and scalable memory bank, achieving strong results on MVTec AD. The method reports 99.5% I-AUROC and 98.5% P-AUROC.
Problem
Pre-trained CNNs can extract biased features from industrial images, while large target-sized memory banks may overestimate abnormal-feature normality and increase inference time.
Method
CFA applies transfer learning with a learnable patch descriptor, coupled-hypersphere-based feature adaptation, hard-negative supervision, and a scalable memory bank.
Results
99.5% I-AUROC and 98.5% P-AUROC were achieved on the MVTec AD benchmark.
Takeaways & Limitations
Feature adaptation to the target dataset enables target-oriented anomaly localization with a memory bank compressed independently of target-dataset size.
Abstract
from arXiv · showhide
For a long time, anomaly localization has been widely used in industries. Previous studies focused on approximating the distribution of normal features without adaptation to a target dataset. However, since anomaly localization should precisely discriminate normal and abnormal features, the absence of adaptation may make the normality of abnormal features overestimated. Thus, we propose Coupled-hypersphere-based Feature Adaptation (CFA) which accomplishes sophisticated anomaly localization using features adapted to the target dataset. CFA consists of (1) a learnable patch descriptor that learns and embeds target-oriented features and (2) scalable memory bank independent of the size of the target dataset. And, CFA adopts transfer learning to increase the normal feature density so that abnormal features can be clearly distinguished by applying patch descriptor and memory bank to a pre-trained CNN. The proposed method outperforms the previous methods quantitatively and qualitatively. For example, it provides an AUROC score of 99.5% in anomaly detection and 98.5% in anomaly localization of MVTec AD benchmark. In addition, this paper points out the negative effects of biased features of pre-trained CNNs and emphasizes the importance of the adaptation to the target dataset. The code is publicly available at https://github.com/sungwool/CFA_for_anomaly_localization.
1. Introduction
Anomaly localization must distinguish normal from abnormal features despite incomplete outlier coverage and feature mismatch between ImageNet-trained CNNs and industrial images. CFA adapts features to the target dataset and uses a scalable memory bank, achieving strong MVTec AD performance.
- Anomaly localization produces a heatmap showing an outlier’s location and whether an outlier is present.The heatmap can also support explaining the anomaly’s cause.
- Because all possible outliers cannot be included during learning, prior methods typically distinguish abnormal samples by modeling normal-feature distributions.Examples include reconstruction-based unsupervised learning and self-supervised approaches that synthesize abnormal noise.
- Industrial images differ from ImageNet, causing pre-trained CNNs to extract unfitted, biased features that hinder precise normal–abnormal discrimination.Using only mid-level features reduces but does not fundamentally solve this mismatch.
- A memory bank sized with the target dataset can overestimate abnormal-feature normality and increase inference time when it contains many unfitted features.These limitations motivate reducing the memory bank while retaining core normal features.
- CFA adapts a pre-trained CNN through transfer learning, softboundary regression, a learnable patch descriptor, and a scalable memory bank independent of target-dataset size.The loss densely clusters normal features in a minimum-radius hypersphere, while coupled memorized features support discriminative patch descriptors.
- 99.5% I-AUROC and 98.5% P-AUROC were achieved on MVTec AD, while approximately 99.9% of memory-bank activations were decreased.The paper reports these as image-level anomaly detection and pixel-level anomaly localization results, respectively.
2. Related Works
Related methods commonly infer normal features with pre-trained CNNs and compare test patches against memory banks. However, these approaches retain biased features and memory banks whose size remains tied to the target dataset.
- Memory bank-based methods acquire normal features with pre-trained CNNs because collecting all possible outlier samples is costly and impossible.They then compare test-sample patch features with stored normal features using distances such as Euclidean or Mahalanobis distance.
- Prior methods use feature maps, location-wise normal distributions, or mid-level features to model normality and compute anomaly scores.Some also use neighbor information and greedy coreset subsampling to reduce memory and computational costs.
- Existing methods generally use large-dataset-biased features without adaptation, while memory-bank size remains proportional to the target dataset.The memory bank therefore cannot be adjusted to an arbitrary size.
3. Proposed Method
CFA adapts pretrained CNN features to a target dataset by clustering normal patch features around coupled hyperspheres and using a scalable memory bank. Its scoring function combines feature matching with certainty to localize anomalies while avoiding memory growth with dataset size.
- Feature adaptation: CFA uses transfer learning, a learnable patch descriptor, and target-dataset normal features to reduce bias in pretrained CNN representations.Patch features are extracted from multiple CNN depths, resized to a common resolution, concatenated, and adapted toward target-oriented representations.
- Coupled-hypersphere supervision: Normal patch features are attracted toward nearest memorized features inside hyperspheres, increasing their concentration in the adapted feature space.The attraction loss uses K nearest neighbors and Euclidean distance to supervise embeddings toward hyperspheres of radius r.
- Coupled-hypersphere supervision: Hard negative features repel embeddings from competing hyperspheres, addressing ambiguity that could overestimate the normality of abnormal features.The repulsion loss uses hard negatives defined as the K+j-th nearest neighbors and is optimized jointly with the attraction loss.
- Memory bank compression: The memory bank is initialized and updated across normal target samples using clustering and exponential moving averages, retaining core normal features.The final bank is updated iteratively rather than storing every target feature.
- Memory bank compression: CFA uses a scalable memory bank whose space complexity is O(HWD′) and is not affected by target-dataset size |X|.The compressed bank stores cluster-center features while updating the bank state for each sample.
- Scoring and localization: CFA generates heatmaps by matching test patch features to memorized features and computes refined anomaly score maps using a certainty-aware scoring function.The function addresses unclear cluster boundaries and uncertain nearest-feature matches in continuously distributed normal features.
4. Experiments
CFA is evaluated on aligned and unaligned industrial-image benchmarks, with ablations examining feature adaptation, memory-bank compression, backbone choice, and qualitative localization. It reports strong detection and localization performance while substantially reducing memory activation and inference cost.
- Experimental setup: CFA is evaluated on MVTec AD and the unaligned RD-MVTec AD dataset using image-level and pixel-level anomaly metrics.The experiments use I-AUROC for anomaly detection, P-AUROC for anomaly localization, and P-AUPRO for more precise localization evaluation.
- Quantitative results: CFA++ exceeds PatchCore by 0.4% in I-AUROC and achieves strong performance across both image-level detection and pixel-level localization.Its P-AUROC is state of the art for object classes, although CFLOW is slightly better when all classes are considered.
- Quantitative results: On RD-MVTec AD, CFA++ degrades by only 0.8% in I-AUROC and exceeds SPADE and PaDiM by 11.5% and 6.6%, respectively.CFA++ also reports a 10.1% higher P-AUPRO score than PaDiM on the unaligned dataset.
- Ablation study: Feature adaptation improves ResNet18 I-AUROC and P-AUROC by up to 14.1% and 5.4%, while Lrep adds 1.1% and 0.3%, respectively.For WRN50-2, Latt increases I-AUROC and P-AUROC by 13.2% and 4.3%, while Lrep adds 0.4% and 0.2%.
- Ablation study: Reducing memory-bank activation by about 99.9% increases throughput up to 2.8 times, while further compression can increase throughput up to 3.6 times.Compression from 25% to about 2% decreases P-AUROC by only 0.08%.
5. Conclusion
CFA addresses biased pre-trained CNN features in industrial anomaly localization by adapting features to the target dataset. Its learnable patch descriptor and memory bank achieve strong MVTec AD performance through transfer learning.
- CFA adapts pre-trained CNN features to industrial target datasets to address bias in anomaly localization.The method targets the mismatch between pre-trained features and industrial-image distributions.
- CFA combines a learnable patch descriptor with a memory bank to obtain target-oriented features.Feature adaptation associates the patch descriptor with a predetermined memory bank through transfer learning.
- CFA achieved state-of-the-art performance on the MVTec AD benchmark in extensive qualitative and quantitative experiments.The paper examines the effectiveness of adapting features to the target dataset.