PaperlayerView exact revision on arXiv

Source-linked AI summary

Template-Based Feature Aggregation Network for Industrial Anomaly Detection

Wei Luo, Haiming Yao, Wenyong Yu

arXiv:2603.22874v1cs.CV

TL;DR

Existing feature-reconstruction methods can learn shortcut solutions that copy anomalous features, undermining industrial anomaly detection. TFA-Net uses a fixed normal template, hierarchical CNN features, and template-based aggregation to filter abnormal information before reconstruction; it reports strong performance on industrial benchmarks, including 98.7% anomaly-detection and 98.3% anomaly-segmentation AUROC on MVTec AD.

  • Problem

    Existing feature-reconstruction methods suffer from shortcut learning by copying input features, while industrial defects are diverse and difficult to comprehensively annotate.

  • Method

    TFA-Net extracts hierarchical features for an input and fixed normal template, aggregates input features onto template features, refines reconstruction details, and segments anomalies using two similarity metrics.

  • Results

    98.7% AUROC for anomaly detection and 98.3% for anomaly segmentation were achieved across all 15 MVTec AD categories, with outstanding performance also reported on MVTec LOCO AD.

  • Takeaways & Limitations

    TFA-Net provides meaningful feature reconstruction rather than simple input replication and is presented as a promising approach for industrial anomaly detection and segmentation.

  • Takeaways & Limitations

    TFAM can lose reconstructed-feature details, reducing performance on some datasets, while ImageNet features may cause false positives, false negatives, and difficulty with complex logical defects.

Abstract

from arXiv · show

Industrial anomaly detection plays a crucial role in ensuring product quality control. Therefore, proposing an effective anomaly detection model is of great significance. While existing feature-reconstruction methods have demonstrated excellent performance, they face challenges with shortcut learning, which can lead to undesirable reconstruction of anomalous features. To address this concern, we present a novel feature-reconstruction model called the \textbf{T}emplate-based \textbf{F}eature \textbf{A}ggregation \textbf{Net}work (TFA-Net) for anomaly detection via template-based feature aggregation. Specifically, TFA-Net first extracts multiple hierarchical features from a pre-trained convolutional neural network for a fixed template image and an input image. Instead of directly reconstructing input features, TFA-Net aggregates them onto the template features, effectively filtering out anomalous features that exhibit low similarity to normal template features. Next, TFA-Net utilizes the template features that have already fused normal features in the input features to refine feature details and obtain the reconstructed feature map. Finally, the defective regions can be located by comparing the differences between the input and reconstructed features. Additionally, a random masking strategy for input features is employed to enhance the overall inspection performance of the model. Our template-based feature aggregation schema yields a nontrivial and meaningful feature reconstruction task. The simple, yet efficient, TFA-Net exhibits state-of-the-art detection performance on various real-world industrial datasets. Additionally, it fulfills the real-time demands of industrial scenarios, rendering it highly suitable for practical applications in the industry. Code is available at https://github.com/luow23/TFA-Net.

1. Introduction

Industrial visual anomaly detection is commonly unsupervised because defects vary widely and comprehensive labeled datasets are impractical. TFA-Net addresses shortcut learning in feature reconstruction by aggregating input features onto a normal template and reports strong benchmark performance.

  • Defect diversity and the impracticality of collecting comprehensive annotations make visual anomaly detection typically unsupervised.
  • Embedding-based methods face memory and inference-speed costs, whereas reconstruction-based methods offer greater potential for practical industrial applications.
  • Feature-reconstruction methods use semantically meaningful CNN features, but existing approaches can simply copy input features through shortcut learning.
  • TFA-Net extracts hierarchical features from an input and fixed normal template, then aggregates input features onto template features to filter abnormal information.
  • Dual-mode anomaly segmentation compares features using Euclidean distance and cosine similarity to improve detection robustness.
  • 98.7% AUROC for anomaly detection and 98.3% for anomaly segmentation were achieved across all 15 MVTec AD categories.

2. Related Work

Related work spans embedding-based and reconstruction-based anomaly detection. Reconstruction methods are practical but can reconstruct defective data, motivating feature-based and template-guided alternatives such as TFA-Net.

  • Embedding-based methods learn discriminative spaces from normal data, using distances from normal clusters to detect or localize anomalies.
  • Reconstruction-based methods encode and decode normal data and use reconstruction error as the anomaly criterion during testing.
  • Autoencoders can generalize strongly enough to reconstruct defective data accurately, producing potential false negatives.

3. The TFA-Net Methodology

TFA-Net’s methodology comprises feature extraction, template-based aggregation, feature-detail refinement, and dual-mode anomaly segmentation. These stages transform multilevel features into reconstructed features and localize defects through feature discrepancies.

  • TFA-Net first extracts multi-level semantic features from input and template images using a pre-trained CNN.
  • The method then applies template-based feature aggregation to combine input information with a normal template.
  • Feature detail refinement reconstructs the aggregated features before testing uses the resulting discrepancies for anomaly localization.

3.1. Overall Architecture

TFA-Net combines pre-trained CNN multiscale feature fusion with ViT-based template aggregation. Similar normal features aggregate strongly onto the template, while dissimilar defect features are suppressed from that aggregation.

  • Overall Architecture: A pre-trained CNN extracts multilevel input and template features, which are resized to a common size and concatenated.
  • Overall Architecture: TFAM uses a normal template so similar normal input features aggregate strongly, whereas dissimilar defect features aggregate weakly.

3.2. Multiple Hierarchical Fusion Feature Extraction

TFA-Net fuses hierarchical CNN features to balance detail and semantic information, using the resulting multi-scale representation as its reconstruction target.

  • Image-level reconstruction struggles with image details and semantically meaningful anomaly criteria.
  • TFA-Net rescales features from multiple levels and concatenates them across channels to form a fused feature representation.
  • Using multi-level fusion features as the reconstruction target improves inspection performance compared with image-level reconstruction.

3.3. Template-based Feature Aggregation Mechanism and Feature Detail Refinement Module

TFAM uses a normal template to aggregate similar input features while filtering dissimilar defect features, after which FDRM refines the reconstructed feature details.

  • TFA-Net introduces TFAM to address shortcut learning, where reconstruction methods copy input features and may reconstruct defects.
  • The model extracts multi-level fused features for both the input and a normal template, then embeds them into token sequences.
  • Positional embeddings enhance TFA-Net’s ability to capture spatial relationships for anomaly detection.
  • TFAM concatenates input and template tokens so self-attention aggregates normal input features into template features.
  • TFAM filters defective features because they have low similarity to normal template features, producing reconstructed maps free from anomalies.
  • Attention maps sum similarities between input and template patches, assigning smaller values to defective regions and larger values to normal regions.
  • The full reconstruction procedure takes projected patch embeddings through TFAM and FDRM before inverse projection produces reconstructed features.
  • Different template images do not significantly affect TFA-Net’s detection performance.

3.4. Discussion on TFAM and FDRM

The discussion examines why TFA-Net retains template features after TFAM and how FDRM complements TFAM in preventing defective feature reconstruction.

  • The analysis compares variants that retain input features, omit FDRM, or retain template features after TFAM.
  • Retaining input features allows defect features to reconstruct because they remain highly similar to themselves, whereas retaining template features blocks low-similarity defects.
  • Because some defect features may still aggregate into template features, FDRM refines and repairs the resulting feature details.

3.5. Training and Testing Procedures

TFA-Net trains with joint Euclidean and cosine feature losses, then segments anomalies using both similarity measures and derives an image-level score from the smoothed anomaly map.

  • 3.5.1. Training procedure: TFA-Net jointly optimizes Euclidean distance and cosine similarity losses when reconstructing input features.
  • 3.5.1. Training procedure: The two loss terms are weighted by w_euc and w_cos, set to 1 and 5, respectively, in the study.
  • 3.5.2. Testing procedure: After training, TFA-Net filters anomalous features from an anomalous image to obtain a reconstructed feature map that eliminates anomalies.
  • 3.5.2. Testing procedure: Dual-mode anomaly segmentation jointly measures input–reconstruction differences using Euclidean distance and cosine similarity.
  • 3.5.2. Testing procedure: The final anomaly map is upsampled, smoothed with a Gaussian filter, and converted into an image-level anomaly score using its standard deviation.
  • Table 1 reports image- and pixel-level AUROC results on the MVTec AD dataset, with highest values bolded and second-highest values underlined.

4. Experiments

Experiments on MVTec AD and MVTec LOCO AD evaluate TFA-Net’s detection, localization, robustness, and component choices. TFA-Net achieves strong benchmark performance, while ablations show benefits from TFAM and dual-mode segmentation alongside trade-offs from patch size and feature-detail loss.

  • MVTec AD: 98.7% image-level AUROC and 98.3% pixel-level AUROC were achieved across 15 MVTec AD categories, exceeding the second-best results by 0.7% and 1.0%.TFA-Net also localized defects across all categories and detected object disappearance in Cable and Transistor.
  • MVTec LOCO AD: TFA-Net achieved 85.4% image AUROC and 72.3% pixel sPRO for structural anomalies on MVTec LOCO AD, while ranking second for logical anomalies.The reported qualitative results show repair and localization of both structural and logical defects.
  • Patch-size ablation: Patch size K=4 balanced reconstruction precision and inference speed, because smaller patches increase sequence complexity while larger patches produce coarser reconstructions and larger projection heads.The study evaluated K values of 2, 4, 8, and 16.
  • Template robustness: Template selection was robust: performance fluctuations stayed below 1% across three pose-diverse datasets, although Screw and Metal Nut varied more than Hazelnut.The reported maximum image/pixel AUROC fluctuations were 0/0.09% for Hazelnut, 0.9/0.21% for Screw, and 0.6/0.37% for Metal Nut.
  • TFAM ablation: TFAM improved average image- and pixel-level AUROC over the model without TFAM, with image-level gains of 6.8%, 5.6%, and 9.6% for Cable, Screw, and Transistor.Pixel-level gains were 3.3% for Cable and 7.9% for Transistor.
  • TFAM limitation: TFAM improved Transistor and Cable performance but slightly reduced Metal Nut and Pill performance because feature-detail loss introduced noise into anomaly maps.This trade-off is attributed to TFAM’s semantic filtering and its possible loss of reconstructed details.

4.4. Analysis of Failure Cases

TFA-Net can produce both false positives and false negatives, particularly with background interference or low defect–background contrast, and remains limited on complex logical defects.

  • False positives arise when background interference is erroneously identified as a defect.This occurs in the Wood, Screw, and Metal nut datasets.
  • False negatives occur when defects have low contrast against the background.The paper reports this issue in the Wood and Zipper datasets.
  • TFA-Net struggles to detect complex logical defects in the Breakfast box and Splicing connectors datasets.
  • The authors identify improved discriminative capacity and more meaningful training tasks as priorities for enhancing robustness and efficacy.

5. Conclusion

The paper introduces TFA-Net, whose TFAM aggregates normal input information onto fixed template features using ViT attention before dual-mode segmentation compares input and reconstructed features. The authors report meaningful reconstruction beyond input duplication and promising anomaly detection and segmentation performance.

  • TFA-Net introduces TFAM, which uses a fixed normal image and ViT multi-head self-attention to aggregate normal input information onto template features.
  • TFAM filters abnormal information and produces reconstruction results that go beyond merely duplicating the input data.
  • Dual-mode anomaly segmentation computes discrepancies between input and reconstructed features to support anomaly detection and segmentation.

CRediT authorship contribution statement

The CRediT statement assigns contributions across conceptualization, methodology, analysis, investigation, writing, validation, supervision, and visualization.

  • Wei Luo contributed to conceptualization, methodology, formal analysis, investigation, writing, and visualization.
  • Haiming Yao contributed to methodology, writing, validation, and supervision.
  • Wenyong Yu contributed to methodology and writing-review and editing.
Loading 2603.22874v1…