PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Unsupervised Anomaly Localization in Industrial Images: A Survey

Xian Tao, Xinyi Gong, Xin Zhang, Shaohua Yan, Chandranath Adak

arXiv:2207.10298v1cs.CV

TL;DR

Industrial anomaly localization must address defects that supervised inspection cannot reliably cover because defect samples and annotations are scarce. This survey synthesizes deep-learning approaches, datasets, challenges, performance, and future directions, reporting strong results for feature-embedding methods while identifying the need for realistic industrial datasets.

  • Problem

    Industrial anomaly detection alone cannot identify interpretable defect locations, while defect-free training distributions and contamination complicate unsupervised localization.

  • Method

    The paper surveys deep-learning methods for unsupervised industrial anomaly localization, organizing approaches by taxonomy, datasets, challenges, performance, and future directions.

  • Results

    Feature-embedding methods achieved 93.9%, 96.5%, and 95.0% pixel AUROC for S-T, SPADE, and DFR on MVTec AD, respectively.

  • Takeaways & Limitations

    Self-supervision and pre-trained deep feature embedding are promising strategies for increasing anomaly-localization performance, with ViT, NF, and 3D localization identified as future directions.

  • Takeaways & Limitations

    Existing methods cannot be adapted to real complex industrial scenarios without realistic and rich industrial anomaly-localization datasets.

Abstract

from arXiv · show

Currently, deep learning-based visual inspection has been highly successful with the help of supervised learning methods. However, in real industrial scenarios, the scarcity of defect samples, the cost of annotation, and the lack of a priori knowledge of defects may render supervised-based methods ineffective. In recent years, unsupervised anomaly localization algorithms have become more widely used in industrial inspection tasks. This paper aims to help researchers in this field by comprehensively surveying recent achievements in unsupervised anomaly localization in industrial images using deep learning. The survey reviews more than 120 significant publications covering different aspects of anomaly localization, mainly covering various concepts, challenges, taxonomies, benchmark datasets, and quantitative performance comparisons of the methods reviewed. In reviewing the achievements to date, this paper provides detailed predictions and analysis of several future research directions. This review provides detailed technical information for researchers interested in industrial anomaly localization and who wish to apply it to the localization of anomalies in other fields.

I. INTRODUCTION

Industrial visual inspection has relied heavily on supervised learning, but costly annotations, scarce defects, unknown defect types, and annotation noise motivate unsupervised anomaly localization. Unlike image-level anomaly detection, anomaly localization produces pixel-level anomaly regions, and this survey reviews the emerging literature and taxonomies around that goal.

  • Motivation: Supervised inspection requires image, bounding-box, or pixel-wise labels, creating substantial annotation costs and dependence on known defect types.Defective samples can also be scarce, and labeling may introduce noise.
  • Motivation: Unsupervised anomaly localization trains on normal images without defective samples, reducing the need to collect anomalous data and annotate training samples.The approach also avoids labeling deviation associated with supervised methods.
  • Task distinction: Anomaly detection classifies images at the image level, whereas anomaly localization produces pixel-level anomaly locations.Darker regions in an anomaly heat map indicate greater anomaly likelihood.
  • Task distinction: Image-level anomaly detection can be uninterpretable because its network may emphasize normal wood texture instead of actual drill-hole defects.The survey uses this failure case to motivate localization in industrial scenes.
  • Survey gap: Earlier surveys covered classical, broad deep anomaly detection, limited localization, or GAN-focused methods, leaving comprehensive industrial anomaly-localization coverage underdeveloped.The surveyed literature also spans evolving comparisons from image-level reconstruction or generation to feature-level comparison and contrastive self-supervision.

C. Contributions to this Article

The article contributes a focused survey of deep learning methods for unsupervised industrial anomaly localization, with a detailed taxonomy, method comparison, and discussion of datasets, metrics, challenges, and future directions. It also distinguishes textural from functional anomalies and identifies practical localization difficulties.

  • Contributions: The paper focuses specifically on deep learning algorithms for unsupervised anomaly localization in industrial images.It excludes medical-image and video anomaly localization.
  • Contributions: Its taxonomy provides a detailed subclassification of recent anomaly-localization methods and uses a Venn diagram to show distinctions and overlaps.The survey also organizes methods by their high-level paradigms and overlapping categories.
  • Contributions: The survey compares existing methods on a public dataset and summarizes benchmark datasets, evaluation metrics, performance, advantages, limitations, and future research directions.Its conclusion frames these analyses as unresolved issues and possible directions for industrial anomaly localization.
  • Anomaly types: Industrial anomalies are categorized as textural anomalies with little semantic information or functional anomalies requiring considerable semantic information.A misplaced transistor can resemble a normal sample in texture, making functional anomalies more difficult to detect.
  • Problems and challenges: Unsupervised localization must handle missing or contaminated normal regions, imaging variation, tiny defects, large-span defects, and inaccurate anomaly boundaries.These conditions can cause normal regions to be judged anomalous and make precise contour segmentation difficult.

B. A Road Map of Anomaly Localization

The survey traces industrial anomaly localization from early sparse-coding and dictionary-learning methods to deep models based on GANs and autoencoder reconstruction. It then organizes current approaches by high-level paradigm and further subdivides representative works, including methods that span multiple categories.

  • Historical development: Before deep learning, industrial anomaly localization largely relied on sparse coding and dictionary learning.Deep anomaly-localization methods began expanding around 2017.
  • Historical development: Early deep anomaly-localization methods used GAN models and autoencoder reconstruction networks.These approaches followed the broader success of deep learning in computer vision.
  • Survey organization: The survey reviews methods by high-level paradigm and breaks each category into representative works.A separate table assigns works according to the Venn-diagram categories.
  • Survey organization: Some anomaly-localization methods belong to multiple categories, so overlapping regions represent their cross-category components.The survey preserves these overlaps when organizing the literature.

A. Image Reconstruction-based Approach

Image reconstruction-based methods train on normal images and localize anomalies through differences between inputs and reconstructions. The survey groups improvements around network structure, latent representation, and loss functions, while noting limitations in generalization and semantic fidelity.

  • Core idea: Autoencoders reconstruct normal images through a low-dimensional latent bottleneck, using input–reconstruction differences as anomaly maps.The approach assumes anomalous regions are reconstructed as normal because training uses only normal images.
  • Network structure: Skip connections and multi-scale architectures improve reconstruction of normal data across datasets with differing domains and complexity.Skip-GANomaly captures multi-scale normal distributions, while related methods reconstruct or fuse features at multiple resolutions.
  • Latent representation: Latent-space methods constrain representations through memory banks, feature clustering, or explicit probability modeling.These strategies replace, structure, or model latent features to improve discrimination and reconstruction of normal patterns.
  • Loss functions: Pixel-level L1 and L2 losses lack semantic information, motivating SSIM and multi-scale gradient-based losses that emphasize structural differences.The survey reports that SSIM loss significantly improves anomaly localization on textured datasets compared with L2 loss.
  • Limitations: Reconstruction methods are intuitive and interpretable, but their performance depends on latent-layer expression of defect-free features and can fail on complicated textures or objects.Powerful generalization may reproduce anomalous features, whereas weak reconstruction can cause over-detection.

B. Generative Model-based Approach

Generative model-based approaches learn the distribution of defect-free data and use generation, reconstruction, or likelihood estimation to localize anomalies. The survey covers VAE, GAN, CycleGAN, and normalizing-flow variants, with normalizing flows reported as strongest for localization.

  • Overview: Generative models learn defect-free data distributions and use the learned distribution to generate or model new data for anomaly localization.The central requirement is obtaining the feature distribution of normal data explicitly or implicitly.
  • VAE: VAE methods impose a prior distribution in latent space, while attention and gradient-based variants derive anomaly maps or improve reconstruction quality.Attention maps use gradients of latent variables, and gradient-descent methods move toward normal data samples.
  • GAN: GAN approaches improve anomaly localization by modifying generator inputs, generator reconstruction constraints, or discriminator structure.GANomaly uses an encode-decode-encode structure, while f-AnoGAN reconstructs latent-space feature vectors with a fixed decoder.
  • CycleGAN: CycleGAN uses two generators and two discriminators to learn mappings between two feature domains while distinguishing real from synthesized images.Its multi-GAN structure supports domain-to-domain translation within the generative-model family.
  • Normalizing flow: 98.62% AUROC and 94.60% AUPRO are reported for CFLOW-AD on MVTec AD localization.The method uses conditional positional encoding within decoder coupling layers.
  • Comparison: Normalizing flows achieve the best localization result among the surveyed generative approaches, whereas VAE and GAN generation can produce poor normal-area reconstructions and false detections.Flows estimate exact likelihoods and measure test features against the estimated defect-free distribution.

C. Deep Feature Embedding-based Approach

Deep feature embedding methods extract representations and estimate anomalies by comparing target features with normal feature structure. The survey emphasizes knowledge distillation and feature modeling, while identifying fine-grained, noise-resistant, multi-scale, and alignment-aware localization as continuing goals.

  • Limitations: Reconstruction-based approaches can ignore image details and over-detect normal regions because they lack feature-level discriminatory information.The survey illustrates this failure with poorly reconstructed defect-free areas near a hazelnut base.
  • Overview: Deep feature embedding methods generate pixel-level anomaly maps by comparing target-image embeddings with embeddings from normal images.Feature extraction commonly uses ImageNet-pretrained or self-supervised models, followed by anomaly estimation.
  • Knowledge distillation: Knowledge distillation uses a pretrained teacher and student networks whose output differences provide anomaly scores.Uninformed Students uses one teacher and multiple students, but multi-patch localization increases computational burden.
  • Feature modeling: Deep feature modeling builds a feature space and estimates normality through clustering, probability fitting, or learned models without separating teacher and student networks.PaDiM models each feature location with a multivariate Gaussian and uses Mahalanobis distance to a standard feature template.
  • Representative results: 98.2% AUROC is reported for FYD on MVTec AD, although its heatmaps still contain interference and coarse defect regions.FYD combines coarse image alignment with fine dense-feature alignment through pixel-wise non-contrastive learning.
  • Research goals: Deep feature embedding research targets fine-grained, noise-resistant localization and extension to multi-scale anomalies and non-aligned datasets.These two issues are identified as central directions in the survey’s comparison of representative methods.

D. Self-Supervised Learning-based Approach

Self-supervised anomaly localization learns visual features from unlabeled images through proxy tasks or contrastive learning. Proxy-task methods focus on pretext-task design, whereas contrastive methods focus primarily on network design.

  • Overview: Self-supervised learning derives visual features from unlabeled images and applies them to anomaly localization.The survey distinguishes proxy-task and contrastive-learning approaches.
  • Proxy tasks: Proxy tasks predict or recover hidden image regions or properties, including image inpainting, relative position prediction, and attribute restoration.These three forms are identified as the primary proxy tasks used by recent self-supervised localization methods.

1) Proxy Tasks:

Proxy-task methods train models on synthetic or transformed inputs so they learn to reconstruct normal content or model spatial and attribute relationships for anomaly localization.

  • Image inpainting: Image inpainting repairs synthetic defects, encouraging networks to reconstruct normal regions and restore abnormal ones during testing.Synthetic defects include random noise or erased regions filled with a fixed color.
  • Limitations: More realistic defect synthesis does not necessarily yield better localization, so synthesis often must be paired with a suitable restoration network.The survey reports no relationship between synthesis realism and positioning results in Table 10.
  • Relative location prediction: Relative location prediction learns neighborhood spatial information by ordering surrounding patches around a central image block.PatchSVDD divides images into 3×3 regions and sorts the eight surrounding blocks.
  • Attribute restoration: Attribute restoration modifies hidden image attributes such as color or orientation before autoencoder reconstruction.The approach transforms the input attribute and then reconstructs the image with an autoencoder.
  • Contrastive predictive coding: Contrastive predictive coding detects and segments anomalies by learning common features among similar image patches and differences among non-similar instances.The image is split into patches, with each line of patches treated as a separate time step.

2) Contrast Learning:

Self-supervised and one-class approaches extend anomaly localization beyond direct reconstruction by learning invariant or compact representations from normal data.

  • Contrastive learning: 93.4% pixel-AUROC was achieved on MVTec AD by a SimCLR-based method contrasting normal samples with locally augmented images.The method generates a pair of negative images during training.
  • Contrastive learning: SimSiam-based localization uses two random transformations, a shared encoder, and a predictor to match representations across branches.This design exploits the Siamese network’s modeling invariance.
  • Future direction: Self-supervised learning frameworks are identified as a relevant future direction for validating anomaly localization potential.The survey summarizes their benefits and drawbacks in Table 8.
  • One-class classification: One-class localization divides images into patches and classifies them as normal or abnormal, enabling coarse anomaly regions.Deep SVDD maps training data into a small feature-space hypersphere, with outside points treated as anomalies.
  • Limitations and future direction: Patch-based anomaly detection can overlook subtle abnormal regions because it concentrates on the entire image’s semantic information.The survey identifies combining anomaly detection with self-supervision or pretrained deep embeddings as a promising improvement.

IV. EXPERIMENTS

Industrial anomaly-localization datasets vary substantially in scale, resolution, texture, defect coverage, and annotation structure, with MVTec AD the most commonly used benchmark.

  • Dataset overview: Five industrial anomaly-localization datasets differ significantly in image quantity, quality, resolution, and texture information.The survey presents them as available datasets for unsupervised learning-based localization.
  • NanoTWICE: NanoTWICE contains 45 nanofibrous-material images at 1024×3696-pixel resolution captured by scanning electron microscopy.Its background is non-cyclical continuous texture, and defect size varies.
  • MVTec AD: MVTec AD contains 15 categories with about 240 normal training images and 100 defective testing images per category.It includes ten object categories and five regular or random texture categories, with original resolutions from 700×700 to 1024×1024 pixels.
  • BTAD: BTAD contains 2,830 images across three classes with resolutions of 1600×1600, 600×600, and 800×600 pixels.Each class includes defect-free training and testing images, while defect types are not illustrated.
  • Fabric dataset: The Fabric dataset has 256×256 images across three patterns, with 25 defect-free and 25 defective samples per pattern.Its defective samples include five defect types and corresponding ground truth.
  • Dataset protocol: Typically, defect-free training samples are used to train anomaly-localization models, while mainly defective remaining samples form the test sets.This split reflects the unsupervised learning setup described for the listed datasets.
  • Kolektor datasets: KolektorSDD is relatively simple with one thin-scratch defect, whereas KolektorSDD2 is a complex, well-annotated real-world surface-inspection dataset.KolektorSDD2 uses color production images with fine-grained segmentation masks.

B. Evaluation Criteria

Anomaly localization is evaluated mainly with pixel AUROC and PRO, with IoU used less often; the survey cautions that AUROC can misrepresent fine-grained localization when anomalies are sparse.

  • Metrics overview: AUROC, PRO score, and IoU are the three primary metrics used for anomaly localization.AUROC is dominant, PRO addresses region overlap, and IoU treats localization as segmentation.
  • AUROC: High AUROC indicates reduced sensitivity to threshold variation when identifying anomalous pixels.Normal pixels are negatives, anomalous pixels are positives, and TPR and FPR quantify classification outcomes.
  • AUROC limitation: AUROC can overstate localization quality when only a tiny fraction of pixels is anomalous because numerous normal pixels suppress the false-positive rate.Methods can achieve about 97% pixel AUROC while still producing coarse anomaly maps with excessive background interference.
  • PRO score: PRO thresholds anomaly scores and computes relative overlap between each ground-truth connected component and the thresholded anomaly region.It is used because AUROC favors large anomalies.
  • Reported comparisons: Tables 10 and 11 report state-of-the-art MVTec AD localization results using pixel AUROC and PRO score, respectively.Figure 13 provides qualitative MVTec AD comparisons for STPM, PatchCore, PaDiM, and CFLOW-AD.
  • IoU: IoU evaluates anomaly localization as a segmentation task, but only very few studies use it.The survey cites works using IoU as an evaluation approach.

C. Performance Comparison

On MVTec AD, feature embedding and auxiliary-data strategies generally outperform pure reconstruction or generative approaches, while network-module choices and real-time constraints shape practical performance. Despite strong overall results, texture-rich Tile and Wood categories and semantically challenging Transistor remain difficult.

  • MVTec AD performance: 94.2% pixel AUROC is the highest reported result for RIAD, yet pure AE-based reconstructions and generative methods remain insufficient on MVTec AD.The survey contrasts RIAD's peak pixel AUROC with the broader weakness of pure reconstruction and generative approaches.
  • MVTec AD performance: 93.9%, 96.5%, and 95.0% pixel AUROC were reported by S-T, SPADE, and DFR, respectively, among representative feature embedding methods.Feature embedding methods improved through feature selection, attention, memory banks, self-organizing features, and feature alignment.
  • Method strategies: Self-supervised methods such as ANOSEG, NSA, and DRAEM outperform original AE-based methods, while contrastive methods provide competitive anomaly-region discrimination.The survey also notes that one-class classification methods are often time-consuming and can localize inaccurately because of local patch processing.
  • MVTec AD performance: Tile, Wood, and Transistor remain difficult MVTec AD datasets; most methods do not reach 95% AUROC on Tile and Wood or ideal performance on Transistor.Tile and Wood contain multi-scale, multi-type texture defects, while Transistor includes a missing defect type with high-level semantic information.
  • Network modules: Combining ViT or NF modules significantly improves localization accuracy over simple CNN modules in deep feature embedding-based approaches.CFLOW-AD and Fastflow, the two best algorithms in the cited comparison, both use NF, while UTRAD and InTra use ViT.
  • Additional data and runtime: Major methods using additional data outperform methods without extra data, and deep feature embedding approaches reach about 20 fps, including 27 fps for CFLOW-AD-WRN50.Additional data includes large pre-trained models, self-supervised synthesis, or proxy tasks; the speed comparison concerns MVTec AD inference.

V. CONCLUSION AND OUTLOOK

The survey synthesizes deep learning-based industrial anomaly localization and identifies unresolved challenges spanning anomaly types, datasets, evaluation, and emerging model directions.

  • The survey organizes recent industrial anomaly localization methods, datasets, performance comparisons, limitations, and future research directions.It highlights structural taxonomy and discusses unresolved issues after significant progress.
  • Functional anomalies: Localization performance drops substantially on datasets containing functional anomalies, which violate underlying scene or object constraints.Existing benchmarks emphasize textural defects such as scratches and dents, while functional defects include invalid object locations or missing required objects.
  • Releasing rich AL datasets: Public anomaly-localization datasets remain insufficiently rich and realistic for complex industrial scenarios.The survey calls for datasets varying lighting, perspective, proportions, shadows, blur, and other imaging conditions.
  • Vision transformer-based method: Vision transformers offer long-distance feature modeling, motivating their combination with normalizing-flow generation models for anomaly localization.The survey also identifies comprehensive modeling of multi-scale anomalous regions as an opportunity for vision-transformer methods.
  • Meaningful model evaluation: High pixel-AUROC can overlap with weak fine-grained localization, motivating boundary-aware modeling or IoU-based evaluation.The survey reports that visualization quality may remain poor despite high pixel-AUROC values.
  • Accurate anomaly types: Real industrial settings contain diverse anomaly types with differing importance, requiring methods that can discriminate among anomaly categories.Clustering anomaly types is described as an initial step rather than a complete solution.
  • Unsupervised 3D anomaly localization: The spread of 3D sensors makes unsupervised anomaly localization in 3D industrial scenes a relevant future direction.A publicly available MVTec 3D AD/AL dataset is cited as recent support for this direction.
Loading 2207.10298v1…