PaperlayerView exact revision on arXiv

Source-linked AI summary

DFR: Deep Feature Reconstruction for Unsupervised Anomaly Segmentation

Jie Yang, Yong Shi, Zhiquan Qi

arXiv:2012.07122v1cs.CV

TL;DR

Small, localized image anomalies are difficult to detect without anomaly priors, especially when defective training examples are scarce. DFR generates multi-scale, context-aware regional features from a pre-trained CNN and reconstructs them with an efficient convolutional autoencoder. The paper reports effective and efficient anomaly detection and state-of-the-art performance across benchmark object and texture categories, while identifying boundary effects as a practical limitation.

  • Problem

    Detecting and segmenting subtle anomalies in small, confined image regions is challenging when defective examples are scarce and training must rely on normal images.

  • Method

    DFR generates dense multi-scale regional representations from hierarchical pre-trained CNN features and detects anomalies by reconstructing those features with a deep convolutional autoencoder.

  • Results

    DFR achieves effective anomaly detection and state-of-the-art results across object and texture categories in extensive experiments.

  • Takeaways & Limitations

    Multi-scale regional representations are beneficial for anomaly detection, while feature scales can be selected to trade compactness against detection metrics.

  • Takeaways & Limitations

    The method can wrongly report anomalies near image boundaries when the foreground fills the image, partly because VGG19 zero-padding introduces out-of-image information.

Abstract

from arXiv · show

Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect visual variations, such as defects on manufacturing products. This paper proposes an effective unsupervised anomaly segmentation approach that can detect and segment out the anomalies in small and confined regions of images. Concretely, we develop a multi-scale regional feature generator that can generate multiple spatial context-aware representations from pre-trained deep convolutional networks for every subregion of an image. The regional representations not only describe the local characteristics of corresponding regions but also encode their multiple spatial context information, making them discriminative and very beneficial for anomaly detection. Leveraging these descriptive regional features, we then design a deep yet efficient convolutional autoencoder and detect anomalous regions within images via fast feature reconstruction. Our method is simple yet effective and efficient. It advances the state-of-the-art performances on several benchmark datasets and shows great potential for real applications.

I. INTRODUCTION

Unsupervised anomaly segmentation is difficult when defects are subtle, localized, and rare, motivating a method that uses multi-scale regional representations and deep feature reconstruction.

  • Manufacturing inspection often lacks sufficient defective samples and must therefore learn from normal product images alone.
  • Small, confined defects create subtle visual deviations that image-level anomaly detectors may fail to localize or segment.
  • Existing approaches include reconstruction-based methods that identify anomalies through large pixel-wise reconstruction errors.
  • Feature-based methods instead construct regional representations and model normality in feature space.
  • DFR generates dense multi-scale regional features from pre-trained CNNs and reconstructs them with an efficient convolutional autoencoder for anomaly localization.

II. RELATED WORK

Prior unsupervised anomaly segmentation methods primarily reconstruct images or model regional features, while DFR combines hierarchical CNN features, multi-scale regional representations, and feature reconstruction.

  • Unsupervised anomaly segmentation methods broadly comprise reconstruction-based and feature-based approaches.
  • Reconstruction-based methods train on normal images and detect anomalies using distances or likelihood-related signals from image reconstructions.
  • Feature-based methods represent local patches or regions and learn the distribution of normal regional features with machine learning models.
  • The benchmark literature reports that existing reconstruction-based and feature-based methods do not work consistently well on diverse anomaly-segmentation categories.
  • DFR extracts hierarchical CNN features, generates dense multi-scale regional representations, and reconstructs them for anomaly detection.

B. Multi-scale Regional Feature Generation

The regional feature generator aligns and aggregates hierarchical CNN feature maps into dense multi-scale descriptions whose receptive fields provide local and contextual information for anomaly detection.

  • The generator aligns hierarchical CNN feature maps and aggregates them into a common regional representation.
  • Aggregation smooths feature variations for robustness and controls the spatial size of the representation through convolution stride.
  • Aggregated feature maps are concatenated, with the resulting channel depth equal to the sum of the component channel depths.
  • Multi-scale regional features describe local characteristics while encoding spatial context from larger receptive fields.
  • Each representation scale corresponds to a CNN layer, while its spatial size corresponds to that layer’s receptive field; scales can be selected flexibly.

C. Deep Feature Reconstruction

The method uses an efficient convolutional autoencoder to compress and reconstruct dense multi-scale regional features. It is trained on regional representations from normal images using averaged pair-wise l2 reconstruction error.

  • C. Deep Feature Reconstruction: The convolutional autoencoder uses only 1 × 1 convolution and ReLU operations to compress dense multi-scale regional representations into a low-dimensional latent space.This design targets effective and fast anomaly detection despite the high dimensionality of the regional features.
  • C. Deep Feature Reconstruction: The autoencoder is trained solely on regional representations from normal images with averaged pair-wise l2-distance reconstruction loss.The reconstructed dense regional representation is compared with its ground-truth representation during training.
  • C. Deep Feature Reconstruction: The input and reconstructed representations are feature maps of size h_o × w_o × c_o, with each regional feature corresponding to a local image region.This preserves the spatial correspondence needed for later anomaly scoring and segmentation.

D. Anomaly Scoring and Segmentation

Anomalies are scored by regional feature reconstruction error and converted into pixel-level segmentations. The method assumes normal-only training cannot reproduce features from anomalous regions, then thresholds the resulting anomaly map.

  • D. Anomaly Scoring and Segmentation: The method compares each input regional feature map with its reconstruction to produce an anomaly score map, then thresholds it to segment anomalous regions.The anomaly map is first inferred from feature reconstruction differences and then binarized.
  • D. Anomaly Scoring and Segmentation: The anomaly score A_i,j(x) is the pair-wise reconstruction error at spatial location (i,j), producing a regional map with size h_o × w_o.Each score corresponds to one regional feature on the input feature map.
  • D. Anomaly Scoring and Segmentation: The regional anomaly map is bilinearly upsampled to the image size to obtain a pixel-wise anomaly map.This converts regional scores into image-resolution localization.
  • D. Anomaly Scoring and Segmentation: The method assumes anomalous regions produce large reconstruction errors because a normal-only autoencoder cannot reproduce their regional features.High reconstruction-error regions therefore receive high anomaly scores.
  • D. Anomaly Scoring and Segmentation: The final anomaly map is binarized using a threshold estimated from an acceptable false positive rate on normal data.For example, an acceptable FPR of 0.005 requires the threshold to limit wrongly classified normal pixels to 0.5 percent.

IV. EXPERIMENTS

The experiments compare the proposed approach with state-of-the-art unsupervised anomaly segmentation methods on the MVTec Anomaly Detection dataset. The evaluation section includes experimental comparisons and further analysis.

  • IV. EXPERIMENTS: The experiments compare the approach with state-of-the-art unsupervised anomaly segmentation methods and analyze the proposed approach in detail.The section is organized around comparative evaluation followed by analysis.
  • IV. EXPERIMENTS: The MVTec Anomaly Detection dataset contains 15 sub-datasets, including 10 object datasets and 5 texture datasets.It is described as a benchmark for unsupervised anomaly segmentation.
  • IV. EXPERIMENTS: Training sets contain only normal images, while testing sets contain both normal and anomalous samples.This train-test structure supports unsupervised anomaly segmentation evaluation.

2) Baselines:

The baselines span reconstruction-based models, deep-feature methods, and student–teacher approaches. The implementation uses pretrained VGG19 features and dataset-dependent autoencoder configurations.

  • 2) Baselines:: The comparison includes CAE methods using l2-distance or SSIM, GAN- and VAE-based reconstruction methods, CNN-FD, and student–teacher models.These baselines represent pixel reconstruction, generative reconstruction, deep-feature modeling, and student–teacher approaches.
  • 2) Baselines:: AE-l2 and AE-ssim detect anomalies by comparing input images with reconstructions using l2-distance and SSIM, respectively.Both approaches are based on convolutional autoencoders.
  • 2) Baselines:: AnoGAN and VAE-grad find a nearest normal image through generative reconstruction before computing differences from the test image.VAE-grad iteratively updates the VAE input by minimizing reconstruction loss.
  • 2) Baselines:: CNN-FD uses deep CNN features and kmeans clustering to model normality and infer anomalies.It uses a shallow clustering model over deep features.
  • 2) Baselines:: The implementation uses ImageNet-pretrained VGG19 features from its first 16 convolutional layers after removing the last 3 dense layers.CNN feature maps are taken from ReLU outputs of convolutional layers.
  • 2) Baselines:: The autoencoder latent dimension is selected with PCA to explain 90% of the variance, while concrete architecture parameters depend on the dataset and feature-map count.The autoencoder architecture is detailed in the appendix.

4) Training Details:

The method is evaluated with pixel-level and region-level segmentation metrics across object and texture categories, outperforming most baselines while retaining an efficient multi-scale pipeline. Results also indicate benefits from transferred deep features and using multiple feature scales.

  • Evaluation Metrics: ROC-AUC and normalized PRO-AUC evaluate segmentation across 7 object and 5 texture categories, with PRO-AUC weighting anomalous regions equally regardless of size.The method is tested using both 16 regional-representation scales and the first 12 scales.
  • Comparisons Against Baselines: Our method outperforms baselines on most categories for ROC-AUC and achieves overwhelming PRO-AUC results across all categories against AE-ssim, AE-l2, and CNN-FD.The exceptions for ROC-AUC are Tile and Transistor.
  • Comparisons Against Baselines: Averaged across categories, the method performs on par with ST-m and shows similar or better results on many categories than ST-m and ST-p65.This comparison concerns the reported ROC-AUC and PRO-AUC metrics.
  • Efficiency and Architecture: Unlike ST, the method realizes multi-scale anomaly detection with one forward pass and requires training only the convolutional autoencoder.ST requires multiple teacher–student networks, whereas the proposed pipeline can flexibly combine scales for specific applications.
  • Feature Representations: Transferred discriminative CNN features outperform reconstruction-based models, while shallow k-means clustering is less effective at exploiting those features.The comparison includes AE-ssim, AnoGAN, ST-m, ST-p65, and CNN-FD.
  • Multi-scale Modeling: Using all 16 feature scales improves average ROC-AUC and PRO-AUC over using fewer scales, supporting multi-scale modeling when anomaly priors are unavailable.The paper also reports that ST-m outperforms ST-p65 on average.

C. Analysis

The analysis evaluates how multi-scale regional representations affect anomaly-segmentation performance across object and texture categories. More scales generally help objects, while texture performance saturates earlier and can sometimes decline.

  • Effectiveness of the multi-scale representation: Across object categories, ROC-AUC and PRO-AUC improve as multi-scale representations include more scales.The evaluation compares representations from f{1:2} through f{1:16}.
  • Effectiveness of the multi-scale representation: Across texture categories, performance generally saturates at f{1:8} or f{1:12} and may slightly degrade with additional scales.Wood is an exception: ROC-AUC increases while PRO-AUC decreases as more scales are included.
  • Effectiveness of the multi-scale representation: Adding more and larger scales makes anomaly maps progressively approach ground truth, removes false anomalous regions, and refines true anomalous regions.The paper attributes this to encoding more spatial context for each image subregion.
  • Effectiveness of the multi-scale representation: Adding smaller scales improves average performance gradually but slightly, while the large-scale-only representation f{12} still provides satisfactory results.The analysis suggests each scale conveys distinct information, although large scales already contain substantial useful information.
  • Effectiveness of the multi-scale representation: The paper concludes that smaller scales can be dropped to build more compact models when maximum detection metrics are not required.Texture categories generally need fewer and smaller feature scales than object categories.

2) Boundary Effects:

The method can falsely report anomalies near image boundaries when the foreground fills the image. Reflection padding relieves these boundary effects and improves average performance.

  • Boundary Effects: Zero padding in the pre-trained VGG19 may inject out-of-image information into boundary features, making them difficult for the CAE to model.The issue is observed especially for texture categories whose foreground fills the image.
  • Boundary Effects: Reflection padding relieves boundary effects and improves averaged performance on both object and texture categories by about 1 percent.Reflection padding uses information from the image itself.

3) Inference Speed:

The inference-speed analysis tests multiple scale settings and reports that DFR remains fast while offering more compact alternatives with small performance degradations.

  • Inference Speed: Inference speed is averaged across object and texture testing categories under different multi-scale regional representations.The corresponding averaged performances are also reported with reflection padding.
  • Inference Speed: DFR reaches over about 100 frames per second even when 12 feature scales are used.This result is presented as evidence of practical applicability.
  • Inference Speed: Using fewer feature scales produces more compact and efficient models with only small performance degradations.The comparison is made across different multi-scale settings.

APPENDIX A MVTEC AD DATASET

The appendix identifies the MVTec AD dataset and describes the architecture used to compress and reproduce multi-scale regional representations.

  • MVTec AD Dataset: The appendix provides detailed statistics for the MVTec AD dataset.These statistics are referenced in Table VII.
  • MVTec AD Dataset: The deep convolutional autoencoder contains six convolutional layers using only 1×1 convolutions and ReLU activations.It compresses and reproduces the multi-scale regional representation f{1:L}(x).
  • Mvtéc AD Dataset: The latent feature dimension is estimated with PCA by retaining 90% of the variance from sampled regional features.A subset of regional features is randomly sampled for the estimate.
Loading 2012.07122v1…