PaperlayerView exact revision on arXiv

Source-linked AI summary

PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization

Thomas Defard, Aleksandr Setkov, Angelique Loesch, Romaric Audigier

arXiv:2011.08785v1cs.CV

TL;DR

Industrial anomaly detection and localization must operate in a one-class setting because anomalous examples are rare and difficult to supervise, while existing approaches can be cumbersome or poorly scalable. PaDiM uses pretrained CNN patch embeddings and position-specific multivariate Gaussian distributions with semantic-level correlations. It achieves state-of-the-art performance on MVTec AD and STC, with low memory and time consumption and robustness on non-aligned data.

  • Problem

    Industrial anomaly detection and localization lack abundant anomalous training examples, while existing methods may require deep-network training or training-size-dependent K-NN inference.

  • Method

    PaDiM extracts patch embeddings from a pretrained CNN and models each patch position with a multivariate Gaussian distribution capturing correlations across CNN semantic levels.

  • Results

    PaDiM achieves state-of-the-art performance for anomaly detection and localization on MVTec AD and STC, including robust results on non-aligned data.

  • Takeaways & Limitations

    Low memory and time consumption, ease of use, and prediction complexity independent of training-set size support PaDiM's suitability for visual industrial control applications.

Abstract

from arXiv · show

We present a new framework for Patch Distribution Modeling, PaDiM, to concurrently detect and localize anomalies in images in a one-class learning setting. PaDiM makes use of a pretrained convolutional neural network (CNN) for patch embedding, and of multivariate Gaussian distributions to get a probabilistic representation of the normal class. It also exploits correlations between the different semantic levels of CNN to better localize anomalies. PaDiM outperforms current state-of-the-art approaches for both anomaly detection and localization on the MVTec AD and STC datasets. To match real-world visual industrial inspection, we extend the evaluation protocol to assess performance of anomaly localization algorithms on non-aligned dataset. The state-of-the-art performance and low complexity of PaDiM make it a good candidate for many industrial applications.

I. INTRODUCTION

PaDiM addresses one-class anomaly detection and localization for industrial inspection, where anomalous examples are scarce and existing approaches face training or scalability challenges. It uses pretrained CNN embeddings, patch-level Gaussian modeling, and cross-level semantic correlations, achieving strong performance on MVTec AD and STC.

  • Motivation: Anomaly localization assigns scores to pixels or patches, producing interpretable anomaly maps beyond image-level anomaly detection.The paper illustrates PaDiM anomaly maps on MVTec AD images.
  • Motivation: One-class learning trains only on normal images because anomalous examples are frequently unavailable and may contain unexpected patterns.At test time, examples differing from the normal training distribution are classified as anomalous.
  • Problem: Existing joint detection-localization methods either require cumbersome deep-network training or incur training-size-dependent K-NN costs.These issues may hinder deployment in industrial contexts.
  • Contribution: PaDiM uses a pretrained CNN for embedding extraction and represents each patch position with a multivariate Gaussian distribution.The Gaussian parameters are learned from patch embeddings across normal training images and pretrained CNN layers.
  • Contribution: PaDiM models correlations between different semantic levels of a pretrained CNN to support anomaly localization.The approach is designed to avoid the scalability issues of K-NN-based inference.
  • Results: PaDiM outperforms existing state-of-the-art methods for anomaly detection and localization on the MVTec AD and STC datasets.The paper also evaluates localization under non-aligned conditions.

II. RELATED WORK

Prior anomaly methods reconstruct normal images or compare learned embeddings with normal references. PaDiM instead models patch embeddings with Gaussian distributions that capture semantic-level correlations while keeping prediction complexity independent of training-set size.

  • Method categories: Anomaly detection and localization methods are categorized as reconstruction-based or embedding similarity-based.The distinction organizes the related approaches discussed by the paper.
  • Reconstruction-based methods: Reconstruction-based methods train autoencoders, variational autoencoders, or GANs on normal images and detect anomalies through reconstruction failure.Reconstruction error can serve as an image-level anomaly score.
  • Embedding similarity-based methods: Embedding similarity-based methods compare image or patch embeddings with references representing normality, but image-level methods often lack localization interpretability.Normal references may be centers, Gaussian parameters, or the full set of normal embeddings.
  • Embedding similarity-based methods: SPADE uses the entire set of normal embeddings with K-NN and reports strong localization results, but its prediction cost grows with the training dataset.This motivates a more compact normality representation.
  • PaDiM: PaDiM models patch embeddings with Gaussian distributions that capture correlations across pretrained CNN semantic levels, making prediction complexity independent of training-set size.The paper reports state-of-the-art performance using this representation.

III. PATCH DISTRIBUTION MODELING

PaDiM extracts spatially corresponding patch embeddings from multiple pretrained CNN layers, optionally reduces their dimensionality, and uses the resulting vectors for anomaly-map generation.

  • Embedding extraction: PaDiM avoids neural-network optimization by using a pretrained CNN to generate patch embedding vectors.Activations from different layers are concatenated to encode multiple feature resolutions and semantic levels.
  • Embedding extraction: Each grid position in the largest activation map is associated with an embedding vector formed from its spatially corresponding CNN activations.The grid has W×H positions, and each position corresponds to a non-overlapping pixel patch at input resolution.
  • Dimensionality reduction: Randomly selecting embedding dimensions is more efficient than PCA for reducing vector size while maintaining state-of-the-art performance.The reduction decreases training and testing complexity.
  • Inference: Test-image patch embeddings are used with the learned parametric representation to produce an anomaly map.The supplied passage introduces the final inference step without detailing its full computation.

B. Learning of the normality

For each spatial patch position, PaDiM assumes normal embeddings follow a multivariate Gaussian distribution and estimates its mean and covariance from normal training images. Covariance regularization ensures invertibility, while cross-level correlations encode relationships among CNN semantic levels.

  • Normality modeling: PaDiM collects the spatially corresponding patch embeddings from N normal training images for each position.The resulting set is denoted Xij.
  • Normality modeling: Each position-specific embedding set is modeled as a multivariate Gaussian N(µij, Σij), with µij as its sample mean and Σij as its sample covariance.Thus, every possible patch position receives its own Gaussian parameters.
  • Covariance estimation: The regularization term ϵI makes each sample covariance matrix full rank and invertible.This supports the subsequent distance computation.
  • Semantic-level correlations: Because embeddings combine semantic levels, each Gaussian covariance matrix captures inter-level correlations.The paper reports that modeling these relationships improves anomaly localization performance.

C. Inference : computation of the anomaly map

PaDiM assigns each test-image patch a Mahalanobis-distance anomaly score against its learned normal distribution, forming an anomaly map whose maximum scores the image. AUROC and PRO-score evaluate localization, while MVTec AD and STC provide industrial and non-aligned settings.

  • Inference: Mahalanobis distance compares each test patch embedding with the learned Gaussian distribution for its position.The distance produces a patch-level anomaly score.
  • Inference: High values in the Mahalanobis-distance matrix identify anomalous areas, and the image anomaly score is the map maximum.This avoids computing and sorting large numbers of distances across the full training set at test time.
  • Evaluation: AUROC measures pixel-level anomaly classification, while PRO-score evaluates connected-region overlap across false-positive rates from 0 to 0.3.PRO-score complements AUROC because AUROC favors large anomalies.
  • Datasets: MVTec AD evaluates one-class industrial quality control with aligned object and texture classes, while STC evaluates surveillance anomalies across variable pedestrian positions.STC contains normal training sequences and anomalous testing events such as vehicles in pedestrian areas or fighting.

B. Experimental setups

The experiments use ImageNet-pretrained CNN backbones and combine embeddings from multiple semantic levels, with random dimensionality reduction. PaDiM and SPADE share preprocessing, while a separately trained VAE supplies a reconstruction-based baseline.

  • Backbones: PaDiM uses ImageNet-pretrained ResNet18, Wide ResNet-50-2, and EfficientNet-B5 backbones.ResNet models extract embeddings from their first three layers; EfficientNet-B5 uses layers 7, 20, and 26.
  • Backbones: Embedding vectors are extracted from multiple semantic levels to combine CNN information while retaining resolution for localization.The model names encode the backbone and dimensionality-reduction method.
  • Dimensionality reduction: Random dimensionality reduction is applied in the experimental configurations.For example, PaDiM-R18-Rd100 denotes ResNet18 with random reduction to 100 dimensions.
  • Preprocessing: MVTec images are resized to 256x256 and center-cropped to 224x224, whereas STC images are resized to 256x256 only.Images and localization maps use bicubic interpolation, and anomaly maps use a Gaussian filter with σ = 4.
  • Baselines: The VAE baseline uses a ResNet18 encoder, an 8x8 convolutional latent variable, augmentation, and 100 Adam-training epochs.It is trained separately on each MVTec AD class using 10,000 images.

V. RESULTS

The experiments test semantic-level correlation and dimensionality reduction for anomaly localization. Results show that correlation improves layer aggregation, while random feature selection preserves performance more effectively than PCA at reduced dimensions.

  • Inter-layer correlation: Table I reports anomaly-localization AUROC and PRO-score for different semantic-level CNN-layer configurations on MVTec AD.The table evaluates how layer selection affects localization.
  • Inter-layer correlation: 1.1p.p. AUROC and 1.8p.p. PRO-score gains show PaDiM-R18 outperforming simple Layer 1+2+3 aggregation.The comparison supports modeling correlations between semantic levels rather than simply summing layer outputs.
  • Dimensionality reduction: Table II compares PCA and random feature selection after reducing 448-dimensional embeddings to 100 or 200 dimensions.The reported tuples are AUROC% and PRO-score% on MVTec AD.
  • Dimensionality reduction: Random dimensionality reduction outperforms PCA by at least 1.3p.p. in AUROC and 1.2p.p. in PRO-score at equal dimensions.The comparison covers all MVTec AD classes.
  • Dimensionality reduction: 0.4p.p. AUROC and 0.3p.p. PRO-score drops result when random reduction shrinks embeddings to 100 dimensions.The reduction substantially lowers PaDiM’s time and space complexity.

B. Comparison with the state-of-the-art

PaDiM achieves strong anomaly-localization and detection results across MVTec AD and STC, including non-aligned surveillance imagery. Its lightweight R18-Rd100 model remains competitive, while EfficientNet-B5 leads image-level detection.

  • MVTec AD localization: PaDiM-WR50-Rd550 outperforms competing methods on average for both MVTec AD localization metrics.On texture classes, it exceeds SPADE by 4.8p.p. in PRO-score and 4.0p.p. in AUROC.
  • MVTec AD localization: PaDiM-R18-Rd100 exceeds all compared models in average MVTec AD AUROC by at least 0.2p.p.This is the lighter PaDiM configuration.
  • STC localization: PaDiM-R18-Rd100 achieves the best STC AUROC with a 2.1p.p. margin.The result is reported for a dataset with highly variable pedestrian positions.
  • Image-level detection: PaDiM-EfficientNet-B5 outperforms every compared image-level detector by at least 2.6p.p. in average AUROC.Unlike MahalanobisAD, PaDiM also produces anomaly segmentation.

C. Anomaly localization on a non-aligned dataset

On non-aligned Rd-MVTec AD images, PaDiM-WR50-Rd550 outperforms the compared models on texture and object classes, while its AUROC declines less than VAE and SPADE.

  • C. Anomaly localization on a non-aligned dataset: PaDiM-WR50-Rd550 outperforms SPADE and VAE on both texture and object classes in PRO-score and AUROC.The experiment averages results across five randomly preprocessed versions of MVTec AD.
  • C. Anomaly localization on a non-aligned dataset: 5.3p.p is the AUROC decline for PaDiM-WR50-Rd550, versus 12.2p.p for VAE and 8.8p.p for SPADE on non-aligned data.
  • C. Anomaly localization on a non-aligned dataset: The results indicate that PaDiM-WR50-Rd550 is more robust to non-aligned images than the other tested methods.

D. Scalability gain

PaDiM trades linear training-time scaling for dataset-size-independent inference and memory requirements, while remaining competitive in speed and adaptable to resource limits.

  • D. Scalability gain: PaDiM training time scales linearly with dataset size because Gaussian parameters use the entire training dataset.It avoids deep neural-network training by using a pretrained CNN.
  • D. Scalability gain: PaDiM inference is around seven times faster than SPADE with an equivalent backbone on MVTec AD.PaDiM-R18-Rd100 has inference time in the same order of magnitude as the VAE implementation while outperforming VAE methods.
  • D. Scalability gain: PaDiM space complexity is independent of training-set size and depends only on image resolution.The model stores the pretrained CNN and Gaussian parameters for each patch.
  • D. Scalability gain: PaDiM-WR50-Rd550 requires seven times less memory than SPADE on the larger STC dataset.

VI. CONCLUSION

PaDiM is a distribution-modeling framework for one-class anomaly detection and localization. It achieves state-of-the-art performance, shows robustness on non-aligned data, and combines low resource use with ease of use for industrial control.

  • VI. CONCLUSION: PaDiM achieves state-of-the-art anomaly detection and localization performance on the MVTec AD and STC datasets.
  • VI. CONCLUSION: PaDiM results show robustness on more realistic non-aligned data.
  • VI. CONCLUSION: Low memory and time consumption plus ease of use make PaDiM suitable for applications such as visual industrial control.
Loading 2011.08785v1…