Source-linked AI summary
VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization
Pankaj Mishra, Riccardo Verk, Daniele Fornasier, Claudio Piciarelli, Gian Luca Foresti
TL;DR
Image anomaly detection needs methods that identify defects and, especially in industrial settings, localize anomalous regions. VT-ADL combines transformer patch embeddings, reconstruction, and Gaussian mixture modeling, while introducing the BTAD dataset. The reported results are at or above state-of-the-art performance, and noise improves one PRO-score configuration from 0.807 to 0.897.
Problem
Prior anomaly-detection methods commonly classify images globally, while fewer methods localize anomalous regions; industrial data also contain relatively few anomalies.
Method
VT-ADL combines patch-based vision-transformer encoding and reconstruction with Gaussian mixture density modeling of encoded normal-image features.
Results
The method performs at or above state-of-the-art techniques, while noise raises PRO score with 150 Gaussians from 0.807 to 0.897.
Takeaways & Limitations
The framework supports both global anomaly detection and localization, and the paper contributes BTAD as a real-world industrial anomaly dataset.
Abstract
from arXiv · showhide
We present a transformer-based image anomaly detection and localization network. Our proposed model is a combination of a reconstruction-based approach and patch embedding. The use of transformer networks helps to preserve the spatial information of the embedded patches, which are later processed by a Gaussian mixture density network to localize the anomalous areas. In addition, we also publish BTAD, a real-world industrial anomaly dataset. Our results are compared with other state-of-the-art algorithms using publicly available datasets like MNIST and MVTec.
I. INTRODUCTION
Industrial anomaly detection is important but often focuses on image-level classification rather than localizing defects. VT-ADL addresses this gap with a vision-transformer model and introduces the BTAD industrial dataset.
- Industrial anomaly detection supports applications including surveillance, defect segmentation, inspection, quality control, medical imaging, and financial transactions.
- Most prior methods learn normality using encoding-decoding schemes or latent-feature distributions, while fewer methods segment local anomalous regions.
- The model adapts a vision transformer for anomaly localization using Gaussian approximation of latent features.
- VT-ADL learns the manifold of normal-class data semi-supervisedly, requiring only normal images during training.
- BTAD contains 2,830 real-world images of three industrial products with body and surface defects.
II. RELATED WORK
Industrial image anomaly detection has shifted from classical methods toward deep learning, especially reconstruction-based approaches. Semi-supervised learning is prevalent because anomalous examples are rare and datasets are highly imbalanced.
- Classical anomaly detection used Bayesian networks, rule-based systems, and clustering algorithms before deep learning became prominent.
- Many deep-learning approaches reconstruct input images and assume models trained only on normal data reconstruct anomalies poorly.
- Fully supervised binary classification is uncommon because real-world anomaly datasets contain far fewer anomalies than normal examples.
- Most solutions use semi-supervised training in which only normal data are available during the training step.
III. PROPOSED MODEL
VT-ADL combines patch-based vision-transformer encoding, image reconstruction, and Gaussian mixture density modeling. Positional embeddings preserve patch locations so encoded features can support anomaly localization.
- A Gaussian mixture density network models the distribution of transformer-encoded features to estimate normal-data density in latent space.The model uses a Gaussian mixture with full covariance matrices and neural-network-estimated parameters.
- The input image is divided into patches, linearly embedded into D dimensions, and augmented with positional embeddings.For an image of resolution H × W with patch size P × P, the number of patches is N = HW/P^2.
- The patch sequence passes through a transformer encoder with multi-headed self-attention, an MLP block, layer normalization, and residual connections.The model omits dropout because it causes instability in the Gaussian approximation network.
- Encoded patches are projected into a reconstruction vector, which the decoder maps back to the original image shape.The decoder maps R^512 to R^(H×W×C).
- Softmax constrains Gaussian mixing weights to be nonnegative and sum to one, while softplus ensures positive standard deviations.The Gaussian means use a linear layer because they have no constraint.
IV. OBJECTIVE AND LOSSES
VT-ADL trains with reconstruction and Gaussian-mixture objectives: reconstruction losses encourage normal-image features, while latent-density modeling captures the normal-feature manifold. The final objective is a weighted sum of negative log-likelihood, MSE, and SSIM losses, with Gaussian noise added before GMM processing.
- Objectives: Training jointly reconstructs normal images and models the manifold of their encoded features with a Gaussian mixture density network.The decoder encourages features relevant to normal data, while the density network models where normal encoded features reside.
- Reconstruction losses: The reconstruction-based component combines Mean Squared Error and Structural Similarity Index losses.MSE measures pixel-wise differences, while SSIM incorporates visual properties that standard MSE can miss.
- Reconstruction losses: MSE compares the original input image X with the decoder output X̂ over image width W and height H.The loss is expressed using the squared Frobenius norm of the input–reconstruction difference.
- Density loss: The Gaussian estimation network fits its parameters by maximum likelihood and minimizes the negative conditional log-likelihood of normal training data.This trains the latent density model around normal-class features.
- Visualization: Figure 3 compares anomalous images, ground-truth masks, and the method’s anomaly scores and localization outputs on MVTec examples.The examples include bottle, cable, capsule, metal nut, and brush categories.
- Combined objective: The final objective is a weighted addition of the three losses, using λ1 = 5 and λ2 = 0.5 for all datasets.Gaussian noise N(0, 0.2) is added to transformer-embedded features before they enter the GMM for regularization.
V. EXPERIMENTAL RESULTS
The experimental section evaluates VT-ADL through dataset descriptions, training and testing procedures, comparative results, and the introduction of BTAD, a manually labeled industrial anomaly dataset.
- Experimental scope: The experiments report dataset use, training and testing procedures, comparative results, and the introduction of BTAD.BTAD contains normal and defective industry-oriented product images with pixel-wise manually labeled anomaly masks.
A. Datasets
The study evaluates anomaly detection on MNIST, MVTec, and BTAD, including both general benchmark data and real-world industrial products. BTAD uses normal-only training and provides pixel-wise masks for anomalous test images.
- MNIST: MNIST contains 60K grayscale handwritten-digit images, with one class treated as normal and all other classes as anomalies.The paper uses MNIST as a baseline dataset for comparison with state-of-the-art approaches.
- MVTec Dataset: MVTec contains 5,354 high-resolution color images spanning textures and objects with 70 types of real-world product anomalies.Images are scaled to 550 × 550 pixels and center-cropped to 512 × 512 pixels before model input.
- BTAD Dataset: BTAD contains RGB images of three industrial products, with 400, 1,000, and 399 training images for products 1, 2, and 3.Images are scaled to 512 during training, and anomalous images include pixel-wise ground-truth masks.
- BTAD Dataset: BTAD training uses normal-class data only, while testing combines reconstruction losses with the maximum log-likelihood loss for global anomaly detection.The supplied passage truncates after describing global anomaly detection.
- Training setup: The experimental setup reports training hyperparameters in Table I.The supplied passages identify the table and its title but do not provide the hyperparameter values.
B. Results
VT-ADL was evaluated on MNIST, MVTec, and BTAD using classification and localization metrics, matching or exceeding state-of-the-art methods in the reported comparisons.
- MNIST: On MNIST, VT-ADL almost always outperformed the compared anomaly detection techniques in AUC.Each row used one digit class as normal and the remaining classes as anomalies.
- MVTec: The MVTec evaluation reports PRO up to a 30% average per-pixel false-positive rate and measures overlap between ground-truth and predicted anomaly regions.The metric averages region overlap across multiple thresholds.
- MVTec: On MVTec, the proposed methods performed at par with recent state-of-the-art algorithms and outperformed them in 7 product categories.Comparative results were taken from prior work.
- BTAD: BTAD results report PR-AUC for VT-ADL alongside convolutional autoencoders trained with MSE and MSE+SSIM losses.BTAD is a real-world industrial dataset with pixel-wise anomaly masks.
- Configuration analysis: The Gaussian-approximation study plots PRO score as the number of Gaussians changes.The figure is used to examine mixture-model configuration.
C. Gaussian mixture model tuning
The authors tuned the Gaussian mixture size on MVTec and examined whether noise in transformer-encoded features improves generalization.
- Gaussian-mixture size: Increasing the number of Gaussians initially raises the PRO score, after which the score becomes constant.The trend was measured on MVTec using Fig. 4.
- Noise addition: With noise added, PRO score with 150 Gaussians is 0.897 versus 0.807 without noise.The reported comparison evaluates noise addition in transformer-encoded features.
VI. CONCLUSIONS
The paper concludes that VT-ADL combines reconstruction and patch-based learning for anomaly detection and localization, while introducing BTAD as a real-world industrial dataset.
- Conclusions: VT-ADL detects anomalies globally through reconstruction and localizes them using a Gaussian mixture model over encoded image patches.The framework combines transformer-based patch processing with reconstruction-based detection.
- Conclusions: The reported results are at par with or outperform other state-of-the-art techniques.The conclusion summarizes the comparative evaluations.
- Conclusions: The paper publishes BTAD, a real-world industrial dataset for anomaly detection.BTAD includes industrial product images and supports the anomaly detection task.