Source-linked AI summary
Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection
Marco Rudolph, Tom Wehrbein, Bodo Rosenhahn, Bastian Wandt
TL;DR
The paper tackles automatic defect detection when defective examples are unavailable and unseen defect types may occur unpredictably. It introduces CS-Flow, a fully convolutional cross-scale normalizing flow for flexible density estimation on multiscale feature maps, achieving state-of-the-art image-level detection on MVTec AD and Magnetic Tile Defects.
Problem
Defect detection must learn from non-defective examples because defective parts may be unavailable and new defect types can occur unpredictably.
Method
CS-Flow jointly models multiscale pretrained feature maps with a fully convolutional normalizing flow that preserves spatial arrangement.
Results
CS-Flow sets a new state-of-the-art in image-level defect detection on the MVTec AD and Magnetic Tile Defects datasets.
Takeaways & Limitations
Preserved spatial structure enables defect localization, while cross-scale processing incorporates local and global image context.
Takeaways & Limitations
Autoencoders and GANs perform poorly on defect detection tasks because different anomalies have inconsistent reconstruction-error characteristics.
Abstract
from arXiv · showhide
In industrial manufacturing processes, errors frequently occur at unpredictable times and in unknown manifestations. We tackle the problem of automatic defect detection without requiring any image samples of defective parts. Recent works model the distribution of defect-free image data, using either strong statistical priors or overly simplified data representations. In contrast, our approach handles fine-grained representations incorporating the global and local image context while flexibly estimating the density. To this end, we propose a novel fully convolutional cross-scale normalizing flow (CS-Flow) that jointly processes multiple feature maps of different scales. Using normalizing flows to assign meaningful likelihoods to input samples allows for efficient defect detection on image-level. Moreover, due to the preserved spatial arrangement the latent space of the normalizing flow is interpretable which enables to localize defective regions in the image. Our work sets a new state-of-the-art in image-level defect detection on the benchmark datasets Magnetic Tile Defects and MVTec AD showing a 100% AUROC on 4 out of 15 classes.
1. Introduction
The paper addresses defect detection when defective examples are unavailable and defects can be previously unseen. It proposes CS-Flow, a fully convolutional normalizing flow that jointly models multiscale feature maps while preserving spatial information for detection and localization.
- Motivation: Defect detection must operate without defective training examples because defects can appear unpredictably in unknown forms.Manual inspection is costly and error-prone, while standard classification cannot handle formerly unseen defect types.
- Motivation: Existing approaches either lose information by averaging feature maps or require strong statistical priors that restrict density estimation.The paper motivates a more flexible representation and density model for fine-grained defect detection.
- Method: CS-Flow jointly estimates likelihoods on multiscale feature maps using a fully convolutional normalizing flow with cross-scale interactions.The method models feature distributions through a bijective mapping to latent variables with meaningful input likelihoods.
- Method: Preserving spatial arrangement makes the latent space interpretable and enables defective regions to be localized in images.The fully convolutional architecture retains spatial structure during density estimation.
- Results: The method achieves state-of-the-art image-level defect detection on the MVTec AD and Magnetic Tile Defects datasets.The paper reports this result as a principal contribution.
2. Related Work
Related work spans generative models, pretrained-feature methods, and normalizing flows for anomaly and defect detection. CS-Flow addresses limitations from reconstruction errors, restrictive feature distributions, lost spatial context, and costly multi-pass processing.
- Generative Models: Generative anomaly detectors identify anomalies through reconstruction failure, but autoencoders and GANs perform poorly on defect detection tasks.Different defect sizes, shapes, and structures produce inconsistent reconstruction errors; high-frequency structures and small defect areas are particularly problematic.
- Methods Based on Pretrained Networks: Pretrained-network methods extract universal features and commonly detect defects with traditional statistical approaches.Examples include one-class SVMs, unimodal Gaussian modeling with Mahalanobis distance, and patch-based multilevel feature modeling.
- Methods Based on Pretrained Networks: Gaussian feature models are limited because normal distributions can be inappropriate for feature distributions.The paper instead learns the feature distribution by maximum likelihood estimation without assuming a predefined distribution.
- Methods Based on Pretrained Networks: DifferNet applies normalizing flows after average pooling, losing contextual and positional information from full-sized feature maps.Its compensation strategy uses 64 rotated versions of each image, significantly increasing computational complexity.
- Normalizing Flows: Normalizing flows transform data bijectively into a tractable latent distribution, enabling exact likelihood evaluation and inverse generation.The paper enhances Real-NVP to operate across interacting multiple scales while avoiding fully connected and squeeze layers.
- Normalizing Flows: CS-Flow uses multiple feature scales and fully convolutional processing to retain semantic, contextual, and positional information for defect detection.Its coupling-block architecture is based on affine transformations and processes feature maps without fully connected layers.
3. Method
CS-Flow models defect-free image features with a bijective, fully convolutional flow that jointly processes multiple scales, estimates likelihoods, and preserves spatial information for localization.
- Feature extraction and density estimation: The method extracts multiscale feature maps from defect-free images and performs density estimation on those features rather than raw RGB images.A pretrained feature extractor remains fixed during training, while features from different image resolutions retain fine-grained positional and contextual information.
- Detection and localization: Image-level anomaly decisions use a likelihood threshold, while fully convolutional processing preserves positions for local defect scoring and visualization.Local anomaly scores aggregate feature values along channels, and regions with high output norms identify candidate defective areas.
- Cross-scale flow: CS-Flow transforms feature tensors bijectively and in parallel into a latent space with a multivariate standard normal target distribution.Cross-scale interactions share information between feature-map resolutions while jointly estimating the likelihood of their compound representation.
- Cross-scale flow: The flow is built from affine coupling blocks whose subnetworks regress element-wise scale and shift parameters for successive transformations.The coupling architecture is based on Real-NVP, with channel-wise splitting and permutation before the parts manipulate each other.
- Cross-scale flow: Soft-clamping restricts scale components to (−α, α), limiting extreme scaling produced by exponentiation.The restriction is applied to the scale outputs of the coupling subnetworks to preserve model stability.
- Learning objective: Training maximizes feature likelihood under the change-of-variables formulation, equivalently minimizing negative log-likelihood for the Gaussian latent density.The Jacobian determinant contribution simplifies to a sum over scale outputs, and gradient norms are limited to 1 for stabilization.
4. Experiments
Experiments evaluate CS-Flow on MVTec AD and Magnetic Tile Defects using image-level AUROC, likelihood distributions, localization visualizations, and ablations of multiscale processing and coupling depth.
- Datasets: The evaluation uses MVTec AD and Magnetic Tile Defects, with MVTec AD containing 10 object and 5 texture classes and diverse defect types.MTD includes grayscale magnetic-tile images with breaks and blowholes, alongside substantial variation among defect-free examples.
- Detection: 98.7% average AUROC on MVTec AD matched or exceeded competing methods in 12 of 15 categories.Image-level AUROC is used to compare defect detection performance, and the reported result considerably closes the gap to 100%.
- Detection: 99.3% AUROC on MTD established a new state of the art, with likelihoods clearly distinguishing defective from non-defective parts except for a few cases.The negative-log-likelihood histogram shows the two groups are almost completely separable, while the ROC comparison places performance 0.7% below an optimal ROC.
- Localization: The localization maps identify anomalous regions of varied sizes, colors, patterns, and shapes, but are not intended to provide pixel-precise segmentations.Maps are formed from the channel-wise sum of squared highest-scale outputs and can be dilated by convolutional receptive fields.
- Ablation Studies: Joint multiscale training outperformed single-scale and concatenated-multiscale baselines, with the concatenation strategy reducing AUROC by 0.7%.The results support the cross-convolutional multiscale procedure over upscaled feature concatenation.
- Ablation Studies: Performance improved with more coupling blocks up to nblocks = 4 and then saturated, while simultaneous training across all 15 MVTec AD classes achieved 98.2% average AUROC.The multi-class experiment indicates that the model handles multimodal normal-data distributions.
5. Conclusion
The paper presents a semi-supervised normalizing-flow method that detects and localizes defects using multiscale feature tensors. CS-Flow integrates cross-scale context to assign likelihoods and achieves state-of-the-art performance on MVTec AD and MTD.
- 5. Conclusion: CS-Flow detects and localizes defects by applying normalizing flows to feature tensors at different scales.Cross-convolution blocks integrate context within and between multiscale feature maps to assign likelihoods.
- 5. Conclusion: The method addresses weaknesses of approaches using overly simplified data representations or restricted distribution models.It leverages multiscale feature context while learning likelihoods for defect detection.
- 5. Conclusion: CS-Flow achieves state-of-the-art performance on the MVTec AD and Magnetic Tile Defects datasets.The conclusion identifies video anomaly detection as a possible direction for future refinement.