Source-linked AI summary
Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows
Marco Rudolph, Bastian Wandt, Bodo Rosenhahn
TL;DR
Manufacturing defects are rare, subtle, and often unknown in advance, making reliable supervised detection difficult. DifferNet estimates densities of multi-scale convolutional features with normalizing flows, derives transformation-aware anomaly scores, and backpropagates them for localization. The paper reports strong detection performance on MVTec AD and Magnetic Tile Defects, including competitive results with few training images.
Problem
Defect detection must handle rare, visually subtle anomalies whose types are often unknown and for which defective training examples may be unavailable.
Method
DifferNet applies normalizing-flow density estimation to multi-scale convolutional image features and combines likelihoods across transformations to score and localize defects.
Results
DifferNet achieves state-of-the-art detection performance on MVTec AD and Magnetic Tile Defects and remains competitive with few training examples.
Takeaways & Limitations
Likelihood-based scoring supports robust defect detection and label-free localization without requiring a large training set.
Takeaways & Limitations
Generative-model comparisons are framed around anomaly-type dependence, including sensitivity to defective-area size, structure, and frequency content.
Abstract
from arXiv · showhide
The detection of manufacturing errors is crucial in fabrication processes to ensure product quality and safety standards. Since many defects occur very rarely and their characteristics are mostly unknown a priori, their detection is still an open research question. To this end, we propose DifferNet: It leverages the descriptiveness of features extracted by convolutional neural networks to estimate their density using normalizing flows. Normalizing flows are well-suited to deal with low dimensional data distributions. However, they struggle with the high dimensionality of images. Therefore, we employ a multi-scale feature extractor which enables the normalizing flow to assign meaningful likelihoods to the images. Based on these likelihoods we develop a scoring function that indicates defects. Moreover, propagating the score back to the image enables pixel-wise localization. To achieve a high robustness and performance we exploit multiple transformations in training and evaluation. In contrast to most other methods, ours does not require a large number of training samples and performs well with as low as 16 images. We demonstrate the superior performance over existing approaches on the challenging and newly proposed MVTec AD and Magnetic Tile Defects datasets.
1. Introduction
DifferNet addresses defect detection when defective examples are unavailable and defect types are unknown. It estimates likelihoods from multi-scale image features, supports label-free localization, and achieves strong results with small training sets.
- Motivation: Manufacturing defects are difficult to detect because they are rare, visually subtle, and often unknown before production.Consequently, supervised approaches cannot reliably cover all defect types.
- Approach: DifferNet uses a normalizing flow to assign likelihoods to convolutional-network features and derive an anomaly score for each image.Multi-scale inputs improve feature descriptiveness, while multiple image transformations make scoring more flexible and robust.
- Localization: Anomaly localization requires no training labels, pixel-wise optimization, or sub-image detection.The method propagates likelihood-based information back to the input to identify anomalous regions.
- Results: DifferNet remains competitive with a low number of training examples.The contribution specifically emphasizes applicability to small training sets.
- Results: DifferNet achieves state-of-the-art detection performance on MVTec AD and Magnetic Tile Defects.These are challenging datasets for defect detection.
2. Related Work
Prior image anomaly-detection methods use pretrained features, generative models, or density estimators, but each has limitations for subtle defect detection. DifferNet combines multi-scale feature processing with normalizing-flow density estimation to address these challenges.
- Pretrained Networks: Pretrained-feature methods commonly use One-Class-SVM or nearest-neighbor scores, while localization may require costly evaluation over overlapping patches.These techniques are reported as insufficiently robust for defect detection.
- Generative Models: Generative anomaly detectors often rely on reconstruction or generation, but autoencoders can reconstruct anomalies too accurately because they generalize strongly.GAN-based methods similarly measure differences between original and reconstructed data.
- Generative Models: Generative-model performance depends heavily on anomaly size, structure, and frequency content, while instance-specific structures can dominate reconstruction error.The paper contrasts these constraints with DifferNet’s handling of more varied defect types.
- Normalizing Flows: Normalizing flows learn bijective transformations between data distributions and well-defined densities, allowing likelihood evaluation and sample generation.Their mappings operate in both directions through affine-transform layers.
- Normalizing Flows: Earlier normalizing-flow anomaly-detection work showed promise but did not address visual data.DifferNet applies density estimation to image features rather than directly modeling high-dimensional images.
3. Method
DifferNet estimates densities of pretrained multi-scale image features with a normalizing flow, then uses likelihoods and their transformations to detect and localize anomalies. The flow is trained by likelihood-based optimization and evaluated with transformation-averaged anomaly scores.
- Feature density estimation: DifferNet maps pretrained image features from feature space Y to latent space Z with a bijective normalizing flow and computes image likelihoods.The latent distribution is modeled as z ∼ N(0, I), giving every feature input a corresponding latent representation and likelihood.
- Flow architecture: The flow uses Real-NVP coupling layers composed of permutations followed by scale and shift operations.Each block splits the input into two parts, which regress multiplicative and additive components to transform their counterparts.
- Flow architecture: Exponential scaling preserves non-zero affine coefficients, while soft-clamping restricts scaling values to (−α, α) for stability.The flow’s scale subnetworks use differentiable functions and constrain scaling components to support convergence.
- Training: Training maximizes feature likelihood under the change-of-variables formulation, implemented as minimizing a negative log-likelihood loss.The Jacobian log determinant contributes to the loss through the sum of scaling coefficients before exponentiation.
- Scoring function: The anomaly score averages negative log-likelihoods across transformed images, classifying an image as anomalous when the score exceeds threshold θ.Transformations include rotations and brightness or contrast manipulations.
- Localization: Localization backpropagates negative log-likelihood to the input, blurs channel gradients with a Gaussian kernel, sums their absolute values, and averages rotated maps.This produces a gradient map indicating how strongly pixels influence anomaly-related error.
4. Experiments
The experiments evaluate DifferNet on MVTec AD and Magnetic Tile Defects, focusing on detection accuracy, score behavior, data requirements, and localization. Results show strong defect detection, meaningful anomaly scores and localizations, and robustness across training configurations and multimodal data.
- Detection: DifferNet outperforms existing methods in almost every MVTec AD category, with gains reaching 15%.It achieves at least 84% AUROC in all 15 categories.
- Detection: DifferNet achieves about 50% recall without false positives on 191 defect-free MTD test images.Its anomaly scores separate a large subset of defective samples from non-defective samples.
- Localization: The method produces accurate localizations across many anomaly types, sizes and shapes in MVTec AD.Localization is obtained by backpropagating the negative log-likelihood loss to the input image.
- Ablation Studies: Multi-scale features improve AUROC by 4.7%, while transformed training supports multi-transform inference, generalization and data augmentation.Transformations in training are useful when they are also applied during inference.
- Ablation Studies: Training on all 15 MVTec AD categories yields a mean AUROC of 90.2%, indicating that the method handles multimodal distributions.The normalizing flow uses 12 coupling blocks in this setting.
5. Conclusion
DifferNet detects defects using normalizing-flow density estimation over multi-scale image features and combines transformed-image likelihoods into a robust anomaly score. Its design also supports defect localization through image gradients, handles multimodal distributions, and does not require large training sets.
- DifferNet estimates feature densities with normalizing flows at multiple image scales to detect defects.
- Likelihoods from several transformations of one image are combined to compute a robust anomaly score.
- Image gradients from the scoring function enable defect localization without training labels or pixel-wise optimization.
- The method scales to multimodal distributions, resembling real-world settings.
- DifferNet does not require a large amount of training samples.