Source-linked AI summary
Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders
Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, Carsten Steger
TL;DR
Pixel-wise reconstruction errors can exaggerate edge-localization errors and miss structural defects whose intensities remain similar. The paper replaces them with SSIM-based reconstruction and anomaly measures, achieving significant gains over pixel-wise approaches across real-world fabric datasets and performance comparable to methods using image priors.
Problem
Pixel-wise reconstruction errors create large residuals around slight edge misalignments and miss structural defects when pixel intensities remain roughly consistent.
Method
The paper uses SSIM as both reconstruction loss and anomaly measure, comparing local image regions through luminance, contrast, and structural information.
Results
SSIM significantly improves unsupervised defect segmentation on nanofibrous and woven-fabric datasets, reaching performance on par with approaches using image priors.
Takeaways & Limitations
Structural-similarity metrics provide more useful defect residuals than per-pixel losses for defects expressed primarily through structural alterations.
Abstract
from arXiv · showhide
Convolutional autoencoders have emerged as popular methods for unsupervised defect segmentation on image data. Most commonly, this task is performed by thresholding a pixel-wise reconstruction error based on an $\ell^p$ distance. This procedure, however, leads to large residuals whenever the reconstruction encompasses slight localization inaccuracies around edges. It also fails to reveal defective regions that have been visually altered when intensity values stay roughly consistent. We show that these problems prevent these approaches from being applied to complex real-world scenarios and that it cannot be easily avoided by employing more elaborate architectures such as variational or feature matching autoencoders. We propose to use a perceptual loss function based on structural similarity which examines inter-dependencies between local image regions, taking into account luminance, contrast and structural information, instead of simply comparing single pixel values. It achieves significant performance gains on a challenging real-world dataset of nanofibrous materials and a novel dataset of two woven fabrics over the state of the art approaches for unsupervised defect segmentation that use pixel-wise reconstruction error metrics.
1. INTRODUCTION
Unsupervised defect segmentation trains on non-defective images, avoiding the annotation burden and incomplete defect coverage required by supervised methods. Standard per-pixel reconstruction errors are vulnerable to edge misalignment and structurally altered defects, motivating SSIM-based reconstruction accuracy.
- Supervised inspection methods require extensive defect annotations and prior knowledge of possible defect types, while defective samples may be scarce.
- Unsupervised defect segmentation trains exclusively on non-defective samples to identify defective regions in test images.
- Per-pixel ℓp distances produce high residuals from slight edge-localization errors and miss structural differences when pixel colors remain roughly consistent.
- SSIM compares local image regions and emphasizes visually salient reconstruction changes, enabling more accurate defect segmentation than pixel-wise ℓ2 residuals.
- The proposed approach reaches performance on par with unsupervised methods using additional priors such as handcrafted features or pretrained networks.
2. RELATED WORK
Prior unsupervised defect-segmentation methods use pretrained features, GANs, adversarially enhanced autoencoders, or variational latent-space measures. The paper identifies their shared reliance on pixel-wise assumptions and proposes SSIM to model local structural dependencies instead.
- Anomaly segmentation differs from anomaly classification because it targets subtle deviations within otherwise known image structures.
- Pretrained CNN feature clustering lacks guaranteed general applicability because useful features and layer selections may vary across tasks.
- GAN-based methods can be slow because latent-sample optimization requires many steps, and mode collapse may omit non-defective data modes.
- Adversarially improved autoencoders still use per-pixel defect comparisons and may require aligned inputs, which is impossible for many unstructured applications.
- VAE reconstruction probabilities inherit per-pixel-loss problems and can become a segmentation bottleneck when separate pixel-level forward passes are needed.
- SSIM replaces the mutual-independence assumption by comparing local patches through luminance, contrast, and structural statistics.
3. METHODOLOGY
The methodology uses autoencoders trained on defect-free images to reconstruct inputs and segment unseen defects from residual maps. It compares pixel-wise, variational, feature-matching, and SSIM-based approaches, motivating SSIM because it captures local luminance, contrast, and structural relationships.
- Autoencoders for Unsupervised Defect Segmentation: Autoencoders encode images through a low-dimensional bottleneck and decode them into reconstructions, with CNNs parameterizing both functions.Choosing d much smaller than the input dimensionality prevents simple copying and encourages meaningful feature extraction.
- Autoencoders for Unsupervised Defect Segmentation: Training exclusively on defect-free images enables segmentation of unseen defects by comparing each test image with its reconstruction.The comparison produces a spatial residual map over image locations.
- Variational Autoencoder: VAEs model latent variables probabilistically and can obtain spatial reconstruction probabilities by decoding multiple samples from the approximate posterior.The method evaluates per-pixel reconstruction probability from decoded latent samples.
- Feature Matching Autoencoder: Feature matching autoencoders add a feature-space discrepancy between input and reconstruction to the ℓ2 loss, while defect residuals remain based on per-pixel ℓ2 distance.The feature extractor may use early layers of a pretrained classification CNN, and λ weights the two loss terms.
- SSIM Autoencoder: VAEs and feature-matching autoencoders do not satisfactorily improve residual maps over deterministic ℓ2 autoencoders because their evaluation remains pixel-wise.The paper instead adapts both loss and evaluation to capture local interdependencies with SSIM.
- Structural Similarity: SSIM compares K × K patches through luminance, contrast, and structure components, weighted by α, β, and γ.Luminance uses mean intensity, contrast uses variance, and structure uses covariance; c1 and c2 stabilize the computation.
- Structural Similarity: Sliding a K × K window across an image yields a pixel-location SSIM map, and differentiability allows SSIM to serve as a gradient-based training loss.SSIM lies in [−1, 1] and equals 1 exactly when two patches are identical.
4. EXPERIMENTS
The experiments evaluate the proposed SSIM autoencoder on nanofibrous materials and two woven-fabric texture datasets using qualitative and ROC-based comparisons. SSIM reveals defects that pixel-wise methods miss and substantially improves segmentation performance, including a nanofibre AUC increase from 0.688 to 0.966.
- Datasets: The study introduces a public woven-fabric dataset with two textures, defect-free training images, defective test images, and pixel-accurate annotations.It contains 100 defect-free images per texture for training and validation and 50 defective images, including cuts, roughened areas, and contaminations.
- Training and Evaluation Procedure: Evaluation reconstructs overlapping 128×128 test patches, averages predictions to reduce striding artifacts, thresholds residual maps, and applies morphological opening.The stride is reduced to 30 pixels, and ROC curves are computed from pixel-level true- and false-positive rates.
- Qualitative Results: Only the SSIM residual map reveals defects accurately when both ℓ2 and SSIM autoencoders remove defects from woven-fabric reconstructions.The same qualitative behavior is observed on the NanoTWICE nanofibrous-material dataset.
- Quantitative Results: 0.688 to 0.966: changing only the loss function improves nanofibrous-material AUC and reaches performance comparable to a reported 0.974 state-of-the-art value.SSIM significantly outperforms the tested autoencoding architectures using per-pixel losses, while similar performance leaps occur on both texture datasets.
- Quantitative Results: At a 5% false-positive rate, more than 50% of nanofibre defects overlap the ground truth by more than 91%.This exceeds the 85% minimal overlap reported by Napoletano et al. for the same setting.
- Sensitivity and Defect Types: SSIM is insensitive to varied latent dimensions, window sizes, and training-patch sizes once the latent dimension is sufficiently large.With d = 500, k = 11, and 128×128 patches, a forward pass takes 2.23 ms on a Tesla GPU; SSIM also segments high- and low-contrast defects while reducing edge-alignment artifacts.
5. CONCLUSION
The paper finds that structural-similarity-based perceptual losses improve unsupervised defect segmentation over per-pixel residuals, including for structurally altered defects with roughly consistent intensities. SSIM often brings results from nearly unusable segmentations to performance comparable with approaches using additional image priors.
- Conclusion: Per-pixel losses fail when defects differ mainly in structure rather than intensity, while SSIM captures these anomalies.Treating pixels as mutually independent prevents per-pixel residuals from representing interdependencies between local image regions.
- Conclusion: Perceptual losses outperform commonly used per-pixel residuals, whereas VAE-based probabilistic errors and feature-matching regularization do not improve segmentation.Those alternatives do not address the problems caused by treating pixels as mutually independent.
- Conclusion: SSIM is less sensitive to small edge-location inaccuracies because it compares local patches using luminance, contrast, and structure.
- Conclusion: SSIM often improves segmentation from almost unusable results to performance on par with methods relying on pretrained-network image priors.
- Conclusion: Different SSIM autoencoder hyperparameter settings do not significantly alter NanoTWICE defect-segmentation performance.
- Conclusion: Metal-pin defects detectable through gray-value differences are found by both ℓ2 and SSIM, while structurally different defects are segmented well only by SSIM.