Source-linked AI summary
DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection
Xuan Zhang, Shiyu Li, Xi Li, Ping Huang, Jiulong Shan, Ting Chen
TL;DR
Visual anomaly detection must identify anomalous images and pixels despite scarce, diverse anomaly examples, while existing student-teacher methods lack direct anomalous constraints and adaptive multi-level fusion. DeSTSeg combines synthetic anomalies, a denoising student encoder-decoder, a pretrained teacher, and a segmentation network. On a surface-anomaly benchmark, it reports improvements over prior state of the art across image-, pixel-, and instance-level tasks.
Problem
Visual anomaly detection is commonly trained as one-class classification and segmentation because anomalous samples are rare and diverse, while student-teacher methods lack direct anomalous constraints and empirically fuse multi-level features.
Method
DeSTSeg trains a denoising student on synthetic anomalous images to match teacher features from clean images, then trains a segmentation network to adaptively fuse student-teacher features.
Results
DeSTSeg outperforms the previous state of the art by 0.1% AUC for image-level anomaly detection, 5.6% AP for pixel-level anomaly localization, and 4.9% IAP for instance-level anomaly detection.
Takeaways & Limitations
The proposed denoising student-teacher network and segmentation network considerably boost performance across image-level, pixel-level, and instance-level anomaly tasks.
Takeaways & Limitations
Failure cases include noisy backgrounds where tiny fibers and stains are highlighted, and the acceptability of these detections is left for future work.
Abstract
from arXiv · showhide
Visual anomaly detection, an important problem in computer vision, is usually formulated as a one-class classification and segmentation task. The student-teacher (S-T) framework has proved to be effective in solving this challenge. However, previous works based on S-T only empirically applied constraints on normal data and fused multi-level information. In this study, we propose an improved model called DeSTSeg, which integrates a pre-trained teacher network, a denoising student encoder-decoder, and a segmentation network into one framework. First, to strengthen the constraints on anomalous data, we introduce a denoising procedure that allows the student network to learn more robust representations. From synthetically corrupted normal images, we train the student network to match the teacher network feature of the same images without corruption. Second, to fuse the multi-level S-T features adaptively, we train a segmentation network with rich supervision from synthetic anomaly masks, achieving a substantial performance improvement. Experiments on the industrial inspection benchmark dataset demonstrate that our method achieves state-of-the-art performance, 98.6% on image-level AUC, 75.8% on pixel-level average precision, and 76.4% on instance-level average precision.
1. Introduction
Visual anomaly detection is commonly trained with only normal data, but student-teacher methods may fail to separate anomalous features and typically fuse multi-level discrepancies empirically. DeSTSeg addresses these issues with denoising feature learning and a segmentation network for adaptive fusion.
- Motivation: Visual anomaly detection identifies corrupted images and anomalous pixels, but the rarity and diversity of anomalies make comprehensive anomalous training data difficult to acquire.The task is therefore formulated as one-class classification and segmentation using normal data for training.
- Motivation: Student-teacher methods train a student to mimic a pretrained teacher on normal samples, expecting feature discrepancies on unseen anomalies.Prior work aggregated discrepancies from multiple feature-pyramid levels, but anomalous inputs were not directly constrained during training.
- DeSTSeg: DeSTSeg trains a denoising student on synthetic anomalous inputs to match teacher features from corresponding clean images.The framework combines a pretrained teacher, a denoising student network, and a segmentation network.
- Evaluation: DeSTSeg outperforms state-of-the-art methods on image-level, pixel-level, and instance-level anomaly detection tasks on MVTec AD.The paper also reports ablation studies validating the proposed components.
- DeSTSeg: The segmentation network adaptively fuses multi-level student-teacher similarities instead of relying on empirical inference-time fusion.Synthetic anomalies provide supervision for learning the segmentation output.
2. Related Works
Related work addresses anomaly detection through reconstruction, density estimation, memory banks, knowledge distillation, and simulated anomalies. These approaches rely on normal data or pseudo-anomalies, with different limitations in reconstruction fidelity, distributional assumptions, and fine-grained anomaly detection.
- Anomaly Detection and Localization: Reconstruction methods train autoencoders, variational autoencoders, or generative adversarial networks on normal data and use reconstruction differences as pixel-level anomaly scores.Over-generalization can allow anomalous regions to be reconstructed accurately.
- Anomaly Detection and Localization: Parametric density methods model normal features with distributions such as multivariate Gaussians, while normalizing flows approximate more flexible distributions.The Gaussian assumption is described as too strict for some recent approaches.
- Anomaly Detection and Localization: Memory-based approaches build a memory bank from normal training data for anomaly detection.
- Knowledge Distillation: Knowledge-distillation methods use a pretrained teacher and trainable student on anomaly-free data, expecting anomalous features to differ between networks.Prior solutions include ensemble learning to improve discrimination across anomaly types.
- Anomaly Simulation: Anomaly simulation enables supervised training without anomalous data, but rotation and cutout perform poorly for fine-grained anomalous patterns.CutPaste is presented as a simple yet effective alternative.
3. Method
DeSTSeg combines a pre-trained teacher, a denoising student encoder-decoder, and a segmentation network. Synthetic anomalies supervise feature denoising and adaptive fusion of multi-level similarities for end-to-end localization.
- Architecture: DeSTSeg uses a pre-trained teacher network, a denoising student network, and a segmentation network.The student and teacher process paired inputs during training, while the segmentation network localizes anomalous regions.
- Denoising student-teacher training: The student receives anomalous images while the fixed teacher receives clean images, minimizing feature discrepancies to reconstruct normal features in feature space.The paired training design explicitly encourages distinct student and teacher features when both process anomalous inputs.
- Synthetic anomaly generation: Synthetic anomalies are generated online by masking normal images with binarized Perlin noise and blending external anomaly content using opacity β.The anomaly image combines masked external content and the original normal image; β is randomly selected between 0.15 and 1.
- Denoising student-teacher network: An encoder-decoder student is used because reconstructing normal features requires both local and global information across feature levels.The paper reports that a complete encoder-decoder student performed better in preliminary experiments than using the teacher as encoder with a reversed student decoder.
- Segmentation network: A segmentation network adaptively fuses multi-level teacher-student similarities using synthetic anomaly masks, rather than summing feature distances empirically.Frozen student and teacher features are upsampled, concatenated, and optimized with focal and L1 losses; inference produces an anomaly map and a top-T image score.
- Segmentation network: Bilinear downsampling with threshold 0.5 preserves contiguous binary anomaly regions without the over- or under-estimation caused by alternative implementations.The comparison includes floor, ceil, and nearest-neighbor implementations, which respectively nearly erase, thicken, or interrupt the mask.
4. Experiments
Experiments evaluate DeSTSeg on MVTec AD across image-, pixel-, and instance-level anomaly detection, with visual analyses and ablations of its design choices. Results show strong benchmark performance, while failure cases and category-specific trade-offs identify practical boundaries.
- Dataset: MVTec AD contains 15 categories with normal training images, anomalous and normal evaluation images, and pixel-level test annotations.The categories include 10 objects and 5 textures; image sizes range from 700 × 700 to 1024 × 1024 pixels.
- Image-level results: DeSTSeg reaches state-of-the-art average performance for image-level anomaly detection on MVTec AD.Image-level evaluation uses AUC, with category-specific results reported in supplementary material.
- Pixel-level results: 5.6% AP improvement over state-of-the-art is reported for pixel-level localization, with AUC comparable to PatchCore.The method achieves the highest or near-highest score in most categories, supporting generalization across industrial scenes.
- Instance-level results: 57.8% IAP@90 is achieved on average, meaning 57.8% pixel-level precision when 90% of anomaly instances are detected.The method reaches state-of-the-art performance for both IAP and IAP@90; hard categories can have relatively high standard deviations.
- Category-specific analysis: Higher-resolution methods perform better for small or thin anomalies in grid, screw, and tile, but require more memory and computation.Memory-based methods outperform DeSTSeg on cable, while DeSTSeg is comparable or better on the remaining categories.
- Failure analysis: Failure cases include ambiguous ground truths and noisy backgrounds that cause tiny fibers and stains to be highlighted.The authors note that some broad predictions may remain useful despite imperfect alignment with the ground truth.
- Ablation studies: Ablations evaluate denoising training, the student encoder-decoder, segmentation-network fusion, segmentation loss, and segmentation-network inputs.The L1 segmentation loss improves performance, while concatenated features and cosine-distance inputs are suboptimal relative to the element-wise product input.
5. Conclusion
DeSTSeg combines denoising student-teacher features with segmentation-guided adaptive fusion for anomaly detection. On a surface anomaly benchmark, it surpasses prior state-of-the-art across image-, pixel-, and instance-level tasks.
- DeSTSeg integrates a denoising student-teacher network with a segmentation network that adaptively fuses features.
- 0.1% AUC improvement over prior state-of-the-art is reported for image-level anomaly detection.
- 5.6% AP improvement over prior state-of-the-art is reported for pixel-level anomaly localization.
- 4.9% IAP improvement over prior state-of-the-art is reported for instance-level anomaly detection.