Source-linked AI summary
Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection
Nicolae-Catalin Ristea, Neelu Madan, Radu Tudor Ionescu, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B. Moeslund, Mubarak Shah
TL;DR
Anomaly detection commonly learns from normal data while using masked-information reconstruction errors to identify abnormal examples. The paper proposes SSPCAB, a self-supervised masked-convolution and channel-attention block, and reports performance improvements across image and video anomaly-detection frameworks and benchmarks. The authors also identify reduced gains when the block is placed near the input.
Problem
Anomaly detection must learn from normal training data, while existing masked-reconstruction functionality is not integrated as a flexible architectural building block.
Method
SSPCAB combines masked dilated convolution, channel attention, and its own reconstruction loss to predict masked receptive-field information from context.
Results
SSPCAB brings performance improvements in almost all integrated cases and enables new state-of-the-art levels on Avenue and ShanghaiTech.
Takeaways & Limitations
The block is generic enough to integrate into existing and future reconstruction methods, as well as frameworks that do not rely on reconstruction.
Takeaways & Limitations
SSPCAB does not bring significant gains when placed near the input, a limitation the authors aim to investigate in future work.
Abstract
from arXiv · showhide
Anomaly detection is commonly pursued as a one-class classification problem, where models can only learn from normal training samples, while being evaluated on both normal and abnormal test samples. Among the successful approaches for anomaly detection, a distinguished category of methods relies on predicting masked information (e.g. patches, future frames, etc.) and leveraging the reconstruction error with respect to the masked information as an abnormality score. Different from related methods, we propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block. The proposed self-supervised block is generic and can easily be incorporated into various state-of-the-art anomaly detection methods. Our block starts with a convolutional layer with dilated filters, where the center area of the receptive field is masked. The resulting activation maps are passed through a channel attention module. Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field. We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video, providing empirical evidence that shows considerable performance improvements on MVTec AD, Avenue, and ShanghaiTech. We release our code as open source at https://github.com/ristea/sspcab.
1. Introduction
The paper introduces SSPCAB, a self-supervised neural block that reconstructs masked receptive-field information and can be integrated into diverse anomaly-detection architectures. Across image and video benchmarks, the block improves detection performance, including Avenue RBDC rising from 41% to 62%.
- Anomaly detection is typically framed as one-class classification using normal training samples to identify unfamiliar test examples as anomalies.
- Masked-information methods use reconstruction error for missing patches, frames, or regions as an abnormality score.
- SSPCAB integrates masked-information reconstruction into a neural block, enabling masking at any network layer and incorporation into varied architectures.
- SSPCAB combines dilated masked convolution, channel attention, and a reconstruction loss that trains the block to predict masked areas from context.The dilation rate controls the context level from local to global, while attention recalibrates reconstruction maps.
- 41% to 62%: adding SSPCAB increases Liu et al.'s region-based detection criterion on Avenue, while experiments also report new state-of-the-art results on Avenue and ShanghaiTech.
2. Related Work
Related anomaly-detection methods learn normal behavior through dictionaries, changes, probabilities, distances, or reconstruction. SSPCAB differs by making masked reconstruction a reusable architectural block that integrates across existing frameworks.
- Because anomalies are difficult to anticipate, anomaly-detection methods are generally trained on normal data and evaluated on both normal and abnormal data.
- Existing anomaly-detection approaches include dictionary learning, change detection, probability-based, distance-based, and reconstruction-based methods.
- Reconstruction methods learn normal reconstruction manifolds from normal data, but neural-network generalization can reconstruct abnormal frames with low error.
- Masked-reconstruction approaches erase input information and predict it, using errors on masked patches, frames, or future content for anomaly detection.
- SSPCAB introduces reconstruction functionality as a basic neural building block based on masked convolution and channel attention, integrating into multiple anomaly-detection frameworks.
3. Method
SSPCAB predicts masked receptive-field regions using a masked dilated convolution followed by channel attention, while its self-supervised reconstruction objective can be combined with a host model's loss. The design uses contextual structure and is intended for integration across CNN architectures.
- SSPCAB is designed to predict masked information from contextual information while encouraging CNNs to learn the global structure of local patterns.
- The masked convolution uses four corner sub-kernels around a central masked region, with dilation controlling the receptive-field context.The receptive-field size is k = 2k′ + 2d + 1.
- At each location, masked convolution ignores the center, predicts every channel through separate filters, and produces an output tensor with the input's spatial dimensions.Zero-padding and stride 1 ensure that every input location is used as masked information.
- The channel-attention module globally pools activation maps, computes channel scale factors through fully connected layers, and recalibrates the resulting feature maps.The scale factors use sigmoid and ReLU transformations with a reduction ratio r.
- The block adds a self-supervised mean-squared reconstruction loss between input and output to the host model's objective.The coefficient λ controls the reconstruction loss's importance relative to the host loss.
4. Experiments and Results
Experiments evaluate SSPCAB on industrial-image and video anomaly benchmarks, including its hyperparameter choices and integration into existing models. Across these settings, SSPCAB generally improves detection and localization performance, with especially strong gains on Avenue and selected MVTec AD metrics.
- Data Sets: The evaluation covers MVTec AD for image anomalies and Avenue and ShanghaiTech for video anomalies.MVTec AD contains 15 categories, while Avenue and ShanghaiTech contain 21 and 107 test videos, respectively.
- Evaluation Metrics: Experiments use AP and AUROC for MVTec AD, and frame-level AUC, RBDC, and TBDC for video anomaly detection.The video evaluation reports both macro and micro AUC and includes region- and track-based criteria because frame-level AUC may not assess localization reliably.
- Preliminary Results: Preliminary experiments select MSE loss, k′ = 1, dilation rate d = 1, channel attention, and reduction rate r = 8 for subsequent experiments.Channel attention yields gains of 5.9% in micro AUC, 2.2% in macro AUC, and 4.6% in TBDC.
- Anomaly Detection in Images: On MVTec AD, SSPCAB improves detection AUROC by close to 1% for both CutPaste and DRAEM.For DRAEM localization, AP increases by 1.5%, from 68.4% to 69.9%, although overall localization AUROC does not improve.
- Abnormal Event Detection in Video: On Avenue and ShanghaiTech, SSPCAB produces consistent gains across several integrated models and enables new state-of-the-art results for several metrics.For Liu et al. [39] on Avenue, RBDC increases by 21.22%.
- Abnormal Event Detection in Video: On Avenue test video 18, integrating SSPCAB increases AUC by more than 5% and precisely localizes a person walking in the wrong direction.Figure 4 compares frame-level anomaly scores before and after integration; localization corresponds to the SSPCAB model.
5. Conclusion
The paper introduces SSPCAB, a self-supervised predictive block, and integrates it into multiple anomaly-detection models. Experiments show performance improvements across models and new state-of-the-art results on Avenue and ShanghaiTech.
- SSPCAB is a neural block composed of masked convolution and channel attention that predicts masked receptive-field regions.It is trained self-supervised with its own reconstruction loss.
- The block was integrated into a series of image and video anomaly-detection methods.
- SSPCAB improved performance in almost all evaluated cases across multiple models and benchmarks.Preliminary results indicate that both masked convolution and channel attention contribute to the gains.
- SSPCAB enabled new state-of-the-art results on Avenue and ShanghaiTech.
- Future work will extend SSPCAB with masked 3D convolution and investigate application domains beyond anomaly detection.
6. Supplementary
The supplementary material reports integration choices, ablations, qualitative improvements, runtime overhead, and discussion of SSPCAB's flexibility and limitations. Results include improved localization, higher anomaly-score quality, and less than 0.3 ms added runtime in two frameworks.
- Ablation Study: A single SSPCAB block was generally inserted in place of the penultimate convolutional layer because anomaly-detection datasets lack validation sets.The authors used one configuration to demonstrate generality and utility fairly.
- Ablation Study: Masked-kernel sizes of 1 × 1 and 3 × 3 produced no significant differences in preliminary Avenue experiments.
- Ablation Study: The tested kernel pattern masks the center and connects the receptive field to four corner sub-kernels, extrapolating from middle-frame prediction.
- Qualitative Anomaly Detection Results: SSPCAB qualitatively segmented object and texture anomalies better than baseline DRAEM on MVTec AD.
- Qualitative Anomaly Detection Results: Nearly 4% higher AUC was observed after integrating SSPCAB into Liu et al.'s method on Avenue test video 10.The first abnormal event also received higher frame-level anomaly scores.
- Qualitative Anomaly Detection Results: More than 10% AUC improvement was observed for ShanghaiTech test video 01 0054 after adding SSPCAB to Georgescu et al.'s framework.
- Qualitative Anomaly Detection Results: Almost 6% higher frame-level AUC was observed for ShanghaiTech test video 01 0130 after SSPCAB shifted detection toward the correct moment.
- Qualitative Anomaly Detection Results: ShanghaiTech examples showed precise localization and detection, with frame-level scores more correlated to ground-truth anomalies in one video.