Source-linked AI summary
CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection
Muhammad Zaigham Zaheer, Arif Mahmood, Marcella Astrid, Seung-Ik Lee
TL;DR
Weakly supervised anomaly detection must learn from video-level labels despite rare anomalies and normal segments within anomalous videos. CLAWS combines random batch training, normalcy suppression, and clustering distance learning, achieving 83.03% frame-level AUC on UCF-Crime and 89.67% on ShanghaiTech.
Problem
Weakly supervised anomaly detection must handle video-level labels in which anomalous videos may contain numerous normal scenes, creating noisy segment-level supervision.
Method
CLAWS uses randomly selected batches, full-batch normalcy suppression, and a clustering distance loss to learn more discriminative normal and anomalous representations.
Results
83.03% frame-level AUC is obtained on UCF-Crime and 89.67% on ShanghaiTech, outperforming existing state-of-the-art approaches.
Takeaways & Limitations
The framework demonstrates state-of-the-art results on the UCF-Crime and ShanghaiTech datasets using only video-level labels.
Takeaways & Limitations
Qualitative analysis shows that anomaly scores can remain high after an annotated explosion ends because abnormal aftermath scenes may outlast the dataset’s annotations.
Abstract
from arXiv · showhide
Learning to detect real-world anomalous events through video-level labels is a challenging task due to the rare occurrence of anomalies as well as noise in the labels. In this work, we propose a weakly supervised anomaly detection method which has manifold contributions including1) a random batch based training procedure to reduce inter-batch correlation, 2) a normalcy suppression mechanism to minimize anomaly scores of the normal regions of a video by taking into account the overall information available in one training batch, and 3) a clustering distance based loss to contribute towards mitigating the label noise and to produce better anomaly representations by encouraging our model to generate distinct normal and anomalous clusters. The proposed method obtains83.03% and 89.67% frame-level AUC performance on the UCF Crime and ShanghaiTech datasets respectively, demonstrating its superiority over the existing state-of-the-art algorithms.
1 Introduction
The introduction frames weakly supervised anomaly detection as learning from video-level labels despite normal segments inside anomalous videos and challenges from correlated training. CLAWS addresses these issues with random batch selection, normalcy suppression, and clustering-based representation learning, achieving strong results on two datasets.
- Motivation: Video-level labels avoid costly temporal and spatial annotations, but anomalous videos can contain numerous normal scenes that create noisy supervision.This setting motivates weakly supervised anomaly detection without fine-grained manual annotations.
- Motivation: Existing weakly supervised formulations face rigid segment-count requirements or data correlation from training on complete videos.MIL assumes equal numbers of segments, while whole-video iterations can strengthen correlation, especially for stationary-camera datasets.
- CLAWS contributions: Randomly selected batches of temporally consecutive segments reduce inter-batch correlation while preserving temporal consistency within each batch.Several batches may be extracted from one video, and each iteration samples a batch across the training dataset.
- CLAWS contributions: Normalcy suppression uses full-batch information to reduce normal-feature activation and lower anomaly scores for normal portions of inputs.For inputs containing anomalous portions, it suppresses normal regions while retaining high-anomaly regions; for all-normal inputs, suppression distributes across the batch.
- CLAWS contributions: The clustering distance loss encourages distinct normal and anomalous representations by minimizing or maximizing inter-cluster distance for normal or abnormal videos.The loss operates on intermediate backbone representations to improve discrimination between normal and abnormal events.
- Results: 83.03% frame-level AUC is achieved on UCF-Crime and 89.67% on ShanghaiTech, outperforming existing state-of-the-art approaches.These are the reported headline results for the proposed framework.
2 Related Work
Related work includes one-class anomaly detection, reconstruction and pseudo-supervised approaches, and video-level weakly supervised methods. CLAWS is positioned closest to MIL and noisy-label learning while differing in its batch-based training and added suppression and clustering mechanisms.
- Anomaly detection paradigms: One-class methods learn normal representations and flag deviations as anomalies, but limited coverage of normal behavior can produce false alarms.Reconstruction-based and pseudo-supervised methods provide alternative ways to model normal data or create synthetic anomalies.
- Video-level weak supervision: Multiple Instance Learning treats each video as a bag of segments and ranks the top-scoring segments from normal and anomalous bags.This formulation is a principal precedent for video-level weak supervision.
- Video-level weak supervision: Noisy-label learning addresses normal segments inside anomalous videos, with prior graph-convolutional training performed on one complete video per iteration.CLAWS contrasts this complete-video training with its batch-based organization.
- CLAWS positioning: CLAWS differs from these approaches through batch-based training, normalcy suppression using noise-free normal labels, and a clustering distance loss.Its batches correspond to portions of training videos rather than complete videos.
3 Proposed Architecture
CLAWS Net organizes temporally ordered video features into randomly selected batches, then combines a backbone predictor with normalcy suppression and clustering-based learning. Its objective integrates prediction, clustering-distance, temporal-smoothness, and sparsity losses for weakly supervised anomaly detection.
- Training Data Organization: Video features are extracted from non-overlapping segments, arranged into equal-sized temporally ordered batches, and assigned their parent video's binary label.Videos may produce different numbers of batches, while each batch preserves within-batch temporal order.
- Training Data Organization: Random Batch Selection samples batches arbitrarily across the training dataset, reducing inter-batch correlation while retaining within-batch temporal consistency.The method uses batch-level temporal information for classification and reports improved backbone performance when consecutive-batch order is broken.
- Backbone Network: The backbone network maps each input batch to sigmoid anomaly scores, using video-level labels that assign all batch elements normal or abnormal targets.It comprises two fully connected modules with ReLU and dropout, followed by a sigmoid output layer.
- Normalcy Suppression: Normalcy suppression modules collect information across a batch and learn to suppress anomaly scores for normal portions rather than highlighting them.For normal-video batches, the suppression mechanism distributes probabilities across the batch to minimize values over all feature dimensions.
- Clustering Loss Module: The clustering loss groups intermediate video representations into normal and anomalous clusters, minimizing their distance for normal videos and maximizing it for abnormal videos.Cluster centers are distance-normalized by the video's segment count, and k-means assignments provide the distance used during training.
- Training Losses: The overall training objective combines regression, clustering-distance, temporal-smoothness, and sparsity losses.Temporal smoothness penalizes changes between adjacent predictions, while sparsity reflects the rarity of anomalous events.
4 Experiments
Experiments evaluate CLAWS on UCF-Crime and ShanghaiTech using frame-level AUC, including comparisons with prior work, ablations, and qualitative analyses. The full system reaches 83.03% AUC on UCF-Crime and outperforms the comparison on ShanghaiTech by 13.23% with C3D features.
- Datasets: UCF-Crime contains 128 hours of surveillance video, while ShanghaiTech provides 437 videos across 13 locations under varied lighting and camera angles.The reorganized ShanghaiTech protocol mixes normal and anomalous videos for weakly supervised evaluation.
- Evaluation metric: Frame-level AUC is computed from ROC curves using frame-level test annotations, with larger AUC indicating better discrimination.EER is avoided because normal frames greatly outnumber anomalous frames.
- Experiments on UCF-Crime: CLAWS Net shows superior performance to compared state-of-the-art methods on UCF-Crime using video-level labels.Random batch selection raises BBN performance to 75.95%, while the complete system reaches 83.03% AUC after adding its modules and losses.
- Experiments on ShanghaiTech: 13.23% margin over Zhong et al. is obtained on ShanghaiTech when both methods use similar C3D features.CLAWS also exceeds Zhong et al.'s TSN-based result by 5.23%.
- Ablation: Ablations increase UCF-Crime AUC from 69.5% with BBN to 75.95% with RBS, 78.60% with NSM-1, 80.94% with NSM-2, 81.53% with added losses, and 83.03% with clustering loss.The study uses both bottom-up additions and top-down removals to assess component contributions.
- Qualitative analysis: Normalcy suppression increases separation between normal and anomalous scores, while the complete system is more stable and discriminative in most qualitative cases.One unsuccessful case retains high anomaly scores after an explosion ends, although the aftermath may remain visually abnormal beyond the annotated interval.
5 Conclusions
The proposed weakly supervised anomalous event detection system uses video-level labels and combines random batch training, normalcy suppression, and clustering distance loss. It achieves state-of-the-art results on UCF-Crime and ShanghaiTech.
- Randomly selecting temporally ordered video batches breaks inter-batch correlation while retaining within-batch temporal consistency.A video may be divided into several batches according to its length.
- Normalcy suppression learns to suppress features corresponding to normal portions of an input video during anomaly detection.
- Clustering distance loss improves representations of anomalous and normal events.
- The architecture produces state-of-the-art results on the UCF-Crime and ShanghaiTech datasets.