Source-linked AI summary
MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detection
Yingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton Fok, Xiaojuan Qi, Yik-Chung Wu
TL;DR
Surveillance-video anomaly detection is difficult in long videos because local evidence and feature magnitudes can be confounded by scene variation. MGFN combines global-to-local glance-and-focus processing with Feature Amplification and Magnitude Contrastive learning, and reports state-of-the-art performance on UCF-Crime and XD-Violence.
Problem
Long-video anomaly localization is challenging, while scene attributes can make feature magnitudes inconsistent with anomaly status and weaken magnitude-based detection.
Method
MGFN glances at the whole video before focusing on specific portions, using FAM and MC loss to learn more discriminative, scene-adaptive feature magnitudes.
Results
86.98% AUC on UCF-Crime and 80.11% AP on XD-Violence are reported, outperforming state-of-the-art approaches.
Takeaways & Limitations
The paper supports combining long-term context with local focus and scene-adaptive magnitude learning for weakly supervised video anomaly detection.
Takeaways & Limitations
Feature magnitudes are affected by scene attributes such as object movement, object count, and the number of people, complicating simple magnitude ordering.
Abstract
from arXiv · showhide
Weakly supervised detection of anomalies in surveillance videos is a challenging task. Going beyond existing works that have deficient capabilities to localize anomalies in long videos, we propose a novel glance and focus network to effectively integrate spatial-temporal information for accurate anomaly detection. In addition, we empirically found that existing approaches that use feature magnitudes to represent the degree of anomalies typically ignore the effects of scene variations, and hence result in sub-optimal performance due to the inconsistency of feature magnitudes across scenes. To address this issue, we propose the Feature Amplification Mechanism and a Magnitude Contrastive Loss to enhance the discriminativeness of feature magnitudes for detecting anomalies. Experimental results on two large-scale benchmarks UCF-Crime and XD-Violence manifest that our method outperforms state-of-the-art approaches.
Introduction
Long-video anomaly detection requires global temporal context and scene-aware feature magnitudes because local evidence and raw magnitude ordering can be misleading. MGFN addresses these issues through glance-and-focus processing, Feature Amplification, and Magnitude Contrastive learning.
- Challenges: Long videos are difficult to localize because anomalies may occupy few frames, require normality context, and span varied event types.Video-level weak supervision is more feasible than exhaustive frame-level annotation.
- Existing limitations: Spatial-temporal models often struggle on long videos because they lack global context awareness and focused processing of abnormal frames.
- Magnitude inconsistency: Feature magnitude depends on scene attributes such as object movement, object count, and the number of people, not only anomaly status.A normal video with substantial object movement can have larger magnitudes than an abnormal video.
- Magnitude inconsistency: RTFM's strategy of making abnormal features larger and normal features smaller conflicts with inter-video magnitude distributions and can harm training.The paper also reports overlapping or reversed normal and abnormal magnitudes within the same sequence and poor separability.
- Proposed framework: MGFN first glances at the whole sequence for long-term context, then focuses on specific portions for scene-adaptive anomaly detection.The design is presented as a global-to-local information integration mechanism.
- Proposed framework: FAM enhances feature-magnitude discriminativeness, while MC loss learns scene-adaptive cross-video magnitude distributions and separates normal from abnormal features.MC encourages similar magnitudes within categories and separability between categories.
- Results: 86.98% AUC and 80.11% AP are reported on UCF-Crime and XD-Violence, respectively, outperforming state-of-the-art approaches.
Related Work
Video anomaly detection research includes one-class, unsupervised, and weakly supervised settings, with weak supervision using video-level labels to reduce annotation demands. Related approaches include reconstruction, memory, meta-learning, graph, Siamese, clustering, and transformer-based spatial-temporal models.
- Video Anomaly Detection: Surveillance-video annotation is labor-intensive, motivating one-class, unsupervised, and weakly supervised anomaly-detection settings.Weak supervision uses only video-level annotations.
- Video Anomaly Detection: One-class methods learn normal patterns using ConvLSTM, auto-encoder reconstruction, memory mechanisms, and meta-learning.
- Video Anomaly Detection: Weakly supervised methods use multiple-instance learning, graph convolution, Siamese networks, and clustering to distinguish or localize anomalous events.The related work notes unsatisfactory generalization for one graph-convolution approach.
- Vision Transformer: Transformer architectures have been applied across vision tasks because of their feature-representation capacity and have been extended to video spatial-temporal dependencies.Reported applications include video recognition, video object detection, and untrimmed-video processing.
Our Approach
MGFN explicitly incorporates feature magnitudes, then combines global video context with local clip features for weakly supervised anomaly detection. Its losses promote smooth predictions and scene-adaptive separation between normal and abnormal feature magnitudes.
- Architecture: MGFN extracts clip features, amplifies them with explicit feature norms, and processes them through Glance and Focus Blocks for global-to-local representation learning.FAM incorporates a modulated feature norm as a residual; GB integrates global temporal context, while FB enhances local clip features.
- Glance Block: The Glance Block models correlations among temporal clips so the network uses long-term context and knowledge of normal cases.Its video clip-level transformer generates attention-weighted representations from all clips in a long video.
- Focus Block: The Focus Block uses self-attentional convolution to enhance feature learning within each video clip and capture channel-wise correlations without learnable weights.It applies the feature map as both convolution input and kernel before a two-layer feed-forward network.
- Loss Functions: Training combines sigmoid cross-entropy, temporal smoothness, sparsity, and magnitude-contrastive losses through weighted summation.Temporal regularization smooths prediction scores for adjacent video clips.
- Magnitude Contrastive Loss: MC loss groups same-category feature magnitudes and separates normal-abnormal pairs using top-k normal and abnormal magnitudes.The loss samples B/2 normal and B/2 abnormal videos, increasing cross-category distances while reducing within-category distances.
- Magnitude Contrastive Loss: MC loss learns a scene-adaptive cross-video magnitude distribution rather than universally forcing abnormal magnitudes above normal magnitudes.This permits high-magnitude normal features in scenes with substantial movement while still encouraging category separability.
Experiment
MGFN is evaluated on UCF-Crime and XD-Violence using AUC and AP, with ablations testing its Glance-Focus mechanism, Feature Amplification Mechanism, and Magnitude Contrastive loss. The reported results show improved anomaly detection and localization, including stable normal-video predictions and detection of multiple anomalous clips.
- Benchmark Results: MGFN outperforms prior methods on UCF-Crime, including RTFM by 2.85% with I3D features and MSL by 1.05% AUC with VideoSwin features.The comparisons include one-class, unsupervised, and weakly supervised baselines.
- Benchmark Results: MGFN outperforms prior methods on XD-Violence, exceeding RTFM by 1.38% with I3D features and MSL by more than 1.53% AP with VideoSwin features.The authors report consistent superiority over spatial-temporal feature ensemble approaches.
- Glance-Focus Ablation: The Glance-Focus mechanism produces more stable normal-video predictions and detects abnormal frames that FF, FG, GF-Fusion, and RTFM miss or misclassify.The qualitative comparison uses predicted anomaly scores against ground-truth anomalies on two UCF-Crime videos.
- Glance-Focus Ablation: 82.66% AUC and 72.14% AP from FF increase to 83.1% AUC and 74.30% AP when global context is added after local feature learning.These results support the contribution of global features in the ablation study.
- Benchmark Results: MGFN achieves 85.80% AUC on UCF-Crime and 78.50% AP on XD-Violence, improving over prior global-local approaches.The authors attribute these gains to the Glance-Focus mechanism.
- FAM and MC Loss Ablation: FAM improves performance by more than 1.65% AUC on UCF-Crime and 1.02% AP on XD-Violence, while MC loss adds nearly 2.85% AUC and 3.69% AP.Combining FAM and MC loss further improves performance, indicating compatibility.
Conclusion
The paper introduces MGFN with Glance-and-Focus processing, Feature Amplification, and Magnitude Contrastive loss for weakly supervised video anomaly detection. Experiments on UCF-Crime and XD-Violence report that the approach outperforms state-of-the-art methods by a large margin.
- Conclusion: MGFN integrates global context and local features through a Glance-and-Focus module and uses Magnitude Contrastive loss for anomaly detection.Feature Amplification Mechanism enhances awareness of feature magnitudes, while the loss encourages separation between normal and abnormal magnitudes.
- Conclusion: Experimental results on UCF-Crime and XD-Violence show that MGFN outperforms state-of-the-art works by a large margin.