Source-linked AI summary
Weakly Supervised Video Anomaly Detection via Center-guided Discriminative Learning
Boyang Wan, Yuming Fang, Xue Xia, Jiajie Mei
TL;DR
Video anomaly detection must handle diverse anomalous content and duration while training with limited supervision. The paper proposes AR-Net, which regresses clip-level anomaly scores using only video-level labels and introduces dynamic MIL and center losses; on ShanghaiTech, it reports a new state-of-the-art result with a 4.94% absolute AUC gain.
Problem
Video anomaly detection is challenging because anomalous video content and duration are diverse, while weakly supervised training uses only video-level labels.
Method
AR-Net performs anomaly-score regression for video clips under video-level weak supervision, using dynamic MIL and center losses to improve feature discrimination.
Results
4.94% absolute AUC gain is reported on ShanghaiTech, yielding a new state-of-the-art result for video anomaly detection.
Takeaways & Limitations
The proposed losses produce discriminative feature representations for video anomaly detection under weak supervision.
Takeaways & Limitations
The method fails on anomalies occupying only a local scene region or visually resembling normal behavior, such as skateboarding on a sidewalk.
Abstract
from arXiv · showhide
Anomaly detection in surveillance videos is a challenging task due to the diversity of anomalous video content and duration. In this paper, we consider video anomaly detection as a regression problem with respect to anomaly scores of video clips under weak supervision. Hence, we propose an anomaly detection framework, called Anomaly Regression Net (AR-Net), which only requires video-level labels in training stage. Further, to learn discriminative features for anomaly detection, we design a dynamic multiple-instance learning loss and a center loss for the proposed AR-Net. The former is used to enlarge the inter-class distance between anomalous and normal instances, while the latter is proposed to reduce the intra-class distance of normal instances. Comprehensive experiments are performed on a challenging benchmark: ShanghaiTech. Our method yields a new state-of-the-art result for video anomaly detection on ShanghaiTech dataset
1. INTRODUCTION
The paper frames weakly supervised video anomaly detection as a binary-classification problem using only video-level labels, and proposes AR-Net with two losses for discriminative feature learning. On ShanghaiTech, the approach achieves a new state-of-the-art result with a 4.94% absolute AUC gain.
- Motivation: Unary-classification methods model usual patterns, but unseen normal videos may be falsely flagged because normal training samples cannot cover all normal content.This motivates using both anomalous and normal videos under the binary-classification paradigm.
- Weakly supervised formulation: The paper treats each video as a bag of clips, with anomalous videos as positive bags and normal videos as negative bags.This multiple-instance learning formulation uses video-level labels while assigning anomaly scores to video clips.
- Prior methods: Existing MIL losses rely on pairwise calculations, making detection ability partly dependent on batch size and available graphics memory.The paper contrasts this limitation with its proposed loss design.
- Proposed method: AR-Net combines a dynamic multiple-instance learning loss with a center loss to learn separable features under video-level weak supervision.The dynamic loss uses cross entropy between clip anomaly scores and video labels, while the center loss uses distances from clips to the average anomaly score of each normal video.
- Results: 4.94% absolute AUC gain is reported for the proposed approach on the ShanghaiTech benchmark, establishing a new state-of-the-art result.The experiments are conducted on ShanghaiTech.
2. PROPOSED METHOD
AR-Net extracts clip features and maps them to anomaly-score vectors using only video-level labels. Its DMIL and center losses respectively separate anomalous from normal instances and gather normal-clip scores.
- 2.1. Feature Extraction: AR-Net uses pretrained RGB and optical-flow I3D features, concatenated from their penultimate layers, as clip representations.Each input video is divided into non-overlapping clips of 16 consecutive frames.
- 2.1. Feature Extraction: The feature matrix Xi contains clip features with dimension F × ti and is fed to AR-Net instead of raw video clips.F is the clip-feature dimension and ti is the number of clips in video xi.
- 2.2. Anomaly Regression Network: The FC-Layer applies ReLU and dropout, while the AR-Layer maps features to clip-level anomaly-score vectors using video-level labels.The anomaly scores represent probabilities that instances are anomalies.
- 2.3. Dynamic Multiple-Instance Learning Loss: DMIL selects k-max anomaly scores, with k determined by the video clip count and hyperparameter α, then computes instance-level cross-entropy against the video label.Unlike averaging selected scores, this focuses on individual anomaly scores and limits error propagation from noisy labels.
- 2.4. Center loss for Anomaly Scores Regression: The center loss reduces the intra-class spread of normal-clip anomaly scores by penalizing their distances from each normal video's average anomaly score.It is introduced to address the normal-instance separation problem caused by DMIL's early-training label assignments.
3. EXPERIMENTS
Experiments on ShanghaiTech compare AR-Net with existing methods, loss variants, feature extractors, and qualitative cases. AR-Net achieves the strongest reported frame-level AUC, while results also show benefits from its losses and combined appearance-motion features, alongside failures on visually ambiguous local anomalies.
- Experiments Setup: ShanghaiTech contains 238 training videos and 199 testing videos, with AUC and FAR used to evaluate anomaly detection.The dataset includes untrimmed videos with variable scenes, contents, and durations.
- Comparison Results: 91.24% frame-level AUC is achieved by our method, exceeding at 86.30% and at 82.50% on ShanghaiTech.The method is the only compared approach surpassing 90% AUC in the reported comparison.
- Ablation Study: 89.10% frame-level AUC is obtained with DMIL, compared with 86.50% for the k-max MIL baseline.Adding the proposed center loss reduces FAR to 1/9 of the baseline value.
- Ablation Study: 91.24% frame-level AUC is obtained with I3DConc, compared with 85.38% for I3DRGB and 82.34% for I3DOptical-Flow.I3DConc combines appearance and motion features from the two extractors.
- Qualitative Analysis: Clip-based methods can recognize short-term anomalies that segmented methods may miss when anomalous frames are overwhelmed by normal frames.The paper attributes this distinction to dividing videos into fixed-frame clips rather than 32 non-overlapped segments.
- Qualitative Analysis: Both and our method fail on ‘01 0015’ when the anomaly is local and visually indistinguishable from normal behavior.The paper identifies these scenes as a continuing challenge for current video anomaly detection models.
4. CONCLUSION
The paper proposes a MIL-based anomaly regression network with dynamic and center losses for weakly supervised video anomaly detection. Experiments support its effectiveness, while future work targets temporal relations for greater robustness.
- AR-Net performs video anomaly detection under weak supervision using video-level labels.
- The dynamic loss learns separable features and avoids false alarms caused by interference between clip features.
- The center regression loss suppresses label noise by smoothing anomaly-score distributions.
- Clip-based instance generation benefits short-term anomaly detection, while modeling temporal relations remains future work for improved robustness.