Source-linked AI summary
Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes
Mohammad Sabokrou, Mohsen Fayyaz, Mahmood Fathy, Zahra Moayedd, Reinhard klette
TL;DR
Anomaly detection in crowded scenes must address slow CNN processing and the lack of labeled examples for unknown anomaly classes. The paper converts a pre-trained CNN into an FCN with cascaded outlier detection for efficient anomaly detection and localization. The method is reported as fast and accurate, reaching about 370 fps and outperforming existing methods, while high false-positive rates occur in very crowded situations.
Problem
Anomaly detection with CNNs is limited by slow patch-based processing and the impossibility of training on large labeled samples for nonexistent anomaly classes.
Method
The paper proposes an FCN architecture that adapts a pre-trained CNN to extract regional features and perform cascaded anomaly detection and localization.
Results
About 370 fps is reported, and the proposed method is described as fast and accurate while outperforming existing methods.
Takeaways & Limitations
The method provides a time-efficient approach for anomaly detection in video surveillance while preserving reported accuracy relative to existing methods.
Takeaways & Limitations
The system has a high false-positive rate in situations that are too crowded.
Abstract
from arXiv · showhide
The detection of abnormal behaviours in crowded scenes has to deal with many challenges. This paper presents an efficient method for detection and localization of anomalies in videos. Using fully convolutional neural networks (FCNs) and temporal data, a pre-trained supervised FCN is transferred into an unsupervised FCN ensuring the detection of (global) anomalies in scenes. High performance in terms of speed and accuracy is achieved by investigating the cascaded detection as a result of reducing computation complexities. This FCN-based architecture addresses two main tasks, feature representation and cascaded outlier detection. Experimental results on two benchmarks suggest that detection and localization of the proposed method outperforms existing methods in terms of accuracy.
1. Introduction
Anomaly detection in crowded videos must identify rare or unexpected motion and shapes while handling large data volumes, high computational cost, and the impossibility of exhaustively labeling unknown anomalies. The paper proposes an FCN-based approach that processes full frames, transfers features from a pre-trained CNN, and uses cascaded detection for faster anomaly detection and localization.
- 370 fps is achieved on a standard NVIDIA TITAN GPU for 320 × 240 frames, while anomaly-detection accuracy is reported as comparable to an existing method.The paper reports concurrent convolution and pooling operations and describes the method as simpler and faster in training and testing.
- Crowded-scene anomalies are rare shapes or motions, making detection and localization a challenging video-analysis task.
- Reference-model methods learn normal regions, then classify regions that differ from those models as abnormal.These methods require extensive training samples to describe each region's properties.
- CNN-based anomaly detection is constrained by slow patch-based processing and supervised training that cannot provide samples for nonexistent anomaly classes.The paper identifies both computational cost and the difficulty of collecting labeled examples for unknown anomalies as major problems.
- A cascaded design sends low-confidence regions to a final convolutional layer while modeling normal training regions with two Gaussian models.The pre-trained CNN kernels remain constant in the FCN, while the added layer is trained on the paper's data.
- The proposed method adapts a pre-trained classification CNN into an FCN that extracts regional features from full video frames for concurrent motion and shape description.An additional convolutional layer and transfer learning support feature representation, while the method analyzes deep-layer outputs to detect and localize abnormal regions.
2. Related Work
Prior anomaly-detection methods model motion, appearance, trajectories, or normality with diverse handcrafted, probabilistic, sparse, and deep-learning approaches. However, reported weaknesses include difficulty handling occlusions and excessive complexity for crowded scenes, motivating spatio-temporal and cascaded alternatives.
- Motion and appearance modeling: Existing methods model anomalies through object trajectories, optical flow, dynamic textures, probabilistic PCA, hidden Markov models, and social-force representations.These approaches represent motion or normal patterns using trajectory estimation, optical-flow histograms, mixtures of dynamic textures, MPPCA, HMMs, or social force.
- Motivation for efficient detection: Reported limitations include inefficient handling of occlusions and methods that are too complex for processing crowded scenes.These weaknesses are identified alongside proposals using spatio-temporal low-level features and cascaded deep classifiers, where challenging patches are processed before neighboring patches undergo further classification.
- Learning-based approaches: Deep and feature-learning methods use auto-encoders, PCANet, spatio-temporal convolutions, MLPs, Gaussian mixture models, and clustering or sparse coding.These methods learn representations from gradients, human visual-system features, optical flow, particles, corner features, or global and local activity patterns.
- Spatial-temporal context: Several methods explicitly incorporate spatial and temporal context through motion-context descriptors, informative structural context, local pixel models, or hierarchical local-global frameworks.These approaches model interactions, local patterns, crowd variation, or both global activity and local salient behavior.
3. Proposed Method
The proposed method represents video frames with regional features from a pre-trained FCN, models normal regions, and detects and localizes anomalies through cascaded outlier detection.
- Frame and feature representation: Input frames are represented on progressively smaller grids by intermediate convolutional layers, whose feature vectors describe receptive fields of the input.The method uses L = 3 convolutional layers, with each feature vector derived from a specific input sub-region.
- Normal reference modeling: Normal training regions are collected from the pre-trained FCN and modeled with a Gaussian one-class classifier as a normal reference.The same regional description process is applied to normal training frames to fit the reference distribution.
- Localization: Anomaly locations are recovered by rolling abnormal regional descriptions back through the FCN to identify corresponding regions in the input frame.The localization relies on the approximate invertibility of convolution and mean-pooling operations.
- Cascaded anomaly detection: Gaussian classifier G1 labels regions as normal, abnormal, or suspicious according to their distance from the normal reference and thresholds α and β.Suspicious regions fall between the two thresholds and are passed to the next convolutional stage.
- Cascaded anomaly detection: Only suspicious regions are transformed by a sparse auto-encoder and evaluated by a second Gaussian classifier G2, reducing the regions processed at the next stage.G2 is trained on normal regional features represented by the auto-encoder.
- Architecture and evaluation: Adding a new convolutional layer after C1 and C2 forms the proposed architecture, whose evaluated CNN architecture is reported as best for the studied data.The paper associates the new representation with more discriminative suspicious-region features and both accuracy and speed.
4. Experimental Results
The method is evaluated on UCSD and Subway benchmarks using frame-level and pixel-level anomaly metrics, qualitative outputs, and runtime comparisons. Results report strong detection and localization performance alongside substantially faster processing, while false positives remain in crowded or directionally varied scenes.
- Datasets: The method is evaluated on UCSD and Subway benchmarks for anomaly detection and localization.UCSD includes Ped2 sequences, while Subway contains entrance and exit scenes.
- Evaluation protocol: Evaluation uses frame-level and pixel-level measures, including ROC curves, EER, and AUC.Frame-level detection marks a frame abnormal if one pixel detects an anomaly; pixel-level evaluation requires at least 40% of ground-truth anomaly pixels to be covered.
- Qualitative results: False positives occur when scenes are too crowded or people walk in different directions, including directions opposite to other pedestrians.The latter behavior is treated as abnormal because it is absent from the training video.
- UCSD Ped2: On UCSD Ped2, the proposed method outperforms the considered methods in ROC comparisons except Tan Xiao et al. for frame-level EER.The reported frame-level EER is 11%, versus a best general result of 10% by Tan Xiao et al..
- UCSD Ped2: 15% pixel-level EER is reported for the proposed method, 2% better than the next-best result of 17%.This comparison concerns the pixel-level EER metric on UCSD Ped2.
- Runtime: The system processes a frame in approximately 0.0027 seconds, achieving 370 fps and exceeding the speed of the considered state-of-the-art methods.The reported runtime uses a standard NVIDIA TITAN GPU.
5. Conclusions
The paper introduces a fully convolutional architecture for video anomaly detection and localization. It combines pretrained convolutional features with a newly trained convolutional layer, achieving fast processing while addressing limited training data.
- Contribution: The paper presents a new FCN architecture for generating and describing abnormal regions in videos.The architecture uses fully convolutional processing for anomaly detection and localization.
- Feature representation: Patch-wise FCN operations produce regional features without contextual information from neighboring patches.The paper describes the generated regional features as context-free.
- Architecture: The proposed FCN combines a pretrained AlexNet version with a new convolutional layer whose kernels are trained on the chosen training video.Only the final convolutional layer of the proposed FCN needs to be trained.
- Conclusion: The method addresses limitations in the training samples required to learn a complete CNN while achieving about 370 fps.The conclusion characterizes the approach as both fast and accurate for video anomaly detection.