Source-linked AI summary
Real-Time Anomaly Detection and Localization in Crowded Scenes
Mohammad Sabokrou, Mahmood Fathy, Mojtaba Hosseini, Reinhard Klette
TL;DR
The paper targets reliable anomaly detection and localization in crowded scenes despite limitations in prior descriptors and real-time methods. It combines global and local video representations with Gaussian classifiers, using learned features and patch similarity, and reports strong benchmark performance with real-time operation.
Problem
Anomaly detection in crowded scenes requires reliable localization and real-time operation, while prior methods often fail at localization or are not designed for real-time applications.
Method
The method represents videos with global and local descriptors, combines their classifier outputs, and uses auto-encoder features, adjacent-patch similarity, and Gaussian modeling.
Results
The approach achieves favorable anomaly detection and localization results, with performance and running time superior to state-of-the-art methods, and reaches 25 fps processing power.
Takeaways & Limitations
The system can detect and localize anomalies soon after they occur, while its fusion strategy provides accurate and reliable results across the evaluated datasets.
Takeaways & Limitations
Prior sparse-representation methods commonly fail at localization or real-time deployment, and the UMN dataset has only three anomaly scenes and no pixel-level ground truth.
Abstract
from arXiv · showhide
In this paper, we propose a method for real-time anomaly detection and localization in crowded scenes. Each video is defined as a set of non-overlapping cubic patches, and is described using two local and global descriptors. These descriptors capture the video properties from different aspects. By incorporating simple and cost-effective Gaussian classifiers, we can distinguish normal activities and anomalies in videos. The local and global features are based on structure similarity between adjacent patches and the features learned in an unsupervised way, using a sparse auto- encoder. Experimental results show that our algorithm is comparable to a state-of-the-art procedure on UCSD ped2 and UMN benchmarks, but even more time-efficient. The experiments confirm that our system can reliably detect and localize anomalies as soon as they happen in a video.
1. Introduction
The paper addresses the computational and modeling challenges of anomaly detection in crowded scenes by combining learned and similarity-based descriptors with Gaussian modeling for real-time detection and localization.
- Motivation: High-dimensional descriptors require many training samples and substantial computation, risking reduced predictive power through the curse of dimensionality.These challenges make reliable modeling of complex-scene anomalies difficult.
- Prior limitations: Trajectory-based methods can be computationally expensive and unable to handle occlusion in crowded scenes.They label objects according to how closely their trajectories follow learned normal behavior.
- Prior limitations: Sparse-representation methods can perform favorably for anomaly detection but commonly fail at localization and real-time deployment.The paper identifies these as limitations of much prior work, with one cited exception for real-time design.
- Proposed approach: The proposed system represents videos through two partially independent views and integrates them for simultaneous real-time anomaly detection and localization.The learned representative features are based on auto-encoders rather than low-level features.
- Proposed approach: The method models normal patches with Gaussian distributions and uses Mahalanobis distance to assess test-video relevance.Local and global representations support the final decision through complementary classifiers.
2. Proposed System
The proposed system represents videos through complementary global and local views, modeling normal patches with sparse learned features, neighboring-patch similarities, and Gaussian classifiers. It combines both views to detect and localize anomalies at patch level.
- Overall Scheme: Videos are partitioned into non-overlapping cubic patches covering the entire space-time volume.This patch representation supports subsequent feature extraction and patch-level anomaly localization.
- Overall Scheme: Anomalies are modeled as patches whose spatial, temporal, and occurrence patterns differ from normal patches.Spatial and temporal changes are encoded locally, while occurrence likelihood is represented globally.
- Global descriptors: The global descriptor learns sparse features from raw normal patches with an auto-encoder, reconstructing inputs while optimizing its weights through gradient descent.The hidden-layer activation is regularized toward sparsity, and W1 and W2 map between input, hidden, and output layers.
- Local descriptors: The local descriptor combines SSIM similarities between each patch and 10 spatial-temporal neighbors with temporal inner-patch similarities.The temporal neighbor is restricted to the preceding patch to support early anomaly detection.
- Anomaly Classifier: Two Gaussian classifiers evaluate partially independent global and local feature sets using Mahalanobis distance and threshold-based decisions.A patch is classified as anomalous only when both classifiers reject it; localization follows from the anomalous patches.
- Anomaly detection using feature learning: Feature learning uses 10×10×5 patches, while 40×40×5 test patches are divided into 16 non-overlapping small patches whose learned feature vectors are mean-pooled.This transfers a classifier learned on small patches to larger test regions while avoiding direct learning from high-dimensional large patches.
3. Experimental results and comparisons
The method is evaluated on UCSD ped2 and UMN using frame-, pixel-, and dual-pixel-level measures, with comparisons covering detection quality, localization, and runtime. Results are comparable to or better than competing methods, while supporting real-time use.
- Evaluation setup: The framework is compared with state-of-the-art methods on UCSD ped2 and UMN benchmarks.The evaluation reports ROC, EER, AUC, qualitative comparisons, and runtime.
- Evaluation measures: Dual pixel level adds a precision condition to pixel-level detection, requiring at least β percent of detected anomaly pixels to overlap the ground truth.This measure is designed to penalize irrelevant detected regions or “lucky guesses.”
- Qualitative comparison: The method shows the best qualitative performance among the compared methods in three example scenes.Figure 7 compares Temporal MDT, Spatial MDT, MPPCA, Social force, Optic flow, feature learning only, and combined views.
- UCSD ped2 results: 0.5 percent below Li et al., the method’s frame-level performance outperforms the other reported methods on UCSD ped2 except that method.The comparison uses frame-level ROC and EER analysis.
- UCSD ped2 results: 24 percent EER is reported for pixel-level evaluation, compared with 29.9 percent for Li et al.; the method is 5.9 percent better than the otherwise best result.The pixel-level results also report good dual-pixel performance at β=0.05 percent and 0 percent.
- UMN results: On UMN, the method’s AUC is comparable to the otherwise best result, while its EER is 0.3 percent better than the best previous method.Because UMN has only three anomaly scenes and no pixel-level ground truth, evaluation uses frame-level EER and AUC; only the global detector is used.
4. Conclusions
The paper concludes that combining global and local descriptors with two classifiers supports accurate, reliable anomaly detection and localization. It reports better UCSD performance, lower computational complexity, and real-time operation than competing methods.
- Conclusions: Global and local descriptors feed two classifiers whose fused outputs provide accurate and reliable anomaly detection and localization.Each classifier also performs well independently for anomaly detection, especially the global descriptor on UMN.
- Conclusions: The method introduces a dual-pixel-level metric for region-level anomaly detection and reports better UCSD performance than recent approaches.The conclusion also states that the system achieves these results with better running time than competing methods.
- Conclusions: Low computational complexity enables real-time operation for live-stream surveillance applications.The conclusion frames real-time execution as the practical use case for the method.