Source-linked AI summary
ResnetCrowd: A Residual Deep Learning Architecture for Crowd Counting, Violent Behaviour Detection and Crowd Density Level Classification
Mark Marsden, Kevin McGuinness, Suzanne Little, Noel E. O'Connor
TL;DR
Crowd analysis is challenging and relevant to public safety, while multi-task approaches have been limited by the lack of an appropriately labelled dataset. The paper proposes ResnetCrowd and a 100-image dataset for simultaneous counting, violent-behaviour detection, and density classification. Multi-task training improves all evaluated tasks, with the largest reported gain in violent-behaviour detection.
Problem
Crowd analysis is challenging and relevant to public safety, while prior work focused on separate tasks and multi-task investigation was limited by the lack of an appropriately labelled dataset.
Source: §1 · p. 1Method
ResnetCrowd is a residual deep-learning framework trained and evaluated using a new 100-image dataset fully annotated for crowd counting, violent-behaviour detection, and density classification.
Source: §1 · p. 1Results
Violent-behaviour detection AUC improved by 9% to 0.78 versus the single-task baseline, with small performance improvements across the other tasks.
Source: §5.1 · p. 5Takeaways & Limitations
The experiments demonstrate benefits of multi-task crowd analysis across the evaluated tasks, most notably for violent-behaviour detection.
Source: §5.1 · p. 5, §6 · p. 6
Abstract
from arXiv · showhide
In this paper we propose ResnetCrowd, a deep residual architecture for simultaneous crowd counting, violent behaviour detection and crowd density level classification. To train and evaluate the proposed multi-objective technique, a new 100 image dataset referred to as Multi Task Crowd is constructed. This new dataset is the first computer vision dataset fully annotated for crowd counting, violent behaviour detection and density level classification. Our experiments show that a multi-task approach boosts individual task performance for all tasks and most notably for violent behaviour detection which receives a 9\% boost in ROC curve AUC (Area under the curve). The trained ResnetCrowd model is also evaluated on several additional benchmarks highlighting the superior generalisation of crowd analysis models trained for multiple objectives.
1. Introduction
Crowd analysis matters for public safety, but prior work largely treated counting, behaviour recognition, and density classification as separate tasks. ResnetCrowd and the 100-image Multi Task Crowd dataset introduce a multi-objective approach to all three tasks.
1. Introduction
- The paper frames crowded-scene analysis as a public-safety challenge amid urban congestion and cites stampedes and crushes as examples of preventable loss of life.Source: §1 · p. 1
- A multi-task crowd-analysis approach had not been fully investigated, largely because an appropriately labelled multi-task dataset was lacking.Prior work focused on task-specific systems, while simultaneous learning had boosted individual-task performance in facial analysis.Source: §1 · p. 1
- ResnetCrowd is a residual deep learning framework for simultaneous crowd counting, violent behaviour recognition, and crowd density level classification.The paper proposes a dataset to train the network and reports it as the first computer-vision dataset fully annotated for these tasks.Source: §1 · p. 1
- The Multi Task Crowd dataset contains 100 images fully labelled for crowd counting, violent behaviour detection, and crowd density estimation.Source: §1 · p. 1
- The paper's stated contributions are the dataset, a deep residual architecture for the three tasks, and a quantitative demonstration of multi-objective crowd analysis benefits.Source: §1 · p. 1
2. Related Work
Prior crowd-analysis work developed separate approaches for counting, behaviour recognition, and density estimation, with limited multi-task work. The reviewed methods differ in inputs and representations, while density labels also lack a consistent scheme across datasets.
2. Related Work
- Prior multi-objective crowd-analysis work included density classification to increase crowd-counting robustness, but no technique then combined behaviour recognition with counting or scene occupancy.Multi-task benefits had also been demonstrated in facial analysis, head pose estimation, and speech recognition.Source: §2 · p. 2
- Crowd-counting systems map images to counts or use pixel-wise heatmaps, with heatmaps reported to improve performance on challenging, highly congested scenes.The section also notes deep methods and datasets including UCF CC 50 and Marsden et al. [11], alongside direct regression approaches.Source: §2 · p. 2
- Violent-behaviour recognition using inter-frame motion features requires contiguous frames and cannot classify still images.General crowd-concept detection covers concepts such as “Fight” and “Mob”, but also relies on motion features.Source: §2 · p. 2
- Crowd density level denotes congestion, but datasets lack a shared scheme for assigning labels and their meanings.Labels may be discrete or continuous; inferring discrete labels from true counts is described as a transparent way to reduce subjectivity and human error. Prior examples include texture-based estimation and deep discrete classification.Source: §2 · p. 2
3. Multi Task Crowd Dataset
The Multi Task Crowd dataset was built to support single-frame evaluation of crowd counting, violent behaviour recognition, and density classification. It combines varied crowd scenes with annotations for all three tasks.
3. Multi Task Crowd Dataset
- The dataset targets single-frame analysis; density levels are chosen for their low subjectivity and can be inferred from crowd-count ground truth.Violent behaviour recognition is targeted because of its importance to security personnel.Source: §3 · p. 2
- WWW Crowd was selected for its variation in behaviour and crowd size, while UCF CC 50 had little variation in behaviour and scene content.The paper notes that a publicly available dataset meeting the desired combined criteria had not been produced.Source: §3 · p. 2
- The 100-image dataset is evenly split between violent and non-violent behaviour, with variation in crowd size and scene content.Violent and non-violent subsets were each formed from 50 first frames selected from WWW Crowd clips.Source: §3 · pp. 2–3
- Head locations are marked with single pixels, and Zhang et al.’s method converts them into heatmaps whose integral equals the crowd count.The heatmaps are downscaled to 160 × 90, and density-level labels are inferred from the overall counts.Source: §3 · p. 3
- The dataset provides a jointly labelled resource for single-frame crowd counting, violent behaviour recognition, and density classification.Each image includes a crowd count, head locations, a discrete density level, and binary “Mob” and “Fight” labels.Source: §3 · pp. 2–3
- All tasks are benchmarked using five-fold cross-validation, with each fold designed to represent the overall dataset.Source: §3 · p. 3
4. ResnetCrowd
ResnetCrowd combines a ResNet18-based shared feature extractor with task-specific outputs for crowd counting, behaviour recognition, and density classification. It is trained end-to-end with losses for four supervised outputs.
Architecture
- The shared ResNet18 stem feeds a 1×1-convolution crowd heatmap output and a pooled 64-dimensional representation for task-specific regression and classification heads.The stem uses the first five convolutional layers, batch normalisation and skip connections; its max-pooling layer is removed to preserve feature-map size for pixel-wise counting.Source: §4 · pp. 3–4
- The task-specific outputs cover heatmap-based counting, regression-based counting, violent behaviour detection, and density level classification.The full architecture has 180,934 parameters.Source: §4 · p. 4
Behaviour recognition
- Behaviour recognition predicts separate probability scores for the “Mob” and “Fight” concepts using sigmoid activation and binary cross-entropy.The predicted and ground-truth scores are compared with binary cross-entropy.Source: §4 · p. 4
Density level classification
- Density classification uses softmax and categorical cross-entropy for five mutually exclusive density-level labels.The predicted and ground-truth category probabilities are compared for each example.Source: §4 · p. 4
Crowd counting
- Regression-based counting uses a ReLU output and mean squared error, while heatmap counting trains pixel probabilities with sigmoid and binary cross-entropy.At inference, the overall crowd count is obtained by integrating the estimated heatmap.Source: §4 · pp. 4–5
Training
- Training minimizes the sum of four losses, with images resized to 320 × 180, AdaGrad optimization, and L2 weight regularization at λ = 1 × 10−4.The four losses correspond to the supervised outputs; the image resolution is halved to maintain a suitably high batch size.Source: §4 · pp. 4–5
5. Experimental Results
The experiments compare multi-task ResnetCrowd with single-task baselines and evaluate transfer to additional crowd-analysis benchmarks. The reported results include a 9% AUC gain for violent-behaviour detection, counting trade-offs across density levels, and better anomaly-detection performance than a single-task baseline.
5. Experimental Results
- The evaluation uses 5-fold cross-validation, with horizontal rotations doubling each fold’s training set.Optimization and testing use an Nvidia GeForce GTX 970 GPU with batch size 40.Source: §5 · p. 5
5.1. Multi vs Single Task Learning
- For both counting approaches, ResnetCrowd boosts performance on low-density scenes at the expense of high-density performance.The analysis reports mean absolute error separately for low (0-50 people), medium (50-150 people), and high (150+ people) scenes.Source: §5.1 · p. 5, Table 3 · p. 6
5.2. Transfer Learning
- The transfer experiments assess how training on Multi Task Crowd generalizes to task-specific benchmarks.The benchmarks include violent-behaviour recognition, crowd counting, and crowd-behaviour anomaly detection.Source: §5.2 · p. 5
5.2.1 Violent Behaviour Recognition
- On the WWW crowd test set, ResnetCrowd significantly outperforms the single-task baseline; its AUC falls 15% for “Mob” and 20% for “Fight” versus the state of the art.The comparison uses 80 training frames, versus several million available to Kang et al.Source: §5.2.1 · p. 5, Table 4 · p. 6
5.2.2 Crowd Counting
- On UCF CC 50, heatmap counting improves by 9% over its single-task baseline, while regression counting performs poorly across runs.The authors attribute ResnetCrowd’s lower performance than leading techniques largely to training on lower-density images; no UCF CC 50 fine-tuning was performed.Source: §5.2.2 · pp. 5–6, Table 5 · p. 6
5.1. Multi vs Single Task Learning
- Violent-behaviour detection AUC improves by 9% to 0.78, with small performance improvements across the other tasks.Table 2 compares ResnetCrowd with single-task baselines.Source: §5.1 · p. 5, Table 2 · p. 6
5.2.3 Crowd Behaviour Anomaly Detection
- For anomaly detection, ResnetCrowd significantly outperforms the single-task behaviour-recognition baseline, though it remains far from state of the art.The experiment uses single-frame features and Gaussian-mixture outlier detection; leading methods use inter-frame motion features.Source: §5.2.3 · p. 6, Table 6 · p. 6
6. Conclusions
The paper demonstrates multi-task crowd analysis with a residual-learning approach and constructs a 100-image dataset for evaluation. Future work will explore unsupervised learning to address the lack of labelled crowd data and further increase generalisation.
6. Conclusions
- The paper demonstrates multi-task crowd analysis for simultaneous counting, violent-behaviour detection, and density-level classification.Source: §6 · p. 6
- A 100-image dataset is constructed to evaluate the proposed multi-task architecture.Source: §6 · p. 6
- Future work will investigate unsupervised learning to address scarce labelled crowd data and further increase model generalisation.Source: §6 · p. 6