Source-linked AI summary
Multi-scale Convolutional Neural Networks for Crowd Counting
Lingke Zeng, Xiangmin Xu, Bolun Cai, Suo Qiu, Tong Zhang
TL;DR
Crowd counting must handle substantial scale variation, while existing multi-column or multi-network approaches complicate optimization and increase computation. MSCNN uses a single-column network with multi-scale blobs to extract scale-relevant features, achieving state-of-the-art performance on ShanghaiTech and UCF CC 50 with fewer parameters.
Problem
Crowd counting requires robust accuracy despite scale variation, while existing multi-column or multi-network approaches complicate optimization and consume more computing resources.
Method
MSCNN uses a single-column network with multi-scale blobs comprising different kernel sizes to extract scale-relevant features for density-map regression.
Results
MSCNN achieves state-of-the-art performance on the ShanghaiTech and UCF CC 50 datasets with higher accuracy and robustness and substantially fewer parameters than compared CNN-based methods.
Takeaways & Limitations
The single-column, end-to-end design makes MSCNN more likely to extend to practical crowd-counting applications.
Takeaways & Limitations
The paper frames multi-column or multi-network methods as limited by complicated pre-training and increased parameter-driven computing costs for practical application.
Abstract
from arXiv · showhide
Crowd counting on static images is a challenging problem due to scale variations. Recently deep neural networks have been shown to be effective in this task. However, existing neural-networks-based methods often use the multi-column or multi-network model to extract the scale-relevant features, which is more complicated for optimization and computation wasting. To this end, we propose a novel multi-scale convolutional neural network (MSCNN) for single image crowd counting. Based on the multi-scale blobs, the network is able to generate scale-relevant features for higher crowd counting performances in a single-column architecture, which is both accuracy and cost effective for practical applications. Complemental results show that our method outperforms the state-of-the-art methods on both accuracy and robustness with far less number of parameters.
1. INTRODUCTION
Crowd counting supports public safety but is challenged by occlusion, complex backgrounds, scale variation, and the computational burden of existing approaches. The paper proposes MSCNN, a single-column network using a multi-scale blob to extract scale-relevant features with fewer parameters.
- Motivation: Accurate and robust crowd-count estimation is important for public safety in overcrowded settings such as tourist attractions and public rallies.Overcrowding can cause crowd crushes, blockages, and stampedes.
- Limitations of Existing Methods: Detection-based methods require substantial computation and are limited by person occlusions and complex backgrounds, reducing practical robustness and accuracy.These methods count people by accumulating individually detected persons.
- Limitations of Existing Methods: Single-CNN methods struggle to extract scale-relevant features and address scale variations in crowd images.Recent convolutional neural networks have nevertheless shown effectiveness for crowd counting.
- Limitations of Existing Methods: Multi-column or multi-network approaches complicate global optimization by requiring a pre-trained single network and increase parameters and computing resource demands.These costs make practical application difficult.
- Proposed Method: MSCNN extracts scale-relevant features with a multi-scale blob using different kernel sizes instead of adding columns or networks.The approach outperforms state-of-the-art methods on the ShanghaiTech and UCF CC 50 datasets with a small number of parameters.
2. MULTI-SCALE CNN FOR CROWD COUNTING
MSCNN addresses scale variation in crowd images with a single-column architecture that learns scale-relevant density maps. Its design combines feature remapping, multi-scale extraction, density-map regression, Gaussian-kernel ground truth construction, and Euclidean-distance optimization.
- Motivation: MSCNN learns scale-relevant density maps from original images to address varying person-pixel sizes caused by perspective distortion.The method is motivated by the difficulty of handling scale variations with same-sized kernel combinations in a single network.
- Architecture: The network comprises feature remapping, multi-scale feature extraction, and density map regression.Its first convolution layer remaps image features using single-sized kernels before subsequent processing.
- Architecture: The Multi-Scale Blob extracts scale-relevant features using multiple filters with 9×9, 7×7, 5×5, and 3×3 kernels.The MSB is described as an Inceptionlike model.
- Density-map generation: Each head annotation is represented as δ (x −xi) and convolved with a Gaussian kernel Gσ to form the density map F (x) = H (x) ∗Gσ (x).The resulting density map is accumulated to obtain the crowd count value.
- Optimization: Euclidean distance measures the difference between the model output feature map and the corresponding ground-truth density map for optimization.The loss is defined in Eq. (2), with Θ denoting model parameters and F (Xi; Θ) the model output.
3. EXPERIMENTS
MSCNN is evaluated on the ShanghaiTech and UCF CC 50 crowd-counting datasets using accuracy, robustness, and parameter-count metrics. It achieves state-of-the-art performance on both datasets while using substantially fewer parameters than competing CNN-based methods.
- Evaluation setup: MSCNN is evaluated on ShanghaiTech and UCF CC 50 using MAE, MSE, and PARAMS to measure accuracy, robustness, and computation complexity.All convolutional neural networks are trained with Caffe.
- ShanghaiTech dataset: The ShanghaiTech experiments show state-of-the-art performance, with MSCNN using approximately 7× fewer parameters than MCNN while achieving higher accuracy and robustness.ShanghaiTech contains 1,198 annotated images and 330,165 persons, divided into Parts A and B.
- UCF CC 50 dataset: The UCF CC 50 experiments use five-fold cross-validation because the dataset contains only 50 images with counts ranging from 94 to 4,543 people.Training data are augmented by randomly cropping 36 patches of size 225×225 from each image and flipping them.
- UCF CC 50 results: MSCNN achieves state-of-the-art performance on UCF CC 50 with approximately 5× fewer parameters than CrowdNet, while working more accurately and robustly.Five models are trained using the five training splits, with a fixed learning rate of 1e-7 to guarantee convergence.
4. CONCLUSION
The paper proposes MSCNN, a single-column, end-to-end crowd-counting network that uses multi-scale blobs to extract scale-relevant features without multi-column or multi-network pretraining. It achieves more accurate and robust counting with far fewer parameters.
- 4. CONCLUSION: MSCNN extracts scale-relevant features from crowd images using a single-column network based on multi-scale blobs.The architecture avoids the complexity of multi-column or multi-network models.
- 4. CONCLUSION: MSCNN supports end-to-end training without requiring multi-column or multi-network pretraining.
- 4. CONCLUSION: MSCNN delivers more accurate and robust crowd-counting performance with far fewer parameters than recent CNN-based methods.