Source-linked AI summary
Crowd counting via scale-adaptive convolutional neural network
Lu Zhang, Miaojing Shi, Qiaobo Chen
TL;DR
Crowd counting must handle strong perspective and scale changes, while representative methods perform poorly on sparse scenes. SaCNN uses a single small-filter backbone with multi-layer feature fusion and relative count loss, and experiments report improvements over the state of the art. The paper also evaluates sparse-scene performance using SmartCity.
Problem
Crowd counting requires estimating pedestrian numbers despite perspective and scale changes, and representative approaches perform poorly on scenes with few pedestrians.
Method
SaCNN uses a single small-filter backbone, combines feature maps from multiple layers, and jointly optimizes density-map and relative-count losses.
Results
SaCNN demonstrates efficiency and effectiveness over the state of the art across extensive experiments on standard benchmarks and sparse-scene data.
Takeaways & Limitations
Shared multi-scale features and relative count loss provide a crowd counter adaptable to different pedestrian scales, perspectives, and sparse scenes.
Takeaways & Limitations
The authors report that results on SmartCity are not perfect and identify direct perspective information as future work for weighting feature-map outputs.
Abstract
from arXiv · showhide
The task of crowd counting is to automatically estimate the pedestrian number in crowd images. To cope with the scale and perspective changes that commonly exist in crowd images, state-of-the-art approaches employ multi-column CNN architectures to regress density maps of crowd images. Multiple columns have different receptive fields corresponding to pedestrians (heads) of different scales. We instead propose a scale-adaptive CNN (SaCNN) architecture with a backbone of fixed small receptive fields. We extract feature maps from multiple layers and adapt them to have the same output size; we combine them to produce the final density map. The number of people is computed by integrating the density map. We also introduce a relative count loss along with the density map loss to improve the network generalization on crowd scenes with few pedestrians, where most representative approaches perform poorly on. We conduct extensive experiments on the ShanghaiTech, UCF_CC_50 and WorldExpo datasets as well as a new dataset SmartCity that we collect for crowd scenes with few people. The results demonstrate significant improvements of SaCNN over the state-of-the-art.
1. Introduction
Crowd counting is needed for public safety but is difficult under occlusion, dense crowds, and large perspective and scale changes. SaCNN addresses these challenges with a single-column, scale-adaptive architecture and relative count loss, including a dataset for sparse scenes.
- Accurate pedestrian counting supports crowd control and public safety in settings such as rallies and sports.
- Detection-based methods struggle when pedestrians are heavily occluded or densely distributed.
- Regression-based methods address perspective and scale variation using multi-column CNNs with different filter sizes and density-map integration.
- A single column retained over 70% of multi-column accuracy on some datasets, motivating the proposed SaCNN architecture.
- SaCNN combines multi-layer feature maps and a relative head-count loss to adapt to scale and perspective while improving generalization on sparse crowds.
- SmartCity contains indoor and outdoor high-angle scenes with few pedestrians, averaging 7.4 people per image.
2. Related work
Prior work includes detection- and regression-based counting, with CNN density-map regression improving performance while multi-column designs address scale variation. SaCNN instead uses shared multi-scale features, deconvolution, and relative count loss in a single-column network.
- Crowd-counting methods are categorized as detection-based or regression-based approaches.
- Detection methods suffer severely in dense crowds with high occlusion among people.
- Regression methods estimate scalar counts or density maps from image features using functions such as linear, ridge, or Gaussian regression.
- Density-map regression provides regional counts through integration and leverages CNNs' effectiveness at solving problems locally.
- Multi-column CNNs address head-scale variation but large filters complicate training, while SaCNN uses one filter size and trains from scratch.
- SaCNN combines feature maps from multiple layers using deconvolution rather than upsampling and elementwise summation.
- SaCNN jointly regresses density maps and head counts through a multi-task objective with a proposed relative count loss.
3. Scale-adaptive CNN
SaCNN replaces multi-column CNNs with a single small-filter backbone that combines aligned feature maps from multiple layers to adapt to pedestrian scale and perspective. It trains density-map regression with a relative head-count loss, improving sparse-scene learning while preserving an efficient training design.
- Ground-truth density maps: Ground-truth density maps represent each head with a unit-sum Gaussian around its annotated pixel, so summing the map recovers the total pedestrian count.The map is formed by convolving head-centered delta functions with a normalized Gaussian kernel.
- Architecture: Feature maps from multiple network layers are resized and concatenated so density estimation adapts to pedestrian scale and perspective variations.The architecture progressively adds layers of different scales and aligns their outputs before concatenation.
- Architecture: The two-scale model extracts conv4_3 and conv5_3 features, aligns them with stride-1 pooling, and produces a final density map that improves over the single-scale model.Figure 3 depicts the single-scale and two-scale variants, with density maps and head counts as outputs.
- Architecture: The full scale-adaptive model combines deeper features, upsamples to 1/8 input resolution, and downsamples the ground-truth density map by factor 8.Performance slightly declines at 1/16 resolution, which the authors suggest may result from overly large deeper-layer receptive fields.
- Network loss: Training combines Euclidean density-map loss with a relative head-count loss that emphasizes samples with comparatively large prediction errors, especially in sparse crowds.The model first trains on density maps, then jointly optimizes both losses with weights 1 and 0.1; sparse-scene results show significant improvement.
4. Experiments
Experiments evaluate SaCNN across four datasets, ablations, and comparisons with state-of-the-art methods. Results show benefits from multi-scale features and relative count loss, with strong performance on dense and sparse crowd scenes.
- Datasets: Experiments cover ShanghaiTech, WorldExpo’10, UCF CC 50, and the newly collected SmartCity dataset.SmartCity targets indoor and outdoor scenes with few pedestrians, unlike existing datasets dominated by hundreds or thousands of outdoor pedestrians.
- Ablation studies: The scale-adaptive model achieves the lowest MAE and MSE on both ShanghaiTech PartA and PartB.The ablation compares single-scale, two-scale, and three-scale models trained with density map loss.
- Ablation studies: Relative count loss decreases both MAE and MSE compared with removing count loss on ShanghaiTech PartA and PartB.Absolute count loss instead impairs performance because its magnitude varies drastically among crowd images.
- Benchmark results: 86.8 and 16.2 are SaCNN’s best MAE values on ShanghaiTech PartA and PartB, respectively, while testing is twice as fast as [25].SaCNN reduces MAE versus [25] by 3.6 points on PartA and 5.4 points on PartB; inference takes 293ms versus 580ms per image.
- Benchmark results: On WorldExpo’10, SaCNN without perspective information obtains the best average MAE of 8.5 across scenes.It produces the best scene-level MAE on S1, S2, and S5, with values 2.6, 13.5, and 3.3.
- Benchmark results: On UCF CC 50, SaCNN achieves the best MAE 314.9 and MSE 424.8 among the compared methods.The smallest MSE indicates the lowest variance of predictions across the dataset.
- Sparse-scene results: On SmartCity, relative count loss lowers MAE and MSE by 9.2 and 11.8 points, while SaCNN reaches MAE 8.6.The best MAE is close to SmartCity’s average pedestrian count of 7.4; competing methods and [25] obtain MAE 40.0 and 23.4.
5. Conclusion
SaCNN estimates crowd density maps and pedestrian numbers by combining multi-scale feature maps, while relative count loss improves generalization on scenes with few pedestrians. The paper identifies perspective-aware weighting and combining detection with density estimation as future directions.
- SaCNN concatenates feature maps of different scales to estimate density maps and pedestrian numbers in crowd images.
- Relative count loss is included with density-map training to improve generalization on crowd scenes with few pedestrians.
- SaCNN adapts to pedestrians of different scales and perspectives through its multi-scale feature-map architecture.
- Future work will embed perspective information as pixel-level weighting and investigate combining head detection with density-based counting.