Source-linked AI summary
Fully Convolutional Crowd Counting On Highly Congested Scenes
Mark Marsden, Kevin McGuinness, Suzanne Little, Noel E. O'Connor
TL;DR
The paper addresses accurate crowd counting in highly congested scenes, where prior techniques often perform poorly. It combines redundancy-minimizing augmentation, a deep fully convolutional network, and multi-scale averaging, achieving state-of-the-art performance on Shanghaitech Part B and UCF CC 50 and competitive performance on Part A.
Problem
Accurate, generalizable crowd counting remains difficult in highly congested scenes, where occlusions and scene variation limit performance.
Method
The method combines nonredundant training augmentation, a deep single-column FCN for density heatmaps, and multi-scale count averaging during inference.
Results
The model achieves state-of-the-art performance on Shanghaitech Part B and UCF CC 50, with competitive performance on Shanghaitech Part A.
Takeaways & Limitations
The approach can analyze images with any resolution and aspect ratio and remains effective with substantial test-time downsampling.
Takeaways & Limitations
Processing high-resolution images requires substantial memory and slows inference.
Abstract
from arXiv · showhide
In this paper we advance the state-of-the-art for crowd counting in high density scenes by further exploring the idea of a fully convolutional crowd counting model introduced by (Zhang et al., 2016). Producing an accurate and robust crowd count estimator using computer vision techniques has attracted significant research interest in recent years. Applications for crowd counting systems exist in many diverse areas including city planning, retail, and of course general public safety. Developing a highly generalised counting model that can be deployed in any surveillance scenario with any camera perspective is the key objective for research in this area. Techniques developed in the past have generally performed poorly in highly congested scenes with several thousands of people in frame (Rodriguez et al., 2011). Our approach, influenced by the work of (Zhang et al., 2016), consists of the following contributions: (1) A training set augmentation scheme that minimises redundancy among training samples to improve model generalisation and overall counting performance; (2) a deep, single column, fully convolutional network (FCN) architecture; (3) a multi-scale averaging step during inference. The developed technique can analyse images of any resolution or aspect ratio and achieves state-of-the-art counting performance on the Shanghaitech Part B and UCF CC 50 datasets as well as competitive performance on Shanghaitech Part A.
1 INTRODUCTION
Crowd counting seeks accurate person counts despite occlusion, scene variation, and congestion. This paper explores a fully convolutional approach with redundancy-minimizing augmentation, a deeper architecture, and multi-scale inference.
- Crowd counting provides objective estimates useful for city planning, shopping-pattern analysis, and public safety, but occlusion and scene variation remain major challenges.
- Detection-based methods become computationally demanding and degrade under occlusion, while regression methods avoid explicit detection but have suffered from overfitting.
- Fully convolutional networks produce proportionally sized feature maps and support variable-size inputs, helping preserve image detail for crowded-scene counting.
- The proposed training augmentation minimizes redundancy among samples to improve model generalization and counting performance.
- A deep single-column FCN generates density heatmaps, while multi-scale inference averages counts estimated from differently sized inputs.
- Inference uses a simple averaging step that produces significant performance gains.
2 A FULLY CONVOLUTIONAL NETWORK FOR CROWD COUNTING
The method builds a fully convolutional crowd-counting pipeline that supports variable image sizes, uses nonredundant augmentation and a deep single-column architecture, and averages predictions across scales.
- FCN formulation: FCNs accept images of any resolution and aspect ratio because they omit fully connected layers and produce proportionally sized feature maps.The resulting feature maps support image transformation and pixel-wise prediction tasks.
- Density-map construction: Head annotations are converted into continuous density heatmaps by placing impulses at head locations and smoothing them with adaptive Gaussian kernels.The kernel spread is based on the mean distance to the five nearest heads, with the 0.3 weighting retained from prior work.
- Training set augmentation: Nonoverlapping quadrant crops and horizontal flips reduce pixel redundancy in training samples, improving validation accuracy despite reducing the overall training-set size.The augmentation scheme is motivated by the risk of overfitting repeated pixels in pixel-wise tasks.
- FCN architecture: The six-layer single-column FCN uses 315,000 parameters, ReLU activations except after the final convolution, and a 1×1 output layer for the density heatmap.An element-wise 2D sum integrates the heatmap into the crowd-count estimate, and training minimizes Euclidean heatmap distance.
- FCN architecture: A deeper single-column architecture improves Shanghaitech Part B validation performance over a shallower multi-column FCN under identical hyperparameters and augmentation.The comparison is reported in Table 2.
- Multi-scale inference: Multi-scale inference resizes each test image, estimates a count at every scale, and averages those estimates; the best validation scheme is used thereafter.This design addresses scale and perspective variation, including foreground-background size differences in CCTV scenes.
3 EXPERIMENTS
The method is evaluated on three crowd-counting benchmarks spanning different congestion levels and scene content. It achieves state-of-the-art results on Shanghaitech Part B and UCF CC 50, while cross-dataset results show that density-matched training domains generalize better.
- Evaluation setup: The evaluation covers Shanghaitech Parts A and B and UCF CC 50, using MAE and MSE to compare counting performance.MAE measures estimate accuracy, while MSE measures robustness.
- Shanghaitech results: 10% lower MAE and 19% lower MSE produce state-of-the-art performance on Shanghaitech Part B.The method also achieves competitive performance on Part A, with MSE near the state-of-the-art result.
- UCF CC 50 results: 11% lower MAE and 13% lower MSE improve the state of the art on UCF CC 50.The dataset contains highly challenging images with between 94 and 4500 pedestrians per frame.
- Cross-dataset performance: Cross-dataset performance is strongest when source and target domains contain similar density levels.The reported favorable transfers are Shanghaitech Part B to Part A and Part A to UCF CC 50.
- Computation and accuracy: High-resolution FCN inference improves counting performance but increases GPU memory consumption and slows processing.The study therefore examines reducing test-time image resolution to trade computation speed against accuracy.
4 CONCLUSION
The proposed fully convolutional model achieves state-of-the-art performance on Shanghaitech Part B and UCF CC 50, competitive performance on Part A, and supports varied image inputs. Cross-dataset performance is reported with percentage increases in MAE and MSE highlighted.
- The model achieves state-of-the-art performance on the Shanghaitech Part B and UCF CC 50 datasets.
- The model achieves competitive performance on the Shanghaitech Part A dataset.
- Images of any resolution and aspect ratio can be analysed by the developed approach.
- Table 6 reports cross-dataset performance, highlighting percentage increases in MAE and MSE.