Source-linked AI summary
CrowdNet: A Deep Convolutional Network for Dense Crowd Counting
Lokesh Boominathan, Srinivas S S Kruthiventi, R. Venkatesh Babu
TL;DR
Highly dense still-image crowds are difficult to count because of occlusion, scale variation, and limited annotated training data. The paper combines deep and shallow fully convolutional networks with Gaussian density targets and multi-scale augmentation, and reports state-of-the-art performance on UCF CC 50. However, it tends to underestimate images containing more than 2500 people.
Problem
Highly dense still-image crowd counting is challenged by severe occlusion, large scale variation, and limited training data because dense-image annotation is laborious.
Method
The method combines deep and shallow convolutional networks, Gaussian-blurred density-map ground truth, and multi-scale and high-density patch augmentation.
Results
The method outperforms state-of-the-art methods on the challenging UCF CC 50 dataset.
Takeaways & Limitations
Combining deep and shallow networks with augmentation addresses scale variation and severe occlusion in highly dense crowd images.
Takeaways & Limitations
The method tends to underestimate images containing more than 2500 people, possibly because the dataset has too few training images with such large crowds.
Abstract
from arXiv · showhide
Our work proposes a novel deep learning framework for estimating crowd density from static images of highly dense crowds. We use a combination of deep and shallow, fully convolutional networks to predict the density map for a given crowd image. Such a combination is used for effectively capturing both the high-level semantic information (face/body detectors) and the low-level features (blob detectors), that are necessary for crowd counting under large scale variations. As most crowd datasets have limited training samples (<100 images) and deep learning based approaches require large amounts of training data, we perform multi-scale data augmentation. Augmenting the training samples in such a manner helps in guiding the CNN to learn scale invariant representations. Our method is tested on the challenging UCF_CC_50 dataset, and shown to outperform the state of the art methods.
1. INTRODUCTION
The paper addresses crowd counting in highly dense still images, where occlusion, perspective, scale variation, and limited annotated data challenge existing approaches. It combines deep and shallow convolutional networks with extensive augmentation and reports state-of-the-art results on UCF CC 50.
- Motivation: Crowd counting supports automated analysis by estimating both the number of people and crowd-density distribution.Density estimates can identify regions above safety limits, while counts support event quantification and logistics.
- Challenges: Highly dense crowd counting is difficult because occlusion, perspective, and non-uniform scale make traditional detectors ineffective.Images with more than 2000 people are especially challenging, while annotation is laborious and limits available training data.
- Motivation: Hand-crafted features such as SIFT and HOG often lack robustness to occlusion and large scale variations.The proposed approach instead relies on deep learned features from fully convolutional neural networks.
- Contribution: The method combines shallow and deep convolutional architectures and augments multi-scale training samples to improve scale robustness.The approach was evaluated on the challenging UCF CC 50 dataset.
2. RELATED WORK
Prior crowd-counting methods target sparse scenes, moving objects, or handcrafted representations, while this work focuses on highly dense still-image crowds. It estimates both total count and density distribution without requiring laborious perspective maps.
- Scope: Existing datasets and methods often address sparse crowds or moving objects, whereas highly dense still-image counting remains a distinct setting.Motion-based methods require continuous video streams with adequate frame rates and do not extend to still images.
- Handcrafted methods: Handcrafted approaches combine cues such as HOG head detections, Fourier analysis, and interest points but can lose accuracy under illumination, perspective, and occlusion changes.These methods may also use multi-scale Markov Random Field post-processing.
- Deep methods: Zhang et al. train with perspective maps, whose generation requires manually labeling several pedestrians by their height.The paper presents a simpler training approach while reporting better performance.
- Deep methods: Wang et al. estimate only total crowd count, whereas this network estimates both count and crowd-density distribution.The density map is identified as important for crowd analysis.
3. PROPOSED METHOD
The proposed method combines deep semantic features with shallow blob detection to produce density maps for crowds observed across perspectives and scales. Gaussian ground truth construction and targeted multi-scale augmentation address annotation ambiguity, limited data, and dense-crowd difficulty.
- 3.1 Network Architecture: The model combines deep and shallow convolutional networks to recognize faces or bodies near the camera and head blobs in distant or aerial views.This combination addresses simultaneous semantic and low-level recognition under perspective and scale variation.
- 3.1 Network Architecture: The deep branch adapts VGG-16 features for per-pixel crowd-density prediction rather than whole-image classification.Its architecture is modified to preserve higher spatial resolution for density estimation.
- 3.1 Network Architecture: The deep and shallow predictions are concatenated, processed by a 1x1 convolution, and upsampled to the input size to form the final density map.Summing the predicted density map yields the total crowd count, and training uses an L2 loss against ground truth.
- 3.2 Ground Truth: Gaussian-blurred head annotations normalized to sum to one create learnable density-map ground truth whose total equals the crowd count.Blurring reduces sensitivity to ambiguous annotation positions while indicating which regions contribute to the count.
- 3.3 Data Augmentation: Multi-scale augmentation crops patches from image pyramids at scales 0.5 to 1.2, training the CNN to recognize people across different sizes.The patches are 225 × 225 with 50% overlap.
- 3.3 Data Augmentation: High-density patches are sampled more often because CNNs find highly dense crowds inherently difficult to handle.This augmentation targets insufficient training samples in dense regions.
4. EXPERIMENTS
Experiments evaluate CrowdNet on the highly challenging UCF CC 50 dataset and analyze its multi-scale architecture and density-focused augmentation. The method achieves state-of-the-art crowd-counting performance, while errors increase for images containing more than 2500 people.
- UCF CC 50 contains 50 grayscale images spanning 94 to 4543 people per image and varied scenarios such as concerts, rallies, religious gatherings, and stadiums.
- The evaluation uses 5-fold cross-validation, training on 40 images and validating on 10 images in each fold.Training uses 225 × 225 patches and averages 50,292 training patches per fold.
- Results: The approach achieves state-of-the-art performance on UCF CC 50 using MAE without post-processing.MAE measures the mean absolute difference between actual and predicted counts.
- Results: Predicted counts generally lie close to actual counts, but the method tends to underestimate images containing more than 2500 people.The authors suggest this error could result from too few training images with such large crowds.
- Deep and Shallow Networks: Combining deep and shallow networks quantitatively improves MAE over either network alone on the hardest cross-validation fold.The combined architecture is intended to capture individuals across multiple scales.
- Count based Augmentation: Density-focused augmentation nearly doubles patches on the hardest fold, from 26,385 to 50,891, addressing insufficient training samples for large crowds.Its quantitative advantage is reported in Table 3.
5. CONCLUSION
The paper proposes a deep-learning approach for estimating density and total count in highly dense crowd images. It combines deep and shallow networks with training-image augmentation, and reports state-of-the-art results on UCF CC 50.
- The approach estimates crowd density and total crowd count from highly dense crowd images.
- Combining deep and shallow networks addresses large scale variations and severe occlusion when detecting people.
- Augmenting the training images tackles varying scales and difficulties inherent in highly dense crowds.
- The method outperforms state-of-the-art methods on the challenging UCF CC 50 dataset.