Source-linked AI summary
DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation
Jiang Liu, Chenqiang Gao, Deyu Meng, Alexander G. Hauptmann
TL;DR
Varying crowd density makes detection and regression unreliable in different regimes: detection weakens in congested scenes, while regression overestimates sparse ones. DecideNet jointly produces both density estimates and uses an attention module to weight them adaptively, achieving state-of-the-art performance on three public datasets.
Problem
Spatial and temporal density variation limits crowd counting methods that rely exclusively on detection or regression.
Method
DecideNet jointly estimates detection- and regression-based density maps and uses QualityNet to assign adaptive per-pixel attention weights.
Results
DecideNet achieves state-of-the-art performance on three public crowd-counting datasets.
Takeaways & Limitations
The framework adapts counting mode to local density by combining detection for sparse regions with regression for congested regions.
Takeaways & Limitations
Detection estimates remain inaccurate in crowded scenes because small object sizes and occlusion lower detection confidence.
Abstract
from arXiv · showhide
In real-world crowd counting applications, the crowd densities vary greatly in spatial and temporal domains. A detection based counting method will estimate crowds accurately in low density scenes, while its reliability in congested areas is downgraded. A regression based approach, on the other hand, captures the general density information in crowded regions. Without knowing the location of each person, it tends to overestimate the count in low density areas. Thus, exclusively using either one of them is not sufficient to handle all kinds of scenes with varying densities. To address this issue, a novel end-to-end crowd counting framework, named DecideNet (DEteCtIon and Density Estimation Network) is proposed. It can adaptively decide the appropriate counting mode for different locations on the image based on its real density conditions. DecideNet starts with estimating the crowd density by generating detection and regression based density maps separately. To capture inevitable variation in densities, it incorporates an attention module, meant to adaptively assess the reliability of the two types of estimations. The final crowd counts are obtained with the guidance of the attention module to adopt suitable estimations from the two kinds of density maps. Experimental results show that our method achieves state-of-the-art performance on three challenging crowd counting datasets.
1. Introduction
Crowd density varies across space and time, exposing complementary weaknesses in detection- and regression-based counting. DecideNet combines both estimations and adaptively weights them according to local density and reliability.
- Crowd counting supports higher-level tasks including crowd monitoring, scene understanding, and public safety management.
- Real-world scenes exhibit substantial spatial and temporal density variation, making exclusive reliance on either counting mode insufficient.
- Detection-based counting can work well in sparse scenes but becomes less reliable as density increases, while regression tends to overestimate low-density counts.Detection errors in crowded areas are associated with small targets and occlusion, whereas regression lacks individual localization.
- The framework is designed to use localization-oriented detection in low-density regions and regression-style estimation in congested regions.
- DecideNet estimates detection- and regression-based density maps, then uses QualityNet to assign adaptive attention weights for each pixel.The modules are trained end-to-end with a joint loss, and QualityNet receives additional supervision from detection scores.
- Experimental results report state-of-the-art performance on public datasets with varying crowd densities.
2. Related works
Prior crowd-counting methods primarily use detection or regression. Detection localizes individuals, whereas regression estimates density from image content and is generally more reliable in crowded patches.
- Crowd counting by detection: Detection-based methods localize each person and treat the number of detections as the crowd count.Recent approaches use CNN-based object detectors for end-to-end counting, but localization becomes difficult when objects are partially visible in congested environments.
- Crowd counting by regression: Regression-based methods estimate counts without explicitly locating individuals, often producing density maps whose integrals give patch counts.
- Crowd counting by regression: Regression methods are reliable in crowded settings but tend to overestimate low-density crowds because they lack object-location information.
- DecideNet: DecideNet combines RegNet and DetNet to estimate two density maps before QualityNet adaptively selects their relative contributions.
3. Crowd Counting by DecideNet
DecideNet formulates crowd counting as density-map estimation using complementary detection and regression branches. QualityNet adaptively combines their pixel-level estimates according to their relative reliability across varying-density scenes.
- 3. Crowd Counting by DecideNet: DecideNet estimates crowd density with separate regression and detection maps, then combines them through an attention-guided final map.The three blocks are jointly learned end to end, with the final estimate formed from the two branch outputs.
- 3.2. The RegNet block: RegNet directly predicts pixel-wise density without localizing individual heads, using a fully convolutional architecture.The implemented block contains five convolutional layers and returns a nonnegative regression density map.
- 3.3. The DetNet block: DetNet extends Faster R-CNN for head detection and converts detected bounding-box centers into a location-aware density map with Gaussian convolution.This map concentrates responses around detected head locations.
- 3.4. Quality-aware density estimation: Detection is favored in sparse scenes, whereas regression is preferred in crowded scenes because localization becomes unreliable under small targets and occlusion.Regression captures contextual density information but tends to overestimate sparse scenes without individual-location information.
- 3.4. Quality-aware density estimation: QualityNet evaluates the relative reliability of the two density maps at each pixel and assigns adaptive attention weights for their final combination.It receives the image and aligned density maps, outputs a probabilistic attention map, and uses that map to weight the detection and regression estimates.
4. Model Learning
DecideNet learns regression, detection, and attention components jointly, using component-specific supervision and a quality-aware loss to guide attention toward reliable detection estimates.
- Model Learning: DecideNet treats its regression, detection, and attention parameters as three parts of a multi-task learning problem.The overall loss combines Lreg, Ldet, and Lqua, optimized with annotated training data.
- Model Learning: The RegNet uses pixel-wise mean square error, while DetNet uses bounding-box supervision to reduce classification and localization errors.Bounding-box supervision avoids overwhelming negative background pixels in the detection density map.
- Model Learning: Bounding-box annotations are manually labeled for part of the training points, and their average width and height provide supervision for the remaining points.The datasets provide head centers rather than bounding boxes.
- Model Learning: QualityNet learns from final density-map error and the difference between its probabilistic attention map and the detection score map.The detection-score term acts as a regularizer over QualityNet parameters and incorporates detection quality as prior information.
- Model Learning: The quality-aware loss balances its two error components with the hyper-parameter λ.The loss compares attention reliability with detection scores while also accounting for final density-map error.
5. Experimental Results
DecideNet is evaluated on three real-world crowd-counting datasets and combines detection and regression estimates adaptively, producing strong benchmark and component-analysis results across varying densities.
- Experimental Setup: DecideNet is evaluated on the Mall, ShanghaiTech PartB, and WorldExpo’10 datasets using cropped training patches and validation-based model selection.Training uses 40k iterations, an initial learning rate of 0.005, and halving every 10k steps.
- The Mall dataset: On Mall, DecideNet obtains the minimum MAE and MSE, improving MAE by 0.49 points over regression-based Boosting CNN and achieving MSE 1.90.This result is reported without the ensemble scheme used by MoCNN and Boosting CNN.
- The WorldExpo’10 dataset: On WorldExpo’10, DecideNet achieves average MAE 9.23 across five scenes, improving by 0.17 over the second-best Switching-CNN approach.It also obtains three minimum scene-level MAE errors among the compared methods.
- Effects of different components in DecideNet: Regression estimates overpredict low-count images, detection estimates underpredict high-count images, and DecideNet remains closest to ground truth across the plotted ranges.The quality-aware model outperforms direct late fusion, whose predictions are less stable across images.
- Visualization on density maps: Visualization shows detection-like concentrated peaks in nearby low-density regions and diffuse regression-like maps in remote congested regions.QualityNet assigns the locally preferred estimation pattern, indicating pixel-level reliability assessment.
6. Conclusion
DecideNet is an end-to-end framework that adaptively combines detection- and regression-based crowd-count estimates using attention, achieving state-of-the-art performance on three public datasets.
- DecideNet adaptively adopts detection and regression-based count estimations under attention guidance for varying crowd densities.The framework is motivated by the complementary performance of the two counting approaches.
- DecideNet achieves state-of-the-art performance on three public crowd-counting datasets.
- DecideNet is presented as the first framework to estimate crowd counts by adaptively adopting both counting estimations under an attention mechanism.