Source-linked AI summary
Crowd Counting Using Scale-Aware Attention Networks
Mohammad Asiful Hossain, Mehrdad Hosseinzadeh, Omit Chanda, Yang Wang
TL;DR
Crowd counting requires estimating density maps from arbitrary crowded images despite substantial scale variation. This paper introduces a scale-aware attention network that softly selects appropriate global and local scales, and reports better performance than state-of-the-art methods across several benchmark datasets.
Problem
Crowd counting must estimate pixel-level density maps from arbitrary crowded images, where scale variation complicates density estimation.
Method
A scale-aware attention network uses global and local attentions to softly select and re-weight features at appropriate scales for density estimation.
Results
The proposed model outperforms other state-of-the-art crowd-counting approaches on several benchmark datasets.
Takeaways & Limitations
Global and local scale attention provides a unified approach for adapting crowd-counting features to image-level and local density variation.
Abstract
from arXiv · showhide
In this paper, we consider the problem of crowd counting in images. Given an image of a crowded scene, our goal is to estimate the density map of this image, where each pixel value in the density map corresponds to the crowd density at the corresponding location in the image. Given the estimated density map, the final crowd count can be obtained by summing over all values in the density map. One challenge of crowd counting is the scale variation in images. In this work, we propose a novel scale-aware attention network to address this challenge. Using the attention mechanism popular in recent deep learning architectures, our model can automatically focus on certain global and local scales appropriate for the image. By combining these global and local scale attention, our model outperforms other state-of-the-art methods for crowd counting on several benchmark datasets.
1. Introduction
Crowd counting estimates a density map from an arbitrary crowded image and derives the count by integrating it, despite occlusion, perspective distortion, and scale variation. The paper introduces global and local scale-aware attention to softly select useful scales and reports superior benchmark performance.
- Problem: Crowd counting estimates a pixel-level density map from arbitrary static images, with the final count obtained by integrating the map.The setting assumes dotted annotations, where each object instance is marked by a single image point.
- Challenge: Severe occlusion, perspective distortion, diverse densities, and varying person or head sizes make crowd counting difficult for single-scale models.These conditions motivate architectures that represent multiple scales within crowded scenes.
- Approach: The proposed network uses attention to focus on relevant scale dimensions rather than selecting only one scale for density estimation.Its attention acts as a soft switch, re-weighting features at different scales instead of making a potentially incorrect hard selection.
- Contribution: The paper introduces scale attention for crowd counting, focusing on global and local scales instead of only spatial locations.The authors describe this as the first use of attention models in crowd counting and contrast it with classifier-based scale selection.
- Result: The proposed approach is reported to outperform other state-of-the-art crowd-counting methods on several benchmark datasets.This is the paper's stated overall empirical contribution.
2. Related Work
Prior crowd-counting work includes detection, regression, density-estimation, multi-scale, contextual, and switching approaches. The paper positions its method as a simpler attention-based alternative that softly focuses on global and local scales without switching between distinct counting modes.
- Existing approaches: Existing crowd-counting methods are commonly categorized as detection-based, regression-based, or density-estimation-based approaches.CNN methods often estimate a density map and derive counts from it.
- Existing approaches: Detection-based methods often struggle with high-density crowds, motivating regression methods that learn image-patch-to-count mappings.This describes one historical transition in crowd-counting methodology.
- Scale variation: Scale variation has motivated multi-column, scale-aware, and deep-shallow architectures that process or learn from multiple scale levels.MC-CNN uses three branches at different input scales, while Hydra CNN learns mappings from patches to density maps.
- Context and switching: Other methods use contextual density predictions or switch among branches and counting modes according to estimated density or scale.CP-CNN combines global and local context, Switch-CNN selects a scale branch, and DecideNet switches between regression and detection models.
- Proposed direction: The proposed model uses global and local attention to focus automatically on appropriate scales and is conceptually simpler than DecideNet because it uses one counting mode.The attention mechanism avoids switching between separate regression and detection models.
3. Our Approach
The proposed network combines multi-scale feature extraction with global and local scale attention, then fuses attention-weighted features to estimate a density map. It is trained with density-map and auxiliary global/local scale losses.
- 3. Our Approach: Global and local attention weights capture inter-image and intra-image density variation, enabling adaptive use of features at appropriate scales.The architecture comprises MFE, GSA, LSA, and FN modules.
- 3.1. Multi-Scale Feature Extractor: The multi-branch feature extractor uses three branches with different filter sizes to extract features at three scales, producing feature maps at H/4 × W/4 resolution.The branches process arbitrary-sized input images independently at corresponding scale levels.
- 3.2. Global Scale Attention: Global scale attention produces three normalized scores corresponding to low-, mid-, and high-density labels for each input image.A softmax layer normalizes the scores to sum to one.
- 3.3. Local Scale Attention: Local scale attention produces three pixel-wise attention maps for fine-grained scale information at different image locations.The module uses convolution and max-pooling layers followed by three fully connected layers and a sigmoid output.
- 3.4. Fusion Network: Attention-weighted feature maps are concatenated and passed through the fusion network, which upsamples them to H × W and outputs a one-channel density map.The final density map is converted to a crowd count by summing its entries.
- 3.5. Loss Function: The overall training loss combines density-map loss with auxiliary global- and local-attention losses, whose weights control their relative contributions.The density-map loss uses the predicted and ground-truth maps, while the auxiliary losses encourage attention predictions to match ground-truth scales; empirically, they regularize the model and improve performance.
4. Experiments
The experiments evaluate the proposed method on three crowd-counting benchmarks using standard splits or cross-validation, then analyze component contributions. The model outperforms prior methods across most reported metrics, while both attention modules and auxiliary losses improve performance.
- 4.1. Datasets and Setup: The evaluation uses ShanghaiTech PartB, Mall, and UCF CC 50 with standard splits for the first two datasets and 5-fold cross-validation for UCF CC 50.The experiments apply the same data augmentation used in previous methods across all datasets.
- 4.2. Experimental Results: The model significantly outperforms previous approaches in both MAE and MSE on ShanghaiTech PartB and Mall, and improves MAE on UCF CC 50.On UCF CC 50, it also outperforms most previous approaches in MSE except [20].
- 4.3. Ablation Study: Using both global and local scale attention achieves the best performance compared with the base model or either attention module alone.The base model contains the multiscale feature extractor and fusion network without attention modules.
- 4.3. Ablation Study: Both auxiliary attention losses improve the learned model, with the best performance obtained when LDM, LGSA, and LLSA are used together.LGSA and LLSA provide extra supervision during training in addition to the density-map loss.
5. Conclusion
The paper presents scale-aware attention networks for crowd counting, using soft global and local scale selection within a single density-estimation model. Experiments show that the approach outperforms other state-of-the-art crowd-counting methods while remaining simpler than a model that switches between counting modes.
- 5. Conclusion: The proposed scale-aware attention network softly selects appropriate scales at both global and local levels for crowd counting.The conclusion contrasts this design with DecideNet, which switches between detection and density-estimation models.
- 5. Conclusion: The model outperforms other state-of-the-art approaches for crowd counting in the reported experiments.The conclusion presents this as the main empirical outcome of the paper.
- 5. Conclusion: Qualitative density-map examples cover ShanghaiTech PartB and Mall images alongside their corresponding input images.The figure presents original images and density maps in paired columns for both datasets.