Source-linked AI summary
SCAR: Spatial-/Channel-wise Attention Regression Networks for Crowd Counting
Junyu Gao, Qi Wang, Yuan Yuan
TL;DR
CNN-based crowd-counting methods often emphasize local appearance while lacking large-range contextual and channel-attention modeling. SCAR adds spatial-wise and channel-wise attention to a regression CNN, concatenates the resulting features, and predicts density maps. The paper reports state-of-the-art performance across Shanghai Tech Part A/B, GCC, and UCF_CC_50.
Problem
CNN-based crowd-counting methods emphasize local appearance features but lack large-range pixel-wise contextual and crowd-attention information.
Method
SCAR adds Spatial-wise Attention Model and Channel-wise Attention Model modules to a regression CNN, concatenating their features for density-map prediction.
Results
SCAR achieves state-of-the-art results on four mainstream crowd-counting datasets.
Takeaways & Limitations
The combined model encodes contextual and location information for predicting crowd density maps.
Takeaways & Limitations
The spatial-wise attention model relies on a regularity in global and local crowd-density distributions associated with perspective changes.
Abstract
from arXiv · showhide
Recently, crowd counting is a hot topic in crowd analysis. Many CNN-based counting algorithms attain good performance. However, these methods only focus on the local appearance features of crowd scenes but ignore the large-range pixel-wise contextual and crowd attention information. To remedy the above problems, in this paper, we introduce the Spatial-/Channel-wise Attention Models into the traditional Regression CNN to estimate the density map, which is named as "SCAR". It consists of two modules, namely Spatial-wise Attention Model (SAM) and Channel-wise Attention Model (CAM). The former can encode the pixel-wise context of the entire image to more accurately predict density maps at the pixel level. The latter attempts to extract more discriminative features among different channels, which aids model to pay attention to the head region, the core of crowd scenes. Intuitively, CAM alleviates the mistaken estimation for background regions. Finally, two types of attention information and traditional CNN's feature maps are integrated by a concatenation operation. Furthermore, the extensive experiments are conducted on four popular datasets, Shanghai Tech Part A/B, GCC, and UCF_CC_50 Dataset. The results show that the proposed method achieves state-of-the-art results.
1. Introduction
SCAR addresses the limited contextual and channel-relation modeling of conventional CNN crowd-counting methods with spatial- and channel-wise attention. Its modules are integrated with CNN features to produce density maps, and the paper reports state-of-the-art performance across four datasets.
- Motivation: CNN-based crowd-counting methods often focus on local spatial features, limiting their ability to encode large-range contextual information.Traditional FCNs also fail to model relations between channels and are prone to background estimation errors.
- Method: SCAR combines Local Feature Extraction, an Attention Model, and a Map Regressor in a sequential crowd-counting pipeline.The feature extractor uses the first 10 convolutional layers of VGG-16, while the map regressor uses two 1 × 1 convolutional layers.
- Method: SAM encodes spatial dependencies across the whole feature map, while CAM models relations between channel maps to improve regression and reduce background estimation errors.The two attention outputs are concatenated before density-map prediction.
- Contributions: The paper identifies SAM, CAM, and their combined attention model as its principal contributions to contextual and channel-aware density estimation.The combined model is reported to achieve state-of-the-art results on four mainstream datasets.
2. Related Work
Prior crowd-counting work improves local-feature modeling and introduces contextual or recurrent mechanisms, while attention research develops methods for selecting or weighting visual regions and features.
- CNN-based counting: Earlier CNN counting methods use multi-column architectures, switch layers, and composition loss to model varying densities or improve localization.These approaches primarily address local appearance features or image-patch processing.
- Context modeling: Other counting methods incorporate global context, dilated convolutions, or recurrent spatial modeling to represent contextual features and density variation.The cited examples include CP-CNN, CSRNet, and a recurrent spatial-aware network.
- Attention models: Visual-attention research includes recurrent attention, hard or soft pooling, and classical attention modules for selecting or weighting salient spatial features.These methods motivate attention mechanisms in visual tasks.
3. Methodology
SCAR adds spatial- and channel-wise attention to a CNN regression pipeline for crowd-density estimation. SAM captures long-range spatial context, while CAM models channel relations to emphasize crowd-relevant responses and reduce background estimation errors.
- Overview: The local feature extractor combines a VGG-16 backbone with dilation convolution, producing 1/8-size, 64-channel feature maps with added spatial context.The dilation module enlarges the receptive field relative to the backbone output.
- Overview: SCAR uses two parallel non-local streams to encode spatial- and channel-wise attention, concatenating their feature maps before predicting a one-channel density map.The concatenated attention features are converted to the density map through convolution.
- Implementation: Training uses standard Mean Squared Error loss, and Table 1 specifies the proposed SCAR network architecture and convolutional configuration.The configuration notation includes kernel size, output channels, stride, dilation rate, and ReLU usage.
- Spatial-wise Attention Model: SAM models dependencies across all spatial positions, producing global contextual features and self-attention information for density localization.For C × H × W input features, SAM forms an HW × HW spatial attention map and combines the attended features with the original feature map using a learnable factor.
- Channel-wise Attention Model: CAM models dependencies between channels to strengthen class-specific responses for foreground head regions and background discrimination.CAM computes a C × C channel-attention map and combines its output with the original features through a learnable parameter.
4. Experiments
SCAR is evaluated for counting accuracy and density-map quality across ShanghaiTech, GCC, and UCF CC 50, using standard metrics and visual comparisons. It reports strong benchmark performance, including leading results on ShanghaiTech and GCC.
- Evaluation Metrics: The experiments evaluate counting performance with MAE and MSE, and density-map quality with PSNR and SSIM.MAE and MSE measure counting error, while PSNR and SSIM are full-reference density-map metrics.
- Implementation Details: All images and generated density maps are resized to 576 × 768, with Adam optimization for 400 epochs.Training uses an initial learning rate of 10^-5, batch size 4 per GPU, and a learning-rate multiplier of 0.995 per epoch.
- ShanghaiTech Dataset: SCAR wins three first places and one second place on ShanghaiTech Part A and B, outperforming VGG-16-based Switching-CNN and CSRNet overall.The comparison is reported in Table 2 for mainstream methods on the ShanghaiTech dataset.
- Qualitative Results: Visualizations show predicted density maps that represent regional densities and estimated counts close to ground-truth counts.Figure 4 compares input images, ground truth, and SCAR-predicted density maps on ShanghaiTech Parts A/B and GCC.
- UCF CC 50 Dataset: On UCF CC 50, SCAR obtains the best MAE of 259.0 and the third-place MSE of 374.0 under 5-fold cross-validation.The dataset contains 50 images without a predefined training-testing partition.
- GCC Dataset: SCAR achieves the best GCC MAE under random splitting, cross-camera, and cross-location evaluation: 31.7/55.8/87.2.Using the same VGG-16 backbone as CSRNet, FCN, and SFCN, SCAR achieves 8 best places across 12 metrics.
5. Discussion and Analysis
The experiments analyze module contributions, density-map quality, fusion strategies, and attention visualizations. Together, they show that combining SAM and CAM with concatenation yields the strongest supported results and distinct spatial and channel attention behaviors.
- Ablation Study: The full SCAR model achieves the best counting and density-map quality performance among the ablation settings.Embedding SAM and CAM simultaneously outperforms the individual-module configurations on ShanghaiTech Part B.
- Ablation Study: 11.0/18.8 of MAE/MSE for FCN+SAM is better than 11.5/19.3 of MAE/MSE for FCN+CAM.
- Ablation Study: 23.01/0.881 of PSNR/SSIM for FCN+SAM exceeds 22.45/0.875 of PSNR/SSIM for FCN+CAM.
- Density Map Quality: SCAR obtains 23.93 of PSNR and 0.81 of SSIM on Shanghai Tech Part A, outperforming MCNN, CP-CNN, and CSRNet.
- Attention Feature Fusion: Concatenation produces lower estimation errors than element-wise summation when fusing SAM and CAM features on Shanghai Tech Part B.
- Attention Map Visualization: Spatial attention captures large-range context, while channel attention accurately locates head positions in visualized crowd scenes.
- Attention Map Visualization: SCAR combines context and location information to predict the density map.
6. Conclusion
SCAR uses parallel spatial-wise and channel-wise attention modules atop a VGG-16 backbone to generate density maps and estimate crowd counts. The authors describe the architecture as flexible for encoding large-range contextual information and suggest applying it to other pixel-wise tasks.
- SCAR uses parallel spatial-wise and channel-wise attention modules atop a VGG-16 backbone to generate density maps and estimate crowd counts.
- The authors state that SCAR is flexible because its attention modules can be embedded into any CNN.
- The paper suggests applying SCAR to pixel-wise tasks such as saliency detection and image segmentation, leaving verification for future work.