Source-linked AI summary
Learn to Scale: Generating Multipolar Normalized Density Maps for Crowd Counting
Chenfeng Xu, Kai Qiu, Jianlong Fu, Song Bai, Yongchao Xu, Xiang Bai
TL;DR
Extreme density variance and pattern shifts remain a challenge for crowd counting. The paper learns to scale image patches toward density-level centers using a multipolar center loss. It reports consistent improvements over state-of-the-art methods on three challenging datasets and generalization to unseen datasets with different density distributions.
Problem
Extreme density variances cause density-pattern shifts that remain challenging for crowd counting, including for multi-scale approaches.
Source: §1 · pp. 1–2Method
L2SM learns patch scale factors under a multipolar center loss to bring regions at each density level toward density centers.
Source: §1 · p. 2, §3.3 · p. 5Results
L2SM achieves consistent and significant improvements over state-of-the-art methods on three challenging datasets and generalizes to unseen datasets with significantly different density distributions.
Source: §5 · p. 8Takeaways & Limitations
The authors conclude that L2SM makes density estimation on dense regions more robust and has generalization ability relevant to real applications.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Dense crowd counting aims to predict thousands of human instances from an image, by calculating integrals of a density map over image pixels. Existing approaches mainly suffer from the extreme density variances. Such density pattern shift poses challenges even for multi-scale model ensembling. In this paper, we propose a simple yet effective approach to tackle this problem. First, a patch-level density map is extracted by a density estimation model and further grouped into several density levels which are determined over full datasets. Second, each patch density map is automatically normalized by an online center learning strategy with a multipolar center loss. Such a design can significantly condense the density distribution into several clusters, and enable that the density variance can be learned by a single model. Extensive experiments demonstrate the superiority of the proposed method. Our work outperforms the state-of-the-art by 4.2%, 14.3%, 27.1% and 20.1% in MAE, on ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF-QNRF datasets, respectively.
1. Introduction
Extreme density variation and pattern shifts remain difficult for crowd counting, even with multi-scale approaches. The paper proposes learning to scale image patches so their density distributions approach multiple centers, reporting improvements on challenging datasets and cross-dataset transferability.
1. Introduction
- Large density variances shift crowd-image patterns, making accurate prediction difficult for a single CNN and challenging even for multi-scale approaches.Recent methods also show higher mean relative error on datasets with larger density variances.Source: §1 · pp. 1–2
- L2SM learns patch scale factors under a multipolar center loss, bringing patches at each density level toward an online-updated density center.The method divides an initial density map into patches and density-level groups, then scales the patches and concatenates them into the final map.Source: §1 · p. 2
- L2SM is end-to-end trainable when added to a CNN density-estimation model.The design is intended to scale regions to similar densities and improve density-map quality.Source: §1 · p. 2
- L2SM added to SPN outperforms state-of-the-art methods on three challenging datasets and shows transferability under cross-dataset validation.The experiments include ShanghaiTech, UCF CC 50, and UCF-QNRF.Source: §1 · p. 2
2. Related Work
Related work covers CNN density-map regression, methods for scale and density variation, and approaches to over-fitting. The paper distinguishes its approach by learning to scale dense regions toward multiple similar density levels.
2. Related Work
- CNN-based crowd counting methods regress density maps and have become mainstream after detection- and regression-based approaches.Earlier detection methods struggled with occlusion and dense areas, while regression methods improved counting in dense areas.Source: §2 · p. 2
Methods addressing huge scale and density variance
- Prior methods for scale variation either use explicit scale information or implicitly combine multi-scale features and context.Explicit approaches may rely on scale information that is not readily available; implicit approaches include multi-column networks and changes to receptive fields or training losses.Source: §2 · p. 2
Methods alleviating severe over-fitting
- Small crowd-counting datasets also motivate methods that use unlabeled images or diverse regressors to address over-fitting.The related work links limited dataset size and variety to weak performance and transferability.Source: §2 · p. 3
2. Related Work
- Unlike prior approaches, this paper learns to scale dense regions and uses multipolar center loss to bring regions with varied densities toward multiple similar density levels.The authors connect this density centralization to robust estimation on dense regions and transferability.Source: §2 · p. 3
3. Method
The method addresses density-pattern shifts by generating an initial density map, then scaling selected dense regions and re-predicting them. Multipolar center loss organizes relative density levels, while transformed targets and a joint training objective support the full pipeline.
3.1. Overview
- The pipeline first predicts density with SPN, then selects dense regions for L2SM to scale and re-predict using scaled features.The method is trained end-to-end.Source: §3.1 · p. 3
3.2. Scale Preserving Network
- SPN stacks pooled features from multiple network stages into a pyramid representation and uses it to regress an initial density map.The pooled feature scales are 16 × 16, 8 × 8, 4 × 4, 2 × 2, and 1 × 1.Source: §3.2 · p. 4, Figure 3 · p. 4
3.3. Learning to Scale Module
- L2SM divides the image into non-overlapping regions, selects dense ones, and re-predicts them from scaled features while preserving their counting result.The initial prediction on selected regions is replaced by the resized re-prediction, whose density is multiplied by the squared scale ratio.Source: §3.3 · p. 4
- For selected regions, the method recomputes ground-truth density maps after scaling head locations, enlarging blob distances while keeping the original peaks.This transformation reduces the density-pattern gap between sparse and dense regions.Source: §3.3 · p. 4, Figure 4 · p. 5
- Multipolar center loss brings selected regions’ relative densities toward multiple learned centers, reducing the density-pattern shift between sparse and dense regions.The centers are updated during training alongside the network parameters.Source: §3.3 · p. 5
3.4. Training objective
- The end-to-end objective combines three losses, and optimization updates both network parameters and the density centers.The losses include terms for initial prediction, re-prediction, and multipolar center supervision.Source: §3.4 · p. 5
4. Experiments
Experiments evaluate the method on crowd-counting benchmarks, compare it with prior methods, and test design choices and cross-dataset transfer. Results report strong benchmark performance and generalization to unseen datasets.
4.1. Datasets and Evaluation Metrics
- Experiments use ShanghaiTech, UCF CC 50, and UCF-QNRF, with MAE and MSE as evaluation metrics.UCF CC 50 is evaluated with 5-fold cross-validation; UCF-QNRF images vary widely in crowd size and resolution.Source: §4.1 · p. 5
4.3. Experimental Comparisons
- The proposed method outperforms all competing methods on all three benchmarks; Table 1 presents the quantitative comparison.L2SM beats SANet by 2.8 MAE and 6.1 MSE on ShanghaiTech Part A, and by 1.2 MAE and 2.5 MSE on Part B.Source: Table 1 · p. 6, §4.3 · pp. 6–7
4.2. Implementation Details
- Training defaults to G=5 density groups, C=3 centers (C=5 for UCF CC 50), and K=4 image regions per dimension.At inference, dense regions are selected using their initial average density and replaced with scaled re-predictions; UCF-QNRF images above 1080p are down-sampled and divided into four input tiles.Source: §4.2 · p. 6
4.4. Ablation Study
- With G=5, centralizing more regions slightly improves counting until including every region adds background noise; L2SM also performs much better than S2AD.The authors report that finer group divisions with a proper number of centers perform slightly better.Source: Table 2 · p. 6, §4.4 · p. 7
- Center loss improves counting accuracy across a wide range of its weights, while ground-truth transformation is more accurate than straightforward density-map scaling.Figure 5 compares center-loss weights and the use of ground-truth transformation; the transformation enlarges distances between crowded heads to create regular Gaussian density blobs.Source: Figure 5 · p. 7, §4.4 · p. 8
- Performance remains stable across different K × K image-domain divisions under the one-center setting.Figure 6 displays original images, SPN predictions, L2SM re-predictions on selected regions, and ground-truth density maps.Source: Table 3 · p. 6, §4.4 · p. 8, Figure 6 · p. 8
4.5. Evaluation of Transferability
- Cross-dataset tests show good generalization to unseen datasets, with consistent large-margin gains over D-ConvNet-v1 and MCNN.L2SM performs slightly better than L2R [18] when transferring from ShanghaiTech Part A to UCF CC 50; the authors note L2R uses extra data.Source: Table 4 · p. 7, §4.5 · p. 8
5. Conclusion
L2SM addresses large density variation in crowd counting by centralizing density with multipolar center loss, and the paper reports improvements on three challenging datasets and generalization to unseen datasets with different density distributions.
5. Conclusion
- L2SM uses multipolar center loss to centralize density and scale dense regions to similar density levels, improving dense-region estimation robustness.Experiments on three challenging datasets show consistent improvements over state-of-the-art methods, and L2SM generalizes to unseen datasets with significantly different density distributions.Source: §5 · p. 8