Source-linked AI summary
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Network
Xiaolong Jiang, Zehao Xiao, Baochang Zhang, Xiantong Zhen, Xianbin Cao, David Doermann, Ling Shao
TL;DR
Crowd counting requires accurate density maps despite degraded spatial precision and limited multi-scale fusion in common encoder-decoder networks. TEDnet uses a trellis architecture with multi-path decoding, dense skip connections, and distributed combinatorial losses; it achieves state-of-the-art results on four benchmarks, improving MAE by up to 14%.
Problem
Common encoder-decoder crowd-counting networks lose localization precision through down-sampling and lack hierarchical multi-scale fusion, while MSE neglects spatial correlation and local coherence.
Method
TEDnet combines a spatially preserving multi-scale encoder, hierarchical multi-path decoder with dense skip connections, and distributed combinatorial losses for intermediate outputs.
Results
14%: TEDnet achieves the best overall performance on four benchmark datasets, surpassing state-of-the-art methods by up to 14% in MAE and obtaining the best density-map quality in PSNR and SSIM.
Takeaways & Limitations
TEDnet provides a unified architecture for improving both crowd-counting accuracy and the quality of estimated density maps.
Takeaways & Limitations
TEDnet's pervasive feature fusion exceeds what sparsely linked skip connections in single-path hourglass encoder-decoders can reach, defining the comparison scope described here.
Abstract
from arXiv · showhide
Crowd counting has recently attracted increasing interest in computer vision but remains a challenging problem. In this paper, we propose a trellis encoder-decoder network (TEDnet) for crowd counting, which focuses on generating high-quality density estimation maps. The major contributions are four-fold. First, we develop a new trellis architecture that incorporates multiple decoding paths to hierarchically aggregate features at different encoding stages, which can handle large variations of objects. Second, we design dense skip connections interleaved across paths to facilitate sufficient multi-scale feature fusions and to absorb the supervision information. Third, we propose a new combinatorial loss to enforce local coherence and spatial correlation in density maps. By distributedly imposing this combinatorial loss on intermediate outputs, gradient vanishing can be largely alleviated for better back-propagation and faster convergence. Finally, our TEDnet achieves new state-of-the art performance on four benchmarks, with an improvement up to 14% in terms of MAE.
1. Introduction
Crowd counting methods based on density estimation preserve crowd localization but face degraded spatial precision and insufficient multi-scale feature fusion. TEDnet addresses these issues with a trellis encoder-decoder, dense skip connections, distributed combinatorial losses, and full-image processing, achieving strong benchmark results.
- Density estimation methods generate pixel-wise maps that preserve crowd localization, with total crowd count computed by integrating the density map.
- Classification-oriented CNN encoders progressively down-sample feature maps, degrading spatial resolution and localization precision in heavily occluded, scale-varying scenes.
- Single-path encoder-decoders suffer precision loss from repeated down- and up-sampling, while conventional skip connections lack hierarchical multi-scale feature fusion.
- TEDnet hierarchically assembles multiple encoding-decoding paths to aggregate features at different stages and generate high-quality density maps.
- Dense skip connections progressively fuse multi-scale features, while distributed combinatorial losses supervise local coherence and spatial correlation through intermediate multi-path outputs.
- 14%: TEDnet surpasses state-of-the-art methods by up to 14% for the MAE metric across four benchmark datasets, while achieving the best density-map quality in PSNR and SSIM.
2. Related Work
Related crowd-counting methods trade off detection robustness, direct counting, and localization. Density estimation methods retain localization through pixel-wise regression, while TEDnet emphasizes spatial preservation and hierarchical multi-scale fusion.
- Detection-based methods localize and count targets explicitly, but detection performance is affected in overcrowded scenes.
- Regression-based methods avoid explicit detection by mapping images directly to scalar counts, but they forfeit the ability to perceive crowd distributions.
- Density estimation methods recover localization by conducting pixel-wise regression to produce crowd density maps.
- TEDnet preserves spatial information in the encoding hierarchy and uses a multi-path decoder to fuse spatial and semantic multi-scale features.
3. Trellis Encoder-Decoder Networks
TEDnet preserves spatial precision while using a multi-path trellis decoder to aggregate and fuse multi-scale features. Distributed combinatorial losses supervise intermediate outputs and target spatial coherence in density maps.
- 3.1. Trellis Encoder: TEDnet uses a multi-scale encoder with limited pooling to preserve pixel-wise localization precision while addressing occlusion and scale variation.The encoder uses different kernel sizes and only two 2 × 2 max-pooling layers in its first stages.
- 3.2. Multi-Path Decoder: The multi-path decoder hierarchically aggregates features within paths and fuses different levels across paths through dense skip connections.Three decoding paths operate on features from the last three encoding stages, progressively combining spatial and semantic information.
- 3.2. Multi-Path Decoder: The final density map is generated from the rightmost decoding path after decoded features have aggregated multi-scale spatial and semantic information and spatial resolution is restored.The rightmost path produces the richest decoded feature representation before up-sampling.
- 3.3. Distributed Supervision: Intermediate outputs receive distributed supervision, while dense skip connections provide multiple gradient-flow paths that alleviate gradient vanishing.The architecture produces outputs at the ends of decoding paths, allowing multiple losses to contribute during back-propagation.
- 3.4. Combinatorial Loss: The combinatorial loss combines SAL and SCL to enforce multi-level local coherence and map-wise spatial correlation beyond pixel-wise MSE.SAL computes MSE across progressively abstracted maps, while SCL compares density maps using normalized cross-correlation.
4. Experiments and Results
TEDnet is evaluated through implementation details, ablations, counting accuracy, density-map quality, and parameter comparisons across four crowd-counting benchmarks. The experiments show that its multi-path design, distributed supervision, and combinatorial loss improve convergence, counting, density-map quality, and overall efficiency.
- Implementation details: TEDnet uses full-size images and outputs same-size density maps, avoiding patch-wise boundary artifacts and patch-level statistical shifts.The predicted count is computed as the integral of each density map, while MAE and MSE evaluate counting accuracy; PSNR and SSIM evaluate map quality.
- Ablation Study: The ablation study evaluates trellis multi-path decoding, distributed supervision, and combinatorial loss using MAE and PSNR on ShanghaiTech Part A.Table 1 groups configurations by these three contributions, while Figure 7 illustrates convergence performance for different structures and supervision losses.
- Ablation Study: 2.6% lower MAE and 7.1% higher PSNR result from multi-path decoding, while dense skip connections also produce faster convergence.The comparison uses the same trellis encoder and normal MSE with single supervision on density map Z.
- Ablation Study: 2.8% lower MAE and 1.9% higher PSNR result from distributed supervision, while the combinatorial loss with SAL and SCL improves MAE by 7.2% and PSNR by 4.5%.The combinatorial-loss comparison uses distributed deployment of different losses with λ = 1; configuration 5 combines all three contributions.
- Counting Accuracy: TEDnet achieves the best overall performance across ShanghaiTech, UCF CC 50, WorldExpo’10, and UCF-QNRF, surpassing state-of-the-art methods by up to 14% in MAE.Reported dataset results include 4.2% MAE improvement on ShanghaiTech Part A, 3.5% on UCF CC 50, 14.4% on UCF-QNRF, and 2.4% average MAE improvement on WorldExpo’10.
- Density Map Quality: TEDnet obtains the best reported density-map quality, outperforming the second-best method by 8.1% in PSNR and 8.4% in SSIM on ShanghaiTech Part A.Compared with MCNN, TEDnet improves PSNR by 17.3% and SSIM by 36%, while using only 10% of CSRNet’s parameters and 2.4% of CP-CNN’s.
5. Conclusion
TEDnet is a crowd-counting architecture that combines multi-scale encoding, multi-path decoding, dense skip connections, and distributed combinatorial-loss supervision. Across four benchmarks, it achieves state-of-the-art density-map quality and counting accuracy.
- TEDnet combines a multi-scale encoder with a multi-path decoder to generate high-quality density estimation maps.The architecture preserves localization precision in encoded features while aggregating and fusing multi-scale information during decoding.
- Dense skip connections enable thorough aggregation and fusion of multi-scale features in the decoder.
- Distributed supervision applies a proposed combinatorial loss to intermediate outputs during training.
- Experiments on four benchmarks show state-of-the-art performance in both density-map quality and crowd-counting accuracy.