PaperlayerView exact revision on arXiv

Source-linked AI summary

CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes

Yuhong Li, Xiaofan Zhang, Deming Chen

arXiv:1802.10062v4cs.CV

TL;DR

Highly congested scenes require both accurate counting and spatially informative density maps, while existing multi-column approaches introduce training and redundancy concerns. CSRNet uses a pure-convolutional network with a CNN front-end and dilated-convolution back-end to generate density maps. It achieves state-of-the-art results across crowd datasets and extends effectively to vehicle counting.

  • Problem

    Highly congested-scene analysis needs accurate counts and spatially coherent density maps because identical crowd counts can represent different distributions.

    Source: §1 · p. 1

  • Method

    CSRNet combines the first 10 VGG-16 convolutional layers as a front-end with dilated convolutional layers that enlarge receptive fields without losing resolution.

    Source: §1 · p. 2, §3.1.1 · p. 4

  • Results

    CSRNet delivers state-of-the-art performance across four crowd-counting datasets and achieves 47.3% lower MAE than CP-CNN on ShanghaiTech Part B.

    Source: §1 · p. 2, §4.3.1 · p. 6, §5 · p. 8

  • Takeaways & Limitations

    The pure-convolutional architecture supports end-to-end training, high-quality density-map generation, and extension from crowd counting to vehicle counting.

    Source: §1 · p. 2, §5 · p. 8

    Prior multi-column approaches require more training time, contain redundant branches, and depend on difficult-to-define density-level classification.

    Source: §2.5 · p. 3

Abstract

from arXiv · show

We propose a network for Congested Scene Recognition called CSRNet to provide a data-driven and deep learning method that can understand highly congested scenes and perform accurate count estimation as well as present high-quality density maps. The proposed CSRNet is composed of two major components: a convolutional neural network (CNN) as the front-end for 2D feature extraction and a dilated CNN for the back-end, which uses dilated kernels to deliver larger reception fields and to replace pooling operations. CSRNet is an easy-trained model because of its pure convolutional structure. We demonstrate CSRNet on four datasets (ShanghaiTech dataset, the UCF_CC_50 dataset, the WorldEXPO'10 dataset, and the UCSD dataset) and we deliver the state-of-the-art performance. In the ShanghaiTech Part_B dataset, CSRNet achieves 47.3% lower Mean Absolute Error (MAE) than the previous state-of-the-art method. We extend the targeted applications for counting other objects, such as the vehicle in TRANCOS dataset. Results show that CSRNet significantly improves the output quality with 15.4% lower MAE than the previous state-of-the-art approach.

1. Introduction

Congested-scene analysis must represent spatial crowd distributions, not only total counts, because identical counts can correspond to different scenes. CSRNet addresses this need with a deeper convolutional density-map generator and reports lower MAE than prior methods across several datasets.

Motivation

  • Identical crowd counts can correspond to different spatial distributions, making density maps more informative than counts alone for high-risk decisions.Figure 1 illustrates three ShanghaiTech Part B images containing 95 people but having different spatial arrangements.Source: §1 · p. 1, Figure 1 · p. 1
  • Pixel-by-pixel density prediction requires spatially coherent maps with smooth transitions between neighboring pixels.Irregular crowd clusters and varying camera perspectives make this prediction difficult.Source: §1 · p. 1

Limitations of prior approaches

  • Multi-scale architectures can require substantial training time and introduce ineffective branching as networks become deeper.The introduction identifies MCNN as an example of this branch-structure problem.Source: §1 · p. 1
  • MCNN columns with small, medium, and large kernels show similar estimated-error patterns across 50 ShanghaiTech Part A test cases, suggesting redundant learned features.This behavior conflicts with the intended role of separate columns learning different features for different congestion levels.Source: Figure 1 · p. 1, §1 · p. 2

Contribution

  • CSRNet uses a deeper CNN-based density-map generator and reports lower MAE than the previous state-of-the-art CP-CNN across ShanghaiTech, UCF CC 50, and WorldExpo’10.The reported reductions are 7%, 47.3%, 10.0%, and 2.9% for ShanghaiTech Part A, Part B, UCF CC 50, and WorldExpo’10, respectively; UCSD performance is 1.16 MAE.Source: §1 · p. 2

2. Related work

Related work spans detection-, regression-, density-estimation-, and CNN-based approaches, with CNN methods improving congested-scene analysis. The paper highlights redundancy, training cost, and classification overhead in multi-column CNN designs, motivating a simpler alternative.

Approach categories

  • Crowd-scene methods are commonly classified as detection-based, regression-based, or density-estimation-based approaches.CNN-based solutions are described as outperforming traditional methods in this task.Source: §2 · p. 2

Detection-based methods

  • Detection-based methods perform poorly in highly congested scenes because targeted people are often obscured.Some methods instead detect body parts rather than complete bodies.Source: §2.1 · p. 2

Regression-based methods

  • Regression-based methods learn feature-to-count relations from cropped patches, but local saliency can be overlooked and produce inaccurate regional results.Later work learns mappings to object density maps to incorporate local saliency.Source: §2.2 · p. 2, §2.3 · p. 3

CNN-based methods

  • CNN-based research includes patch-based, end-to-end, selective-sampling, boosting, and density-map prediction methods.These approaches build on CNN success in classification and recognition.Source: §2.4 · p. 3

Limitations of multi-column CNNs

  • Multi-column methods such as Switch-CNN and Contextual Pyramid CNN achieve state-of-the-art results but are difficult to train and can contain redundant branches.Their designs also use density-level classifiers to route input patches among regressors or columns.Source: §2.5 · p. 3
  • Multi-column designs allocate parameters to density-level classification and may leave fewer resources for final density-map generation.Fine-grained density classification also increases the number of columns and design complexity.Source: §2.5 · p. 3

3. Proposed Solution

CSRNet combines a VGG-16 convolutional front-end with a dilated-convolution back-end to generate density maps while preserving resolution and enlarging receptive fields. Its configuration uses small convolutional kernels, selected pooling, geometry-adaptive ground-truth maps, patch augmentation, and end-to-end training.

Network architecture

  • CSRNet uses the first 10 VGG-16 convolutional layers as a front-end and dilated convolutional layers as a back-end for density-map generation.The model uses pure convolutional layers and supports input images with flexible resolutions.Source: §3 · p. 3, §3.1 · p. 3

Dilated convolution

  • Dilated convolution enlarges receptive fields without increasing parameters or computation while preserving feature-map resolution.A k × k kernel becomes k + (k −1)(r −1) with dilation stride r, enabling multi-scale contextual aggregation.Source: §3.1.1 · p. 4
  • Compared with pooling, convolution, and upsampling, dilation with factor = 2 produces an output matching the input dimension and retains more detailed information.The alternative scheme requires downsampling and subsequent upsampling, whereas dilated convolution does not require pooling or deconvolutional layers.Source: §3.1.1 · p. 4, Figure 4 · p. 4

Network configuration

  • The selected architecture keeps three pooling layers and the first ten VGG-16 layers to balance output accuracy with training time, memory, and parameter overhead.CSRNet outputs density maps at 1/8 of the input size and uses bilinear resolution as the input image.Source: §3.1.2 · pp. 4–5

Ground-truth generation

  • Ground-truth density maps blur head annotations with normalized geometry-adaptive Gaussian kernels whose σ_i depends on nearest-neighbor distances.The experiments use β = 0.3 and k = 3; sparse crowds instead use the average head size for blurring.Source: §3.2.1 · p. 5

Training details

  • Training uses cropped and mirrored patches, VGG-16 fine-tuning for the first 10 convolutional layers, Gaussian initialization elsewhere, SGD, and Euclidean distance.The training learning rate is fixed at 1e-6, and the loss compares estimated and ground-truth density maps.Source: §3.2.2 · p. 5, §3.2.3 · p. 5

4. Experiments

CSRNet is evaluated across crowd-counting and vehicle-counting datasets using error, density-map quality, and ablation analyses. The experiments report strong performance across varied scenes, with dilated configurations and ROI-aware evaluation supporting the comparisons.

Density-map evaluation

  • PSNR and SSIM are used to assess generated density-map quality after resizing and normalizing predicted and ground-truth maps.The supplementary material presents density-map samples across five datasets and varied density levels.Source: §4.1 · p. 6, Table 11 · p. 8

Ablation study

  • CSRNet B achieves the lowest error among four equal-parameter configurations on ShanghaiTech Part A and is selected for subsequent experiments.The configurations differ in their back-end dilation rates; dropout provides no significant improvement.Source: §4.2 · p. 6

ShanghaiTech dataset

  • 7% lower MAE than CP-CNN is achieved on ShanghaiTech Part A, while Part B shows 47.3% lower MAE than CP-CNN.CSRNet also achieves the highest SSIM and PSNR for density maps on ShanghaiTech Part A.Source: §4.3.1 · p. 6

Other crowd datasets

  • The evaluation covers datasets with varied crowd densities, perspectives, resolutions, and surveillance settings, including UCF CC 50 and UCSD.The supplied passages identify their estimation-error tables but do not report the corresponding numerical outcomes.Source: §4.3.2 · pp. 6–7, §4.3.4 · p. 7, Table 7 · p. 7

WorldExpo’10 dataset

  • CSRNet achieves the best average accuracy and the best accuracy in 4 of 5 WorldExpo’10 scenes.The dataset uses ROI-masked frames and dot maps during preprocessing.Source: §4.3.3 · p. 7

TRANCOS vehicle counting

  • 67.7%, 60.1%, 48.7%, and 22.2% lower GAME(0), GAME(1), GAME(2), and GAME(3), respectively, are reported against the method in on TRANCOS.GAME(L) sums regional MAE over a grid of 4^L non-overlapping regions; at L = 0 it equals MAE.Source: §4.3.5 · p. 8

5. Conclusion

The conclusion presents CSRNet as an easy-trained architecture for crowd counting and high-quality density-map generation. It attributes the design’s contextual aggregation to dilated convolutions and reports state-of-the-art crowd-counting and vehicle-counting performance.

  • CSRNet is proposed as an easy-trained end-to-end architecture for crowd counting and high-quality density-map generation.Source: §5 · p. 8
  • Dilated convolutional layers aggregate multi-scale contextual information while expanding the receptive field without losing resolution.Source: §5 · p. 8
  • The model achieves state-of-the-art performance on four crowd-counting datasets and best accuracy on the vehicle-counting task.Source: §5 · p. 8

7. Appendix: supplementary material

The appendix supplies additional CSRNet outputs from five datasets to demonstrate the design’s validity. It evaluates density-map quality with PSNR and SSIM across varied density levels.

  • Additional CSRNet results are presented for ShanghaiTech, UCF CC 50, WorldExpo’10, UCSD, and TRANCOS.Source: §7 · p. 11
  • PSNR and SSIM are used to evaluate the quality of generated density maps.Source: §7 · p. 11
  • The supplementary samples represent a variety of density levels across the five datasets.Source: §7 · p. 11
  • Figures 7 through 12 show original or ROI-masked images, ground-truth density maps, and CSRNet-generated density maps.Source: Figure 7 · p. 11, Figure 8 · p. 12, Figure 9 · p. 13

Related papers

Loading 1802.10062v4…