PaperlayerView exact revision on arXiv

Source-linked AI summary

Top-Down Feedback for Crowd Counting Convolutional Neural Network

Deepak Babu Sam, R. Venkatesh Babu

arXiv:1807.08881v2cs.CV

TL;DR

Dense-crowd counting is challenged by small, variable, occluded people and feedforward CNNs that mistake local crowd-like patterns for humans. The paper introduces a bottom-up and top-down CNN architecture whose multiplicative feedback corrects lower-layer activations, achieving better or competitive results across major crowd-counting datasets.

  • Problem

    Dense crowds contain small, variable, occluded people, while feedforward CNN regressors can mistake local crowd-like patterns for humans.

  • Method

    A separate top-down CNN uses high-level bottom-up features to generate multiplicative gating feedback for lower layers of a two-column bottom-up density regressor.

  • Results

    The model delivers better or competitive results on all major crowd-counting datasets, while ablations demonstrate the effectiveness of top-down feedback.

  • Takeaways & Limitations

    Top-down feedback selectively passes legitimate activations and damps spurious responses in crowd-density prediction.

Abstract

from arXiv · show

Counting people in dense crowds is a demanding task even for humans. This is primarily due to the large variability in appearance of people. Often people are only seen as a bunch of blobs. Occlusions, pose variations and background clutter further compound the difficulty. In this scenario, identifying a person requires larger spatial context and semantics of the scene. But the current state-of-the-art CNN regressors for crowd counting are feedforward and use only limited spatial context to detect people. They look for local crowd patterns to regress the crowd density map, resulting in false predictions. Hence, we propose top-down feedback to correct the initial prediction of the CNN. Our architecture consists of a bottom-up CNN along with a separate top-down CNN to generate feedback. The bottom-up network, which regresses the crowd density map, has two columns of CNN with different receptive fields. Features from various layers of the bottom-up CNN are fed to the top-down network. The feedback, thus generated, is applied on the lower layers of the bottom-up network in the form of multiplicative gating. This masking weighs activations of the bottom-up network at spatial as well as feature levels to correct the density prediction. We evaluate the performance of our model on all major crowd datasets and show the effectiveness of top-down feedback.

Introduction

Dense crowds are difficult to count because people appear as small, variable, partially occluded blobs, while CNN regressors can mistake background patterns for humans. The paper proposes top-down feedback to use scene context to correct these false density predictions.

  • Introduction: Dense-crowd counting is difficult because people occupy few pixels and vary in appearance, pose, viewpoint, illumination, and occlusion.
  • Introduction: CNN regressors learn crowd patterns rather than individual-person features, causing clutter, trees, and other crowd-like backgrounds to produce false detections.
  • Introduction: High-level scene context can help evaluate ambiguous low-level features and correct information lost in early feedforward processing.
  • Introduction: The proposed architecture combines a bottom-up density regressor with a separate top-down CNN whose feedback gates lower-layer activations multiplicatively.
  • Introduction: The paper contributes a generic top-down feedback architecture and a crowd-counting system that corrects density predictions.

Related Work

Prior crowd-counting methods progressed from hand-crafted head and crowd features to CNN regressors, but related vision work generally used high-level context without feeding it back into lower layers. This paper distinguishes its approach by driving multiplicative feedback to bottom-up feature activations.

  • Related Work: Early crowd-counting systems used hand-crafted head detectors, HOG, SIFT, and Fourier features, but dense scenes made appearance features unreliable.
  • Related Work: CNN regressors outperformed hand-crafted approaches, while direct count prediction and scene-specific adaptation remained limited by feature learning or required data.
  • Related Work: Several vision approaches use high-level context for the task, but do not provide feedback to the bottom-up network.
  • Related Work: Unlike those approaches, the proposed architecture sends top-down feedback to lower bottom-up layers through multiplicative gating that suppresses unwanted activations.

Our Approach

TDF-CNN combines a bottom-up density regressor with a top-down network that uses scene context to gate lower-level activations and correct crowd predictions. It trains the bottom-up and feedback networks in separate stages, using multiplicative spatial and feature-level gating.

  • Architecture: A two-column bottom-up CNN predicts density maps, while a separate top-down CNN generates feedback from higher-level feature maps.The columns use different receptive fields, and their fused final features produce the density map.
  • Design rationale: Feedback is applied to lower convolutional features rather than the input image so higher layers retain the context needed for prediction.The top-down network uses features before pooling to preserve information that spatial pooling might otherwise discard.
  • Feedback gating: Top-down feedback multiplies gate maps with early bottom-up features, controlling activations across spatial locations, feature maps, and CNN columns.This gating can emphasize the column better suited to a scene’s crowd scale while suppressing spurious responses.
  • Training and inference: The model first predicts with the bottom-up network, then derives feedback maps and applies them to generate a corrected density map.Training likewise proceeds in two stages: the bottom-up parameters are fixed before training the feedback network.
  • Training objective: The bottom-up CNN is pretrained with an L2 density-map loss, whereas the top-down network uses count loss with an L1 regularizer on feedback features.The count objective prioritizes final counting accuracy, and the regularizer encourages sparse feedback activations.

Experiments

TDF-CNN is evaluated on four major crowd-counting datasets using MAE and MSE, with comparisons emphasizing accuracy, robustness, and parameter count. On Shanghaitech, it outperforms other models on both metrics with fewer parameters, while results on UCF CC 50 are competitive with Hydra2s.

  • Evaluation Scheme: During testing, feedback gate features are generated first and then applied to the bottom-up CNN to produce the final density map.The predicted density maps are one-quarter the input-image size because of two pooling layers.
  • Evaluation Scheme: The evaluation uses MAE for accuracy, MSE for estimation variance, and model-parameter count for comparison efficiency.MAE represents model accuracy, while MSE indicates robustness of count prediction.
  • Shanghaitech dataset: TDF-CNN outperforms all other models on Shanghaitech Part A and Part B in both MAE and MSE, using fewer parameters.Table 1 reports the comparison and states that TDF-CNN performs better on all metrics.
  • UCF CC 50 dataset: On UCF CC 50, TDF-CNN performs better than all compared models except Hydra2s.The dataset is evaluated with five-fold cross-validation because it contains 50 highly dense crowd scenes.
  • WorldExpo’10 dataset: On WorldExpo’10, TDF-CNN is trained and tested using the provided regions of interest.The dataset contains relatively sparse crowds averaging 50 people per image and provides perspective maps.

Analysis

Top-down feedback substantially improves crowd-counting accuracy by selectively suppressing spurious activations while preserving useful spatial and feature-level information.

  • Dataset Evaluation: The feedback model achieves better MAE in three of five WorldExpo’10 scenes and a lower average MAE.The supplied table caption reports scene-level and average comparisons without giving the numeric averages.
  • Effectiveness of Feedback: Fine-tuning the bottom-up network with count loss alone performs worse than TDF-CNN and distorts spatial density-map quality to some extent.The comparison separates the feedback mechanism from the effect of count-loss training.
  • Effectiveness of Feedback: Feedback remains effective with a single 9×9 CNN column, reducing MAE by 21.4% relative to the no-feedback model.This supports applying the top-down framework across different bottom-up architectures.
  • Gate Analysis: Gate maps mostly contain values near zero and selectively weight spatial regions across feature maps rather than uniformly suppressing them.Complementary gates allow blocked regions in one map to remain available in another.

Conclusion

The paper addresses false crowd detections from local pattern matching by using top-down scene context to gate bottom-up CNN activations. It reports better or competitive results across major crowd-counting datasets and supports the feedback mechanism through ablations.

  • Conclusion: Top-down feedback carries high-level scene context to correct spurious crowd detections produced by conventional crowd-pattern-based CNNs.A separate top-down CNN generates gating signals for lower-level bottom-up activations.
  • Conclusion: The feedback model delivers better or competitive results on all major crowd-counting datasets.The paper also demonstrates the feedback mechanism’s effectiveness with ablation studies.
Loading 1807.08881v2…