PaperlayerView exact revision on arXiv

Source-linked AI summary

Focal Inverse Distance Transform Maps for Crowd Localization

Dingkang Liang, Wei Xu, Yingying Zhu, Yu Zhou

arXiv:2102.07925v3cs.CV

TL;DR

Crowd localization needs precise individual positions, but density-map regression becomes ambiguous in dense scenes because Gaussian responses blur and overlap. The paper proposes FIDT maps with I-SSIM supervision and LMDS extraction, reporting state-of-the-art localization performance and robustness across negative and extremely dense scenes.

  • Problem

    Density-map regression cannot reliably distinguish individual locations in extremely dense scenes because Gaussian blobs blur and overlap.

  • Method

    The method regresses non-overlapping FIDT maps, uses I-SSIM to preserve local structure, and extracts head centers with LMDS.

  • Results

    The method achieves state-of-the-art localization performance and superior robustness on negative and extremely dense scenes.

  • Takeaways & Limitations

    FIDT map regression provides a practical alternative to density-map regression for crowd localization from point-level annotations.

  • Takeaways & Limitations

    Inference is slower than some real-time methods.

Abstract

from arXiv · show

In this paper, we focus on the crowd localization task, a crucial topic of crowd analysis. Most regression-based methods utilize convolution neural networks (CNN) to regress a density map, which can not accurately locate the instance in the extremely dense scene, attributed to two crucial reasons: 1) the density map consists of a series of blurry Gaussian blobs, 2) severe overlaps exist in the dense region of the density map. To tackle this issue, we propose a novel Focal Inverse Distance Transform (FIDT) map for the crowd localization task. Compared with the density maps, the FIDT maps accurately describe the persons' locations without overlapping in dense regions. Based on the FIDT maps, a Local-Maxima-Detection-Strategy (LMDS) is derived to effectively extract the center point for each individual. Furthermore, we introduce an Independent SSIM (I-SSIM) loss to make the model tend to learn the local structural information, better recognizing local maxima. Extensive experiments demonstrate that the proposed method reports state-of-the-art localization performance on six crowd datasets and one vehicle dataset. Additionally, we find that the proposed method shows superior robustness on the negative and extremely dense scenes, which further verifies the effectiveness of the FIDT maps. The code and model will be available at https://github.com/dk-liang/FIDTM.

I. Introduction

Crowd localization predicts each person’s head point from point-level annotations, but density-map regression struggles to distinguish individuals in dense scenes. The paper proposes FIDT maps, I-SSIM loss, and LMDS to improve localization and robustness.

  • Crowd localization predicts one head point per person from point-level annotations, whereas detection requires expensive bounding-box labels.
  • Gaussian density blobs overlap in dense regions, making individual locations and sizes difficult to recover from regression outputs.
  • FIDT maps use nearest-neighbor distance information so local maxima represent individual head centers without overlap, including in extremely dense crowds.
  • I-SSIM loss emphasizes independent local regions because background values near zero can otherwise produce false local maxima under traditional SSIM.
  • LMDS extracts predicted local maxima with max-pooling and an adaptive threshold, while also helping classify negative samples.
  • The method reports state-of-the-art localization performance and robustness on negative and extremely dense scenes.

II. Related Works

Prior crowd-analysis methods emphasize counting, detection, or localization using density, detection, and distance-based representations. This paper instead combines FIDT regression with local-structure supervision and local-maxima extraction for precise localization.

  • Crowd counting commonly regresses density maps, while crowd localization seeks precise head positions and detection relies on costly bounding-box annotations.
  • FIDT is a non-overlap representation in which higher responses nearer each head center make local maxima encode person locations.
  • Using only MSE can blur predictions and lose local structure, motivating losses that preserve instance-level regions for detecting head-center maxima.
  • Unlike thresholded semantic-like maps, distance-transform approaches provide a basis for using local maxima directly in regression-based crowd localization.
  • The method trains a regressor with MSE and I-SSIM, then applies LMDS to predicted FIDT maps and can estimate boxes with KNN.

A. Focal Inverse Distance Transform Map

The paper replaces overlapping density representations with FIDT maps whose local maxima encode individual head locations, then extracts those maxima using LMDS.

  • FIDT map: FIDT maps represent individual head locations through local maxima while avoiding overlaps between nearby heads.Unlike density maps, FIDT maps remain discriminative in extremely dense crowds.
  • FIDT map: FIDT modifies the IDT response so decay is slower away from head centers while background responses approach zero.The proposed parameters are α = 0.02 and β = 0.75.
  • Regression network: The method uses a high-resolution HRNET-based regressor because FIDT-map centers correspond to local maxima.The base network adds one convolution and two transposed convolution layers as the representation head.
  • Local-Maxima-Detection-Strategy: LMDS applies 3 × 3 max-pooling to find candidate local maxima, then uses an adaptive threshold Ta = 100/255.0 × max(M) to filter false positives.If max(M) is below Tf = 0.10, LMDS returns zero detections for negative samples.
  • Instance-size estimation: Predicted head centers can also support pseudo bounding-box generation by estimating instance size from K-nearest-neighbor distances.A testing-phase threshold restrains estimated sizes in very sparse regions, without affecting localization performance.

C. Independent SSIM Loss

The paper supplements MSE supervision with an Independent SSIM loss that emphasizes structural information in separate head regions and reduces background-driven false maxima.

  • Motivation: MSE-only supervision can introduce blur and lose local structural information in predicted maps.SSIM loss is introduced as a way to improve predicted-map quality.
  • Motivation: Traditional SSIM scans foreground and background together, potentially producing high background responses and false local maxima.This is problematic because localization depends on detecting FIDT-map local maxima.
  • Independent SSIM: I-SSIM computes structural similarity over independent instance regions, using estimated and ground-truth regions for each person.Each instance region is set to 30 × 30 to contain the head while limiting redundant background.
  • Training objective: The final training objective combines MSE loss with the proposed I-SSIM loss.The paper denotes these components as LMS E and LI−S, respectively.

IV. Implement details

The implementation augments training data, sets dataset-dependent crop and resize rules, and uses fixed optimization and bounding-box-generation parameters.

  • Training uses random cropping and horizontal flipping, with 256 × 256 crops for Part A/B and 512 × 512 crops for other datasets.
  • The method sets k = 4, f = 0.1, α = 0.02, and β = 0.75 for bounding-box generation and training.
  • Adam optimization uses a 1e-4 learning rate, 5e-4 weight decay, and batch size 16.
  • Images from NWPU-Crowd, JHU-Crowd++, and UCF-QNRF are resized so their longer side is smaller than 2048.
  • The proposed KNN strategy generates green bounding boxes for qualitative comparison with LSC-CNN.

A. Evaluation metrics

Evaluation measures localization using thresholded point matching and counting using MAE and MSE, with thresholds varying across datasets.

  • Localization matches a predicted point to a ground-truth point when their distance is below threshold σ, then computes Precision, Recall, and F-measure.
  • NWPU-Crowd uses two head-size-related thresholds, with the former criterion stricter than the latter.
  • UCF-QNRF evaluates localization at thresholds from 1 through 100 pixels.
  • JHU-Crowd++, ShanghaiTech Parts A/B, and UCF CC 50 use fixed thresholds σ = 4 and σ = 8.
  • Counting performance is measured with Mean Absolute Error and Mean Square Error over testing images.

B. Dataset

The evaluation covers six crowd datasets with varied density, image conditions, and annotation scales, and compares localization against established methods using reported benchmark results and visualizations.

  • The study evaluates the method on six challenging public datasets and reports localization comparisons using official predictions and evaluation code.
  • NWPU-Crowd contains 5,109 images and 2,133,375 instances, including 351 negative samples for robustness testing.
  • JHU-Crowd++ spans 2,722 training, 500 validation, and 1,600 test images, with counts ranging from 0 to 25,791 persons.
  • UCF-QNRF contains 1,535 images and about one million annotations, while ShanghaiTech provides 1,198 images across Parts A and B.
  • UCF CC 50 contains 50 grayscale images from extremely congested scenes, and TRANCOS contains 1,244 traffic-congestion images with 46,796 annotations.
  • The method improves over TopoCount by 8.7% F-measure on JHU-Crowd++ at σ = 4 and by 17.5% and 1.5% on ShanghaiTech Parts A and B.
  • On UCF CC 50, the method exceeds other localization methods by more than 7% F-measure at σ = 4 and remains robust to degraded images.
  • Qualitative visualizations compare testing images, ground-truth maps, predicted FIDT maps, point maps, and bounding boxes, including comparisons with LSC-CNN.

B. Crowd counting

The local maxima in FIDT maps directly provide counts, enabling counting evaluation alongside localization and comparisons with density-map regression methods.

  • The total crowd count equals the number of local maxima in the predicted FIDT map.
  • The method reduces NWPU-Crowd test-set MSE by 27.4 compared with the localization-based SCALNet method.
  • The method achieves the best counting performance on UCF-QNRF, ShanghaiTech Parts A/B, and UCF CC 50, while reaching state-of-the-art MSE on JHU-Crowd++.
  • Against density-map regression methods, the method outperforms them on NWPU-Crowd, JHU-Crowd++, and UCF CC 50, with comparable performance on three other datasets.
  • Density maps lose position information through severe overlaps, whereas FIDT maps provide nearly accurate individual locations in extremely dense scenes.

C. Evaluation on vehicle dataset

On TRANCOS, the method generalizes FIDT-based localization to vehicle counting and achieves strong performance across global and regional GAME metrics.

  • Evaluation on vehicle dataset: GAME evaluates vehicle-counting error across 4^L non-overlapping image subregions using predicted and ground-truth counts.The metric measures both global and localized counting accuracy.
  • Evaluation on vehicle dataset: The method is evaluated for vehicle counting on the TRANCOS dataset as a test of generalization to a related task.The evaluation follows previous localization methods and uses the GAME metric.
  • Evaluation on vehicle dataset: The proposed method achieves the best performance on GAME(0), GAME(1), and GAME(2), with comparable performance on GAME(3).This indicates accurate global predictions alongside strong localization performance.
  • Generalization on different regressors: The FIDT map can support localization and competitive counting performance with different regressors on NWPU-Crowd.CSRNET and BL variants, including versions with FPN context, are evaluated.

E. Limitation

The method has higher inference latency than some real-time approaches, despite favorable reported complexity relative to other methods.

  • Limitation: Inference is slower than some real-time methods, limiting the current method’s suitability for real-time deployment.The authors identify faster inference as future work.
  • Limitation: MACs for Crowd-SDNet could not be calculated because its official implementation depends on an unavailable old Keras version.This affects the completeness of the complexity comparison.
  • Limitation: The reported complexity analysis uses MACs and FPS on 768 × 1024 images with an NVIDIA RTX 3090.The method achieves the second-lowest MACs and third-highest FPS in the comparison.

VI. Conclusion

The paper concludes that FIDT maps, LMDS, and I-SSIM enable accurate crowd localization with robustness to negative and extremely dense scenes. The authors present FIDT regression as a more practical alternative to density-map regression.

  • Conclusion: FIDT maps represent head centers with non-overlapping local maxima for crowd localization.LMDS extracts the corresponding individual centers from the map.
  • Conclusion: I-SSIM improves the structural information of local maxima by focusing learning on foreground regions.The loss is designed to support recognition of individual centers.
  • Conclusion: Experiments on six public datasets show state-of-the-art localization performance and superior robustness on negative and extremely dense scenes.The conclusion summarizes the reported empirical scope of the method.
Loading 2102.07925v3…