PaperlayerView exact revision on arXiv

Source-linked AI summary

Localization in the Crowd with Topological Constraints

Shahira Abousamra, Minh Hoai, Dimitris Samaras, Chao Chen

arXiv:2012.12482v1cs.CV

TL;DR

Crowd localization is difficult because dot annotations provide limited spatial information, leading to phantom or collapsed predictions. The paper introduces a topological constraint enforced by persistence loss, and reports improved localization plus 7 to 28% gains for density-based counting on public benchmarks. Its counting integration still trades away high-quality localization because density maps do not provide exact locations.

  • Problem

    Crowd localization remains challenging because dot annotations provide limited spatial supervision, making phantom dots and collapsed dots difficult to prevent.

  • Method

    TopoCount predicts a binary mask with one connected component per target dot and enforces local topology using a persistent-homology-based persistence loss.

  • Results

    7 to 28%: integrating TopoCount localization results improves density-based counting algorithms on several public benchmarks, while TopoCount achieves state-of-the-art localization accuracy.

  • Takeaways & Limitations

    Topological reasoning improves localization quality near cluttered regions and supplies spatial configuration information that strengthens density-based counting.

  • Takeaways & Limitations

    The combined counting approach outputs density maps that improve count approximation but cannot provide high-quality localization information.

Abstract

from arXiv · show

We address the problem of crowd localization, i.e., the prediction of dots corresponding to people in a crowded scene. Due to various challenges, a localization method is prone to spatial semantic errors, i.e., predicting multiple dots within a same person or collapsing multiple dots in a cluttered region. We propose a topological approach targeting these semantic errors. We introduce a topological constraint that teaches the model to reason about the spatial arrangement of dots. To enforce this constraint, we define a persistence loss based on the theory of persistent homology. The loss compares the topographic landscape of the likelihood map and the topology of the ground truth. Topological reasoning improves the quality of the localization algorithm especially near cluttered regions. On multiple public benchmarks, our method outperforms previous localization methods. Additionally, we demonstrate the potential of our method in improving the performance in the crowd counting task.

1 Introduction

Crowd localization remains difficult because dot annotations provide limited spatial supervision, causing phantom, missing, or collapsed predictions. TopoCount addresses these errors with local topological constraints and persistence loss, improving localization and downstream counting.

  • Motivation: Crowd localization is challenging because perspective, occlusion, clutter, and imprecise dot annotations limit spatial supervision.Dots may be arbitrarily located within instances, making instance boundaries difficult to learn.
  • Method: TopoCount predicts a binary Topological Map whose connected components correspond one-to-one with target dots.Training uses non-overlapping, slightly dilated dot masks as ground-truth dot maps.
  • Method: The local topological constraint teaches the model to reason about dot arrangement and avoids phantom dots and collapsed dots, especially in dense regions.The constraint is enforced within randomly sampled patches.
  • Method: Persistence loss compares modes in the likelihood map with the true topology using persistent homology.It operates on the neural network’s sigmoid likelihood map rather than directly on the predicted binary mask.
  • Results: TopoCount outperforms previous localization methods across multiple benchmarks and achieves state-of-the-art localization performance.The paper reports improvements across various localization metrics.
  • Results: 7 to 28%: adding TopoCount localization results as input improves density-based counting algorithms on several public benchmarks.The added spatial configuration information complements density-based counting.

2 Related Work

Prior work addresses crowd localization through detection, segmentation, or density estimation, but each approach has limitations in crowded or ambiguously annotated scenes. This paper positions topological information as a new approach for crowd localization and counting.

  • Localization approaches: Detection-based methods include bounding-box prediction and Faster R-CNN, but Faster R-CNN does not scale well with increasing occlusion and clutter.Other approaches jointly learn localization with counting models.
  • Localization approaches: Semantic-segmentation methods represent instances as blobs, splitting blobs with multiple dots and suppressing blobs without dots.Their blob decisions are not robust to perturbed dot locations.
  • Density estimation: Density-estimation methods learn a density function whose integral represents the estimated object count, but derived localization maps are limited in quality.The density representation prioritizes counting over exact spatial localization.
  • Topological methods: Topological information has been used in shape analysis, graph learning, clustering, segmentation, and model regularization.The paper states that its method is the first to exploit topological information for crowd localization and counting.

3 Method: TopoCount

TopoCount formulates localization as predicting a binary topological map whose connected components correspond one-to-one with people, then enforces this structure locally with persistence loss. It combines topological supervision with density-estimation methods to improve counting, while the combined output sacrifices precise localization.

  • Topological localization: TopoCount predicts a binary mask whose connected components represent individual people, with component centers serving as predicted dots.Training expands non-overlapping point annotations into dot masks, while inference thresholds the sigmoid likelihood map.
  • Topological localization: The local topological constraint requires each patch’s number of connected components to equal its number of ground-truth dots.This encodes spatial arrangement without requiring exact dot locations, helping avoid phantom dots and collapsed detections.
  • Persistence loss: Persistence loss uses the likelihood map’s topographic modes and their saliency to retain exactly c salient modes for a patch containing c ground-truth dots.It reinforces the top c modes and suppresses the remaining modes, reducing false positives and separating merged detections.
  • Persistence loss: Mode saliency is computed from maximum–saddle pairs using a merging-tree algorithm with near-linear complexity O(n log n + nα(n)).Persistence is the difference between a mode’s function value and its corresponding saddle value.
  • Counting integration: TopoCount integrates its localization maps with Bayesian and CAN density-estimation methods, improving state-of-the-art counting performance by 7 to 28%.The integration is model-agnostic and is motivated by the additional spatial configuration information in high-quality localization maps.
  • Counting integration: The combined counting approach outputs density maps only, so it cannot provide high-quality localization information despite better count approximation.This is the stated trade-off for improved counting performance.

4 Experiments

TopoCount is evaluated across crowd-counting and localization benchmarks using multiple accuracy measures, ablations, and integration experiments. It achieves strong localization results, competitive direct counting performance, and improves density-based counting when combined with its localization output.

  • Evaluation setup: TopoCount is evaluated on ShanghaiTech, UCF CC 50, UCF QNRF, JHU++, and NWPU-Crowd using localized counting, F1-score matching, and challenge metrics.The evaluation includes GAME, matching accuracy, and the NWPU localization challenge metric.
  • Localization performance: TopoCount achieves the lowest localized-counting error especially at the finest GAME scale, indicating higher localization accuracy by predicted dots.GAME aggregates counting error over increasingly fine image grids; at L=3, the method is especially strong.
  • Localization performance: TopoCount achieves the highest matching scores with a large margin at both σ = 5 and σ = 20, and the highest F-score under both NWPU thresholds.These results cover matching accuracy on ShanghaiTech Part A and UCF QNRF and the NWPU-Crowd localization challenge.
  • Counting performance: TopoCount achieves SOTA performance on JHU++ and ranks mostly second or third on other datasets against SOTA density-based counting methods.Direct counting is compared using Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE).
  • Ablation studies: Adding persistence loss lowers error beyond DICE-only training, with the most robust results for λ between 0.5 and 2.0 and best performance at λ = 1 and λ = 1.5.The experiments use λ = 1 in all reported experiments.
  • Ablation studies: Patch sizes of 30 or 150 perform poorly, while 50 or 100 are mostly similar; 50 performs best at GAME scale L=3 on ShanghaiTech Part A.The selected patch size is 50 for ShanghaiTech and UCF CC 50; larger-scale datasets require separate patch-size experiments.

5 Conclusion

The paper concludes that topological constraints and persistence loss provide flexible localization supervision for sparse and dense crowd regions. TopoCount achieves state-of-the-art localization accuracy and substantially boosts density-based counting when its outputs are used as additional input.

  • 5 Conclusion: TopoCount combines a topological constraint with a persistence loss based on persistent homology for crowd localization.The method focuses on 0-dimensional topology, where connected components correspond to localized dots.
  • 5 Conclusion: The topological constraint is designed to remain flexible across sparse and dense regions.Persistence measures the saliency of likelihood-map modes by tracking connected components across thresholds.
  • 5 Conclusion: TopoCount achieves state-of-the-art localization accuracy and significantly improves density-based counting when its localization results are supplied as additional input.The conclusion links localization quality with gains for density-map estimation methods.

B Additional Training and Implementation Details

The implementation uses a U-Net-style model with persistence computation over likelihood maps and threshold-based postprocessing. Training and preprocessing choices vary by dataset and include cropping, resizing, and Adam optimization.

  • Model Architecture: TopoCount uses a U-Net-style architecture, with detailed per-layer design shown in Figure 5.The implementation section identifies the model family but does not provide the individual layer configuration here.
  • Image Scaling: UCF QNRF images are resized to a maximum side length of 2048, with different longer-side and shorter-side policies for training and testing.The relaxed test resizing is intended to preserve more detail in densely crowded regions.
  • Training Setup: Training uses Adam with learning rate 0.00005 and batch size 1, while crop sizes depend on dataset resolution.Lower-resolution ShanghaiTech images use whole-image training; other datasets use crops capped at 1024 × 1024.
  • Persistence Computation: Persistence is computed from a likelihood map by building a pixel grid graph and sorting nodes by decreasing likelihood.The algorithm outputs paired modes and saddles, P = {(m_i, s_i)}.
  • Persistence Computation: The persistence procedure merges neighboring components according to their earliest birth component and saddle relationships.The supplied pseudocode describes creation, merging, and saddle-point handling for adjacent components.
  • Postprocessing: Postprocessing uses double thresholding with high threshold 0.5 and low threshold 0.4 to select and then grow connected components.The high threshold identifies components representing people, while the lower threshold expands their geometry.

C.1 Additional Qualitative Results

Additional qualitative results compare TopoCount’s topology and density maps with ground-truth annotations and show examples on difficult JHU++ cases.

  • Qualitative Comparisons: TopoCount’s topology maps closely match the spatial arrangement of ground-truth dots, while its integrated density maps also follow the annotations.Figure 6 reports F-scores and counting errors alongside qualitative examples.
  • Difficult Cases: Figures 8 and 9 provide TopoCount examples on difficult cases from the JHU++ dataset.The passage identifies these as additional qualitative results without specifying a numerical outcome.

C.2 Integration with Density Map Results

Integrating TopoCount’s topology map with density-based counting improves density-map structure and recovers crowded regions missed by the baseline.

  • Density Map Quality: TopoCount integration makes a blurry baseline density map more structured and closer to the ground-truth density map.The comparison is shown for a sample in Figure 7 using the baseline method of Ma et al. (2019).
  • Difficult Regions: In a densely crowded shadowed region, the baseline misses the crowd, whereas TopoCount identifies it and passes that information to the integrated density estimator.The integrated baseline+TopoCount result recovers the crowd in the shadow.

C.3 Shanghai Part A and UCF QNRF Localization Accuracy

TopoCount achieves the strongest reported matching accuracy across both small and large Gaussian scales, indicating robust localization under different proximity requirements.

  • Matching Accuracy: TopoCount achieves the highest AP, AR, mAP, and mAR scores with a large margin at both small and large σ values.The evaluation compares σ ∈ {40, 20, 5}; smaller σ requires predictions to be closer to the ground-truth dots.
  • Evaluation Metrics: The evaluation reports AP.5, AR.5, AP.75, AR.75, mAP, and mAR using thresholds from 0.5 to 0.95.The mean metrics use threshold steps of 0.05.

C.4 NYPU-Crowd Online Localization Challenge

The NWPU-Crowd evaluation uses 1024×1024 crops and reports localization performance across adaptive matching thresholds and head-area ranges. Qualitative examples compare TopoCount’s topological maps with baseline and integrated density maps.

  • Evaluation setup: NWPU-Crowd training uses 1024×1024 crops from original-resolution images, including images without heads.The dataset provides dot annotations and head-box dimensions for adaptive matching evaluation.
  • Evaluation setup: The challenge evaluates F-score, precision, and recall under two adaptive matching thresholds, with recall further categorized by head bounding-box area.The stricter threshold is σs = min(w, h)/2, while the other threshold is defined from the box dimensions.
  • Localization results: TopoCount achieves the highest reported recall across the detailed NWPU-Crowd localization results.The supplied passage begins the corresponding result statement but truncates its continuation.
  • Qualitative results: Sample visualizations show original images, ground truth, TopoCount topological maps, and density maps from Bayesian baseline integration.The figures compare the spatial outputs produced by the baseline and TopoCount combination.
  • Qualitative results: Adding TopoCount to the baseline density estimator produces density maps with better topological structure and repairs shadowed regions missed by the baseline.This comparison is illustrated on crowd-counting samples.

C.5 JHU++ Counting Evaluation

The JHU++ counting evaluation trains on original-resolution 1024×1024 crops and reports categorized validation and test performance. Difficult-case visualizations show predicted topological maps and component centers overlaid on input images.

  • Evaluation setup: JHU++ training uses 1024×1024 crops without resizing and excludes training images containing no heads.The dataset includes images with varying difficulty, including rain and fog conditions.
  • Localization evaluation: The evaluation also reports localization accuracy using the metric of Liu, Weng, and Mu (2019).This metric is listed separately from the categorized counting tables.
  • Counting evaluation: The categorized counting results are presented separately for the JHU-CROWD++ validation and test sets.Tables 11 and 12 correspond to the validation and test evaluations, respectively.
  • Qualitative results: Difficult JHU++ cases are visualized with the input image, ground-truth dot map, predicted topological map, and component centers overlaid as red dots.The same visualization layout is described for multiple figure sets.
Loading 2012.12482v1…