Source-linked AI summary
NMS by Representative Region: Towards Crowded Pedestrian Detection by Proposal Pairing
Xin Huang, Zheng Ge, Zequn Jie, Osamu Yoshie
TL;DR
Crowded pedestrian detection remains difficult because standard NMS cannot simultaneously retain highly overlapped pedestrians and avoid false positives. The paper proposes representative-region NMS using visible regions, supported by a paired-box model that predicts full and visible boxes together, and reports state-of-the-art results on CrowdHuman and CityPersons.
Problem
Standard NMS faces a crowded-scene trade-off: low IoU thresholds miss highly overlapped pedestrians, while high thresholds bring more false positives.
Method
R2NMS determines overlap using visible-region IoU, while PBM jointly predicts paired full and visible boxes throughout the detection pipeline.
Results
The proposed approach achieves state-of-the-art results on the CrowdHuman and CityPersons benchmarks.
Takeaways & Limitations
Visible-region suppression provides the paper’s approach for removing redundant boxes in crowded scenes without bringing in many false positives.
Takeaways & Limitations
AdaptiveNMS remains sub-optimal because density estimation, handcrafted density-to-threshold matching, and inaccurate box predictions can be inconsistent.
Abstract
from arXiv · showhide
Although significant progress has been made in pedestrian detection recently, pedestrian detection in crowded scenes is still challenging. The heavy occlusion between pedestrians imposes great challenges to the standard Non-Maximum Suppression (NMS). A relative low threshold of intersection over union (IoU) leads to missing highly overlapped pedestrians, while a higher one brings in plenty of false positives. To avoid such a dilemma, this paper proposes a novel Representative Region NMS approach leveraging the less occluded visible parts, effectively removing the redundant boxes without bringing in many false positives. To acquire the visible parts, a novel Paired-Box Model (PBM) is proposed to simultaneously predict the full and visible boxes of a pedestrian. The full and visible boxes constitute a pair serving as the sample unit of the model, thus guaranteeing a strong correspondence between the two boxes throughout the detection pipeline. Moreover, convenient feature integration of the two boxes is allowed for the better performance on both full and visible pedestrian detection tasks. Experiments on the challenging CrowdHuman and CityPersons benchmarks sufficiently validate the effectiveness of the proposed approach on pedestrian detection in the crowded situation.
1. Introduction
Crowded pedestrian detection is difficult because occlusion creates ambiguous overlaps, making standard NMS trade off missed pedestrians against false positives. The paper addresses this with R2NMS and a Paired-Box Model that jointly predicts full and visible boxes.
- 1. Introduction: Occlusion includes inter-class shielding by background objects and intra-class overlap between pedestrians, creating missing detections and confused instance boundaries.
- 1. Introduction: Crowded scenes expose a standard NMS dilemma: low IoU thresholds miss overlapping pedestrians, whereas high thresholds admit more false positives.
- 1. Introduction: R2NMS uses visible-region overlap instead of full-body overlap to suppress redundant boxes while preserving highly overlapped pedestrians.The approach is motivated by the lower occlusion of visible regions in crowded scenes.
- 1. Introduction: PBM jointly predicts each pedestrian’s full and visible boxes as a paired sample throughout RPN and R-CNN processing.The pairing maintains correspondence and supports feature integration for both detection tasks.
- 1. Introduction: Experiments on CrowdHuman and CityPersons report state-of-the-art results for the proposed approach.
2. Related Works
Related work covers generic one- and two-stage object detectors, occlusion-aware pedestrian detectors, and NMS variants for crowded scenes. The paper’s model combines paired-box processing with representative-region suppression.
- 2. Related Works: Generic detectors are commonly divided into one-stage methods emphasizing speed and two-stage methods emphasizing refined detection performance.
- 2. Related Works: Prior pedestrian methods address occlusion through body parts, penalties on boxes between people, or adaptive density-based NMS.
- 2. Related Works: The Paired-Box Model processes paired proposals and integrates their features before predicting paired full and visible boxes.
3. Method
The method replaces full-body overlap decisions in NMS with visible-region comparisons and uses a Paired-Box Model to maintain corresponding full and visible detections throughout Faster R-CNN.
- 3.2. NMS by Representative Region: R2NMS uses IoU between visible regions rather than full-body boxes to identify redundant detections of the same pedestrian.Different pedestrians can have large full-box overlap but low visible-region overlap, while detections of the same pedestrian have large overlap in both regions.
- 3.1. Analysis on original NMS and Adaptive NMS: A low visible-region IoU threshold can suppress redundant same-pedestrian boxes while avoiding many false positives from distinct crowded pedestrians.Standard NMS faces a trade-off: low thresholds suppress true positives, whereas high thresholds retain more false positives.
- 3.3. Paired-BBox Faster R-CNN: The Paired-Box Model simultaneously predicts full and visible boxes through paired proposal and R-CNN stages based on Faster R-CNN.Its P-RPN generates paired proposals, while PPFE integrates information from both proposals for full- and visible-body detection.
- 3.3. Paired-BBox Faster R-CNN: Full and visible annotations form a pair that serves as the model’s ground-truth unit, with anchor assignment requiring agreement with both boxes.This pairing preserves correspondence between the two predicted boxes throughout the detection pipeline.
- 3.3. Paired-BBox Faster R-CNN: PPFE integrates paired proposal features by direct concatenation or visible-body attention masking.The concatenation approach combines full- and visible-proposal features after RoI Align, while attention highlights pixels inside the visible proposal.
- 3.3. Paired-BBox Faster R-CNN: The reported evaluation uses CrowdHuman and includes a main-results table covering full-body and visible-body MR, AP, and Recall.The table defines MR as lower-is-better and AP and Recall as higher-is-better, with comparisons against the re-implemented baseline.
4. Experiments
Experiments on CrowdHuman and CityPersons evaluate the proposed modules and compare them with baselines and prior methods. PBM with R2NMS improves detection metrics and achieves competitive or superior crowded-pedestrian results.
- 4.3. Detection Results on CrowdHuman: On CrowdHuman, the proposed methods improve MR, AP, and Recall by 2.71%, 4.37%, and 4.85%, respectively, over the strong FPN baseline.The baseline reaches 46.28% MR for full-body detection and 55.57% MR for visible-body detection.
- 4.3. Detection Results on CrowdHuman: PBM's PPFE feature integration improves MR-V and MR, with attention-based PPFE outperforming direct feature concatenation.Direct concatenation improves MR-V and MR by 1.99% and 1.11%, respectively; the attention mechanism performs better.
- 4.3. Detection Results on CrowdHuman: R2NMS greatly boosts AP despite less than 1% MR improvement, indicating better preservation of true positives and reduction of false positives.The paper attributes the metric difference to MR and AP covering different score ranges.
- 4.4. State-of-the-art Comparison on CrowdHuman: MR decreases from 49.73% to 43.35% and AP increases from 84.71% to 89.29% when PBM with R2NMS is compared with published methods on CrowdHuman.The comparison is against AdaptiveNMS and Repulsion Loss on the CrowdHuman validation set.
- 4.5. Detection Results on CityPersons: CityPersons reasonable-set MR decreases from 13.8% to 11.1%, while heavy-occlusion performance improves by 5.7% over the baseline.The reported results use full-body detection on CityPersons validation subsets.
- 4.5. Detection Results on CityPersons: PBM with R2NMS outperforms all published methods on CityPersons reasonable validation subsets, although it is slightly worse than MGAN because of a weaker baseline.The reported comparison includes Adapted FasterRCNN, ATT-part, Repulsion Loss, OR-CNN, AdaptiveNMS, and MGAN.
5. Conclusion
The paper proposes R2NMS, which uses visible-region overlap to suppress redundant full-body boxes, together with PBM for paired full and visible box prediction. Experiments on CrowdHuman and CityPersons report state-of-the-art results for crowded pedestrian detection.
- 5. Conclusion: R2NMS uses IoU between visible regions to decide whether full-body boxes overlap, while PBM predicts paired full and visible boxes.The paired predictions support the representative-region suppression strategy.
- 5. Conclusion: Experiments on CrowdHuman and CityPersons report state-of-the-art results for the proposed approach.The conclusion presents these results as evidence of the method's effectiveness in crowded pedestrian detection.