Source-linked AI summary
Adaptive NMS: Refining Pedestrian Detection in a Crowd
Songtao Liu, Di Huang, Yunhong Wang
TL;DR
Crowded-scene pedestrian detection remains difficult because overlapping, similar-looking pedestrians challenge detectors and fixed-threshold greedy NMS. The paper introduces adaptive-NMS, which learns density-based suppression thresholds, and reports state-of-the-art performance on CityPersons and CrowdHuman, including 10.8% MR−2 and 49.73% MR−2.
Problem
Crowded pedestrian detection remains difficult because similar-looking pedestrians can heavily occlude one another, while greedy-NMS must trade off missing overlapping objects against retaining false positives.
Method
Adaptive-NMS uses a learned subnetwork to predict an instance-specific suppression threshold from object density, increasing it in crowded regions and reverting to the standard threshold in sparse regions.
Results
Adaptive-NMS achieves state-of-the-art performance on CityPersons and CrowdHuman, reaching 10.8% MR−2 on CityPersons and 49.73% MR−2 on CrowdHuman.
Takeaways & Limitations
The method improves bounding-box refinement for crowded pedestrian detection across both two-stage and single-stage detectors.
Takeaways & Limitations
CNN-based detectors often assign correlated scores to neighboring regions, making it difficult to produce compact boxes that reliably support a high fixed NMS threshold.
Abstract
from arXiv · showhide
Pedestrian detection in a crowd is a very challenging issue. This paper addresses this problem by a novel Non-Maximum Suppression (NMS) algorithm to better refine the bounding boxes given by detectors. The contributions are threefold: (1) we propose adaptive-NMS, which applies a dynamic suppression threshold to an instance, according to the target density; (2) we design an efficient subnetwork to learn density scores, which can be conveniently embedded into both the single-stage and two-stage detectors; and (3) we achieve state of the art results on the CityPersons and CrowdHuman benchmarks.
1. Introduction
Crowded pedestrian scenes expose a fundamental limitation of fixed-threshold greedy-NMS: lower thresholds remove overlapping true positives, while higher thresholds retain false positives. The paper proposes adaptive-NMS, which learns instance density to set dynamic suppression thresholds and improves crowded-detection benchmarks.
- Motivation: Crowded scenes remain difficult because similar pedestrians can heavily occlude one another, unlike ordinary isolated occlusion.On CityPersons, 26.4% of pedestrians overlap above 0.3 IoU, with average pairwise overlap of 0.32 per image.
- Motivation: Greedy-NMS creates a fixed-threshold dilemma: lower thresholds miss highly overlapped objects, whereas higher thresholds increase false positives.Even accurate, highly overlapping ground-truth boxes can be suppressed during post-processing.
- Proposed approach: Adaptive-NMS raises suppression thresholds as instances gather and occlude each other, and lowers them when instances are separated.An auxiliary learnable subnetwork predicts an adaptive threshold for each instance from density.
- Proposed approach: The method addresses crowded detection while remaining applicable to both single-stage and two-stage detectors.The experiments report improvements for both detector families on CityPersons and CrowdHuman.
- Results: 10.8% MR−2 on CityPersons and 49.73% MR−2 on CrowdHuman are reported as state-of-the-art results.These results are reported for crowded pedestrian detection benchmarks.
2. Related Work
Prior work improves pedestrian and generic object detection through stronger detectors, compact proposals, learned suppression, and alternatives to greedy-NMS. Adaptive-NMS instead uses density-aware thresholding to distinguish crowded true positives from nearby false positives while retaining greedy-NMS efficiency.
- Pedestrian detection: Pedestrian detection progressed from hand-crafted feature methods to CNN-based detectors, including single-stage and two-stage architectures.Related work covers traditional detectors, deep feature models, multiscale methods, and occlusion-aware approaches.
- Crowd and occlusion handling: Occlusion-focused methods learn body parts or compact proposals, while some crowd-specific methods use regression or aggregation penalties to reduce sensitivity to NMS thresholds.A recurrent LSTM alternative removes NMS but suffers from scale variations.
- Non-Maximum Suppression: Greedy-NMS remains a widely used post-processing component across computer-vision detection tasks.It is used in edge detection, feature-point detection, and object detection.
- Non-Maximum Suppression: Soft-NMS lowers neighboring detection scores instead of discarding them, whereas learning-NMS methods learn suppression functions from boxes and scores.The cited learning-NMS network is specifically designed and complex, while soft-NMS still treats highly overlapped boxes as false positives.
- Adaptive-NMS: Adaptive-NMS differs by learning object density around each ground-truth instance as its suppression threshold.Its related-work motivation is the distinct challenge posed by many highly overlapped true positives in crowded pedestrian scenes.
3. Method
The method replaces fixed-threshold NMS with an adaptive threshold based on predicted object density, preserving likely neighboring true positives in crowds while pruning false positives in sparse regions. A lightweight density-prediction subnet supports both detector architectures with negligible additional computational cost.
- Greedy-NMS Revisit: Greedy-NMS can remove true positives in crowds at low thresholds, but high thresholds retain correlated-score false positives.Low thresholds suppress overlapping detections of distinct people; high thresholds preserve more true positives while increasing false positives.
- Greedy-NMS Revisit: Soft-NMS reduces neighboring detection scores instead of discarding them, but still penalizes highly overlapped boxes as likely false positives.Its overlap-based penalty remains inconsistent with crowded scenes, where highly overlapped boxes may correspond to different people.
- Adaptive-NMS: Adaptive-NMS sets each selected box’s suppression threshold to the larger of the fixed threshold and its predicted density.The density is defined from the maximum IoU with another ground-truth object and indicates crowd occlusion.
- Adaptive-NMS: Boxes below the fixed overlap threshold are retained, while boxes near a crowded selected instance receive a higher threshold and are more likely preserved.For sparse instances, the fixed threshold continues to prune highly overlapped neighboring proposals that are more likely false positives.
- Adaptive-NMS: Adaptive-NMS has the same computational complexity as greedy-NMS and soft-NMS, requiring only an additional N-element predicted-density list.It can also operate with both greedy-NMS and soft-NMS.
- Density Prediction: Density prediction is treated as Smooth-L1 regression using surrounding information rather than only individual-object features.The three-convolutional-layer subnet uses detector outputs and features, including a large final kernel, and is compatible with one-stage and two-stage detectors.
4. Experiments
Experiments evaluate adaptive-NMS on CityPersons and CrowdHuman using two-stage and single-stage detectors, ablations, density-stratified analysis, visual comparisons, and state-of-the-art benchmarks. Adaptive-NMS improves crowded pedestrian detection while preserving efficiency comparable to conventional NMS.
- Experimental setup: Experiments use CityPersons and CrowdHuman to evaluate adaptive-NMS with both two-stage and single-stage pedestrian detectors.The setups include Faster R-CNN and RFB Net, with the density sub-network trained jointly with the base detectors.
- Ablation study: 14.5% MR−2 is achieved by adapted Faster R-CNN with traditional greedy-NMS in the ablation baseline.Greedy-NMS and soft-NMS are compared with adaptive-NMS using the best reported conventional settings.
- Analysis: Adaptive-NMS performs better on higher-density groups and keeps more highly-overlapped true positives while removing false positives in sparse regions.The density-stratified analysis and visual comparisons report these complementary effects across crowd conditions.
- Ablation study: 1.6% MR−2 and 1.2% MR−2 reductions are obtained by adaptive-NMS for Faster R-CNN and RFB Net, respectively.The corresponding Faster R-CNN result drops to 12.9% MR−2, while RFB Net changes from 13.9% MR−2 to 12.7% MR−2.
- State-of-the-art comparison: 0.9% MR−2 is reduced for Faster R-CNN with AggLoss and adaptive-NMS on CityPersons validation, reaching 11.9% versus 12.8% for [49].RFB Net with adaptive-NMS and AggLoss reaches 12.0% MR−2; the evaluation separates Partial, Bare, and Heavy occlusion subsets.
- CrowdHuman: 2.62% MR−2 and 2.19% MR−2 reductions are reported when replacing greedy-NMS with adaptive-NMS for FPN and RFB Net on CrowdHuman.These results are reported as evidence of adaptive-NMS effectiveness for detectors operating in crowded scenes.
5. Conclusions
Adaptive-NMS refines bounding boxes in crowded scenarios by predicting a density-dependent suppression threshold for each instance, achieving state-of-the-art results on CityPersons and CrowdHuman.
- Adaptive-NMS uses an additionally learned sub-network to predict each instance’s suppression threshold according to density.