Source-linked AI summary
Is Faster R-CNN Doing Well for Pedestrian Detection?
Liliang Zhang, Liang Lin, Xiaodan Liang, Kaiming He
TL;DR
Pedestrian detection remains challenging for Faster R-CNN despite its general object-detection success, particularly because small instances and hard negatives are poorly handled. The paper uses a pedestrian-tailored RPN with boosted forests on shared high-resolution features, achieving competitive benchmark performance and reporting a 40% relative Caltech improvement at IoU 0.7.
Problem
Faster R-CNN has limited pedestrian-detection accuracy despite strong general object-detection performance, while leading pedestrian detectors commonly use hybrid hand-crafted and deep features.
Method
The paper combines a pedestrian-tailored RPN with cascaded Boosted Forests that reuse shared, higher-resolution convolutional features and perform hard-negative bootstrapping.
Results
The method produces competitive results across Caltech, INRIA, ETH, and KITTI, including a 40% relative improvement on Caltech at IoU 0.7 and 0.5 second per image test-time speed.
Takeaways & Limitations
RPN can serve as a competitive stand-alone pedestrian detector, while suitable feature resolution and bootstrapping enable a deep-learning system without hand-crafted features.
Abstract
from arXiv · showhide
Detecting pedestrian has been arguably addressed as a special topic beyond general object detection. Although recent deep learning object detectors such as Fast/Faster R-CNN [1, 2] have shown excellent performance for general object detection, they have limited success for detecting pedestrian, and previous leading pedestrian detectors were in general hybrid methods combining hand-crafted and deep convolutional features. In this paper, we investigate issues involving Faster R-CNN [2] for pedestrian detection. We discover that the Region Proposal Network (RPN) in Faster R-CNN indeed performs well as a stand-alone pedestrian detector, but surprisingly, the downstream classifier degrades the results. We argue that two reasons account for the unsatisfactory accuracy: (i) insufficient resolution of feature maps for handling small instances, and (ii) lack of any bootstrapping strategy for mining hard negative examples. Driven by these observations, we propose a very simple but effective baseline for pedestrian detection, using an RPN followed by boosted forests on shared, high-resolution convolutional feature maps. We comprehensively evaluate this method on several benchmarks (Caltech, INRIA, ETH, and KITTI), presenting competitive accuracy and good speed. Code will be made publicly available.
1 Introduction
Faster R-CNN succeeds in general object detection but has limited pedestrian-detection accuracy. The paper attributes this gap to low-resolution features for small pedestrians and insufficient hard-negative mining, motivating an RPN-plus-Boosted-Forest baseline.
- Pedestrian detection remains distinct from general object detection, with leading systems generally combining hand-crafted and deep convolutional features.Hand-crafted features appear important for state-of-the-art pedestrian detection.
- Faster R-CNN combines an RPN proposer with a downstream Fast R-CNN classifier but has not achieved competitive results on popular pedestrian datasets.
- Low-resolution feature maps can make RoI pooling produce nondiscriminative features for small pedestrian instances.The paper gives 28×70-pixel pedestrians on Caltech as an example and identifies collapsing pooling bins as the mechanism.
- Hard background instances dominate false predictions in pedestrian detection, but Fast/Faster R-CNN lacks careful attention to these hard negatives.
- The proposed baseline uses an RPN followed by Boosted Forests on shared convolutional features to address feature resolution and hard-negative mining.The approach avoids traditional hand-crafted features while reusing deep features from the RPN.
- 40% relative improvement is reported on Caltech at IoU 0.7, with 0.5 second per image test-time speed and competitive results across Caltech, INRIA, ETH, and KITTI.
2 Related Work
Prior pedestrian detectors commonly use two-stage pipelines, boosted classifiers, or hybrid features. The paper positions its approach against these methods by using deep convolutional features with region proposals and boosted classification.
- ICF and its extensions use channel-feature pyramids with boosted classifiers, and boosting remains a key building block for pedestrian detection.
- Several pedestrian systems use hand-crafted-feature detectors to generate proposals before R-CNN-style classification or neural-network processing.
- CompACT combines hybrid hand-crafted and deep features, while CCF uses boosted classifiers on deep-feature pyramids without region proposals.
3 Approach
The approach combines a pedestrian-tailored RPN with a cascaded Boosted Forest classifier. It uses pedestrian-shaped multiscale anchors, shared high-resolution features, and bootstrapping for hard-negative mining.
- The pipeline uses an RPN to generate candidate boxes and convolutional feature maps, followed by a Boosted Forest classifier.
- The pedestrian-tailored RPN uses one 0.41 aspect ratio and nine scales beginning at 40 pixels with a 1.3× scaling stride.
- The RPN retains a 16-pixel stride for proposal generation, while the a trous trick is used only during feature extraction to increase resolution.
- Boosted Forests can pool features from multiple convolutional layers and resolutions without the fully connected dimensional constraints of Faster R-CNN.Examples include Conv3_3, Conv4_3, and higher-resolution a trous features.
- The classifier is trained with six bootstrapping stages, progressively adding hard negatives while increasing the number of trees from 64 to 1536.
- RPN proposal scores initialize the boosting process as a stage-0 classifier, and training uses NMS-filtered proposals ranked by score.
4 Experiments and Analysis
The experiments evaluate the method on four pedestrian benchmarks using dataset-specific protocols and metrics. The supplied passages emphasize proposal-quality evaluation on Caltech and benchmark coverage rather than complete result tables.
- The method is evaluated on Caltech, INRIA, ETH, and KITTI, with IoU 0.5 used by default to determine true positives.
- Fig. 3 compares RPN with SCF, LDCF, and Checkerboards using recall versus IoU for averages of 1, 4, or 100 proposals per image.
- INRIA and ETH are used to assess generalization capability and are evaluated using MR^-2.
- KITTI reports PASCAL-style mean Average Precision across Easy, Moderate, and Hard difficulty levels.
4.2 Ablation Experiments
Ablations on Caltech show that the RPN is a strong stand-alone pedestrian detector, achieving high proposal recall and competitive miss rate.
- The ablation experiments evaluate proposal quality and detector performance on the Caltech dataset.
- With 100 proposals per image, the RPN achieves >95% recall at an IoU of 0.7.
- RPN as a stand-alone pedestrian detector achieves an MR of 14.9%.This outperforms all but two state-of-the-art competitors on Caltech.
How important is feature resolution?
Feature resolution strongly affects classifier accuracy: Fast R-CNN degrades with low-resolution features, while higher-resolution features substantially improve boosted-forest results.
- 20.2% MR is obtained by Fast R-CNN, compared with 14.9% for the same RPN proposals.Training the downstream Fast R-CNN classifier therefore degrades the stand-alone RPN result.
- 16.2% MR with a trous Conv5 features partially alleviates Fast R-CNN’s low-resolution problem.Reducing the feature-map stride from 16 pixels to 8 pixels improves the result, but it still trails stand-alone RPN or R-CNN.
- 13.7% MR is achieved by Boosted Forests using a trous Conv5 features, versus 18.2% with Conv5_3 features.
- 11.5% MR results from concatenating Conv3_3 and Conv4_3 features, while Conv3_3 and Conv4_3 alone yield 12.4% and 12.6%.Conv2_2 yields 15.9%, reflecting weaker shallow-layer representations.
- 9.6% MR is achieved by combining Conv3_3 with a trous Conv4_3 features.This gives the best result but requires extra computation to recompute Conv4 features.
How important is bootstrapping?
The bootstrapping ablation evaluates whether hard-negative mining, rather than the classifier structure alone, drives the boosted-forest advantage.
- After six bootstrapping stages, Fast R-CNN reaches 14.3% MR, close to the 13.7% of its Boosted Forest counterpart.Using the same bootstrapped training set improves Fast R-CNN over its non-bootstrapped version.
- The comparison tests with and without bootstrapping on the Caltech set.
4.3 Comparisons with State-of-the-art Methods
The proposed method outperforms prior methods on Caltech, especially under the stricter IoU threshold, while maintaining favorable speed.
- 9.6% MR on Caltech with original annotations beats CompactACT-Deep’s 11.7%.
- 7.3% MR−2 and 16.8% MR−4 on corrected annotations are both 2 points better than previous best methods.
- On Caltech, the method achieves 9.6% MR without hand-crafted features, supporting high-resolution features and bootstrapping as key factors.
- 23.5% MR at IoU 0.7 yields a relative improvement of approximately 40% over the closest competitors.
- The method is as fast as CompactACT-Deep and much faster than CCF, balancing speed and accuracy through shared RPN and BF features.
INRIA and ETH
The method achieves stronger miss rates than prior leading methods on both the INRIA and ETH pedestrian datasets.
- 6.9% MR on INRIA is considerably better than the best available competitor’s 11.2%.
- 30.2% MR on ETH is 5 points better than the previous leading method, TA-CNN.
KITTI
On KITTI, the method provides competitive pedestrian-detection accuracy while retaining fast speed.
- The method achieves competitive accuracy and fast speed on the KITTI dataset.
5 Conclusion and Discussion
The paper concludes that RPN plus boosted forests provides a simple, self-contained alternative to Faster R-CNN and hybrid-feature pedestrian detectors. Bootstrapping remains important even with deep neural networks.
- RPN plus BF combines arbitrary-resolution features with bootstrapping for hard-negative mining.
- The KITTI evaluation compares accuracy and timing, with some region-proposal timing records estimated or ignored.
- The method is self-contained and does not rely on hybrid hand-crafted features.
- Using the same bootstrapping strategy and RoI features, BF and Fast R-CNN achieve similar results.