PaperlayerView exact revision on arXiv

Source-linked AI summary

Is Faster R-CNN Doing Well for Pedestrian Detection?

Liliang Zhang, Liang Lin, Xiaodan Liang, Kaiming He

arXiv:1607.07032v2cs.CV

TL;DR

Pedestrian detection remains challenging for Faster R-CNN despite its general object-detection success, particularly because small instances and hard negatives are poorly handled. The paper uses a pedestrian-tailored RPN with boosted forests on shared high-resolution features, achieving competitive benchmark performance and reporting a 40% relative Caltech improvement at IoU 0.7.

  • Problem

    Faster R-CNN has limited pedestrian-detection accuracy despite strong general object-detection performance, while leading pedestrian detectors commonly use hybrid hand-crafted and deep features.

    Source: §1 · pp. 1–2

  • Method

    The paper combines a pedestrian-tailored RPN with cascaded Boosted Forests that reuse shared, higher-resolution convolutional features and perform hard-negative bootstrapping.

    Source: §1 · p. 3, §3.1 · p. 4, §5 · p. 12

  • Results

    The method produces competitive results across Caltech, INRIA, ETH, and KITTI, including a 40% relative improvement on Caltech at IoU 0.7 and 0.5 second per image test-time speed.

    Source: §1 · p. 3

  • Takeaways & Limitations

    RPN can serve as a competitive stand-alone pedestrian detector, while suitable feature resolution and bootstrapping enable a deep-learning system without hand-crafted features.

    Source: §1 · p. 3, Is RPN good for pedestrian detection? · p. 7, §5 · p. 13

Abstract

from arXiv · show

Detecting pedestrian has been arguably addressed as a special topic beyond general object detection. Although recent deep learning object detectors such as Fast/Faster R-CNN [1, 2] have shown excellent performance for general object detection, they have limited success for detecting pedestrian, and previous leading pedestrian detectors were in general hybrid methods combining hand-crafted and deep convolutional features. In this paper, we investigate issues involving Faster R-CNN [2] for pedestrian detection. We discover that the Region Proposal Network (RPN) in Faster R-CNN indeed performs well as a stand-alone pedestrian detector, but surprisingly, the downstream classifier degrades the results. We argue that two reasons account for the unsatisfactory accuracy: (i) insufficient resolution of feature maps for handling small instances, and (ii) lack of any bootstrapping strategy for mining hard negative examples. Driven by these observations, we propose a very simple but effective baseline for pedestrian detection, using an RPN followed by boosted forests on shared, high-resolution convolutional feature maps. We comprehensively evaluate this method on several benchmarks (Caltech, INRIA, ETH, and KITTI), presenting competitive accuracy and good speed. Code will be made publicly available.

1 Introduction

Faster R-CNN succeeds in general object detection but has limited pedestrian-detection accuracy. The paper attributes this gap to low-resolution features for small pedestrians and insufficient hard-negative mining, motivating an RPN-plus-Boosted-Forest baseline.

  • Pedestrian detection remains distinct from general object detection, with leading systems generally combining hand-crafted and deep convolutional features.Hand-crafted features appear important for state-of-the-art pedestrian detection.Source: §1 · pp. 1–2
  • Faster R-CNN combines an RPN proposer with a downstream Fast R-CNN classifier but has not achieved competitive results on popular pedestrian datasets.Source: §1 · p. 2
  • Low-resolution feature maps can make RoI pooling produce nondiscriminative features for small pedestrian instances.The paper gives 28×70-pixel pedestrians on Caltech as an example and identifies collapsing pooling bins as the mechanism.Source: §1 · p. 2, Figure 1 · p. 2
  • Hard background instances dominate false predictions in pedestrian detection, but Fast/Faster R-CNN lacks careful attention to these hard negatives.Source: Figure 1 · p. 2, §1 · p. 3
  • The proposed baseline uses an RPN followed by Boosted Forests on shared convolutional features to address feature resolution and hard-negative mining.The approach avoids traditional hand-crafted features while reusing deep features from the RPN.Source: §1 · p. 3, Figure 2 · p. 3
  • 40% relative improvement is reported on Caltech at IoU 0.7, with 0.5 second per image test-time speed and competitive results across Caltech, INRIA, ETH, and KITTI.Source: §1 · p. 3

2 Related Work

Prior pedestrian detectors commonly use two-stage pipelines, boosted classifiers, or hybrid features. The paper positions its approach against these methods by using deep convolutional features with region proposals and boosted classification.

  • ICF and its extensions use channel-feature pyramids with boosted classifiers, and boosting remains a key building block for pedestrian detection.Source: §2 · p. 3
  • Several pedestrian systems use hand-crafted-feature detectors to generate proposals before R-CNN-style classification or neural-network processing.Source: §2 · p. 4
  • CompACT combines hybrid hand-crafted and deep features, while CCF uses boosted classifiers on deep-feature pyramids without region proposals.Source: §2 · p. 4

3 Approach

The approach combines a pedestrian-tailored RPN with a cascaded Boosted Forest classifier. It uses pedestrian-shaped multiscale anchors, shared high-resolution features, and bootstrapping for hard-negative mining.

  • The pipeline uses an RPN to generate candidate boxes and convolutional feature maps, followed by a Boosted Forest classifier.Source: §3 · p. 4
  • The pedestrian-tailored RPN uses one 0.41 aspect ratio and nine scales beginning at 40 pixels with a 1.3× scaling stride.Source: §3.1 · p. 4
  • The RPN retains a 16-pixel stride for proposal generation, while the a trous trick is used only during feature extraction to increase resolution.Source: §3.1 · pp. 4–5
  • Boosted Forests can pool features from multiple convolutional layers and resolutions without the fully connected dimensional constraints of Faster R-CNN.Examples include Conv3_3, Conv4_3, and higher-resolution a trous features.Source: §3.2 · p. 5
  • The classifier is trained with six bootstrapping stages, progressively adding hard negatives while increasing the number of trees from 64 to 1536.Source: §3.3 · p. 5
  • RPN proposal scores initialize the boosting process as a stage-0 classifier, and training uses NMS-filtered proposals ranked by score.Source: §3.3 · p. 6, §3.4 · p. 6

4 Experiments and Analysis

The experiments evaluate the method on four pedestrian benchmarks using dataset-specific protocols and metrics. The supplied passages emphasize proposal-quality evaluation on Caltech and benchmark coverage rather than complete result tables.

  • The method is evaluated on Caltech, INRIA, ETH, and KITTI, with IoU 0.5 used by default to determine true positives.Source: §4.1 · p. 6
  • Fig. 3 compares RPN with SCF, LDCF, and Checkerboards using recall versus IoU for averages of 1, 4, or 100 proposals per image.Source: Figure 3 · p. 7
  • INRIA and ETH are used to assess generalization capability and are evaluated using MR^-2.Source: §4.1 · p. 7
  • KITTI reports PASCAL-style mean Average Precision across Easy, Moderate, and Hard difficulty levels.Source: §4.1 · p. 7

4.2 Ablation Experiments

Ablations on Caltech show that the RPN is a strong stand-alone pedestrian detector, achieving high proposal recall and competitive miss rate.

How important is feature resolution?

Feature resolution strongly affects classifier accuracy: Fast R-CNN degrades with low-resolution features, while higher-resolution features substantially improve boosted-forest results.

How important is bootstrapping?

The bootstrapping ablation evaluates whether hard-negative mining, rather than the classifier structure alone, drives the boosted-forest advantage.

  • After six bootstrapping stages, Fast R-CNN reaches 14.3% MR, close to the 13.7% of its Boosted Forest counterpart.Using the same bootstrapped training set improves Fast R-CNN over its non-bootstrapped version.Source: How important is bootstrapping? · p. 9
  • The comparison tests with and without bootstrapping on the Caltech set.Source: Figure 4 · p. 10, Figure 5 · p. 10

4.3 Comparisons with State-of-the-art Methods

The proposed method outperforms prior methods on Caltech, especially under the stricter IoU threshold, while maintaining favorable speed.

  • 9.6% MR on Caltech with original annotations beats CompactACT-Deep’s 11.7%.Source: Caltech · p. 10
  • 7.3% MR−2 and 16.8% MR−4 on corrected annotations are both 2 points better than previous best methods.Source: Caltech · p. 10, Figure 6 · p. 11
  • On Caltech, the method achieves 9.6% MR without hand-crafted features, supporting high-resolution features and bootstrapping as key factors.Source: Caltech · p. 10
  • 23.5% MR at IoU 0.7 yields a relative improvement of approximately 40% over the closest competitors.Source: Caltech · pp. 10–11
  • The method is as fast as CompactACT-Deep and much faster than CCF, balancing speed and accuracy through shared RPN and BF features.Source: Caltech · p. 11, Table 4 · p. 11

INRIA and ETH

The method achieves stronger miss rates than prior leading methods on both the INRIA and ETH pedestrian datasets.

KITTI

On KITTI, the method provides competitive pedestrian-detection accuracy while retaining fast speed.

  • The method achieves competitive accuracy and fast speed on the KITTI dataset.Source: KITTI · p. 12

5 Conclusion and Discussion

The paper concludes that RPN plus boosted forests provides a simple, self-contained alternative to Faster R-CNN and hybrid-feature pedestrian detectors. Bootstrapping remains important even with deep neural networks.

  • RPN plus BF combines arbitrary-resolution features with bootstrapping for hard-negative mining.Source: §5 · p. 12
  • The KITTI evaluation compares accuracy and timing, with some region-proposal timing records estimated or ignored.Source: Table 5 · p. 13
  • The method is self-contained and does not rely on hybrid hand-crafted features.Source: §5 · p. 13
  • Using the same bootstrapping strategy and RoI features, BF and Fast R-CNN achieve similar results.Source: §5 · p. 13

Related papers

Loading 1607.07032v2…