PaperlayerView exact revision on arXiv

Source-linked AI summary

R-FCN: Object Detection via Region-based Fully Convolutional Networks

Jifeng Dai, Yi Li, Kaiming He, Jian Sun

arXiv:1605.06409v3cs.CV

TL;DR

Object detection needs both shared, translation-invariant computation and translation-variant localization, but existing designs sacrifice accuracy or efficiency. R-FCN uses position-sensitive score maps and pooling to achieve competitive PASCAL VOC accuracy, including 83.6% mAP, at 170ms per image and 2.5–20× faster than Faster R-CNN.

  • Problem

    Fully convolutional classifiers offer shared computation, but object detection additionally requires translation-variant localization, creating an accuracy–efficiency design dilemma.

  • Method

    R-FCN uses shared fully convolutional architectures with position-sensitive score maps and a position-sensitive RoI pooling layer, trained end-to-end without following learnable layers.

  • Results

    83.6% mAP on PASCAL VOC 2007 with ResNet-101 was achieved at 170ms per image, 2.5× to 20× faster than Faster R-CNN.

  • Takeaways & Limitations

    R-FCN achieves accuracy competitive with Faster R-CNN while being much faster during both training and inference.

  • Takeaways & Limitations

    The presented R-FCN system is intentionally kept simple and does not incorporate orthogonal extensions developed for fully convolutional or region-based methods.

Abstract

from arXiv · show

We present region-based, fully convolutional networks for accurate and efficient object detection. In contrast to previous region-based detectors such as Fast/Faster R-CNN that apply a costly per-region subnetwork hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image. To achieve this goal, we propose position-sensitive score maps to address a dilemma between translation-invariance in image classification and translation-variance in object detection. Our method can thus naturally adopt fully convolutional image classifier backbones, such as the latest Residual Networks (ResNets), for object detection. We show competitive results on the PASCAL VOC datasets (e.g., 83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is achieved at a test-time speed of 170ms per image, 2.5-20x faster than the Faster R-CNN counterpart. Code is made publicly available at: https://github.com/daijifeng001/r-fcn

1 Introduction

R-FCN addresses the conflict between translation-invariant image classification and translation-variant object localization with a fully convolutional, position-sensitive architecture. Using ResNet-101, it achieves competitive PASCAL VOC accuracy with substantially faster test-time inference than Faster R-CNN.

  • Problem: Region-based detectors traditionally split into a shared fully convolutional subnetwork and an RoI-wise subnetwork without shared computation.This decomposition historically reflected classification architectures such as AlexNet and VGG Nets.
  • Problem: Naïvely removing hidden layers from the RoI-wise subnetwork produces considerably inferior detection accuracy despite modern classifiers being fully convolutional.ResNets and GoogLeNets are fully convolutional by design, motivating this otherwise natural construction.
  • Motivation: Object detection requires balancing translation invariance for classification with translation variance in localization representations.Shifting an object within an image should not affect classification, while shifts within a candidate box should inform overlap quality.
  • Method: R-FCN incorporates translation variance into a fully convolutional network through position-sensitive score maps produced by specialized convolutional output layers.Each score map encodes position relative to a spatial location, enabling position-specific RoI pooling.
  • Results: 83.6% mAP on PASCAL VOC 2007 and 82.0% on 2012 are achieved at 170ms per image, 2.5× to 20× faster than Faster R-CNN + ResNet-101.These results use ResNet-101 as the backbone.

2 Our approach

R-FCN is a two-stage detector that shares fully convolutional image computation between an RPN and position-sensitive RoI classification, making region-wise computation nearly cost-free. Its design uses position-sensitive score maps and selective RoI pooling, with convolutional bounding-box regression and end-to-end training.

  • Overview: R-FCN uses an RPN to propose candidate RoIs, shares features between the RPN and detector, and computes learnable convolutional layers over the entire image.The per-RoI computational cost is negligible.
  • Position-sensitive score maps & Position-sensitive RoI pooling: The final convolutional layer produces k^2 position-sensitive score maps per category, with k^2(C + 1) output channels including background.The k × k maps encode relative spatial positions within each RoI.
  • Position-sensitive score maps & Position-sensitive RoI pooling: Position-sensitive RoI pooling divides each RoI into a k × k grid and selectively pools each bin from its corresponding score map, then averages the scores to produce a (C + 1)-dimensional RoI vector.End-to-end training shepherds the convolutional layers to learn specialized position-sensitive score maps.
  • Bounding-box regression: Bounding-box regression uses a sibling 4k^2-dimensional convolutional layer, position-sensitive pooling, and average voting to obtain a 4-dimensional box parameter vector.The parameterization is t = (t_x, t_y, t_w, t_h).
  • Training: The RoI layer has no learnable layer after it, enabling nearly cost-free region-wise computation and speeding both training and inference.Training combines cross-entropy classification loss with bounding-box regression loss.
  • À trous and stride: 2.6 points: the à trous modification improves mAP by 2.6 points by reducing ResNet-101’s effective stride from 32 pixels to 16 pixels.The modification increases score-map resolution while leaving the RPN unaffected.

3 Related Work

Prior object detectors range from region-wise computation in R-CNN to semiconvolutional designs that share image-level convolutional computation but retain per-region subnetworks. Other approaches use fully convolutional sliding windows or anchor-based box prediction, while some generate holistic detections with fully connected layers.

  • Region-based detectors: R-CNN applies convolutional networks independently to cropped, warped regions, so computation is not shared among regions.This established the effectiveness of combining region proposals with deep networks.
  • Region-based detectors: SPPnet, Fast R-CNN, and Faster R-CNN share convolutional computation across the image but use another subnetwork to evaluate individual regions.The passage characterizes these methods as “semiconvolutional.”
  • Fully convolutional detectors: OverFeat detects objects by sliding multi-scale windows over shared convolutional feature maps.The passage presents OverFeat as an example of a detector that can be viewed as fully convolutional.
  • Fully convolutional detectors: Faster R-CNN’s RPN is a fully convolutional detector that predicts boxes relative to multi-size reference boxes, with class-specific variants also applicable.The original RPN is class-agnostic, whereas the evaluated counterpart can be class-specific.
  • Holistic detectors: Some object detectors use fully connected layers to generate holistic detection results for an entire image.These methods form another family distinct from region-wise and fully convolutional detectors.

4 Experiments

Experiments on PASCAL VOC and MS COCO show that R-FCN combines competitive detection accuracy with substantially lower per-region computation and faster testing. Position-sensitive representations are central to this performance, while accuracy improves with deeper backbones and remains competitive across proposal methods.

  • Position sensitivity: 68.9% mAP for naïve Faster R-CNN versus 76.4% for standard Faster R-CNN shows the importance of preserving spatial information by inserting RoI pooling before conv5.The class-specific RPN obtains 67.6% mAP, about 9 points below standard Faster R-CNN’s 76.4%.
  • PASCAL VOC: R-FCN reaches 76.6% mAP on VOC 2007, matching standard Faster R-CNN’s 76.4% while encoding spatial information without a learnable layer after RoI pooling.Experiments train on VOC 2007 and 2012 trainval and evaluate on the VOC 2007 test set using mAP.
  • Position sensitivity: R-FCN with k = 1 cannot converge, while naïve Faster R-CNN with 1 × 1 RoI pooling reaches only 61.7% mAP.Setting k = 1 removes explicit spatial information within each RoI through global pooling.
  • Efficiency: 0.17s per image makes R-FCN 2.5× faster than Faster R-CNN’s 0.42s with 300 test-time RoIs, while R-FCN has negligible per-region cost.The comparison uses k × k = 7 × 7 and a single Nvidia K40 GPU.
  • PASCAL VOC: 83.6% mAP is achieved by R-FCN after MS COCO pretraining and PASCAL VOC fine-tuning, while multi-scale training with single-scale testing yields 80.5% mAP.The 80.5% mAP setting samples training scales from 400 to 800 pixels and tests at 600 pixels.
  • Backbones and proposals: R-FCN accuracy increases from ResNet-50 to ResNet-101 but saturates at ResNet-152, and remains competitive when using Selective Search or Edge Boxes proposals.These proposal-method comparisons use ResNet-101.

5 Conclusion and Future Work

R-FCN is presented as a simple, accurate, and efficient fully convolutional object detector that adopts ResNet-style backbones and outpaces Faster R-CNN in training and inference. The authors retain a simple system while anticipating benefits from orthogonal advances in segmentation and detection.

  • Conclusion: R-FCN provides a simple, accurate, and efficient framework for object detection.The system is region-based and fully convolutional.
  • Conclusion: R-FCN naturally adopts fully convolutional image-classification backbones such as ResNets.These backbones are fully convolutional by design.
  • Conclusion: R-FCN achieves accuracy competitive with Faster R-CNN while training and inference are much faster.The conclusion states the speed advantage for both training and inference.
  • Future Work: The authors expect R-FCN to benefit from orthogonal extensions of fully convolutional networks and region-based object-detection methods.They intentionally keep the presented system simple and anticipate benefits from progress in these fields.
  • Conclusion: 83.6% mAP is reported on the PASCAL VOC 2007 test set using ResNet-101 and 07+12+COCO training data.The figure uses a 0.6 score threshold for display and reports 170ms per image on one Nvidia K40 GPU.
Loading 1605.06409v3…