PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast R-CNN

Ross Girshick

arXiv:1504.08083v2cs.CV

TL;DR

Object detection requires processing numerous rough object proposals and refining their locations, but prior ConvNet detectors relied on slow, multi-stage pipelines. Fast R-CNN uses single-stage joint learning with region-based features to address these limitations. It trains VGG16 9× faster than R-CNN and achieves 66% mAP versus 62% on PASCAL VOC 2012.

  • Problem

    Object detection must process numerous candidate locations and refine their rough localization, while existing approaches often compromise speed, accuracy, or simplicity.

    Source: §1 · p. 1

  • Method

    Fast R-CNN uses a single-stage, multi-task training algorithm that jointly learns proposal classification and spatial-location refinement, with RoI pooling producing fixed-size feature maps.

    Source: §1 · p. 1, §2.1 · p. 2

  • Results

    Fast R-CNN trains VGG16 9× faster than R-CNN and achieves a VOC12 mAP of 66% versus 62% for R-CNN.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The method improves speed and accuracy while avoiding multi-stage training, feature caching to disk, and fixed convolutional layers.

    Source: §1.1 · p. 2, §6 · p. 8

    SPPnet’s fine-tuning algorithm cannot update convolutional layers preceding spatial pyramid pooling, limiting the accuracy of very deep networks.

    Source: §1.1 · p. 2

Abstract

from arXiv · show

This paper proposes a Fast Region-based Convolutional Network method (Fast R-CNN) for object detection. Fast R-CNN builds on previous work to efficiently classify object proposals using deep convolutional networks. Compared to previous work, Fast R-CNN employs several innovations to improve training and testing speed while also increasing detection accuracy. Fast R-CNN trains the very deep VGG16 network 9x faster than R-CNN, is 213x faster at test-time, and achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains VGG16 3x faster, tests 10x faster, and is more accurate. Fast R-CNN is implemented in Python and C++ (using Caffe) and is available under the open-source MIT License at https://github.com/rbgirshick/fast-rcnn.

1. Introduction

Object detection requires processing many roughly localized proposals and refining them precisely, while prior ConvNet detectors used slow, multi-stage pipelines. Fast R-CNN introduces single-stage joint learning to improve detector training, speed, and accuracy while avoiding several R-CNN and SPPnet limitations.

Motivation

  • Object detection is challenging because many candidate locations must be processed and their rough locations refined precisely.These requirements create trade-offs among speed, accuracy, and simplicity.Source: §1 · p. 1

Fast R-CNN

  • Fast R-CNN jointly learns proposal classification and spatial refinement in a single-stage training algorithm.Its listed advantages include a multi-task loss, updates to all network layers, and no disk storage for feature caching.Source: §1 · p. 1, §1.2 · p. 2

Results

  • 9× faster than R-CNN and 3× faster than SPPnet is Fast R-CNN VGG16 training, while its VOC12 mAP reaches 66% versus 62% for R-CNN.At runtime, the detection network processes images in 0.3s excluding object proposal time.Source: §1 · p. 1

Prior limitations

  • R-CNN uses a multi-stage pipeline that fine-tunes a ConvNet, trains SVMs, and learns bounding-box regressors separately.Its training also requires extracting proposal features and writing them to disk.Source: §1.1 · p. 1
  • 47s / image is the VGG16 R-CNN detection time at test-time on a GPU.R-CNN performs a separate ConvNet forward pass for each object proposal without sharing computation.Source: §1.1 · p. 1
  • SPPnet shares convolutional computation but cannot fine-tune convolutional layers preceding spatial pyramid pooling, limiting very deep-network accuracy.It retains a multi-stage pipeline and disk-based feature extraction.Source: §1.1 · pp. 1–2

2. Fast R-CNN architecture and training

Fast R-CNN processes the whole image once, pools each object proposal into a fixed-size representation, and jointly predicts class probabilities and bounding-box refinements. Its single-stage, feature-sharing training updates all network layers while optimizing classification and localization together.

Architecture

  • The network computes a shared convolutional feature map, applies RoI pooling to each proposal, and sends fixed-length features through fully connected layers.RoI pooling uses max pooling to produce an H × W feature map independent of each proposal’s size.Source: §2 · p. 2, §2.1 · p. 2, Figure 1 · p. 2

Training procedure

  • Hierarchical mini-batch sampling selects images first and RoIs second, allowing RoIs from the same image to share computation and memory.The example configuration samples N = 2 images and R = 128 RoIs, with 64 RoIs per image.Source: §2.3 · p. 3
  • A single fine-tuning stage jointly trains the softmax classifier and bounding-box regressors instead of using separate classifier, SVM, and regressor stages.Back-propagation through RoI pooling routes gradients through the max-pooling argmax selections.Source: §2.3 · pp. 3–4

Architecture

  • Each RoI produces softmax probabilities over K + 1 categories and class-specific bounding-box regression offsets.The two sibling outputs jointly support object classification and localization.Source: Figure 1 · p. 2, §2.3 · p. 3

Training objective

  • Fast R-CNN jointly optimizes classification and bounding-box regression with a multi-task loss, ignoring localization loss for background RoIs.The loss balances the two tasks with λ, and experiments use λ = 1.Source: §2.3 · p. 3
  • The localization objective uses a robust L1 loss that is less sensitive to outliers than the L2 loss used by R-CNN and SPPnet.The paper states that this reduces sensitivity to exploding gradients when regression targets are unbounded.Source: §2.3 · p. 3

3. Fast R-CNN detection

At test time, Fast R-CNN scores pre-computed object proposals with a forward pass, producing class probabilities and class-specific box refinements. Fully connected layers dominate detection cost when many RoIs are processed, motivating truncated-SVD compression for additional speed.

Test-time pipeline

  • Detection uses a forward pass over an image and a list of pre-computed object proposals, typically around 2000 RoIs.The method can also evaluate larger proposal sets, approximately 45k RoIs.Source: §3 · p. 4
  • For each RoI, the network outputs class posterior probabilities and a separate refined bounding-box prediction for each object class.Detection confidence for class k is assigned from the estimated probability Pr(class = k | r).Source: §3 · p. 4

Post-processing

  • Non-maximum suppression is performed independently for each object class using the algorithm and settings from R-CNN.Source: §3 · p. 4

Acceleration

  • Nearly half of the forward-pass time can be spent in fully connected layers because detection processes many RoIs.This contrasts with whole-image classification, where convolutional layers dominate computation.Source: §3.1 · p. 4
  • Truncated SVD factorizes a u × v weight matrix into two fully connected layers, reducing parameters from uv to t(u + v).The reduction is significant when t is much smaller than min(u, v), and the method gives speedups when the number of RoIs is large.Source: §3.1 · p. 4

4. Main results

Fast R-CNN achieves strong VOC detection accuracy while substantially reducing training and testing costs. Its results also show that fine-tuning convolutional layers improves VGG16 accuracy over SPPnet-style training.

  • Fast R-CNN achieves state-of-the-art mAP on VOC07, VOC10, and VOC12.Source: §4 · p. 4

VOC 2010 and 2012 results

  • 65.7% mAP is achieved on VOC12, increasing to 68.4% with extra data.Source: §4.2 · p. 5
  • 68.8% mAP with the enlarged 07++12 training set surpasses SegDeepM’s 67.2%.Source: §4.2 · p. 5

VOC 2007 results

  • Fast R-CNN improves VOC07 mAP over SPPnet from 63.1% to 66.9% through fine-tuning convolutional layers.Source: §4.3 · p. 5

Training and testing time

  • 9× faster VGG16 training reduces time from 84 hours to 9.5 hours compared with R-CNN.Source: §4.4 · p. 6
  • 213× faster testing than R-CNN is achieved with truncated SVD, while detection time falls by more than 30% with a 0.3 percentage-point mAP drop.Source: §4.4 · p. 6

Which layers to fine-tune?

  • Freezing VGG16’s convolutional layers decreases mAP from 66.9% to 61.4%, supporting their importance for deep-network fine-tuning.Source: §4.5 · p. 6

5. Design evaluation

The design evaluation tests multi-task training, scale handling, training-data size, classifier choice, and proposal density. Results favor joint training, single-scale processing for larger networks, more data, softmax, and sparse proposals.

Multi-task training

  • Multi-task training improves pure classification accuracy by +0.8 to +1.1 mAP points across models S, M, and L.Stage-wise training also improves mAP over classification-only training, but underperforms multi-task training.Source: §5.1 · p. 7

Scale invariance

  • Single-scale detection offers the best speed/accuracy tradeoff for larger networks, while multi-scale processing provides only a small mAP increase at high compute cost.VGG16 achieves 66.9% mAP with single-scale processing, compared with 66.0% for R-CNN using “infinite” scales.Source: Table 7 · p. 7, §5.2 · p. 7

Training data

  • Tripling training images from the VOC07 trainval set to 16.5k improves VOC07 test mAP from 66.9% to 70.0%.For VOC10 and VOC2012, a 21.5k-image dataset raises mAP from 66.1% to 68.8% and from 65.7% to 68.4%, respectively.Source: §5.3 · p. 7

Classifier choice

  • Softmax slightly outperforms post-hoc SVM training by +0.1 to +0.8 mAP points across all three networks.The comparison supports one-shot fine-tuning relative to multi-stage training approaches.Source: §5.4 · p. 8, Table 8 · p. 8

Proposal density

  • Increasing selective-search proposals raises mAP before causing a slight decline, while Average Recall does not reliably track mAP as proposal count varies.Using only dense boxes yields 52.9% mAP, and SVMs with hard negative mining perform worse at 49.3%.Source: §5.5 · p. 8

6. Conclusion

Fast R-CNN is presented as a clean, fast update to R-CNN and SPPnet, combining state-of-the-art detection results with experiments that yield new insights. In particular, sparse object proposals appear to improve detector quality, while dense proposals remain an open opportunity for acceleration.

Contribution

  • Fast R-CNN is a clean and fast update to R-CNN and SPPnet.The paper also reports state-of-the-art detection results and detailed experiments.Source: §6 · p. 8

Conclusion

  • Sparse object proposals appear to improve detector quality, and Fast R-CNN makes this issue practical to investigate.The paper notes that undiscovered techniques might allow dense boxes to perform as well as sparse proposals and further accelerate detection.Source: §6 · p. 8

Related papers

Loading 1504.08083v2…