PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast R-CNN

Ross Girshick

arXiv:1504.08083v2cs.CV

TL;DR

Object detection requires processing numerous rough object proposals and refining their locations, but prior ConvNet detectors relied on slow, multi-stage pipelines. Fast R-CNN uses single-stage joint learning with region-based features to address these limitations. It trains VGG16 9× faster than R-CNN and achieves 66% mAP versus 62% on PASCAL VOC 2012.

  • Problem

    Object detection must process numerous candidate locations and refine their rough localization, while existing approaches often compromise speed, accuracy, or simplicity.

  • Method

    Fast R-CNN uses a single-stage, multi-task training algorithm that jointly learns proposal classification and spatial-location refinement, with RoI pooling producing fixed-size feature maps.

  • Results

    Fast R-CNN trains VGG16 9× faster than R-CNN and achieves a VOC12 mAP of 66% versus 62% for R-CNN.

  • Takeaways & Limitations

    The method improves speed and accuracy while avoiding multi-stage training, feature caching to disk, and fixed convolutional layers.

  • Takeaways & Limitations

    SPPnet’s fine-tuning algorithm cannot update convolutional layers preceding spatial pyramid pooling, limiting the accuracy of very deep networks.

Abstract

from arXiv · show

This paper proposes a Fast Region-based Convolutional Network method (Fast R-CNN) for object detection. Fast R-CNN builds on previous work to efficiently classify object proposals using deep convolutional networks. Compared to previous work, Fast R-CNN employs several innovations to improve training and testing speed while also increasing detection accuracy. Fast R-CNN trains the very deep VGG16 network 9x faster than R-CNN, is 213x faster at test-time, and achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains VGG16 3x faster, tests 10x faster, and is more accurate. Fast R-CNN is implemented in Python and C++ (using Caffe) and is available under the open-source MIT License at https://github.com/rbgirshick/fast-rcnn.

1. Introduction

Object detection requires processing many roughly localized proposals and refining them precisely, while prior ConvNet detectors used slow, multi-stage pipelines. Fast R-CNN introduces single-stage joint learning to improve detector training, speed, and accuracy while avoiding several R-CNN and SPPnet limitations.

  • Motivation: Object detection is challenging because many candidate locations must be processed and their rough locations refined precisely.These requirements create trade-offs among speed, accuracy, and simplicity.
  • Fast R-CNN: Fast R-CNN jointly learns proposal classification and spatial refinement in a single-stage training algorithm.Its listed advantages include a multi-task loss, updates to all network layers, and no disk storage for feature caching.
  • Results: 9× faster than R-CNN and 3× faster than SPPnet is Fast R-CNN VGG16 training, while its VOC12 mAP reaches 66% versus 62% for R-CNN.At runtime, the detection network processes images in 0.3s excluding object proposal time.
  • Prior limitations: R-CNN uses a multi-stage pipeline that fine-tunes a ConvNet, trains SVMs, and learns bounding-box regressors separately.Its training also requires extracting proposal features and writing them to disk.
  • Prior limitations: 47s / image is the VGG16 R-CNN detection time at test-time on a GPU.R-CNN performs a separate ConvNet forward pass for each object proposal without sharing computation.
  • Prior limitations: SPPnet shares convolutional computation but cannot fine-tune convolutional layers preceding spatial pyramid pooling, limiting very deep-network accuracy.It retains a multi-stage pipeline and disk-based feature extraction.

2. Fast R-CNN architecture and training

Fast R-CNN processes the whole image once, pools each object proposal into a fixed-size representation, and jointly predicts class probabilities and bounding-box refinements. Its single-stage, feature-sharing training updates all network layers while optimizing classification and localization together.

  • Architecture: The network computes a shared convolutional feature map, applies RoI pooling to each proposal, and sends fixed-length features through fully connected layers.RoI pooling uses max pooling to produce an H × W feature map independent of each proposal’s size.
  • Training procedure: Hierarchical mini-batch sampling selects images first and RoIs second, allowing RoIs from the same image to share computation and memory.The example configuration samples N = 2 images and R = 128 RoIs, with 64 RoIs per image.
  • Training procedure: A single fine-tuning stage jointly trains the softmax classifier and bounding-box regressors instead of using separate classifier, SVM, and regressor stages.Back-propagation through RoI pooling routes gradients through the max-pooling argmax selections.
  • Architecture: Each RoI produces softmax probabilities over K + 1 categories and class-specific bounding-box regression offsets.The two sibling outputs jointly support object classification and localization.
  • Training objective: Fast R-CNN jointly optimizes classification and bounding-box regression with a multi-task loss, ignoring localization loss for background RoIs.The loss balances the two tasks with λ, and experiments use λ = 1.
  • Training objective: The localization objective uses a robust L1 loss that is less sensitive to outliers than the L2 loss used by R-CNN and SPPnet.The paper states that this reduces sensitivity to exploding gradients when regression targets are unbounded.

3. Fast R-CNN detection

At test time, Fast R-CNN scores pre-computed object proposals with a forward pass, producing class probabilities and class-specific box refinements. Fully connected layers dominate detection cost when many RoIs are processed, motivating truncated-SVD compression for additional speed.

  • Test-time pipeline: Detection uses a forward pass over an image and a list of pre-computed object proposals, typically around 2000 RoIs.The method can also evaluate larger proposal sets, approximately 45k RoIs.
  • Test-time pipeline: For each RoI, the network outputs class posterior probabilities and a separate refined bounding-box prediction for each object class.Detection confidence for class k is assigned from the estimated probability Pr(class = k | r).
  • Post-processing: Non-maximum suppression is performed independently for each object class using the algorithm and settings from R-CNN.
  • Acceleration: Nearly half of the forward-pass time can be spent in fully connected layers because detection processes many RoIs.This contrasts with whole-image classification, where convolutional layers dominate computation.
  • Acceleration: Truncated SVD factorizes a u × v weight matrix into two fully connected layers, reducing parameters from uv to t(u + v).The reduction is significant when t is much smaller than min(u, v), and the method gives speedups when the number of RoIs is large.

4. Main results

Fast R-CNN achieves strong VOC detection accuracy while substantially reducing training and testing costs. Its results also show that fine-tuning convolutional layers improves VGG16 accuracy over SPPnet-style training.

  • Fast R-CNN achieves state-of-the-art mAP on VOC07, VOC10, and VOC12.
  • VOC 2010 and 2012 results: 65.7% mAP is achieved on VOC12, increasing to 68.4% with extra data.
  • VOC 2010 and 2012 results: 68.8% mAP with the enlarged 07++12 training set surpasses SegDeepM’s 67.2%.
  • VOC 2007 results: Fast R-CNN improves VOC07 mAP over SPPnet from 63.1% to 66.9% through fine-tuning convolutional layers.
  • Training and testing time: 9× faster VGG16 training reduces time from 84 hours to 9.5 hours compared with R-CNN.
  • Training and testing time: 213× faster testing than R-CNN is achieved with truncated SVD, while detection time falls by more than 30% with a 0.3 percentage-point mAP drop.
  • Which layers to fine-tune?: Freezing VGG16’s convolutional layers decreases mAP from 66.9% to 61.4%, supporting their importance for deep-network fine-tuning.

5. Design evaluation

The design evaluation tests multi-task training, scale handling, training-data size, classifier choice, and proposal density. Results favor joint training, single-scale processing for larger networks, more data, softmax, and sparse proposals.

  • Multi-task training: Multi-task training improves pure classification accuracy by +0.8 to +1.1 mAP points across models S, M, and L.Stage-wise training also improves mAP over classification-only training, but underperforms multi-task training.
  • Scale invariance: Single-scale detection offers the best speed/accuracy tradeoff for larger networks, while multi-scale processing provides only a small mAP increase at high compute cost.VGG16 achieves 66.9% mAP with single-scale processing, compared with 66.0% for R-CNN using “infinite” scales.
  • Training data: Tripling training images from the VOC07 trainval set to 16.5k improves VOC07 test mAP from 66.9% to 70.0%.For VOC10 and VOC2012, a 21.5k-image dataset raises mAP from 66.1% to 68.8% and from 65.7% to 68.4%, respectively.
  • Classifier choice: Softmax slightly outperforms post-hoc SVM training by +0.1 to +0.8 mAP points across all three networks.The comparison supports one-shot fine-tuning relative to multi-stage training approaches.
  • Proposal density: Increasing selective-search proposals raises mAP before causing a slight decline, while Average Recall does not reliably track mAP as proposal count varies.Using only dense boxes yields 52.9% mAP, and SVMs with hard negative mining perform worse at 49.3%.

6. Conclusion

Fast R-CNN is presented as a clean, fast update to R-CNN and SPPnet, combining state-of-the-art detection results with experiments that yield new insights. In particular, sparse object proposals appear to improve detector quality, while dense proposals remain an open opportunity for acceleration.

  • Contribution: Fast R-CNN is a clean and fast update to R-CNN and SPPnet.The paper also reports state-of-the-art detection results and detailed experiments.
  • Conclusion: Sparse object proposals appear to improve detector quality, and Fast R-CNN makes this issue practical to investigate.The paper notes that undiscovered techniques might allow dense boxes to perform as well as sparse proposals and further accelerate detection.
Loading 1504.08083v2…