PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting RCNN: On Awakening the Classification Power of Faster RCNN

Bowen Cheng, Yunchao Wei, Honghui Shi, Rogerio Feris, Jinjun Xiong, Thomas Huang

arXiv:1803.06799v3cs.CV

TL;DR

Faster RCNN-style detectors leave classification errors that limit detection accuracy, especially among hard false positives. The paper introduces a decoupled classifier refinement module that trains separately from the base detector, achieving new state-of-the-art results on PASCAL VOC and COCO. The method also remains limited by unresolved similar-class false positives and slower inference.

  • Problem

    Hard false positives reveal that Faster RCNN classification may be limited by shared features, multi-task optimization, and fixed receptive fields.

  • Method

    DCR samples high-confidence false positives from a base detector and trains a separate RCNN-styled classifier to refine their classification.

  • Results

    DCR achieves new state-of-the-art results, including 84.0% mAP on VOC2007, 81.2% on VOC2012, and 43.1% on COCO2015 test-dev.

  • Takeaways & Limitations

    Decoupled features, decoupled optimization, and adaptive receptive fields are proposed as design principles for stronger Faster RCNN-style detectors.

  • Takeaways & Limitations

    Some similar-class false positives remain unresolved, while the best DCR model runs slower than baseline Faster RCNN at 1.39 s/image.

Abstract

from arXiv · show

Recent region-based object detectors are usually built with separate classification and localization branches on top of shared feature extraction networks. In this paper, we analyze failure cases of state-of-the-art detectors and observe that most hard false positives result from classification instead of localization. We conjecture that: (1) Shared feature representation is not optimal due to the mismatched goals of feature learning for classification and localization; (2) multi-task learning helps, yet optimization of the multi-task loss may result in sub-optimal for individual tasks; (3) large receptive field for different scales leads to redundant context information for small objects.We demonstrate the potential of detector classification power by a simple, effective, and widely-applicable Decoupled Classification Refinement (DCR) network. DCR samples hard false positives from the base classifier in Faster RCNN and trains a RCNN-styled strong classifier. Experiments show new state-of-the-art results on PASCAL VOC and COCO without any bells and whistles.

1 Introduction

The paper studies the underused classification potential of Faster RCNN-style detectors by analyzing hard false positives. It proposes DCR, a separate classifier refinement module, and reports strong gains across standard benchmarks.

  • Few studies examine the full classification potential of Faster RCNN-style detectors despite their strong benchmark performance.
  • Correcting all false positives hypothetically raises VOC2007 mAP from 79.8% to 86.8%, while removing only high-confidence false positives improves mAP by 3.0%.
  • The paper attributes hard false positives to mismatched classification-localization features, multi-task optimization, and scale-insensitive receptive fields.
  • DCR samples high-confidence false positives from a base classifier and trains a separate RCNN-styled classifier to refine classification.
  • DCR improves detection performance by 2.7% mAP on VOC2007 over a strong baseline and reduces hard false positives.
  • With a ResNet-101 backbone, the method reports 84.0% mAP on VOC2007, 81.2% on VOC2012, and 43.1% on COCO2015 test-dev.

2 Related Work

Related work develops region-based detectors, progressively improving proposal generation, shared computation, deformable features, and multi-scale representations. Classifier cascades provide a conceptual precedent for refining predictions with later-stage classifiers.

  • RCNN uses selective search, region-level deep features, and SVM classifiers, while SPPNet and Fast RCNN improve feature sharing and end-to-end training.
  • Faster RCNN integrates proposal generation through a Region Proposal Network, enabling an efficient two-stage detector.
  • R-FCN, Deformable ConvNets, and FPN extend Faster RCNN with position-sensitive pooling, deformable operations, and top-down multi-scale features.
  • Classifier Cascade: Classifier cascades train later-stage classifiers using difficult or misclassified examples from earlier classifiers.
  • Classifier Cascade: The paper’s DCR resembles classifier-cascade methods but targets classification refinement rather than primarily efficient rejection.

3 Problems with Faster RCNN

The paper identifies three detector design problems: hard classification errors, feature-sharing conflicts, and multi-task compromises, with fixed receptive fields adding excess context for small objects. DCR separates classification from localization and adapts input scale.

  • Faster RCNN hard false positives include low-IoU partial boxes, misclassification between similar classes, and confidently misclassified backgrounds.
  • Hard false positives are attributed to feature sharing, joint classification-localization optimization, and receptive fields that do not adapt to object scale.
  • Problem with Feature Sharing: Classification and localization require different feature properties, making a shared backbone potentially suboptimal for region proposal classification.
  • Problem with Feature Sharing: Faster RCNN minimizes L_detection = L_cls+L_bbox, which may converge to a compromise rather than optimize either task individually.
  • Problem with Optimization: Multi-task learning may become a bottleneck as classification backbones strengthen, because their capacity may not be fully exploited.
  • Problem with Receptive Field: ROI pooling creates a fixed receptive field, so small objects may receive excessive context that makes classification harder.

4 Revisiting RCNN for Improving Faster RCNN

The paper revisits RCNN as a complement to Faster RCNN, arguing that decoupled, stronger classification can correct hard false positives while preserving localization.

  • 4.1 Learning from RCNN Design: Using Faster RCNN proposals with RCNN alone reduces detection from 79.8% to 44.7%, indicating that classification—not box coordinates—causes the degradation.RCNN does not modify box coordinates, and many small-intersection boxes are classified as objects instead of background.
  • 4.1 Learning from RCNN Design: RCNN complements Faster RCNN because its classifier is stronger but unaware of object location, whereas Faster RCNN is weaker but location-aware.Multiplying the two classification scores improves mAP from 79.8% to 81.7%.
  • 4.2 Decoupled Classification Refinement: The proposed DCR module decouples classification and localization by adding a crop-resize layer and strong classifier to the base detector.DCR takes the original image and Faster RCNN boxes as inputs and refines the base classifier’s results.
  • 4.2 Decoupled Classification Refinement: DCR shares no features with the detector backbone and prevents error propagation, allowing the base detector to focus on localization and DCR on classification.This design preserves classification-oriented translation-invariant features and independently optimizes the two components.
  • 4.3 Training: Training proceeds in two stages: first Faster RCNN converges, then DCR is trained on mini-batches sampled from its hard false positives.The DCR parameters are initialized through ImageNet pretraining.
  • 4.3 Training: Table 1 organizes ablations by sampling heuristic, hard-false-positive threshold, sampling size, ROI scale, classifier depth, base detector, and matched model size.The baseline is Faster RCNN with a ResNet-101 backbone, while DCR uses ResNet-50.

5 Experiments

Experiments evaluate DCR on PASCAL VOC using multiple training and inference settings. The ablations show that performance is robust to sampling choices and that accuracy-speed trade-offs depend on ROI size and classifier depth.

  • 5.1 PASCAL VOC: The PASCAL VOC experiments train on the union of VOC 2007 and VOC 2012 trainval sets, including horizontal flips, and evaluate mAP@0.5 on VOC 2007 test.Ablations primarily use ResNet-50 as the DCR classifier.
  • 5.2 Sampling Heuristics: Even random sampling improves mAP by 2.0%, while hard-false-positive-only sampling improves it by 1.6%.Adding foreground examples contributes another 0.2%, whereas adding background examples reduces performance by 1.1%.
  • 5.3 Hard False Positive Threshold: A hard-false-positive threshold of 0.3 gives the best performance, with a gain of 2.5%, while results remain relatively insensitive across thresholds from 0.2 to 0.4.The threshold comparison uses [0.2, 0.25, 0.3, 0.35, 0.4].
  • 5.4 Sampling Size: The difference between the best and worst sampling sizes is only 0.3%, so smaller ROI batches can improve training efficiency without a severe performance drop.The tested sampling sizes are 8, 16, 32, and 64.
  • 5.5 ROI Scale and Speed: Inference speed varies linearly with input-image area, while very small inputs such as 56 × 56 cause a severe accuracy drop.The study compares ROI sizes of 56 × 56, 112 × 112, 224 × 224, and 320×320.
  • 5.6 Classifier Depth and Speed: Deeper DCR classifiers produce more accurate predictions but require more test time, and classifier accuracy correlates with detector accuracy.Depths 18, 34, 50, 101, and 152 are compared as a guideline for selecting the DCR module.
  • 5.7 Main Results: Table 2 reports PASCAL VOC 2007 test detection results for the evaluated methods.The supplied passage identifies the table and evaluation split but does not provide its numerical entries.

Generalization to more advanced object detectors

DCR improves strong Faster RCNN-style detectors across PASCAL VOC and COCO, including detectors using DCN and FPN, by reducing hard false positives.

  • Generalization to more advanced object detectors: DCR remains effective on Deformable ConvNets, whose classifier still produces hard false positives.The method is evaluated on Faster RCNN and DCN.
  • Generalization to more advanced object detectors: 84.0% mAP is achieved on PASCAL VOC 2007 with DCN-DCR without extra data, multi-scale training/testing, ensembles, or post-processing tricks.The result is reported as a new state of the art.
  • Generalization to more advanced object detectors: 81.2% mAP is achieved on the PASCAL VOC 2012 test set using a single model without post-processing tricks.The authors describe this as the first result above 81.0% on that test set.
  • Generalization to more advanced object detectors: On COCO minival, DCR improves Faster RCNN from 30.0% to 33.1% AP, Faster RCNN with DCN from 34.4% to 37.2%, and FPN from 38.2% to 40.2%.FPN+DCN also improves from 41.4% to 42.6%.
  • Generalization to more advanced object detectors: On COCO test-dev, DCR improves Faster RCNN from 30.5% to 33.9% AP, Faster RCNN+DCN from 35.2% to 38.1%, FPN from 38.8% to 40.7%, and FPN+DCN from 41.7% to 43.1%.The authors report a similar trend to COCO minival and state that their best model achieves state-of-the-art performance.

6 Conclusion

The paper analyzes error modes in region-based detectors and proposes DCR around three design principles: decoupled features, decoupled optimization, and adaptive receptive fields.

  • 6 Conclusion: The paper analyzes error modes of state-of-the-art region-based object detectors and studies their potential for accuracy improvement.
  • 6 Conclusion: The authors hypothesize that good object detectors should use decoupled features, decoupled optimization, and adaptive receptive fields.
  • 6 Conclusion: DCR is presented as a simple, effective, and widely applicable module that achieves new state-of-the-art results.
  • 6 Conclusion: Future work includes studying detector architecture, adaptive feature representation in multi-task learning, and DCR efficiency.

RCNN

The supplementary section identifies the paper’s authors and their affiliations at the University of Illinois at Urbana-Champaign and IBM T.J. Watson Research Center.

  • RCNN: The paper lists Bowen Cheng, Yunchao Wei, Honghui Shi, Rogerio Feris, Jinjun Xiong, and Thomas Huang as authors.
  • RCNN: Bowen Cheng, Yunchao Wei, and Thomas Huang are affiliated with the University of Illinois at Urbana-Champaign.
  • RCNN: Honghui Shi, Rogerio Feris, and Jinjun Xiong are affiliated with IBM T.J. Watson Research Center.

1 Differences with Related Works

DCR differs from prior classifier and hard-example methods in its purpose, training design, and emphasis on correcting high-confidence false positives from a base detector.

  • 1 Differences with Related Works: Unlike RCNN’s SVM classifier, DCR is an end-to-end softmax classifier built to correct false positives produced by a base detector.
  • 1 Differences with Related Works: RCNN samples foreground and background with a fixed ratio, whereas DCR specifically emphasizes hard false positives made by Faster RCNN.
  • 1 Differences with Related Works: Online Hard Example Mining selects ROIs with the highest combined classification and localization losses rather than DCR’s correction-focused sampling.
  • 1 Differences with Related Works: Focal Loss down-weights well-classified examples, but its dense-anchor setting mainly down-weights background losses instead of focusing on hard false positives.

2 More Discussions

DCR suppresses hard false positives substantially, while improving detection sensitivity to occlusion and object size. Its accuracy gains come with slower inference than Faster RCNN.

  • False-positive suppression: DCR reduces the number of hard false positives by almost three times on the VOC2007 test set.The best DCR model runs at 1.39 s/image on a 1080 Ti GPU, slower than baseline Faster RCNN.
  • False-positive analysis: The analysis categorizes false positives as localization, similar-object, other-object, or background errors as confidence scores decrease.These categories use overlap with ground-truth objects and predefined similar classes to distinguish error types.
  • Object characteristics: DCR is compared with Faster RCNN on sensitivity to occlusion, truncation, size, aspect ratio, viewpoint, and visible parts.Normalized AP measures detector sensitivity across these six object characteristics.
  • Object characteristics: DCR improves performance particularly for occluded and small objects by focusing on the object area through an adaptive receptive field.The paper attributes this improvement to reduced sensitivity to occlusion and object size.

3 Visualization

Visualization shows DCR suppressing hard false positives involving object parts, similar objects, and background regions. Some similar-object errors remain, but their confidence scores can drop substantially.

  • Visualization: DCR suppresses all three illustrated kinds of hard false positives to some extent.The examples cover object parts, similar objects, and background regions.
  • Object-part errors: A cat head classified by Faster RCNN with confidence 0.98 is eliminated by DCR.This illustrates suppression of a false positive involving only part of an object.
  • Similar-object errors: DCR suppresses most similar-object false positives, but residual errors can remain with substantially reduced confidence scores.Two unsuppressed examples drop from 0.96 to 0.38 and from 0.92 to 0.52; the paper leaves resolving such cases open.
  • Background errors: A background region near a ground-truth object classified as a boat by Faster RCNN is eliminated by DCR.This example represents a background false positive.

Supplementary materials 19

The supplementary figure compares hard false positives between Faster RCNN and the proposed methods. The accompanying material also contains related-work references.

  • Figure comparison: The figure compares hard false positives with confidence scores higher than 0.3 between Faster RCNN and the proposed methods.Red boxes mark ground-truth objects, while green boxes mark hard false positives.
  • Figure encoding: Red boxes denote ground-truth objects in the visualization.The figure uses this marking to distinguish target objects from detections.
  • Figure encoding: Green boxes denote hard false positives in the visualization.These boxes identify the detections being compared across methods.
Loading 1803.06799v3…