PaperlayerView exact revision on arXiv

Source-linked AI summary

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

Zhaowei Cai, Nuno Vasconcelos

arXiv:1906.09756v1cs.CV

TL;DR

High-quality object detection is difficult because low IoU thresholds admit noisy detections, whereas high thresholds cause overfitting and mismatch with inference hypotheses. Cascade R-CNN uses sequential detectors with increasing thresholds and applies the same cascade at inference, achieving consistent gains across challenging detection datasets and extending to instance segmentation.

  • Problem

    Low IoU thresholds produce noisy detections, while higher thresholds often degrade performance through vanishing positives and detector–hypothesis quality mismatch.

  • Method

    Cascade R-CNN sequentially trains detectors with increasing IoU thresholds, resampling each stage's outputs for the next and applying the cascade during inference.

  • Results

    The architecture achieves consistent gains across COCO, PASCAL VOC, KITTI, CityPersons, and WiderFace, and also improves instance segmentation over Mask R-CNN.

  • Takeaways & Limitations

    Cascade R-CNN addresses overfitting during training and quality mismatch during inference across multiple detectors, datasets, and detection or instance-segmentation settings.

  • Takeaways & Limitations

    The evaluation omits some detectors because their single-model COCO test-dev results were not publicly available.

Abstract

from arXiv · show

In object detection, the intersection over union (IoU) threshold is frequently used to define positives/negatives. The threshold used to train a detector defines its \textit{quality}. While the commonly used threshold of 0.5 leads to noisy (low-quality) detections, detection performance frequently degrades for larger thresholds. This paradox of high-quality detection has two causes: 1) overfitting, due to vanishing positive samples for large thresholds, and 2) inference-time quality mismatch between detector and test hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, composed of a sequence of detectors trained with increasing IoU thresholds, is proposed to address these problems. The detectors are trained sequentially, using the output of a detector as training set for the next. This resampling progressively improves hypotheses quality, guaranteeing a positive training set of equivalent size for all detectors and minimizing overfitting. The same cascade is applied at inference, to eliminate quality mismatches between hypotheses and detectors. An implementation of the Cascade R-CNN without bells or whistles achieves state-of-the-art performance on the COCO dataset, and significantly improves high-quality detection on generic and specific object detection datasets, including VOC, KITTI, CityPerson, and WiderFace. Finally, the Cascade R-CNN is generalized to instance segmentation, with nontrivial improvements over the Mask R-CNN. To facilitate future research, two implementations are made available at \url{https://github.com/zhaoweicai/cascade-rcnn} (Caffe) and \url{https://github.com/zhaoweicai/Detectron-Cascade-RCNN} (Detectron).

1 INTRODUCTION

The paper frames high-quality detection as a conflict between rejecting close false positives and retaining enough positive training examples. Cascade R-CNN addresses this through sequentially trained detectors with increasing IoU thresholds and matching inference stages.

  • Object detection jointly requires foreground recognition, class assignment, and accurate bounding-box localization.The two-stage R-CNN framework combines classification and bounding-box regression as a multi-task solution.
  • IoU thresholds define detection quality, but the common 0.5 criterion admits close false positives and makes them difficult to reject.Training examples under u = 0.5 are rich and diverse but noisy for high-quality detection.
  • Detectors trained at u = 0.5, 0.6, and 0.7 perform best near their respective training thresholds, rather than uniformly across IoU levels.The same threshold dependence appears in localization, classification loss, and detection performance.
  • Raising the training threshold can degrade detection because high thresholds reduce positives through overfitting and create a mismatch with lower-quality inference hypotheses.The paper calls this failure mode the paradox of high-quality detection.
  • Cascade R-CNN trains a sequence of increasingly selective R-CNN detectors, using each stage's output to train the next and applying the cascade during inference.Because bounding-box regression usually improves output IoU, each stage supplies a better hypothesis distribution for the next threshold.
  • A vanilla Cascade R-CNN surpasses almost all previous single-model detectors on COCO, with consistent 2∼4-point gains and larger gains under stricter localization metrics.The architecture also extends to instance segmentation and improves over Mask R-CNN.

2 RELATED WORK

Related work develops two-stage and one-stage detection architectures, scale-aware proposal mechanisms, and earlier multi-stage localization procedures. Cascade R-CNN builds on these directions while extending cascading to detector stages rather than only proposal generation or iterative localization.

  • R-CNN, SPP-Net, Fast R-CNN, and Faster R-CNN established two-stage detection with shared region-wise features and, later, an integrated proposal network.Faster R-CNN became a cornerstone architecture for modern object detection.
  • MS-CNN and FPN address scale variation by generating proposals across multiple feature layers or resolutions.These methods use different receptive-field sizes or top-down connections to improve scale coverage.
  • YOLO and SSD pursue computational efficiency through one-stage detection, but their accuracy is typically below that of two-stage detectors.YOLO produces sparse results in a single backbone pass, while SSD uses multiple-resolution feature maps.
  • Earlier multi-stage methods used iterative bounding-box regression, accurate proposal generation, or sequential object localization.These approaches are related in spirit but precede the Cascade R-CNN framework.
  • Subsequent work applied cascading to single-shot detectors and explored additional regression steps, supporting the idea's applicability across architectures.The cited work reports improvements for high-quality single-shot detection and localization.
  • Figure 3 compares frameworks using image input, backbone convolutions, region-wise pooling, network heads, bounding boxes, classifications, and shared proposals B0.The caption identifies the symbols used to compare the architectures.

3 HIGH QUALITY OBJECT DETECTION

High-quality detection is difficult because IoU thresholds trade off diverse positive samples against rejecting close false positives, while detector and hypothesis qualities must be matched. The section reviews the two-stage detection components, training objectives, and the resulting challenges for stricter detection.

  • 3.1 Object Detection: Object detection combines recognition and localization, assigning class labels and accurate bounding boxes within a two-stage R-CNN framework.Faster R-CNN uses a proposal sub-network followed by a region-of-interest detection head with classification and bounding-box outputs.
  • 3.1.1 Bounding Box Regression: Bounding-box regression learns to transform candidate boxes into target boxes using smooth L1 loss on normalized coordinate offsets.The offsets encode center, width, and height differences, and normalization helps balance regression with classification in multi-task learning.
  • 3.2 Detection Quality: IoU determines whether hypotheses are positive or negative, and the training threshold u defines detector quality: larger values favor tighter alignment.The same threshold is commonly used to select samples for classification and regression, although the tasks do not require identical thresholds.
  • 3.2 Detection Quality: The standard u = 0.5 compromise provides rich positives but permits close false positives, whereas high thresholds make large positive training sets difficult.Most proposal hypotheses are low quality, so detectors must discriminate among finely separated positive and negative cases.
  • 3.3 Challenges to High Quality Detection: Cascade R-CNN uses cascade regression to resample hypotheses toward higher IoU, keeping successive positive sets roughly constant as u increases.The provided discussion describes this resampling mechanism and its progressively higher-quality sample distributions across cascade stages.
  • 3.3 Challenges to High Quality Detection: High-quality detection requires increasing both hypothesis and detector quality, because improving only one leaves a quality mismatch or too few suitable hypotheses.The section identifies this coupled requirement as part of the paradox of high-quality detection and notes that historical metrics often emphasize low-quality regimes.

4 CASCADE R-CNN

Cascade R-CNN extends R-CNN with sequentially trained detector stages that progressively improve hypotheses and match detector quality to proposal quality. Cascaded regression and detection reduce overfitting and quality mismatch, distinguishing the method from iterative BBox and integral-loss approaches.

  • Architecture: Cascade R-CNN is a multi-stage extension of Faster R-CNN that combines cascaded bounding-box regression with cascaded detection.The architecture uses the RPN for proposal detection and adds multiple detector stages.
  • Cascaded Bounding Box Regression: Each regressor is optimized for the bounding-box distribution produced by the preceding regressor, progressively improving hypotheses.The cascade composes stage-specific regressors rather than repeatedly applying one regressor.
  • Cascaded Detection: Cascade regression resamples examples toward higher IoU, keeping successive stages’ positive sets roughly constant as detector thresholds increase.This addresses the scarcity of positive examples at high IoU thresholds.
  • Cascaded Detection: Sequential cascade learning reduces high-threshold overfitting, specializes deeper detectors for higher IoUs, and improves later-stage regression after outlier removal.The same cascade is applied during inference so higher-quality detectors process higher-quality hypotheses.
  • Differences from Previous Works: Unlike iterative BBox, Cascade R-CNN uses cascade regression during training and inference, avoiding distribution discrepancy and using multiple specialized regressors.The paper reports more precise localization than iterative BBox without human engineering.
  • Differences from Previous Works: Unlike integral loss, Cascade R-CNN maintains nearly constant positive counts across IoU thresholds rather than exposing high-quality classifiers to rapidly shrinking positive sets.The cited comparison identifies shrinking positives as a source of overfitting for high-quality classifiers.

5 INSTANCE SEGMENTATION

The paper extends Cascade R-CNN to instance segmentation by adding Mask R-CNN-style segmentation branches at different cascade locations. Three strategies vary whether one branch is placed early or late, or whether every detection stage receives a branch.

  • Extension to Instance Segmentation: Cascade R-CNN is extended to instance segmentation by adding a segmentation branch similar to Mask R-CNN.The extension builds on the observation that stronger object detection generally improves instance segmentation.
  • Cascade Mask R-CNN Strategies: The paper evaluates one segmentation head at the first stage, one at the last stage, and a segmentation branch at every cascade stage.These strategies address both branch placement and the number of segmentation branches.
  • Cascade Mask R-CNN Strategies: Later segmentation placement provides more positive training examples, whereas per-stage branches maximize sample diversity for mask prediction.Highly overlapping instances may be less helpful for pixel-wise segmentation than for patch-based detection.
  • Inference: All three strategies predict masks on patches from the final detection stage, while the per-stage design combines three segmentation branches at inference.The single-head designs use the corresponding single segmentation branch for final mask prediction.
  • Results: Cascade Mask R-CNN architectures outperform Mask R-CNN in the reported experiments.

6 EXPERIMENTAL RESULTS

The evaluation spans multiple datasets and baseline architectures, with most experiments conducted on COCO. The setup measures performance across IoU qualities and uses a standard three-head cascade with thresholds 0.5, 0.6, and 0.7.

  • Experimental Setup: Experiments evaluate Cascade R-CNN across multiple datasets and baseline network architectures.The baselines include Faster R-CNN, MS-CNN, R-FCN, FPN, and Mask R-CNN.
  • Datasets and Metrics: COCO supplies approximately 118k training images, 5k validation images, and approximately 20k test-dev images without provided annotations.COCO AP averages AP across IoU thresholds from 0.5 to 0.95 at intervals of 0.05.
  • Implementation: The default Cascade R-CNN uses four stages: one RPN and three detection heads with thresholds U = {0.5, 0.6, 0.7}.Subsequent stages resample using all regressed outputs from the preceding stage.
  • Baseline Architectures: The evaluation covers object detection baselines with VGG-Net and ResNet backbones and uses Mask R-CNN with ResNet backbones for instance segmentation.The experiments use end-to-end training instead of multi-step training unless otherwise noted.
  • Specific-Category Datasets: The study also evaluates MS-CNN-based Cascade R-CNN on KITTI, CityPersons, and WiderFace.MS-CNN serves as the baseline detector for these specific object-category datasets.

6.2 Quality Mismatch

Quality-mismatch experiments show that detector performance depends on matching detector quality with proposal quality, while comparisons indicate Cascade R-CNN is strongest at high IoU levels. Its gains are especially large for strict localization metrics.

  • Quality Mismatch: The u = 0.5 detector outperforms u = 0.6 at low IoU levels but underperforms it at higher levels, whereas u = 0.7 underperforms both.These results motivate examining proposal quality at inference.
  • Quality Mismatch: Matching test proposals to detector quality significantly improves the u = 0.6 and u = 0.7 detectors.Cascade stage proposals provide higher-quality hypotheses for the corresponding detectors.
  • Quality Mismatch: More precise hypotheses improve every Cascade R-CNN detector, with larger gains for higher-quality detectors.This pattern is reported across detector qualities and cascade stages.
  • Comparison with Iterative BBox: Iterative BBox localization degrades with repeated application, while cascade regression improves at later stages and outperforms iterative BBox.The comparison attributes the difference to specialized cascade regressors rather than repeatedly applying one regressor.
  • Comparison with Integral Loss: The integral-loss classifier at u = 0.6 is best across IoU levels, u = 0.7 is worst, and their ensemble shows no visible gain.
  • Overall Comparison: Cascade R-CNN achieves the best performance at all IoU levels, with gains of 0.8 for AP50, 6.1 for AP80, and 8.7 for AP90.The reported gains are mild at low IoU and significant at higher IoU.

6.4 Ablation Experiments

Ablations show that cascade performance improves through stage specialization, increasing IoU thresholds, and multi-stage design, with three stages offering the best cost–AP trade-off.

  • Stage-wise Comparison: The second cascade stage substantially improves performance, while the third remains comparable and the ensemble of all classifiers performs best overall.Deeper stages favor higher-quality localization and encourage features that also benefit earlier stages through backbone sharing.
  • IoU Thresholds: Increasing IoU thresholds across stages improves performance beyond using the same threshold for all heads.Higher thresholds make later detectors more selective against close false positives and better matched to precise hypotheses.
  • Regression Statistics: Updating regression statistics stage by stage supports effective multi-task learning, although performance is not highly sensitive to their exact values.The statistics normalize bounding-box regression distance vectors for each cascade stage.
  • Stage Losses: The decay scheme weights later-stage losses more lightly, using w_t = 1/2^(t−1), and is adopted for the remaining experiments.Head learning rates are rescaled to ensure sufficient training under either loss-weighting scheme.
  • Number of Stages: Three stages provide the best trade-off between computational cost and AP, whereas a fourth stage slightly decreases overall performance despite stronger high-IoU results.Adding a second stage significantly improves the baseline, and a third provides further nontrivial gains.

6.5 Comparison with the state-of-the-art

Across COCO comparisons and implementations, Cascade R-CNN delivers strong gains over single- and multi-stage baselines with small computational overhead and consistent generalization across architectures and backbones.

  • COCO Comparison: Approximately 4 points of improvement over FPN+ enables the vanilla Cascade R-CNN to outperform almost all single-model detectors across COCO evaluation metrics.The comparison includes detectors using various training and inference enhancements, making some comparisons unfair.
  • COCO Comparison: 5.2 points above SNIP are achieved by an enhanced Cascade R-CNN using multi-scale processing, a stronger backbone, and additional supervision.The enhanced model also outperforms MegDet, which reports 50.6 mAP.
  • Generalization Capacity: Cascade R-CNN gains remain consistent across Faster R-CNN, R-FCN, and FPN+ baseline detectors.The architecture is designed to combine with different two-stage detector frameworks.
  • Efficiency: The Cascade R-CNN adds parameters linearly with the number of stages but incurs small computational overhead because heads require much less computation than the backbone.Timing is reported per image on a single Titan Xp GPU in the corresponding implementation comparison.
  • Generalization Capacity: 3∼4 points of improvement are observed across backbones, codebases, and implementations, including independent PyTorch and TensorFlow reproductions.These results indicate reliable gains across detector architectures and implementation environments.

6.7 Proposal Evaluation

Proposal evaluation shows that bounding-box regression stages substantially improve proposal recall, supporting later high-quality detection.

  • Proposal Recall: Proposal recall rises from 59.1 to 70.7 for AP1k after adding a bounding-box regression stage.The additional regression stage produces a significant recall improvement, while a third stage adds a smaller but non-negligible gain.
  • Proposal Recall: The high proposal recall achieved by the cascade secures the subsequent high-quality object-detection task.The first stage is close to the FPN baseline, with later regression stages providing the main recall gains.

6.8 Instance Segmentation by Cascade Mask R-CNN

Cascade Mask R-CNN extends the cascade to instance segmentation and improves baseline performance, though gains are smaller at high quality and vary by strategy cost.

  • Performance: All three Cascade Mask R-CNN strategies improve over the Mask R-CNN baseline, but object-detection gains exceed instance-segmentation gains at high quality.An AP90 improvement of 8.7 points for object detection falls to 1.8 points for instance segmentation.
  • Strategy Comparison: Strategy (c) outperforms strategy (b) because it reduces the mismatch between the stage used to train the mask head and the stage used for testing.Strategy (b) trains the mask head in the first stage but tests after the last stage.
  • Strategy Comparison: Adding a mask branch to every stage provides no noticeable benefit over strategy (c) while requiring substantially more computation and memory.Strategy (b) is selected for the remaining experiments because it offers the best cost–AP trade-off.
  • Backbone Robustness: Cascade Mask R-CNN gains are consistent across the evaluated backbone networks, including the strongest model with augmentation and a 1.44x schedule.Detection results are also reported because the architecture performs both object detection and instance segmentation.

6.9 Results on PASCAL VOC

Cascade R-CNN significantly improves detection performance on PASCAL VOC and additional specific detection datasets. Evaluation emphasizes high-quality localization using stricter IoU-aware metrics where applicable.

  • PASCAL VOC: Cascade R-CNN significantly improves overall AP for all evaluated detector architectures on PASCAL VOC.VOC2007 and VOC2012 trainval were used for training, with VOC2007 test used for evaluation under COCO metrics.
  • KITTI: 0.87 points and 1.9 points improve KITTI car detection over MS-CNN for the Moderate and Hard regimes, respectively.The KITTI car task uses AP at IoU=0.7 and is described as fairly saturated.
  • CityPersons: CityPersons evaluation reports miss-rate at IoU=0.5 and additionally at IoU=0.75 to assess higher-quality pedestrian detection.The stricter IoU=0.75 metric is described as more commensurate with high-quality detection.
  • WiderFace: WiderFace evaluation uses AP at IoU=0.5 across easy, medium, and hard subsets, with additional evaluation at IoU=0.75.The dataset contains faces spanning different detection difficulties and varies substantially in scale, pose, and occlusion.

7 CONCLUSION

The paper concludes that Cascade R-CNN addresses overfitting during training and quality mismatch during inference through sequential multi-stage training and cascaded inference. It reports consistent gains across diverse detection and instance-segmentation settings.

  • Conclusion: Cascade R-CNN addresses overfitting during training and quality mismatch during inference in high-quality object detection.Stages are trained sequentially, using one detector’s output to train the next, and the same cascade is applied at inference.
  • Conclusion: Consistent performance gains occur across COCO, PASCAL VOC, KITTI, CityPersons, and WiderFace for generic and specific object detection.The gains also appear across multiple detector architectures, backbone networks, and detection and instance-segmentation techniques.
  • Conclusion: Cascade R-CNN is presented as useful for future object detection and instance segmentation research efforts.The conclusion bases this view on gains observed across datasets, detector types, backbones, and techniques.
Loading 1906.09756v1…