Source-linked AI summary
Cascade R-CNN: Delving into High Quality Object Detection
Zhaowei Cai, Nuno Vasconcelos
TL;DR
High-IoU object detection is difficult because single detectors face overfitting and mismatch between training and inference hypotheses. Cascade R-CNN uses sequentially higher-IoU detectors and surpasses prior single-model detectors on COCO with consistent gains across detector architectures.
Problem
Single object detectors struggle to suppress close false positives while performing well across IoU levels because high-IoU training has few positives and inference hypotheses have mismatched quality.
Method
Cascade R-CNN sequentially trains and applies increasingly selective detector stages, resampling progressively improved hypotheses so training and inference distributions match.
Results
The vanilla Cascade R-CNN surpasses previous state-of-the-art single-model detectors on COCO and delivers consistent 2∼4-point gains across two-stage detector architectures.
Takeaways & Limitations
Cascade R-CNN provides a broadly applicable architecture for high-quality object detection across detector architectures, with improvements independent of baseline strength.
Takeaways & Limitations
The paper notes that multi-threshold classifier ensembles remain limited by rapidly shrinking positive sets and low-quality proposals at inference.
Abstract
from arXiv · showhide
In object detection, an intersection over union (IoU) threshold is required to define positives and negatives. An object detector, trained with low IoU threshold, e.g. 0.5, usually produces noisy detections. However, detection performance tends to degrade with increasing the IoU thresholds. Two main factors are responsible for this: 1) overfitting during training, due to exponentially vanishing positive samples, and 2) inference-time mismatch between the IoUs for which the detector is optimal and those of the input hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, is proposed to address these problems. It consists of a sequence of detectors trained with increasing IoU thresholds, to be sequentially more selective against close false positives. The detectors are trained stage by stage, leveraging the observation that the output of a detector is a good distribution for training the next higher quality detector. The resampling of progressively improved hypotheses guarantees that all detectors have a positive set of examples of equivalent size, reducing the overfitting problem. The same cascade procedure is applied at inference, enabling a closer match between the hypotheses and the detector quality of each stage. A simple implementation of the Cascade R-CNN is shown to surpass all single-model object detectors on the challenging COCO dataset. Experiments also show that the Cascade R-CNN is widely applicable across detector architectures, achieving consistent gains independently of the baseline detector strength. The code will be made available at https://github.com/zhaoweicai/cascade-rcnn.
1. Introduction
Cascade R-CNN addresses high-quality object detection by using sequentially trained detector stages with increasing IoU thresholds. This design resamples progressively improved hypotheses to reduce positive-sample imbalance and aligns inference with detector quality.
- Motivation: Object detection must jointly recognize object classes and localize accurate bounding boxes while suppressing close false positives.Close false positives are bounding boxes that are close but incorrect.
- Motivation: The commonly used u = 0.5 threshold defines loose positives, so detectors frequently produce noisy detections and few close false positives are difficult to suppress.The paper defines hypothesis quality by IoU with ground truth and detector quality by its training IoU threshold u.
- Motivation: Simply increasing u can degrade detection because higher thresholds yield exponentially fewer positive training samples from low-quality hypothesis distributions.This positive-sample collapse is particularly problematic for neural networks.
- Cascade R-CNN: Cascade R-CNN uses multi-stage R-CNN detectors trained sequentially with increasing IoU thresholds, making later stages more selective against close false positives.Each stage uses the preceding stage’s output, motivated by regressors producing output IoUs almost invariably better than input IoUs.
- Cascade R-CNN: Resampling progressively improved hypotheses provides a suitable training distribution for each next higher-IoU detector rather than mining hard negatives.The procedure adjusts bounding boxes so each stage aims to find a good distribution for the next stage.
- Results: 2∼4 points are the consistent gains observed when Cascade R-CNN is built with different two-stage R-CNN detectors.A vanilla implementation reportedly surpasses previous state-of-the-art single-model detectors on COCO, especially under higher-quality evaluation metrics.
2. Related Work
Related work established two-stage detectors as dominant, improved their efficiency through region-wise features and RPNs, and developed one-stage and multi-stage alternatives. These approaches motivate Cascade R-CNN’s positioning within efficient and iterative detection architectures.
- Two-stage detection: R-CNN popularized two-stage detection by combining proposal detection with region-wise classification.This formulation became predominant in recent object detection research.
- Two-stage detection: SPP-Net and Fast-RCNN reduced redundant CNN computation through region-wise feature extraction, while Faster-RCNN added an RPN for further speedup.These developments progressively improved two-stage detector efficiency.
- One-stage detection: One-stage architectures gained popularity for computational efficiency, following sliding-window designs and including YOLO and SSD.YOLO enables real-time detection with fair performance using an efficient backbone, while SSD uses multi-resolution feature maps for varied object scales.
- Multi-stage detection: Prior multi-stage methods iterated bounding-box regression, generated accurate proposals, or embedded cascade architectures within object-detection networks.Examples include the multi-region detector, CRAFT, AttractioNet, and approaches cited as [25].
3. Object Detection
This section introduces two-stage object detection, bounding-box regression, and IoU-based labeling, then identifies why a single detector or naïve ensemble struggles across quality levels. The main limitations are detector mismatch at high IoUs and rapidly shrinking positive training sets as thresholds increase.
- Two-Stage Detection: Faster R-CNN uses an image-wide proposal sub-network H0 followed by a region-of-interest detection head H1 that assigns each hypothesis a classification score and bounding box.The architecture is depicted in Figure 3(a).
- Bounding-Box Regression: Bounding-box regression learns a function f(x, b) to transform a candidate box b into a target box g using localization loss on coordinate offsets.The offsets encode translation and scale differences, and Fast R-CNN uses a smoothed L1 localization loss.
- Bounding-Box Regression: Iterative BBox applies the same regressor repeatedly to refine boxes, but a regressor trained at u = 0.5 is suboptimal for higher-IoU hypotheses and degrades boxes above IoU 0.85.The iterative procedure is represented as repeated composition of f.
- IoU-Based Detection Quality: IoU threshold u determines whether a hypothesis is positive or negative and therefore defines detector quality, but high thresholds yield too few positives while low thresholds encourage acceptance of close false positives.A single classifier is consequently difficult to optimize uniformly across all IoU levels.
- IoU-Based Detection Quality: A naïve ensemble of classifiers optimized for multiple IoU thresholds still fails because the corresponding losses operate on different numbers of positives, which decrease quickly as u increases.The approach also requires ensembling the classifiers at inference.
4. Cascade R-CNN
Cascade R-CNN frames object detection as cascaded regression: specialized detectors progressively improve hypotheses while training and inference use matching, stage-specific distributions. Resampling toward higher-IoU examples keeps successive positive sets roughly constant in size, enabling increasingly selective detectors to be effectively trained.
- Cascade architecture: Cascade R-CNN decomposes difficult uniform regression into a sequence of simpler steps using specialized regressors.Each regressor is optimized for the sample distribution arriving at its stage, progressively improving hypotheses.
- Relation to iterative BBox: Unlike iterative BBox post-processing, cascaded regression resamples hypothesis distributions across stages and is applied during both training and inference.It therefore changes which hypotheses subsequent regressors process rather than merely refining boxes after detection.
- Hypothesis resampling: Cascade regression resamples hypotheses toward higher IoU, keeping successive positive-example sets roughly constant as detector quality increases.The resulting distributions become increasingly tilted toward high-quality examples, while some outliers are removed by increasing IoU thresholds.
- Stage-wise training: At stage t, each R-CNN classifier ht and regressor ft is optimized for an increasing IoU threshold ut, with ut > ut−1.The stage-specific objective produces a sequence of increasingly higher-quality detectors.
- Inference: The same cascade procedure improves hypothesis quality during inference, so higher-quality detectors operate only on higher-quality hypotheses.Applying the cascade at both training and inference removes discrepancy between their hypothesis distributions.
5. Experimental Results
Experiments show that Cascade R-CNN improves detection across stages, proposal qualities, and baseline architectures, with especially strong gains at higher IoU levels. The three-stage cascade offers the best overall trade-off while adding only small computational overhead.
- Proposal Quality: Each detector improves with more precise hypotheses, and higher-quality detectors obtain larger gains; the u = 0.7 detector benefits most from deeper cascade-stage proposals.The u = 0.7 detector performs poorly with low-quality first-stage proposals but much better with more precise deeper-stage hypotheses.
- Ablation Studies: Cascade regression outperforms iterative BBox at later stages, whereas sharing one regressor across integral-loss classifiers yields no visible ensemble gain.Iterative BBox performance drops after repeated application, while the cascade regressor improves on later-stage hypotheses.
- Overall Results: The cascade achieves the best performance across all evaluation metrics, with gains that are mild at low IoU thresholds but significant at higher ones.Iterative BBox and the integral loss detector improve the baseline only marginally.
- Stage-wise Comparison: The first detection stage already outperforms the baseline, the second improves substantially, and the third is equivalent to the second.The gains are attributed to multi-stage multi-task learning, while later stages specialize for their corresponding hypothesis distributions.
- Design Trade-offs: The three-stage cascade achieves the best trade-off, while a fourth stage slightly decreases overall AP despite performing best at high IoU levels; computational overhead remains small.Parameter growth is linear in the baseline detector heads, and the overhead is small during both training and testing.
- Detection Performance: Cascade R-CNN improves baseline detectors by 2∼4 points independently of their strength and adds ∼4 points over the strong FPN+ baseline.The gains are consistent on val and test-dev, and the method outperforms all single-model detectors under every evaluation metric.
6. Conclusion
The paper proposes Cascade R-CNN, a multi-stage framework for high-quality object detection that addresses training overfitting and inference-time quality mismatch. It achieves solid, consistent COCO improvements and applies across many detector architectures.
- Conclusion: Cascade R-CNN is a multi-stage object detection framework designed for high-quality object detectors.The architecture was proposed to address detector design challenges.
- Conclusion: The architecture avoids overfitting during training and quality mismatch during inference.These are the two problems the proposed architecture was shown to avoid.
- Conclusion: Cascade R-CNN delivers solid, consistent detection improvements on the challenging COCO dataset and applies to many object detection architectures.The conclusion presents these results as evidence that the framework may support future object detection research.