Source-linked AI summary
Cascade RPN: Delving into High-Quality Region Proposal Network with Adaptive Convolution
Thang Vu, Hyunjun Jang, Trung X. Pham, Chang D. Yoo
TL;DR
Conventional RPN depends on heuristically chosen anchors and heuristic feature alignment, limiting region-proposal quality. Cascade RPN uses single-anchor multi-stage refinement with progressively stricter sample criteria and adaptive convolution, achieving higher proposal recall and detector mAP.
Problem
Conventional RPN heuristically defines anchors and aligns features to anchors, while anchor scales and aspect ratios require tuning.
Method
Cascade RPN uses one anchor per location, progressively stricter anchor-free then anchor-based criteria, and adaptive convolution guided by refined anchors.
Results
Cascade RPN achieves 13.4 AR points over conventional RPN and improves mAP by 3.1 points in Fast R-CNN and 3.5 points in Faster R-CNN.
Takeaways & Limitations
A simple two-stage Cascade RPN surpasses existing region proposal methods and improves detection performance when integrated into two-stage detectors.
Takeaways & Limitations
Iterative refinement can mismatch anchors with their represented features, while deformable convolution does not enforce alignment to anchor changes.
Abstract
from arXiv · showhide
This paper considers an architecture referred to as Cascade Region Proposal Network (Cascade RPN) for improving the region-proposal quality and detection performance by \textit{systematically} addressing the limitation of the conventional RPN that \textit{heuristically defines} the anchors and \textit{aligns} the features to the anchors. First, instead of using multiple anchors with predefined scales and aspect ratios, Cascade RPN relies on a \textit{single anchor} per location and performs multi-stage refinement. Each stage is progressively more stringent in defining positive samples by starting out with an anchor-free metric followed by anchor-based metrics in the ensuing stages. Second, to attain alignment between the features and the anchors throughout the stages, \textit{adaptive convolution} is proposed that takes the anchors in addition to the image features as its input and learns the sampled features guided by the anchors. A simple implementation of a two-stage Cascade RPN achieves AR 13.4 points higher than that of the conventional RPN, surpassing any existing region proposal methods. When adopting to Fast R-CNN and Faster R-CNN, Cascade RPN can improve the detection mAP by 3.1 and 3.5 points, respectively. The code is made publicly available at \url{https://github.com/thangvubk/Cascade-RPN.git}.
1 Introduction
Cascade RPN targets conventional RPN’s heuristic anchor design and feature-alignment limitations through single-anchor, multi-stage refinement and adaptive convolution. It improves region proposal recall and detection performance over conventional baselines.
- Motivation: Conventional RPN heuristically initializes anchors and relies on uniform feature-map strides for feature alignment.Anchor scales and aspect ratios require tuning, while the alignment rule is only heuristically guaranteed.
- Motivation: Iterative RPN lowers regression variance across stages but provides marginal recall gains because regressed anchors break feature alignment.Stage-2 regression should be easier after stage 1, yet the anchor changes without corresponding feature alignment.
- Cascade RPN: Cascade RPN uses a single anchor per location, combines anchor-free and anchor-based positive-sample criteria, and adapts convolution to refined anchors.Adaptive convolution learns features sampled within anchors, acting as a lightweight RoIAlign layer.
- Results: 13.4 AR points higher than conventional RPN is achieved by a simple two-stage Cascade RPN on COCO, surpassing existing region proposal methods.The method also improves mAP by 3.1 points in Fast R-CNN and 3.5 points in Faster R-CNN.
2 Related Work
Related work spans one- and two-stage object detection, region-proposal methods, multi-stage RPN refinement, and anchor-based versus anchor-free sample discrimination. The paper positions Cascade RPN as addressing alignment and anchor-design issues in these approaches.
- Object Detection: Two-stage detectors first generate region proposals and then refine and classify them with a region-wise detector.Examples include R-CNN, Fast R-CNN, Faster R-CNN, Cascade R-CNN, and HTC.
- Region Proposals: Region proposals act as an attention mechanism supporting accurate bounding boxes while maintaining computational tractability.Earlier proposal methods used superpixel grouping or window scoring.
- Multi-Stage RPN: Multi-stage RPN methods refine proposals repeatedly but can ignore the feature misalignment caused by changing regressed boxes.Deformable convolution is used by some methods, although alignment to anchor geometry is not explicitly enforced.
- Anchor-based vs. Anchor-free Criterion: Anchor-based criteria require multiple heuristically tuned scales and aspect ratios, whereas anchor-free criteria use prediction-point distance to object centers.Anchor-free methods use fewer hyperparameters but can be limited in complex scenes.
3 Region Proposal Network and Variants
The conventional RPN uniformly places anchors, predicts box transformations, and filters regressed anchors into proposals. Iterative refinement reuses regressed anchors but creates mismatch between anchors and represented features; adaptive convolution addresses this alignment issue.
- Region Proposal Network: Anchors are uniformly initialized over an image feature map, with each anchor represented by center coordinates and width and height.The anchor set uses a fixed feature-map stride.
- Region Proposal Network: The regression branch predicts a transformation from each anchor to its target ground-truth box.The regressor takes image features as input and is trained with a bounding-box loss.
- Region Proposal Network: Regressed anchors are obtained by inverse transformation and filtered with non-maximum suppression to produce sparse proposals.The classification branch supplies the objectness scores used in filtering.
- Iterative RPN and Variants: Iterative RPN treats regressed anchors as the next stage’s initial anchors, but changing anchor positions and shapes causes feature mismatch.It repeats the standard transformation and proposal steps across stages.
- Iterative RPN and Variants: Deformable convolution attempts to transform features with anchor geometry, but lacks a constraint enforcing alignment; Cascade RPN uses adaptive convolution instead.Adaptive convolution is designed to systematically ensure the alignment rule.
4 Cascade RPN
Cascade RPN uses anchor-guided adaptive convolution and progressively stricter sample-discrimination metrics to refine proposals while maintaining feature-anchor alignment. Its pipeline trains multi-stage regressors and a classifier, then applies NMS to produce proposals.
- 4.1 Adaptive Convolution: Standard convolution samples features on a regular grid, whereas adaptive convolution replaces that grid with an anchor-inferred offset field.Adaptive convolution therefore guides sampling using the input anchor rather than fixed locations.
- 4.1 Adaptive Convolution: Adaptive offsets combine center displacement with shape-dependent offsets determined by the projected anchor and kernel size.Fractional offsets are sampled using bilinear interpolation.
- 4.1 Adaptive Convolution: Adaptive convolution samples within anchors to preserve alignment between anchors and features, unlike deformable convolution, which learns offsets without this stated anchor constraint.The first Cascade RPN stage uses dilated convolution because anchor center offsets are zero, preserving spatial feature order across stages.
- 4.2 Sample Discrimination Metrics: Cascade RPN uses a single anchor per location and progressively strengthens positive-sample requirements, beginning with an anchor-free center-region criterion and continuing with anchor-based IoU thresholds.This progression addresses the differing sample sufficiency and strictness of anchor-free and anchor-based criteria.
- 4.3 Cascade RPN: The pipeline computes anchor offsets, regresses anchors stage by stage, scores final anchors for objectness, and derives proposals with NMS.The algorithm initializes anchors uniformly over the image and outputs a proposal set P.
- 4.4 Learning: Cascade RPN is trained end-to-end with a multi-task loss combining stage-weighted regression losses and classification loss, balanced by λ.The implementation uses binary cross entropy for classification and IoU loss for regression.
5 Experiments
Experiments on COCO evaluate Cascade RPN for region proposal quality, detection, component contributions, qualitative refinement, stage count, and Cascade R-CNN integration. Across these evaluations, adaptive alignment, mixed sample metrics, and multi-stage refinement produce stronger proposals and detection results, while two stages balance accuracy and inference time.
- Experimental Setting: Experiments use COCO 2017 train, val, and test-dev splits, with ResNet50-FPN and a default two-stage model using one anchor per location.Proposal and ablation analyses use val, while benchmark detection results use test-dev.
- Detection Performance: 40.1 and 40.6 mAP are achieved when Cascade RPN is integrated into Fast R-CNN and Faster R-CNN, compared with 37.0 and 37.1 mAP for RPN.The detector experiments use an IoU threshold of 0.65 and 300 proposals.
- Component-wise Analysis: 71.5 AR1000 is reached after adding alignment, mixed anchor-free and anchor-based metrics, and regression statistics; IoU loss adds 0.2 points.The final gains over conventional RPN are 16.5, 14.7, and 13.4 points for AR100, AR300, and AR1000.
- Acquisition of Alignment: 67.8 AR1000 is obtained with both center and shape alignment, compared with 58.0 without alignment and 64.1 with center alignment alone.Center alignment maintains positional alignment, while shape alignment maintains semantic scope.
- Qualitative Evaluation: Second-stage proposals are more accurate and cover more objects than first-stage proposals in qualitative examples.The comparison uses the first and second rows of Figure 4.
- Number of Stages: The two-stage Cascade RPN achieves the best trade-off between AR1000 and inference time among the evaluated stage counts.The three-stage variant uses an IoU threshold of 0.75 at its third stage.
6 Conclusion
Cascade RPN improves region proposal quality and object detection performance by addressing conventional RPN limitations in anchor definition and feature alignment.
- Cascade RPN systematically addresses conventional RPN limitations in heuristically defining anchors and aligning features to anchors.
- AR improves by 13.4 points over the baseline, surpassing existing region proposal methods.
- Detection mAP improves by 3.1 points with Fast R-CNN and 3.5 points with Faster R-CNN.