Source-linked AI summary
Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Training
Hongkai Zhang, Hong Chang, Bingpeng Ma, Naiyan Wang, Xilin Chen
TL;DR
Dynamic R-CNN addresses the mismatch between fixed detector settings and changing proposal quality during training. It dynamically adjusts label-assignment IoU thresholds and SmoothL1 loss parameters, improving COCO detection performance with no extra overhead.
Problem
Fixed label assignment is difficult because proposal labels depend on ambiguous IoU thresholds, while regression-label distributions shift as proposal quality improves.
Method
Dynamic R-CNN uses Dynamic Label Assignment and Dynamic SmoothL1 Loss to adapt classification and regression training based on proposal statistics.
Results
Dynamic R-CNN improves the baseline by 1.9 points AP and 5.5 points AP90 on MS COCO with no extra cost.
Takeaways & Limitations
The dynamic design generalizes across detectors, backbones, and training or testing techniques while adding no inference overhead.
Abstract
from arXiv · showhide
Although two-stage object detectors have continuously advanced the state-of-the-art performance in recent years, the training process itself is far from crystal. In this work, we first point out the inconsistency problem between the fixed network settings and the dynamic training procedure, which greatly affects the performance. For example, the fixed label assignment strategy and regression loss function cannot fit the distribution change of proposals and thus are harmful to training high quality detectors. Consequently, we propose Dynamic R-CNN to adjust the label assignment criteria (IoU threshold) and the shape of regression loss function (parameters of SmoothL1 Loss) automatically based on the statistics of proposals during training. This dynamic design makes better use of the training samples and pushes the detector to fit more high quality samples. Specifically, our method improves upon ResNet-50-FPN baseline with 1.9% AP and 5.5% AP$_{90}$ on the MS COCO dataset with no extra overhead. Codes and models are available at https://github.com/hkzhang95/DynamicRCNN.
1 Introduction
Two-stage detector training uses fixed settings despite changing proposal quality, creating a mismatch for high-quality detection. Dynamic R-CNN adapts label assignment and regression loss to proposal statistics, improving COCO performance without extra overhead.
- Fixed label assignment and regression settings cannot fit changing proposal distributions during training, limiting high-quality detector training.
- Proposal quality and the number of positive proposals improve during training, motivating dynamic rather than fixed training settings.
- Dynamic Label Assignment progressively adjusts the IoU threshold using proposal statistics for positive and negative sample assignment.
- Dynamic SmoothL1 Loss adjusts β according to regression-label distributions, adapting the loss shape to high-quality samples.
2 Related Work
Region-based detectors generate proposals and refine them through classification and regression, while prior approaches largely ignore the changing quality of proposals during training.
- Region-based object detectors: Region-based detectors generate candidate proposals, sample them with a foreground-background ratio, then classify and refine their locations in a second stage.
- Region-based object detectors: Prior multi-stage methods improve proposals across stages but are effective yet time-consuming and overlook training's inherent dynamic property.
- Bounding box regression: Bounding-box regression performance depends on relative loss weighting, and prior methods adjust regression normalization or gradients across stages.
3 Dynamic Quality in the Training Procedure
Object detection training must handle changing proposal quality in both classification and localization. Fixed IoU thresholds and regression normalization become poorly matched as proposals improve.
- Object detection jointly solves recognition and localization, requiring training procedures for both tasks to achieve high-quality detection.
- 3.1 Proposal Classification: IoU-based label assignment is ambiguous for intermediate-overlap proposals, such as a proposal with IoU 0.5.
- 3.1 Proposal Classification: Faster R-CNN uses predefined positive and negative IoU thresholds, making proposal labels essentially hand-crafted.
- 3.1 Proposal Classification: Directly raising the IoU threshold is impractical early in training because positive samples are scarce.
- 3.2 Bounding Box Regression: Regression-label means and standard deviations decrease as proposals improve, reducing high-quality samples' contributions under fixed SmoothL1 normalization.
4 Dynamic R-CNN
Dynamic R-CNN adapts classification and localization training to changing proposal distributions through Dynamic Label Assignment and Dynamic SmoothL1 Loss.
- 4 Dynamic R-CNN: Dynamic R-CNN adjusts its second-stage classifier and regressor to fit proposal-distribution changes during training.
- 4.1 Dynamic Label Assignment: Dynamic Label Assignment updates the current IoU threshold from proposal IoU statistics, raising it as proposal quality improves.
- 4.1 Dynamic Label Assignment: The DLA pipeline assigns positive and negative labels after adapting the IoU threshold to the evolving proposal distribution.
- 4.2 Dynamic SmoothL1 Loss: SmoothL1 β controls the range of the softer loss, and smaller β accelerates gradient saturation so accurate samples contribute more.
- 4.2 Dynamic SmoothL1 Loss: Dynamic SmoothL1 Loss changes the loss shape according to regression-label statistics to focus progressively on high-quality samples.
- Dynamic R-CNN introduces only one additional hyperparameter relative to the baseline, with updates based on recorded batch statistics.
5 Experiments
Experiments on COCO show that Dynamic R-CNN consistently improves detection quality across backbones, training and testing settings, detector variants, and high-IoU metrics, while adding negligible computational or inference overhead.
- 5.3 Main Results: 39.1% AP with ResNet-50 is 1.8 points higher than the FPN-based Faster R-CNN baseline, while ResNet-101 also provides +1.9 points.The gains remain consistent when combined with longer schedules, multi-scale training and testing, and deformable convolution.
- 5.3 Main Results: 49.2% AP with the best combination is 2.3 points higher than the Faster R-CNN baseline.The combined setting progressively adds a 2× longer schedule, multi-scale training, multi-scale testing, and deformable convolution.
- 5.4 Ablation Experiments: 1.9 points AP and 5.5 points AP90 are the overall gains over the baseline.Dynamic Label Assignment contributes 1.2 box AP points, while Dynamic SmoothL1 Loss raises box AP from 37.0 to 38.0.
- 5.4 Ablation Experiments: The IoU threshold increases and SmoothL1 β decreases during training across different settings.The values are clipped to 0.4 and 1.0 respectively at the beginning, and the overall trends are stable across KI and Kβ.
- 5.5 Efficiency: 13.9 FPS is achieved by Dynamic R-CNN versus 11.2 FPS for Cascade R-CNN under ResNet-50-FPN.The method adds only mean or median calculations over short vectors during training and does not slow inference.
- 5.7 Comparison with State-of-the-Arts: 42.0% AP is achieved with ResNet-101 without bells and whistles, while the enhanced Dynamic R-CNN* reaches 50.1% AP.The enhanced configuration uses image pyramids, deformable convolutions, and Soft-NMS, and is reported to outperform previous detectors.
6 Conclusion
The paper identifies fixed training schemes as limiting detector performance and proposes Dynamic R-CNN to exploit training dynamics without extra cost.
- Dynamic R-CNN analyzes detector training through a dynamic viewpoint, finding that fixed schemes limit overall performance.
- Its Dynamic Label Assignment and Dynamic SmoothL1 Loss components improve detector training without additional cost.
- Extensive experiments across detectors and backbones validate Dynamic R-CNN’s universality and effectiveness.
7 Appendix
Appendix experiments extend Dynamic R-CNN beyond two-stage detection, showing gains on RetinaNet and PASCAL VOC with shared training settings.
- 7.1 Effectiveness on One-stage Detectors: Dynamic training improves RetinaNet by 0.7 points box AP over its baseline on the COCO validation set.
- 7.1 Effectiveness on One-stage Detectors: RetinaNet’s relatively fixed anchor inputs limit Dynamic Label Assignment’s effect, while adjusting β has little impact because its baseline β is already small.
- 7.2 Experimental Results on PASCAL VOC dataset: Using the same hyperparameters as on MS COCO, the VOC results support similar conclusions about Dynamic R-CNN’s effectiveness and universality.
- 7.2 Experimental Results on PASCAL VOC dataset: 1.5 points AP and 3.5 points AP90 improvements over Faster R-CNN are reported on PASCAL VOC.