Source-linked AI summary
Side-Aware Boundary Localization for More Precise Object Detection
Jiaqi Wang, Wenwei Zhang, Yuhang Cao, Kai Chen, Jiangmiao Pang, Tao Gong, Jianping Shi, Chen Change Loy, Dahua Lin
TL;DR
Precise object localization is difficult for conventional center-and-scale regression, especially with large anchor-target displacements, and existing precision-oriented methods can add computational overhead. SABL instead localizes box sides with side-aware branches and two-step bucketing, improving multiple detector baselines on COCO with limited or no extra inference time.
Problem
Conventional bounding box regression struggles with precise localization under large anchor-target displacements, while some existing improvements add considerable computational overhead.
Method
SABL extracts side-aware boundary features, selects a bucket for each box side, and fine-regresses the boundary position from that bucket.
Results
SABL improves Faster R-CNN, RetinaNet, and Cascade R-CNN by 3.0%, 1.7%, and 0.9% AP, respectively.
Takeaways & Limitations
The method provides consistent performance gains across two-stage, single-stage, and cascade detection pipelines.
Takeaways & Limitations
SABL improves overall localization and AP but can slightly lower AP50, so higher classification accuracy may require additional methods.
Abstract
from arXiv · showhide
Current object detection frameworks mainly rely on bounding box regression to localize objects. Despite the remarkable progress in recent years, the precision of bounding box regression remains unsatisfactory, hence limiting performance in object detection. We observe that precise localization requires careful placement of each side of the bounding box. However, the mainstream approach, which focuses on predicting centers and sizes, is not the most effective way to accomplish this task, especially when there exists displacements with large variance between the anchors and the targets. In this paper, we propose an alternative approach, named as Side-Aware Boundary Localization (SABL), where each side of the bounding box is respectively localized with a dedicated network branch. To tackle the difficulty of precise localization in the presence of displacements with large variance, we further propose a two-step localization scheme, which first predicts a range of movement through bucket prediction and then pinpoints the precise position within the predicted bucket. We test the proposed method on both two-stage and single-stage detection frameworks. Replacing the standard bounding box regression branch with the proposed design leads to significant improvements on Faster R-CNN, RetinaNet, and Cascade R-CNN, by 3.0%, 1.7%, and 0.9%, respectively. Code is available at https://github.com/open-mmlab/mmdetection.
1 Introduction
SABL replaces conventional center-and-scale regression with side-aware boundary localization and a two-step bucketing scheme designed for precise localization under large anchor-target displacements. Across detector families, it improves COCO AP with limited added inference cost.
- Bounding box regression commonly predicts center offsets and relative scale factors, but large-variance anchor-target displacements make precise localization difficult.
- SABL localizes each bounding-box side using surrounding context, first selecting a boundary bucket and then regressing an offset from its centerline.
- 3.0% AP improvement: Faster R-CNN reaches 41.8% versus 38.8% AP with around 10% extra inference time.
- 1.7% AP improvement: RetinaNet reaches 40.5% versus 38.8% AP without extra inference time.
- 0.9% AP improvement: Cascade R-CNN reaches 43.3% versus 42.4% AP.
2 Related Work
Related work spans two-stage and single-stage detection pipelines, while SABL offers boundary-focused bucketing as an efficient alternative to methods that add localization complexity or overhead.
- Two-stage detectors generate region proposals before classification and coordinate refinement, whereas single-stage detectors predict bounding boxes directly.
- SABL decomposes localization for each boundary with bucketing and uses bucketing confidence to improve classification results.
- SABL performs localization in one pass while achieving gains on both two-stage and single-stage pipelines and retaining their efficiency.
3 Side-Aware Boundary Localization
SABL extracts side-aware features and localizes each boundary through bucket estimation followed by fine regression. Bucket confidences also guide rescoring, and the design extends to single-stage detectors with convolutional features.
- 3.1 Side-Aware Feature Extraction: SABL aggregates RoI features along horizontal and vertical axes, then splits them into left, right, top, and bottom side-aware features.
- 3.2 Boundary Localization with Bucketing: The boundary-localization scheme divides candidate regions into buckets, estimates the boundary’s bucket, and regresses its precise offset from the bucket centerline.
- 3.2 Boundary Localization with Bucketing: The nearest bucket is positive, the second-nearest is ignored for bucket classification, and both nearest buckets train the fine regressor.
- 3.3 Bucketing-Guided Rescoring: Bucket-estimation confidences are averaged across four boundaries and multiplied with classification scores for NMS ranking.
- 3.4 Application to Single-Stage Detectors: For single-stage detectors, convolutional layers produce localization features at each feature-map location before the same bucketing and fine-regression steps.
4 Experiments
Experiments on COCO evaluate SABL across detector architectures and ablate its components. SABL improves localization and detection performance, including strong gains at high IoU thresholds, while maintaining modest computational cost.
- Results: SABL improves Faster R-CNN by 3.0% AP with around 10% extra inference time and improves RetinaNet by 1.7% AP without extra inference time.The evaluations use ResNet-101 with FPN on COCO test-dev.
- Results: SABL improves Cascade R-CNN performance by 0.9%, demonstrating gains on a strong cascade baseline.
- Results: SABL achieves the best performance among compared methods while retaining efficiency and outperforming recent localization-focused two-stage detectors.The comparisons include iterative regression and keypoint-based localization pipelines.
- Ablation Study: On Faster R-CNN, SABL raises AP from 36.4% to 39.7% and improves AP90 by 10.5% using a ResNet-50 FPN baseline.The overall improvement is 3.3% AP, especially at high IoU thresholds.
- Ablation Study: Boundary localization with bucketing reaches 38.3% AP versus 36.4% for conventional box regression, while combining it with SAFE reaches 39.0% AP.The bucketing design first selects a boundary bucket and then applies fine regression.
- Analysis: SABL localizes proposals more precisely across IoU bins and produces more positive boxes at all thresholds, especially at IoU ≥0.9.The method nevertheless has slightly lower AP50, which the authors associate with classification sensitivity rather than localization quality.
5 Conclusion
The paper proposes SABL as a replacement for conventional bounding box regression, using side-aware features, two-step bucketing, and rescoring to improve localization across detection pipelines.
- SABL replaces conventional bounding box regression with side-aware features focused on object boundaries.
- A lightweight two-step bucketing scheme locates objects accurately from side-aware features.
- SABL adds a rescoring mechanism that uses bucketing confidence to retain high-quality bounding boxes.
- SABL delivers consistent and significant performance gains across various object detection pipelines.
A Extensions of SABL in COCO Challenge 2019
The authors extend SABL to a COCO Challenge 2019 system with additional architectural and training components, achieving strong single-model and ensemble results on COCO test-dev.
- The system combines SABL with Mask R-CNN, or with HTC, CAFA, and CARAFE for stronger configurations.
- The reported system includes synchronized batch normalization, switchable whitening, deformable convolutions, multiscale training, stronger augmentation, and model ensembling.
- 57.8% APbox and 51.3% APmask are achieved with multiple-model ensemble on COCO 2017 test-dev.
- 56.0% APbox and 49.4% APmask are achieved with a single model on COCO 2017 test-dev.
- The ensemble result exceeds the 2018 COCO Winner Entry by 1.7% APbox and 2.3% APmask.
B Content-Aware Feature Aggregation (CAFA)
CAFA addresses content-insensitive multi-scale feature fusion by introducing content-aware aggregation, combining CARAFE-based upsampling with deformable convolution before feature summation.
- CAFA improves multi-scale feature fusion by accounting for the content of features at different scales.
- CAFA uses CARAFE for upsampling and applies DCNv2 before summing upsampled features with lateral features.
- CAFA adds 1.6% APbox and 1.1% APmask on FPN, and 2.3% APbox and 1.9% APmask on PAFPN.
- 39.2% APbox versus 39.5% APbox is reported for CAFA and NAS-FPN on COCO 2017 val at 640 × 640 scale.
- CAFA with PAFPN uses two pyramid networks with simpler top-down and bottom-up pathways.
C Visual Results Comparison
Visual comparisons on COCO 2017 val show more precise localization for Faster R-CNN with SABL than for the baseline under matched backbone and training settings.
- Faster R-CNN with SABL shows more precise localization results than the Faster R-CNN baseline.
- Both comparisons use a ResNet-101 with FPN backbone and a 1x training schedule.