PaperlayerView exact revision on arXiv

Source-linked AI summary

Anchor-free Oriented Proposal Generator for Object Detection

Gong Cheng, Jiabao Wang, Ke Li, Xingxing Xie, Chunbo Lang, Yanqing Yao, Junwei Han

arXiv:2110.01931v2cs.CV

TL;DR

Oriented detectors commonly rely on horizontal boxes, which can produce poor overlaps and difficult regression targets. The paper proposes the anchor-free AOPG and releases DIOR-R, reporting strong mAP results across three datasets.

  • Problem

    Horizontal boxes used in oriented detectors can have small IoUs with oriented ground truths and extremely large regression targets.

  • Method

    AOPG removes horizontal-box operations by predicting coarse oriented boxes anchor-free, aligning features, refining proposals, and applying a Fast R-CNN head.

  • Results

    64.41%, 75.24% and 96.22% mAP are achieved on DIOR-R, DOTA and HRSC2016, respectively, without bells and whistles.

  • Takeaways & Limitations

    The authors report that AOPG obtains large gains over the baseline on DIOR-R, DOTA and HRSC2016.

Abstract

from arXiv · show

Oriented object detection is a practical and challenging task in remote sensing image interpretation. Nowadays, oriented detectors mostly use horizontal boxes as intermedium to derive oriented boxes from them. However, the horizontal boxes are inclined to get small Intersection-over-Unions (IoUs) with ground truths, which may have some undesirable effects, such as introducing redundant noise, mismatching with ground truths, detracting from the robustness of detectors, etc. In this paper, we propose a novel Anchor-free Oriented Proposal Generator (AOPG) that abandons horizontal box-related operations from the network architecture. AOPG first produces coarse oriented boxes by a Coarse Location Module (CLM) in an anchor-free manner and then refines them into high-quality oriented proposals. After AOPG, we apply a Fast R-CNN head to produce the final detection results. Furthermore, the shortage of large-scale datasets is also a hindrance to the development of oriented object detection. To alleviate the data insufficiency, we release a new dataset on the basis of our DIOR dataset and name it DIOR-R. Massive experiments demonstrate the effectiveness of AOPG. Particularly, without bells and whistles, we achieve the accuracy of 64.41%, 75.24% and 96.22% mAP on the DIOR-R, DOTA and HRSC2016 datasets respectively. Code and models are available at https://github.com/jbwang1997/AOPG.

I. INTRODUCTION

The paper identifies fundamental problems in using horizontal boxes for oriented detection and proposes AOPG, alongside the DIOR-R dataset, to address proposal quality and data insufficiency.

  • Most oriented detectors use horizontal anchors or proposals as references for generating oriented bounding boxes.
  • Horizontal boxes can include background and multiple objects, have small IoUs with oriented ground truths, and create extremely large regression targets.These issues introduce irrelevant information, weaken ground-truth matching, and hurt model robustness.
  • AOPG predicts coarse oriented boxes anchor-free, aligns features with AlignConv, refines boxes, and uses a Fast R-CNN head for final detection.
  • DIOR-R contains 23463 remote sensing images and 192512 instances across 20 common classes, addressing the shortage of large-scale oriented-detection data.
  • AOPG achieves 64.41%, 75.24% and 96.22% mAP on DIOR-R, DOTA and HRSC2016, respectively, without bells and whistles.
  • The authors report large gains over the baseline on DIOR-R, DOTA and HRSC2016.

II. RELATED WORK

Related work includes two-stage and one-stage detection paradigms, oriented detectors that still rely on horizontal boxes, and anchor-free methods motivating AOPG.

  • Two-stage detectors generate RoIs before extracting RoI features, whereas one-stage detectors directly predict detections and often address negative-sample imbalance with Focal Loss.
  • Oriented detection methods improve angle handling, localization, or polygon representation through mechanisms such as IoU-Smooth L1 Loss, feature encoding, and four-point prediction.
  • RoI Transformer and S2ANet improve horizontal-proposal processing, but still retain horizontal boxes and their associated low-IoU and large-shape-distance problems.
  • Anchor-free methods such as FCOS and FoveaBox regress boxes from points, while CornerNet and CenterNet detect object points or point pairs.
  • AOPG applies an anchor-free scheme to generate oriented proposals directly from points without horizontal boxes.

III. OUR APPROACH

The approach compares horizontal- and oriented-box schemes, then uses feature-pyramid CLMs and oriented proposal refinement to generate proposals without horizontal-box operations.

  • AOPG: AOPG uses an FPN and applies a Coarse Location Module on each feature map to generate oriented boxes from feature points.
  • Horizontal-Box Scheme V.S. Oriented-Box Scheme: The comparison evaluates Faster R-CNN's horizontal scheme and AOPG's oriented scheme using IoU and regression-target distributions on DOTA.
  • Horizontal-Box Scheme V.S. Oriented-Box Scheme: Most horizontal proposals have IoUs below 0.5, whereas all sampled oriented proposals have IoUs above 0.5.
  • Horizontal-Box Scheme V.S. Oriented-Box Scheme: Horizontal-proposal regression targets are asymmetric and include many extreme values, increasing detector-training difficulty.
  • Horizontal-Box Scheme V.S. Oriented-Box Scheme: Oriented-proposal regression targets are regular and small compared with those of horizontal proposals.
  • AOPG: The study concludes that horizontal-box operations hurt oriented-detection accuracy, motivating AOPG's high-quality oriented proposal generation.

B. Coarse Location Module

The Coarse Location Module assigns feature-map points to ground truths by scale and central-region membership, then predicts scores, distances, and angles to generate oriented boxes.

  • Region Assignment: CLM assigns each ground truth to an FPN level according to its size and labels points in its central region as positives.The feature maps are P2–P6, with strides 4, 8, 16, 32, and 64.
  • Region Assignment: A central region preserves the ground-truth center and angle while scaling width and height by the central rate σ.It is represented as Bσ = (xgt, ygt, σwgt, σhgt, θgt).
  • Region Assignment: A point becomes positive when its transformed coordinates satisfy |x′| < σwgt/2 and |y′| < σhgt/2.The transformation converts image coordinates into the ground-truth coordinate system.
  • Training: The CLM has score, distance, and angle branches that predict coarse-box scores, side distances, and orientations at each feature-map position.The distance branch is trained only on positive samples, while the score branch uses points from the region assignments.
  • Training: The distance targets are normalized as t∗gt = log(tgt/z), with z defined separately for each feature map.The angle branch directly uses ground-truth angles because θgt lies in the symmetric interval [−π/4, π/4].
  • Training: By combining predicted distances and angles, CLM generates one oriented box at each location.The paper compares these coarse oriented boxes with horizontal RPN proposals.

C. Oriented Box Refinement

The refinement stage aligns features with coarse oriented boxes, then predicts foreground and regression maps to generate accurate oriented proposals.

  • Feature Alignment by AlignConv: AlignConv derives deformable-convolution offsets from coarse oriented boxes rather than a separate small network.Its sampling points follow a regular distribution within each coarse oriented box.
  • Feature Alignment by AlignConv: The feature-alignment operation uses input and output features, deformable-convolution kernel weights, regular-grid vectors, and position offsets.
  • High-quality Oriented Proposals: Two 1×1 convolution layers separately produce classification and regression maps after alignment.These maps identify foreground regions and refine the coarse oriented boxes.
  • High-quality Oriented Proposals: Coarse boxes with IoUs above 0.7 are positives, those below 0.3 are negatives, and intermediate boxes are ignored during proposal refinement.

D. Oriented Object Detection

The second-stage detector uses a modified Fast R-CNN head to classify proposals and regress final oriented boxes, adding angle-bias prediction to the regression branch.

  • D. Oriented Object Detection: The modified Fast R-CNN head predicts classification scores and regresses the final oriented bounding boxes.
  • D. Oriented Object Detection: An angle parameter is added to the regression branch to predict angle bias.

A. Datasets and Training Details

The experiments use DOTA, HRSC2016, and DIOR-R, with DIOR-R providing oriented annotations across a large collection of remote-sensing images and instances.

  • Datasets: DOTA2 contains 2806 images and 188282 instances across 15 common object classes.
  • Datasets: DOTA results use training and validation sets for training and the testing set for evaluation through the DOTA evaluation server.
  • Datasets: HRSC2016 contains 1061 images, with 436 training images, 181 validation images, and 444 testing images.Evaluation uses PASCAL VOC07 and VOC12 metrics.
  • Datasets: DIOR-R reuses DIOR images while adding oriented bounding-box annotations for 23463 images and 192518 instances across 20 common classes.
  • Training Details: Training uses ResNet50 with FPN, oriented Faster R-CNN hyperparameters, an NMS threshold of 0.8, and one NVIDIA RTX 2080 Ti with batch size 2.
  • Training Details: DOTA images are cropped into 1024×1024 patches, while DIOR-R retains the original 800×800 image size.

B. Comparison with State-of-the-Art Methods

AOPG achieves competitive or leading accuracy across DIOR-R, DOTA, and HRSC2016, while maintaining practical inference speed on DOTA.

  • DIOR-R: 64.41% mAP: AOPG achieves dominant accuracy on the DIOR-R dataset over the compared oriented detectors.Faster R-CNN OBB and RetinaNet OBB reach 59.54% and 57.55% mAP, while Gliding Vertex and RoI Transformer reach 60.06% and 63.87%.
  • DOTA: 75.24% mAP and 75.39% mAP: AOPG surpasses other advanced methods on DOTA with ResNet50-FPN and ResNet101-FPN backbones, respectively.With multi-scale training and testing plus rotated augmentation, it reaches 80.66% and 80.19% mAP.
  • DOTA: 80.66% mAP and 80.19% mAP: AOPG obtains the highest reported DOTA results with multi-scale training and testing and rotated augmentation.The strongest results use ResNet50-FPN and ResNet101-FPN, respectively.
  • Speed: 12.4 FPS: AOPG runs on DOTA with ResNet50, only 2.5 FPS slower than Faster R-CNN OBB while improving mAP by 6.19%.The paper characterizes this as an accuracy improvement with acceptable time consumption.
  • HRSC2016: 90.34% and 96.22% mAP: AOPG performs competitively on HRSC2016 under the VOC07 and VOC12 metrics, respectively.Both results are reported as competitive with current state-of-the-art methods.

C. Comparisons of Different Oriented Proposal Generators

On the DOTA validation set, the anchor-free CLM outperforms the compared oriented proposal generators, and AlignConv further improves proposal quality and detection performance.

  • Visualization: RPN, CLM, and CLM+ are the proposal-generation methods visualized in Fig. 12.CLM+ denotes CLM with AlignConv.
  • DOTA validation: 73.69% mAP: CLM alone outperforms RoI-Trans, anchor-based OPG, and FCOS-based OPG on the DOTA validation set.The compared methods reach 73.12%, 72.53%, and 72.17% mAP, respectively.
  • Refinement: 74.16% mAP: Adding AlignConv to CLM improves its DOTA validation accuracy over CLM alone.AlignConv also raises CLM Recall from 89.82% to 90.68%.
  • Refinement: 0.82, 0.88, and 0.86 points: AlignConv increases Recall for anchor-based OPG, FCOS-based OPG, and CLM, respectively.The corresponding mAP gains are 0.56, 0.52, and 0.47 points.

D. Evaluations of AOPG with Different Frameworks

Replacing RPN with AOPG improves detection accuracy in both Cascade R-CNN and Double Head frameworks, showing gains across different architectures.

  • Cross-framework evaluation: 76.31% and 75.56% mAP: Replacing RPN with AOPG raises Cascade R-CNN and Double Head accuracy, respectively.The baseline frameworks reach 73.45% and 72.03% mAP before replacement.
  • Cross-framework evaluation: 2.86↑ and 3.53↑: AOPG provides these mAP gains over the original Cascade R-CNN and Double Head frameworks, respectively.The results display consistent improvements after replacing the original RPN.
  • Conclusion: AOPG contributes to increased detection accuracy and obtains large gains over the baseline on DIOR-R, DOTA, and HRSC2016.The conclusion reports this result without additional bells and whistles.
Loading 2110.01931v2…