Source-linked AI summary
Oriented Object Detection in Aerial Images with Box Boundary-Aware Vectors
Jingru Yi, Pengxiang Wu, Bo Liu, Qiaoying Huang, Hui Qu, Dimitris Metaxas
TL;DR
Aerial images contain densely packed objects at arbitrary orientations, challenging horizontal and anchor-based detection. The paper extends a center-keypoint detector with Cartesian BBAVectors and separate horizontal/rotational box handling, reporting better representation performance and favorable benchmark results.
Problem
Aerial objects are densely packed and arbitrarily oriented, while existing methods mainly rely on two-stage anchor-based detectors and direct box-parameter regression.
Method
The method detects object center keypoints, then regresses four-quadrant BBAVectors and separately classifies horizontal versus rotational bounding boxes.
Results
BBAVectors perform better than directly predicting width, height, and angle, while the proposed detector achieves state-of-the-art performances on DOTA and HRSC2016.
Takeaways & Limitations
A single-stage, anchor-free center-keypoint detector can capture oriented bounding boxes with BBAVectors and competes favorably with state-of-the-art methods.
Abstract
from arXiv · showhide
Oriented object detection in aerial images is a challenging task as the objects in aerial images are displayed in arbitrary directions and are usually densely packed. Current oriented object detection methods mainly rely on two-stage anchor-based detectors. However, the anchor-based detectors typically suffer from a severe imbalance issue between the positive and negative anchor boxes. To address this issue, in this work we extend the horizontal keypoint-based object detector to the oriented object detection task. In particular, we first detect the center keypoints of the objects, based on which we then regress the box boundary-aware vectors (BBAVectors) to capture the oriented bounding boxes. The box boundary-aware vectors are distributed in the four quadrants of a Cartesian coordinate system for all arbitrarily oriented objects. To relieve the difficulty of learning the vectors in the corner cases, we further classify the oriented bounding boxes into horizontal and rotational bounding boxes. In the experiment, we show that learning the box boundary-aware vectors is superior to directly predicting the width, height, and angle of an oriented bounding box, as adopted in the baseline method. Besides, the proposed method competes favorably with state-of-the-art methods. Code is available at https://github.com/yijingru/BBAVectors-Oriented-Object-Detection.
1. Introduction
Aerial imagery contains densely packed objects at arbitrary orientations, making horizontal boxes unsuitable and motivating oriented detection. The paper extends a center-keypoint detector with BBAVectors, separates horizontal and rotational boxes, and reports favorable performance.
- Motivation: Aerial objects vary in scale, texture, orientation, and density, so horizontal bounding boxes can misalign with object boundaries.Oriented bounding boxes are preferred to capture arbitrarily oriented objects in complex aerial scenes.
- Limitations of Existing Methods: Existing oriented detectors mainly use two-stage anchor-based designs with complicated anchor choices and positive-negative imbalance.These methods commonly describe boxes using center, width, height, and angle parameters.
- Proposed Approach: The proposed method extends CenterNet by detecting object centers and regressing BBAVectors in four Cartesian quadrants instead of width, height, and angle.BBAVectors represent the top, right, bottom, and left boundaries of oriented boxes.
- Proposed Approach: Horizontal and rotational bounding boxes are handled separately to reduce errors when BBAVectors lie near quadrant boundaries.The method distinguishes HBBs from RBBs, with RBBs covering all oriented boxes except horizontal ones.
- Reported Contributions: The single-stage, anchor-free detector achieves better BBAVector performance than direct width-height-angle regression and state-of-the-art results on DOTA and HRSC2016.The paper characterizes the model as fast and accurate while reporting favorable competition with existing methods.
2. Related Work
Related work contrasts horizontal and oriented detection and highlights the limited use of keypoint-based, anchor-free detectors for oriented aerial objects. The paper positions its CenterNet extension against a direct width-height-angle baseline.
- Detection Representations: Horizontal detectors use HBBs, whereas oriented detection uses OBBs to represent objects appearing at arbitrary directions.The distinction motivates adapting horizontal detection approaches for oriented imagery.
- Proposed Architecture: The proposed architecture uses a U-shaped network with skip connections to produce heatmap, offset, box-parameter, and orientation maps.The input is resized to 608 × 608 before processing, and the output maps have 152 × 152 resolution.
- Keypoint-Based Detection: Keypoint-based detectors provide anchor-free solutions and show speed and accuracy advantages over anchor-based detectors in horizontal detection.Their application to oriented object detection had been limited.
- Baseline: The paper builds a CenterNet baseline that directly regresses oriented-box width, height, and angle, called Center+wh+θ.This baseline provides the comparison for the proposed BBAVector representation.
3. Method
The method extends a keypoint-based detector by locating object centers and decoding oriented boxes from boundary-aware vectors, with separate handling for corner cases.
- 3.1. Architecture: The network uses a U-shaped architecture with a ResNet101 Conv1-5 backbone and produces feature maps at one-quarter input resolution.Deep and shallow features are combined through upsampling and skip connections.
- 3.2. Heatmap: A K-channel sigmoid heatmap detects object centers by assigning one channel to each object category.The predicted heatmap value at a center point serves as the detection confidence.
- 3.2. Heatmap: The heatmap ground truth places a box-size-adaptive 2D Gaussian around each oriented-box center, while a variant focal loss reduces penalties inside Gaussian bumps.Only center points are positive; other locations, including Gaussian-bump points, are treated as negative with reduced penalty inside the bumps.
- 3.3. Offset: An offset map corrects quantization from floating-point image centers to integer heatmap locations, and offsets are optimized with smooth L1 loss.At inference, predicted offsets adjust extracted center points before rescaling them to the input image.
- 3.4. Box Parameters: Instead of predicting width, height, and angle, the detector regresses four quadrant-distributed BBAVectors and two external horizontal box-size parameters.The box parameter vector is b = [t, r, b, l, we, he], with four two-dimensional BBAVectors and two external size parameters.
- 3.5. Orientation: To address corner cases near the Cartesian axes, the method classifies boxes as horizontal or rotational and processes the two types separately.The orientation class uses the intersection-over-union between the oriented and horizontal bounding boxes and is trained with binary cross-entropy.
4. Experiments
The experiments evaluate the proposed detector on DOTA and HRSC2016, including implementation, decoding, state-of-the-art comparisons, orientation-classification ablations, and baseline comparisons.
- Datasets: The method is evaluated on DOTA and HRSC2016, two public aerial-image datasets.DOTA-v1.0 contains 2,806 images and 188,282 instances; HRSC2016 contains 1,061 images with separate training, validation, and testing splits.
- Implementation and decoding: Inference extracts the top 500 heatmap center points, adjusts them with offsets, and decodes RBB or HBB boxes from predicted parameters.RBB corners are formed by combining boundary-aware vectors with the adjusted center, while HBB corners use external width and height parameters.
- Comparison with state-of-the-art methods: 72.32% mAP on DOTA: BBAVectors+rh exceeds ROI Transformer+FPN by 2.76% and runs faster than ROI Transformer.BBAVectors+rh adds orientation classification and external OBB size parameters; a larger training batch produces about 3 points higher performance.
- Comparison with state-of-the-art methods: 12.88 FPS on HRSC2016: BBAVectors+rh is 2.18x faster than ROI Transformer, while BBAVectors+r achieves 88.2% AP.BBAVectors+rh performs 0.4% higher than BBAVectors+r on this dataset.
- Ablation studies: 0.71% on DOTA and 0.4% on HRSC2016: BBAVectors+rh improves over BBAVectors+r by separating HBB and RBB predictions.The classification strategy addresses corner cases where objects nearly align with the xy-axes and vector types are difficult to differentiate near quadrant boundaries.
- Comparison with baseline: 4.82% on HRSC2016 and 2.74% on DOTA: BBAVectors+r outperforms the Center+wh+θ baseline.The comparison uses the same architecture and training procedure, differing in the output box parameter and orientation map.
5. Conclusion
The paper introduces a single-stage, anchor-free oriented object detector based on center points and box boundary-aware vectors. BBAVectors outperform direct width, height, and angle regression, with results surpassing state-of-the-art methods on HRSC2016 and DOTA.
- The proposed method detects object center points and uses box boundary-aware vectors to capture oriented bounding boxes.The vectors provide the paper’s representation for oriented boxes within a center keypoint-based detector.
- The detector is single-stage and free of anchor boxes.This design distinguishes it from the anchor-based oriented detection methods discussed in the paper.
- BBAVectors perform better than directly learning oriented-box width, height, and angle.The comparison uses the baseline that directly regresses these spatial parameters.
- The method demonstrates superiority over state-of-the-art methods on the HRSC2016 and DOTA datasets.