Source-linked AI summary
Oriented RepPoints for Aerial Object Detection
Wentong Li, Yijie Chen, Kaixuan Hu, Jianke Zhu
TL;DR
Aerial detection must represent arbitrary orientations in cluttered scenes without relying on problematic direct angle regression. Oriented RepPoints learns adaptive point sets, converts them into oriented boxes, and selects high-quality samples; experiments across four datasets demonstrate efficacy.
Problem
Aerial targets are often arbitrary-oriented and cluttered, while direct orientation regression can suffer loss discontinuity and regression inconsistency.
Method
Oriented RepPoints uses adaptive points, three oriented conversion functions, APAA quality assessment and assignment, and a spatial constraint penalizing points outside oriented boxes.
Results
Experiments on four challenging aerial datasets demonstrate the efficacy of Oriented RepPoints, with 75.97% mAP and +5.86% gains when all four APAA quality terms are used.
Takeaways & Limitations
Adaptive points provide a fine-grained representation for capturing geometric features of arbitrary-oriented, cluttered, and non-axis-aligned aerial targets.
Abstract
from arXiv · showhide
In contrast to the generic object, aerial targets are often non-axis aligned with arbitrary orientations having the cluttered surroundings. Unlike the mainstreamed approaches regressing the bounding box orientations, this paper proposes an effective adaptive points learning approach to aerial object detection by taking advantage of the adaptive points representation, which is able to capture the geometric information of the arbitrary-oriented instances. To this end, three oriented conversion functions are presented to facilitate the classification and localization with accurate orientation. Moreover, we propose an effective quality assessment and sample assignment scheme for adaptive points learning toward choosing the representative oriented reppoints samples during training, which is able to capture the non-axis aligned features from adjacent objects or background noises. A spatial constraint is introduced to penalize the outlier points for roust adaptive learning. Experimental results on four challenging aerial datasets including DOTA, HRSC2016, UCAS-AOD and DIOR-R, demonstrate the efficacy of our proposed approach. The source code is availabel at: https://github.com/LiWentomng/OrientedRepPoints.
1. Introduction
Aerial detection must handle arbitrary orientations, dense layouts, and complex context, while direct angle regression suffers from discontinuity and inconsistency. Oriented RepPoints addresses these issues with adaptive points, oriented conversion functions, and APAA-based point selection.
- Aerial targets are frequently non-axis aligned, arbitrarily oriented, densely distributed, and surrounded by complex context.
- Direct orientation regression suffers from loss discontinuity and regression inconsistency caused by bounded periodic angles and rotated-box orientation definitions.
- RepPoints offers fine-grained point representations but its original conversion functions produce only upright boxes and do not precisely estimate aerial orientations.
- Oriented RepPoints learns adaptive points that represent diverse orientations, shapes, poses, and underlying geometric structure in aerial instances.
- Three oriented conversion functions produce oriented bounding boxes, while APAA assesses classification, localization, orientation, and point-wise feature correlation during training.
- The paper reports promising qualitative and quantitative results from experiments on four challenging aerial datasets.
2. Related Work
Prior aerial detectors largely extend conventional detectors with orientation regression or geometric transformations, while sample assignment methods select higher-quality training examples. Oriented RepPoints instead uses adaptive point representations within an anchor-free framework.
- Aerial targets differ from generic objects because they are often arbitrarily oriented and densely distributed.
- Many aerial detectors derive from classical detectors by predicting bounding-box rotation angles, while others regress quadrilaterals or transform angular representations.
- Spatial transformations and feature-alignment modules have been used to learn non-axis-aligned representations under oriented-box supervision.
- Simple IoU-threshold assignment may not ensure high-quality samples because of noise and hard cases, motivating learning-to-match assignment strategies.
- Its framework is anchor-free, uses backbone-FPN feature encoding, and applies APAA during training alongside an oriented conversion function.
3. Oriented RepPoints
Oriented RepPoints represents aerial objects with adaptive point sets and converts them into oriented boxes, avoiding direct angle regression. It further learns representative points through quality-aware assignment and spatial regularization.
- Adaptive point representation: Adaptive point sets capture geometric structure across aerial objects with diverse orientations, shapes, and poses.The points are initialized from object centers and refined to adapt to each instance.
- Oriented conversion: Three oriented conversion functions transform learned points into object representations: MinAeraRect, NearestGTCorner, and ConvexHull.MinAeraRect is used during post-processing, while the differentiable NearestGTCorner and ConvexHull functions support training.
- Spatial constraint: A spatial constraint penalizes learned points that fall outside the ground-truth box because clutter and adjacent objects can attract them.The constraint is applied through a spatial loss based on outlier points in each point set.
- Quality assessment: The adaptive points quality measure combines classification, localization, orientation alignment, and point-wise feature correlation.Localization uses converted oriented boxes, while orientation alignment uses Chamfer distance between predicted and ground-truth contour samples.
- Dynamic assignment: APAA dynamically selects top-k point-set samples according to quality scores as positive training samples during refinement.The scheme sorts initialization-stage samples for each object and assigns only selected positive point sets to the target ground-truth box.
4. Experiments
Experiments evaluate Oriented RepPoints through ablations and comparisons across four aerial datasets. Results show benefits from oriented conversion, adaptive-point quality assessment, spatial constraints, and the proposed representation.
- Ablation Study: 68.89% mAP is achieved with the ConvexHull conversion, exceeding 49.69% mAP from the original min-max RepPoints baseline.The oriented MinAeraRect, NearestGTCorner, and ConvexHull conversions improve performance progressively in the reported comparison.
- Ablation Study: +1.39% and +1.46% mAP improvements over angle-based orientation regression are obtained with ResNet-50-FPN and ResNet-101-FPN, respectively.The comparison evaluates adaptive points against direct angle-based orientation regression using two backbones.
- Ablation Study: The spatial constraint improves learning by enforcing adaptive points to remain on their owner instances, especially for weak-feature or background-similar categories.The cited categories include Helicopter, Baseball Diamond, Bridge, and Roundabout.
- Ablation Study: 75.97% mAP and +5.86% gains are obtained when all four quality-assessment terms are used in the APAA scheme.The best sampling ratio reported for dynamic top-k assignment is σ = 0.4, and APAA outperforms the compared assignment schemes without complicated operations.
- State-of-the-art Comparison: 77.63% mAP is achieved on DOTA with Swin-T-FPN, while ResNet-50-FPN and ResNet-101-FPN obtain 75.97% and 76.52% mAP, respectively.The reported DOTA results use single-scale evaluation for comparison with previous methods.
- State-of-the-art Comparison: The method achieves the best reported performance on UCAS-AOD at 90.11% mAP and on DIOR-R at 66.71% mAP.On HRSC2016, it is best under the VOC2012 metric and second-best under VOC2007 with ResNet-50-FPN.
5. Conclusion
The proposed detector uses adaptive points to capture geometric features of arbitrary-oriented, cluttered, non-axis aligned aerial targets. Quality assessment, sample assignment, and spatial constraints support robust adaptive point learning.
- Adaptive points provide a fine-grained representation for capturing key geometric features of arbitrary-oriented aerial targets.
- Quality assessment and sample assignment select high-quality point samples for training the detector.
- A spatial constraint penalizes points outside the oriented box to improve adaptive points learning robustness.
- Experiments on four testbeds report promising results demonstrating the efficacy of the proposed approach.