PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning RoI Transformer for Detecting Oriented Objects in Aerial Images

Jian Ding, Nan Xue, Yang Long, Gui-Song Xia, Qikai Lu

arXiv:1812.00155v1cs.CV

TL;DR

Aerial detection is challenged by misaligned horizontal proposals and the computational cost of dense rotated anchors. The RoI Transformer learns rotated regions from horizontal ones and extracts rotation-invariant features, achieving strong results on aerial benchmarks with low added cost. Its comparisons and conclusion support using it as an alternative to deformable RoI pooling when oriented-box annotations are available.

  • Problem

    Horizontal proposals can misalign with densely packed, arbitrarily oriented objects, while rotated-anchor designs substantially increase computational complexity.

  • Method

    The RoI Transformer learns RRoIs from HRoIs and applies Rotated Position Sensitive RoI Align to extract rotation-invariant features for classification and regression.

  • Results

    The method achieves state-of-the-art performance on challenging aerial datasets, improves detectors, and outperforms deformable PS RoI pooling by 3.85 points in the reported comparison.

  • Takeaways & Limitations

    The RoI Transformer can be integrated into other detector architectures and serves as an optional substitution for deformable RoI pooling when oriented-box annotations are available.

Abstract

from arXiv · show

Object detection in aerial images is an active yet challenging task in computer vision because of the birdview perspective, the highly complex backgrounds, and the variant appearances of objects. Especially when detecting densely packed objects in aerial images, methods relying on horizontal proposals for common object detection often introduce mismatches between the Region of Interests (RoIs) and objects. This leads to the common misalignment between the final object classification confidence and localization accuracy. Although rotated anchors have been used to tackle this problem, the design of them always multiplies the number of anchors and dramatically increases the computational complexity. In this paper, we propose a RoI Transformer to address these problems. More precisely, to improve the quality of region proposals, we first designed a Rotated RoI (RRoI) learner to transform a Horizontal Region of Interest (HRoI) into a Rotated Region of Interest (RRoI). Based on the RRoIs, we then proposed a Rotated Position Sensitive RoI Align (RPS-RoI-Align) module to extract rotation-invariant features from them for boosting subsequent classification and regression. Our RoI Transformer is with light weight and can be easily embedded into detectors for oriented object detection. A simple implementation of the RoI Transformer has achieved state-of-the-art performances on two common and challenging aerial datasets, i.e., DOTA and HRSC2016, with a neglectable reduction to detection speed. Our RoI Transformer exceeds the deformable Position Sensitive RoI pooling when oriented bounding-box annotations are available. Extensive experiments have also validated the flexibility and effectiveness of our RoI Transformer. The results demonstrate that it can be easily integrated with other detector architectures and significantly improve the performances.

1 Introduction

Aerial imagery makes oriented, densely packed object detection difficult because horizontal proposals can misalign with objects, while rotated proposals increase anchor complexity. The RoI Transformer learns rotated regions and extracts rotation-invariant features to improve detection efficiently.

  • Birdview imagery contains arbitrarily oriented objects amid complex backgrounds and varied appearances, making detection challenging.
  • Horizontal RoIs can contain several crowded instances, creating misalignment that complicates accurate feature extraction, classification, and localization.
  • Rotated-proposal methods reduce mismatching but require many anchors across angles, scales, and aspect ratios, increasing computational complexity.
  • The RoI Transformer combines a supervised RRoI learner with Rotated Position Sensitive RoI Align for rotation-invariant features and subsequent classification and localization.
  • The method achieves state-of-the-art performance on public aerial datasets and can be embedded into other detector architectures with significant performance improvements.

2 Related Work

Prior methods either suffer from region-feature misalignment or incur substantial cost from rotated proposal generation. The paper addresses these issues while retaining efficient RoI-wise processing for densely populated aerial scenes.

  • HRoI-based methods can misalign region features with instances, whereas RRoI-based methods avoid this problem to a certain extent.
  • Rotated-anchor designs multiply anchors across scales, aspect ratios, and angles, increasing network computation; one example uses 3×5×6 = 90 anchors per location.
  • Spatial Transformer and deformable convolution or RoI pooling model geometric variation, but are designed for general deformation rather than oriented-box supervision.
  • 2.3 Light RoI-wise Operations: RoI-wise computation is an efficiency bottleneck because operations are not shared, especially when images contain hundreds of densely packed instances.
  • The RoI Transformer architecture learns rotated regions from estimated horizontal regions, warps feature maps for rotation invariance, and trains both layers end to end.

3 RoI Transformer

The RoI Transformer learns rotated regions from horizontal proposals, then extracts rotation-invariant features and computes rotation-aware targets for oriented detection.

  • 3.1 RRoI Learner: The RRoI Learner predicts rotated RoI parameters from feature maps associated with horizontal RoIs using fully connected layers.The predicted offsets are decoded into RRoI location, dimensions, and orientation, with Smooth L1 optimization.
  • 3.2 Rotated Position Sensitive RoI Align: RPS RoI Align divides each rotated RoI into K × K bins and outputs a K × K × C feature map through position-sensitive sampling.Sampling coordinates are determined by the RRoI, and Eq. (3) is typically implemented with bilinear interpolation.
  • 3.3 RoI Transformer for Oriented Object Detection: The RoI Transformer combines RRoI learning with RPS RoI Align to produce rotation-invariant pooled features and better initialization for later regression.The matched RRoI is closer to the ground-truth oriented box than the matched horizontal RoI.
  • 3.3 RoI Transformer for Oriented Object Detection: Oriented-box IoU is computed over polygon intersections for matching and non-maximum suppression, with RRoIs treated as true positives above an IoU of 0.5.For long, thin boxes, small angle jitter can still produce very low IoU and complicate suppression.
  • 3.1 RRoI Learner: The learner uses location-, size-, and angle-relative offsets defined in the RRoI coordinate system rather than the image coordinate system.This coordinate binding makes the regression targets rotation-invariant.

4 Experiments and Analysis

Experiments evaluate the RoI Transformer on DOTA and HRSC2016, through ablations, comparisons with deformable PS RoI pooling, and state-of-the-art benchmarks. The method improves accuracy while maintaining lightweight computation and can be integrated into detector architectures.

  • Experimental Setup: Experiments use DOTA and HRSC2016, with Light-Head R-CNN OBB as the baseline framework for evaluating oriented object detection.DOTA provides large-scale multi-category aerial imagery, while HRSC2016 focuses on challenging ship detection.
  • Complexity: The RoI Transformer uses fewer learned localization parameters than deformable PS RoI pooling and runs faster at inference, but trains slightly slower.It uses 5 localization parameters; inference takes 0.17s versus 0.206s per image, while training takes 0.475s versus 0.445s.
  • Comparison with Deformable PS RoI Pooling: The RoI Transformer exceeds deformable PS RoI pooling by 3.85 points in detection accuracy after deformable pooling improves the Light-Head R-CNN OBB baseline by 5.6 points.The authors attribute the gain to improved geometry modeling and regression targets relative to the RRoI rather than the HRoI.
  • Ablation Studies: Adding the RoI Transformer improves mAP by 4.87 points in the ablation study, while contextual RRoI enlargement adds 2.86 AP points and removing RRoI NMS adds about 1.5 mAP points.The contextual region enlarges the long and short RRoI sides by factors of 1.2 and 1.4, respectively.
  • Comparisons with the State-of-the-art: On DOTA, the RoI Transformer reaches 69.56 mAP with the Light-Head OBB FPN baseline, a 2.6-point improvement, and achieves 67.74 mAP without FPN.Its ship-category performance improves by 26.34 points over the previous best result of 57.25.
  • Comparisons with the State-of-the-art: On HRSC2016, the method reaches 86.16 mAP without FPN, exceeding RRD by 1.86 points while using five rather than thirteen aspect ratios.The qualitative results also include closely arranged ships that are difficult to distinguish with horizontal rectangles.

5 Conclusion

The RoI Transformer models geometric transformations to address misalignment between region features and objects, improving oriented object detection on DOTA and HRSC with negligible computation-cost increase.

  • The RoI Transformer models geometry transformations to solve misalignment between region features and objects.
  • It brings significant improvements for oriented object detection on the challenging DOTA and HRSC datasets with negligible computation-cost increase.
  • Compared with deformable RoI pooling, it is more reasonable when oriented bounding-box annotations are available and can serve as an optional substitution.
Loading 1812.00155v1…