Source-linked AI summary
ReDet: A Rotation-equivariant Detector for Aerial Object Detection
Jiaming Han, Jian Ding, Nan Xue, Gui-Song Xia
TL;DR
Aerial detection must handle arbitrary object orientations, but ordinary CNNs model rotation inefficiently and require substantial augmentation. ReDet combines rotation-equivariant networks with RiRoI Align to produce rotation-invariant features, achieving state-of-the-art results across three datasets while reducing parameters by 60%.
Problem
Aerial objects have arbitrary orientations, while ordinary CNNs require redundant parameters and substantial rotation-augmented data to model orientation variation.
Method
ReDet uses rotation-equivariant networks for feature extraction and RiRoI Align to align spatial and orientation dimensions when extracting rotation-invariant RoI features.
Results
ReDet achieves state-of-the-art 80.10, 76.80 and 90.46 mAP on DOTA-v1.0, DOTA-v1.5 and HRSC2016, respectively, with 60% fewer parameters.
Takeaways & Limitations
The method combines rotation equivariance and rotation invariance for accurate aerial object detection with a better model size versus accuracy trade-off.
Abstract
from arXiv · showhide
Recently, object detection in aerial images has gained much attention in computer vision. Different from objects in natural images, aerial objects are often distributed with arbitrary orientation. Therefore, the detector requires more parameters to encode the orientation information, which are often highly redundant and inefficient. Moreover, as ordinary CNNs do not explicitly model the orientation variation, large amounts of rotation augmented data is needed to train an accurate object detector. In this paper, we propose a Rotation-equivariant Detector (ReDet) to address these issues, which explicitly encodes rotation equivariance and rotation invariance. More precisely, we incorporate rotation-equivariant networks into the detector to extract rotation-equivariant features, which can accurately predict the orientation and lead to a huge reduction of model size. Based on the rotation-equivariant features, we also present Rotation-invariant RoI Align (RiRoI Align), which adaptively extracts rotation-invariant features from equivariant features according to the orientation of RoI. Extensive experiments on several challenging aerial image datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016, show that our method can achieve state-of-the-art performance on the task of aerial object detection. Compared with previous best results, our ReDet gains 1.2, 3.5 and 2.6 mAP on DOTA-v1.0, DOTA-v1.5 and HRSC2016 respectively while reducing the number of parameters by 60\% (313 Mb vs. 121 Mb). The code is available at: \url{https://github.com/csuhan/ReDet}.
1. Introduction
Aerial objects have arbitrary orientations, making rotation handling a central challenge for oriented detection. ReDet addresses this with rotation-equivariant feature extraction and RiRoI Align, achieving strong accuracy with fewer parameters.
- Aerial objects often have arbitrary orientations, so detection commonly uses oriented bounding boxes rather than horizontal boxes.
- Regular CNN features are not rotation-equivariant, making RRoI-warped features unstable as object orientation changes.Approximate rotation invariance therefore requires larger networks and more training samples.
- ReDet combines rotation-equivariant backbones with RiRoI Align to extract rotation-invariant features in both spatial and orientation dimensions.RiRoI Align aligns orientation channels through circular switching and feature interpolation.
- 80.10, 76.80 and 90.46 mAP are achieved on DOTA-v1.0, DOTA-v1.5 and HRSC2016, respectively.Compared with previous best results, the gains are 1.2, 3.5 and 2.6 mAP, respectively.
- 60% fewer parameters are used than the baseline, reducing model size from 313 Mb to 121 Mb while improving the model size versus accuracy trade-off.
2. Related Works
Related work addresses oriented detection through OBBs, rotation-invariant features, and rotation-equivariant networks. ReDet extends these ideas to detection by combining equivariant backbones with instance-level invariant feature extraction.
- Oriented object detectors use OBBs because aerial objects can have large aspect ratios, arbitrary orientations, and dense distributions.
- Prior detectors use approaches including additional anchors, RoI transformations, quadrilateral or mask representations, feature alignment, and dynamic refinement.
- 2.2. Rotation-equivariant Networks: Rotation-equivariant networks introduce weight sharing across translations and rotations, with prior work extending equivariance to larger discrete or continuous groups.
- 2.3. Rotation-invariant Object Detection: Regular CNNs require more parameters and training samples to approximate rotation invariance, while oriented detection additionally needs instance-level rotation-invariant features.
- 2.3. Rotation-invariant Object Detection: RRoI Align cannot produce completely rotation-invariant features from regular CNNs, whereas RiRoI Align operates on equivariant features across spatial and orientation dimensions.
3. Preliminaries
The preliminaries define equivariance as predictable feature transformation under input transformation, then specialize it to translation and rotation-equivariant networks. They also explain how rotation-invariant instance features can be obtained by compensating for an RoI’s orientation.
- Equivariance means that transforming an input produces a correspondingly predictable transformation of its feature.
- When the output-space transformation is identical, equivariance becomes invariance.
- CNN translation equivariance extends to rotation equivariance through group convolutions over combined translation and rotation groups.The rotation group may be a cyclic group C_N containing discrete rotations.
- Rotation-equivariant networks use layers that preserve rotation transformations while sharing weights across translations and rotations.
- Rotation-invariant instance features remain unchanged under RoI rotations and can be obtained by applying the transformation associated with the RoI orientation.The orientation θ determines the transformation T_r used to construct the invariant representation.
4. Rotation-equivariant Detector
ReDet combines a rotation-equivariant backbone with RiRoI Align to produce rotation-invariant RoI features. RiRoI Align performs spatial and orientation alignment, including channel switching and interpolation for angles outside the discrete rotation group.
- ReDet uses rotation-equivariant backbone features, an RPN, and a RoI Transformer to generate rotated RoIs before classification and box regression.
- The rotation-equivariant backbone shares filters across orientations and enriches orientation information in feature maps.
- Ordinary RRoI warping aligns only spatial features, leaving orientation channels misaligned.
- RiRoI Align achieves rotation-invariant RoI features through spatial alignment followed by orientation alignment.
- Orientation alignment circularly switches channels according to an index and interpolates neighboring orientation features when the RoI angle is outside C_N.The interpolation handles rotations not represented by the discrete equivariant group.
- Orientation pooling after RRoI Align preserves only the strongest orientation response and discards features from other orientations.
5. Experiments and Analysis
Experiments evaluate ReDet through dataset protocols, ablations, comparisons with rotation augmentation, and state-of-the-art benchmarks. Across DOTA-v1.0, DOTA-v1.5, and HRSC2016, ReDet achieves strong detection accuracy, efficient parameterization, and improved localization.
- Datasets and settings: DOTA-v1.0 contains 2,806 images and 188,282 instances across 15 categories, while DOTA-v1.5 contains 402,089 instances and more extremely small objects.DOTA-v1.5 is described as more challenging than DOTA-v1.0.
- Datasets and settings: The experiments train on training and validation sets, test on held-out test sets, and use 1024×1024 crops, multi-scale data, and random rotation for DOTA.HRSC2016 images are resized to 800 × 512 while preserving aspect ratio.
- Ablation studies: ReResNet50+ReFPN under C8 improves detection mAP by 1.83 with only 1/8 the parameters, while also outperforming corresponding ordinary backbones in Faster R-CNN OBB and RetinaNet OBB.The reported parameter comparison is 103 Mb versus 12 Mb.
- Ablation studies: 66.86 mAP is achieved by RiRoI Align with l = 2 interpolation, improving 0.87 mAP over RRoI Align; max pooling decreases mAP and l = 4 gains only 0.33 mAP.The authors suggest that excessive interpolation may hurt equivariance and relations between orientations.
- Ablation studies: Compared with rotation augmentation, ReDet* improves mAP by 2.59 with 18% extra training time, while a 2x augmented baseline is 0.68 higher but takes twice as long.The comparison uses similar parameter amounts for ReDet* and the baseline.
- State-of-the-art comparisons: ReDet reaches 80.10 mAP on DOTA-v1.0, about 76.80 OBB mAP on DOTA-v1.5, and 90.46 mAP on HRSC2016, with improvements also reported for AP75 and small-instance categories.On DOTA-v1.0, the single-scale model reaches 76.25 mAP; on HRSC2016, VOC2012 mAP is 97.63.
6. Conclusions
ReDet combines a rotation-equivariant backbone with RiRoI Align to extract rotation-invariant features for aerial object detection, with experiments on DOTA and HRSC2016 demonstrating effectiveness.
- ReDet combines a rotation-equivariant backbone with RiRoI Align to extract rotation-invariant features from rotation-equivariant features.