Source-linked AI summary
Adaptive Rotated Convolution for Rotated Object Detection
Yifan Pu, Yiru Wang, Zhuofan Xia, Yizeng Han, Yulin Wang, Weihao Gan, Zidong Wang, Shiji Song, Gao Huang
TL;DR
Rotated object detection challenges standard backbones because orientations vary across images and within each image. The paper proposes ARC, which adaptively rotates and conditionally combines convolution kernels to address this variation. ARC improves multiple oriented detectors on DOTA and HRSC2016, while remaining efficient and achieving state-of-the-art performance with Oriented R-CNN on DOTA.
Problem
Standard backbones may be sub-optimal for rotated object detection because object orientations vary across images and within each image.
Method
ARC adaptively rotates convolution kernels according to input-dependent angles and conditionally combines multiple rotated kernels for convolution.
Results
ARC-equipped backbones improve multiple oriented object detectors on DOTA and HRSC2016, with state-of-the-art performance reported on DOTA using Oriented R-CNN.
Takeaways & Limitations
ARC is a plug-and-play module for convolutional backbones that improves oriented-object representation while remaining efficient.
Abstract
from arXiv · showhide
Rotated object detection aims to identify and locate objects in images with arbitrary orientation. In this scenario, the oriented directions of objects vary considerably across different images, while multiple orientations of objects exist within an image. This intrinsic characteristic makes it challenging for standard backbone networks to extract high-quality features of these arbitrarily orientated objects. In this paper, we present Adaptive Rotated Convolution (ARC) module to handle the aforementioned challenges. In our ARC module, the convolution kernels rotate adaptively to extract object features with varying orientations in different images, and an efficient conditional computation mechanism is introduced to accommodate the large orientation variations of objects within an image. The two designs work seamlessly in rotated object detection problem. Moreover, ARC can conveniently serve as a plug-and-play module in various vision backbones to boost their representation ability to detect oriented objects accurately. Experiments on commonly used benchmarks (DOTA and HRSC2016) demonstrate that equipped with our proposed ARC module in the backbone network, the performance of multiple popular oriented object detectors is significantly improved (\eg +3.03\% mAP on Rotated RetinaNet and +4.16\% on CFA). Combined with the highly competitive method Oriented R-CNN, the proposed approach achieves state-of-the-art performance on the DOTA dataset with 81.77\% mAP. Code is available at \url{https://github.com/LeapLabTHU/ARC}.
1. Introduction
Rotated object detection must handle objects whose orientations vary across images and coexist within the same image, exposing limitations in conventional backbones. ARC addresses this by adapting convolution kernels to input orientations and conditionally combining multiple rotated kernels, improving detectors on standard benchmarks.
- Arbitrary object orientations make accurately locating objects challenging in applications including scene text, face, and aerial image detection.
- Conventional backbone architectures may be sub-optimal because they largely ignore orientation variation across images and within individual images.
- ARC predicts input-dependent rotation angles and adaptively rotates convolution kernels to adjust their parameters for each input.
- ARC conditionally combines multiple individually rotated kernels before convolution, enabling adaptation to multiple object orientations within an image.
- ARC functions as a plug-and-play replacement for convolution layers in backbones such as ResNet.
- ARC-equipped backbones improve multiple oriented object detectors on DOTA and HRSC2016, achieving state-of-the-art performance with Oriented R-CNN on DOTA-v1.0.
2. Related work
Prior rotated detection research has emphasized detector components and object representations, while backbone feature extraction has received less attention. ARC instead adapts convolution parameters to object orientation, extending dynamic parameter adjustment to rotated-object backbones.
- Rotated detection research includes specialized detector necks, proposal networks, RoI extraction, heads, label assignment, and flexible object representations.
- Backbone feature extractor design has been comparatively underexplored despite progress in rotated detector design.
- Rotation-equivariant operations preserve orientation information but do not address orientation variation within an image and across the dataset.
- Dynamic networks adapt architecture or parameters to inputs, with sample-wise, spatial-wise, and temporal-wise categories.
- Dynamic parameter adaptation includes input-conditioned parameter adjustment, parameter generation, kernel-shape adaptation, and feature rescaling.
- ARC belongs to parameter adjusting methods and uses adaptive rotated kernel parameters to enhance backbone representation for rotated object detection.
3. Method
ARC adapts convolution kernels to input-dependent orientations and combines multiple rotated kernels efficiently, addressing objects with varying directions across and within images. The module is implemented as a plug-and-play backbone component and evaluated across rotated detectors and benchmarks.
- Kernel rotation: ARC replaces fixed-direction convolution with kernels rotated by angles predicted from each input feature map.The rotation samples weights from the kernel space using bilinear interpolation and applies the procedure across all input channels and output kernels.
- Routing function: The routing function predicts rotation angles and combination weights from the input feature map in a data-dependent manner.It uses depthwise convolution, pooling, and separate prediction branches for angles and weights.
- Conditional computation: Multiple individually rotated kernels are combined with predicted weights to represent objects with multiple orientations within one image.The module uses conditional parameterization so the weighted kernels can be applied in one convolution operation rather than separate convolutions.
- Conditional computation: The combined-kernel formulation preserves the result of separately convolving and summing feature maps while performing the heavy convolution only once.This provides the intended multi-orientation representation with improved computational efficiency.
- Backbone integration and evaluation: ARC can replace convolution layers in existing backbones, with ARC-ResNet replacing 3 × 3 convolutions in the last three ResNet stages while retaining 1 × 1 convolutions.The method is evaluated with multiple detectors on DOTA and with popular rotated detectors on HRSC2016.
4. Experiments
Experiments on DOTA and HRSC2016 evaluate ARC across detector architectures, backbones, training strategies, and ablations. ARC consistently improves oriented detection performance while preserving efficiency and achieves 81.77% mAP with multi-scale Oriented R-CNN on DOTA.
- Experiment settings: Experiments evaluate ARC on DOTA-v1.0 and HRSC2016 using multiple rotated object detectors and standard training configurations.DOTA contains 2,806 images and 188,282 instances; HRSC2016 contains 1,061 images.
- Effectiveness on various architectures: ARC improves both single-stage and two-stage detectors on DOTA, including +3.03% mAP for Rotated RetinaNet and +4.16% mAP for CFA.Other reported gains include +2.62% for R3Det, +1.36% for S2ANet, +1.60% for Rotated Faster R-CNN, and +1.54% for Oriented R-CNN.
- Effectiveness on various architectures: ARC also improves performance on HRSC2016, with gains of 1.27% mAP for Rotated RetinaNet, 2.12% for S2ANet, and 1.84% for Oriented R-CNN.These results support compatibility with multiple detector architectures across both evaluated datasets.
- Comparison with state-of-the-art methods: 81.77% mAP is achieved on DOTA with ResNet-50 under multi-scale training and testing, surpassing the compared existing methods.ARC improves Oriented R-CNN by 1.54% with ResNet-50 and 1.59% with ResNet-101 under single-scale training and testing.
- Ablation studies: +1.16% mAP results from adaptive kernel rotation on Oriented R-CNN with DOTA, while additional rotated kernels further increase detector performance.The conditional computation design addresses multiple object orientations within an image.
- Ablation studies: Increasing the number of kernels raises mAP with only a 0.002% FLOPs increase and less than a 2.7% FPS drop versus the baseline.Replacing more backbone stages also improves performance steadily, leading to replacement of the last three stages.
5. Conclusion
The paper proposes ARC for rotated object detection, combining adaptive kernel rotation with efficient conditional computation to capture objects at varying orientations. As a plug-in backbone module, ARC improves detector performance on rotated-object benchmarks while remaining efficient.
- ARC adaptively rotates convolution kernels according to object orientations in images.
- Efficient conditional computation gives the network flexibility to capture multiple oriented objects within an image.
- ARC plugs into convolutional backbones and improves various oriented object detectors on commonly used rotated object detection benchmarks while remaining efficient.