Source-linked AI summary
Learning a Rotation Invariant Detector with Rotatable Bounding Box
Lei Liu, Zongxu Pan, Bin Lei
TL;DR
Arbitrarily rotated objects are difficult to localize and separate from backgrounds because traditional bounding boxes are not robust to orientation changes. The paper introduces DRBox, which replaces them with rotatable bounding boxes and multi-angle priors that learn object orientation. On satellite-image detection of vehicles, ships, and airplanes, DRBox outperforms Faster R-CNN and SSD and outputs object orientation angles.
Problem
Traditional bounding boxes are rotation-variant, making arbitrarily oriented objects difficult to locate accurately and distinguish when orientations vary.
Method
DRBox reconstructs a convolutional detection framework with rotatable bounding boxes and multi-angle prior boxes that encode local orientation information.
Results
DRBox outperforms Faster R-CNN and SSD on satellite-image detection of vehicles, ships, and airplanes, with better detection results in the reported scenes.
Takeaways & Limitations
DRBox provides rotation-invariant detection while also outputting object orientation angles for arbitrarily oriented targets.
Abstract
from arXiv · showhide
Detection of arbitrarily rotated objects is a challenging task due to the difficulties of locating the multi-angle objects and separating them effectively from the background. The existing methods are not robust to angle varies of the objects because of the use of traditional bounding box, which is a rotation variant structure for locating rotated objects. In this article, a new detection method is proposed which applies the newly defined rotatable bounding box (RBox). The proposed detector (DRBox) can effectively handle the situation where the orientation angles of the objects are arbitrary. The training of DRBox forces the detection networks to learn the correct orientation angle of the objects, so that the rotation invariant property can be achieved. DRBox is tested to detect vehicles, ships and airplanes on satellite images, compared with Faster R-CNN and SSD, which are chosen as the benchmark of the traditional bounding box based methods. The results shows that DRBox performs much better than traditional bounding box based methods do on the given tasks, and is more robust against rotation of input image and target objects. Besides, results show that DRBox correctly outputs the orientation angles of the objects, which is very useful for locating multi-angle objects efficiently. The code and models are available at https://github.com/liulei01/DRBox.
1. Introduction
The paper motivates rotation-invariant object detection for aerial and satellite imagery, where arbitrary object orientations expose limitations of traditional bounding boxes. It introduces DRBox with rotatable boxes that preserve orientation information in detection results.
- Motivation: Traditional bounding boxes are rotation-variant structures that become a shortcoming when target-object orientations vary.The paper contrasts this limitation with its rotatable bounding box, which incorporates orientation into the data structure.
- Motivation: Arbitrary object orientations make rotation invariance especially important in aerial and satellite-image detection.Top-down viewpoints can produce arbitrary orientations, while remote-sensing satellites generate large volumes of imagery for important detection tasks.
- Related work: Existing detection research includes handcrafted-feature methods, region-based deep detectors, and box-based methods such as YOLO and SSD.The related work traces progression from DPM, selective search, and EdgeBoxes to R-CNN-family and single-shot detectors.
- Related work: Recent rotation-handling methods adjust orientation or seek rotation-insensitive features, whereas this paper makes rotation information useful for feature extraction.The proposed direction aims for rotatable detection results while maintaining rotation-invariant detector performance.
- Related work: ORN extracts the orientation of a whole image for classification, but DRBox is designed to estimate orientation locally for each detected object.DRBox associates angle estimation with multiple prior boxes so object rotations can be represented during detection.
2. Rotatable bounding box
RBox extends the bounding box with an orientation parameter, tightly matching rotated objects and addressing BBox’s localization, background, and dense-object separation problems. DRBox uses angle-aware overlap during training and ordinary IoU for suppressing inaccurate-angle predictions.
- BBox limitations: Traditional BBoxes fit rotated ships with varying sizes and aspect ratios, so their dimensions do not represent the targets’ physical size.Near 45 degrees, about 60% of the BBox can contain background pixels.
- BBox limitations: Dense targets are difficult to separate with BBoxes because the boxes include substantial background and do not tightly follow object outlines.The paper presents these effects as major disadvantages of traditional BBoxes in rotated-object detection.
- RBox design: RBox is a rectangle with an angle parameter and five parameters defining an object’s location, size, and orientation.Compared with BBox, it surrounds the target outline more tightly.
- RBox overlap criteria: For RBoxes, IoU measures overlap for evaluating box distance and is used during non-maximum suppression.The intersection of two RBoxes can be a polygon with no more than eight sides, making Boolean overlap computation more complex than for BBoxes.
- RBox overlap criteria: ArIoU incorporates the angle difference between two RBoxes and decreases monotonically as that difference changes from 0 to 90 degrees.ArIoU is used for training to enforce learning the correct object angle, while IoU is used for NMS.
3. Rotation invariant detection
DRBox extends single-shot detection with rotatable prior boxes so the network predicts object locations, sizes, and orientation angles. Multi-angle priors, angle-aware matching and regression, and pyramid processing support detection across arbitrary orientations and object scales.
- Network structure: DRBox predicts each object's location, size, and angle using predefined rotatable prior RBoxes and decodes the offsets into exact predicted boxes.Each prior RBox produces confidence scores and a five-dimensional parameter-offset vector; non-maximum suppression removes repeated predictions.
- Multi-angle priors: Multi-angle prior RBoxes rotate through a series of angles at each location, enabling angle-specific predictions while fixed aspect ratios reduce the number of priors.The convolutional structure slides priors over image locations, while their rotations provide the key distinction from traditional box-based methods.
- Training: Ground-truth RBoxes are matched to prior RBoxes when ArIoU(P, G) > 0.5, making matched priors positive samples for location and angle regression.Because ArIoU selects priors with suitable angles, the training assignments provide initial angle information for learning.
- Training: The DRBox objective extends SSD loss with RBox regression, using confidence classification and smooth L1 regression for predicted and ground-truth RBox parameters.The regression formulation includes location, size, and angle-related terms.
- Training: The angle regression term applies a tangent function to account for angular periodicity, and minimizing it trains the network to learn the correct object angle.Equations 6a, 6b, and 6c represent location, size, and angle regression, respectively.
- Input and architecture: Pyramid input rescales images into overlapping 300 × 300 sub-images, applies DRBox separately, and uses whole-image NMS to suppress duplicate detections across tiles.The architecture also has a 108 × 108 pixel receptive field, limiting detection of targets larger than that scope.
- Efficiency: DRBox reaches 70-80 fps on the reported hardware, while the pyramid strategy adds no more than 4/3 times the processing cost.On 1600 × 1600 pixel inputs, the reported processing speed is 1600 × 1600 pixels^2 per second; SSD and Faster R-CNN reach 70 fps and 20 fps, respectively, under the stated setup.
4. Experiments and results
Experiments on a GoogleEarth satellite-image dataset evaluate DRBox against BBox-based SSD and Faster R-CNN for vehicles, ships, and airplanes. DRBox achieves stronger detection performance and greater robustness to rotated inputs and objects, while also predicting object directions.
- Dataset and setup: The GoogleEarth dataset covers vehicles from Beijing, ships near Chinese rivers, wharfs, ports, and the East China Sea, and airplanes from 15 Chinese airports.Objects are annotated with RBoxes indicating location, size, and angle.
- Dataset and setup: DRBox, SSD, and Faster R-CNN use the same convolution architecture and training-data augmentation, with dataset-optimized anchors, prior boxes, and hyperparameters.SSD and Faster R-CNN serve as BBox-based benchmarks.
- Detection results: DRBox successfully detects ships in ports and open water, small vehicles near one another or hidden in building shadows, and airplanes of different sizes.For vehicles, each car is approximately 20 pixels long and 9 pixels wide.
- Detection results: DRBox predicts vehicle head directions in addition to locations, including directions that match the stated right-side traffic convention.The paper describes direction prediction as challenging even for humans.
- Detection results: DRBox outperforms SSD and Faster R-CNN on all BEP, AP, and mAP indexes listed in Table 2.The P-R curves of SSD and Faster R-CNN remain below DRBox's curve in the reported test.
- Rotation robustness: DRBox remains robust across rotation tests, with more concentrated rotated-image recall curves and approximately stable recall across object-angle scopes, while SSD is unstable.Faster R-CNN is slightly more robust than DRBox to airplane-target rotation in Table 3, although DRBox outperforms the other methods otherwise.
5. Conclusions
DRBox replaces traditional bounding boxes with rotatable bounding boxes to estimate object orientation and improve rotation-robust detection. Across satellite-image tasks, it outperforms traditional detectors and remains more robust under image rotation.
- Conclusion: DRBox replaces traditional bounding boxes with RBoxes, enabling orientation-angle estimation for arbitrarily oriented objects.The detector is designed to be rotation invariant through its ability to estimate object orientations.
- Conclusion: DRBox outperforms Faster R-CNN and SSD on satellite-image object detection.The evaluated tasks include ship, vehicle, and airplane detection.
- Future applicability: RBox training enforces learning of multi-scale local orientation information and may extend to proposal-based frameworks such as R-FCN or Faster R-CNN.The paper identifies this orientation information as potentially useful for other orientation-sensitive tasks.
- Detection examples: DRBox detects ships, vehicles, and airplanes across open water, ports, complex backgrounds, and repair scenes in the illustrated examples.The examples include correctly indicating vehicle head directions.
- Detection performance: DRBox achieves the best precision-recall performance in each ship, vehicle, and airplane detection task.Figure 4 compares precision-recall curves for DRBox, SSD, and Faster R-CNN.
- Rotation robustness: DRBox produces more concentrated precision-recall curves than SSD when input images are rotated across multiple angles.The concentration is reported as an indication of greater robustness to input-image rotation.