Source-linked AI summary
Gliding vertex on the horizontal bounding box for multi-oriented object detection
Yongchao Xu, Mingtao Fu, Qimeng Wang, Yukang Wang, Kai Chen, Gui-Song Xia, Xiang Bai
TL;DR
Horizontal bounding boxes do not adequately represent ubiquitous multi-oriented objects, while alternative methods face angle sensitivity, grouping or post-processing costs, and vertex confusion. The paper glides horizontal-box vertices along corresponding sides using four length ratios and adds an obliquity factor to select horizontal or oriented detections. Without bells and whistles, the resulting detector outperforms some state-of-the-art methods across aerial-image, scene-text, and fisheye-pedestrian benchmarks while remaining efficient.
Problem
Horizontal bounding boxes inadequately capture orientation and scale for multi-oriented objects, while existing alternatives face angle sensitivity, confusion, or costly post-processing.
Method
The method regresses four side-specific gliding ratios and an area-ratio obliquity factor to represent oriented objects and choose horizontal versus oriented detection.
Results
The detector outperforms some state-of-the-art methods on multiple aerial-image, scene-text, and fisheye-pedestrian benchmarks while being more efficient.
Takeaways & Limitations
The proposed representation and divide-and-conquer strategy provide a robust and fast approach for detecting multi-oriented objects across these benchmark settings.
Takeaways & Limitations
Segmentation-based alternatives can produce split or merged components requiring heavy, time-consuming post-processing.
Abstract
from arXiv · showhide
Object detection has recently experienced substantial progress. Yet, the widely adopted horizontal bounding box representation is not appropriate for ubiquitous oriented objects such as objects in aerial images and scene texts. In this paper, we propose a simple yet effective framework to detect multi-oriented objects. Instead of directly regressing the four vertices, we glide the vertex of the horizontal bounding box on each corresponding side to accurately describe a multi-oriented object. Specifically, We regress four length ratios characterizing the relative gliding offset on each corresponding side. This may facilitate the offset learning and avoid the confusion issue of sequential label points for oriented objects. To further remedy the confusion issue for nearly horizontal objects, we also introduce an obliquity factor based on area ratio between the object and its horizontal bounding box, guiding the selection of horizontal or oriented detection for each object. We add these five extra target variables to the regression head of faster R-CNN, which requires ignorable extra computation time. Extensive experimental results demonstrate that without bells and whistles, the proposed method achieves superior performances on multiple multi-oriented object detection benchmarks including object detection in aerial images, scene text detection, pedestrian detection in fisheye images.
1 INTRODUCTION
Horizontal bounding boxes inadequately represent multi-oriented objects, while existing rotated, quadrangle, grouping, and segmentation approaches retain accuracy, ambiguity, or efficiency limitations. The proposed framework glides horizontal-box vertices using four side-specific ratios and an obliquity factor, outperforming some state-of-the-art methods without bells and whistles.
- Motivation: Horizontal bounding boxes lose accurate orientation and scale information for ubiquitous multi-oriented objects such as aerial objects and scene texts.This creates problems for applications including aerial-image change detection and recognition of sequential characters in multi-oriented scene texts.
- Limitations of existing methods: Rotated-box methods are sensitive to angle errors, while quadrangle regression can suffer confusion among sequential label points.A minor angle deviation can cause a substantial IoU drop, and directly regressing vertices introduces point-order ambiguity.
- Proposed framework: The method adds five target variables to the Faster R-CNN regression head and requires ignorable extra computation time.The five variables comprise four gliding ratios and one obliquity factor.
- Limitations of existing methods: Part-grouping and segmentation alternatives rely on costly post-processing, including heuristic grouping or handling split and merged components.The grouping process is described as heuristic and time-consuming, while segmentation often requires heavy post-processing.
- Proposed framework: The proposed representation glides each horizontal-box vertex along its corresponding side and regresses four relative length ratios.Limiting each offset to its corresponding side is intended to facilitate offset learning and avoid sequential-point confusion.
- Proposed framework: An obliquity factor based on object-to-horizontal-box area ratio guides selection between horizontal and oriented detection for nearly horizontal objects.The contribution summary describes this selection scheme as addressing confusion for nearly horizontal objects.
2 RELATED WORK
Related work adapts horizontal detectors to aerial and scene-text objects using quadrangles, rotated boxes, enhanced features, or segmentation. The proposed method instead targets general multi-oriented detection with side-constrained vertex gliding and divide-and-conquer selection for nearly horizontal versus oriented objects.
- Aerial-image detection: Aerial-image detection is difficult because objects exhibit large scale variations and arbitrary orientations.DOTA baselines replace Faster R-CNN horizontal-box regression with four-vertex quadrangle regression.
- Aerial-image detection: Aerial-image methods include rotated proposals, RoI transformation, image cascades, multidimensional attention, and joint global-local context enhancement.These approaches address rotated localization, multi-scale features, or complex backgrounds through additional proposal or feature mechanisms.
- Scene-text detection: Oriented scene-text detectors mainly use regression-based rotated boxes or quadrangles, alongside segmentation-based approaches.The paper focuses on regression-based methods that predict entire texts with rotated-box or quadrangle representations.
- Positioning of the proposed method: Compared with related work, the proposed method targets general and ubiquitous multi-oriented objects using a simple framework.Its representation glides horizontal-box vertices on corresponding sides and combines this with selection for nearly horizontal and oriented objects.
3 PROPOSED METHOD
The method represents oriented objects by gliding horizontal-box vertices along corresponding sides, then uses an obliquity factor to choose horizontal or oriented detection. It adds these variables to Faster R-CNN while retaining its basic architecture.
- Motivation: Horizontal bounding boxes poorly capture orientation and scale for multi-oriented objects, while rotated boxes and quadrangles face angle sensitivity or vertex-order ambiguity.These issues affect applications including aerial-image change detection and recognition of sequential scene-text characters.
- Multi-Oriented Object Representation: All αi equal 1 for horizontal objects, while an obliquity factor r measures the object-to-horizontal-box area ratio.Nearly horizontal objects have r close to 1, whereas extremely slender oriented objects have r close to 0.
- Network and Training: The network adds five extra target variables to the Faster R-CNN head, and its regression loss combines horizontal-box, four-ratio, and obliquity-factor losses.The extra ground-truth variables are calculated from the underlying object, while λ1, λ2, and λ3 balance the regression terms.
- Detection Selection: During testing, candidates with r above threshold tr use horizontal boxes; other candidates use the oriented representation, followed by nonmaximum suppression.This divides selection between horizontal detection for nearly horizontal objects and oriented detection for other objects.
4 EXPERIMENTS
Experiments evaluate the proposed representation across aerial images, long scene text, and fisheye pedestrian detection, using standard datasets and comparisons with baseline and state-of-the-art methods. Results show accurate orientation handling, strong benchmark performance, and low runtime overhead.
- Experimental setup: The evaluation covers DOTA aerial images, RCTW-17 long text, and MW-18Mar fisheye pedestrian detection, with dataset-specific metrics and training protocols.DOTA uses mAP, RCTW-17 uses F-measure, and MW-18Mar uses miss rates and LAMR.
- Object detection in aerial images: On DOTA, the method achieves 73.39% mAP without FPN and 75.02% mAP with FPN, outperforming the cited state-of-the-art method by 5.65% mAP without FPN.With FPN, it improves the cited state-of-the-art method by 3.86% mAP.
- Ablation study: Gliding-offset regression is accurate for oriented objects, while obliquity-factor regression remains generally accurate with MAE below 5.3%.Gliding-offset precision decreases for nearly horizontal objects with r > 0.8, motivating obliquity-based selection.
- Object detection in aerial images: Qualitative comparisons show accurate detection across orientations, whereas RBox Reg. suffers from angle errors and Vertex Reg. from vertex-order confusion.The comparison rotates an input image by different angles before testing the methods.
- Object detection in aerial images: The method outperforms RBox Reg. and Vertex Reg. by 6.30% and 11.37% mAP at standard evaluation, with ignorable runtime cost.At IoU threshold 0.7, the respective improvements increase to 25.93% and 15.98%.
- Long text detection in natural scenes: On long-text benchmarks, the method improves competing results by 0.5% on MSRA-TD500 and by 5.8% single-scale or 0.9% multi-scale on RCTW-17 while being more efficient.The method also correctly detects texts of arbitrary orientations.
- Pedestrian detection in fisheye images: For fisheye pedestrian detection, qualitative results report more accurate detections and lower missing rates than all baseline methods.The benchmark uses images captured with fisheye cameras and evaluates miss rates across false positives per image.
5 CONCLUSION
The paper presents a representation and divide-and-conquer strategy for detecting multi-oriented objects. Experiments show accurate, efficient detection across aerial images, scene texts, and fisheye pedestrians, outperforming some state-of-the-art methods on multiple benchmarks.
- The proposed method combines a representation for oriented objects with a divide-and-conquer strategy for multi-oriented detection.
- The resulting detector is described as robust and fast for multi-oriented objects.
- The method accurately detects objects in aerial images, scene texts, and fisheye pedestrian images.
- Extensive experiments show performance exceeding some state-of-the-art methods on multiple benchmarks while being more efficient.