PaperlayerView exact revision on arXiv

Source-linked AI summary

ARS-DETR: Aspect Ratio-Sensitive Detection Transformer for Aerial Oriented Object Detection

Ying Zeng, Yushi Chen, Xue Yang, Qingyun Li, Junchi Yan

arXiv:2303.04989v3cs.CVcs.AI

TL;DR

AP50 can tolerate substantial angle deviation, motivating AP75 for high-precision oriented detection. ARS-DETR addresses this setting with aspect-ratio-sensitive angle encoding, rotated feature alignment, and dynamic training components, achieving competitive results across aerial datasets.

  • Problem

    AP50 has a large tolerance for angle deviation, making it unsuitable for accurately reflecting high-precision oriented object detection.

  • Method

    ARS-DETR combines aspect-ratio-aware AR-CSL, rotated deformable attention, aspect-ratio-sensitive matching and loss, and denoising for oriented detection.

  • Results

    ARS-DETR achieves competitive high-precision oriented detection performance across DOTA-v1.0, DIOR-R, and OHD-SJTU.

  • Takeaways & Limitations

    AP75 is advocated as a more suitable metric for high-precision oriented detection, while aspect-ratio-sensitive training components improve the detector’s reported performance.

Abstract

from arXiv · show

Existing oriented object detection methods commonly use metric AP$_{50}$ to measure the performance of the model. We argue that AP$_{50}$ is inherently unsuitable for oriented object detection due to its large tolerance in angle deviation. Therefore, we advocate using high-precision metric, e.g. AP$_{75}$, to measure the performance of models. In this paper, we propose an Aspect Ratio Sensitive Oriented Object Detector with Transformer, termed ARS-DETR, which exhibits a competitive performance in high-precision oriented object detection. Specifically, a new angle classification method, calling Aspect Ratio aware Circle Smooth Label (AR-CSL), is proposed to smooth the angle label in a more reasonable way and discard the hyperparameter that introduced by previous work (e.g. CSL). Then, a rotated deformable attention module is designed to rotate the sampling points with the corresponding angles and eliminate the misalignment between region features and sampling points. Moreover, a dynamic weight coefficient according to the aspect ratio is adopted to calculate the angle loss. Comprehensive experiments on several challenging datasets show that our method achieves competitive performance on the high-precision oriented object detection task.

I. INTRODUCTION

Aerial images contain densely arranged objects with varied scales, aspect ratios, and orientations, making oriented boxes more suitable than horizontal boxes. ARS-DETR targets high-precision oriented detection with aspect-ratio-aware angle modeling, feature alignment, matching, and loss strategies.

  • Aerial objects vary in scale, aspect ratio, orientation, and density, so horizontal boxes may include background or overlap neighboring objects.
  • Existing detectors often add angle prediction to generic detection, but their angle estimates can remain inaccurate even when AP50 is high.
  • Angle prediction is difficult because of periodic boundary discontinuity and because objects with different aspect ratios have different angle sensitivities.
  • ARS-DETR introduces AR-CSL, which smooths angle labels using SkewIoU across aspect ratios while removing the prior window-radius hyperparameter.
  • The method combines rotated deformable attention, aspect-ratio-sensitive matching and loss, and denoising to align features and dynamically adjust training.
  • Experiments on DOTA-v1.0, DIOR-R, and OHD-SJTU report competitive performance for high-precision oriented object detection.

II. RELATED WORK

Related work addresses oriented detection through rotated representations, angle classification, and Transformer architectures. The paper builds on these directions while focusing on aspect-ratio-sensitive angle encoding and feature processing.

  • Rotated detectors replace horizontal proposals or regions with oriented representations, while feature-alignment methods address mismatches in refined single-stage detection.
  • B. Angle Classification-based Oriented Object Detection: Classification-based angle prediction avoids boundary discontinuity; CSL uses 180 categories and Gaussian smoothing, while later methods add coding, weighting, or multi-grained representations.
  • B. Angle Classification-based Oriented Object Detection: AR-BCL uses aspect-ratio-based bidirectional coding for square-like objects, whereas the paper’s angle encoding is designed to be hyperparameter-free.
  • C. DETR and Its Variants: DETR provides end-to-end set prediction without hand-designed anchors or NMS, and Deformable DETR samples adaptively around reference points using multi-level features.
  • C. DETR and Its Variants: O2DETR and AO2-DETR apply DETR to oriented detection, but their angle regression does not address boundary discontinuity or embed angle information into DETR.
  • A. Oriented Object Detection: The paper analyzes angle–aspect-ratio relationships and argues that AP50 is inadequate for evaluating high-precision oriented detection.

A. Angle and Aspect Ratio

Objects with different aspect ratios respond differently to angle deviation, while AP50 tolerates substantial deviations. AP75 imposes a narrower, aspect-ratio-dependent tolerance and is argued to better reflect high-precision oriented detection.

  • SkewIoU trends split at aspect ratio k = 1.5 under AP50.For 1 ≤ k ≤ 1.5, SkewIoU remains above 0.5 across angle deviations; for k > 1.5, it decays rapidly as deviation increases.
  • Small-aspect-ratio objects are less sensitive to angle deviation, whereas large-aspect-ratio objects are more sensitive but retain substantial AP50 tolerance.
  • AP75 requires angle deviation to remain within a specific range for every aspect ratio, with a narrower range as aspect ratio increases.Predictions outside this range are not judged positive under AP75.
  • Achieving high AP75 is more difficult for rotated than horizontal detection because rotated boxes contain less redundant area and are more sensitive to errors.
  • AP75 can distinguish detector performance that appears similar under AP50.The passage gives S2A-Net versus Rotated ATSS as an example where AP50 ordering differs from AP75 ordering.
  • AP75 is presented as a balanced intermediate metric because AP50 and AP55 can mask angle errors, while AP90 and AP95 yield strongly degraded performance.AP50:95 averages across metrics, giving lower-threshold metrics substantial influence.

IV. METHOD

ARS-DETR combines a Deformable DETR architecture with angle-aware encoding and rotated attention for oriented aerial object detection. Its design addresses CSL’s fixed smoothing, angle-granularity insensitivity, and window-radius hyperparameter.

  • ARS-DETR adopts Deformable DETR and develops rotated deformable attention, denoising training, aspect-ratio-sensitive matching, and loss for oriented detection.The framework extracts multi-scale features, uses encoder-decoder attention, and selects Top-K proposals as object queries.
  • CSL converts angle prediction into circular classification by dividing angles into 180 categories and smoothing labels with a Gaussian window.The first and last angle categories are treated as adjacent to avoid boundary discontinuity.
  • CSL uses a fixed-radius Gaussian label function without considering object aspect ratio, although SkewIoU varies substantially across adjacent angles for different aspect ratios.This makes a single rigid smoothing pattern unsuitable for all objects.
  • CSL produces consistent smoothing outcomes under different angle discrete granularities, although adjacent-angle correlation should weaken as granularity increases.The paper identifies this insensitivity as unreasonable for angle classification.
  • The CSL window radius is a hyperparameter whose best value becomes difficult to determine when angle granularity changes.This adds a practical tuning burden to classification-based oriented detection.

2) Design of Aspect Ratio Aware Circle Smooth Label:

AR-CSL replaces CSL’s fixed Gaussian smoothing with SkewIoU-based, aspect-ratio-aware label construction. The resulting labels vary with object shape and angle granularity without introducing window-radius hyperparameters.

  • Design of Aspect Ratio Aware Circle Smooth Label: AR-CSL uses SkewIoU under each angle deviation to generate angle-category labels that reflect aspect-ratio-dependent correlations.The method calculates bounding-box SkewIoU, normalizes the values, and assigns them to the current angle-category bin.
  • Design of Aspect Ratio Aware Circle Smooth Label: AR-CSL dynamically calculates smoothing values from object aspect ratios instead of applying one fixed smoothing function.The comparison presents this behavior for objects with different aspect ratios.
  • Design of Aspect Ratio Aware Circle Smooth Label: AR-CSL varies adjacent-category smoothing values when the angle discrete granularity changes because it accounts for the corresponding angle deviation.This addresses CSL’s granularity insensitivity.
  • Design of Aspect Ratio Aware Circle Smooth Label: AR-CSL introduces no hyperparameters because its smoothing values are determined by SkewIoU and angle deviation.The paper presents this as making the method more convenient to use.

C. Rotated Deformable Attention Module

The Rotated Deformable Attention module embeds angle information into DETR and rotates sampling points to align extracted features with oriented objects.

  • The simple DETR-based oriented detector predicts angles but does not embed angle information, causing feature misalignment.
  • Given a query feature and rotated reference box, RDA predicts dynamic sampling offsets constrained within the reference box.
  • RDA updates angle information after each layer and uses it to align sampling points with features.
  • RDA rotates sampling points according to the reference-box angle, correcting the mismatch caused by horizontal reference boxes.
  • Compared with deformable offsets or fixed rotated offsets, RDA combines learnable offsets with rotation to improve sampling alignment.

D. Denoising Training

Denoising Training addresses unstable DETR matching by reconstructing noisy ground-truth labels, boxes, and angles as an auxiliary task with fixed assignments.

  • Bipartite matching instability can slow DETR convergence and hinder performance.
  • Denoising Training generates noisy targets with fixed assignments, helping mitigate matching instability and accelerate convergence.
  • Positive and negative noisy labels are created by preserving ground-truth classes for positives and assigning background to negatives.
  • Noisy boxes are produced by randomly moving ground-truth boundaries, with larger noise scales assigned to negative samples.
  • The decoder denoises noisy box proposals and reconstructs the ground-truth boxes.
  • Noisy angles shift ground-truth angles with uniformly sampled perturbations, using a periodic function to keep them within range.

E. Aspect Ratio Sensitive Matching and Loss

Aspect Ratio Sensitive Matching and Loss dynamically increase the influence of angle accuracy for objects whose elongated shapes make angle deviations more consequential.

  • DETR matches predictions to oriented-object targets by minimizing a cost over permutations of the prediction set.
  • The matching cost incorporates class, angle, and horizontal-box terms, using focal, L1, GIoU, and cross-entropy losses.
  • ARM introduces a dynamic angle-loss coefficient because objects with larger aspect ratios are more sensitive to angle deviations.
  • With ARM, large-aspect-ratio ground truths favor predictions with more similar angles during matching.
  • ARL applies the same aspect-ratio-sensitive principle to dynamically adjust the angle loss during training.

V. EXPERIMENTS

The experiments evaluate ARS-DETR on multiple aerial oriented-object detection datasets using standardized training settings and comparisons of angle-labeling strategies.

  • DOTA-v1.0 contains 2,806 aerial images, 188,282 instances, and 15 oriented-object categories.
  • The experiments compare AR-CSL with CSL under different smoothing radii and angle-discretization granularities on DOTA-v1.0.
  • DIOR-R contains 23,463 images and 192,518 oriented-object instances across 20 categories.
  • OHD-SJTU provides small- and large-scale datasets, with OHD-SJTU-L containing six categories and 113,435 instances.
  • The models use PyTorch with MMRotate, AdamW, a 10^-4 initial learning rate, and dataset-specific 3x or 9x schedules.
  • A detector comparison evaluates the use of CSL and AR-CSL on DOTA-v1.0.

B. Ablation Studies

The ablations show that AR-CSL improves high-precision angle classification without CSL’s radius tuning, generalizes across detectors, and achieves the best AP75 among compared angle-classification methods.

  • Studies on AR-CSL: AR-CSL reaches about 45.71% AP75 without tuning hyperparameters, while CSL’s radius changes performance by up to 1.25% AP75.On Deformable DETR with ω=1, CSL ranges from 42.82% to 44.07% AP75 depending on R.
  • Studies on AR-CSL: 67.98% AP50 and 39.18% AP75 are obtained with RetinaNet, while FCOS reaches 71.60% AP50 and 39.74% AP75 using AR-CSL.AR-CSL also performs well on AP75 when transferred across detector architectures.
  • Studies on AR-CSL: AR-CSL achieves 45.71% AP75, exceeding the compared angle-classification methods on Deformable DETR.The comparison includes regression, CSL, POE, and AR-BCL; AR-BCL improves AP75 by 0.6% over CSL.

2) Studies on ARS-DETR:

The ARS-DETR component studies attribute gains to direct angle prediction, rotated feature alignment, aspect-ratio-sensitive matching and loss, and denoising training.

  • Angle prediction types and ways: Direct angle prediction is best for both regression and classification among the four direct-versus-residual prediction combinations.The study attributes residual prediction’s weaker optimization to angle periodicity.
  • Rotated Deformable Attention: RDA improves AP75 by 1.02% and 0.49%, reaching 48.06% and 48.62% from 47.04% and 48.13% in the two ablations.After RDA, sampling points better align with objects and high-attention points cover more object parts.
  • Aspect Ratio sensitive Matching and Loss: Using ARM and ARL independently improves AP75 by about 0.59% and 0.54%, while combining them yields a 1.09% gain.The results support aspect-ratio-sensitive matching and loss as complementary components.
  • Denoising training: Denoising training improves AP50 by 0.76% and AP75 by 1.33%, reaching 73.14% and 47.04%.Adding angle noise further raises performance to 74.16% AP50 and 49.41% AP75 when γ=0.05.

C. Comparison with state-of-the-art methods

Across aerial benchmarks, ARS-DETR is especially competitive under AP75, supporting stricter evaluation than AP50 for oriented detection.

  • DOTA-v1.0: 49.41% AP75 with ResNet50 and 51.77% AP75 with Swin-T are achieved on DOTA-v1.0.The corresponding AP50 results are 74.16% and 75.47%.
  • DOTA-v1.0: ARS-DETR surpasses RoI Trans by 0.55%, Oriented Reppoints by 2.85%, CFA by 2.86%, and GWD by 4.2% on DOTA-v1.0 AP75.With ResNet50, ARS-DETR is weaker than several advanced detectors on AP50 but leads on AP75.
  • Evaluation metric: AP50 can rank detectors differently from AP75, so the paper advocates AP75 as a more stringent measure of high-precision oriented detection.Examples include S2A-Net at 75.29% AP50 and 40.08% AP75, versus PSC at 72.87% AP50 and 46.18% AP75.
  • DIOR-R: 66.12% AP50 and 45.81% AP75 are obtained on DIOR-R with an R-50 backbone.The results are reported using a 3x training schedule.
  • OHD-SJTU: ARS-DETR achieves 46.08% AP75 on OHD-SJTU-L and 80.67% AP75 on OHD-SJTU-S without additional bells and whistles.The paper reports that these results surpass other advanced oriented object detectors.
Loading 2303.04989v3…