PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Coarse-to-Fine Learning for Oriented Tiny Object Detection

Chang Xu, Jian Ding, Jinwang Wang, Wen Yang, Huai Yu, Lei Yu, Gui-Song Xia

arXiv:2304.08876v1cs.CV

TL;DR

Oriented tiny-object detectors face mismatch and imbalance in label assignment. DCFL dynamically models priors, label assignment, and object representations using a coarse-to-fine assigner; experiments report improvements, including state-of-the-art 57.66% mAP on DOTA-v2.0 OBB.

  • Problem

    Oriented tiny objects present label-assignment challenges because priors, features, and instances mismatch, while positive-sample learning is imbalanced.

    Source: §1 · pp. 1–2

  • Method

    DCFL dynamically models priors, label assignment, and object representations, progressively assigning labels through a coarse-to-fine scheme.

    Source: §1 · p. 2

  • Results

    Experiments show convincing improvements; DCFL achieves state-of-the-art 57.66% mAP on the DOTA-v2.0 OBB benchmark.

    Source: §4.3 · p. 5, §6 · p. 8

  • Takeaways & Limitations

    Experiments support DCFL as a promising approach for oriented tiny-object detection across six datasets.

    Source: §1 · p. 2, §6 · p. 8

    The similarity measure assumes equal contributions from the prior and ground-truth distributions, with α set to 0.5.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Detecting arbitrarily oriented tiny objects poses intense challenges to existing detectors, especially for label assignment. Despite the exploration of adaptive label assignment in recent oriented object detectors, the extreme geometry shape and limited feature of oriented tiny objects still induce severe mismatch and imbalance issues. Specifically, the position prior, positive sample feature, and instance are mismatched, and the learning of extreme-shaped objects is biased and unbalanced due to little proper feature supervision. To tackle these issues, we propose a dynamic prior along with the coarse-to-fine assigner, dubbed DCFL. For one thing, we model the prior, label assignment, and object representation all in a dynamic manner to alleviate the mismatch issue. For another, we leverage the coarse prior matching and finer posterior constraint to dynamically assign labels, providing appropriate and relatively balanced supervision for diverse instances. Extensive experiments on six datasets show substantial improvements to the baseline. Notably, we obtain the state-of-the-art performance for one-stage detectors on the DOTA-v1.5, DOTA-v2.0, and DIOR-R datasets under single-scale training and testing. Codes are available at https://github.com/Chasel-Tsui/mmrotate-dcfl.

1. Introduction

Oriented tiny objects in aerial images are difficult to detect because their extreme shapes and sizes expose weaknesses in label assignment. DCFL addresses the resulting prior-feature-instance mismatch and imbalanced supervision by dynamically updating priors, labels, and object representations in a coarse-to-fine pipeline.

1. Introduction

  • Oriented bounding boxes reduce background regions, making them useful for arbitrarily oriented aerial objects, but aerial images also contain many tiny objects.The paper identifies accurate label assignment as especially challenging for tiny objects with extreme geometries.Source: §1 · p. 1
  • Static assignment rules cannot adapt positive and negative samples to ground-truth shape or filter low-quality samples, often yielding suboptimal performance.Earlier oriented detectors preset angled anchors or refine anchors before applying static rules such as MaxIoU.Source: §1 · pp. 1–2
  • Fixed priors and feature locations can miss tiny objects’ main bodies, while extreme shapes and orientations lead to mismatched priors, features, and instances.The authors describe a lack of positive samples for ground truths far from predefined anchor shapes, angles, or scales, and limited high-quality feature points for tiny objects.Source: §1 · p. 2
  • DCFL’s Prior Capturing Block adaptively adjusts prior locations, then GJSD selects cross-FPN-layer coarse positive candidates before posterior-based reranking and DGMM filtering.GJSD expands candidates to nearby spatial locations and adjacent FPN layers; the full design is incorporated into an end-to-end one-stage detector without additional branches.Source: §1 · p. 2
  • DCFL dynamically models the prior, label assignment, and ground-truth representation, using coarse-to-fine assignment to provide more balanced supervision for diverse instances.The authors report experiments on six datasets, without specifying results in these passages.Source: §1 · p. 2

2. Related Work

Prior work addresses oriented detection through prior design and adaptive assignment, while tiny-object methods target scale, sparse positives, context, and feature quality; DCFL dynamically models the prior, assignment, and ground-truth representation together.

2.1. Oriented Object Detection

  • Oriented detectors evolved from rotated-anchor tiling to horizontal-anchor proposals and oriented RPNs; one-stage methods use box or point priors, mostly fixed except S2A-Net [17].Rotated RPN [36] tiles 54 anchors per location; RoI Transformer [10] transforms horizontal proposals, while Oriented R-CNN [56] predicts oriented proposals from horizontal anchors.Source: §2.1 · pp. 2–3
  • Oriented assignment methods dynamically reweight anchors with DAL [38], adapt sample selection with SASM and GGHL, or assess point quality with Oriented Reppoints [26].DAL responds to inconsistency between input-prior IoU and predicted IoU; GGHL fits an instance’s main body with a 2-D Gaussian heatmap.Source: §2.1 · p. 3

2.2. Tiny Object Detection

  • Tiny-object scale methods reduce image-pyramid cost with FPN, use receptive-field branches in TridentNet [27], or normalize object scales through SNIP [46] and SNIPER [47].A vanilla multiresolution image pyramid incurs substantial computation.Source: §2.2 · p. 3
  • Tiny objects’ low anchor IoU or limited feature-point coverage leaves few positives; ATSS [68], NWD [57], and RFLA [58] address sample quantity or scale balance.NWD replaces IoU to sample more positives, while RFLA uses outliers for scale-balanced learning.Source: §2.2 · p. 3
  • Context methods exploit local and global cues with MRCNN and ION [3], or reason across instances using Relation Network [22] and transformer detectors [4].These methods address the limited discriminative features of tiny objects by leveraging surrounding context.Source: §2.2 · p. 3
  • Feature-enhancement approaches use GANs or super-resolution, including PGAN [25], MT-GAN, and other super-resolution methods.MT-GAN trains an image-level super-resolution model to improve small-object RoI features.Source: §2.2 · p. 3
  • DCFL jointly models the prior, label assignment, and ground-truth representation, embedding its dynamic prior in an end-to-end one-stage detector without an auxiliary branch.This contrasts with RoI-Transformer [10] and S2A-Net [17].Source: §2.2 · p. 3

3. Method

DCFL dynamically models priors, label assignment, and ground-truth representation to address mismatch, using coarse prior matching followed by finer posterior constraints to assign labels.

3.1. Dynamic Prior

  • A Prior Capturing Block uses dilated and deformable convolutions to update priors, while regression-branch offsets guide classification feature extraction for better task alignment.Each feature point initializes a prior location; the updated location parameterizes a Gaussian prior.Source: §3.1 · p. 4

3.2. Coarse Prior Matching

  • Cross-FPN-layer candidates extend to nearby spatial locations and adjacent levels; GJSD ranks priors, and the top K per ground-truth object become coarse positives.The two Gaussian distributions contribute equally to GJSD (α = 0.5); this candidate range avoids single-layer restrictions while narrowing the all-layer search.Source: §3.2 · p. 4

3.3. Finer Dynamic Posterior Matching

  • Posterior matching re-ranks coarse candidates by predicted classification confidence and rotated IoU, then uses a geometry-and-semantic-center DGMM to filter distant samples.The top Q candidates by possibility of becoming true predictions form medium positives; the DGMM’s geometry center is the object center, and its semantic center averages medium-positive sample locations.Source: §3.3 · pp. 4–5

4. Experiments

Across six datasets, DCFL achieves strong oriented-detection results and shows gains on generic small-object benchmarks. Ablations support its coarse-to-fine assignment, dynamic priors, and parameter choices.

4.1. Datasets

  • Experiments cover DOTAv1.0, DOTAv1.5, DOTA-v2.0, DIOR-R, VisDrone, and MS COCO.Ablations use DOTA-v2.0 train/val; benchmark comparisons train on the datasets’ specified train or trainval splits and evaluate on test or val sets.Source: §4.1 · p. 5

4.2. Implementation Details

  • Experiments use an ImageNet-pretrained ResNet-50 with FPN by default, SGD, Focal classification loss, IoU regression loss, and random flipping.Training uses one RTX 3090 GPU, batch size 4, learning rate 0.005, momentum 0.9, and weight decay 0.0001. DOTA-v1.0 and DOTA-v2.0 use 1024 × 1024 patches with 200 overlap for 12 epochs; settings vary for other datasets.Source: §4.2 · p. 5

4.3. Main Results

  • 57.66% mAP achieves state-of-the-art on DOTA-v2.0 OBB, while 51.57% mAP surpasses tested one-stage detectors without bells and whistles.These results are reported under single-scale training and testing.Source: §4.3 · pp. 5–6, Table 1 · p. 6
  • 71.03% mAP achieves state-of-the-art on DIOR-R; DCFL also improves tiny-object detection and yields a notable AP0.5 boost on HBB datasets.The reported tiny-object examples include vehicles, ships, and storage tanks; the HBB evaluation discards angle on VisDrone, MS COCO, and DOTA-v2.0 HBB.Source: §4.3 · p. 6, Table 4 · p. 6, Table 5 · p. 6

4.4. Ablation Study

  • RetinaNet-OBB starts at 51.70% mAP, and progressively adding posterior-re-ranked MPS and DGMM on the CPS improves results; CPS alone is too coarse for final positives.Ablations report mAP at IoU 0.5 and use one prior per feature point for fair comparisons.Source: §4.4 · p. 7, Table 7 · p. 7
  • 59.15% mAP is the best CPS construction result, achieved by GJSD among the tested distribution distances.KLD, GWD, and GJSD enable cross-FPN-layer candidates extending to adjacent layers; the compared single-layer and all-layer approaches do not yield the best performance.Source: §4.4 · p. 7
  • Fixing sample locations slightly reduces performance, while DCN adds 0.34 mAP points and dilated convolution slightly enhances mAP in PCB.Guiding classification-head offsets with regression-head offsets is used to improve branch alignment after separate DCN application slightly deteriorates accuracy.Source: §4.4 · p. 7
  • K = 16 and Q = 12 perform best, and the DGMM threshold g = 0.8 yields the highest mAP when wi,1 is 0.7.Performance changes only marginally as CPS/MPS/FPS become coarser and stricter, supporting robustness to parameter selection.Source: §4.4 · p. 7

5. Analysis

The analysis finds that DCFL balances positive-sample quantity and predicted-IoU quality across object angles and scales, improves predictions for extreme-shaped tiny objects, and maintains 20.9 FPS on DOTA-v2.0 validation.

Visualization

  • DCFL eliminates false-negative and false-positive predictions, especially for extreme-shaped oriented tiny objects, and dynamically samples priors that better fit each instance’s main body.The visualizations support the paper’s claims of dynamic modeling and mismatch alleviation.Source: Figure 5 · p. 8, Figure 7 · p. 8, §5 · p. 8

Reconciliation of imbalance problems

  • DCFL compensates positive samples for outlier angles and scales and balances predicted-IoU quality across angles and scales, unlike RetinaNet’s periodic quantity and scale-related quality imbalances.These results are from the models’ last training epoch.Source: Figure 6 · p. 8, §5 · p. 8

Speed

  • DCFL runs at 20.9 FPS on DOTA-v2.0 validation, compared with R3Det at 16.2, S2A-Net at 18.9, and RetinaNet at 20.8 FPS.The authors also report that DCFL is lighter in parameters and GFLOPs; Table 8 compares these quantities with 1024×1024 input, but no values are supplied here.Source: §5 · p. 8, Table 8 · p. 8

6. Conclusion

DCFL addresses mismatched feature priors and imbalanced positive samples in oriented tiny-object label assignment. It combines a dynamic prior with a coarse-to-fine assigner, and experiments and analyses show improvements.

6. Conclusion

  • DCFL identifies mismatched feature priors and unbalanced positive samples as obstacles to label assignment for oriented tiny objects.Source: §6 · p. 8
  • Its dynamic prior alleviates mismatch, while its coarse-to-fine assigner mitigates imbalance by reformulating the prior, label assignment, and ground-truth representation dynamically.Source: §6 · p. 8
  • Experiments and analyses show convincing improvements from DCFL.Source: §6 · p. 8

Related papers

Loading 2304.08876v1…