Source-linked AI summary
Dynamic Coarse-to-Fine Learning for Oriented Tiny Object Detection
Chang Xu, Jian Ding, Jinwang Wang, Wen Yang, Huai Yu, Lei Yu, Gui-Song Xia
TL;DR
Oriented tiny-object detection suffers from severe mismatch and imbalance in label assignment because extreme shapes and limited features undermine supervision. DCFL dynamically models priors, assignments, and object representations through coarse-to-fine learning, achieving state-of-the-art results on several benchmarks. Its coarse matching uses equally weighted Gaussian distributions.
Problem
Extreme geometry and limited features in oriented tiny objects cause severe mismatch and imbalance among priors, features, instances, and supervision.
Method
DCFL dynamically models the prior, label assignment, and ground-truth representation, using coarse prior matching followed by finer posterior constraints.
Results
State-of-the-art performance includes 57.66% mAP on DOTA-v2.0 OBB and 71.03% mAP on DIOR-R, with notable improvements on tiny objects.
Takeaways & Limitations
Experiments on six datasets show convincing improvements from DCFL over the baseline for oriented tiny object detection.
Takeaways & Limitations
The coarse matching weights the prior and ground-truth Gaussian distributions equally, using α = 0.5.
Abstract
from arXiv · showhide
Detecting arbitrarily oriented tiny objects poses intense challenges to existing detectors, especially for label assignment. Despite the exploration of adaptive label assignment in recent oriented object detectors, the extreme geometry shape and limited feature of oriented tiny objects still induce severe mismatch and imbalance issues. Specifically, the position prior, positive sample feature, and instance are mismatched, and the learning of extreme-shaped objects is biased and unbalanced due to little proper feature supervision. To tackle these issues, we propose a dynamic prior along with the coarse-to-fine assigner, dubbed DCFL. For one thing, we model the prior, label assignment, and object representation all in a dynamic manner to alleviate the mismatch issue. For another, we leverage the coarse prior matching and finer posterior constraint to dynamically assign labels, providing appropriate and relatively balanced supervision for diverse instances. Extensive experiments on six datasets show substantial improvements to the baseline. Notably, we obtain the state-of-the-art performance for one-stage detectors on the DOTA-v1.5, DOTA-v2.0, and DIOR-R datasets under single-scale training and testing. Codes are available at https://github.com/Chasel-Tsui/mmrotate-dcfl.
1 School of Electronic Information, Wuhan University 2 School of Computer Science, Wuhan University
The section lists affiliations with the School of Electronic Information and School of Computer Science at Wuhan University.
- The listed affiliations are the School of Electronic Information and School of Computer Science at Wuhan University.
1. Introduction
Oriented tiny object detection is hindered by extreme geometry, limited features, and label-assignment mismatch and imbalance. DCFL addresses these issues by dynamically modeling priors, representations, and assignments with coarse-to-fine supervision.
- Extreme geometry in oriented tiny objects hampers accurate label assignment.
- Static assignment cannot adapt to ground-truth shape or filter low-quality samples, leading to sub-optimal performance.
- Position priors, sampled features, and instances mismatch because fixed, uniformly located priors often deviate from tiny objects’ main bodies.
- Existing detectors produce biased and imbalanced supervision, including too few positive samples for ground truths with extreme angles and scales.
- DCFL dynamically reformulates priors, label assignment, and ground-truth representation, then progressively assigns labels from coarse to fine.
- The method introduces a dynamic Prior Capturing Block that adaptively adjusts prior locations while retaining their physical meaning.
2. Related Work
Related work covers oriented priors, adaptive label assignment, multiscale learning, and small-object enhancement. DCFL combines dynamic modeling of priors, assignments, and representations in an end-to-end one-stage detector.
- Prior for Oriented Objects: Oriented detectors evolved from preset rotated anchors toward one-stage box- and point-prior designs, but most retain fixed priors.
- Label Assignment: Adaptive assignment methods use prediction-aware matching, shape-aware sampling, or Gaussian heatmaps to improve oriented-object supervision.
- Multi-scale Learning: Feature Pyramid Networks and alternative multiscale strategies reduce or redistribute the computational cost of scale-aware detection.
- Label Assignment: Tiny objects often receive few positives because they have low anchor IoU or cover limited feature points; proposed remedies include ATSS, NWD, and RFLA.
- Context Information: Context-based methods exploit surrounding information because tiny objects lack discriminative features.
- Feature Enhancement: Super-resolution and GAN-based methods enhance small-object features through image-level or region-level processing.
- DCFL jointly addresses prior mismatch and unbalanced learning in an end-to-end one-stage detector without an auxiliary branch.
3. Method
DCFL builds a dynamic coarse-to-fine learning pipeline that updates priors, assigns labels progressively, and represents instances with finer Gaussian mixtures. It combines cross-FPN coarse matching with posterior re-ranking and DGMM constraints to obtain balanced, higher-quality positive samples.
- The detector maps feature-point priors through a detection head to classification scores and box locations, then matches priors with ground truths for supervision.Static assignment uses handcrafted matching, whereas dynamic assignment combines prior and posterior information through prediction-aware mapping.
- Dynamic learning formulation: DCFL dynamically updates priors with a learnable block and reformulates matching as a coarse-to-fine process using a finer DGMM ground-truth representation.The dynamic prior, label assignment, and ground-truth representation are jointly modeled to alleviate mismatch.
- Dynamic prior: The Prior Capturing Block updates prior locations from feature-point positions and offset sets, then fits each dynamic location with a 2-D Gaussian prior.The dynamic location serves as the Gaussian mean, while covariance is computed from the prior representation and feature-map stride.
- Coarse prior matching: Cross-FPN coarse matching expands candidates around each ground truth across nearby spatial locations and adjacent FPN layers, avoiding both single-layer rigidity and all-layer looseness.GJSD measures similarity between Gaussian priors and ground truths, while top-K ranking forms Coarse Positive Samples.
- Finer dynamic posterior matching: Posterior matching re-ranks coarse candidates by predicted classification and location quality before DGMM filtering produces finer positive samples.The possibility of becoming a true prediction combines classification confidence with rotated IoU; DGMM uses geometry and semantic centers to represent each instance.
4. Experiments
Experiments across six datasets evaluate DCFL under specified training settings, showing strong performance for oriented tiny-object detection and consistent gains over baselines. Ablations attribute these gains to the coarse-to-fine assignment design, dynamic prior, CPS construction, and PCB components.
- Main Results: 57.66% mAP establishes state-of-the-art performance on the DOTA-v2.0 OBB benchmark under single-scale training and testing.The model also reaches 51.57% mAP without bells and whistles, surpassing all tested one-stage object detectors.
- Main Results: About 10 points of improvement over the baseline can be expected for tiny oriented objects such as small vehicles, ships, and storage tanks.The reported gains cover tested datasets and object categories rather than a single benchmark only.
- Main Results: 71.03% mAP achieves state-of-the-art performance on DIOR-R, with notable improvements on representative tiny objects.The reported tiny-object categories include vehicles, bridges, and windmills.
- Main Results: Discarding the angle still produces a notable AP0.5 boost over the baseline on VisDrone, MS COCO, and DOTA-v2.0 HBB.This evaluates DCFL's versatility on generic small-object detection datasets.
- Ablation Study: The baseline RetinaNet-OBB reaches 51.70% mAP, while progressively adding posterior re-ranked MPS and DGMM improves performance.The CPS alone is too coarse to serve as the final positive-sample set; it is used with finer components in the full pipeline.
- Ablation Study: 59.15% mAP is obtained by the GJSD-based Cross-FPN-layer CPS, outperforming the compared CPS construction paradigms.Cross-FPN-layer candidates extend beyond the main layer, and GJSD's scale-invariance is identified as the main reason for its best performance.
5. Analysis
The analyses examine imbalance, visualization behavior, and efficiency, showing that DCFL improves supervision balance, fits dynamic priors to instances, and maintains high inference speed.
- Analysis setup: The analysis evaluates quality and quantity imbalance separately and visualizes predicted results alongside sampled dynamic priors.Figure 6 presents the two imbalance dimensions, while Figures 5 and 7 visualize predictions and priors.
- Imbalance analysis: DCFL compensates positive samples for outlier angles and scales while improving and balancing predicted IoU across angles and scales.RetinaNet exhibits periodic variation in positive-sample quantity and predicted IoU across object geometry.
- Visualization: DCFL dynamically generates and samples priors that better fit each instance’s main body, reducing false-negative and false-positive predictions for extreme-shaped oriented tiny objects.The visualization analysis supports dynamic modeling and mismatch alleviation.
- Speed: DCFL reaches 20.9 FPS on the DOTA-v2.0 validation set with a single RTX3090 GPU, compared with 20.8 FPS for RetinaNet.The authors also report that DCFL is lighter in parameters and GFLOPs, as summarized in Table 8.
6. Conclusion
The paper proposes DCFL for oriented tiny object detection by addressing mismatched priors and unbalanced positive samples through dynamic modeling and coarse-to-fine assignment.
- Conclusion: DCFL identifies mismatched feature priors and unbalanced positive samples as obstacles to label assignment for oriented tiny objects.The scheme reformulates the prior, label assignment, and ground-truth representation dynamically.
- Conclusion: The proposed dynamic prior alleviates mismatch, while the coarse-to-fine assigner mitigates imbalance through dynamically reformulated learning components.The paper reports convincing improvements from extensive experiments and analyses.