Towards Resolving the Challenge of Long-tail Distribution in UAV Images for Object Detection

1. Introduction

• UAV detection remains challenging because targets are very small in high-resolution images and spatially nonuniformly distributed.Prior work addressed these challenges by augmenting inputs with image-cropping strategies.Source: §1 · p. 1
• Class imbalance persists in cropped patches, so cropping addresses spatial nonuniformity but not the long-tail distribution.DMNet crops patches based on object density to even out spatial distribution; head-class objects still dominate the resulting patches.Source: Figure 1 · p. 1, §1 · p. 1
• Faster R-CNN AP is 51.7 for VisDrone car but 7.7 for awn., illustrating the performance gap between head and tail classes.Car, pedestrian, and person account for more than 70% of VisDrone targets, while tricycle and awn. have few samples.Source: §1 · p. 2, Table 1 · p. 2
• Natural-image long-tail methods are not directly transferable: high-resolution UAV images typically require batch sizes of 1 or 2, and image-level repeat sampling is ineffective when one image contains many head-class targets.Two-phase methods use different input distributions for representation learning and classification, and are reported effective on natural-scene datasets.Source: §1 · p. 2
• DSHNet combines Class-Biased Samplers (CBS), which separately prioritize tail- and head-class proposals, with Bilateral Box Heads (BBH), which process each group separately.At test time, each head predicts only its corresponding head or tail classes.Source: §1 · p. 2
• DSHNet substantially outperforms baseline models and generic long-tail approaches across detectors, and pairing it with image cropping yields new state-of-the-art results on VisDrone and UAVDT.The paper identifies this as a contribution across various object detectors and network backbones.Source: §1 · p. 2

2. Related work

• General detectors are categorized as anchor-free or anchor-based; anchor-based methods include two-stage and one-stage detectors.Two-stage methods generate candidate regions and refine them with a box-regression head; one-stage methods directly regress locations and boxes from anchors.Source: §2 · p. 2
• UAV cropping methods generate sub-images from spatial information and fuse local-patch and global-image detections at test time, while addressing spatial distribution rather than class imbalance.The overview describes a proposal sub-network for cropping and non-maximum suppression as one possible fusion rule.Source: §2 · p. 3
• Re-sampling over-samples tail classes or under-samples head classes, but duplication can overfit and discarding head samples can impair generalization; re-weighting can cause optimization difficulty on large-scale datasets.The cited overview presents these as two common approaches to long-tail detection.Source: §2 · p. 3
• Two-phase methods first train on the original distribution, then fine-tune the classifier on balanced data with the representation fixed.They build on findings that re-balanced inputs improve classifier learning but harm feature learning.Source: §2 · p. 3
• Applying classification-style long-tail methods to detection assumes roughly equal target counts across classes in each batch, an assumption difficult to satisfy on UAV datasets.The related-work discussion attributes this difficulty to the large gap in target counts across classes shown in Fig. 1.Source: §2 · p. 3

3.1. Long-tailed object detection in UAV images

• Balanced-batch methods face UAV-specific constraints: high-resolution images limit batch size, and head-class objects can dominate within individual images.Thus, balancing images alone may leave object classes imbalanced.Source: §3.1 · p. 3

3.2. Overview of DSHNet

• DSHNet pairs head- and tail-biased samplers with corresponding box heads while retaining the Faster R-CNN backbone and RPN.Both heads compute losses over all classes during training; at inference, each predicts its corresponding classes and their predictions are fused.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3. Class-biased samplers

• CBS resamples proposals at the object level, prioritizing tail classes in CBS(T) and head classes in CBS(H), then filling any shortfall from the other class group.The Faster R-CNN example samples 512 proposals, including 128 positives; CBS(T) prioritizes tail-class proposals among the positives.Source: §3.3 · p. 4

3.4. Bilateral box heads

• BBH processes the two biased proposal groups with separate classifiers, preserving the original input distribution for the backbone while allowing the heads to focus on their corresponding classes.The two losses are combined with a balance coefficient; both heads train on all classes, but each predicts only its corresponding classes at inference.Source: §3.4 · pp. 4–5

3.5. Plug-and-play on base models

• DSHNet can be applied to one-stage and cascaded detectors by adapting sampling and box-head arrangements to each architecture.For cascaded detectors, the method doubles the samplers and heads; inference averages class scores over three heads and uses the latest head’s box regressions.Source: §3.5 · p. 5

4.1. Datasets

• On VisDrone, the 10,209-image dataset is evaluated on its 548-image validation set because the test server is closed; UAVDT has 23,258 training and 15,069 test images.The implementation uses Faster R-CNN, RetinaNet, and Cascade R-CNN baselines; batch size is 2 on one 11GB NVIDIA 1080Ti GPU.Source: §4.1 · p. 5, §4.2 · pp. 5–6

4.3. Experimental results

• 3.0 overall AP on Cascade R-CNN (23.2 vs. 26.2), while Faster R-CNN with ResNeXt101 rises from 22.4 to 25.8 on VisDrone.On UAVDT, the tail-class bus AP on FRCNN-R50 rises from 3.4 to 14.7.Source: §4.3 · p. 6, Table 2 · p. 6, Table 3 · p. 7
• DSHNet also improves VisDrone results over MMF (22.6 vs. 24.6) and BGS (23.0 vs. 24.6); SimCal is below the base model (20.0 vs. 21.7).With DMNet-cropped patches, DSHNet boosts overall AP by 2.2 points (28.1 vs. 30.3).Source: §4.3 · p. 7

4.4. Ablation study

• Doubling random-sampler samples leaves performance on par with the base model (21.6 vs. 21.7), while using random sampling with BBH reaches only 22.1 vs. 21.7, highlighting CBS's role.These ablations use Faster R-CNN with ResNet50 on VisDrone.Source: §4.4 · p. 8, Table 4 · p. 8
• Class-exclusive sampling scores below CBS (24.1 vs. 24.6), and removing BBH drops performance to 21.7 from 24.6.The authors report that adding other classes can benefit classifier generalization and that BBH enhances classifier discriminative power.Source: §4.4 · p. 8, Table 5 · p. 8
• Restricting each BBH branch to its assigned class group improves AP by 2 (22.6 vs. 24.6) compared with letting both branches predict all classes.At test time, BBH(T) predicts tail classes and BBH(H) predicts head classes before aggregation.Source: §4.4 · p. 8
• BBH's best reported AP is 24.6 for λ between 2.0 and 4.0; performance declines above 4.0, and the experiments use λ = 2.0.The authors report that inference is only slightly slower than the base models.Source: §4.4 · p. 8, Figure 4 · p. 8, Table 6 · p. 8

• Class-Biased Samplers separately perform biased sampling on proposals for tail and head classes.Source: §5 · p. 8
• Bilateral Box Heads process tail- and head-biased proposals with two classifiers and compute their losses separately.The heads also achieve loss re-weighting by computing losses for head and tail classes respectively.Source: §5 · p. 8
• DSHNet significantly improves base models and achieves new state-of-the-art results on two UAV benchmarks.Source: §5 · pp. 8–9

Appendix

• The appendix presents these figures as additional qualitative results comparing DSHNet with the base model.Source: Appendix · p. 11
• Figure A1 compares Faster R-CNN (ResNet50), DSHNet on that base model, and ground truth on UAVDT.Ground-truth boxes are red for tail classes and green for head classes.Source: Figure A1 · p. 11, Appendix · p. 11
• Figure A2 compares Faster R-CNN (ResNet50), DSHNet on that base model, and ground truth on VisDrone.Ground-truth boxes are red for tail classes and green for head classes.Source: Appendix · p. 11, Figure A2 · p. 12