Source-linked AI summary
UFPMP-Det: Toward Accurate and Efficient Object Detection on Drone Imagery
Yecheng Huang, Jiaxin Chen, Di Huang
TL;DR
Drone detection is challenged by many tiny instances, inefficient coarse-to-fine processing, and confusion among similar categories with variable appearances. UFPMP-Det combines foreground-region packing into a single mosaic with multi-proxy classification and BoIW-guided optimal transport, and reports new state-of-the-art results on VisDrone and UAVDT at higher speed.
Problem
Drone imagery combines a high proportion of very small instances with inefficient existing processing and low distinctiveness among similar categories.
Method
UFPMP-Det merges and packs coarse foreground regions into a mosaic for one inference, then uses multi-proxy learning with BoIW-guided optimal transport for classification.
Results
UFPMP-Det reaches new state-of-the-art performance on VisDrone and UAVDT while improving detection accuracy and efficiency.
Takeaways & Limitations
The approach provides a unified strategy for improving both accuracy and speed in drone-image object detection.
Takeaways & Limitations
Existing coarse-to-fine pipelines remain constrained by background-heavy sub-regions and multiple inferences per image.
Abstract
from arXiv · showhide
This paper proposes a novel approach to object detection on drone imagery, namely Multi-Proxy Detection Network with Unified Foreground Packing (UFPMP-Det). To deal with the numerous instances of very small scales, different from the common solution that divides the high-resolution input image into quite a number of chips with low foreground ratios to perform detection on them each, the Unified Foreground Packing (UFP) module is designed, where the sub-regions given by a coarse detector are initially merged through clustering to suppress background and the resulting ones are subsequently packed into a mosaic for a single inference, thus significantly reducing overall time cost. Furthermore, to address the more serious confusion between inter-class similarities and intra-class variations of instances, which deteriorates detection performance but is rarely discussed, the Multi-Proxy Detection Network (MP-Det) is presented to model object distributions in a fine-grained manner by employing multiple proxy learning, and the proxies are enforced to be diverse by minimizing a Bag-of-Instance-Words (BoIW) guided optimal transport loss. By such means, UFPMP-Det largely promotes both the detection accuracy and efficiency. Extensive experiments are carried out on the widely used VisDrone and UAVDT datasets, and UFPMP-Det reports new state-of-the-art scores at a much higher speed, highlighting its advantages.
Introduction
Drone imagery makes object detection difficult because it contains many very small instances, inefficient coarse-to-fine processing, and confusing category and appearance variations. UFPMP-Det addresses these challenges with unified foreground packing and multi-proxy detection.
- Drone imagery contains a much higher proportion of very small instances than general detection benchmarks such as PASCAL VOC and MS COCO.
- Limited onboard computing makes time-consuming image pyramids unsuitable, while coarse-to-fine methods retain background-heavy regions and require multiple chip inferences.
- Semantically similar categories and substantial viewpoint, altitude, and weather variation entangle inter-class similarities with intra-class differences.
- UFPMP-Det targets small instances with Unified Foreground Packing and classification confusion with a Multi-Proxy Detection Network.
Related Work
Prior detection research developed broad anchor-based and anchor-free methods and coarse-to-fine strategies for small objects, but drone imagery retains efficiency and category-distinctiveness challenges. UFPMP-Det is positioned to address both.
- Generic object detectors are commonly organized into anchor-based and anchor-free streams.
- Drone-image studies mainly use coarse-to-fine pipelines to handle the high proportion of small instances.
- Existing drone pipelines include substantial background in generated sub-regions and require multiple inferences for one image, limiting efficiency.
- Small-object methods have been evaluated on general or specialized detection tasks, whereas drone imagery combines distinctive scale distributions with semantically similar categories.
The Proposed UFPMP-Det Approach
UFPMP-Det addresses drone imagery through a two-stage pipeline: UFP creates higher-foreground-ratio mosaics, and MP-Det classifies their objects using multiple proxies and BoIW-guided optimal transport.
- UFP converts raw drone images into mosaics with higher foreground ratios before MP-Det performs inference on them.
- MP-Det uses multi-proxy learning with Bag-of-Instance-Words-guided optimal transport to model complex object distributions.
Unified Foreground Packing
Unified Foreground Packing merges coarse foreground regions, adaptively enlarges smaller clusters, and packs the adjusted regions into one mosaic for a single fine-grained inference.
- UFP merges extracted foreground sub-areas into clustered regions, adaptively enlarges small clusters, and packs them into a unified mosaic.
- Foreground Region Generation: Foreground Region Generation expands coarse boxes around their centers and greedily merges boxes when their combined area is at least the enclosing convex box area.
- Foreground Region Scale Equalization: UFP equalizes cluster scales by estimating the average region scale and enlarging regions below a fixed size such as 96 × 96.
- Packing all cluster regions into one mosaic lets fine-grained detection run once instead of independently on every region, saving time.
Multi-Proxy Detection Network
MP-Det addresses drone-image classification difficulty by representing each category with multiple proxies and using BoIW-guided distribution matching to keep those proxies diverse and data-aligned.
- MP-Det overview: MP-Det combines a Multi-Proxy Classification Head with a Bag-of-Instance-Words model to address complex object distributions.The module targets confusion between inter-class similarities and intra-class variations.
- Single-proxy limitation: The conventional head models category i with one weight vector and implicitly assumes a single class center at w_i/∥w_i∥.Its conditional probability is computed from the backbone feature x and weight vector w_i through a sigmoid classifier.
- Multi-proxy classification: Multiple proxies model a category whose instances span several centers, producing a more flexible and accurate decision boundary than a single center.The multi-proxy head assigns K, K > 1, proxies to each category and optimizes their conditional probabilities with cross-entropy or focal loss.
- BoIW-guided distribution modeling: BoIW estimates nonuniform intra-class proxy distributions from a queue-based class vocabulary and K-means cluster sizes instead of assuming uniform priors.The vocabulary is updated with minibatch positive-instance features, and cluster probabilities are sorted to align proxies with the highest-probability clusters.
- Training objectives: MP-Det training combines conventional detection, optimal-transport, and contrastive losses, while an adaptive DBSCAN procedure estimates the proxy count K.The contrastive loss increases intra-class correlations and decreases inter-class correlations.
Experimental Results and Analysis
Experiments on VisDrone and UAVDT show that UFPMP-Det improves detection accuracy while reducing the inefficiency of conventional region-based processing. Ablations further support unified foreground packing, multi-proxy learning, adaptive proxy estimation, BoIW design, and optimal-transport matching.
- Evaluation Protocol: UFPMP-Det is evaluated on VisDrone and UAVDT, with VisDrone validation scores reported because its test set is unavailable.VisDrone contains 10,209 high-resolution images; UAVDT provides separate training and test sets.
- Comparison with State-of-the-art Methods: 4.9%, 5.8% and 3.5% AP improvements over the second-best methods are achieved with ResNet-50, ResNet-101 and ResNeXt-101 on VisDrone, respectively.The ResNet-50 model also surpasses the result obtained with the deeper ResNeXt-101 backbone.
- On UFP: UFP produces fewer packed images while achieving the best accuracy among the evaluated packing methods.Its higher foreground ratio and lower proportion of small objects facilitate subsequent detection.
- On MP-Det: MP-Head improves baseline AP by 0.4%, and BoIW adds a further 0.5% on VisDrone with ResNet-101.These ablations isolate the contributions of the two MP-Det components.
- On MP-Det: Adaptive K-proxy estimation outperforms manual setting, while MP-Det obtains its highest reported score with BoIW size N = 200.The adaptive setting uses DBSCAN on instance features, and N = 200 is adopted thereafter.
- On MP-Det: With optimal transport, instance features gather near multiple proxies more uniformly and proxies from different classes become more separated.The qualitative t-SNE comparison links OT-based matching to more discriminative proxy arrangements.
- On Hyper-parameter β: β = 1.5 gives UFP its highest reported performance, while increasing β decreases the foreground ratio.β affects both the number of clustered regions and the average recall of raw images.
Conclusion
UFPMP-Det combines unified foreground packing with multi-proxy detection for drone imagery. Its experiments on two benchmarks report new state-of-the-art performance, supporting the approach’s effectiveness.
- Conclusion: UFPMP-Det addresses very small instances with single mosaics that increase foreground ratios and improve accuracy and efficiency.Its MP-Det module models complex object distributions with multiple proxies and a BoIW-guided optimal transport loss.
- Conclusion: Extensive experiments on two benchmarks report new state-of-the-art scores for UFPMP-Det.The conclusion highlights the approach’s overall effectiveness on drone imagery.