Source-linked AI summary
Clustered Object Detection in Aerial Images
Fan Yang, Heng Fan, Peng Chu, Erik Blasch, Haibin Ling
TL;DR
Aerial-image detection must handle tiny objects and inefficient processing caused by sparse, non-uniform layouts. ClusDet combines cluster proposal, scale estimation, and dedicated detection, reporting state-of-the-art results across three datasets while reducing computation.
Problem
Aerial-image detectors struggle with small targets and sparse, non-uniform target distributions that make high-resolution detection inefficient.
Method
ClusDet integrates CPNet cluster proposals, ScaleNet scale estimation, and dedicated DetecNet fine detection in an end-to-end framework.
Results
State-of-the-art performance is reported on VisDrone, UAVDT, and DOTA with less computation.
Takeaways & Limitations
ClusDet reduces chips for fine detection while improving detection precision and small-object handling through clustered regions and scale estimation.
Takeaways & Limitations
CPNet requires supervised cluster labels, but existing public datasets do not provide cluster ground truth.
Abstract
from arXiv · showhide
Detecting objects in aerial images is challenging for at least two reasons: (1) target objects like pedestrians are very small in pixels, making them hardly distinguished from surrounding background; and (2) targets are in general sparsely and non-uniformly distributed, making the detection very inefficient. In this paper, we address both issues inspired by observing that these targets are often clustered. In particular, we propose a Clustered Detection (ClusDet) network that unifies object clustering and detection in an end-to-end framework. The key components in ClusDet include a cluster proposal sub-network (CPNet), a scale estimation sub-network (ScaleNet), and a dedicated detection network (DetecNet). Given an input image, CPNet produces object cluster regions and ScaleNet estimates object scales for these regions. Then, each scale-normalized cluster region is fed into DetecNet for object detection. ClusDet has several advantages over previous solutions: (1) it greatly reduces the number of chips for final object detection and hence achieves high running time efficiency, (2) the cluster-based scale estimation is more accurate than previously used single-object based ones, hence effectively improves the detection for small objects, and (3) the final DetecNet is dedicated for clustered regions and implicitly models the prior context information so as to boost detection accuracy. The proposed method is tested on three popular aerial image datasets including VisDrone, UAVDT and DOTA. In all experiments, ClusDet achieves promising performance in comparison with state-of-the-art detectors. Code will be available in \url{https://github.com/fyangneil}.
1. Introduction
Aerial-image detection is challenged by small targets and sparse, non-uniform distributions. ClusDet addresses both through clustered region selection, scale estimation, and dedicated fine detection.
- Aerial images challenge detectors because targets are small relative to high-resolution images and sparsely, non-uniformly distributed.
- Uniformly partitioning images improves resolution but wastes computation on sparse or empty regions.More than 73% of grid-partitioned chips are sparse, including 23% with zero objects.
- ClusDet predicts object clusters, estimates their scales, rescales cluster chips, and applies a dedicated detector before fusing local and global detections.
- ClusDet reduces computation through CPNet, improves scale handling through ScaleNet, and uses DetecNet to model clustered-region context.
- State-of-the-art performance is reported on VisDrone, UAVDT, and DOTA with less computation.
2. Related work
Generic detectors perform less effectively on high-resolution aerial imagery because objects are small and image-wide processing is inefficient. ClusDet instead predicts clustered regions for fine detection.
- Detectors successful on natural images degrade on high-resolution aerial images, where objects occupy small relative scales.
- Prior aerial-image methods include context-driven search, contextual proposal exploration, reinforcement-learning region selection, and uniform image partitioning.
- ClusDet differs from region-search methods by predicting object-cluster regions before extracting them for fine detection.This design is presented as reducing computation compared with uniform-chip processing.
3.1. Overview
ClusDet detects aerial-image objects through cluster extraction, fine detection on cluster chips, and fusion with global-image detections.
- CPNet predicts clustered regions from image feature maps, after which iterative cluster merging reduces noisy cluster chips.
- ScaleNet estimates an appropriate object scale for cluster chips, which are then used with initial global detections for final fusion.
3.2. Cluster Region Extraction
Cluster region extraction uses CPNet to generate cluster proposals and ICM to reduce dense, overlapping proposals before fine detection.
- Cluster region extraction comprises initial cluster generation with CPNet and cluster reduction with iterative cluster merging.
- Cluster Proposal Sub-network (CPNet): CPNet predicts cluster locations and scales from high-level feature maps using regression and classification subnets.Unlike RPN, CPNet proposes larger cluster regions and therefore requires a larger receptive field.
- Cluster Proposal Sub-network (CPNet): CPNet training is supervised, but public datasets lack cluster ground truth, so the paper generates training labels with a simple strategy.
- Cluster Region Extraction: ICM cleans dense, messy CPNet proposals whose overlap and size would otherwise impose a heavy computation burden.
- Cluster Proposal Sub-network (CPNet): A cluster is defined as a rectangular aerial-image region containing at least three objects.
- Cluster Region Extraction: ICM repeatedly selects the highest-scoring cluster, merges sufficiently overlapping clusters, removes merged inputs, and stops at the preset maximum.The process uses overlap threshold τop and maximum cluster count Nmax.
3.3. Fine Detection on Cluster Chip
Fine detection uses ScaleNet to estimate cluster-object scales, applies partition and padding to normalize chips, and sends the refined regions to DetecNet.
- Fine Detection: Unlike approaches that directly resize cropped chips, ClusDet rescales chips using estimated object scales and then performs fine detection with DetecNet.DetecNet can use a state-of-the-art detector and standard backbone networks such as VGG, ResNet, or ResNeXt.
- Scale Estimation Sub-network (ScaleNet): ScaleNet treats cluster scale estimation as regression and uses fully connected networks to predict a relative scale offset for each cluster.It receives backbone feature maps, cluster bounding boxes, and initial global-image detections as inputs.
- Scale Estimation Sub-network (ScaleNet): ScaleNet projects cluster detections onto feature-map space, pools each cluster to a fixed-size map, and converts it into a feature vector through fully connected layers.The network produces one scale-regression output per cluster.
- Scale Estimation Sub-network (ScaleNet): The estimated relative offset is defined from reference and estimated object scales, with smoothly L1 regression loss aggregated over cluster boxes.The passage defines t_i = (p_i−s_i)/p_i, where s_i is the estimated scale and M is the number of cluster boxes.
- Partition and Padding (PP): Partition and padding adjust each cluster chip so object scales remain within a reasonable detector range before fine detection.If the detector-space scale is too large, the cluster is padded proportionally; otherwise, it is partitioned into two equal chips, while padded-region detections are ignored.
3.4. Final Detection with Local-Global Fusion
Final detection combines local detections from cluster chips with global detections from the whole image using standard NMS post-processing.
- Local-Global Fusion: The final aerial-image detections fuse local cluster-chip results and global whole-image results with standard non-max suppression.Global detections come from the detection subnet, and existing modern detectors can be used for global detection.
4. Experiments
Experiments evaluate ClusDet on three aerial-image datasets, using ablations and comparisons to examine accuracy, efficiency, scale estimation, and cluster selection. Results show that cluster-aware processing improves the accuracy–computation trade-off, while dataset characteristics affect the usefulness of uniform partitioning.
- Experimental setup: Experiments use VisDrone, UAVDT, and DOTA, comparing ClusDet with evenly image partitioning and representative state-of-the-art detectors.Precision is measured with AP, AP50, and AP75; efficiency uses the number of forwarded images and inference time.
- Ablation study: EIP improves overall detection and especially small-object AP on VisDrone, but increases forwarded images 6 times, from 548 to 3,288.It also decreases large-object AP because partitioning truncates large objects and produces false positives.
- Ablation study: DetecNet+CPNet processes 1,945 images versus 3,288 for FRCNN+EIP while achieving better performance on VisDrone.Cluster-aware selection preserves large-object performance by avoiding truncation and incorporates spatial distribution information.
- Ablation study: ScaleNet raises VisDrone AP to 26.7 with a ResNet50 backbone and boosts AP50 by 2–3 points across backbones.APs also increases by 1.6 points with the ResNeXt101 backbone, while processed images rise to 2,716 because some chips are partitioned.
- Ablation study: For TopN above 4, forwarded images increase while AP fluctuates around 27, indicating repetitive computation and the importance of cluster merging.TopN is empirically set to 3 on VisDrone, 2 on UAVDT, and 5 on DOTA.
- Quantitative results: On VisDrone, ClusDet outperforms Faster R-CNN and RetinaNet across backbone settings, whereas multi-scale testing can produce background false positives for EIP methods.On UAVDT, EIP decreases performance from 11.0 to 6.1, while ClusDet is superior to FRCNN+FPN and its EIP variant; on DOTA, it achieves similar performance with dramatically fewer chips and higher AP75.
5. Conclusion
The paper presents ClusDet as an end-to-end framework that unifies object clustering and detection for aerial images. Experiments report reduced detection chips, improved precision, and state-of-the-art performance across three public datasets.
- 5. Conclusion: ClusDet unifies object clustering and detection end to end, predicting clustered regions to reduce the number of chips processed for detection.The framework also uses cluster-based scale estimation for small-object detection and models prior context information to improve precision.
- 5. Conclusion: Extensive experiments show state-of-the-art performance on VisDrone, UAVDT, and DOTA with less computation.