PaperlayerView exact revision on arXiv

Source-linked AI summary

Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark

Chang Xu, Jinwang Wang, Wen Yang, Huai Yu, Lei Yu, Gui-Song Xia

arXiv:2206.13996v1cs.CV

TL;DR

Tiny aerial objects are difficult to detect because they contain few pixels and IoU-based assignment is highly sensitive to location deviations. The paper introduces NWD-RKA and relabels AI-TOD as AI-TOD-v2; across four datasets, the method improves detection and reaches state-of-the-art performance on AI-TOD-v2.

  • Problem

    Tiny aerial objects provide limited appearance information, while IoU-based label assignment is sensitive to location deviations and can produce imbalanced training samples.

  • Method

    The paper models bounding boxes as 2-D Gaussian distributions, measures their similarity with Normalized Wasserstein Distance, ranks anchors with RKA, and releases meticulously relabeled AI-TOD-v2.

  • Results

    NWD-RKA consistently improves tiny-object detection across four datasets and achieves state-of-the-art performance on AI-TOD-v2.

  • Takeaways & Limitations

    NWD-RKA provides a detector-agnostic replacement for IoU threshold assignment, while AI-TOD-v2 offers a more reliable tiny-object training and validation benchmark.

  • Takeaways & Limitations

    The normalization constant C is experimentally robust only within a certain range.

Abstract

from arXiv · show

Tiny object detection (TOD) in aerial images is challenging since a tiny object only contains a few pixels. State-of-the-art object detectors do not provide satisfactory results on tiny objects due to the lack of supervision from discriminative features. Our key observation is that the Intersection over Union (IoU) metric and its extensions are very sensitive to the location deviation of the tiny objects, which drastically deteriorates the quality of label assignment when used in anchor-based detectors. To tackle this problem, we propose a new evaluation metric dubbed Normalized Wasserstein Distance (NWD) and a new RanKing-based Assigning (RKA) strategy for tiny object detection. The proposed NWD-RKA strategy can be easily embedded into all kinds of anchor-based detectors to replace the standard IoU threshold-based one, significantly improving label assignment and providing sufficient supervision information for network training. Tested on four datasets, NWD-RKA can consistently improve tiny object detection performance by a large margin. Besides, observing prominent noisy labels in the Tiny Object Detection in Aerial Images (AI-TOD) dataset, we are motivated to meticulously relabel it and release AI-TOD-v2 and its corresponding benchmark. In AI-TOD-v2, the missing annotation and location error problems are considerably mitigated, facilitating more reliable training and validation processes. Embedding NWD-RKA into DetectoRS, the detection performance achieves 4.3 AP points improvement over state-of-the-art competitors on AI-TOD-v2. Datasets, codes, and more visualizations are available at: https://chasel-tsui.github.io/AI-TOD-v2/

1. Introduction

Tiny aerial objects provide little appearance information, while IoU-based assignment is highly sensitive to location and scale, producing poor supervision. The paper addresses these issues with NWD-RKA and improved dataset annotations, reporting gains across four datasets.

  • Tiny aerial objects contain extremely limited appearance information, making discriminative feature learning difficult and causing many detection failures.
  • IoU-based assignment is unsuitable for tiny-object positional evaluation because minor deviations can flip labels and create severe positive/negative and scale imbalances.IoU can remain constant for non-overlapping or mutually inclusive boxes, while larger objects receive more positive samples than very tiny ones.
  • NWD models bounding boxes as 2-D Gaussian distributions and uses normalized Wasserstein distance to measure similarity even when boxes barely overlap.Its smoother response to location deviation and scale balance can provide more positive samples for tiny objects.
  • RKA ranks anchor candidates by NWD scores rather than assigning labels solely through IoU thresholds, ensuring each instance receives sufficient positive training samples.
  • More than 50,000 missed instances were added to AI-TOD-v2, which contains 28,036 images and 752,754 instances with a mean absolute object size of 12.7 pixels.The dataset also includes eight categories and a benchmark with additional baseline detectors.
  • NWD-RKA applies to anchor-based detectors and achieves 24.7 AP and 57.2 AP0.5 on AI-TOD-v2, outperforming state-of-the-art competitors by a large margin.Improvements are also reported on AI-TOD, VisDrone2019, and DOTA-v2.0.

2. Related Work

Prior tiny-object detection work spans feature learning, context, augmentation, training strategies, super-resolution, and detector adaptation. This paper instead focuses on replacing IoU-based assignment with a similarity metric and customized assignment strategy, alongside a tiny-object benchmark.

  • Public aerial datasets such as DIOR, DOTA, xView, and VisDrone generally contain objects larger than those targeted by AI-TOD-v2.
  • AI-TOD-v2 is dedicated to tiny aerial-object detection, with a mean absolute object size of 12.7 pixels and about 86% of instances smaller than 16 pixels.
  • Existing tiny-object strategies include multi-scale feature learning, context-based detection, data augmentation, training strategies, and label assignment methods.
  • Data augmentation improves tiny-object detection by increasing training data or tiny-object representation through operations such as oversampling, copy-pasting, resizing, and rotation.
  • Super-resolution and detector-adaptation methods enhance image resolution, feature representation, contextual information, or background suppression for small-object detection.
  • The proposed method differs by designing a similarity metric and customized label assignment strategy to replace IoU-based assignment in object detectors.

3. Dataset

AI-TOD-v2 relabels AI-TOD to reduce annotation noise while preserving its images, classes, and horizontal-box representation. The dataset and benchmark emphasize extremely small objects and provide statistical and similarity analyses for evaluation.

  • Dataset Optimization: AI-TOD-v2 retains AI-TOD’s 28,036 images, eight classes, and horizontal bounding-box representation while revising its annotations.The relabeling process adjusts existing annotations and addresses missed or inaccurate object locations.
  • Dataset Optimization: The relabeling workflow combines detector visualizations, expert review, volunteer annotation, team decisions, voting, and double-blind quality checks.Uncertain images and annotations receive additional review before precise locations and categories are finalized.
  • Dataset Statistics: AI-TOD-v2 contains 752,745 annotated instances, 52,133 more than AI-TOD, indicating substantial additions during precise relabeling.The dataset reports object counts across image sets and categories in Table 1.
  • Dataset Statistics: The dataset is highly imbalanced across eight categories, with vehicles far more numerous than wind-mills, reflecting real-world class distributions.The categories include airplane, bridge, storage-tank, ship, swimming-pool, vehicle, person, and wind-mill.
  • Dataset Characteristics: AI-TOD-v2 has a mean absolute object size of 12.7 pixels, with 73.4% of objects classified as tiny and 12.4% as very tiny.Most categories fall within the very-tiny and tiny size ranges; the dataset is characterized as having the smallest object size in the earth observation community.
  • Annotation Evaluation: Similarity analyses compare NWD-RKA predictions with AI-TOD-v2 annotations, finding about 33.0 AP overall and below 15.0 APvt for very tiny objects.The reported differences indicate that relabeling produced annotations distinct from predictions generated from AI-TOD.

4. Methodology

The methodology models tiny bounding boxes as 2-D Gaussian distributions, compares them with a normalized Wasserstein metric, and ranks anchors to improve assignment in anchor-based detectors.

  • Normalized Gaussian Wasserstein Distance: NWD models each horizontal bounding box as a 2-D Gaussian distribution whose mean is the box center and whose spread reflects its dimensions.The box center supplies the Gaussian mean, while the inscribed ellipse determines the Gaussian scales.
  • Normalized Gaussian Wasserstein Distance: Wasserstein distance converts similarity between bounding boxes into a distance between their corresponding Gaussian distributions.The distance is defined for two 2-D Gaussian distributions using their means and covariance matrices.
  • Normalized Gaussian Wasserstein Distance: NWD maps Gaussian Wasserstein distance through an exponential transformation to produce a smooth similarity value in (0, 1] comparable to IoU.The constant C is experimentally robust within a certain range.
  • Ranking-based Assignment: RKA ranks anchors by NWD score for each ground-truth box, assigning positive labels to the Top k anchors and negative labels to the remainder.This replaces threshold-based label assignment with per-instance ranking.
  • NWD-RKA: NWD-RKA combines NWD with RKA as a replacement for standard IoU-based assignment and can be integrated into anchor-based detectors without inference-stage cost.The schematic distinguishes IoU-threshold, IoU-RKA, and NWD-RKA assignment strategies.

5. Analysis

The analysis identifies metric and assignment imbalances caused by IoU for tiny objects, then shows how NWD-RKA produces smoother, more scale-balanced similarities and more positive supervision.

  • Analysis of Metrics: NWD offers smoothness to location deviation, scale balance, and similarity measurement for non-overlapping or mutually inclusive bounding boxes.These properties distinguish NWD from IoU for tiny-object comparisons.
  • Analysis of NWD-RKA: 709,060 positive samples were assigned in one epoch with NWD-RKA, 490,240 more than the baseline.The increased positive samples provide supervision for classification and regression heads.
  • Analysis of NWD-RKA: The method addresses positive/negative and scale imbalance by supplementing positive samples and balancing assignment across tiny-object scales.The analysis connects these assignment changes to improved supervision for tiny-object detection.
  • Analysis of NWD-RKA: NWD-RKA reconciles the number of positive samples across object scales, whereas very-tiny objects under Faster R-CNN almost receive none.This supports more balanced optimization across instances of different scales.

6. Experiments

Experiments evaluate NWD-RKA across detectors, assignment metrics, anchor settings, hyperparameters, and aerial datasets. The method consistently improves tiny-object detection, with DetectoRS reaching 24.7 AP on AI-TOD-v2.

  • Benchmark: 24.7 AP is achieved by DetectoRS with NWD-RKA on the AI-TOD-v2 test set, establishing the reported state-of-the-art result.The benchmark includes more than 15 baseline detectors and six proposed detectors.
  • Improvements over baselines: NWD-based detectors improve AP by 1.6, 8.6, 7.1, and 8.6 points for RetinaNet, Faster R-CNN, Cascade R-CNN, and DetectoRS, respectively.The gains are reported for four anchor-based detector architectures.
  • Assignment metrics: 20.2 AP is obtained by NWD for label assignment, outperforming the compared IoU-based and GWD metrics.The result supports NWD as a more suitable tiny-object measurement in the reported comparison.
  • Ablation study: 21.4 AP is obtained when NWD and RKA are combined, compared with 20.2 AP for NWD alone and 17.9 AP for RKA alone.The ablation reports complementary effects between the two modules over the 12.8 AP baseline.
  • Anchor settings: 22.2 AP is achieved with anchor scale 4, while performance remains relatively stable across different anchor settings.The authors describe this stability as robustness to anchor-size choices.
  • Feature resolution: NWD-RKA remains superior with HRNet, reaching 22.6 AP for Faster R-CNN under the stronger backbone.HRNet alone improves Faster R-CNN by 1.7 AP points over the ResNet-50 version.

FCOS Faster R-CNN DetectoRS NWD+RKA

NWD-RKA improves detection across additional aerial datasets and reduces false negatives and false positives in visualized results. Its DetectoRS gains are reported on AI-TOD, VisDrone2019, and DOTA-v2.0.

  • Experiments on other datasets: DetectoRS with NWD-RKA gains 8.6, 1.7, and 1.1 AP points on AI-TOD, VisDrone2019, and DOTA-v2.0, respectively.The improvement is reported across all tested detectors and datasets, with particularly notable gains for tiny objects.
  • Visualization: NWD-RKA greatly reduces false negatives in visualized DetectoRS results by providing sufficient supervision from positive tiny-object samples.The comparison uses baseline detectors in the first four rows and an NWD-RKA DetectoRS detector in the fifth row.
  • Visualization: NWD-RKA also handles false-positive detections more effectively than the visualized RetinaNet baseline.The passage attributes this observation to higher-quality positive and negative sample assignments.

7. Conclusion

The paper contributes NWD-RKA for tiny-object detection and releases the relabeled AI-TOD-v2 dataset with a corresponding benchmark. Experiments across four datasets report large performance improvements and state-of-the-art performance on AI-TOD-v2.

  • Method: NWD-RKA combines normalized Wasserstein distance with ranking-based assignment and can be embedded into anchor-based detectors.The strategy is presented as a label-assignment approach for tiny objects in aerial images.
  • Dataset: AI-TOD-v2 is a meticulously optimized dataset and benchmark intended to support tiny-object detection in aerial images.The paper states that AI-TOD-v2 has the smallest average object size among aerial image datasets.
  • Conclusion: Experiments on four datasets show large performance improvements, with state-of-the-art performance achieved on AI-TOD-v2.The conclusion summarizes the reported experimental findings without specifying a single detector or metric value.
  • Impact: The open-access dataset and benchmark are intended to facilitate fair comparisons and encourage further research in tiny-object detection.The stated scope includes the earth vision and computer vision communities.
Loading 2206.13996v1…