PaperlayerView exact revision on arXiv

Source-linked AI summary

Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges

Jian Ding, Nan Xue, Gui-Song Xia, Xiang Bai, Wen Yang, Micheal Ying Yang, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, Liangpei Zhang

arXiv:2102.12219v2cs.CV

TL;DR

Object detection in aerial images lacks large-scale, precise benchmarks and must handle arbitrary orientations, scale variation, and domain-specific detector design. The paper introduces DOTA-v2.0 with OBB annotations, builds a unified code library, and evaluates broad algorithmic baselines. The resulting analyses show that aerial detectors require different hyperparameter and module choices from natural-image detectors, while the resources support fairer and more reproducible research.

  • Problem

    Aerial object detection lacks large-scale benchmarks with precise OBB annotations, despite challenges from arbitrary orientations and scale variation.

  • Method

    The paper expands DOTA to DOTA-v2.0, provides an oriented-detection code library, and evaluates 10 algorithms across over 70 configurations under unified settings.

  • Results

    Aerial detectors require hyperparameter selections and module designs that differ substantially from those used for natural images.

  • Takeaways & Limitations

    DOTA, its code library, and evaluation resources provide benchmarks for fair comparisons and support reproducible research in Earth vision.

  • Takeaways & Limitations

    OBB annotations can be inaccurate because object boundaries are sometimes ambiguous, and modeling inaccurate OBB annotations remains future work.

Abstract

from arXiv · show

In the past decade, object detection has achieved significant progress in natural images but not in aerial images, due to the massive variations in the scale and orientation of objects caused by the bird's-eye view of aerial images. More importantly, the lack of large-scale benchmarks has become a major obstacle to the development of object detection in aerial images (ODAI). In this paper,we present a large-scale Dataset of Object deTection in Aerial images (DOTA) and comprehensive baselines for ODAI. The proposed DOTA dataset contains 1,793,658 object instances of 18 categories of oriented-bounding-box annotations collected from 11,268 aerial images. Based on this large-scale and well-annotated dataset, we build baselines covering 10 state-of-the-art algorithms with over 70 configurations, where the speed and accuracy performances of each model have been evaluated. Furthermore, we provide a code library for ODAI and build a website for evaluating different algorithms. Previous challenges run on DOTA have attracted more than 1300 teams worldwide. We believe that the expanded large-scale DOTA dataset, the extensive baselines, the code library and the challenges can facilitate the designs of robust algorithms and reproducible research on the problem of object detection in aerial images.

1 INTRODUCTION

Aerial object detection remains difficult because overhead imagery creates arbitrary orientations, large scale variation, crowded scenes, and domain-specific detector requirements. The paper addresses these gaps with DOTA-v2.0, a unified code library, and comprehensive baselines and evaluations.

  • Challenges: OBB representations localize oriented objects more precisely than HBBs and help distinguish densely packed instances.The paper identifies arbitrary orientation as a key difference from natural images and links OBBs to precise localization and rotation-invariant features.
  • Challenges: Aerial images contain arbitrary object orientations, substantial scale variation, and uneven spatial distributions, complicating object detection.Small vehicles may appear at arbitrary angles, while instances range from sparse regions to locally crowded areas.
  • Dataset gap: Existing aerial datasets are limited in scale or realism, while xView uses HBBs rather than precise OBB annotations.These limitations motivate a large-scale dataset with oriented annotations for realistic aerial scenes.
  • Contributions: DOTA-v2.0 contains approximately 1.8 million OBB-annotated instances from 11,268 aerial images across 18 categories.The dataset is described as the largest public Earth vision object detection dataset.
  • Contributions: The paper evaluates 10 algorithms and over 70 model configurations using a unified code library, hardware, software platform, and settings.The evaluations report speed and accuracy analyses intended to guide aerial detector design.
  • Contributions: The dataset, code library, and evaluation server support benchmarking and reproducible research, while DOTA challenges attracted approximately 1300 registrations.The paper presents these resources as mechanisms for advancing aerial object detection research.

2 RELATED WORK

Related work covers benchmark datasets, aerial detection methods, and code libraries. It emphasizes that aerial imagery combines large images, many instances, arbitrary orientations, scale variation, crowded objects, and domain-specific implementation needs.

  • Datasets for Conventional Object Detection: DOTA is comparable in scale to large natural-image detection datasets and surpasses them in average instances per image.The comparison accounts for bounding-box quantities and average bounding-box counts per image.
  • Datasets for Object Detection in Aerial Images: Aerial detection datasets have often targeted specific categories, motivating DOTA's broader benchmark design.The related-work discussion contrasts specialized building, vehicle, ship, and plane datasets with the need for more general resources.
  • Datasets for Object Detection in Aerial Images: DOTA is designed around substantial annotated data, large images, precise OBB annotations, and balanced image sources.These properties are presented as desirable characteristics for aerial detection datasets.
  • Deep Models for Object Detection in Aerial Images: Aerial detectors address orientation and scale variation with rotation-aware features, OBB-based methods, and feature or image pyramids.Examples include rotation-invariant layers, active rotating filters, RoI Transformer, S2A-Net, and pyramid approaches.
  • Deep Models for Object Detection in Aerial Images: Crowded objects challenge HBB-based NMS, so aerial detection methods use rotated NMS and oriented object detection formulations.Precise OBB detections are required to better handle overlapping instances represented by HBBs.
  • Deep Models for Object Detection in Aerial Images: Extremely large aerial images exceed GPU memory limits, making patch splitting a practical baseline strategy.The paper follows naive patch-based solutions after noting that downsampling loses detailed information.
  • Code Libraries for Object Detection: The enriched MMDetection library adds oriented-detection operators and enables evaluation of 10 aerial object-detection algorithms.The broader motivation is to make comparisons easier through modular, standardized code libraries.

3 CONSTRUCTION OF DOTA

DOTA is constructed from heterogeneous aerial sources with oriented annotations designed to represent rotated and densely packed objects. Its annotation workflow combines expert guidance, volunteer labeling, and quality checks.

  • Image collection: DOTA images are collected from Google Earth, GF-2 and JL-1 satellites, and CycloMedia airborne imagery to reduce sensor and resolution bias.
  • OBB representation: DOTA uses oriented bounding boxes (OBBs) because horizontal boxes cannot accurately outline or distinguish densely distributed oriented objects.OBBs are represented by the four image coordinates of their vertices, arranged clockwise.
  • Oriented annotation: Annotators click the four corners of each OBB, while categories with distinct head and tail use a selected first point to encode object direction.
  • Quality control: Experts provide training examples, plain volunteers create initial annotations, and senior volunteers plus authors perform double checks.
  • Annotation limitations: Annotation errors include missed tiny objects and inaccurate boundaries, with future work suggested for modeling inaccurate OBB annotations.

4 PROPERTIES OF DOTA

DOTA combines imagery from multiple sources and provides spatial-resolution information relevant to object-scale analysis. Its examples and metadata reflect varied acquisition platforms and visual conditions.

  • Image sources: DOTA-v2.0 contains Google Earth, GF-2 and JL-1 satellite, and CycloMedia airborne images, with Google Earth contributing most positive samples.
  • Metadata: DOTA-v2.0 omits image geolocation because the task emphasizes visual cues rather than geographic position.
  • Spatial resolution: Ground sample distance (GSD) indicates the distance between pixel centers on Earth and can support object-size estimation, filtering, detector inputs, and scale normalization.
  • Examples: Figures 4 and 5 provide annotated-image examples and typical images from Google Earth, GF&JL satellites, and CycloMedia.

4.3 Various Instance Orientations

DOTA captures the broad orientation diversity characteristic of overhead imagery. Its instances can occur at arbitrary angles, supporting research on rotation-invariant features and oriented detection.

  • Orientation distribution: Unlike many text and face datasets whose orientations remain within narrower gravity-related ranges, DOTA lacks that directional restriction.
  • Spatial statistics: Figure 6 reports the ground sample distance statistics for 30% of DOTA-v2.0 images.
  • Orientation distribution: DOTA objects have approximately equal probabilities of arbitrary angles across [−π, π].

4.4 Various Instances Pixel Sizes

DOTA contains substantial variation in instance size across and within categories, including many small objects. These scale properties increase detection difficulty and distinguish its distribution from several other datasets.

  • Size definition: DOTA-v2.0 defines small instances as 10–50 pixels high, medium as 50–300, and large as above 300 pixels.Instance size is measured using the height of the horizontal bounding box.
  • Aspect ratios: Figure 8 compares aspect-ratio distributions for oriented and horizontal bounding boxes in DOTA.
  • Size distribution: DOTA-v2.0 includes more small instances than DOTA-v1.0, with some objects approximately 10 pixels high.
  • Scale variation: Instance scales vary greatly both within and between categories, making detection more challenging.

4.5 Various Instance Aspect Ratios (ARs)

DOTA instances vary significantly in aspect ratio, a property used to guide anchor-based model design.

  • DOTA instances exhibit significant variation in aspect ratio.Aspect ratios are measured for both original oriented bounding boxes and their axis-aligned bounding boxes.

4.6 Various Instance Densities of the Images

DOTA contains images with highly variable instance densities, from extremely sparse patches to patches containing up to 1000 instances. Density also differs substantially across object categories.

  • Up to 1000 instances can occur in one DOTA image patch, while another may contain only one.
  • DOTA’s per-image instance counts vary more widely than those of natural image datasets.
  • Storage tank, ship, and small vehicle are the three densest DOTA categories.Density is based on each instance’s distance to the closest instance in the same category, classified as dense, normal, or sparse.

4.7 DOTA Versions

DOTA evolves from a limited initial benchmark into a larger, more diverse dataset that adds tiny-object annotations, new categories, and multi-source imagery. Its evaluation uses held-out test subsets through an online server.

  • Motivation and expansion: DOTA-v2.0 expands beyond DOTA-v1.0 by annotating tiny objects and collecting more large-size GF-2 and airborne images with lower foreground ratios.These changes target tiny and normal object detection, large-scale images with few objects, and generalization across multi-source overhead imagery.
  • DOTA-v1.0: DOTA-v1.0 contains 2,806 images, 188,282 instances, and 15 categories.Its training, validation, and testing proportions are 1/2, 1/6, and 1/3, respectively.
  • DOTA-v1.5: DOTA-v1.5 adds annotations for instances smaller than 10 pixels and introduces the container crane category with 402,089 instances.It uses the same images and dataset splits as DOTA-v1.0.
  • DOTA-v2.0: DOTA-v2.0 contains 11,268 images, 1,793,658 instances, and 18 categories, adding airport and helipad categories beyond DOTA-v1.5.It uses training, validation, test-dev, and test-challenge subsets, with smaller training and validation proportions than the test set.
  • Evaluation: DOTA-v2.0 test-dev and test-challenge images are released without annotations and evaluated through the evaluation server.The experiments in this paper use the test-dev subset.

5 BENCHMARKS

The benchmark defines HBB and OBB detection tasks, standardizes evaluation and implementations, and adapts detectors to oriented objects and large aerial images. It also releases supporting code and development tools.

  • Evaluation tasks and metrics: The benchmark evaluates detection with horizontal bounding boxes and oriented bounding boxes using confidence scores and PASCAL VOC 07 mAP.HBBs are rectangles, whereas OBBs are quadrilaterals.
  • Unified benchmark implementation: A unified code library based on MMDetection makes algorithms easier to compare under consistent implementations and settings.
  • Large-image processing: Large images are cropped into 1,024×1,024 patches with stride 824, then detections are mapped back and merged with NMS.
  • Oriented-object baselines: OBB baselines either replace HBB heads with OBB heads or treat OBB annotations as coarse masks for mask-based prediction.The OBB Head regresses oriented-box offsets, while the Mask Head predicts pixel-level classifications from regions of interest.
  • Oriented-object baselines: Replacing standard heads produces models including Faster R-CNN OBB, RetinaNet OBB, and Faster R-CNN H-OBB, with additional evaluations using deformable pooling and RoI Transformer modules.
  • Tools: The released code library and development kit support visualization, OBB IoU and mAP computation, evaluation, and cropping or merging large images.The OBB IoU implementation includes CPU and GPU versions.

6 RESULTS

The evaluation covers more than 70 experiments across baseline detectors, backbones, geometric modules, proposal settings, augmentation strategies, and difficult aerial scenes. Results show that oriented representations and geometry-aware designs are important, while aerial datasets and scenes remain substantially challenging.

  • Benchmark Results: Over 70 experiments evaluate 10 two-stage and one-stage algorithms across DOTA-v1.0, DOTA-v1.5, and DOTA-v2.0.Baseline evaluations report HBB and OBB mAP where supported, using unified experimental settings.
  • Benchmark Results: OBB mAP is usually slightly lower than HBB mAP because oriented detection requires more precise localization.
  • Backbones and Overall Performance: Faster R-CNN OBB + RoI Transformer outperforms the other evaluated methods on the speed-accuracy curve.The comparison uses different backbones and measures speed on a single Tesla V100.
  • Geometric Modules: RoI Transformer performs better than Deformable RoI Pooling, indicating an advantage for carefully designed geometry transformation modules in aerial images.Dpool improves Faster R-CNN OBB at most times, but RoI Transformer performs better than Dpool.
  • Number of Proposals: 8,000 proposals give the highest performance for Faster R-CNN OBB + RoI Transformer, although later experiments use 2,000 proposals because more proposals increase computation.Increasing proposals from 1,000 to 10,000 improves mAP by 2.2 points for Faster R-CNN + RoI Transformer and 1.39 points for Faster R-CNN OBB.
  • State-of-the-Art Results: On DOTA-v1.0, the RoI Transformer reaches 73.76 OBB mAP and 79.82 with matched augmentation, exceeding Li et al. by 3.46 OBB-mAP points.The matched-augmentation result also exceeds Li et al. by 1.96 HBB-mAP points.

7 CONCLUSION

DOTA provides a large-scale, OBB-annotated benchmark and code library for evaluating aerial-image detection. The experiments show that aerial-image detectors require settings different from natural-image detectors, supporting fairer comparisons and broader research.

  • Dataset and benchmark: 1,793,658 instances in DOTA are annotated with oriented bounding boxes, providing a large-scale basis for aerial-image detection research.The dataset covers real-world aerial imagery and 18 common categories.
  • Dataset and benchmark: The code library evaluates both oriented and horizontal aerial-image detectors to support benchmark comparisons.It is intended to provide a common basis for evaluating algorithms.
  • Experimental findings: Hyperparameter selection and module design for aerial images differ substantially from those used for natural images.The conclusion gives the number of proposals as an example of an aerial-specific design choice.
  • Experimental findings: DOTA can supplement natural-scene datasets in efforts toward universal object detection.This is presented as an implication of the observed differences between aerial and natural-image detector settings.
  • Future work: Future work will extend the dataset, host more challenges, and integrate additional oriented-object-detection algorithms into the code library.The authors also connect these resources to broader questions in Earth-vision and general computer vision.
Loading 2102.12219v2…