Source-linked AI summary
DOTA: A Large-scale Dataset for Object Detection in Aerial Images
Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, Liangpei Zhang
TL;DR
Aerial object detection needs datasets that capture substantial variation and realistic complexity, but existing annotated datasets are limited. The paper introduces DOTA, a large expert-annotated aerial dataset with flexible quadrilateral annotations and detector benchmarks. Cross-dataset results show performance differences of 35.8 and 15.6 for YOLOv2-A and YOLOv2-D, respectively, while low DOTA results indicate its greater challenge.
Problem
Existing aerial datasets often use ideal conditions and do not adequately represent the scale, orientation, crowding, and complexity of real aerial scenes.
Method
The paper constructs DOTA from aerial images annotated by experts using arbitrary quadrilateral bounding boxes and evaluates object detection algorithms on the dataset.
Results
35.8 and 15.6 are the performance differences across datasets for YOLOv2-A and YOLOv2-D, respectively, and both models obtain low results on DOTA.
Takeaways & Limitations
DOTA provides a challenging benchmark for aerial object detection and supports oriented bounding-box detection for practical applications.
Takeaways & Limitations
The annotation method may become more precise and robust with more elaborately designed protocols, while alternative protocols may improve crowdsourced annotation efficiency.
Abstract
from arXiv · showhide
Object detection is an important and challenging problem in computer vision. Although the past decade has witnessed major advances in object detection in natural scenes, such successes have been slow to aerial imagery, not only because of the huge variation in the scale, orientation and shape of the object instances on the earth's surface, but also due to the scarcity of well-annotated datasets of objects in aerial scenes. To advance object detection research in Earth Vision, also known as Earth Observation and Remote Sensing, we introduce a large-scale Dataset for Object deTection in Aerial images (DOTA). To this end, we collect $2806$ aerial images from different sensors and platforms. Each image is of the size about 4000-by-4000 pixels and contains objects exhibiting a wide variety of scales, orientations, and shapes. These DOTA images are then annotated by experts in aerial image interpretation using $15$ common object categories. The fully annotated DOTA images contains $188,282$ instances, each of which is labeled by an arbitrary (8 d.o.f.) quadrilateral To build a baseline for object detection in Earth Vision, we evaluate state-of-the-art object detection algorithms on DOTA. Experiments demonstrate that DOTA well represents real Earth Vision applications and are quite challenging.
1 Introduction
Aerial object detection remains difficult because objects vary greatly in scale, orientation, shape, and crowding, while existing datasets inadequately represent real-world complexity. DOTA addresses this gap with a large, expert-annotated benchmark and baseline evaluations.
- Challenges: Aerial images contain objects with huge scale variation, arbitrary orientations, extreme aspect ratios, and dense or sparse distributions.These properties distinguish Earth Vision detection from conventional natural-image detection.
- Dataset gap: Existing aerial datasets often use ideal conditions with clear backgrounds and sparsely distributed instances, limiting their representation of problem complexity.The paper identifies dataset bias and insufficiently realistic annotations as central obstacles.
- DOTA: DOTA contains 2806 aerial images of about 4000 × 4000 pixels, 188,282 instances, and 15 common categories annotated by experts with oriented bounding boxes.The dataset spans different sensors and platforms and includes varied object scales, orientations, and shapes.
- Contribution: DOTA is presented as the largest annotated Earth Vision object dataset with diverse categories, suitable for developing and evaluating aerial-image detectors.The authors also plan to expand its size and scope to reflect evolving real-world conditions.
- Contribution: The paper benchmarks state-of-the-art object detection algorithms on DOTA as a baseline for future algorithm development.The benchmark is intended to advance Earth Vision research and pose algorithmic questions for conventional computer vision.
2 Motivations
The paper motivates DOTA as a missing large-scale, challenging aerial benchmark whose data volume, class coverage, annotation detail, and realism better match practical Earth Vision needs. It contrasts DOTA with existing aerial and general-purpose datasets, especially in oriented annotation and instance density.
- Motivation: Aerial object detection lacked a dataset combining large image numbers with detailed annotations, hindering development of deep learning-based Earth Vision methods.The paper connects this need to applications including vehicle counting, remote object tracking, and unmanned driving.
- Desired properties: A suitable aerial benchmark should provide many images, many instances per category, properly oriented annotations, and diverse object classes.These properties are intended to bring the dataset closer to real-world applications.
- Existing datasets: Existing aerial datasets commonly suffer from insufficient data and classes, limited annotation detail, and low image resolution.Vehicle-only or narrow-class datasets also restrict applicability to complicated scenes.
- Annotation design: DOTA uses oriented bounding boxes that better enclose aerial objects and differentiate crowded instances from one another.This annotation choice distinguishes DOTA from general object detection benchmarks such as PASCAL VOC, ImageNet, and MSCOCO.
- Dataset comparison: DOTA surpasses other general object detection datasets by a large margin in average instances per image.Table 2 compares DOTA with PASCAL VOC, ImageNet, and MSCOCO using image, category, and bounding-box statistics.
3 Annotation of DOTA
DOTA is constructed to reduce aerial-dataset bias through diverse imagery, broad categories, and detailed oriented annotations. Its arbitrary quadrilateral labels and explicit point-ordering protocol are designed for crowded, variably oriented objects.
- Dataset construction: DOTA collects images from multiple sensors, platforms, cities, and resolutions, recording geographical coordinates and capture times to avoid duplicates.
- Category selection: Experts select and annotate 15 categories based on their prevalence and value for real-world applications, including separate large- and small-vehicle classes.
- Category selection: DOTA surpasses NWPU VHR-10 in both category count and instances per category.
- Annotation method: Horizontal boxes can overlap heavily for oriented objects, so DOTA uses arbitrary quadrilateral bounding boxes instead of standard or angle-based rectangles.
- Annotation method: The first quadrilateral point is deliberately assigned to encode an object's head when visual cues exist; otherwise, annotators generally use the top-left point.
- Dataset split: The dataset is split into one-half training, one-sixth validation, and one-third testing images to approximately match data distributions.
4 Properties of DOTA
DOTA captures aerial-scene complexity through full-image annotation, balanced instance sizes, spatial-resolution metadata, varied aspect ratios, and dense per-instance labeling. These properties target the scale, shape, and crowding variation encountered in real aerial imagery.
- Image and annotation scale: DOTA annotates original full images ranging from about 800 × 800 to about 4000 × 4000 pixels without partitioning them into pieces.
- Spatial resolution: Spatial-resolution information indicates actual instance size and supports adaptation to same-category size variation and fine-grained distinctions such as small boats versus large warships.
- Dataset examples: The dataset includes samples from each category, with six large-vehicle examples rather than three.
- Spatial resolution: Spatial resolution also supports filtering mislabeled outliers by comparing object sizes within a narrow resolution range for the same category.
- Instance size: DOTA achieves a good balance between small and middle instances, making its size distribution more similar to real-world scenes.
- Aspect ratio: DOTA contains greatly varied instance aspect ratios, including many instances with large aspect ratios relevant to anchor-based model design.
- Instance density: Images can contain up to 2000 instances, and dense areas are annotated instance by instance because aerial viewpoints rarely produce occlusion.
5 Evaluations
The evaluation benchmarks horizontal and oriented detection on DOTA, including adaptations for large images and oriented boxes. Results show that aerial scenes remain difficult, especially for small, crowded, elongated, and arbitrarily oriented objects.
- Two tasks evaluate detection with horizontal bounding boxes (HBB) and oriented bounding boxes (OBB), regardless of how methods were trained.
- 1024 × 1024 patches with stride 512 enable CNN processing of DOTA’s large images, while split objects with Ui < 0.7 are labeled difficult.Testing recombines patch results and applies class-based NMS with thresholds 0.3 for HBB and 0.1 for oriented experiments.
- SSD performs much lower than other HBB models, which the authors attribute to random cropping degrading performance with numerous small training instances.The comparison also indicates substantial instance-size differences between aerial and general object detection.
- Faster R-CNN is modified to predict four-vertex oriented bounding boxes because existing state-of-the-art detectors are not designed for oriented objects.The oriented regression target attaches each rectangular RoI to a ground-truth quadrilateral.
- Small vehicles, large vehicles, and ships perform far below large discrete objects such as planes, swimming pools, and tennis courts.The authors relate the weaker performance to small size and dense placement in aerial images.
- HBB has lower localization precision than OBB for densely packed and oriented objects, while both representations remain unsatisfactory in extremely dense scenes.Large-aspect-ratio objects are difficult for OBB regression but often appear easier under HBB because their axis-aligned aspect ratios are more ordinary.
6 Cross-dataset validations
The cross-dataset validation tests whether training on DOTA or UCAS-AOD transfers across datasets. The results show substantial performance differences and indicate that DOTA contains broader patterns while remaining challenging.
- Cross-dataset generalization is evaluated by training YOLOv2 on either UCAS-AOD or DOTA and testing with HBB annotations on both datasets.The experiment uses 1110 UCAS-AOD images for training and 400 for testing.
- 35.8 and 15.6 are the performance differences for YOLOv2-A and YOLOv2-D, respectively, across the two datasets.
- The results suggest that DOTA covers UCAS-AOD while containing additional patterns and properties not shared by UCAS-AOD.Both models obtain low results on DOTA, reflecting its greater challenge.
7 Conclusion
The paper presents DOTA as a large-scale benchmark for oriented object detection in aerial images. Its annotations and challenging scenes are intended to support practical Earth Vision evaluation and broader algorithmic study.
- DOTA is a large-scale aerial oriented-object dataset described as much larger than existing datasets in the field.
- The dataset annotates many well-distributed oriented objects with oriented bounding boxes and establishes a benchmark for aerial object detection.The paper also demonstrates the feasibility of producing oriented boxes by modifying a mainstream detector.
- DOTA targets densely packed small instances and extremely large instances with arbitrary orientations in large images.The authors characterize these cases as meaningful and challenging for object detection research.