PaperlayerView exact revision on arXiv

Source-linked AI summary

Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark

Ke Li, Gang Wan, Gong Cheng, Liqiu Meng, Junwei Han

arXiv:1909.00133v2cs.CV

TL;DR

Existing optical remote-sensing object-detection datasets and methods remain limited, particularly in dataset scale, diversity, and adaptation to remote-sensing characteristics. This paper reviews deep learning progress, proposes the large-scale DIOR benchmark, and evaluates representative detectors to establish a baseline for future research.

  • Problem

    Existing datasets have small numbers of images and categories, insufficient image diversity and object variation, while transferred detection methods remain imperfect for remote-sensing rotations, scale changes, and cluttered backgrounds.

  • Method

    The paper reviews deep learning object-detection methods and introduces the publicly available DIOR benchmark, then evaluates representative detection approaches on it.

  • Results

    Representative detectors are benchmarked on DIOR, with RetinaNet using ResNet-101 and PANet using ResNet-101 each achieving 66.1% mAP.

  • Takeaways & Limitations

    DIOR provides a benchmark for developing and validating deep learning methods and a performance baseline for future research in earth observation.

  • Takeaways & Limitations

    Existing methods remain an open research problem because remote-sensing images differ substantially from natural scenes in rotation, scale variation, and complex backgrounds.

Abstract

from arXiv · show

Substantial efforts have been devoted more recently to presenting various methods for object detection in optical remote sensing images. However, the current survey of datasets and deep learning based methods for object detection in optical remote sensing images is not adequate. Moreover, most of the existing datasets have some shortcomings, for example, the numbers of images and object categories are small scale, and the image diversity and variations are insufficient. These limitations greatly affect the development of deep learning based object detection methods. In the paper, we provide a comprehensive review of the recent deep learning based object detection progress in both the computer vision and earth observation communities. Then, we propose a large-scale, publicly available benchmark for object DetectIon in Optical Remote sensing images, which we name as DIOR. The dataset contains 23463 images and 192472 instances, covering 20 object classes. The proposed DIOR dataset 1) is large-scale on the object categories, on the object instance number, and on the total image number; 2) has a large range of object size variations, not only in terms of spatial resolutions, but also in the aspect of inter- and intra-class size variability across objects; 3) holds big variations as the images are obtained with different imaging conditions, weathers, seasons, and image quality; and 4) has high inter-class similarity and intra-class diversity. The proposed benchmark can help the researchers to develop and validate their data-driven methods. Finally, we evaluate several state-of-the-art approaches on our DIOR dataset to establish a baseline for future research.

1. Introduction

The paper identifies inadequate surveys and limited remote-sensing detection datasets as barriers to progress, then introduces DIOR as a large-scale public benchmark and evaluates representative methods on it.

  • Remote-sensing object detection differs substantially from natural-scene detection, particularly in rotation, scale variation, and cluttered backgrounds.
  • Existing remote-sensing datasets are often small in images and categories, with insufficient image diversity and object variation.
  • DIOR contains 23,463 images spanning 20 object categories, providing a large-scale public benchmark for optical remote-sensing object detection.
  • DIOR combines large object-size variation, diverse imaging conditions, and high inter-class similarity with intra-class diversity.
  • The paper surveys more than 110 papers across computer vision and earth observation, then benchmarks representative detection methods on DIOR for future research.

2. Review on Object Detection in Computer Vision Community

The computer-vision review covers datasets and deep-learning detectors, including region proposal-based and regression-based streams, along with efficiency, feature-pyramid, and scale-variation advances.

  • Datasets: Natural-scene detection datasets range from PASCAL VOC’s 20 classes to MSCOCO’s more than 200,000 images and 80 categories, and ImageNet’s 200 classes.
  • Detection frameworks: Deep-learning object detectors are broadly divided into region proposal-based methods and regression-based methods.
  • Region Proposal-based Methods: Region proposal-based detectors separate proposal generation from classification, while Faster R-CNN replaces handcrafted proposals with an RPN sharing convolutional computation.
  • Feature pyramids: FPN and PANet improve detection by exploiting hierarchical features and adding localization information through bottom-up path augmentation.
  • Scale variation: SNIP reduces scale variation during training, while SNIPER maintains SNIP-level performance and reduces processed pixels by a factor of 3.
  • Regression-based Methods: Regression-based detectors simplify detection into one-stage prediction, directly estimating bounding boxes and class probabilities without candidate proposals.

3. Review on Object Detection in Earth Observation Community

The review surveys optical remote-sensing object-detection datasets and deep-learning methods, tracing dataset diversity and the shift from proposal-based pipelines toward broader approaches. It highlights persistent limitations from proposal generation and the mismatch between natural-scene detectors and earth-observation imagery.

  • Object Detection Datasets of Optical Remote Sensing Images: Earlier surveys and datasets provided foundations for earth-observation object detection, but deep-learning methods and publicly available datasets required broader review.The reviewed datasets span vehicle, building, ship, and multi-class geospatial detection, with varying image sources, resolutions, and annotation formats.
  • Object Detection Datasets of Optical Remote Sensing Images: 30 images and 1319 annotated cars define TAS, illustrating the small scale of some early aerial-image datasets.TAS contains arbitrarily oriented cars, low-resolution imagery, and substantial shadows from buildings and trees.
  • Object Detection Datasets of Optical Remote Sensing Images: 2806 aerial images and 188282 object instances make DOTA a comparatively large multi-class dataset spanning 15 geospatial object categories.DOTA combines imagery from different sensors and platforms with multiple resolutions.
  • Deep Learning Based Object Detection Methods in Earth Observation Community: Region proposal methods, including R-CNN and Faster R-CNN variants, have been widely adapted for multi-class geospatial-object detection.Examples include rotation-invariant CNNs and multi-angle anchors introduced within proposal-based pipelines.
  • Deep Learning Based Object Detection Methods in Earth Observation Community: Hand-designed proposal generation makes proposal-based approaches time-consuming and can reduce detection performance because low-level features produce poor proposals.The proposal-generation stage accounts for much of the running time in these systems.
  • Deep Learning Based Object Detection Methods in Earth Observation Community: Natural-scene detectors remain insufficient for earth-observation imagery because rotation, scale variation, and cluttered backgrounds differ substantially across domains.Existing methods partially address these factors with prior knowledge or specialized models, but the task remains open for further research.

4. Proposed DIOR Dataset

DIOR addresses limitations in existing remote-sensing detection datasets by offering broad scale, object-size variation, image diversity, and challenging class similarities. It contains many manually labeled images and instances across 20 categories, designed to support deep-learning research.

  • DIOR selects 20 classes by combining categories common to existing datasets with additional objects chosen for prevalence or real-world application value.
  • DIOR is presented as one of the largest and most diverse publicly available earth-observation detection datasets, supporting development and evaluation of deep-learning methods.
  • 23463 images and 192472 manually labeled object instances span 20 common categories at 800×800 pixels and 0.5m–30m spatial resolutions.
  • The dataset includes broad object-size variation from spatial-resolution differences and both between-class and within-class differences.
  • 23463 images from more than 80 countries capture varied weathers, seasons, imaging conditions, image quality, viewpoints, illumination, backgrounds, poses, occlusion, and appearance.
  • Fine-grained classes with semantic overlap and varied colors, shapes, and scales create high inter-class similarity and intra-class diversity.

5. Benchmarking Representative Methods

The paper benchmarks 12 representative deep learning object detectors on DIOR using a trainval/test split, reporting performance patterns and remaining challenges across object categories.

  • Experimental Setup: 11,725 images form the trainval set and 11,738 form the test set, with similar category distributions targeted across both subsets.The trainval set combines training and validation data; images may contain multiple object classes.
  • Experimental Setup: 12 representative object detection methods are selected, spanning approaches widely used in natural-scene and earth-observation imagery.The selections include eight region proposal-based approaches.
  • Experimental Results: 66.1% mAP is achieved by both RetinaNet with ResNet-101 and PANet with ResNet-101, the highest reported overall value.Deeper backbones generally provide stronger representations and higher detection accuracy, following ResNet-101 and Hourglass-104 over ResNet50 and Darknet-53 over VGG16.
  • Experimental Results: Feature pyramid networks significantly improve detection accuracy for objects spanning a wide range of scales.FPN is incorporated into detectors including RetinaNet and PANet.
  • Experimental Results: YOLOv3 generally performs better on small objects such as vehicles, storage tanks, and ships than the other evaluated methods.The passage identifies especially strong small-object performance for the ship category.
  • Experimental Results: Bridge, harbor, overpass, and vehicle categories remain difficult, with low detection accuracies and substantial improvement space across almost all categories.The paper attributes these difficulties to relatively low image quality and complex, cluttered aerial-image backgrounds, characterizing DIOR as challenging.

6. Conclusions

The paper surveys deep learning object detection progress in computer vision and earth observation, introduces a large-scale public benchmark, and evaluates representative methods on it. DIOR supports method development and validation, while the reported experiments provide a baseline for future research.

  • Conclusions: The paper reviews benchmark datasets and state-of-the-art deep learning object detection approaches across computer vision and earth observation.The review covers recent progress in both communities.
  • Conclusions: The paper proposes a large-scale, publicly available object detection benchmark dataset for the earth observation community.The dataset is intended to support further exploration and validation of deep learning-based methods.
  • Conclusions: Evaluations of representative object detection methods on the proposed dataset provide a useful performance baseline for future research.The conclusion presents the experimental results as a baseline rather than a final solution.
Loading 1909.00133v2…