PaperlayerView exact revision on arXiv

Source-linked AI summary

FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery

Xian Sun, Peijin Wang, Zhiyuan Yan, Feng Xu, Ruiping Wang, Wenhui Diao, Jin Chen, Jihao Li, Yingchao Feng, Tao Xu, Martin Weinmann, Stefan Hinz, Cheng Wang, Kun Fu

arXiv:2103.05569v2cs.CV

TL;DR

Existing remote-sensing detection datasets are limited in scale, category richness, temporal coverage, and geographic information, while detector performance depends strongly on data quality and quantity. FAIR1M introduces a large, fine-grained benchmark with oriented-box annotations and evaluates detection and classification methods, whose results show substantial dataset difficulty and benefits from hierarchical training.

  • Problem

    Existing remote-sensing detection datasets have limited scale, category information, temporal coverage, and geographic metadata, although deep-learning performance depends strongly on data quality and quantity.

  • Method

    FAIR1M collects over 15,000 images and 1 million instances with oriented bounding boxes across 5 categories and 37 sub-categories, and benchmarks detection and classification models, including a staged hierarchical detector.

  • Results

    The benchmark is challenging for generic detectors, while staged hierarchical training yields 32.46% mAP and coarse-category learning improves fine-grained detection accuracy.

  • Takeaways & Limitations

    FAIR1M provides a large, fine-grained benchmark intended to foster future research in high-resolution remote-sensing object recognition.

  • Takeaways & Limitations

    Annotations use a 16-pixel threshold, so objects below the threshold are excluded according to image-resolution and down-sampling considerations.

Abstract

from arXiv · show

With the rapid development of deep learning, many deep learning-based approaches have made great achievements in object detection task. It is generally known that deep learning is a data-driven method. Data directly impact the performance of object detectors to some extent. Although existing datasets have included common objects in remote sensing images, they still have some limitations in terms of scale, categories, and images. Therefore, there is a strong requirement for establishing a large-scale benchmark on object detection in high-resolution remote sensing images. In this paper, we propose a novel benchmark dataset with more than 1 million instances and more than 15,000 images for Fine-grAined object recognItion in high-Resolution remote sensing imagery which is named as FAIR1M. All objects in the FAIR1M dataset are annotated with respect to 5 categories and 37 sub-categories by oriented bounding boxes. Compared with existing detection datasets dedicated to object detection, the FAIR1M dataset has 4 particular characteristics: (1) it is much larger than other existing object detection datasets both in terms of the quantity of instances and the quantity of images, (2) it provides more rich fine-grained category information for objects in remote sensing images, (3) it contains geographic information such as latitude, longitude and resolution, (4) it provides better image quality owing to a careful data cleaning procedure. To establish a baseline for fine-grained object recognition, we propose a novel evaluation method and benchmark fine-grained object detection tasks and a visual classification task using several State-Of-The-Art (SOTA) deep learning-based models on our FAIR1M dataset. Experimental results strongly indicate that the FAIR1M dataset is closer to practical application and it is considerably more challenging than existing datasets.

1. Introduction

Existing remote-sensing detection datasets are limited in scale, fine-grained categories, image quality, and geographic information. FAIR1M addresses these gaps with a large, fine-grained benchmark, a score-aware evaluation metric, and baseline detection and classification tasks.

  • Dataset gaps: Existing datasets need larger scales because detector generalization should be evaluated using many object instances and images.Small datasets can allow apparently strong algorithms to overfit and perform poorly on larger datasets.
  • Dataset gaps: Remote-sensing applications require finer categories than generic labels, such as distinguishing Airbus 350 from Boeing 747 within Airplane.Existing datasets often provide coarse annotations, limiting large-scale fine-grained type recognition.
  • Dataset gaps: Existing imagery also lacks geographic characteristics and may suffer quality degradation from clouds, fog, or improper cleaning.Relevant geographic information includes temporal and spatial properties such as resolution, longitude, and latitude.
  • FAIR1M benchmark: FAIR1M contains more than 15,000 images and 1 million instances annotated with oriented boxes across 5 categories and 37 sub-categories.The images come from Gaofen satellites and Google Earth, span multiple continents, and include geographic information and multi-temporal imagery.
  • Evaluation and baselines: The paper introduces a score-aware evaluation metric and benchmarks fine-grained detection and visual classification with representative state-of-the-art models.It also proposes a cascaded hierarchical detection network as a baseline for future work.

2. Related Work

Existing remote-sensing detection datasets provide important benchmarks, but remain limited in scale, category richness, image diversity, and practical coverage. FAIR1M is positioned as a larger, more comprehensive dataset using fine-grained categories and oriented annotations.

  • Generic remote-sensing datasets include NWPU VHR-10, HRRSD, DOTA, and DIOR, but their scale and annotation coverage differ substantially.NWPU VHR-10, for example, contains 715 images and 3,775 instances.
  • The related-work figures summarize FAIR1M's category samples, category-instance distribution, per-image instance and category distributions, and geographic distribution across continents.
  • Fine-grained datasets such as VEDAI and MTARSI focus on vehicles or airplane types, while xView and SIMD address fine-grained multi-class recognition.VEDAI contains 1,210 images and 3,700 instances; xView contains over 1 million horizontal boxes in 1,127 images across 60 categories.
  • FAIR1M applies oriented bounding boxes and selects fine-grained categories spanning airplanes, ships, vehicles, courts, and roads.

3. Details of the FAIR1M Dataset

FAIR1M combines globally collected high-resolution imagery with fine-grained categories, quality control, and oriented annotations. Its design targets practical remote-sensing recognition by representing varied object scales, orientations, and difficult category distinctions.

  • 3.1. Data Collection: FAIR1M contains more than 15,000 images collected from Gaofen satellites and Google Earth across more than 100 airports, harbors, and cities worldwide.The images have spatial resolutions ranging from 0.3m to 0.8m.
  • 3.1. Data Collection: The dataset preprocessing removes imagery with clouds, noise, and bright spots and applies block adjustment and orthographic processing to improve image quality and positioning consistency.
  • 3.2. Category Design: FAIR1M covers five object categories and 37 fine-grained categories selected with practical applications in mind, including nine vehicle types and additional court and road types.
  • 3.3. Annotation: Oriented rectangular annotations record four vertices clockwise, with the top-left vertex designated as the first point and positive object direction.
  • 3.3. Annotation: A 16-pixel annotation threshold reflects neural-network down-sampling, so objects below the relevant resolution-dependent size are not annotated.For example, ships longer than 13 meters need not be annotated at 0.8m spatial resolution.
  • 3.3. Annotation: Three checking stages use paired re-annotation, annotation merging, and supervisor review to control annotation quality.
  • 3.4. Characteristics of the Dataset: FAIR1M is characterized by broad scale and orientation variation, high within-class variation, and strong between-class similarity.
  • 3.5. Dataset Splits: The dataset is divided into training, validation, and testing subsets with proportions 1/2, 1/6, and 1/3, respectively.Images and ground truth are provided for training and validation, while testing provides images and server-based evaluation.

4. Algorithm Analysis

FAIR1M evaluates fine-grained oriented and horizontal detection, visual classification, and cross-dataset generalization, exposing challenges from fine-grained categories, object variation, and scene complexity. The proposed CHODNet uses hierarchical staged training and achieves improved fine-grained detection performance, while error and classification analyses identify category confusion and missed small objects as major difficulties.

  • Tasks: FAIR1M benchmarks fine-grained oriented object detection, horizontal object detection, and visual categorization across 37 categories.The evaluation includes oriented and horizontal bounding-box detection plus fine-grained visual classification.
  • Evaluation Metrics: FIoU and mAP_F evaluate localization and fine-grained classification sensitivity beyond generic object-detection mAP.mAP_F averages AP_F across categories and is more sensitive to fine-grained classification scores.
  • Benchmarks: Five oriented and three horizontal detectors show that fine-grained detection is difficult, with rare categories such as C919 and ARJ21 receiving particularly low accuracy.Objects with many instances and obvious features are generally easier, while category imbalance produces uneven accuracy.
  • Error Analysis: Classification errors account for most errors, while missed detections strongly affect ships and vehicles because their objects are small.Removing similar-object and missed-detection errors improves performance more than removing dissimilar-object errors for most categories.
  • Hierarchical Object Detection Method: CHODNet independently learns external and internal representations through a cascaded hierarchy aligned with FAIR1M’s coarse-to-fine categories.Its staged training prioritizes coarse-category features before focusing on the fine-grained stage, using weighted losses.
  • Hierarchical Object Detection Method: 32.46% mAP is obtained by CHODNet, and adding a coarse-category branch improves fine-grained object-detection accuracy.The method is also tested on multi-temporal images from Gaoqi airport and Dalian port.
  • Fine-grained Image Classification: Fine-grained classification remains challenging: MMAL-Net performs relatively well on FAIR-Airplane except for rare C919 and ARJ21, but struggles more with ships and vehicles.Motorboat and Small Car have relatively high confidence, while remaining categories show confusion with other categories.
  • Cross-dataset Validations: 27.92% mAP and 6.77% mAP are the reported cross-dataset differences, with ROI Transformer variants obtaining lower results on FAIR1M.The comparison indicates that FAIR1M contains more object types and patterns than DOTA and is more challenging for cross-dataset evaluation.

5. Conclusion

FAIR1M is proposed as a challenging dataset for fine-grained object detection and recognition in high-resolution remote sensing imagery. The paper introduces its dataset characteristics and benchmarks state-of-the-art algorithms to foster future research.

  • FAIR1M targets fine-grained object detection and recognition in high-resolution remote sensing imagery.
  • The dataset’s challenge reflects fine-grained types, varied sizes and orientations, within-class variation, between-class similarity, complex scenes, and geographic information.
  • The paper introduces FAIR1M’s collection, categories, annotations, and characteristics, then builds an object detection benchmark with state-of-the-art algorithms.
  • The authors aim for FAIR1M to enhance fine-grained object recognition research because coarse recognition does not adequately meet remote-sensing interpretation requirements.
Loading 2103.05569v2…