Source-linked AI summary
RarePlanes: Synthetic Data Takes Flight
Jacob Shermeyer, Thomas Hossler, Adam Van Etten, Daniel Hogan, Ryan Lewis, Daeil Kim
TL;DR
RarePlanes addresses limited openly available, very-high-resolution overhead data for testing synthetic imagery in aircraft detection and attribute classification. It combines large synthetic and real satellite datasets with fine-grained annotations and benchmarks their use. The experiments report that a 90% synthetic and 10% real blend achieves nearly equivalent aircraft-identification performance to using 100% real data.
Problem
Openly available overhead datasets are scarce, while synthetic data requires paired real imagery with shared features to quantify its value.
Method
RarePlanes combines AI.Reverie-generated synthetic satellite imagery, real WorldView-3 scenes, and 10 fine-grained aircraft attributes for benchmarking overhead vision.
Results
A blend of 90% synthetic and 10% real data delivers nearly equivalent performance to 100% real data for aircraft identification.
Takeaways & Limitations
RarePlanes supports studying synthetic data, domain adaptation, detection, instance segmentation, and zero-shot learning in overhead imagery.
Abstract
from arXiv · showhide
RarePlanes is a unique open-source machine learning dataset that incorporates both real and synthetically generated satellite imagery. The RarePlanes dataset specifically focuses on the value of synthetic data to aid computer vision algorithms in their ability to automatically detect aircraft and their attributes in satellite imagery. Although other synthetic/real combination datasets exist, RarePlanes is the largest openly-available very-high resolution dataset built to test the value of synthetic data from an overhead perspective. Previous research has shown that synthetic data can reduce the amount of real training data needed and potentially improve performance for many tasks in the computer vision domain. The real portion of the dataset consists of 253 Maxar WorldView-3 satellite scenes spanning 112 locations and 2,142 km^2 with 14,700 hand-annotated aircraft. The accompanying synthetic dataset is generated via AI.Reverie's simulation platform and features 50,000 synthetic satellite images simulating a total area of 9331.2 km^2 with ~630,000 aircraft annotations. Both the real and synthetically generated aircraft feature 10 fine grain attributes including: aircraft length, wingspan, wing-shape, wing-position, wingspan class, propulsion, number of engines, number of vertical-stabilizers, presence of canards, and aircraft role. Finally, we conduct extensive experiments to evaluate the real and synthetic datasets and compare performances. By doing so, we show the value of synthetic data for the task of detecting and classifying aircraft from an overhead perspective.
1. Introduction
RarePlanes addresses the lack of large, openly available synthetic overhead datasets by combining realistic synthetic and real satellite imagery for aircraft detection and fine-grained classification. Its annotations and experiments support evaluating how synthetic data can reduce reliance on manually labeled real imagery.
- 1. Introduction: RarePlanes fills a gap in openly licensed overhead data, where small heterogeneous objects, sensor variation, illumination, weather, geography, and seasonal conditions complicate dataset creation.Synthetic data requires a paired real dataset with shared features to establish a quantitative baseline.
- 1. Introduction: RarePlanes combines an expansive synthetic overhead dataset with real satellite imagery to evaluate aircraft detection and attribute classification.The dataset is generated from realistic airport environments and includes varied weather, illumination, look angle, biomes, and aircraft distributions.
- 1. Introduction: 50,000 synthetic images cover 9,331.2 km^2 with approximately 630,000 annotations, while 253 real WorldView-3 images cover 2,142 km^2 across 112 locations with approximately 14,700 aircraft.
- 1. Introduction: Each aircraft has 10 fine-grained attributes and 33 sub-attribute choices, enabling customizable classes for detection, instance segmentation, and zero-shot learning.The attributes include geometry, propulsion, engine and stabilizer counts, canards, and aircraft role.
- 1. Introduction: The dataset provides three contribution pillars: paired real and synthetic overhead imagery, detailed aircraft annotations, and experiments comparing their value for overhead aircraft recognition.
2. Related Work
RarePlanes connects synthetic, geospatial, and fine-grained attribute dataset traditions while extending their combination to very-high-resolution overhead aircraft imagery. Its annotation and comparison design supports customizable classes and evaluation of synthetic data for overhead vision tasks.
- 2. Related Work: RarePlanes occupies the intersection of synthetic, geospatial, and fine-grained attribution datasets, combining characteristics from all three domains.Table 1 compares RarePlanes with datasets from these categories.
- 2.1. Synthetic Datasets: Synthetic datasets have supported replacement or augmentation of real data across autonomous driving, indoor scenes, occlusion, and other computer vision tasks.Combined synthetic-real datasets have been used for object detection, semantic segmentation, and instance segmentation.
- 2. Related Work: Open overhead datasets remain scarce, and domain adaptation is relevant because models trained in one location often generalize poorly to new areas.
- 2.1. Synthetic Datasets: RarePlanes includes bounding boxes, diamond polygons, and synthetic-only full instance segmentation annotations.
3. The RarePlanes Dataset and Statistics
RarePlanes combines real and procedurally generated overhead imagery with detailed aircraft annotations spanning geometry, engines, tails, and operational role. Its imagery covers diverse locations, environmental conditions, resolutions, and look angles.
- Annotations, Features, and Attributes: Aircraft roles include three civil transport size classes and four military categories, distinguishing heterogeneous aircraft styles and sizes.The role labels include small, medium, and large civil transport or utility aircraft, military transport or utility or AWAC, bomber, fighter or interceptor or attack, and trainer.
- Real imagery: Real imagery spans 2,142 km2 across 112 locations in 22 countries, with geographic and seasonal sampling designed to increase heterogeneity.Scenes vary in look angle, azimuth, sun elevation, season, cloud cover, snow, and clear-sky conditions.
- Synthetic imagery: Synthetic imagery covers 9,331.2 km2 at simulated 0.3 m GSD, using procedural GIS and Unreal Engine environments with variable look angles and biomes.The synthetic set contains 50,000 1920×1080 images and 629,551 aircraft annotations across five biomes.
4. Experiments, Results, and Discussion
The experiments compare models trained on real data, synthetic data, and synthetic data fine-tuned with approximately 10% real data for detection and segmentation. Synthetic-only training suffers a domain gap, while fine-tuning substantially recovers performance relative to real-only training.
- Experimental setup: The evaluation trains benchmark detection and segmentation networks on real-only, synthetic-only, and synthetic-plus-real fine-tuning subsets.The fine-tuning subset contains approximately 10% of the real training data, and models are evaluated on the real test dataset.
- Evaluation: Performance is measured with COCO average precision, including per-class AP, mAP, mAP50, and average recall.Qualitative detection examples compare ground truth, real-trained, synthetic-trained, and fine-tuned models.
- Results and Discussion: Synthetic-only training yields 55%–75% of real AP, although average recall remains 70%–80% of real recall.False positives include clutter, nearby objects, snow patches, ground markings, and airport vehicles.
- Results and Discussion: Fine-tuning synthetic-trained models with approximately 10% real data reaches 91–96% of real-only mAP.The experiments evaluate object detection and instance segmentation on the real test dataset, with results reported in Table 2.
5. Conclusions
RarePlanes provides paired real and synthetic satellite imagery for studying aircraft identification from overhead views. Its benchmark results indicate that a 90% synthetic and 10% real blend can approach performance achieved with entirely real data.
- Conclusions: A 90% synthetic and 10% real blend delivers nearly equivalent aircraft-identification performance to 100% real data.The dataset also supports research on domain adaptation and additional computer vision tasks through its fine-grained annotations.
- Conclusions: RarePlanes combines real and synthetically generated satellite imagery with detailed aircraft annotations for overhead computer vision research.The authors identify detection, instance segmentation, and zero-shot learning as potential uses.