Source-linked AI summary
xView: Objects in Context in Overhead Imagery
Darius Lam, Richard Kuzma, Kevin McGee, Samuel Dooley, Michael Laielli, Matthew Klaric, Yaroslav Bulatov, Brendan McCord
TL;DR
Public satellite imagery is largely unlabeled, while overhead applications require robust detection across varied resolutions, scales, classes, and scene contexts. The paper introduces xView, a large, diverse, quality-controlled overhead object-detection dataset, and shows its scope through broad class, geographic, and object-scale coverage. xView contains over 1 million labeled objects in 60 classes across over 1,400 km^2 and supports future overhead detection research and realistic few-shot learning settings.
Problem
Most public-domain satellite information is unlabeled, while overhead imagery applications require detection across diverse resolutions, object scales, classes, and contexts.
Method
The paper constructs xView from geographically diverse satellite imagery, using 1 km^2 UTM-grid chips and extensive annotation with three-stage quality control.
Results
xView contains over 1 million labeled objects in 60 classes across over 1,400 km^2, with objects ranging from 3 meters to greater than 3,000 meters and over 80% of classes fine grained.
Takeaways & Limitations
xView is intended as a general-purpose and unifying dataset for overhead object detection, including realistic few-shot learning with high- and low-instance-count categories.
Takeaways & Limitations
The baseline analysis indicates that additional techniques such as supervised pre-training could improve performance.
Abstract
from arXiv · showhide
We introduce a new large-scale dataset for the advancement of object detection techniques and overhead object detection research. This satellite imagery dataset enables research progress pertaining to four key computer vision frontiers. We utilize a novel process for geospatial category detection and bounding box annotation with three stages of quality control. Our data is collected from WorldView-3 satellites at 0.3m ground sample distance, providing higher resolution imagery than most public satellite imagery datasets. We compare xView to other object detection datasets in both natural and overhead imagery domains and then provide a baseline analysis using the Single Shot MultiBox Detector. xView is one of the largest and most diverse publicly available object-detection datasets to date, with over 1 million objects across 60 classes in over 1,400 km^2 of imagery.
1. Introduction
xView is designed to address limitations in overhead object-detection data by combining broad geographic and class diversity with high-resolution, multi-scale imagery. Its construction targets challenges in resolution, imbalanced distributions, fine-grained recognition, and dataset coverage.
- Motivation: Few labeled overhead datasets limit comparability, while existing public datasets often have low class counts, narrow scope, poor geographic diversity, or few instances.xView was developed as a general-purpose satellite-imagery object-detection dataset for both computer-vision and remote-sensing communities.
- Improve Minimum Resolution and Multi-Scale Recognition: Objects range from 3 meters to over 3,000 meters, with xView providing 0.3-meter GSD imagery and higher resolution than many public satellite datasets.The range corresponds to approximately 10 to 10,000 pixels at 0.3-meter GSD, supporting research on low-resolution and multi-scale recognition.
- Improve Learning Efficiency: xView includes both rare and abundant object classes, capturing the uneven distributions found in real-world imagery.This distribution is intended to support research on imbalanced classification and localization.
- Improve Detection of Fine Grained Classes: Over 80% of xView classes are fine grained across 7 parent classes, including 8 distinct truck child classes.Examples include pickup truck, utility truck, and cargo truck, distinguishing objects that share a broader category.
- Dataset Construction: The dataset contains 1 million labeled objects across over 1,400 km2, collected across varied scene types, geographic locations, physical environments, and collection geometries.Image sampling sought uniform coverage across relevant scene types and their varying appearances; annotation used RGB and 8-band multispectral chips with axis-aligned bounding boxes.
2. Related Work
Existing overhead datasets provide valuable imagery but remain limited in class diversity, geographic coverage, or task scope. xView addresses these gaps with a broad, multi-class, multi-location object-detection dataset designed for satellite imagery.
- xView combines benchmark capabilities associated with PASCAL VOC and quality-control methodologies associated with COCO for overhead object detection.
- SpaceNet contains around 5 million building masks in five locations, but its object scope is narrower than xView’s multi-class detection focus.
- COWC contains around 32,700 labeled cars, whereas xView includes multiple vehicle types and broader overhead imagery variation.
3. Dataset Details
xView is constructed from geographically varied satellite imagery using hierarchical classes, systematic area sampling, focused image chips, and staged annotation quality control. The resulting dataset covers over 1,400 km^2 with 60 classes and approximately 1 million labeled objects, while preserving variation in object scale, class frequency, and location.
- Image Annotation: xView uses QGIS and an in-house plugin to create axis-aligned bounding boxes on imagery prepared with orthorectification, pan-sharpening, and atmospheric correction.
- Dataset Details: The dataset organizes 60 selected classes hierarchically into general parent categories and specific child categories.
- Image Collection: AOIs span mines, ports, airfields, and coastal, inland, urban, and rural regions across multiple continents to reduce biased image sampling.
- Image Collection: 1 km^2 UTM-grid chips provide a repeatable extraction process while supporting focused annotation and completion tracking.
- Image Annotation: Annotation uses trained workers, category fallback from child to parent labels, exhaustive chip labeling, and exclusion of objects approximately ≤20% visible.
- Image Annotation: Quality control has worker, supervisory, and expert stages covering label, geometry, duplication, exhaustiveness, and gold-standard precision and recall checks.Expert review requires precision 0.75 and recall 0.95 at 0.5 IoU against gold data.
- Dataset Statistics: Over 1,400 km^2, 60 classes, and approximately 1 million labeled objects define the released xView dataset.Buildings and small cars have the highest instance counts, consistent with their stated real-world prevalence.
- Dataset Statistics: The train, test, and validation splits contain 59.2%, 20.0%, and 20.8% of objects, respectively, with at least five instances of every category per split.
4. Algorithmic Analysis
The authors evaluate SSD on vanilla, multi-resolution, and augmented xView datasets to establish a detection baseline. Multi-resolution training performs best, while results remain relatively low and favor larger, contextually simpler objects.
- Experimental setup: SSD was evaluated on vanilla, multi-resolution, and multi-resolution-augmented xView datasets using 300^2-pixel chips and a shared vanilla test set.The experiments used a 70-30% train-test split and evaluated all variants on the vanilla test dataset.
- Results: 0.2590 total mean average precision: multi-resolution training outperformed vanilla at 0.1456 and augmented training at 0.1549.The comparison measures total mean average precision across the three experimental datasets.
- Results: The best-detected classes generally had large pixel areas and uniform backgrounds, while small cars were detected more poorly despite their high instance count.Top-performing classes included planes, buildings, passenger cars, and other relatively contextually discernible categories.
- Results: Multi-resolution and augmented training continued improving, whereas vanilla performance plateaued; the augmentation result may reflect regularization under the fixed training time.All three experiments were capped at seven days for consistency.
- Results: The pickup-truck and hut/tent classes each had over one thousand instances but scored below 1% mAP across all experiments.This demonstrates that instance count alone did not ensure strong per-class detection performance.
- Interpretation: The SSD baseline achieves relatively low mean average precision, indicating that xView remains difficult for object detection.The authors note that additional training time or supervised pre-training could improve performance.
5. Conclusion and Future Work
The paper concludes that xView provides a large, diverse, quality-controlled benchmark for overhead object detection. Its release is intended to support general detection research and future applications including few-shot learning, demographic studies, and humanitarian efforts.
- Conclusion: xView contains over 1 million instances in 60 classes and was produced through extensive annotation with three-stage quality control.The dataset includes land-use objects, vehicles, and mini-scenes.
- Conclusion: The dataset offers greater class and visual-context variety than other overhead datasets and is intended as a general-purpose, unifying resource.The authors position xView for both overhead object detection research and broader computer vision use.
- Future Work: Few-shot learning is identified as a future direction because xView includes realistic category distributions rather than evenly distributed instances.The paper connects this direction to quickly adaptable computer vision tools for disaster relief.
- Future Work: The RGB release can serve as a standalone object-detection benchmark and support research connecting satellite imagery with computer vision.Suggested application areas include demographic studies and humanitarian efforts.
6. Appendix Overview
The appendix provides additional examples, dataset comparisons, category organization, and quality-control illustrations. These materials show how xView annotations and imagery relate to other overhead datasets and how classes are structured.
- Appendix overview: The appendix includes additional dataset examples, comparisons, and images that were re-labeled.
- Quality control: Quality-control examples contrast pre- and post-remediation chips, including objects that were unlabeled or wrongly labeled.The post-quality-control examples show corrected annotations.
- Dataset examples: Axis-aligned boxes can include substantial background around elongated or rotated objects such as cargo ships and shipping crates.The limitation follows from object orientation relative to the satellite and the use of axis-aligned boxes.
- Dataset comparisons: Comparisons with COWC and SpaceNet show differences in car visibility, building prominence, perspective, and lighting across datasets.The xView example includes long shadows associated with a low sun angle.
- Class taxonomy: The class taxonomy organizes all 60 classes under parent and child denominations, with a final None category for classes without a parent.
- Dataset examples: Fully annotated imagery uses different bounding-box shadings to denote object classes.The examples use DigitalGlobe imagery.