Source-linked AI summary
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
Patrick Helber, Benjamin Bischke, Andreas Dengel, Damian Borth
TL;DR
The paper addresses limited suitable training data for land use and land cover classification from satellite imagery. It introduces and benchmarks the georeferenced, multispectral EuroSAT dataset, whose RGB band combination achieves 98.57% overall classification accuracy, and demonstrates applications in change detection and map assistance.
Problem
Existing land use and land cover datasets are small-scale or rely on data sources unsuitable for the intended Earth observation applications.
Method
The paper constructs EuroSAT from openly accessible Sentinel-2 imagery as 27,000 labeled, georeferenced 64x64-pixel patches across 10 classes and benchmarks CNNs over 13 spectral bands.
Results
98.57% overall classification accuracy is achieved by the RGB band combination on EuroSAT.
Takeaways & Limitations
The trained classifier is demonstrated for detecting land use or land cover changes and assisting with keeping geographical maps up to date.
Abstract
from arXiv · showhide
In this paper, we address the challenge of land use and land cover classification using Sentinel-2 satellite images. The Sentinel-2 satellite images are openly and freely accessible provided in the Earth observation program Copernicus. We present a novel dataset based on Sentinel-2 satellite images covering 13 spectral bands and consisting out of 10 classes with in total 27,000 labeled and geo-referenced images. We provide benchmarks for this novel dataset with its spectral bands using state-of-the-art deep Convolutional Neural Network (CNNs). With the proposed novel dataset, we achieved an overall classification accuracy of 98.57%. The resulting classification system opens a gate towards a number of Earth observation applications. We demonstrate how this classification system can be used for detecting land use and land cover changes and how it can assist in improving geographical maps. The geo-referenced dataset EuroSAT is made publicly available at https://github.com/phelber/eurosat.
I. INTRODUCTION
The paper targets land use and land cover classification from openly accessible satellite imagery, motivated by limitations in existing datasets. It introduces EuroSAT and benchmarks CNN-based classification using Sentinel-2 spectral data.
- Existing land use and land cover datasets are often small-scale or unsuitable for the intended Earth observation applications.
- EuroSAT contains 27,000 labeled images spanning 10 land use and land cover classes.
- The dataset is based on Sentinel-2 imagery and covers 13 spectral bands.
- Every EuroSAT image is geo-referenced, and the dataset is publicly released for reuse.
- The paper benchmarks the dataset with Convolutional Neural Networks and evaluates each Sentinel-2 spectral band.
II. RELATED WORK
The related-work section surveys prior land use and land cover datasets from remotely sensed imagery and reviews state-of-the-art image-classification methods.
- Prior studies use remotely sensed aerial and satellite image datasets for land use and land cover classification.
- The section reviews state-of-the-art image classification methods applied to land use and land cover classification.
- The review covers both datasets and classification methods as the two main components of prior work.
A. Classification Datasets
Prior classification datasets vary in scale, imagery source, spatial resolution, spectral content, and class coverage. The paper positions EuroSAT against these limitations while noting CNNs as a leading classification approach.
- A. Classification Datasets: The UCM land use dataset contains 21 classes with 100 RGB aircraft images per class at about 30 cm per pixel.
- A. Classification Datasets: AID contains 30 classes with 200 to 400 high-resolution images per class from Google Earth imagery.
- A. Classification Datasets: Commercial and preprocessed very-high-resolution datasets may be unsuitable for real-world Sentinel-2 Earth observation applications.
- A. Classification Datasets: The Brazilian Coffee Scene dataset covers coffee and non-coffee crops using 1,423 images per class with red, green, and near-infrared bands.
- A. Classification Datasets: SAT-6 contains six classes from 1-meter aerial imagery, using 28x28-pixel patches with red, green, and blue bands.
- A. Classification Datasets: Convolutional Neural Networks became a state-of-the-art image-classification method after strong results on image-classification challenges.
III. DATASET ACQUISITION
The paper uses Sentinel-2 imagery within ESA’s Copernicus Earth observation program and constructs a georeferenced patch dataset from satellite images collected across Europe.
- III. DATASET ACQUISITION: ESA operates Sentinel satellites through its Copernicus program to improve Earth observation.
- III. DATASET ACQUISITION: Sentinel-2A and Sentinel-2B are identical land-monitoring satellites in a two-satellite constellation.
- III. DATASET ACQUISITION: The dataset construction gathers satellite images of European cities distributed across more than 34 countries.
- III. DATASET ACQUISITION: The collected imagery is converted into 27,000 georeferenced and labeled 64x64-pixel image patches.
A. Satellite Image Acquisition
The dataset draws Sentinel-2A imagery from European Urban Atlas cities across 34 countries, selecting low-cloud scenes and seeking broad geographic and temporal variation.
- Sentinel-2A images were gathered for cities distributed across 34 European countries.
- The images were associated with cities covered by the European Urban Atlas.
- Low-cloud satellite images were selected using ESA-provided cloud-level values.
- The acquisition strategy sought many countries and year-round imagery to capture high intra-class variance.
B. Dataset Creation
EuroSAT is a georeferenced Sentinel-2 dataset designed for land-use and land-cover benchmarking, with 10 visible classes and 27,000 manually checked patches spanning 13 bands.
- The dataset covers all 13 Sentinel-2 spectral bands, whose table lists identification, spatial resolution, and central wavelength.
- The dataset defines 10 land-use and land-cover classes chosen to be visible at 10 meters per pixel and sufficiently represented in the European Urban Atlas.
- All 27,000 images were manually checked multiple times, with mislabeled, snow-covered, and ice-covered samples removed.
- Atmospheric correction was not applied, so some retained images exhibit color casts.
- EuroSAT uses openly accessible imagery, lower spatial resolution than the closest benchmark, 10 classes, and a georeferenced release.
IV. DATASET BENCHMARKING
The benchmark evaluates CNN classifiers against BoVW and prior initialization choices on EuroSAT. Deep CNNs outperform BoVW and shallow CNNs, with ResNet-50 achieving the strongest EuroSAT performance.
- Comparative Evaluation: All CNN approaches outperform BoVW, while deep CNNs generally outperform shallow CNNs.
- Comparative Evaluation: Fine-tuned pretrained networks achieve about 2% higher classification accuracy than randomly initialized networks.
- Comparative Evaluation: ResNet-50 performs best on the introduced EuroSAT land-use and land-cover classes.
- Comparative Evaluation: Deep CNNs outperform previous results on AID, SAT-6, and BCS by about 2-4%.
- Comparative Evaluation: The ResNet-50 confusion matrix shows occasional confusion between agricultural classes and between highway and river.
B. Band Evaluation
The band evaluation compares single Sentinel-2 bands with color-infrared, shortwave-infrared, and RGB combinations using fine-tuned ResNet-50 models. Band combinations outperform single bands, while RGB performs best on the introduced classes.
- The single-band experiment uses one band replicated across all three CNN input channels.
- The RGB confusion matrix reports fine-tuned ResNet-50 performance using satellite images in RGB color space.
- Band-combination images outperform single-band images.
- RGB images perform best among the evaluated combinations on the introduced land-use and land-cover classes.
V. APPLICATIONS
The paper demonstrates that its classification system supports practical Earth observation applications. These applications include land use and land cover change detection and assistance in keeping geographical maps up-to-date.
- 98.57% overall accuracy paves the way for the demonstrated Earth observation applications.
- The applications include detecting land use and land cover changes and assisting with geographical map updates.
A. Land Use and Land Cover Change Detection
The trained classifier detects land use and land cover changes by comparing classification results for 64x64 patches from the same spatial region at different times. The examples show demolished industrial buildings, newly built residential buildings, and deforestation.
- 64x64 image regions from the same spatial location are compared across time, with differing classifications indicating change.
- Industrial buildings near Shanghai were demolished between December 2015 and December 2016.
- Residential buildings were built in the highlighted Dallas area between August 2015 and March 2017.
- A large land area near Villamontes, Bolivia was deforested between October 2015 and September 2016.
B. Assistance in Mapping
The patch-based classifier can assist in updating geographical maps by producing classification results across image patches. The paper demonstrates this use with industrial areas and OpenStreetMap coverage.
- 64x64 patch classification cannot provide finely graduated per-pixel segmentation but can help keep maps up-to-date.
- A sliding-window system can verify tagged areas, identify mistagged areas, or support large-area tagging.
- In the Shanghai example, many industrial buildings visible in satellite imagery were not covered by the corresponding OpenStreetMap map.
- The conclusion presents the classifier as a first step toward monitoring Earth’s land surface and updating geographical maps.