Source-linked AI summary
BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding
Gencer Sumbul, Marcela Charfuelan, Begüm Demir, Volker Markl
TL;DR
Remote sensing lacked large annotated archives suitable for deep learning, while existing labels and ImageNet transfer introduced important mismatches. The paper constructs BigEarthNet, a large multi-label Sentinel-2 benchmark, and reports that even a simple network trained from scratch outperforms fine-tuning a state-of-the-art ImageNet model in the evaluated scene-classification setting.
Problem
Remote sensing lacked a large annotated training archive, and existing archives commonly used single labels despite images containing multiple land-cover classes.
Method
The paper constructs BigEarthNet from Sentinel-2 tiles, dividing them into multi-label image patches annotated using the CLC 2018 database.
Results
A simple neural network trained on BigEarthNet from scratch was effective compared with fine-tuning a state-of-the-art deep-learning model pre-trained on ImageNet.
Takeaways & Limitations
BigEarthNet provides a large-scale benchmark archive for remote sensing image understanding and supports research on large-scale remote sensing image archives.
Takeaways & Limitations
The authors recommend excluding 70,987 images fully covered by seasonal snow, cloud, or cloud shadow from training and testing.
Abstract
from arXiv · showhide
This paper presents the BigEarthNet that is a new large-scale multi-label Sentinel-2 benchmark archive. The BigEarthNet consists of 590,326 Sentinel-2 image patches, each of which is a section of i) 120x120 pixels for 10m bands; ii) 60x60 pixels for 20m bands; and iii) 20x20 pixels for 60m bands. Unlike most of the existing archives, each image patch is annotated by multiple land-cover classes (i.e., multi-labels) that are provided from the CORINE Land Cover database of the year 2018 (CLC 2018). The BigEarthNet is significantly larger than the existing archives in remote sensing (RS) and thus is much more convenient to be used as a training source in the context of deep learning. This paper first addresses the limitations of the existing archives and then describes the properties of the BigEarthNet. Experimental results obtained in the framework of RS image scene classification problems show that a shallow Convolutional Neural Network (CNN) architecture trained on the BigEarthNet provides much higher accuracy compared to a state-of-the-art CNN model pre-trained on the ImageNet (which is a very popular large-scale benchmark archive in computer vision). The BigEarthNet opens up promising directions to advance operational RS applications and research in massive Sentinel-2 image archives.
1. INTRODUCTION
Remote sensing deep learning needs large annotated archives, but existing public resources are too small and often use single labels. BigEarthNet addresses this gap with a large multi-label Sentinel-2 archive for training and evaluating remote sensing models.
- Publicly available remote sensing archives lacked the large number of annotated images needed to train effective deep networks.This shortage was identified as an important bottleneck for applying deep learning in remote sensing.
- Existing remote sensing images commonly used single high-level category labels even though one image can contain multiple land-cover classes.The paper motivates multi-label annotation as a better representation of the content of remote sensing images.
- BigEarthNet contains 590,326 Sentinel-2 image patches annotated with multi-labels from the 2018 CORINE Land Cover database.The archive is proposed as a sufficient source for remote sensing image analysis with deep learning.
- The experiments train a shallow CNN on BigEarthNet and compare it with Inception-v2 pre-trained on ImageNet for remote sensing image scene classification.The comparison tests whether an archive-specific training source can support remote sensing analysis.
- The paper positions BigEarthNet as a basis for advancing algorithms that analyze large-scale remote sensing image archives.
2. LIMITATIONS OF EXISTING REMOTE SENSING IMAGE ARCHIVES
Existing remote sensing benchmarks are limited by small sample sizes, single category labels, and a mismatch with computer-vision pre-training data. These constraints make accurate deep-learning characterization of remote sensing imagery difficult.
- Most established remote sensing archives contain small numbers of images with single category labels.The paper lists UC Merced, WHU-RS19, RSSCN7, SIRI-WHU, AID, NWPU-RESISC45, RSI-CB, EuroSat, and PatternNet as examples.
- Small archives are insufficient for training modern deep neural networks with enough parameters to accurately characterize remote sensing image semantics.
- Fine-tuning ImageNet-pre-trained models is limited by differences between computer-vision and remote-sensing image properties and label definitions.
- Sentinel-2 imagery has 13 spectral bands and varying, lower spatial resolutions compared with computer-vision images.The passage presents these differences as one reason transfer learning from ImageNet may not generally reduce the semantic gap.
3. THE BIGEARTHNET ARCHIVE
BigEarthNet is constructed from Sentinel-2 tiles into hundreds of thousands of multi-label patches linked to CLC 2018 land-cover classes. The archive includes quality-control guidance identifying images that should be excluded from training and testing.
- BigEarthNet was constructed as a large-scale remote sensing benchmark by selecting Sentinel-2 tiles acquired between June 2017 and May 2018.The archive construction is intended to overcome limitations of existing remote sensing archives.
- The selected tiles were divided into 590,326 non-overlapping patches with resolutions of 120 × 120, 60 × 60, and 20 × 20 pixels for 10m, 20m, and 60m bands.
- Each patch was assigned one or more land-cover labels from the CORINE Land Cover database updated in 2018.The archive uses Level-3 CLC class labels within a three-level hierarchy.
- The archive includes Sentinel-2 tiles from 10 European countries with cloud cover below 1%.The tiles were atmospherically corrected with ESA’s sen2cor tool, and Sentinel-2 band 10 was excluded.
- Visual inspection identified 70,987 images fully covered by seasonal snow, cloud, or cloud shadow.The authors suggest excluding these images from training and testing for scene classification, retrieval, and search using only BigEarthNet images.
4. EXPERIMENTAL RESULTS
The experiments evaluate shallow CNNs trained from scratch on BigEarthNet against an ImageNet-pre-trained Inception-v2 model for multi-label Sentinel-2 scene classification. Using all spectral bands improves results over RGB-only inputs, and the reported improvements are statistically significant.
- The evaluation compares shallow CNNs trained from scratch on BigEarthNet with an ImageNet-pre-trained Inception-v2 network.The CNNs use three convolutional layers, fully connected layers, max-pooling, and either RGB or all spectral bands.
- 70,987 images covered by seasonal snow, cloud, or cloud shadow were removed before randomly splitting the remainder into 60% training, 20% validation, and 20% test sets.Both training approaches used 100 epochs and stochastic gradient descent to minimize sigmoid cross-entropy loss.
- The S-CNN-RGB outperforms Inception-v2 across all reported metrics when both models use only RGB channels.This comparison is reported in Table 4.
- Using all spectral bands with S-CNN-All produces more promising results than using only RGB bands.The same behavior is observed for Level-1 and Level-2 CLC class labels.
- Performance improvements across all metrics are statistically significant with p ≪0.0001.The authors note that deeper CNNs trained from scratch could potentially yield further improvements.
5. CONCLUSION
The conclusion presents BigEarthNet as a large-scale, multi-label Sentinel-2 benchmark for remote sensing image understanding. It reports that even a simple CNN trained from scratch on the archive is effective compared with fine-tuning an ImageNet-pre-trained model, while noting planned archive expansion.
- BigEarthNet contains 590,326 Sentinel-2 image patches annotated with multi-labels for remote sensing image understanding.
- Training a simple neural network from scratch on BigEarthNet is reported as effective compared with fine-tuning a state-of-the-art ImageNet-pre-trained deep learning model.
- The archive is intended to support deep learning for large-scale remote sensing image archives.
- The authors plan to regularly enrich BigEarthNet by increasing the number of annotated Sentinel-2 images.