Source-linked AI summary
DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentation
Aysim Toker, Lukas Kondmann, Mark Weber, Marvin Eisenberger, Andrés Camero, Jingliang Hu, Ariadna Pregel Hoderlein, Çağlar Şenaras, Timothy Davis, Daniel Cremers, Giovanni Marchisio, Xiao Xiang Zhu, Laura Leal-Taixé
TL;DR
Local land-cover change requires both frequent observations and pixel-wise semantic labels, which existing datasets often do not provide together. DynamicEarthNet supplies daily multispectral imagery for 75 global areas with monthly labels for seven LULC classes, evaluates baseline uses of the time series, and introduces a change-focused protocol. Daily-data training achieves 43.6% mIoU, compared with 37.9% for monthly and weekly results, while the dataset excludes snow and ice from quantitative evaluation because it appears in only two cubes.
Problem
Monitoring local land-cover change requires high-frequency satellite observations and pixel-precise labels, but existing datasets often lack one or both.
Method
DynamicEarthNet provides daily multispectral imagery for 75 global areas, monthly pixel-wise labels for seven LULC classes, baseline evaluations, and a semantic-change evaluation protocol.
Results
43.6% mIoU is obtained by training on daily data, improving over the 37.9% mIoU from monthly and weekly results.
Takeaways & Limitations
The dataset supports semi-supervised and spatio-temporal settings for studying land-cover evolution with dense time-series observations and high-quality annotations.
Takeaways & Limitations
Snow and ice occur in only two cubes, so this class is absent from validation and test sets and excluded from quantitative evaluation.
Abstract
from arXiv · showhide
Earth observation is a fundamental tool for monitoring the evolution of land use in specific areas of interest. Observing and precisely defining change, in this context, requires both time-series data and pixel-wise segmentations. To that end, we propose the DynamicEarthNet dataset that consists of daily, multi-spectral satellite observations of 75 selected areas of interest distributed over the globe with imagery from Planet Labs. These observations are paired with pixel-wise monthly semantic segmentation labels of 7 land use and land cover (LULC) classes. DynamicEarthNet is the first dataset that provides this unique combination of daily measurements and high-quality labels. In our experiments, we compare several established baselines that either utilize the daily observations as additional training data (semi-supervised learning) or multiple observations at once (spatio-temporal learning) as a point of reference for future research. Finally, we propose a new evaluation metric SCS that addresses the specific challenges associated with time-series semantic change segmentation. The data is available at: https://mediatum.ub.tum.de/1650201.
1. Introduction
DynamicEarthNet addresses the need for frequent observations and pixel-precise labels for monitoring local land-cover change by providing daily multispectral imagery, monthly annotations, and a change-focused evaluation protocol.
- Motivation: Existing datasets commonly lack either frequent observations or pixel-wise annotations, limiting their suitability for local change monitoring.Available labeled datasets may contain only static images or yearly revisits, while coarser annotations may be irregular or monthly.
- Contribution: DynamicEarthNet provides daily multispectral satellite observations for 75 global areas, paired with monthly pixel-wise labels for seven LULC classes.The classes are impervious surfaces, water, soil, agriculture, wetlands, snow & ice, and forest & other vegetation.
- Challenges: Satellite-image segmentation presents domain-specific challenges because labeled regions have intricate shapes and standard vision methods do not necessarily transfer well.The paper also argues that common segmentation metrics are not optimal for semantic change segmentation.
- Contribution: The proposed evaluation protocol models both binary change and semantic segmentation for semantic change segmentation.This directly targets the two central properties identified for the task.
- Experiments: Baseline experiments evaluate how daily observations can support semantic-change detection through additional training data or multiple observations.The paper uses these baselines as reference points for exploiting the dataset’s time-series structure.
2. Related work
Prior satellite benchmarks span semantic segmentation, change detection, and time-series analysis, but commonly provide limited spatial coverage, sparse temporal sampling, or annotations that differ from DynamicEarthNet’s dense pixel-wise LULC labels.
- Semantic segmentation: Earlier satellite-segmentation datasets often contain fewer areas of interest, while newer aerial and satellite benchmarks may target objects or instance segmentation rather than dense pixel-wise semantic labels.DynamicEarthNet focuses on semantic annotations defined for every pixel.
- Change detection: Existing change-detection datasets are often binary, bitemporal, regionally limited, or separated by yearly observations rather than full sequences.HRSCD and Hi-UCD provide multi-class semantic change detection but cover specific regions and show only one observation per year for a few years.
- Time-series analysis: Time-series datasets such as Earthnet2021, fMoW, and MUDS provide temporal imagery for forecasting, bounding boxes, or urbanization monitoring but not the same combination of dense LULC labels and daily observations.Their intended applications and annotation types differ from DynamicEarthNet’s semantic change-segmentation setting.
- Satellite datasets: Compared with related benchmarks, DynamicEarthNet combines dense semantic annotations with daily time-series observations across many areas of interest.The comparison includes revisit time, image count, data source, GSD, annotation types, and annotated objects.
- Change detection: The paper motivates a standardized multi-class change-detection benchmark because existing datasets and evaluation metrics differ widely and are often not public.Prior approaches commonly consider only two snapshots, frequently years apart.
3. The DynamicEarthNet dataset
DynamicEarthNet combines daily, cloud-free multi-spectral imagery from 75 global AOIs with monthly pixel-wise labels for seven LULC classes. Its consistent two-year observations support analysis of both short-term and persistent land-cover change.
- 3.2. Pixel-wise labels: Monthly annotations provide pixel-wise labels for seven LULC classes at the same 3-meter, 1024x1024 resolution as the imagery.Labels correspond to the first day of each month and cover impervious surfaces, agriculture, forest and other vegetation, wetlands, soil, water, and snow and ice.
- 3.2. Pixel-wise labels: Labels are constructed for temporal consistency by updating an initial manual map when perceptible changes occur and applying three quality-control gates.The gates address annotation accuracy, topological correctness, and format correctness.
- 3.3. Dataset statistics: DynamicEarthNet provides 54,750 satellite images and 1,800 monthly ground-truth annotations across 75 AOIs over two years.Each AOI contains 730 daily images and 24 labeled dates.
- 3.3. Dataset statistics: Class occurrences are imbalanced, with forest and other vegetation and soil more prevalent than less frequent classes such as wetlands.No AOI contains all seven classes, and snow and ice is absent from validation and test sets because it occurs in only two cubes.
- 3.4. Advantages over existing benchmarks: The dataset supports observing day-by-day construction and deforestation alongside persistent effects over multiple months.The daily revisit rate enables short-term change detection, while the two-year span captures longer-term evolution.
4. Semantic change segmentation
Semantic change segmentation evaluates both whether pixels changed and which semantic classes changed. The proposed SCS metric combines binary change detection with semantic accuracy on changed pixels.
- Problem definition: The task generalizes binary change detection by assigning semantic classes to pixels whose labels change between consecutive observations.This preserves both the location and the type of land-cover change.
- Evaluation protocol: SCS combines a class-agnostic binary change score with semantic segmentation accuracy among changed pixels into one signal.The two components address missed changes and incorrect semantic labels for changed pixels.
- Evaluation protocol: Binary change measures overlap between predicted and ground-truth change maps using a Jaccard or intersection-over-union score.The binary component focuses on identifying which pixels changed.
- Evaluation protocol: Semantic change computes a class-averaged Jaccard score between ground-truth and predicted labels, conditioned on pixels with ground-truth change.This isolates semantic classification performance where change actually occurs.
- Evaluation protocol: The final SCS score is the arithmetic mean of the binary-change and semantic-change components, computed after accumulating confusion matrices across time-series.Both components contribute equally to the final score.
5. Experiments
Experiments evaluate spatio-temporal and semi-supervised baselines for LULC and semantic change segmentation under monthly, weekly, and daily sampling. Weekly inputs generally benefit spatio-temporal models, while daily data works best for semi-supervised segmentation but can destabilize sequence models.
- Land cover and land use segmentation: Weekly temporal inputs improve all three spatio-temporal architectures over the supervised baseline on validation, with U-TAE and U-ConvLSTM generalizing best on test.The supervised U-Net baseline reaches 33.5% validation mIoU and 37.6% test mIoU.
- Land cover and land use segmentation: Daily sequences of 28–31 images considerably reduce spatio-temporal performance, indicating that weekly sampling is optimal for this benchmark.Highly correlated daily images can make training unstable when labeled data is limited.
- Land cover and land use segmentation: Additional unlabeled weekly and daily images improve over the fully supervised semi-supervised baseline, with daily data producing the strongest result.The semi-supervised approach uses context-aware consistency training on unlabeled images and DeepLabv3+ as its backbone.
- Land cover and land use segmentation: 43.6% mIoU on the test set is achieved by semi-supervised training with daily data, improving over 37.9% for monthly and weekly settings.Agriculture and wetlands remain difficult classes, with agriculture often confused with forest or soil and wetlands with soil or water.
- Semantic change segmentation: Binary change detection is particularly difficult, with most accuracies around 10% while change covers only 5% of pixels on average.Visually similar transitions, such as forest and other vegetation to soil, create many potential false positives.
- Semantic change segmentation: mIoU alone does not guarantee strong semantic change performance, so SCS separates binary change BC from semantic change SC.The results show that semantic-change and binary-change performance can be somewhat decoupled.
6. Conclusion
DynamicEarthNet combines daily multi-spectral imagery with monthly annotations across diverse areas of interest, enabling evaluation settings unavailable in temporally sparse benchmarks. The paper also introduces an evaluation protocol for semantic change segmentation and demonstrates the value of dense time-series data.
- 6. Conclusion: DynamicEarthNet provides daily multi-spectral imagery and monthly semantic annotations for seven common LULC classes across diverse areas of interest.This combination distinguishes it from benchmarks that are temporally sparse or lack comparable ground-truth labels.
- 6. Conclusion: The dataset enables semi-supervised learning and spatio-temporal methods at an unprecedented temporal resolution.These settings arise from combining dense time-series observations with high-quality annotations.
- 6. Conclusion: The proposed semantic change segmentation protocol uses metrics targeting distinct common errors in multi-class change prediction.The authors suggest that the benchmark can support development of techniques designed for daily multi-spectral data.
A. Dataset details
The dataset’s core imagery comes from Planet Fusion, which supplies multi-spectral surface reflectance together with per-pixel quality metadata. The metadata records whether observations are raw or gap-filled and identifies the temporal source of filled pixels.
- A.1. Planet metadata: Planet Fusion forms the core of DynamicEarthNet and provides RGB-plus-near-infrared surface reflectance imagery.Planet also supplies quality-assurance information alongside the imagery.
- A.1. Planet metadata: Per-pixel quality metadata records the distance and direction to the observation day, including whether a value was filled from a nearby observation.A pixel value of -1 indicates filling from the previous day.
A.2. Sentinel 2 auxiliary images
Sentinel-2 imagery is provided as monthly composites from the same locations to support comparisons with Planet Fusion. Its lower temporal and spatial resolution is complemented by more spectral channels, but cloud-related quality issues affect some composites.
- A.2. Sentinel 2 auxiliary images: Sentinel-2 offers 13 channels and a five-day revisit rate, whereas Planet Fusion provides four channels with higher temporal and spatial resolution.The additional Sentinel-2 channels include short-wave infrared information.
- A.2. Sentinel 2 auxiliary images: Monthly Sentinel-2 composites from the same locations enable direct comparisons between Sentinel-2 and Planet Fusion imagery.The composites are formed from multiple Sentinel-2 images collected throughout each month.
- A.2. Sentinel 2 auxiliary images: 26% of monthly Sentinel-2 composites have minor quality issues and around 5% have major quality issues.Cloud coverage can occlude all Sentinel-2 images for an area during a month, compromising the resulting composite.
- A.2. Sentinel 2 auxiliary images: Users should investigate whether Sentinel-2 cubes or months are affected by the recorded quality problems.The dataset includes a quality-assessment spreadsheet identifying affected months and areas.
B. Evaluation protocol details
The evaluation protocol compares existing metrics and motivates the proposed metric for semantic change segmentation.
- The protocol compares alternative metrics according to their efficacy for semantic change segmentation.
B.1. Semantic change
Semantic change evaluation separates binary change from semantic classification by assessing changed pixels under a dedicated protocol. The proposed approach uses ground-truth changed pixels to decouple errors and supports both bi-temporal and multi-temporal evaluation.
- The protocol distinguishes changed-pixel sets from predictions, ground truth, and their intersection when defining semantic-change evaluation.
- The proposed metric evaluates semantic classification on ground-truth changed pixels, decoupling semantic-change scores from binary-change errors.Using predicted-change pixels could allow artificially reducing predicted changes to manipulate the score, whereas the selected ground-truth subset keeps interpretation intuitive.
- Pixel accuracy is rejected because 95% unchanged pixels allow always predicting no change to obtain a misleading 95% score.
- The protocol adapts IoU rather than Cohen’s kappa because IoU provides class-specific information without allowing frequent classes to dominate the aggregate.
- The metric accepts separate binary-change and semantic maps, enabling decoupled predictions and correction of prior frame-level semantic mistakes.
- The evaluation extends from bi-temporal pairs to varying-length intervals using the dataset’s consistent multi-temporal labels.The reported multi-temporal evaluation includes 552 pairs across each area of interest.
- The multi-temporal metric generally produces slightly higher values because its smoothing effect reduces emphasis on identifying the exact change frame.
C. Implementation details
The experiments use four-band satellite imagery with dataset-level per-band normalization and standard augmentation. Training settings differ across spatio-temporal experiments, including an extended schedule for daily samples.
- The dataset uses four spectral bands—RGB plus near-infrared—and normalizes each band using dataset-wide mean and standard deviation.
- Images are randomly resized between 0.5 and 2 times, cropped to 512 × 512, and horizontally flipped during augmentation.
- Snow and ice are excluded from test and validation sets because that class is scarce.
- Spatio-temporal models use Adam with learning rate 1e-4, batch size 4, and up to 100 training epochs.
- The daily-sample spatio-temporal experiment uses 200 epochs to ensure convergence.
D. Additional qualitative results
Additional qualitative analyses compare spatio-temporal and semi-supervised predictions and inspect class-specific errors. They show temporal-density effects and recurring confusion among uncommon land-cover classes.
- Weekly training achieves the best validation results for CAC, while monthly and daily settings struggle with wetlands and agriculture.
- Confusion matrices reveal frequent wetland-to-soil errors and more stable predictions for common classes such as forest and soil.
- Among spatio-temporal methods, 3D-Unet performs best on the challenging impervious-surface class.
- The visual comparisons show spatio-temporal models receiving 30 or 31 daily samples to predict one monthly semantic map.
- The qualitative figures compare CAC predictions across monthly, weekly, and daily training setups against inputs and ground-truth maps.