Source-linked AI summary
Detecting Urban Changes with Recurrent Neural Networks from Multitemporal Sentinel-2 Data
Maria Papadomanolaki, Sagar Verma, Maria Vakalopoulou, Siddharth Gupta, Konstantinos Karantzalos
TL;DR
Satellite change detection remains challenging because handcrafted approaches rely on preprocessing and postprocessing, and registration errors and illumination changes can complicate interpretation. This paper combines U-Net spatial features with convolutional LSTM temporal modeling, reporting that LSTM models perform best with all available dates while added dates reduce simple U-Net performance.
Problem
Satellite change detection is challenging: traditional handcrafted methods depend heavily on preprocessing and postprocessing, and registration errors or illumination changes can affect accuracy.
Source: §1 · p. 1Method
A U-Net computes spatial features from multitemporal inputs while convolutional LSTM blocks model temporal patterns, with additional Sentinel-2 dates enriching OSCD.
Source: §1 · p. 2, §2 · p. 2, §2.1 · p. 2Results
LSTM models performed best with all available temporal information, whereas adding dates reduced simple U-Net performance on OSCD testing images.
Source: §3 · p. 3Takeaways & Limitations
OSCD results indicate recurrent networks can boost both F1 and Overall Accuracy rates for urban change detection.
Source: §4 · p. 4
Abstract
from arXiv · showhide
\begin{abstract} The advent of multitemporal high resolution data, like the Copernicus Sentinel-2, has enhanced significantly the potential of monitoring the earth's surface and environmental dynamics. In this paper, we present a novel deep learning framework for urban change detection which combines state-of-the-art fully convolutional networks (similar to U-Net) for feature representation and powerful recurrent networks (such as LSTMs) for temporal modeling. We report our results on the recently publicly available bi-temporal Onera Satellite Change Detection (OSCD) Sentinel-2 dataset, enhancing the temporal information with additional images of the same region on different dates. Moreover, we evaluate the performance of the recurrent networks as well as the use of the additional dates on the unseen test-set using an ensemble cross-validation strategy. All the developed models during the validation phase have scored an overall accuracy of more than 95%, while the use of LSTMs and further temporal information, boost the F1 rate of the change class by an additional 1.5%.
1. INTRODUCTION
Monitoring land-cover change can inform understanding of urban sprawl and city-infrastructure planning, but satellite change detection remains challenging. This paper combines spatial feature extraction with recurrent temporal modeling to investigate pixel-wise urbanization detection from multitemporal imagery.
1. INTRODUCTION
- Land-cover change from human structures and natural phenomena informs monitoring topics including urban sprawl and city-infrastructure planning.Modeling change over time can help understand future land-use and land-cover tendencies.Source: §1 · p. 1
- Using more than two temporal images is presented as more robust to clouds, shadows, and vegetation or agricultural alterations when detecting change.These confounds are illustrated in the top and bottom examples, respectively.Source: Figure 1 · p. 1
- Traditional handcrafted methods rely heavily on preprocessing and postprocessing, making them less adaptable to large-area imagery; registration errors and illumination changes also complicate detection.Illumination changes may not correspond to semantic changes.Source: §1 · p. 1
- Patch-based deep methods process pixels individually and are time-consuming, while fully convolutional siamese U-Nets lack appropriate temporal-pattern modeling.Prior work also used recurrent layers to learn sequential information from bi-temporal patches, and U-Net-based fully convolutional approaches were proposed to detect changed regions.Source: §1 · p. 2
- The proposed model uses U-Net spatial features and fully convolutional LSTM layers to model temporal change patterns for pixel-wise urbanization detection.Experiments enrich the OSCD dataset with additional dates for each city, while avoiding fully connected layers.Source: §1 · p. 2
2. METHODOLOGY
The model encodes each date with a U-Net and uses convolutional recurrent blocks to model temporal relationships before decoding a change-probability map. Experiments enrich OSCD with additional dates and train on labeled cities using patch augmentation and cross-validation.
2. METHODOLOGY
- The U-Net encoder uses convolution, batch normalization, and ReLU blocks, ending at one-fourth the input dimensions with 256 feature planes.The encoder increases depth across five blocks and uses 2x2 max pooling in the later blocks.Source: §2 · p. 2
- Convolutional recurrent blocks at five encoder levels model temporal relationships among date-specific features, replacing fully connected LSTM operations with convolutional structures.The resulting recurrent weights are convolutional layers in an end-to-end trainable framework.Source: §2 · p. 2
- The decoder upsamples the final temporal volume and concatenates each upsampled feature map with the corresponding encoder temporal pattern before producing an urbanization-probability heatmap.This combines higher-resolution with lower-resolution information while maintaining spatial and temporal knowledge.Source: §2 · p. 2
2.1. Dataset and Implementation Details
- OSCD provides 13 Sentinel-2 channels for 24 cities, with ground truth for 14; the 14 labeled image pairs train the model and the remaining cities are used for testing.Additional Sentinel-2 dates were acquired for the provided cities, with dates selected to correspond as closely as possible to similar seasons.Source: §2.1 · p. 2
- Training used 32x32 patches with stride 6 when change pixels were included and 32 otherwise, flipping patches with >5% change pixels and inverse-frequency class weights; 32,421 patches from 14 training cities informed five-model averaged test predictions.Source: §2.1 · pp. 2–3
3. EXPERIMENTAL RESULTS AND DISCUSSION
On OSCD testing images, LSTM models performed best with all available temporal information, while adding dates reduced performance for simple U-Net. Qualitative examples show multi-date approaches often reduce bare-soil false positives, with a noted exception in Dubai.
3. EXPERIMENTAL RESULTS AND DISCUSSION
- LSTM models performed best with all available temporal information, while additional dates reduced simple U-Net performance; qualitative panels show fewer bare-soil false positives with multi-date inputs.The Dubai five-date example retains bare-soil false positives that are eliminated in the three-date example.Source: §3 · p. 3, Figure 3 · p. 4
- Over 50% recall was maintained across all dates-and-channels combinations, while the 2/4 U-Net detected change more successfully than the 2/13 version by F1.The authors base experiments on four higher-resolution channels because 13-channel volumes require more computation and registration preprocessing.Source: §3 · p. 3
- LSTM frameworks distinguish clouds from change in the Milano example, and multi-date approaches appear to reduce confusion between urban change and bare-soil change.Clouds and improper registration are cited as possible sources of false positives.Source: §3 · pp. 3–4
4. CONCLUSION
The conclusion reports that recurrent networks boost both F1 and Overall Accuracy on OSCD, and identifies further recurrent architectures and very high-resolution imagery as future directions.
4. CONCLUSION
- On OSCD, recurrent networks can boost both F1 and Overall Accuracy rates.The conclusion reports these results across experiments with various architecture and input combinations.Source: §4 · p. 4
- Future work will investigate other types and combinations of recurrent fully convolutional architectures on high-resolution images.Source: §4 · p. 4
- The authors also plan to investigate these models for change detection on very high-resolution satellite imagery.Source: §4 · p. 4