Source-linked AI summary
S2Looking: A Satellite Side-Looking Dataset for Building Change Detection
Li Shen, Yao Lu, Hao Chen, Hao Wei, Donghai Xie, Jiabao Yue, Rui Chen, Shouye Lv, Bitao Jiang
TL;DR
Existing change-detection datasets are largely near-nadir, while deep-learning methods require large, high-quality training data for building-change surveillance. The paper introduces S2Looking, a globally distributed side-looking rural dataset with a benchmark, and reports that its added complexity challenges change-detection algorithms. The dataset’s practical scope is constrained by registration difficulty and false detections of irrelevant structures.
Problem
Most change-detection datasets use near-nadir imagery, limiting coverage of subtle incremental building changes and practical applications.
Method
The paper constructs S2Looking from large-scale, off-nadir bitemporal satellite imagery of rural areas and establishes a benchmark for change-detection algorithms.
Results
S2Looking contains 5000 bitemporal image pairs and 65,920 expert-based change annotations, and preliminary tests find it more challenging than near-nadir datasets.
Takeaways & Limitations
S2Looking provides a baseline for evaluating algorithms on large-scale side-looking imagery and for studying the added complexity of rural building-change detection.
Takeaways & Limitations
Variation between image pairs makes feature-point registration difficult, while current models can misidentify greenhouses and small reservoirs as building changes.
Abstract
from arXiv · showhide
Building-change detection underpins many important applications, especially in the military and crisis-management domains. Recent methods used for change detection have shifted towards deep learning, which depends on the quality of its training data. The assembly of large-scale annotated satellite imagery datasets is therefore essential for global building-change surveillance. Existing datasets almost exclusively offer near-nadir viewing angles. This limits the range of changes that can be detected. By offering larger observation ranges, the scroll imaging mode of optical satellites presents an opportunity to overcome this restriction. This paper therefore introduces S2Looking, a building-change-detection dataset that contains large-scale side-looking satellite images captured at various off-nadir angles. The dataset consists of 5000 bitemporal image pairs of rural areas and more than 65,920 annotated instances of changes throughout the world. The dataset can be used to train deep-learning-based change-detection algorithms. It expands upon existing datasets by providing (1) larger viewing angles; (2) large illumination variances; and (3) the added complexity of rural images. To facilitate {the} use of the dataset, a benchmark task has been established, and preliminary tests suggest that deep-learning algorithms find the dataset significantly more challenging than the closest-competing near-nadir dataset, LEVIR-CD+. S2Looking may therefore promote important advances in existing building-change-detection algorithms. The dataset is available at https://github.com/S2Looking/.
1. Introduction
Building-change detection supports applications including military surveillance and crisis management, but existing datasets are dominated by near-nadir imagery. S2Looking addresses this gap with a large-scale, globally distributed side-looking rural dataset and benchmark for deep-learning change detection.
- Deep-learning change-detection performance depends heavily on training-dataset scale, quality, and completeness, motivating large-scale annotated datasets.
- Near-nadir datasets provide relatively simple observed building features, limiting change-detection algorithms’ coverage of practical applications.
- Scroll imaging captures multiple views of the same object from different angles, offering side-looking imagery with more ground-object detail than overhead-only imagery.
- Unlike SpaceNet MVOI, S2Looking focuses on bitemporal building-change detection rather than multiangle imagery at one location.
- S2Looking contains 5000 bitemporal pairs of 0.5∼0.8 m/pixel rural images and 65,920 expert-based change annotations collected from globally distributed areas.
- The paper contributes a dataset-construction pipeline, a side-looking rural benchmark, and a preliminary evaluation of the dataset’s added complexity.
2. Materials and Methods
S2Looking is designed to extend building-change detection beyond near-nadir imagery by providing globally distributed rural, side-looking satellite data and a processing pipeline. Its off-nadir geometry, rural context, and registration challenges create a more complex benchmark for change-detection models.
- Dataset scope: S2Looking expands change-detection datasets with rural, globally distributed imagery captured at varying large off-nadir angles.It targets limitations of near-nadir datasets and differs from LEVIR-CD+ in its rural coverage and side-looking acquisition.
- Acquisition challenges: Side-looking acquisition improves imaging range and revisit opportunities but introduces geometric inconsistency and displacement for tall objects across viewing angles.These effects complicate change detection because the same structures may appear differently in bitemporal images.
- Dataset characteristics: The dataset includes 71.9% on-nadir and 28.1% off-nadir image pairs, with off-nadir angles ranging from −35° to +40°.The average absolute off-nadir angle is 9.86°, with a standard deviation of 12.197°.
- Data processing: Construction selects repeatedly imaged rural locations, retains bitemporal pairs with IoU greater than 0.7 and time spans greater than 1 year, then crops coordinate-based slices.Intersecting areas are cropped into 3000 × 3000 slices and resized to 1024 × 1024 for GPU-oriented processing.
- Challenge design: The dataset’s rural, side-looking imagery supports analysis of subtle building changes but requires models to distinguish them from directional projection effects and unrelated land-cover variation.Examples of unrelated structures include greenhouses and small reservoirs, while rural land-use changes can be more subtle than simple construction or destruction.
3. Results
The benchmark evaluates classic and state-of-the-art deep-learning methods using Precision, Recall, and F1-score on LEVIR-CD+ and S2Looking. S2Looking is substantially more difficult, particularly because of its side-looking imagery.
- Benchmark setup: Nine deep-learning methods were evaluated on both LEVIR-CD+ and S2Looking, including four classic and five state-of-the-art methods.The state-of-the-art methods were selected because they had previously performed well on LEVIR-CD+.
- Evaluation metrics: The benchmark takes three-channel multispectral image pairs as input and produces a single-channel prediction map compared with pixel-precise ground-truth labels.Performance is evaluated using Precision, Recall, and F1-score.
- Evaluation metrics: Precision measures exclusion of irrelevant structures, Recall measures identification of changed regions, and F1-score provides an overall evaluation.Higher F1-score values indicate better prediction results.
- Benchmark results: Each algorithm required more epochs to converge and obtained lower F1-scores on S2Looking than on LEVIR-CD+.The evaluation results are reported in Table 3 and visualized in Figures 8 and 9.
- Benchmark results: The evaluated methods’ F1-scores on S2Looking were at least 25% lower than on LEVIR-CD+, confirming its greater difficulty.This comparison establishes S2Looking as a more challenging benchmark than LEVIR-CD+.
- On-nadir analysis: On the ±15° on-nadir subset, F1-scores were about 46.4% greater than the overall-dataset averages, identifying side-looking imagery as the principal difficulty.The subset contained 3597 image pairs, with results reported in Table 4.
4. Discussion
Benchmarking shows that S2Looking is substantially more difficult than LEVIR-CD+, exposing weaknesses in existing change-detection models. BiT performed best, but side-looking effects, registration errors, and complex rural scenes still caused errors.
- Benchmark findings: Comparisons on S2Looking reveal weaknesses in how existing algorithms handle side-looking effects and related dataset challenges.The benchmark is intended to provide insights into potential model improvements.
- FC-Net: FC-Net performed poorly on S2Looking because its simple structure struggled with the dataset’s complex problems.Its limited feature-representation capacity led predictions to follow image contrast rather than whole building structures.
- DTCDSCN: DTCDSCN better identified changed regions, building-boundary changes, side-looking effects, and shadows than FC-Net, but confused some rural structures with changed buildings.It missed small prefabricated houses and misrecognized greenhouses, cultivated land, and hardened ground.
- STANet: STANet-PAM outperformed STANet-BAM and STANet-Base on F1-score across both datasets, but performed worse than DTCDSCN on S2Looking.The authors attribute this pattern to greater vulnerability to side-looking effects and illumination differences.
- CDNet: CDNet produced better detection results than previous methods on both datasets, while remaining vulnerable to small structures and large appearance differences.Its deep feature extractor handled moderate illumination variances and small registration errors, but its pixelwise discrimination process was less suited to large variations.
- BiT: BiT outperformed all other methods on both datasets, although side-looking boundary expansion and lower registration accuracy still caused incorrect predictions.Only small prefabricated houses on hills were specifically reported as misrecognized.
- Overall assessment: More sophisticated models are needed for S2Looking, although the evaluated methods were generally robust to seasonal and illumination variations.The authors characterize building-change detection on the dataset as solvable.
5. Conclusions
The paper introduces S2Looking as a side-looking rural building-change dataset and benchmark, finding that contemporary methods perform much worse on it than on LEVIR-CD+. The authors identify registration and model-recognition problems as priorities for future refinement.
- Contributions: S2Looking assembles a large collection of bitemporal side-looking rural satellite-image pairs for building-change detection and algorithm benchmarking.The dataset uses the camera-scroll capability of modern optical satellites.
- Benchmark conclusions: Contemporary change-detection methods performed as much as 25% lower in F1-score on S2Looking than on LEVIR-CD+.The comparison assessed the added challenge posed by S2Looking.
- Implications: Benchmark analysis identified potential weaknesses in existing change-detection pipelines that may inspire further developments.
- Future improvements: Registration is constrained by variation between captured image pairs, making feature-point matching difficult and limiting model performance.The authors suggest that improved registration techniques could enhance models trained on S2Looking.
- Future improvements: Current change-detection models can misidentify irrelevant structures such as greenhouses and small reservoirs as building changes.The authors frame refinement of both the dataset and existing methodologies as an open direction.