PaperlayerView exact revision on arXiv

Source-linked AI summary

Super-resolution-based Change Detection Network with Stacked Attention Module for Images with Different Resolutions

Mengxi Liu, Qian Shi, Andrea Marinoni, Da He, Xiaoping Liu, Liangpei Zhang

arXiv:2103.00188v1eess.IVcs.CVcs.LG

TL;DR

Change detection across bi-temporal images with different resolutions remains challenging for high-resolution imagery, where traditional subpixel-based methods can accumulate errors. SRCDNet combines super-resolution with stacked attention and metric learning, and outperformed baselines on the reported initial-HR and resolution-difference experiments.

  • Problem

    More precise change detection is needed for high-resolution images at different spatial resolutions because traditional subpixel-based methods can accumulate errors.

    Source: §1 · p. 3

  • Method

    SRCDNet combines GAN-based super-resolution, Siamese multi-scale feature extraction with stacked attention, and deep metric learning to generate change maps.

    Source: §3.1 · p. 5, §7 · p. 24

  • Results

    SRCDNet achieved the best performance among baseline methods on initial-HR images and the highest accuracies in 4× and 8× resolution-difference experiments, with F1 scores of 87.40% on BCDD and 92.94% on CDD.

    Source: §1 · p. 4

  • Takeaways & Limitations

    When bi-temporal images have equal resolution, SRCDNet can omit its super-resolution module and reduce to CDNet, which the authors say improves model generality.

    Source: §3.1 · p. 7

    The described setup compares HR imagery at T1 with LR imagery at T2, with resolution differences of 4× or 8×.

    Source: §3.1 · p. 6

Abstract

from arXiv · show

Change detection, which aims to distinguish surface changes based on bi-temporal images, plays a vital role in ecological protection and urban planning. Since high resolution (HR) images cannot be typically acquired continuously over time, bi-temporal images with different resolutions are often adopted for change detection in practical applications. Traditional subpixel-based methods for change detection using images with different resolutions may lead to substantial error accumulation when HR images are employed; this is because of intraclass heterogeneity and interclass similarity. Therefore, it is necessary to develop a novel method for change detection using images with different resolutions, that is more suitable for HR images. To this end, we propose a super-resolution-based change detection network (SRCDNet) with a stacked attention module. The SRCDNet employs a super resolution (SR) module containing a generator and a discriminator to directly learn SR images through adversarial learning and overcome the resolution difference between bi-temporal images. To enhance the useful information in multi-scale features, a stacked attention module consisting of five convolutional block attention modules (CBAMs) is integrated to the feature extractor. The final change map is obtained through a metric learning-based change decision module, wherein a distance map between bi-temporal features is calculated. The experimental results demonstrate the superiority of the proposed method, which not only outperforms all baselines -with the highest F1 scores of 87.40% on the building change detection dataset and 92.94% on the change detection dataset -but also obtains the best accuracies on experiments performed with images having a 4x and 8x resolution difference. The source code of SRCDNet will be available at https://github.com/liumency/SRCDNet.

2. Dept. of Physics and Technology, UiT the Arctic University of Norway, Tromsø, Norway · 1. Introduction

SRCDNet addresses change detection between high- and low-resolution images by learning a super-resolved image, enhancing features with stacked attention, and comparing temporal features. The paper reports that it outperformed baselines on two datasets and in experiments with 4× and 8× resolution differences.

1. Introduction

  • Change detection identifies surface changes between bi-temporal images and supports applications including land and resource investigation, ecological monitoring and protection, and urban planning.Remote-sensing imagery is used in these applications for its broad coverage and high temporal efficiency.Source: §1 · p. 2
  • Deep learning methods use CNNs to extract hierarchical spatial and semantic features from HR images, including classification-based and metric-learning-based approaches.Earlier change-detection methods such as CVA and PCA mainly exploit spectral information, while multispectral satellite imagery often has low spatial resolution.Source: §1 · p. 2
  • Different-resolution imagery is often needed when HR images are unavailable at every time point, while downsampling loses spatial detail and conventional LR interpolation lacks semantic information.These conditions motivate change detection using bi-temporal images with different resolutions for large-scale applications.Source: §1 · p. 2
  • Subpixel-based methods, including approaches by Ling et al., Wang et al., Li et al., and Wu et al., map coarse imagery to fine-resolution change maps but depend on prior fine-resolution land-cover maps.That dependence can accumulate errors, a problem the paper says is more severe for HR images because of intraclass heterogeneity and interclass similarity.Source: §1 · p. 3
  • SRCDNet learns an SR image from LR imagery to address temporal resolution mismatch, then compares multiscale bi-temporal features with a contrastive-loss distance map and thresholds it for change decisions.The paper presents the network as an end-to-end approach intended to avoid error accumulation in traditional subpixel-based methods.Source: §1 · p. 3
  • A stacked attention module of five CBAM blocks enhances useful hierarchical features, aiming to make feature pairs more distinguishable for metric-learning-based change decisions.Source: §1 · p. 3
  • F1 scores of 87.40% on BCDD and 92.94% on CDD were the best among baseline methods, and SRCDNet achieved the highest accuracies in the 4× and 8× resolution-difference experiments.The experiments also considered the initial HR images.Source: §1 · p. 4

2. Related Work

Related work reviews FCN/U-Net and metric-learning approaches to change detection, RNN and attention strategies for HR imagery, and deep-learning methods for super-resolution. It highlights limitations involving HR-image heterogeneity, limited spectral and temporal information, and attention to shallow features.

2.1. Deep learning–based change detection

  • FCN-based methods, especially U-Net variants, use encoder–decoders for pixel-wise change detection, with early fusion, Siamese inputs, and multi-scale feature fusion among explored designs.Daudt et al. examined early fusion, Siamese difference, and Siamese concatenation; Peng et al. proposed U-Net++ with multi-scale feature fusion.Source: §2.1 · p. 4
  • Metric-learning methods replace decoder upsampling with feature-distance maps, minimizing distances for unchanged pairs and maximizing them for changed pairs.Reported losses include improved triplet loss, contrastive loss (Zhang et al. 2018), and batched contrastive loss for class imbalance.Source: §2.1 · p. 4

2.2. Change detection strategies

  • RNNs and LSTMs capture spectral or temporal relationships, but infrequent spectral information in HR images and limited temporal information in bi-temporal images constrain their use.Examples include LSTM blocks in an FCN and convolutional LSTM with a 3D FCN for joint spectral–spatial–temporal features.Source: §2.2 · pp. 4–5
  • Attention methods strengthen spatial-temporal relations or emphasize change information, but memory limits have generally left shallow-feature enhancement unaddressed.Prior examples include self-attention and dual CBAMs.Source: §2.2 · p. 5

2.3. Super resolution

  • Deep-learning super-resolution progressed from CNN-based SRCNN to VDSR and DRCN, with SRGAN applying GANs to reconstruction and achieving state-of-the-art performance at the time.The cited progression includes SRCNN, VDSR, DRCN, and SRGAN, building on GANs.Source: §2.3 · p. 5

3. Methodology

SRCDNet addresses differing temporal-image resolutions by reconstructing the lower-resolution image before change detection. Its stacked attention and metric-learning components refine multi-scale features and produce the change map.

3.1. Overview

  • For HR-at-T1/LR-at-T2 inputs with N=4 or 8, SRCDNet reconstructs T2 through a GAN before shared feature extraction and metric-learning change mapping.When the temporal images have equal spatial resolution, the SR module can be removed, reducing SRCDNet to a change detection network.Source: §3.1 · pp. 5–7

3.2. Super resolution module

  • The generator produces SR images from LR inputs; adversarial training against a discriminator distinguishing SR from HR images encourages outputs more similar to original HR images.The generator uses five residual blocks and M sub-pixel convolution layers, where M = log2 N. The discriminator inherits VGG-19 structure.Source: §3.2 · p. 7

3.3. Change detection module

  • A Siamese ResNet-18 extractor uses five CBAMs to refine and fuse multi-scale features; Euclidean distances between refined temporal features are contrastively trained for threshold-based change mapping.Four CBAMs refine residual-block outputs before fusion, and a fifth refines feature pairs; training increases distances for changed areas and reduces them for unchanged areas.Source: §3.3 · pp. 7–8

3.4. Loss function

  • Training optimizes the generator, discriminator, and change network, with generator optimization following the discriminator and change network.The generated LR-derived image is passed successively to the discriminator and change detection network.Source: §3.4 · p. 9

3.4.1. Discriminator

  • The discriminator is trained to assign HR images probabilities near 1 and SR images probabilities near 0.It classifies the generator’s SR output against the HR image.Source: §3.4.1 · p. 9

3.4.2. Change detection module

  • The change network applies batch contrastive loss to its CBAM-enhanced Euclidean distance map, minimizing distances for unchanged areas and maximizing them for changed areas.The final change map is obtained by threshold segmentation of the distance map.Source: §3.4.2 · p. 9

3.4.3. Generator

  • The generator combines image, content, adversarial, and change losses to promote pixel-wise and perceptual similarity while improving change detection.Image loss uses mean squared error, content loss compares pretrained VGG-19 feature maps, and the change loss is the batch contrastive loss.Source: §3.4.3 · pp. 9–10

4. Settings

Experiments used two real change-detection datasets, five baseline methods, three simulated resolution-difference settings, and standard training and evaluation procedures.

4.1. Dataset

  • BCDD provides 0.2-m pre/post-earthquake image pairs; 7,434 nonoverlapping 256×256 patches were split 8:1:1 for training, validation, and testing, with training rotations for augmentation.The dataset covers building reconstruction and renewal.Source: §4.1 · p. 10
  • CDD contains 16,000 season-varying 256×256 Google Earth image pairs, split into 10,000 training, 3,000 validation, and 3,000 testing samples, with 3–100 cm spatial resolution.Its changes include common objects such as buildings, roads, and forests, as well as detailed objects such as cars and tanks.Source: §4.1 · pp. 10–11

4.2. Baselines

  • Five comparison baselines comprised FC-EF, FC-Siam-diff, FC-Siam-conc (Daudt et al. 2018), BiDateNet, and STANet.FC-EF fuses the image pair; the Siamese variants use feature differences or concatenation; BiDateNet models temporal dependencies with LSTM blocks; STANet uses spatial-temporal attention.Source: §4.2 · pp. 11–12

4.3. Comparative experiments

  • Comparative experiments tested equal-resolution inputs (X1), fourfold downsampling of T2 followed by bicubic resizing (X4), and eightfold downsampling followed by similar processing (X8).X8 was used to test performance under a large spatial difference; X1 tests the model as a common change-detection model.Source: §4.3 · p. 12

4.4. Implementations

  • SRCDNet training used PyTorch, 100 Adam epochs at an initial learning rate of 0.0001, batch size eight, and generator-loss weights α=0.006, β=0.001, and λ=0.001.Baselines ran on a GeForce RTX 2080ti; performance was measured with precision, recall, F1-score, and IoU.Source: §4.4 · pp. 12–13

5. Experiments and Analysis

Across X1, X4, and X8 tests, SRCDNet records the strongest reported detection results; visual comparisons describe more precise boundaries and recovery of small changes as resolution differences widen.

5.1. Performance on X1 experiments

  • 87.40% F1 and 77.63% IoU on BCDD, and 92.94% F1 and 86.81% IoU on CDD, were SRCDNet’s highest X1 scores.On BCDD, SRCDNet also had 90.13% recall and 84.84% precision; STANet ranked second with 84.96% F1 and 73.86% IoU. On CDD, STANet scored 91.44% F1 and 84.23% IoU.Source: §5.1 · p. 13
  • On X1, SRCDNet produced precise building footprints, detected a building reduction, and extracted small changes while maintaining their boundaries and shapes.On CDD, STANet extracted more small changes but showed spillover; SRCDNet’s visual results were more precise.Source: §5.1 · p. 14

5.2. Performance on X4 experiments

  • On X4, SRCDNet restored building boundaries better than bicubic interpolation, reduced pseudo-changes, and detected detailed changes such as alleys and cars that comparison methods rarely detected.The passages attribute the small decline in BCDD accuracy partly to large buildings being mitigated by 4× bicubic interpolation; detailed CDD changes were harder to reconstruct.Source: §5.2 · p. 15

5.3. Performance on X8 experiments

  • 81.69% F1 and 69.05% IoU on BCDD, and 83.32% F1 and 71.40% IoU on CDD, were SRCDNet’s highest reported X8 scores.On CDD, SRCDNet’s F1 and IoU were 6.70% and 10.46% lower than in X4.Source: §5.3 · p. 17
  • On X8, SRCDNet produced more precise building boundaries and more complete change results, recovering building information and small changes often missed in bicubic-based comparisons.Missed or incomplete detections on CDD included alleys, cars, and some building changes after 8× downsampling.Source: §5.3 · p. 17

6. Discussion

Across 4× and 8× resolution differences, combining the stacked attention and super-resolution modules produced the strongest change-detection results. The SRM also improved restored-image quality, with the gains varying by resolution difference.

6.1. Ablation study with 4× resolution difference

  • At 4×, SRCDNet achieved the highest F1 on BCDD (85.66%) and CDD (90.02%), versus base-model lows of 77.53% and 86.56%, respectively.Adding SAM alone yielded F1 values of 84.07% on BCDD and 88.77% on CDD; adding SRM alone yielded 79.81% and 87.53%, respectively. Qualitatively, the combined model produced the most precise change maps among the baselines.Source: Table 2 · p. 19, §6.1 · pp. 19–20

6.2. Ablation study with 8× resolution difference

  • At 8× on BCDD, SRCDNet reached F1 81.69% and IoU 69.05%, above SAM-only F1 77.22%, SRM-only F1 75.17%, and base F1 72.26%.Qualitatively, models with SRM better captured building boundaries and alleviated missed alarms on BCDD.Source: §6.2 · p. 20, Figure 7 · p. 22
  • At 8× on CDD, SRCDNet achieved the highest F1 (83.24%); SRM-only F1 was 80.59%, SAM-only was 79.91%, and base was 77.99%.Unlike the previous experiments, SRM integration outperformed SAM integration on CDD, possibly because it recovered image information useful for extracting small changes; SRM models also yielded more comprehensive change results.Source: §6.2 · p. 20, Figure 7 · p. 22

6.3. Comparisons on different restored images

  • Across the restoration comparisons, bicubic images had the lowest PSNR and SSIM, while SRCDNet-generated SR images had the highest values, slightly exceeding Base+SRM.These comparisons used PSNR and SSIM to evaluate restored images; the results support a positive effect from combining SAM and SRM.Source: §6.3 · p. 23, Table 3 · p. 23
  • SRM integration increased PSNR and SSIM more in X4 than X8 experiments, indicating greater image-information loss and restoration difficulty with the larger resolution difference.The passage attributes the larger X4 metric increase to more image information being recovered through SRM than in X8 experiments.Source: §6.3 · p. 23

7. Conclusion · of remote sensing, vol. 10, pp. 989--1003, 1989.

SRCDNet combines super-resolution, multi-scale Siamese feature extraction, stacked attention, and deep metric learning for change detection between bi-temporal images at different scales. Comparative experiments on BCDD and CDD found it performed best against the SOTA baselines on same-resolution images and outperformed other methods under different resolutions.

7. Conclusion

  • SRCDNet achieved the best results among SOTA baselines on same-resolution images and outperformed other comparative methods for change detection under different resolutions.Comparative experiments were conducted on BCDD and CDD.Source: §7 · p. 24
  • Its super-resolution module uses a generator and discriminator to restore LR images to HR image size and generate realistic super-resolution images.Source: §7 · p. 24
  • A Siamese feature extractor obtains multi-scale features from super-resolution and HR images, while stacked attention captures useful channel-wise and spatial information.Source: §7 · p. 24
  • SRCDNet uses deep metric learning to produce the final change map for bi-temporal images at different scales.Source: §7 · p. 24

Related papers

Loading 2103.00188v1…