PaperlayerView exact revision on arXiv

Source-linked AI summary

A Transformer-Based Siamese Network for Change Detection

Wele Gedara Chaminda Bandara, Vishal M. Patel

arXiv:2201.01293v7cs.CV

TL;DR

Change detection must identify relevant changes in paired remote-sensing images despite nuisance variation, while prior ConvNet-based methods struggle with long-range spatial-temporal details. ChangeFormer uses a Siamese hierarchical transformer, learned multi-scale feature differences, and a lightweight MLP decoder; it outperforms existing methods on two datasets.

  • Problem

    Change detection requires recognizing relevant changes between co-registered multi-temporal images while handling irrelevant variations and limitations in ConvNet-based long-range context modeling.

    Source: §1 · p. 1

  • Method

    ChangeFormer combines a Siamese hierarchical transformer encoder, learned multi-scale feature differences, and a lightweight MLP decoder for change-map prediction.

    Source: §1 · p. 1, §2.1.3 · p. 3

  • Results

    ChangeFormer achieves better F1, IoU, and OA performance than existing state-of-the-art methods on LEVIR-CD and DSIFN-CD.

    Source: §4 · p. 4

  • Takeaways & Limitations

    A hierarchical transformer with a lightweight decoder can perform well for change detection without depending on deep ConvNets.

    Source: §5 · p. 4

    ConvNet-based attention methods struggle to relate long-range spatial-temporal details because they reweight ConvNet-derived bi-temporal features.

    Source: §1 · p. 1

Abstract

from arXiv · show

This paper presents a transformer-based Siamese network architecture (abbreviated by ChangeFormer) for Change Detection (CD) from a pair of co-registered remote sensing images. Different from recent CD frameworks, which are based on fully convolutional networks (ConvNets), the proposed method unifies hierarchically structured transformer encoder with Multi-Layer Perception (MLP) decoder in a Siamese network architecture to efficiently render multi-scale long-range details required for accurate CD. Experiments on two CD datasets show that the proposed end-to-end trainable ChangeFormer architecture achieves better CD performance than previous counterparts. Our code is available at https://github.com/wgcban/ChangeFormer.

1. INTRODUCTION

Change detection seeks relevant changes between co-registered images while rejecting nuisance variation. ChangeFormer addresses limits of ConvNet-based context modeling with a hierarchical transformer encoder and lightweight MLP decoder.

  • Change detection identifies relevant changes between co-registered images acquired at distinct times.Source: §1 · p. 1
  • A strong CD model must distinguish meaningful object and environmental changes from seasonal, shadow, atmospheric, and illumination variations.Source: §1 · p. 1
  • Existing CD methods mainly use ConvNets, adding stacked, dilated, or attention-based layers to enlarge receptive fields and capture long-range context.Source: §1 · p. 1
  • ConvNet-based attention reweights bi-temporal features but struggles to relate long-range spatial-temporal details globally.Source: §1 · p. 1
  • ChangeFormer replaces the required ConvNet dependency with a hierarchical transformer encoder and lightweight MLP decoder for CD.Source: §1 · p. 1

2. METHOD

ChangeFormer processes paired images through a Siamese hierarchical transformer and predicts change masks from learned multi-scale feature differences.

  • ChangeFormer uses a Siamese hierarchical transformer encoder to extract coarse and fine features from bi-temporal images.Source: §2 · p. 1
  • Four feature difference modules compute multi-scale differences, which a lightweight MLP decoder fuses to predict the change mask.Source: §2 · p. 1

2.1. Hierarchical Transformer Encoder

The hierarchical encoder supplies multi-level features for change detection while reducing self-attention cost and adding positional information. Difference modules then learn scale-specific distances between pre-change and post-change representations.

Hierarchical Transformer Encoder

  • The encoder generates ConvNet-like multi-level features from bi-temporal inputs for subsequent change-difference processing.Source: §2.1 · p. 1
  • Its transformer encoder outputs feature maps at progressively changing resolutions and channel dimensions for multi-scale processing.Source: §2.1 · p. 2

Transformer Block

  • Self-attention forms the transformer encoder’s main building block.Source: §2.1.1 · p. 2
  • Sequence reduction lowers self-attention complexity from O((HW)^2) to O((HW)^2/R) by reducing the sequence length with ratio R.Source: §2.1.1 · p. 2
  • The positional encoding uses MLP layers and a 3 × 3 depth-wise convolution, allowing test images with resolutions different from training images.Source: §2.1.1 · p. 2

Hierarchical Transformer Encoder

  • Downsampling progressively produces lower-resolution feature maps with increased channel capacity across transformer layers.Source: §2.1.2 · p. 2

Difference Module

  • Four Difference Modules compute differences between multi-level pre-change and post-change features using convolution, ReLU, and batch normalization.Source: §2.1.3 · pp. 2–3
  • Unlike absolute differencing, the proposed Difference Module learns an optimal distance metric at each scale during training.Source: §2.1.3 · p. 3

2.2. MLP Decoder

The lightweight MLP decoder aggregates multi-scale feature differences, progressively aligns and fuses them, then upsamples the fused representation to predict the change mask.

  • Each multi-scale feature difference map is processed by an MLP to unify its channel dimension and upsample it to H/4×W/4.Source: §2.2.1 · p. 3
  • The aligned feature difference maps are concatenated and fused through another MLP layer.Source: §2.2.2 · p. 3
  • The fused feature map is upsampled to H × W using a 2D transposed convolution with S = 4 and K = 3.Source: §2.2.3 · p. 3
  • A final MLP layer predicts the change mask at H × W × Ncls resolution, with Ncls = 2 for change and no-change.Source: §2.2.3 · p. 3

3. EXPERIMENTAL SETUP

Experiments use LEVIR-CD and DSIFN-CD with defined patching and train/validation/test splits, while training applies augmentation, CE loss, and AdamW optimization.

Datasets

  • Experiments use two public CD datasets: LEVIR-CD and DSIFN-CD.Source: §3.1 · p. 3
  • LEVIR-CD contains 1024 × 1024 building-image pairs, cropped into 256 × 256 patches with 7120/1024/2048 train/val/test samples.Source: §3.1 · p. 3
  • DSIFN-CD contains changes across landcover objects and yields 14400/1360/192 train/val/test samples from 256 × 256 patches.Source: §3.1 · p. 3

Training

  • Training uses augmentation, Cross-Entropy loss, and AdamW with weight decay 0.01 and initial learning rate 0.0001.Source: §3.2 · p. 3

Evaluation

  • Performance is evaluated primarily with change-class F1 and IoU, alongside precision, recall, and overall accuracy.Source: §3.3 · p. 3

4. RESULTS AND DISCUSSION

ChangeFormer is compared with convolutional, attention-based, Siamese, and transformer-based methods on two datasets using quantitative tables and qualitative visual results.

Compared methods

  • The comparison includes FC-EF, FC-Siam-Di, FC-Siam-Conc, DTCDSCN, STANet, IFNet, SNUNet, and BIT.Source: §4 · pp. 3–4

Quantitative results

  • Table 1 reports average quantitative results for different CD methods on LEVIR-CD and DSIFN-CD.Source: Table 1 · p. 4

Qualitative results

  • Figure 2 presents qualitative results on LEVIR-CD and DSIFN-CD for visual comparison across methods.Source: Figure 2 · p. 4

Quantitative results

  • ChangeFormer improves previous SOTA F1/IoU/OA by 1.2/2.2/0.1% on LEVIR-CD and 20.0/44.3/6.4% on DSIFN-CD.Source: §4 · p. 4

Qualitative results

  • The authors report that ChangeFormer captures finer details than the other compared methods in the visual results.Source: §4 · p. 4

5. CONCLUSION

The paper concludes that a hierarchical transformer encoder in a Siamese network with a lightweight MLP decoder can perform change detection without relying on deep ConvNets.

  • The proposed transformer-based Siamese network uses a hierarchical transformer encoder and a simple MLP decoder for change detection.Source: §5 · p. 4
  • The method outperforms several recent ConvNet-, attention-, and ConvNet+Transformer-based CD methods on IoU, F1 score, and overall accuracy.Source: §5 · p. 4
  • The conclusion states that deep-ConvNet dependence is unnecessary for achieving strong CD performance with this architecture.Source: §5 · p. 4

Related papers

Loading 2201.01293v7…