Source-linked AI summary
A Transformer-Based Siamese Network for Change Detection
Wele Gedara Chaminda Bandara, Vishal M. Patel
TL;DR
Change detection must identify relevant changes in paired remote-sensing images despite nuisance variation, while prior ConvNet-based methods struggle with long-range spatial-temporal details. ChangeFormer uses a Siamese hierarchical transformer, learned multi-scale feature differences, and a lightweight MLP decoder; it outperforms existing methods on two datasets.
Problem
Change detection requires recognizing relevant changes between co-registered multi-temporal images while handling irrelevant variations and limitations in ConvNet-based long-range context modeling.
Method
ChangeFormer combines a Siamese hierarchical transformer encoder, learned multi-scale feature differences, and a lightweight MLP decoder for change-map prediction.
Results
ChangeFormer achieves better F1, IoU, and OA performance than existing state-of-the-art methods on LEVIR-CD and DSIFN-CD.
Takeaways & Limitations
A hierarchical transformer with a lightweight decoder can perform well for change detection without depending on deep ConvNets.
Takeaways & Limitations
ConvNet-based attention methods struggle to relate long-range spatial-temporal details because they reweight ConvNet-derived bi-temporal features.
Abstract
from arXiv · showhide
This paper presents a transformer-based Siamese network architecture (abbreviated by ChangeFormer) for Change Detection (CD) from a pair of co-registered remote sensing images. Different from recent CD frameworks, which are based on fully convolutional networks (ConvNets), the proposed method unifies hierarchically structured transformer encoder with Multi-Layer Perception (MLP) decoder in a Siamese network architecture to efficiently render multi-scale long-range details required for accurate CD. Experiments on two CD datasets show that the proposed end-to-end trainable ChangeFormer architecture achieves better CD performance than previous counterparts. Our code is available at https://github.com/wgcban/ChangeFormer.
1. INTRODUCTION
Change detection seeks relevant changes between co-registered images while rejecting nuisance variation. ChangeFormer addresses limits of ConvNet-based context modeling with a hierarchical transformer encoder and lightweight MLP decoder.
- Change detection identifies relevant changes between co-registered images acquired at distinct times.
- A strong CD model must distinguish meaningful object and environmental changes from seasonal, shadow, atmospheric, and illumination variations.
- Existing CD methods mainly use ConvNets, adding stacked, dilated, or attention-based layers to enlarge receptive fields and capture long-range context.
- ConvNet-based attention reweights bi-temporal features but struggles to relate long-range spatial-temporal details globally.
- ChangeFormer replaces the required ConvNet dependency with a hierarchical transformer encoder and lightweight MLP decoder for CD.
2. METHOD
ChangeFormer processes paired images through a Siamese hierarchical transformer and predicts change masks from learned multi-scale feature differences.
- ChangeFormer uses a Siamese hierarchical transformer encoder to extract coarse and fine features from bi-temporal images.
- Four feature difference modules compute multi-scale differences, which a lightweight MLP decoder fuses to predict the change mask.
2.1. Hierarchical Transformer Encoder
The hierarchical encoder supplies multi-level features for change detection while reducing self-attention cost and adding positional information. Difference modules then learn scale-specific distances between pre-change and post-change representations.
- Hierarchical Transformer Encoder: The encoder generates ConvNet-like multi-level features from bi-temporal inputs for subsequent change-difference processing.
- Hierarchical Transformer Encoder: Its transformer encoder outputs feature maps at progressively changing resolutions and channel dimensions for multi-scale processing.
- Transformer Block: Self-attention forms the transformer encoder’s main building block.
- Transformer Block: Sequence reduction lowers self-attention complexity from O((HW)^2) to O((HW)^2/R) by reducing the sequence length with ratio R.
- Transformer Block: The positional encoding uses MLP layers and a 3 × 3 depth-wise convolution, allowing test images with resolutions different from training images.
- Hierarchical Transformer Encoder: Downsampling progressively produces lower-resolution feature maps with increased channel capacity across transformer layers.
- Difference Module: Four Difference Modules compute differences between multi-level pre-change and post-change features using convolution, ReLU, and batch normalization.
- Difference Module: Unlike absolute differencing, the proposed Difference Module learns an optimal distance metric at each scale during training.
2.2. MLP Decoder
The lightweight MLP decoder aggregates multi-scale feature differences, progressively aligns and fuses them, then upsamples the fused representation to predict the change mask.
- Each multi-scale feature difference map is processed by an MLP to unify its channel dimension and upsample it to H/4×W/4.
- The aligned feature difference maps are concatenated and fused through another MLP layer.
- The fused feature map is upsampled to H × W using a 2D transposed convolution with S = 4 and K = 3.
- A final MLP layer predicts the change mask at H × W × Ncls resolution, with Ncls = 2 for change and no-change.
3. EXPERIMENTAL SETUP
Experiments use LEVIR-CD and DSIFN-CD with defined patching and train/validation/test splits, while training applies augmentation, CE loss, and AdamW optimization.
- Datasets: Experiments use two public CD datasets: LEVIR-CD and DSIFN-CD.
- Datasets: LEVIR-CD contains 1024 × 1024 building-image pairs, cropped into 256 × 256 patches with 7120/1024/2048 train/val/test samples.
- Datasets: DSIFN-CD contains changes across landcover objects and yields 14400/1360/192 train/val/test samples from 256 × 256 patches.
- Training: Training uses augmentation, Cross-Entropy loss, and AdamW with weight decay 0.01 and initial learning rate 0.0001.
- Evaluation: Performance is evaluated primarily with change-class F1 and IoU, alongside precision, recall, and overall accuracy.
4. RESULTS AND DISCUSSION
ChangeFormer is compared with convolutional, attention-based, Siamese, and transformer-based methods on two datasets using quantitative tables and qualitative visual results.
- Compared methods: The comparison includes FC-EF, FC-Siam-Di, FC-Siam-Conc, DTCDSCN, STANet, IFNet, SNUNet, and BIT.
- Quantitative results: Table 1 reports average quantitative results for different CD methods on LEVIR-CD and DSIFN-CD.
- Qualitative results: Figure 2 presents qualitative results on LEVIR-CD and DSIFN-CD for visual comparison across methods.
- Quantitative results: ChangeFormer improves previous SOTA F1/IoU/OA by 1.2/2.2/0.1% on LEVIR-CD and 20.0/44.3/6.4% on DSIFN-CD.
- Qualitative results: The authors report that ChangeFormer captures finer details than the other compared methods in the visual results.
5. CONCLUSION
The paper concludes that a hierarchical transformer encoder in a Siamese network with a lightweight MLP decoder can perform change detection without relying on deep ConvNets.
- The proposed transformer-based Siamese network uses a hierarchical transformer encoder and a simple MLP decoder for change detection.
- The method outperforms several recent ConvNet-, attention-, and ConvNet+Transformer-based CD methods on IoU, F1 score, and overall accuracy.
- The conclusion states that deep-ConvNet dependence is unnecessary for achieving strong CD performance with this architecture.