PaperlayerView exact revision on arXiv

Source-linked AI summary

ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model

Hongruixuan Chen, Jian Song, Chengxi Han, Junshi Xia, Naoto Yokoya

arXiv:2404.03425v7eess.IVcs.AIcs.CV

TL;DR

Remote sensing change detection needs broader context than CNNs readily capture, while Transformer self-attention is costly on large datasets. ChangeMamba applies VMamba encoders and three spatiotemporal relationship mechanisms to binary, semantic, and building-damage detection, outperforming CNN- and Transformer-based approaches on five benchmark datasets. The paper also reports robustness to degraded data and identifies future extensions toward multimodal, time-series, and foundation-model settings.

  • Problem

    CNNs may lack broad spatial context, whereas Transformers impose high computational costs, motivating exploration of Mamba for remote sensing change detection.

  • Method

    ChangeMamba uses VMamba encoders and three Mamba-based spatiotemporal relationship mechanisms for binary, semantic, and building-damage change detection.

  • Results

    The three frameworks outperform current CNN- and Transformer-based approaches on five benchmark datasets without complex training strategies or tricks.

  • Takeaways & Limitations

    The results demonstrate the potential of Mamba architecture for accurate, efficient, and robust remote sensing change detection.

  • Takeaways & Limitations

    Future work includes adapting Mamba to remote sensing characteristics and extending it to multimodal, time-series, and foundation-model tasks.

Abstract

from arXiv · show

Convolutional neural networks (CNN) and Transformers have made impressive progress in the field of remote sensing change detection (CD). However, both architectures have inherent shortcomings: CNN are constrained by a limited receptive field that may hinder their ability to capture broader spatial contexts, while Transformers are computationally intensive, making them costly to train and deploy on large datasets. Recently, the Mamba architecture, based on state space models, has shown remarkable performance in a series of natural language processing tasks, which can effectively compensate for the shortcomings of the above two architectures. In this paper, we explore for the first time the potential of the Mamba architecture for remote sensing CD tasks. We tailor the corresponding frameworks, called MambaBCD, MambaSCD, and MambaBDA, for binary change detection (BCD), semantic change detection (SCD), and building damage assessment (BDA), respectively. All three frameworks adopt the cutting-edge Visual Mamba architecture as the encoder, which allows full learning of global spatial contextual information from the input images. For the change decoder, which is available in all three architectures, we propose three spatio-temporal relationship modeling mechanisms, which can be naturally combined with the Mamba architecture and fully utilize its attribute to achieve spatio-temporal interaction of multi-temporal features, thereby obtaining accurate change information. On five benchmark datasets, our proposed frameworks outperform current CNN- and Transformer-based approaches without using any complex training strategies or tricks, fully demonstrating the potential of the Mamba architecture in CD tasks. Further experiments show that our architecture is quite robust to degraded data. The source code will be available in https://github.com/ChenHongruixuan/MambaCD

I. INTRODUCTION

Remote sensing change detection benefits from deep learning but still faces a trade-off between CNNs’ local context and Transformers’ computational cost. ChangeMamba introduces Mamba-based frameworks for three change-detection subtasks, combining global spatial modeling with spatiotemporal feature interaction.

  • Transformer self-attention models global pixel relationships but has quadratic image-size complexity, increasing computational overhead on large remote sensing datasets.
  • The paper explores Mamba for remote sensing change detection, addressing binary, semantic, and building-damage detection tasks.
  • ChangeMamba uses VMamba encoders and three Mamba-compatible mechanisms to model global spatial context and spatiotemporal relationships across multi-temporal images.
  • The proposed frameworks achieve competitive or state-of-the-art performance on five benchmark datasets without complex training strategies or tricks.
  • CNNs struggle with long-distance dependencies, which are important for detecting sparse and scattered changed regions.

C. State Space Model

State space models map input sequences to outputs through hidden states and can scale linearly with sequence length. Mamba extends S4 with selective information processing and hardware-aware recursive computation.

  • SSM-based models map a one-dimensional input sequence to an output sequence through an N-dimensional hidden state.
  • In the continuous formulation, A governs state evolution while B and C serve as projection parameters.
  • S4 discretizes continuous state space systems using a time-scale parameter and zero-order hold transformation.
  • The discretized model can compute sequence outputs through a global convolution with a structured kernel.
  • Mamba adds a selection mechanism for filtering irrelevant information and recalling relevant information, alongside hardware-aware recursive computation.

B. Problem Statement

The paper distinguishes binary, semantic, and building-damage change detection by the information each task must predict. These tasks progress from locating change to identifying semantic transitions or post-event damage states.

  • 1) Binary Change Detection:: BCD predicts binary change or non-change maps from multi-temporal image pairs, with category-agnostic and single-category variants.
  • SCD extends BCD by identifying what changed through from-to semantic information between land-cover classes.
  • SCD additionally requires land-cover labels for the two temporal images and predicts their land-cover maps alongside binary change maps.
  • 3) Building Damage Assessment:: BDA is a one-to-many SCD task that identifies building damage locations and predicts post-event building states given pre-event building information.

C. Network Architecture

The three architectures use weight-sharing siamese VMamba encoders, while task-specific decoders process multi-level features for change and semantic outputs.

  • MambaBCD uses a weight-sharing siamese VMamba encoder to extract robust, representative features from multi-temporal input images.
  • Its tailored change decoder learns spatio-temporal relationships from multi-level features through three mechanisms before producing a binary change result.
  • MambaSCD adds semantic decoders that use encoder features to predict land-cover maps for the temporal images alongside binary change detection.
  • The semantic change information from T1 → T2 is obtained by masking the predicted semantic map with the binary change map.

3) MambaBDA:

MambaBDA adapts the shared VMamba-based design for building damage assessment by predicting pre-event building locations and classifying damage from multi-temporal features.

  • MambaBDA predicts only pre-event building location maps because BDA requires land-cover mapping for pre-event images rather than both temporal images.
  • Its change decoder learns spatio-temporal relationships from multi-temporal features to classify building damage levels.
  • The obtained building location map can support object-based post-processing of the damage classification map to improve accuracy.
  • VMamba uses a 2D cross-scan mechanism to address the challenge of modeling spatial context in noncausal image data.
  • The shared encoder has four stages that downsample inputs, model spatial context with VSS blocks, and output stage-level features for downstream decoders.

E. Task-Specific Decoders

The decoders model relationships between multi-temporal features using three Mamba-compatible mechanisms, then progressively fuse and refine features for change prediction.

  • The three mechanisms are spatio-temporal sequential modeling, cross modeling, and parallel modeling.
  • Sequential modeling orders tokens from the two temporal phases in temporal sequence.
  • Cross modeling cross-orders tokens from the two temporal phases to model their relationships.
  • Parallel modeling concatenates tokens from both temporal phases along the channel dimension before joint modeling.
  • The change decoder uses STSS blocks with three VSS blocks, then fuses current-stage outputs with previous-stage features and progressively upsamples them.

3) Semantic Decoder:

The semantic decoder progressively reconstructs land-cover maps from multi-level features, while the training objectives combine cross-entropy with Lovasz-softmax losses for task-specific outputs.

  • Semantic Decoder: The semantic decoder has four stages that model global spatial context, upsample features, and fuse them with higher-resolution lower-level maps.
  • The networks use commonly used loss functions without focal loss, deep supervision, or multi-scale training and testing strategies.
  • BCD: BCD optimization uses cross-entropy loss, with Lovasz-softmax added to address imbalance between changed and unchanged pixels.
  • SCD: SCD additionally optimizes pre-event and post-event land-cover mapping with cross-entropy losses, while also using Lovasz-softmax for sample imbalance.

3) BDA:

BDA evaluates both building localization and damage classification, using cross-entropy losses and Lovasz-softmax to address sample imbalance. The broader benchmark context includes the five datasets used for experiments.

  • BDA: BDA separates building localization from damage classification and applies cross-entropy loss to both tasks.The corresponding losses are defined for the localization and classification branches.
  • BDA: Lovasz-softmax is incorporated into the final BDA loss to address sample imbalance.
  • Datasets: The experiments use five benchmark datasets, whose information is summarized in Table I.
  • SYSU-CD: The SYSU dataset contains 20,000 pairs of 0.5 meter/pixel aerial images from Hong Kong, spanning 2007–2014.It includes 256 × 256 patches and is split into training, validation, and test sets at a 6:2:2 ratio.
  • LEVIR-CD+: LEVIR-CD+ contains 985 pairs of 0.5 meters/pixel very-high-resolution images documenting building construction changes over 5–14 years.
  • WHU-CD: WHU-CD comprises aerial imagery of Christchurch captured in 2012 and 2016 at 0.3 meters/pixel, emphasizing large and sparse building changes.

4) SECOND [20]:

The SECOND dataset supports semantic change detection across six land-cover classes using paired aerial imagery from several Chinese cities. The study evaluates models with task-specific semantic-change metrics and reports dataset information in Table I.

  • SECOND: SECOND contains 4,662 pairs of 512 × 512 aerial images at 0.5–3 meter/pixel resolution for semantic change detection.
  • SECOND: The dataset covers Hangzhou, Chengdu, and Shanghai and includes six land-cover classes, including vegetation, water, buildings, and playgrounds.
  • Dataset information: Table I summarizes the five benchmark datasets used in the experiments.
  • Evaluation: SCD performance is evaluated with OA, F1, mean IoU, and separated Kappa coefficient, with higher values preferred.
  • Evaluation: BDA evaluation separately measures building localization and damage classification using F1-based quantities.Damage levels include no damage, minor damage, major damage, and destroyed.

C. Comparison Methods

The comparison covers CNN- and Transformer-based methods across BCD, SCD, and BDA. ChangeMamba reports strong task-specific performance, lower computational cost in selected comparisons, and improved spatio-temporal modeling results.

  • Comparison setup: The study compares ChangeMamba with representative CNN and Transformer methods across BCD, SCD, and BDA.
  • BCD: MambaBCD-Base and MambaBCD-Small achieve the highest and second-highest OA, F1, IoU, and KC across three BCD datasets.The reported datasets are SYSU, LEVIR-CD+, and WHU-CD.
  • BCD: 56.9% lower GFLOPs and 0.88%, 1.33%, and 3.00% higher F1 scores are reported for MambaBCD-Tiny versus BIT-18 across the three datasets.
  • SCD: MambaSCD-Small surpasses ScanNet on all four SCD metrics without using advanced SCD-specific loss functions.
  • SCD: MambaSCD produces “from-to” transition matrices close to the ground truth and identifies semantic categories across changed regions.
  • BDA: MambaBDA exceeds representative BDA approaches, with its improvement over DamFormer mainly arising from damage classification rather than building localization.
  • BDA: Compared with DamFormer, the three MambaBDA models improve F overall by 2.64%, 4.12%, and 4.39%, respectively.Another passage reports F1 improvement from 5.64% to 9.46% for similar parameters and lower GFLOPs.
  • BDA: MambaBDA localizes buildings and differentiates damage levels across varied building types and disaster events.

E. Different Spatio-Temporal Modeling Methods

The paper compares several approaches for modeling spatio-temporal relationships and finds advantages for Mamba-based mechanisms across the three change-detection tasks. Combining the mechanisms further improves performance over individual mechanisms.

  • Comparison: The comparison includes FPN concatenation, 3D-convolution TST, RNN-based modeling, and Transformer-based modeling.
  • Comparison: Mamba-based spatio-temporal modeling improves F1 by 0.77% on BCD, SeK by 0.52% on SCD, and F1 by 1.10% on BDA versus Transformer-based modeling.
  • Ablation: Using any one proposed modeling mechanism significantly improves detection performance over the baseline.
  • Ablation: Spatiotemporal cross modeling is slightly better than the other individual mechanisms, while combining all three further improves network performance.
  • Feature visualization: After spatio-temporal relationships are modeled in the decoder, changed regions progressively show high response values in extracted features.

F. Comparison to Other Backbone Networks

ChangeMamba combines VMamba’s linear-complexity global-context modeling with spatio-temporal relationship mechanisms, achieving competitive change-detection performance and strong robustness. Comparisons show advantages over several Transformer and CNN backbones in memory usage, degraded-data resilience, and global-context learning.

  • Backbone comparison: Mamba outperforms Swin-Transformer and Mix-Former on SYSU BCD, whose efficiency-oriented designs limit global contextual information.VMamba reduces computational consumption to a linear relationship with the number of tokens, avoiding those information-loss mechanisms.
  • Backbone comparison: MambaBCD-Small has a GPU-memory trend similar to advanced CNN architectures, while using significantly less memory than Transformer-based ChangeFormerV5.ChangeFormerV5 already uses strided convolutions to reduce self-attention overhead; lightweight ChangeFormerV2 surpasses MambaBCD and two CNN models in this comparison.
  • Robustness comparison: An 8.73% F1 decrease under Gaussian blur demonstrates stronger robustness for MambaBCD than SwinTransUNet’s 14.83% and ChangeFormerV5’s 18.43% decreases.Under Gaussian noise, only ChangeFormerV5 and MambaBCD avoid the significant F1 decreases observed for the other methods.
  • Robustness comparison: MambaBCD’s F1 decreases only 2.88% when scale increases from 1 to 1.5 and 1.18% when scale decreases to 0.75.At a scaling factor of 0.5, all methods show a larger F1 decrease.
  • Overall comparison: Across five benchmark datasets, the three ChangeMamba architectures achieve competitive performance on BCD, SCD, and BDA compared with CNN- and Transformer-based approaches.The paper attributes this performance to VMamba feature extraction and three spatio-temporal relationship modeling mechanisms.
Loading 2404.03425v7…