PaperlayerView exact revision on arXiv

Source-linked AI summary

Changer: Feature Interaction is What You Need for Change Detection

Sheng Fang, Kaiyu Li, Zhe Li

arXiv:2209.08290v1cs.CV

TL;DR

Change detection must identify changed locations in bi-temporal imagery, but temporal domain differences make feature interaction important. MetaChanger explores alternative interaction layers, including ChangerAD and parameter-free ChangerEx, and adds FDAF for alignment and fusion; Changer variants achieve competitive performance across multiple datasets. The authors present ChangerAD and ChangerEx as starting baselines for future MetaChanger design, while noting evaluation under more learning settings and related tasks remains future work.

  • Problem

    Change detection identifies where change has occurred in bi-temporal images, but temporal domain differences can make the change of interest ambiguous.

  • Method

    MetaChanger is a general architecture with alternative interaction layers, including aggregation-distribution and parameter- and computation-free feature exchange, plus FDAF for interactive alignment and fusion.

  • Results

    ChangerAD and ChangerEx achieve competitive performance on multiple change-detection datasets, with extensive ablations demonstrating ChangerEx’s robustness and extensibility.

  • Takeaways & Limitations

    ChangerAD and ChangerEx provide starting baselines for future MetaChanger designs focused on interaction strategies.

  • Takeaways & Limitations

    The paper identifies evaluation under more learning settings and related tasks, such as semantic change detection, as future work.

Abstract

from arXiv · show

Change detection is an important tool for long-term earth observation missions. It takes bi-temporal images as input and predicts "where" the change has occurred. Different from other dense prediction tasks, a meaningful consideration for change detection is the interaction between bi-temporal features. With this motivation, in this paper we propose a novel general change detection architecture, MetaChanger, which includes a series of alternative interaction layers in the feature extractor. To verify the effectiveness of MetaChanger, we propose two derived models, ChangerAD and ChangerEx with simple interaction strategies: Aggregation-Distribution (AD) and "exchange". AD is abstracted from some complex interaction methods, and "exchange" is a completely parameter\&computation-free operation by exchanging bi-temporal features. In addition, for better alignment of bi-temporal features, we propose a flow dual-alignment fusion (FDAF) module which allows interactive alignment and feature fusion. Crucially, we observe Changer series models achieve competitive performance on different scale change detection datasets. Further, our proposed ChangerAD and ChangerEx could serve as a starting baseline for future MetaChanger design.

1. Introduction

Change detection predicts changed locations from bi-temporal images, making feature interaction a central design question. MetaChanger addresses this with alternative interaction layers, simple ChangerAD and ChangerEx variants, and FDAF for alignment and fusion.

  • Motivation: Change detection predicts where changes occur from bi-temporal images, while focusing on changed areas can suit long-term observation and semi-automated applications.It need not predict land-cover categories across every pixel, and it tolerates some misdetection.
  • Motivation: Temporal domain differences and siamese processing can make the model’s interpretation of the change of interest ambiguous.Climate changes and preprocessing corrections affect targets and backgrounds differently across image domains.
  • MetaChanger: MetaChanger is a general architecture that emphasizes bi-temporal feature interaction during feature extraction through alternative interaction and fusion layers.Its encoder can use hierarchical ConvNets or Transformers.
  • Derived models: ChangerAD abstracts aggregation-distribution interaction, while ChangerEx exchanges bi-temporal feature maps without additional parameters or computation.Exchanged features are mixed in subsequent convolution or token-mixer operations.
  • Fusion: FDAF provides interactive fusion and alignment for bi-temporal features to address side-looking and misalignment in multi-temporal remote-sensing images.The module is intended to improve alignment before subsequent propagation.
  • Results: Experiments across multiple datasets show competitive Changer-series performance, with ChangerAD and ChangerEx serving as starting baselines for future MetaChanger designs.The contributions also report extensive experiments demonstrating that simple interaction strategies can improve MetaChanger performance.

2. Related work

Related change-detection methods commonly use metric-based or classification-based formulations, while MetaChanger instead isolates bi-temporal feature interaction. The paper positions feature exchange as a simpler alternative to more complex interaction mechanisms.

  • Change-detection formulations: Metric-based methods align bi-temporal images in feature space, whereas classification-based methods directly predict change as a dense classification task.Classification-based models fuse bi-temporal features and generate the change map with a top-level classifier.
  • MetaChanger: MetaChanger differs by simplifying change-detection processing while specifically focusing on interaction between bi-temporal features.It is presented as a framework for exploring interaction strategies rather than as a separate formulation from existing model types.
  • Feature fusion: Feature fusion combines multiple feature sources, levels, or scales, and may use parameter-free operations, attention, or alignment mechanisms.Alignment-based fusion commonly uses flow fields or deformable convolution for spatial alignment.
  • Feature interaction: Feature interaction is defined here as correlation or communication during feature extraction before fusion, including aggregation-distribution patterns abstracted from co-attention.AD aggregates features and distributes attention maps to reweight bi-temporal channels.
  • Feature exchange: Feature exchange provides a simpler interaction alternative that introduces no extra computation compared with complex cross-attention and affinity-based methods.Unlike multimodal channel exchange, it targets semantic differences between spatially corresponding change-detection domains.

3. Method

MetaChanger is a general change-detection framework that inserts alternative interaction layers into a siamese encoder and combines bi-temporal features for change-map prediction. Its variants explore aggregation-distribution and parameter-free feature exchange, while FDAF explicitly aligns and fuses bi-temporal features.

  • MetaChanger: MetaChanger uses a siamese encoder-decoder with interactive encoders, a decoder, and a fusion-projection head to predict a bi-temporal change map.The encoder can use hierarchical ConvNets or Transformers, and the decoder is lightweight.
  • MetaChanger: The framework processes multilevel features through shared-weight stages before applying interaction layers to correlate the two temporal branches.Each hierarchy first generates stage features and then feeds both temporal outputs into an interaction layer.
  • ChangerAD: ChangerAD aggregates bi-temporal features, extracts a global co-feature with pooling and an MLP, and adaptively reweights each branch through sigmoid attention maps.The MLP first squeezes and then expands the channel dimension.
  • ChangerEx: Feature exchange is motivated by mutual contextual learning and by making the two temporal feature distributions more similar, enabling some automatic domain adaptation.The paper argues exchange is feasible because bi-temporal images share spatial semantic correspondence and temporal order is usually not the change-detection target.
  • ChangerEx: ChangerEx exchanges bi-temporal feature maps in spatial or channel dimensions using a parameter- and computation-free operation, with later layers mixing exchanged features.Its exchange mask determines whether positions or channels are exchanged; channel exchange detaches gradients from exchanged channels.
  • Flow Dual-Alignment Fusion: FDAF addresses side-looking and registration misalignment by resampling each feature map through a deformable optical-flow field and comparing corrected features with the other original map.This explicit bidirectional alignment is used for interactive feature fusion and feeds the resulting differences into subsequent propagation.

4. Experiments

Experiments evaluate MetaChanger and its interaction variants across S2Looking and LEVIR-CD, with ablations examining placement, exchange ratios, window sizes, learnability, visualization, and backbone generalization.

  • Datasets: S2Looking contains 5,000 image pairs, while LEVIR-CD contains 637 bi-temporal image pairs for change-detection evaluation.S2Looking images span 1–3 years at 0.5–0.8 m/pixel; LEVIR-CD images are 1024 × 1024 at 0.5 m/pixel.
  • Main results: ChangerVanilla achieves competitive performance, FDAF improves F1-score, and ChangerAD and ChangerEx improve over ChangerAlign with slight or no extra cost.The gains are reported as especially promising on the more challenging S2Looking dataset.
  • Stage placement: AD performs best when inserted in the last three stages, while channel exchange is strongest in the latter two stages and spatial exchange in earlier stages.The authors use spatial exchange in shallow layers and channel exchange in deeper layers for ChangerEx.
  • Exchange settings: Exchange-ratio changes from 1/32 to 1/2 produce relatively slight performance differences, making exchange presence more important than its exact ratio.The ablation also reports insensitivity to additional exchange hyperparameters.
  • Exchange settings: Spatial-exchange window size is robust from 1 × 1 to 8 × 8, and preserving more complete spatial structure does not improve performance.This result concerns spatial exchange on S2Looking.
  • Learnable exchange: Learnable channel exchange slightly improves over unlearnable exchange, whereas learnable spatial exchange causes a dramatic performance drop.The authors state that only simple learnable exchanges were tested and identify further exploration as future work.
  • Backbones and visualization: With ResNeSt101, ChangerEx gains 1.68 and reaches an F1-Score of 67.61.Feature exchange also activates disappeared building regions in the opposite temporal heat map, according to Grad-CAM visualization.

5. Conclusion

The conclusion presents MetaChanger as a framework for studying feature interaction in change detection and reports competitive ChangerAD and ChangerEx results across multiple datasets. It also identifies broader evaluation and improved interaction methods as future directions.

  • Contributions: MetaChanger explores feature interactions in change detection through simple aggregation-distribution and feature-exchange interaction layers.The derived models are ChangerAD and ChangerEx.
  • Conclusions: ChangerAD and ChangerEx achieve competitive performance on multiple change-detection datasets.The conclusion also reports robustness and extensibility from extensive ChangerEx ablations.
  • Future work: Future work will evaluate MetaChanger under more learning settings and related tasks, including semantic change detection.The authors also hope to encourage further research on MetaChanger interaction methods.

A. Compared methods

The compared methods include classification-based UNet-like models, multi-task and siamese CNNs, hybrid CNN-transformers, and transformer-based siamese architectures for change detection.

  • Classification-based models: FC-EF, FC-Siam-Conc, and FC-Siam-Diff are classification-based UNet-like models using early or medium fusion of bi-temporal inputs.The latter two use siamese encoders with concatenation or difference fusion.
  • Multi-task models: DTCDSCN jointly performs change detection and semantic segmentation and introduces dual attention over channels and spatial positions.Its dual-attention module is intended to improve feature representation.
  • Siamese CNNs: STANet uses a siamese CNN with spatial-temporal attention, including base, basic-attention, and pyramid-attention variants.The base model uses a weight-sharing CNN feature extractor.
  • Siamese CNNs: CDNet is a tuned siamese CNN paired with instance-level augmentation that synthesizes diverse building changes using generative adversarial training.The augmentation targets bi-temporal images containing changed building instances.
  • Transformer-based models: BiT combines shallow convolutional blocks with deeper transformer blocks using cross-attention to model spatial-temporal context.It is a hybrid CNN-transformer model.
  • Transformer-based models: ChangeFormer is a transformer-based siamese network combining a hierarchical transformer encoder with an MLP decoder for long-range details.Its architecture is organized as a siamese network.

B. Pseudo code

The spatial-exchange pseudo-code obtains an exchange mask along the W dimension and broadcasts it across C × H × W for scale-stable testing.

  • Spatial exchange: Spatial exchange derives the mask in the W dimension and broadcasts it across the C × H × W feature dimensions.The stated design allows more stable testing at different scales than obtaining the mask in the HW dimension.
Loading 2209.08290v1…