PaperlayerView exact revision on arXiv

Source-linked AI summary

Exchanging Dual Encoder-Decoder: A New Strategy for Change Detection with Semantic Guidance and Spatial Localization

Sijie Zhao, Xueliang Zhang, Pengfeng Xiao, Guangjun He

arXiv:2311.11302v1cs.CV

TL;DR

Binary change detection remains challenging for MESD and DED in feature fusion, intraclass changes, and multiview building changes. The paper proposes SGSLN, using an exchanging dual encoder-decoder with semantic guidance and spatial localization. Experiments report superior performance with high efficiency across three change-detection scenarios.

  • Problem

    MESD can suffer bitemporal feature interference, while DED is difficult to apply to intraclass and multiview building change detection.

    Source: §I · pp. 1–2

  • Method

    SGSLN combines an exchanging dual encoder-decoder backbone with temporal fusion attention and half-convolution units for semantic guidance and spatial localization.

    Source: A. Overall Structure of the Proposed… · p. 5, §VI · p. 15

  • Results

    SGSLN achieves superior performance with high efficiency and outperforms all compared models across ICCD, SVBCD, and MVBCD experiments.

    Source: §VI · p. 15

  • Takeaways & Limitations

    The authors expect SGSLN to serve as a baseline for binary change detection and exploration of more diverse scenarios.

    Source: §VI · p. 15

    EDED requires many multitemporal images and costly binary-label annotation for supervised training, limiting its suitability when labeled data are scarce or absent.

    Source: D. Expectations and Limitations · p. 14

Abstract

from arXiv · show

Change detection is a critical task in earth observation applications. Recently, deep learning-based methods have shown promising performance and are quickly adopted in change detection. However, the widely used multiple encoder and single decoder (MESD) as well as dual encoder-decoder (DED) architectures still struggle to effectively handle change detection well. The former has problems of bitemporal feature interference in the feature-level fusion, while the latter is inapplicable to intraclass change detection and multiview building change detection. To solve these problems, we propose a new strategy with an exchanging dual encoder-decoder structure for binary change detection with semantic guidance and spatial localization. The proposed strategy solves the problems of bitemporal feature inference in MESD by fusing bitemporal features in the decision level and the inapplicability in DED by determining changed areas using bitemporal semantic features. We build a binary change detection model based on this strategy, and then validate and compare it with 18 state-of-the-art change detection methods on six datasets in three scenarios, including intraclass change detection datasets (CDD, SYSU), single-view building change detection datasets (WHU, LEVIR-CD, LEVIR-CD+) and a multiview building change detection dataset (NJDS). The experimental results demonstrate that our model achieves superior performance with high efficiency and outperforms all benchmark methods with F1-scores of 97.77%, 83.07%, 94.86%, 92.33%, 91.39%, 74.35% on CDD, SYSU, WHU, LEVIR-CD, LEVIR- CD+, and NJDS datasets, respectively. The code of this work will be available at https://github.com/NJU-LHRS/official-SGSLN.

I. INTRODUCTION

Binary change detection must handle multiple changed-object categories, difficult building boundaries, and view-induced spatial differences, while common network architectures have important limitations. The paper proposes EDED and SGSLN to combine bitemporal semantic guidance with spatial localization and decision-level fusion.

I. INTRODUCTION

  • Binary labels omit object semantics, while shadows and dense buildings complicate single-view boundaries and multiview differences can create false positives.These challenges affect intraclass, single-view building, and multiview building change detection.Source: §I · p. 1
  • MESD feature-level fusion can contaminate changed-object features, whereas DED depends on accurate target segmentation and struggles with intraclass and multiview changes.In intraclass change detection, binary labels do not specify object types; in multiview building detection, view differences can be mistaken for changes.Source: §I · pp. 1–2
  • EDED separately locates changed objects in each temporal image and fuses their features at the decision level, addressing MESD interference and DED limitations in ICCD and MVBCD.Bitemporal semantic features guide changed-area identification, while spatial features localize objects in each phase.Source: §I · pp. 2–3
  • The temporal fusion attention module focuses on important parts across bitemporal features, while the half-convolution unit uses one quarter the parameters and computation of conventional convolution.The two modules target temporal feature fusion and network resource use, respectively.Source: §I · p. 3
  • SGSLN combines the EDED backbone, temporal fusion attention, and half-convolution units for binary change detection.Its backbone uses semantic features to determine changed areas and phase-specific spatial features to locate changed objects.Source: §I · p. 3

II. RELATED WORKS

Classical change detection includes several comparison and transformation strategies, each with stated trade-offs. Deep learning approaches use different encoder-decoder arrangements to process bitemporal imagery.

A. Classical Change Detection Methods

  • Classical methods include layer arithmetic, post-classification comparison, and direct classification, with trade-offs involving map errors, training-data demands, or subtle-change detection.Post-classification overlays thematic maps; direct classification uses a multitemporal stack, while layer arithmetic compares image values or derived features.Source: A. Classical Change Detection Methods · p. 3
  • Transformations such as PCA and MAD emphasize between-image variance, CVA interprets change magnitude and direction, and hybrid methods combine localization with identification.The cited discussion notes challenges including locating changes across transformation components, ambiguity in CVA, and the value of combining comparison stages.Source: A. Classical Change Detection Methods · p. 3

B. Deep Learning Based Change Detection Method

1) Single Encoder-Decoder:

The section contrasts encoder-decoder designs and outlines how the proposed network processes bitemporal images. The designs differ in when and how they encode, fuse, and localize temporal information.

1) Single Encoder-Decoder:

  • SED combines or differences bitemporal images before encoding, which can leave early layers without informative deep features for each individual image.The cited discussion associates this with broken change-map boundaries and poor object internal compactness.Source: 1) Single Encoder-Decoder · p. 4

2) Multiple Encoder and Single Decoder:

  • MESD extracts features with multiple encoders and merges them in one decoder, but bitemporal feature fusion can cause features to contaminate each other.The related methods discussed focus on feature enhancement while potentially neglecting temporal information between the features.Source: 2) Multiple Encoder and Single Decoder · p. 4

3) Dual Encoder-Decoder:

  • DED segments target objects in both images and fuses their results, but its reliance on accurate segmentation limits its fit for ICCD and MVBCD.Binary labels make object segmentation difficult in ICCD, while differing imaging angles can cause false detections in MVBCD.Source: 3) Dual Encoder-Decoder · pp. 4–5

A. Overall Structure of the Proposed Network

  • The proposed EDED uses two weight-sharing encoders and decoders plus a fusion decoder, with half-exchanged bitemporal features informing semantic changed-area guidance.Each encoder and decoder uses half-convolution units and convolutional block attention modules.Source: A. Overall Structure of the Proposed… · p. 5
  • Phase-specific decoders use spatial features to locate changed objects, and temporal fusion attention combines bitemporal decoder features into the final change mask.The phase decoders produce half-size masks, while the fusion decoder produces a mask at the input image size.Source: A. Overall Structure of the Proposed… · p. 5

B. Exchanging Dual Encoder-Decoder Backbone

EDED exchanges bitemporal features so each branch can identify changed areas, then uses phase-specific spatial features to locate changed objects. Decision-level fusion avoids MESD’s feature interference and extends DED’s strategy to intraclass and multiview building change detection.

B. Exchanging Dual Encoder-Decoder Backbone

  • Alternating channel exchange gives each encoder branch bitemporal features for identifying rough changed areas.The exchange alternates half of the input features in the channel dimension; the deeper encoder blocks use the resulting bitemporal semantic features.Source: B. Exchanging Dual Encoder-Decoder… · p. 5
  • Each branch’s decoder uses phase-specific spatial features to refine rough changed areas into complete changed objects, supervised by change labels.A fusion decoder then combines the two phases’ localized changed-object features to produce the change map.Source: B. Exchanging Dual Encoder-Decoder… · p. 5
  • EDED uses encoder blocks 1–3 for spatial features and blocks 4–5 for semantic features; change blocks fuse upsampled and high-resolution features for multiscale identification.The described block design uses HCUs and CBAMs to support lightweight feature extraction and enhancement.Source: B. Exchanging Dual Encoder-Decoder… · p. 5, Figure 5 · p. 7
  • EDED fuses phase-specific changed-object features at the decision level, avoiding MESD’s bitemporal feature interference and addressing DED’s inapplicability to ICCD and MVBCD.The bitemporal semantic features support changed-area identification across object categories and help distinguish real changes from view-induced pseudochanges.Source: B. Exchanging Dual Encoder-Decoder… · p. 7

C. Half Convolutional Unit

The HCU reduces convolutional cost by processing half the input channels and reusing the other half. TFAM uses temporal information with channel and spatial attention to weight and fuse bitemporal features.

C. Half Convolutional Unit

  • HCU splits input channels, convolves one half, then concatenates and shuffles it with the residual half.For downsampling, the convolution uses stride 2 and max pooling downsamples the residual features.Source: C. Half Convolutional Unit · p. 7, Figure 6 · p. 7
  • HCU uses one quarter the parameters and computation of conventional convolution while maintaining effective feature extraction.The shuffle operation supports cross-channel interaction, while retaining half the input features supports gradient back-propagation and feature reuse.Source: C. Half Convolutional Unit · p. 7

D. Temporal Fusion Attention Module

  • Existing convolution- and attention-enhancement fusion methods focus on features before or after fusion, respectively, and both neglect temporal information between bitemporal features.Simple fusion is also described as susceptible to noise interference in bitemporal features.Source: D. Temporal Fusion Attention Module · p. 7
  • TFAM uses channel and spatial attention to identify important feature parts, while temporal information identifies important parts between the two time points.Its channel branch globally pools each bitemporal feature across spatial dimensions and uses one-dimensional convolutions to derive channel weights.Source: D. Temporal Fusion Attention Module · p. 7
  • TFAM combines bitemporal channel and spatial weights, applies them to the bitemporal features, and summarizes the weighted features for fusion.The channel weights are normalized with softmax so their sum is one.Source: D. Temporal Fusion Attention Module · pp. 7–8

IV. EXPERIMENTAL SETTINGS AND RESULTS

The evaluation covers six datasets across intraclass, single-view building, and multiview building change detection, comparing three SGSLN model sizes with 18 state-of-the-art methods. The dataset descriptions specify the evaluated data and, for several datasets, the sample preparation and splits.

IV. EXPERIMENTAL SETTINGS AND RESULTS

  • Three SGSLN variants, with maximum encoder feature channel sizes of 128, 256, and 512, are compared with 18 state-of-the-art methods across six datasets and three scenarios.The scenarios are ICCD (CDD and SYSU), SVBCD (WHU, LEVIR-CD, and LEVIR-CD+), and MVBCD (NJDS); larger channel size means more parameters and computation.Source: §IV · p. 8

1) Intraclass Change Detection:

  • CDD uses season-varying images of varied changing objects, while SYSU covers complex changes including road expansion, vegetation change, and suburban growth.CDD excludes changes due to seasonal differences and brightness; SYSU follows a 6:2:2 train/validation/test split.Source: 1) Intraclass Change Detection · p. 8

2) Single-View Building Change Detection:

  • WHU contains two-period aerial imagery of large-scale building changes, while LEVIR-CD covers building growth and decline in very-high-resolution imagery.WHU is cropped into 256×256 patches and split 7:1:2; LEVIR-CD is cropped into overlapping 256×256 patches and split 7:1:2.Source: 2) Single-View Building Change Detection · p. 8
  • LEVIR-CD+ extends LEVIR-CD with 985 image pairs acquired from 2002 to 2020 and approximately 80,000 building instances.It follows the same splitting and cropping approach as LEVIR-CD.Source: 2) Single-View Building Change Detection · p. 9

3) Multiview Building Change Detection:

The NJDS experiments use Nanjing imagery spanning low-, middle-, and high-rise buildings, with a defined train/validation/test split. The section also describes the benchmark architecture groups and training augmentations.

3) Multiview Building Change Detection:

  • NJDS contains 2014 and 2018 Nanjing image pairs with low-, middle-, and high-rise buildings, split into 540 training, 152 validation, and 1,827 testing pairs.Images were cropped into nonoverlapping 256×256-pixel patches.Source: 3) Multiview Building Change Detection · p. 9

B. Benchmark Methods

  • The comparison covers SED, MESD, and DED networks, using the same dataset split and data except that SFCCD additionally uses bitemporal segmentation labels.The benchmark groups include six SED methods, eight MESD methods, and four DED methods.Source: B. Benchmark Methods · p. 9

1) Data Augmentation:

  • Training augmentation includes geometric and image transformations, plus randomly exchanging bitemporal input order with probability 0.5.The transformations include flipping, transposing, shifting, scaling, rotation, and one of several image adjustments or corruptions.Source: 1) Data Augmentation · p. 9

2) Training and Inference:

The section specifies training and evaluation procedures, compares EDED with MESD and DED across scenarios, and reports ablations and intraclass-detection results. SGSLN/512 records the highest reported F1-scores on CDD and SYSU among the compared models.

2) Training and Inference:

  • Training uses 250 epochs, binary cross-entropy plus dice loss, and validation-selected checkpoints; WHU and LEVIR-CD+ use LEVIR-CD-pretrained initialization.The model is trained on one RTX A5000 GPU with batch size 64; validation skips the first 30 epochs.Source: 2) Training and Inference · p. 9

3) Evaluation Metrics:

  • Evaluation uses precision, recall, F1-score, and IoU; F1 balances precision and recall as their harmonic mean.Precision reflects false positives, while recall reflects false negatives.Source: 3) Evaluation Metrics · p. 9

D. Ablation Study

  • EDED outperforms MESD and DED in all three scenarios, especially ICCD and MVBCD; DED exceeds MESD in ICCD and SVBCD but trails it in MVBCD.The comparison uses SYSU, LEVIR-CD, and NJDS.Source: D. Ablation Study · pp. 9–10, Table II · p. 10
  • EDED avoids DED's false positives from multiview spatial differences and detects most changed areas in ICCD where DED has many false negatives.Figure 8 contrasts EDED and DED results on NJDS and SYSU.Source: D. Ablation Study · p. 10
  • With consistent parameters and computation, HCU improves efficiency and performance; TFAM improves performance, and using both further improves change detection.The ablations compare HCU with conventional convolution and test TFAM's bitemporal feature fusion.Source: D. Ablation Study · pp. 9–10, Table III · p. 10

1) Intraclass Change Detection:

  • 0.9777 F1-score on CDD and 0.8307 F1-score on SYSU are the highest reported for SGSLN/512 among compared models.On CDD, it exceeds the second-best TransUNetCD by 0.6%; on SYSU, it exceeds the second-best DARNet by 2.04%.Source: Table IV · p. 10, 1) Intraclass Change Detection · p. 10

2) Single-View Building Change Detection:

On the three single-view building datasets, SGSLN/512 achieves the highest reported IoU and F1-score among compared models. Its inference examples also address shadows, dense buildings, and an apparent false change in WHU.

2) Single-View Building Change Detection:

3) Multiview Building Change Detection:

On NJDS, SGSLN/512 achieves the highest reported IoU and F1-score among compared models and detects changes despite multiview spatial differences. The efficiency test reports strong F1-scores with relatively low computational costs.

3) Multiview Building Change Detection:

4) Efficiency Test:

  • 0.9168 F1-score with 0.381 M parameters, 0.8045 FLOP computation, and 96 seconds of training time is reported for SGSLN/128; SGSLN/256 and /512 surpass all compared models in F1-score.The efficiency comparison is conducted on WHU.Source: 4) Efficiency Test · pp. 12–13, Table X · p. 13

V. DISCUSSION

The discussion identifies encoding block 3 as the strongest exchange position and explains how the bitemporal branches and their supervision support localization and training.

A. Exchanging Position

  • Exchanging bitemporal features after encoding block 3 gives SGSLN/512 its best performance among the tested positions.Earlier exchange leaves too few spatial features for localization; later exchange loses information through downsampling and can miss small changes.Source: Table XI · p. 13, A. Exchanging Position · p. 13

B. Bitemporal Branches

  • On LEVIR-CD+, the T1 branch marks only rough changed-building areas, while the T2 and fusion branches precisely locate the new buildings in T2.The fusion branch combines bitemporal features to detect all changed buildings.Source: B. Bitemporal Branches · pp. 13–14
  • Adding supervision to both bitemporal branches improves F1-score by 0.8% and IoU by 1.34% over supervising only the fusion branch on LEVIR-CD+.The authors attribute the gain to shorter gradient back-propagation paths that enable sufficient training of shallow layers.Source: Table XII · p. 14, B. Bitemporal Branches · p. 14

C. Transferability

Cross-domain transfer from LEVIR-CD to WHU remains weak, despite SGSLN/512 outperforming other methods in that evaluation. The paper reports stronger performance across change-detection scenarios overall, while noting limits from labeled-data requirements and binary outputs.

C. Transferability

  • SGSLN/512's F1-score falls from 0.9233 on LEIVR-CD to 0.5411 on WHU cross-domain, versus 0.9486 when trained on WHU.SGSLN/512 nonetheless leads the compared models on WHU by F1-score and IoU in the cross-domain evaluation.Source: Table XIII · p. 14, C. Transferability · p. 14

D. Expectations and Limitations

  • EDED reports higher F1-scores than MESD and DED in ICCD, SVBCD, and MVBCD, using bitemporal semantic features to handle varied changes and spatial differences.The reported gains are 2.20% and 1.24% over MESD and DED on SYSU, 1.01% and 0.44% on LEVIR-CD, and 3.50% and 3.93% on NJDS.Source: D. Expectations and Limitations · p. 14
  • EDED requires substantial multitemporal imagery and binary-label annotation for supervised training, limiting its adaptability with few or no labels; binary outputs also cannot classify changed objects.The latter boundary limits EDED's use in semantic change detection.Source: D. Expectations and Limitations · pp. 14–15

VI. CONCLUSION

  • Across ICCD, SVBCD, and MVBCD, SGSLN achieves superior performance with high efficiency and outperforms all compared models.The paper expects SGSLN to serve as a baseline for exploring more diverse change-detection scenarios.Source: §VI · p. 15
Loading 2311.11302v1…