PaperlayerView exact revision on arXiv

Source-linked AI summary

FCCDN: Feature Constraint Network for VHR Image Change Detection

Pan Chen, Danfeng Hong, Zhengchao Chen, Xuan Yang, Baipeng Li, Bing Zhang

arXiv:2105.10860v2cs.CV

TL;DR

Change detection lacks effective supervision for learning robust change features, despite its importance in Earth observation. FCCDN constrains bitemporal feature extraction and fusion with a dual encoder-decoder, nonlocal feature pyramid, dense fusion module, and SSL strategy, achieving state-of-the-art performance on two building datasets and semantic segmentation without segmentation labels.

  • Problem

    Existing change-detection networks provide limited constraints for learning relationships between bitemporal features and suppressing background variation.

  • Method

    FCCDN combines a dual encoder-decoder backbone, nonlocal multiscale feature fusion, dense bitemporal feature fusion, and self-supervised feature constraints.

  • Results

    FCCDN achieves state-of-the-art performance on LEVIR-CD and WHU and obtains bitemporal semantic segmentation results without semantic segmentation labels.

  • Takeaways & Limitations

    The method provides accurate bitemporal semantic segmentation without semantic segmentation labels, saving labeling work for change-detection applications.

  • Takeaways & Limitations

    FCCDN requires many labeled data for robust training, and its SSL strategy may not provide additional supervision for intra-class changes or unknown semantic categories.

Abstract

from arXiv · show

Change detection is the process of identifying pixelwise differences in bitemporal co-registered images. It is of great significance to Earth observations. Recently, with the emergence of deep learning (DL), the power and feasibility of deep convolutional neural network (CNN)-based methods have been shown in the field of change detection. However, there is still a lack of effective supervision for change feature learning. In this work, a feature constraint change detection network (FCCDN) is proposed. We constrain features both in bitemporal feature extraction and feature fusion. More specifically, we propose a dual encoder-decoder network backbone for the change detection task. At the center of the backbone, we design a nonlocal feature pyramid network to extract and fuse multiscale features. To fuse bitemporal features in a robust way, we build a dense connection-based feature fusion module. Moreover, a self-supervised learning-based strategy is proposed to constrain feature learning. Based on FCCDN, we achieve state-of-the-art performance on two building change detection datasets (LEVIR-CD and WHU). On the LEVIR-CD dataset, we achieve an IoU of 0.8569 and an F1 score of 0.9229. On the WHU dataset, we achieve an IoU of 0.8820 and an F1 score of 0.9373. Moreover, for the first time, the acquisition of accurate bitemporal semantic segmentation results is achieved without using semantic segmentation labels. This is vital for the application of change detection because it saves the cost of labeling.

1. Introduction

FCCDN addresses limited supervision for change-feature learning by constraining bitemporal feature extraction and fusion. Its architecture combines a dual encoder-decoder backbone, multiscale and dense fusion modules, and self-supervised feature constraints, achieving strong results on two building datasets.

  • Motivation: FCCDN targets insufficient constraints on bitemporal feature relationships and the suppression of background variation in change detection.Unconcerned differences such as seasonal vegetation changes and building offsets can produce pseudochanges.
  • Approach: The proposed method extracts target-object features, suppresses background features, and fuses bitemporal features through feature constraints.The work addresses network design, multiscale extraction and fusion, bitemporal fusion, and SSL-based constraints.
  • Architecture: FCCDN uses a dual encoder-decoder backbone to support change detection.This backbone is presented as the first listed architectural contribution.
  • Architecture: The method introduces a nonlocal feature pyramid network for multiscale feature extraction and fusion.The NL-FPN is designed at the backbone center to enhance multiscale representations.
  • Feature supervision: A self-supervised learning task constrains feature extraction using additional losses on changed and unchanged areas.The contribution also reports semantic segmentation results without semantic segmentation labels.
  • Evaluation: FCCDN was validated on LEVIR-CD and WHU building change detection datasets, where experiments reported advantages over recent methods.The authors also highlight bitemporal building extraction results as significant.

2. Related work

Related work covers fully convolutional change-detection networks and self-supervised learning. Existing approaches use single- or double-stream architectures, while feature fusion and the role of SSL remain active concerns.

  • Fully convolutional networks: Fully convolutional networks support end-to-end pixelwise change detection by using semantic information.They have been applied to remote-sensing change detection and achieved state-of-the-art results on several datasets.
  • Fully convolutional networks: FCN-based change-detection architectures are broadly divided into single-stream and double-stream networks.Single-stream models process concatenated or differential bitemporal images, whereas double-stream models use separate feature-extraction streams.
  • Feature fusion: Double-stream networks commonly use two weight-sharing streams, but unchanged-object features can still vary substantially between bitemporal maps.Effective fusion therefore remains difficult; prior methods use concatenation, differences, or attention mechanisms.
  • Feature fusion: Attention-based fusion modules address bitemporal feature relationships but often require substantial computation and memory.The related work describes attention mechanisms as a response to feature misalignment and complex backgrounds.
  • Self-supervised learning: Self-supervised learning obtains training annotations from image-internal information through pretext tasks rather than human labels.Examples include colorization, inpainting, rotation, and jigsaw puzzles.
  • Self-supervised learning: In remote sensing, SSL has been used for feature extraction, consistent representations, clustering-based change maps, and multiview change detection.Prior change-detection studies mainly used SSL as a pretraining strategy.

3. Methodology

FCCDN constrains bitemporal feature extraction and fusion through a dual encoder-decoder backbone, nonlocal multiscale processing, dense feature fusion, and self-supervised auxiliary segmentation. These components target feature quality, multiscale context, temporal misalignment, and background suppression in change detection.

  • Overall architecture: FCCDN combines a DED backbone, NL-FPN, DFM blocks, and two self-supervised segmentation branches with a change branch.The DED backbone includes a dual decoder, while NL-FPN augments multiscale features and DFMs fuse bitemporal features to produce change features.
  • Dual encoder-decoder network: The DED backbone adds a weight-sharing dual decoder that filters invalid information and reconstructs useful information before change-feature generation.Change features are generated by fusing corresponding features from the dual decoder and then passed to the change decoder.
  • Non-local feature pyramid network: NL-FPN extracts and fuses multiscale features nonlocally by considering similarities between any pair of pixel positions.Its nonlocal blocks are added to the upsampling stage of an FPN to strengthen intraclass correlations and increase interclass separation.
  • Dense fusion module: DFM uses sum and difference branches with densely connected, weight-sharing streams to enhance edges, generate change regions, and reduce pseudochanges from feature misalignment.Dense residual connections combine multiple features and make later feature maps more aligned.
  • Self-supervised feature constraint: The SSL strategy adds two auxiliary semantic-segmentation branches whose pseudolabel-based learning constrains feature learning and guides extraction of target-object features.The branches are intended to suppress background features; their segmentation results are reported to perform well and further improve change detection.

4. Experimental Results

FCCDN is evaluated on the LEVIR-CD and WHU building change detection datasets, where the experiments report state-of-the-art performance relative to recently proposed methods.

  • Benchmark evaluation: FCCDN outperforms recently proposed change detection methods and achieves state-of-the-art performance on the LEVIR-CD and WHU datasets.The evaluation uses two building change detection datasets and reports experimental results after describing datasets, implementation, metrics, and detailed results.

4.1. Datasets

The experiments use the LEVIR-CD and WHU building change detection datasets, with dataset-specific resolutions and splitting strategies.

  • The study presents a brief overview of both datasets in Table 1 and provides further dataset details in the corresponding subsections.
  • LEVIR-CD contains 637 VHR image patches from Google Earth at 0.5 m resolution and 1024×1024 size.Its standard split assigns 70% of samples to training, 10% to validation, and 20% to testing.
  • LEVIR-CD experiments compare 256×256 crops with 128-pixel overlap against 512×512 crops with 256-pixel overlap.The larger slices are intended to provide more context and are feasible because FCCDN has low computational cost.
  • WHU contains two-period aerial images at 0.3 m resolution, acquired in 2012 and 2016, with varied large-scale building changes.

4.2. Implementation Details

The implementation normalizes and augments dataset slices, trains FCCDN with PyTorch and AdamW, and selects checkpoints using validation performance before testing.

  • Dataset slices are normalized before network input using dataset mean and standard deviation.The pre- and post-normalization slices are denoted img and img′, respectively.
  • Training applies random flipping, transposing, rotation, zooming, HSV shifting, and Gaussian noise augmentation.
  • FCCDN uses BCE plus dice coefficient loss, AdamW optimization, an initial learning rate of 0.002, and weight decay of 0.001.Training stops when the learning rate is about to be reduced for the fourth time.
  • FCCDN achieves good validation performance after no more than 220 epochs, and the highest-validation-accuracy weights are saved for testing.
  • Figure 10 compares FCCDN and the baseline on validation sets for LEVIR-CD and WHU, using red and gray curves respectively.Green triangles mark the best validation accuracy.
  • Testing uses normalized original LEVIR-CD slices or prepared WHU slices, with checkpoints selected by the highest validation F1 score.

4.3. Evaluation Metrics

Evaluation uses IoU and F1 score to quantify change detection performance, with higher values indicating better results.

  • IoU and F1 score are the evaluation metrics used for the change detection task.
  • IoU and F1 values range from 0 to 1, and higher values indicate better performance.
  • Precision is identified as a separately calculated evaluation quantity in the metrics subsection.
  • Recall is calculated as TP divided by TP plus FN.TP denotes true positive and FN denotes false negative.
  • FP denotes false positive in the metric definitions.

4.4. Results

FCCDN components consistently improve change-detection performance in ablations and achieve state-of-the-art results on LEVIR-CD and WHU. Visualizations also show unsupervised bitemporal building segmentation, while shadows and small changes remain challenging.

  • Ablation study: DED improves over FCS by 1% IoU and 0.6% F1, reaching 0.8380 IoU and 0.9119 F1 for FCS.FCS has high precision (0.9301) but low recall (0.8944), missing many changes.
  • Ablation study: 0.8500 IoU and 0.9189 F1 are achieved by DED + NL-FPN, while FCS + NL-FPN reaches 0.8430 IoU and 0.9148 F1.Both backbones benefit from NL-FPN.
  • Ablation study: DFM raises F1 from 0.9119 to 0.9161 with FCS and from 0.9176 to 0.9192 with DED.The ablation results indicate improved change-detection accuracy when DFM performs bitemporal feature fusion.
  • Ablation study: SSL raises IoU from 0.8478 to 0.8530 and F1 from 0.9176 to 0.9206 when tested with the DED backbone.The architectures were evaluated using averages over three runs.
  • LEVIR-CD dataset: FCCDN outperforms all competitors on LEVIR-CD and achieves new state-of-the-art results, including a 2% F1 advantage over DDCNN.Most comparison methods use the same test set, except DDCNN, which uses a different split.
  • LEVIR-CD dataset: Building shadows obscure edges, causing notable boundary differences and making exact boundaries difficult to identify.The issue is especially visible in selected rows of the LEVIR-CD visualization.
  • LEVIR-CD dataset: FCCDN produces unsupervised bitemporal building segmentation results from change labels alone, reducing the need for additional segmentation labeling.Figure 11 presents the change masks and segmentation results alongside bitemporal imagery and ground truth.
  • WHU dataset: 0.8820 IoU and 0.9373 F1 are the highest reported FCCDN scores on WHU, with competitive efficiency and the best F1 score.Visual results compare FCCDN with DDCNN and show bitemporal building segmentation outputs.

5. Discussion

The discussion extends feature constraints to multiclass change detection, adapting contrastive or SSL supervision to the available annotations. Results show gains from these strategies, with pseudolabel supervision suppressing unconcerned changes.

  • Season-varying dataset: The season-varying dataset lacks clear changed-object categories, so the SSL strategy is simplified to a contrastive loss.The contrastive loss pulls unchanged features together and pushes changed features apart.
  • Season-varying dataset: 0.9157 IoU and 0.9560 F1 are achieved with contrastive loss, compared with 0.9081 IoU and 0.9519 F1 for DED.The strategy adds two auxiliary branches whose outputs feed the auxiliary loss.
  • SECOND dataset: SECOND provides six landcover classes, enabling additional semantic segmentation supervision for changed objects.The multiclass SSL strategy adds semantic segmentation branches for unchanged pixels using pseudolabels.
  • SECOND dataset: Pseudolabel supervision improves model performance on SECOND and effectively suppresses unconcerned changes in the final results.The evaluation uses binary-change-mask mIoU and the separated kappa coefficient for multiclass change detection.
  • HRSCD dataset: HRSCD supplies complete bitemporal semantic segmentation labels, allowing change labels to be obtained by comparing the two land-cover maps.Its classes include no information, artificial surfaces, agricultural areas, forests, wetlands, and water.
  • Strategy comparison: SSL outperforms contrastive loss on both LEVIR-CD and SECOND, with gains of 0.29% IoU and 0.16% F1 on LEVIR-CD.On SECOND, SSL improves performance by 0.2% in IoU.

6. Conclusions

FCCDN introduces an architecture that constrains bitemporal feature extraction, feature fusion, and overall feature learning for VHR change detection. It achieves state-of-the-art performance while producing unsupervised bitemporal semantic segmentation results, but remains limited by labeled-data requirements and assumptions behind its SSL strategy.

  • FCCDN combines a DED backbone, NL-FPN, DFM, and an SSL-based strategy to constrain feature extraction, fusion, and overall feature learning.The architecture targets both bitemporal feature extraction and feature fusion.
  • FCCDN achieves state-of-the-art performance with relatively high efficiency on two building change detection tasks.
  • FCCDN obtains bitemporal semantic segmentation results in an unsupervised way on the experimental datasets.The paper describes this capability as vital for better Earth observations.
  • FCCDN requires many labeled data to train a robust model because it is a supervised learning algorithm.Building a change detection dataset is described as time-consuming.
  • The SSL-based strategy may not provide additional supervision for intra-class changes or when object semantic categories are unknown.Its auxiliary segmentation branches depend on knowing semantic categories in the bitemporal images.
Loading 2105.10860v2…