PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images

Wele Gedara Chaminda Bandara, Vishal M. Patel

arXiv:2204.08454v3cs.CVcs.AIcs.LGeess.IV

TL;DR

Remote-sensing change detection depends on costly annotations, while unlabeled imagery is widely available. This paper uses consistency regularization on perturbed deep feature differences to train with labeled and unlabeled image pairs. With only 10% of annotated training data, the method reaches closer to supervised performance.

  • Problem

    Supervised change detection requires extensive, costly pixel-wise annotations, while unsupervised methods can struggle to distinguish changes of interest from irrelevant changes.

    Source: §1 · p. 2

  • Method

    The method combines supervised cross-entropy with an unsupervised loss that enforces consistent predictions under small perturbations of deep feature-difference maps.

    Source: §1 · p. 3, Learning from unlabeled data… · pp. 8–9

  • Results

    With 10% of the annotated training data, the method can almost reach supervised results and outperforms SemiCDNet by a significant margin.

    Source: §1 · p. 3, §7 · p. 14

  • Takeaways & Limitations

    The reported experiments show better generalizability and transferability than the state-of-the-art semi-supervised change-detection methods.

    Source: §7 · p. 14

    The consistency rationale depends on low- and high-density regions aligning with the ground-truth change and no-change boundary.

    Source: §3.1 · p. 5

Abstract

from arXiv · show

Remote-sensing (RS) Change Detection (CD) aims to detect "changes of interest" from co-registered bi-temporal images. The performance of existing deep supervised CD methods is attributed to the large amounts of annotated data used to train the networks. However, annotating large amounts of remote sensing images is labor-intensive and expensive, particularly with bi-temporal images, as it requires pixel-wise comparisons by a human expert. On the other hand, we often have access to unlimited unlabeled multi-temporal RS imagery thanks to ever-increasing earth observation programs. In this paper, we propose a simple yet effective way to leverage the information from unlabeled bi-temporal images to improve the performance of CD approaches. More specifically, we propose a semi-supervised CD model in which we formulate an unsupervised CD loss in addition to the supervised Cross-Entropy (CE) loss by constraining the output change probability map of a given unlabeled bi-temporal image pair to be consistent under the small random perturbations applied on the deep feature difference map that is obtained by subtracting their latent feature representations. Experiments conducted on two publicly available CD datasets show that the proposed semi-supervised CD method can reach closer to the performance of supervised CD even with access to as little as 10% of the annotated training data. Code available at https://github.com/wgcban/SemiCD

1 Introduction

Remote-sensing change detection needs labeled examples to distinguish relevant changes from irrelevant variation, but annotation is costly and existing approaches have limits in domain shift and use of unlabeled data. The paper proposes consistency regularization in the deep feature-difference domain, reporting near-supervised performance with only 10% labeled training data.

1 Introduction

  • With 10% of LEVIR training data labeled and 90% used unsupervised, the method is only 3.1% below the result using 100% labeled data.The comparison is for LEVIR; the figure caption reports the 3.1% gap.Source: Figure 1 · p. 2, §1 · p. 3
  • Supervised CD requires costly pixel-wise annotation and can overfit under domain shift, while unsupervised methods struggle to distinguish relevant from irrelevant changes.Remote-sensing domain shifts can arise from sensors, terrain, seasons, and illumination.Source: §1 · pp. 1–2
  • The authors report that the cluster assumption fails in the image domain but holds in the deep feature-difference domain, where they enforce prediction consistency under small perturbations.The consistency loss is combined with supervised cross-entropy to use labeled and unlabeled image pairs.Source: §1 · pp. 2–3

2 Related works

  • Earlier semi-supervised methods often focused on individual image pairs, while a GAN-based approach for large unlabeled datasets assumes labeled and unlabeled change maps share a distribution.The paper contrasts these limitations with consistency training on feature differences, which it says permits unlabeled images from another domain.Source: Semi-supervised CD methods on… · p. 4
  • Applying consistency directly to bi-temporal images is challenging because perturbations can introduce changes that confuse the change-detection network.The paper therefore reformulates consistency regularization for the feature-difference domain.Source: §1 · p. 2, Consistency regularization · p. 5

3 Method

The paper tests whether the cluster assumption for change detection holds in input imagery or hidden feature differences, then uses consistency regularization on feature differences to train with unlabeled image pairs. The model combines supervised cross-entropy with an unsupervised loss that encourages consistent predictions under small random perturbations.

3.1 The cluster assumption in CD

  • For change detection, the cluster assumption requires density-region boundaries to align with ground-truth change/no-change boundaries; it holds in hidden feature differences, but not in the input image domain.The visualization estimates local distances in input patches and neighboring hidden feature-difference vectors; change regions have high average distance in the latter domain.Source: §3.1 · pp. 5–6

3.2 Proposed semi-supervised CD approach

  • The network uses an encoder, a feature-difference module, and a decoder to produce change maps from pre- and post-change images.A shared Siamese encoder extracts representations; the feature-difference module computes their absolute difference and applies pyramid pooling before decoding.Source: CD network architecture · pp. 7–8
  • For unlabeled image pairs, the method perturbs the hidden feature-difference map and trains predictions from perturbed maps to match the unperturbed prediction.The unsupervised loss measures dissimilarity between these predictions, using mean squared error in the experiments.Source: Learning from unlabeled data… · pp. 8–9
  • The method considers random feature noise, feature dropping, guided feature cutout, content/object masking, and feature-VAT as perturbations.Feature-VAT applies an adversarial perturbation in the direction that most alters the feature-difference vector.Source: Learning from unlabeled data… · p. 9
  • The final objective adds the unsupervised consistency loss to supervised cross-entropy, with its weight ramped from 0 to 1 over the first T iterations and held at 1 afterward.The supervised phase trains on labeled pairs and their ground-truth change masks.Source: Learning from labeled data (supervised… · p. 8, Learning from unlabeled data… · p. 9

4 Experimental setup

Experiments use the LEVIR and WHU building change-detection datasets, with labeled and unlabeled training examples drawn from each original training set. The study reports change-class IoU and overall pixel accuracy on the default test sets.

Datasets

  • LEVIR and WHU are building change-detection datasets; training uses non-overlapping 256 × 256 patches and the datasets’ default train, validation, and test splits.For each semi-supervised setting, a selected fraction of training examples remains labeled and the rest are used without their change labels.Source: Datasets · p. 9

Performance metrics

  • The experiments report change-class Intersection over Union (IoU_c) and overall pixel accuracy (OA), with model selection on validation sets and results on test sets.The table covers methods on LEVIR→LEVIR and WHU→WHU across percentages of labeled data.Source: Table 1 · p. 10

5 Results and discussion

The experiments compare the proposed semi-supervised CD method with supervised and semi-supervised baselines across within-dataset, cross-dataset, and mixed-dataset settings. Results report stronger performance than SemiCDNet, including with few labeled samples and when unlabeled data comes from another dataset.

5 Results and discussion

  • The evaluation compares the proposed method with SemiCDNet and reimplemented semi-supervised segmentation methods AdvNet and s4GAN.The study reports quantitative and qualitative comparisons.Source: §5 · p. 10

5.1 Experiments within the same dataset

  • With 5% and 10% labeled data, the proposed method exceeds SemiCDNet by at least a 5% increase in IoU on same-dataset tests.The comparison covers LEVIR→LEVIR and WHU→WHU; all semi-supervised methods outperform supervised CD at the same labeled-data amount.Source: §5.1 · p. 11

5.2 Model generalizability/transferability

  • With 5% and 10% labeled training data, the proposed method achieves an improvement of more than 22% over SOTA methods in cross-dataset change-detection testing across LEVIR and WHU.The authors suggest this may reflect consistent results under small perturbations on hidden feature difference maps.Source: §5.2 · p. 11

5.3 Cross-dataset experiments

  • Adding unlabeled WHU data to LEVIR training generally improves CD results, and the proposed method outperforms SemiCDNet in all reported cases.For the {LEVIR(100%), WHU(unsup.)}→LEVIR qualitative comparison, the method predicts difficult changes more accurately than the methods considered.Source: §5.3 · pp. 13–14, Table 3 · p. 13, Figure 5 · p. 13

6 Ablation study

The ablation examines how feature-difference-map perturbations contribute to semi-supervised CD when the labeled-data amount is held fixed.

6 Ablation study

  • The LEVIR ablation uses 10% labeled samples and progressively adds Feature Noise, Feature Drop, Guided Feature Cutout, Context & Object masking, and Feature-VAT.The starting point is the supervised, no-perturbation case.Source: §6 · p. 14
  • Adding the different perturbations and enforcing consistency of the resulting change predictions significantly improves CD with the same amount of annotated training images.The reported comparisons are quantitative and qualitative.Source: §6 · p. 14

7 Conclusion

The conclusion presents consistency under perturbations of hidden feature differences as the proposed semi-supervised CD method’s central mechanism. It reports gains over existing semi-supervised CD and improved generalizability and transferability.

7 Conclusion

  • The method uses unlabeled remote-sensing data by enforcing consistency of predicted change probability maps under random perturbations to hidden feature difference maps.The conclusion reports that it can improve CD performance.Source: §7 · p. 14
  • The method nearly reaches supervised results with limited annotated samples and outperforms SemiCDNet by a significant margin.Source: §7 · p. 14
  • The method reports better generalizability and transferability, including when testing on an unseen dataset or using unlabeled data from a different dataset.The conclusion contrasts this with GAN-based methods that assume a distribution for unlabeled change masks.Source: §7 · p. 14

1 “Cluster assumption” in CD

The section illustrates the cluster assumption in change detection across image and hidden feature-difference domains. It reports that the assumption holds in the latter domain, motivating consistency regularization there.

1 “Cluster assumption” in CD

  • The examples contrast the pre-change image, post-change image, and ground-truth change mask with cluster-assumption views in two domains.The two domains are the bi-temporal image domain and the hidden feature difference domain.Source: Figure 1 · p. 19
  • The cluster assumption does not hold in the bi-temporal image domain but holds in the deep feature difference domain.The paper connects this observation to consistency regularization under small perturbations of hidden feature differences.Source: §1 · pp. 19–20

2 Additional qualitative results for LEVIR→LEVIR

Additional LEVIR→LEVIR qualitative examples show the proposed semi-supervised approach producing change maps described as much better than SOTA methods across four labeled-data settings.

2.1 Additional qualitative examples for LEVIR(5%)→LEVIR

  • At LEVIR(5%)→LEVIR, the proposed semi-supervised approach produces change maps described as much better than SOTA methods.Additional qualitative examples are shown in Fig. 2.Source: §2.1 · p. 21, Figure 2 · p. 21

2.2 Additional qualitative examples for LEVIR(10%)→LEVIR

  • At LEVIR(10%)→LEVIR, additional examples likewise show change maps described as much better than SOTA methods.The examples are shown in Fig. 3.Source: §2.2 · p. 22, Figure 3 · p. 22

2.3 Additional qualitative examples for LEVIR(20%)→LEVIR

  • At LEVIR(20%)→LEVIR, the examples are described as showing much better semi-supervised change maps than SOTA methods.The additional examples are captioned as Fig. 4.Source: §2.3 · p. 23, Figure 4 · p. 23

2.4 Additional qualitative examples for LEVIR(40%)→LEVIR

  • At LEVIR(40%)→LEVIR, the examples are described as showing much better semi-supervised change maps than SOTA methods.The additional examples are captioned as Fig. 5.Source: §2.4 · p. 24, Figure 5 · p. 24

3 Additional qualitative results for WHU→WHU

Additional WHU-CD qualitative examples cover four labeled-data settings, and the accompanying text describes the semi-supervised change maps as much better than SOTA methods.

3.1 Additional qualitative examples for WHU(5%)→WHU

  • At WHU(5%)→WHU, additional examples show change maps described as much better than SOTA methods.The examples are shown in Fig. 6.Source: §3.1 · p. 25, Figure 6 · p. 25

3.2 Additional qualitative examples for WHU(10%)→WHU

  • Figure 7’s qualitative examples show the proposed semisupervised approach generating much better change maps than SOTA methods, further demonstrating its effectiveness in leveraging freely available unlabeled remote-sensing images to improve change-detection performance.The prose labels Fig. 7 WHU(5%)→WHU, whereas its caption labels it WHU(10%)→WHU.Source: §3.2 · p. 26, Figure 7 · p. 26

3.3 Additional qualitative examples for WHU(20%)→WHU

  • At WHU(20%)→WHU, the examples are described as showing much better semi-supervised change maps than SOTA methods.The additional examples are shown in Fig. 8.Source: §3.3 · p. 27, Figure 8 · p. 27

3.4 Additional qualitative examples for WHU(40%)→WHU

  • At WHU(40%)→WHU, the examples are described as showing much better semi-supervised change maps than SOTA methods.The additional examples are shown in Fig. 9.Source: §3.4 · p. 28, Figure 9 · p. 28

4 Additional qualitative examples for model transferability: LEVIR→WHU

The section presents qualitative examples for models trained on LEVIR and tested on WHU, with labeled-data proportions ranging from 5% to 40%.

4.1 Examples for LEVIR(5%)→WHU

  • Additional qualitative transferability examples are provided for LEVIR(5%)→WHU.These examples are shown in Fig. 10.Source: Figure 10 · p. 29

4.2 Examples for LEVIR(10%)→WHU

  • Additional qualitative transferability examples are provided for LEVIR(10%)→WHU.These examples are shown in Fig. 11.Source: Figure 11 · p. 30

4.3 Examples for LEVIR(20%)→WHU

  • Additional qualitative transferability examples are provided for LEVIR(20%)→WHU.These examples are shown in Fig. 12.Source: Figure 12 · p. 31

4.4 Examples for LEVIR(40%)→WHU

  • Additional qualitative transferability examples are provided for LEVIR(40%)→WHU.These examples are shown in Fig. 13.Source: Figure 13 · p. 32

5 Additional qualitative results for cross-domain experiments

This section describes combining labeled LEVIR data with unlabeled WHU data and provides additional qualitative examples captioned as LEVIR→WHU transferability results.

5 Additional qualitative results for cross-domain experiments

  • The cross-domain experiments combine labeled and unlabeled data from two datasets, using labeled LEVIR data and unlabeled WHU data.The section describes evaluating performance with different proportions of labeled LEVIR data.Source: §5 · p. 33

5.1 Additional qualitative examples for {LEVIR(5%),WHU(unsup.)→LEVIR}

  • Fig. 14 provides additional qualitative examples captioned for LEVIR(5%)→WHU.Source: Figure 14 · p. 33

5.2 Additional qualitative examples for {LEVIR(10%),WHU(unsup.)→LEVIR}

  • Fig. 15 provides additional qualitative examples captioned for LEVIR(10%)→WHU.Source: Figure 15 · p. 34

5.3 Additional qualitative examples for {LEVIR(20%),WHU(unsup.)→LEVIR}

  • Fig. 16 provides additional qualitative examples captioned for LEVIR(20%)→WHU.Source: Figure 16 · p. 35

5.4 Additional qualitative examples for {LEVIR(40%),WHU(unsup.)→LEVIR}

  • Fig. 17 provides additional qualitative examples captioned for LEVIR(40%)→WHU.Source: Figure 17 · p. 36

Related papers

Loading 2204.08454v3…