PaperlayerView exact revision on arXiv

Source-linked AI summary

DASNet: Dual attentive fully convolutional siamese networks for change detection of high resolution satellite images

Jie Chen, Ziyang Yuan, Jian Peng, Li Chen, Haozhe Huang, Jiawei Zhu, Yu Liu, Haifeng Li

arXiv:2003.03608v2cs.CV

TL;DR

Remote sensing change detection methods lack robustness to pseudo-changes, compounded by severe imbalance between unchanged and changed samples. DASNet combines dual attention with weighted double-margin contrastive loss to address these issues. It reports improved F1 performance on both CDD and BCDD compared with baselines.

  • Problem

    Existing change-detection methods have low robustness to pseudo-changes, while unchanged samples greatly outnumber changed samples.

  • Method

    DASNet uses dual attention to capture long-range dependencies and weighted double-margin contrastive loss to balance changed and unchanged feature-pair contributions.

  • Results

    DASNet achieved SOTA results with F1 scores of 91.9% on CDD and 89.8% on BCDD, with maximum baseline increases of 2.9% and 4.2%, respectively.

  • Takeaways & Limitations

    The Siamese structure learns change representations, while attention helps describe local change features and recognize pseudo-changes on both datasets.

Abstract

from arXiv · show

Change detection is a basic task of remote sensing image processing. The research objective is to identity the change information of interest and filter out the irrelevant change information as interference factors. Recently, the rise of deep learning has provided new tools for change detection, which have yielded impressive results. However, the available methods focus mainly on the difference information between multitemporal remote sensing images and lack robustness to pseudo-change information. To overcome the lack of resistance of current methods to pseudo-changes, in this paper, we propose a new method, namely, dual attentive fully convolutional Siamese networks (DASNet) for change detection in high-resolution images. Through the dual-attention mechanism, long-range dependencies are captured to obtain more discriminant feature representations to enhance the recognition performance of the model. Moreover, the imbalanced sample is a serious problem in change detection, i.e. unchanged samples are much more than changed samples, which is one of the main reasons resulting in pseudo-changes. We put forward the weighted double margin contrastive loss to address this problem by punishing the attention to unchanged feature pairs and increase attention to changed feature pairs. The experimental results of our method on the change detection dataset (CDD) and the building change detection dataset (BCDD) demonstrate that compared with other baseline methods, the proposed method realizes maximum improvements of 2.1\% and 3.6\%, respectively, in the F1 score. Our Pytorch implementation is available at https://github.com/lehaifeng/DASNet.

I. INTRODUCTION

The introduction identifies weak robustness to pseudo-changes and severe changed–unchanged sample imbalance as central problems in high-resolution remote sensing change detection. DASNet addresses them with dual attention and weighted double-margin contrastive loss, reporting strong results on CDD and BCDD.

  • Available change-detection methods have low robustness to pseudo-changes because their features poorly distinguish changed from unchanged areas and are sensitive to factors such as noise, angle, shadow, and context.
  • Unchanged samples substantially outnumber changed samples, creating a severe distribution imbalance in change-detection data.
  • DASNet uses dual attention to locate changed areas and obtain more discriminant feature representations by capturing long-range dependencies.
  • Weighted double-margin contrastive loss balances changed and unchanged regions while increasing changed-pair distances and reducing unchanged-pair distances.
  • 91.9% and 89.8% F1 scores were reached on CDD and BCDD, respectively, with maximum increases of 2.9% and 4.2% over selected baselines.

II. RELATED WORKS

Related work spans traditional threshold-based methods and deep-learning approaches for remote sensing change detection. DASNet instead directly measures changes using discriminative dual-attention features and WDMC loss.

  • Remote sensing change detection methods are categorized into traditional and deep-learning approaches based on their underlying techniques.
  • Traditional methods use pixel or object feature differences and ratios, then identify changes through thresholds.
  • Deep-learning methods extract features from multitemporal images with neural networks to determine ground-object changes.
  • DASNet directly measures changes and captures long-range dependencies through dual attention to obtain more discriminative feature representations.

III. METHODOLOGY

DASNet processes paired high-resolution images with a Siamese convolutional module, dual spatial and channel attention, and metric-learning loss. Attention aggregates contextual information, while WDMC separates changed and unchanged feature pairs.

  • The network first uses Siam-Conv on multitemporal image pairs to generate local features, then applies dual attention to establish connections and obtain global context.
  • Spatial Attention Mechanism: Spatial attention models relationships between all feature pixels and selectively aggregates long-range contextual information into local representations.
  • Channel Attention Mechanism: Channel attention models relationships across channels, and its output is aggregated with spatial-attention output to improve feature representations.
  • WDMC Loss Function: WDMC loss decreases distances between unchanged pixel pairs and increases distances between changed pixel pairs in feature space.
  • Spatial Attention Mechanism: Spatial attention produces global-context features in which similar semantic features promote one another, improving within-class compactness and distinguishing changes from pseudo-changes.

C. Channel Attention Mechanism

The channel attention module models relationships among feature channels to enhance semantically related representations. Its output combines channel-wise information with the original features, improving change discrimination.

  • Channel relationships: Channel attention establishes relationships between high-level feature channels whose semantic responses are interdependent.These relationships can enhance correlated feature maps and improve semantic feature representations for distinguishing changes.
  • Attention-map construction: The module reshapes F ∈ R^C×H×W to R^C×N and computes a channel attention map through matrix multiplication and softmax.Here, N = H × W, and the resulting map represents channel relationships.
  • Attention-map construction: The channel attention map weights relationships between channels, with larger values indicating stronger connections.F_xji measures the impact of the ith channel on the jth channel.
  • Output formation: The module multiplies the reshaped features by the attention map, scales the result by γ, and adds it elementwise to F.The resulting representation is then aggregated with the output of the spatial attention module.
  • Output formation: γ starts at 0 and learns increasing weight, producing each channel as a weighted sum of all-channel features and the original feature.This models long-range semantic dependencies, enhances feature identifiability, and highlights changed regions.

D. WDMC Loss Function

The WDMC loss addresses imbalanced changed and unchanged pairs in contrastive training. It uses separate margins, weights, and deep supervision to balance their contributions and improve change recognition.

  • Weighted double-margin design: For each pixel pair, f0 and f1 are the unchanged and changed image feature maps, d_i,j is their feature distance, and y distinguishes unchanged from changed pairs.The margin m is enforced for changed feature pairs, with y = 0 for unchanged and y = 1 for changed pairs.
  • Motivation: Traditional contrastive loss treats changed and unchanged feature pairs unevenly during training, while unchanged samples greatly outnumber changed samples.Imaging differences in unchanged areas can hinder optimization, and changed pairs contribute only when their distance exceeds the margin.
  • Weighted double-margin design: WDMC loss uses separate margins and weights for unchanged and changed feature pairs to balance their punishment and dataset contributions.m1 and m2 are the respective margins, while w1 and w2 are the respective pair weights.
  • Weighted double-margin design: The loss weights are based on changed and unchanged pixel-pair frequencies, reducing the influence of the larger unchanged region population.The parameters balance the network’s interest in changed and unchanged areas.
  • Deep supervision: Deep supervision applies WDMC loss to spatial-attention, channel-attention, and final-output feature pairs, with λ_i weighting each loss.This strategy is intended to enhance hidden-layer feature representations.

E. Implementation Details

The implementation uses configurable Siam-Conv backbones and a parameterized WDMC loss. Backbone modifications preserve higher-resolution features, while loss parameters balance changed and unchanged regions.

  • Network structure: The Siam-Conv design selects either VGG16 or ResNet50 as its basic network.For VGG16, only the first five convolution modules are retained and the last module’s max-pooling layer is removed.
  • Network structure: The ResNet50 variant removes downsampling operations and uses dilated convolutions in its last two ResNet blocks.These are the stated structural modifications for the ResNet50 Siam-Conv network.
  • Loss parameters: The loss design uses four parameters to balance unchanged-region and changed-region feature-pair contributions and enhance changed-region identification.The weights w1 and w2 are dataset pixel ratios, whereas margins m1 and m2 require manual adjustment.

IV. EXPERIMENTS AND DISCUSSION

The experiments evaluate DASNet on CDD and BCDD using comparative and ablation studies, with datasets exhibiting substantially more unchanged than changed pixels.

  • The evaluation compares the proposed method with other change-detection methods on the CDD and BCDD datasets and includes ablation experiments.These experiments assess both the overall method and its proposed structure and loss function.
  • Datasets: CDD contains multisource bitemporal images with varied resolutions and seasonal differences, while BCDD contains Christchurch building images captured in 2012 and 2016.CDD includes 11 original image pairs; BCDD images are divided into nonoverlapping 256 × 256-pixel pairs.
  • Class distribution: 0.147 and 0.045: the changed-to-unchanged pixel ratios in CDD and BCDD, respectively.The changed area is much smaller than the unchanged area in both datasets.
  • Dataset examples: CDD training examples show unchanged images, changed images, and labels in separate rows.The figure presents selected multitemporal training images from CDD.
  • Dataset examples: BCDD training examples show unchanged images, changed images, and labels in separate rows.The figure presents selected multitemporal training images from BCDD.

B. Metrics and Implementation details

The study evaluates change detection with precision, recall, F1, and overall accuracy, then examines distance metrics, WDMC loss, and parameter settings.

  • Metrics: Precision, recall, F1 score, and overall accuracy are used to evaluate change-detection predictions.Higher precision indicates fewer false detections, higher recall indicates fewer missed predictions, and F1 and OA summarize overall performance.
  • Metrics: The overall accuracy metric is computed from TP, TN, FP, and FN as OA = TP + TN TP + TN + FP + FN.TP, FP, TN, and FN denote true positives, false positives, true negatives, and false negatives.
  • Implementation details: The experiments use PyTorch, Adam with a 1e-4 learning rate, batch size 8, m1 = 0.3, m2 = 2.2, and λ1 = λ2 = λ3 = 1.Training used 3 × GTX TITAN XP.
  • Distance metric: The l2 distance produces cleaner backgrounds and higher foreground contrast than cosine similarity in DASNet distance maps.The resulting visualization is described as providing stronger change-recognition performance.
  • WDMC loss: The WDMC loss addresses the imbalance between numerous unchanged pairs and fewer changed pairs by balancing their penalties.It is designed to strengthen identification of change information and robustness to pseudo-changes.
  • Loss comparison: WDMC loss is more resistant to pseudo-changes from seasonal and sensor changes than traditional contrastive loss in Siam-Conv visualizations.The Siam-Conv comparison isolates the loss function from the dual-attention mechanism.

D. Ablation Study for Attention Modules

The ablation study evaluates spatial, channel, and combined attention modules, finding that combining them improves the network’s comprehensive performance.

  • Ablation setup: The ablation study tracks the change process of η and γ across 30 training epochs.These parameter trajectories accompany the attention-module experiments.
  • Attention ablation: The dual attention module improves model performance comprehensively compared with baseline Siamese networks.The study compares spatial attention, channel attention, and their combination using VGG16 and ResNet50 backbones.
  • Attention ablation: The combined attention module uses spatial and channel information together to improve comprehensive network performance.Both individual attention mechanisms improve network accuracy, while their combination fully utilizes spatial and channel feature information.
  • Attention ablation: The attention ablation is reported on the CDD dataset, with CAM denoting channel attention and SAM denoting spatial attention.Table IV compares the attention configurations.

E. Visualization of the Dual Attention Mechanism Effect

Visualization results indicate that DASNet’s dual-attention mechanism better separates changed and unchanged features and focuses more on real areas of interest while resisting pseudo-changes.

  • t-SNE visualization: DASNet’s feature representations show clearer separation between changed and unchanged regions than Siam-Conv in t-SNE visualization.The comparison visualizes the final feature layers of Siam-Conv and DASNet, with changed and unchanged feature vectors shown in separate colors.
  • Grad-CAM visualization: Grad-CAM visualization shows DASNet’s attention mask covers the target area better than Siam-Conv’s mask.The visualization compares fused changed images with Grad-CAM masks from Siam-Conv and DASNet.
  • Pseudo-change robustness: Siam-Conv attends to pseudo-changes from seasonal variation, whereas DASNet focuses more on real changes.The reported pseudo-changes include seasonal changes in trees and agricultural fields.
  • Performance comparison: 2.9% and 4.2% F1-score improvements were reported on CDD and BCDD, respectively, over the other optimal change detection methods.Comparative experiments were conducted on both datasets, with visualized results reported for each.
  • Qualitative results: DASNet performed well when recognizing small changes, under substantial color differences, and amid pseudo-change interference.The qualitative examples also include a change that is difficult to observe in the image.
  • Building-change detection: On BCDD, DASNet focuses more on building-change areas while treating road changes as pseudo-changes.The figure-based comparison reports both change identification and resistance to pseudo-change interference.

V. CONCLUSIONS AND FUTURE WORK

The paper concludes that DASNet combines a Siamese architecture, dual attention, and WDMC loss for high-resolution remote sensing change detection. It reports strong performance on CDD and BCDD, while identifying small-sample, open-world, and noisy settings as future research areas.

  • Conclusions: DASNet directly measures changes by learning implicit metrics for high-resolution remote sensing images.The method uses spatial and channel attention to obtain better feature representations.
  • Conclusions: Spatial and channel attention help minimize distances between unchanged areas and maximize distances between changed areas.The WDMC loss balances the influences of changed and unchanged regions on the network.
  • Conclusions: DASNet performed well on both the CDD and BCDD datasets compared with other baseline methods.The conclusion attributes useful change representations to the Siamese structure and pseudo-change recognition to the attention mechanism.
  • Future work: Future work will address small samples, open-world environments, and noisy environments to improve change-detection mobility and robustness.These settings are identified as directions for further research.
Loading 2003.03608v2…