PaperlayerView exact revision on arXiv

Source-linked AI summary

EHCTNet: Enhanced Hybrid of CNN and Transformer Network for Remote Sensing Image Change Detection

Junjie Yang, Haibo Wan, Zhihai Shang

arXiv:2501.01238v1cs.CVcs.LG

TL;DR

Remote-sensing change detection needs models that focus on changes of interest rather than background; EHCTNet combines hybrid CNN-Transformer features with frequency refinement and KAN-based token mining. It achieves higher Recall than state-of-the-art methods and detects more continuous, intact changes.

  • Problem

    Existing hybrid methods may be designed for general computer vision rather than remote sensing, and background modeling can overshadow identification of changes of interest.

    Source: Introduction · p. 1

  • Method

    EHCTNet combines dual-branch CNN-Transformer multi-scale feature extraction with frequency refinement and KAN-based attention for semantic token mining.

    Source: Proposed Method · p. 3, Conclusion · p. 12

  • Results

    EHCTNet achieves higher Recall than the compared state-of-the-art models and detects more continuous, intact changes of interest.

    Source: Comparisons on DSIFN-CD · p. 9, Conclusion · p. 12

  • Takeaways & Limitations

    EHCTNet is more suitable for remote-sensing change detection focused on changes of interest than approaches that emphasize background information.

    Source: Comparisons on DSIFN-CD · p. 9

    The work focuses on single-source data.

    Source: Conclusion · p. 12

Abstract

from arXiv · show

Remote sensing (RS) change detection incurs a high cost because of false negatives, which are more costly than false positives. Existing frameworks, struggling to improve the Precision metric to reduce the cost of false positive, still have limitations in focusing on the change of interest, which leads to missed detections and discontinuity issues. This work tackles these issues by enhancing feature learning capabilities and integrating the frequency components of feature information, with a strategy to incrementally boost the Recall value. We propose an enhanced hybrid of CNN and Transformer network (EHCTNet) for effectively mining the change information of interest. Firstly, a dual branch feature extraction module is used to extract the multi scale features of RS images. Secondly, the frequency component of these features is exploited by a refined module I. Thirdly, an enhanced token mining module based on the Kolmogorov Arnold Network is utilized to derive semantic information. Finally, the semantic change information's frequency component, beneficial for final detection, is mined from the refined module II. Extensive experiments validate the effectiveness of EHCTNet in comprehending complex changes of interest. The visualization outcomes show that EHCTNet detects more intact and continuous changed areas and perceives more accurate neighboring distinction than state of the art models.

Introduction

EHCTNet targets missed and discontinuous detections in remote-sensing change detection by combining local and global feature learning with sequential frequency and semantic processing. Its dual-branch design and specialized modules aim to improve recall and preserve the continuity and integrity of changed objects.

  • Very high-resolution change detection is challenged by limited spectral information, spectral variability, occlusions, and inconsistent object features across images.These conditions motivate models that extract high-level semantic change information from bi-temporal imagery.Source: Introduction · p. 1
  • Existing methods that model consistent background information may overshadow changes of interest, although applications such as disaster response require high recall even at the cost of precision.Source: Introduction · pp. 1–2
  • EHCTNet combines dual-branch CNN and Transformer blocks to integrate local and global features and enhance multi-scale representations for remote-sensing change detection.The proposed architecture also incorporates spectral and spatial attention mechanisms.Source: Introduction · p. 2
  • HFFT, CKSA, and BFFT sequentially extract first-order features, semantic tokens, and second-order semantic differences to capture subtle and high-level changes and improve recall.Source: Introduction · p. 2
  • EHCTNet’s visualizations show more continuous, internally intact changed objects and improved discrimination between adjacent regions.Source: Introduction · p. 2

Related Work

Related work traces change detection from hand-designed conventional methods to deep networks and CNN–Transformer hybrids, while identifying tradeoffs in semantic discrimination, complexity, and training stability. EHCTNet is positioned as a hybrid with token mining and refinement modules intended to improve feature learning and recall.

Change Detection

  • Conventional methods use manually designed features that are interpretable but limited in representing complex, high-level object information.Examples include image algebra, PCA, clustering, SVMs, random forests, and decision trees; their detection capabilities are generally considered inferior to deep learning approaches.Source: Change Detection · p. 2
  • Deep change-detection methods include single-stream, siamese, and GAN-based approaches, each with limitations in semantic discrimination, complexity, or training stability.Single-stream models may miss discriminative semantic information, siamese models can be complex and difficult to optimize, and GAN training instability can affect feature representation and performance.Source: Change Detection · pp. 2–3

Hybridization Architectures

  • CNN–Transformer hybrids combine CNNs’ local feature extraction with Transformers’ global relationships, but existing models may overlook background consistency needed to reduce false positives.The discussion also cautions that a shift toward diffusion models risks neglecting changed-area identification, where recall can be paramount.Source: Hybridization Architectures · p. 3
  • EHCTNet addresses gaps in existing hybrid models with token mining and refinement modules intended to enhance feature learning and incrementally improve recall.The paper also notes that existing methods lack strategies tailored to specific remote-sensing scenarios.Source: Hybridization Architectures · p. 3

Proposed Method

EHCTNet combines dual-branch CNN–Transformer feature extraction with two frequency-refinement stages around KAN-based semantic token mining to produce a change map.

Proposed Method

  • EHCTNet comprises feature extraction, refined modules I and II, enhanced token mining based on a Transformer, and a detection head.The first refinement supplies image-level frequency features; the second refines semantic difference information for final detection.Source: Proposed Method · p. 3

Feature extraction module

  • Two parameter-sharing HCT branches use ResNet50 encoders and three Transformer decoder blocks to fuse hierarchical local features with multi-scale global context.The third fusion integrates spatial detail from the first CNN module, while a learnable variable weights local and global contributions.Source: Feature extraction module · p. 3

Refined module I

  • FFT maps raw features to spectral space, learnable gating weights frequency components, and inverse FFT returns detailed first-order features through a residual connection.The residual connection retains characteristics of the raw feature image.Source: Refined module I · p. 4

Enhanced token mining based Transformer module

  • KAN-based channel and spatial attention forms two condensed token sets; a Transformer models their intra- and inter-set relations, decodes context into dual semantic maps, and absolute subtraction yields a semantic change map.Replacing fully connected layers with KAN layers is part of the CKSA design.Source: Enhanced token mining based… · pp. 4–5

Refined module II

  • Refined module II applies FFT, weighted gating, inverse FFT, and a residual connection to the rescaled semantic difference map, producing second-order semantic difference information.The map is rescaled to the original image dimensions before frequency refinement.Source: Refined module II · p. 5

Detection head / Loss function

  • A full convolutional detection head outputs a two-class change map at the original image dimensions, and the method uses cross-entropy loss for binary classification.The output has dimensions RH0×W0×2, where H0 and W0 denote the input image height and width.Source: Detection head · p. 6, Loss function · p. 6

Experiment and Analysis · Datasets · Evaluation metrics

The experiments compare EHCTNet with state-of-the-art change-detection methods on LEVIR-CD and DSIFN-CD. The section describes each dataset and the metrics used to assess change-detection performance.

Experiment and Analysis

  • EHCTNet is compared with state-of-the-art change-detection methods on two large-scale, high-spatial-resolution remote-sensing datasets: LEVIR-CD and DSIFN-CD.Source: Experiment and Analysis · p. 6

Datasets

  • LEVIR-CD contains 637 original 1024 × 1024 image pairs; this study used 256 × 256 patches, with 7,120 training, 1,024 validation, and 2,048 testing pairs.Source: Datasets · p. 6
  • DSIFN-CD comprises 2 m satellite image pairs from six Chinese cities; after augmentation and patching, this study used 14,400 training, 1,360 validation, and 192 testing pairs.The source images were curated from Google Earth and split into 394 sub-image pairs before augmentation.Source: Datasets · pp. 6–7

Evaluation metrics

  • Precision measures correct positive predictions among predicted positives, while Recall measures correctly identified actual positives; high Recall matters because missed changes are costly.Source: Evaluation metrics · p. 7
  • IoU measures overlap with the ground truth, prioritizing changed areas; MIoU averages IoU scores across regions of interest.IoU ranges from 0 (no overlap) to 1 (perfect overlap).Source: Evaluation metrics · pp. 7–8
  • Overall Accuracy (OA) measures correctly predicted areas across the dataset but does not show effectiveness on individual classes, especially with class imbalance.Source: Evaluation metrics · p. 8
  • The authors identify Recall, F1-score, and IoU as suitable change-detection metrics and also adopt OA to assess performance across change and background categories.F1-score has historically been a primary metric because it considers both Precision and Recall.Source: Evaluation metrics · p. 8

Implementation details

EHCTNet was trained end-to-end with SGD, and the best validation-performing model was evaluated on the test set.

Implementation details

  • Training used PyTorch on an NVIDIA GeForce RTX 3060 with SGD, a linear learning-rate policy, learning rate 0.01, batch size 8, weight decay 0.0005, and momentum 0.99.Validation was conducted after each epoch, and the best-performing model was saved for test evaluation.Source: Implementation details · p. 8

Comparison with State-of-the-art methods

EHCTNet achieves the highest reported Recall, F1, IoU, and OA scores on both LEVIR-CD and DSIFN-CD, with particularly strong Recall improvements over BIT and VcT. The comparisons cover nine state-of-the-art methods.

Comparison with State-of-the-art methods

  • The comparison evaluates EHCTNet against nine methods: four CNN-based architectures and five Transformer-based architectures, with BIT and VcT selected as primary benchmarks.BIT is described as a change-detection Transformer, while VcT focuses on background information.Source: Comparison with State-of-the-art methods · p. 8

Comparisons on LEVIR-CD

  • On LEVIR-CD, EHCTNet records the best Recall, F1, IoU, and OA scores, exceeding BIT by +3.69%, +1.60%, +1.37%, and +0.13%, and VcT by +11.38%, +4.96%, +4.12%, and +0.39%, respectively.The authors associate its Recall performance with identifying changed areas for applications such as detecting illegal construction.Source: Comparisons on LEVIR-CD · pp. 8–9

Comparisons on DSIFN-CD

  • On DSIFN-CD, EHCTNet achieves the highest Recall, F1, IoU, and OA, beating BIT by +15.83%, +6.07%, +4.51%, and +1.19%, and VcT by +25.49%, +6.80%, +4.35%, and +0.29%, respectively.The authors report that EHCTNet captures continuous, intact changes of interest, including increments in artificial structures and facilities.Source: Comparisons on DSIFN-CD · p. 9

Feature extraction improvement

EHCTNet’s HCT feature extractor outperformed ResNet 18 on LEVIR-CD and DSIFN-CD, while EHCTNet achieved the highest evaluation metrics among the compared methods and an IoU of 90.38% on LEVIR-CD.

Feature extraction improvement

  • On LEVIR-CD, HCT improved Recall, F1, IoU, and OA over ResNet 18 by +3.69%, +2.2%, +1.79%, and +0.18%, respectively.Source: Feature extraction improvement · p. 9
  • On DSIFN-CD, HCT improved Recall, F1, IoU, and OA over ResNet 18 by +8.75%, +5.61%, +4.05%, and +1.29%, respectively.Source: Feature extraction improvement · p. 9
  • EHCTNet achieved the highest evaluation metrics among compared methods, including an IoU of 90.38% on LEVIR-CD.The hybrid feature extractor provides more detailed features for perceiving high-level change information; the shallow-layer feature extraction baseline also performed competitively against other well-designed methods.Source: Feature extraction improvement · p. 9

Ablation studies and analysis

Ablations show that semantic token mining and frequency refinement improve change detection, while combining both refined modules yields the best reported performance and boosts Recall on both datasets.

Influence of enhanced token mining based Transformer module

  • Semantic token mining improves LEVIR-CD Recall, F1 score, IoU, and OA by +4.61%, +2.36%, +2.00%, and +0.21%, and DSIFN-CD metrics by +16.93%, +4.79%, +3.27%, and +0.45%.The module forms concept-based token bundles, uses a Transformer encoder to model global semantic relationships, and decodes semantic information into pixel-level feature maps.Source: Influence of enhanced token mining… · p. 10

Role of refined module I

  • Refined module I improves LEVIR-CD Recall, F1 score, IoU, and OA by +0.66%, +0.07%, +0.06%, and +0.00%, and DSIFN-CD metrics by +0.28%, +0.31%, +0.27%, and +0.12%.Placed between feature extraction and token mining, it extracts localized frequency information such as lines, edges, and corners to represent object extent and transitional boundaries.Source: Role of refined module I · p. 10

Impact of refined module II

  • Refined module II improves LEVIR-CD Recall, F1 score, IoU, and OA by +0.85%, +0.20%, +0.17%, and +0.01%, and DSIFN-CD Recall by +0.99%.It applies FFT, weighted gating, and inverse FFT to frequency components of the semantic difference map, then adds those components back to enhance change information.Source: Impact of refined module II · p. 10
  • EHCTNet’s refined modules together improve Recall by 0.97% on LEVIR-CD and 1.26% on DSIFN-CD; the combination achieves the best evaluation metrics on LEVIR-CD and 89.01% Recall on DSIFN-CD.The DSIFN-CD result is reported under overall high evaluation metrics.Source: Impact of refined module II · p. 10

Visualization

EHCTNet’s visualizations show progressive feature and semantic processing, with more intact change maps at later stages. Compared with BIT, VcT, and the baseline, EHCTNet more clearly distinguishes changes of interest and produces continuous detections with accurate neighboring boundaries.

Visualization comparison to SOTA

  • EHCTNet better identifies unchanged areas and produces more intact, continuous detections with clearer neighboring distinctions than BIT, VcT, and the baseline.The comparison covers LEVIR-CD and DSIFN-CD test scenes, including scenes with different lighting and dense buildings.Source: Figure 8 · p. 11, Visualization comparison to SOTA · p. 11

Network visualization

  • High-level and second-order semantic difference maps are more intact than semantic pixel maps, and the final heatmap shows progressively perceived changes.EHCTNet proceeds from raw features through HFFT-enhanced first-order features and semantic pixel features, then applies subtraction and BFFT before the detection head produces the heatmap.Source: Network visualization · pp. 10–11

Conclusion

EHCTNet combines CNN-Transformer feature extraction with token and frequency refinement for remote-sensing change detection, reporting higher Recall and improved edge detection and continuity. Its evaluation covered LEVIR-CD and DSIFN-CD, while single-source input remains a limitation and multi-source mining is future work.

Conclusion

  • EHCTNet improves edge detection and continuity, achieves higher Recall than state-of-the-art methods, and identifies subtle changes from complex reconstructions and solar radiation.The conclusion reports effectiveness on complex reconstructions and solar radiation.Source: Conclusion · p. 12
  • EHCTNet combines CNN-Transformer multi-scale feature extraction, KAN-based valuable-token selection, symmetric FFT refinement of feature and semantic frequencies, and a detection head.Its five modules are feature extraction, refined modules I and II, enhanced token mining-based Transformer, and a detection head.Source: Conclusion · pp. 11–12
  • Experiments on LEVIR-CD and DSIFN-CD, including ablations, validated effectiveness, but single-source data limits multimodal use; future work will mine multi-source imagery.The authors identify multi-source remote-sensing image change mining as future work to enhance versatility and effectiveness.Source: Conclusion · p. 12

Author contributions statement

The statement assigns complementary research, writing, and resource roles to Z.S., H.W., and J.Y., and records all authors’ approval of the published manuscript.

Author contributions statement

  • Z.S., H.W., and J.Y. shared conceptualization and review/editing; J.Y. and H.W. handled methodology and original drafting, while J.Y. led validation and data curation.J.Y. and Z.S. conducted formal analysis and investigation; Z.S. and H.W. contributed resources. All authors read and agreed to the published manuscript.Source: Author contributions statement · p. 14
Loading 2501.01238v1…