PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-domain Hyperspectral Image Classification based on Bi-directional Domain Adaptation

Yuxiang Zhang, Wei Li, Wen Jia, Mengmeng Zhang, Ran Tao, Shunlin Liang

arXiv:2507.02268v1cs.CVeess.IV

TL;DR

Cross-domain HSI classification must handle substantial spectral shifts between scenes while preserving discriminative domain-specific information. BiDA learns independent source and target adaptive spaces with a triple-branch transformer, CMCA, distillation, and ARS. Experiments report significant improvements over domain-adaptation methods, including a 3%~5% gain in cross-temporal tree species classification over the most advanced comparison method.

  • Problem

    Existing HSI domain-adaptation methods rely on shared spaces and invariant features, but significant spectral shifts and weak intra-domain feature extraction can reduce target inter-class separability.

  • Method

    BiDA uses independent source and target branches, a coupled CMCA branch, bi-directional distillation, and ARS to learn cross-domain and generalized intra-domain features.

  • Results

    BiDA performs significantly better than some state-of-the-art domain-adaptation approaches across cross-temporal and cross-scene airborne and satellite datasets.

  • Takeaways & Limitations

    BiDA improves cross-domain HSI classification by combining independent adaptive-space learning with inter-domain correlation modeling and noise-based intra-domain reinforcement.

Abstract

from arXiv · show

Utilizing hyperspectral remote sensing technology enables the extraction of fine-grained land cover classes. Typically, satellite or airborne images used for training and testing are acquired from different regions or times, where the same class has significant spectral shifts in different scenes. In this paper, we propose a Bi-directional Domain Adaptation (BiDA) framework for cross-domain hyperspectral image (HSI) classification, which focuses on extracting both domain-invariant features and domain-specific information in the independent adaptive space, thereby enhancing the adaptability and separability to the target scene. In the proposed BiDA, a triple-branch transformer architecture (the source branch, target branch, and coupled branch) with semantic tokenizer is designed as the backbone. Specifically, the source branch and target branch independently learn the adaptive space of source and target domains, a Coupled Multi-head Cross-attention (CMCA) mechanism is developed in coupled branch for feature interaction and inter-domain correlation mining. Furthermore, a bi-directional distillation loss is designed to guide adaptive space learning using inter-domain correlation. Finally, we propose an Adaptive Reinforcement Strategy (ARS) to encourage the model to focus on specific generalized feature extraction within both source and target scenes in noise condition. Experimental results on cross-temporal/scene airborne and satellite datasets demonstrate that the proposed BiDA performs significantly better than some state-of-the-art domain adaptation approaches. In the cross-temporal tree species classification task, the proposed BiDA is more than 3\%$\sim$5\% higher than the most advanced method. The codes will be available from the website: https://github.com/YuxiangZhang-BIT/IEEE_TCSVT_BiDA.

I. INTRODUCTION

BiDA addresses cross-domain HSI classification when spectral shifts make shared adaptive spaces and unidirectional adaptation inadequate. It combines independent source and target spaces with cross-domain interaction, distillation, and intra-domain reinforcement.

  • Motivation: Cross-scene HSI models face uncertain target scenes and illumination-driven spectral differences across airborne acquisitions.Single-scene classification assumes matching source and test distributions, which limits adaptability to new scenes.
  • Limitations of Existing Adaptation: Existing methods force source and target data into a shared space, but significant spectral shifts make an optimal shared solution difficult.They also emphasize invariant representations while overlooking generalized intra-domain features, reducing target inter-class separability.
  • Proposed Framework: BiDA uses a triple-branch architecture with independent source and target adaptive spaces plus a coupled branch for cross-domain classification.The framework targets airborne and satellite HSIs and constructs semantic tokens from spatial-spectral characteristics.
  • Training Strategy: Bi-directional distillation uses coupled-branch predictions as soft labels to supervise both source and target branches.This supervision reinforces independent adaptive-space learning.
  • Training Strategy: ARS applies intra-domain consistency constraints under noise to encourage generalized feature extraction in source and target data.The strategy uses teacher-student training and is designed for domain-adaptation models.
  • Proposed Framework: CMCA performs bi-directional cross-attention between source and target tokens to perceive inter-domain correlations.The coupled branch supports feature interaction between the two domains.

A. Semantic Tokenizer

BiDA’s semantic tokenizer jointly encodes HSI spatial-spectral information into compact tokens rather than using standard spatial patch tokenization. It generates attention-weighted tokens from 13×13×d HSI patches.

  • Tokenizer Design: The semantic tokenizer jointly encodes the strong spatial and multispectral characteristics of HSI data.It is designed specifically for transformer-based processing of hyperspectral inputs.
  • Tokenizer Design: Unlike ViT patch tokenization, the tokenizer uses a learned spatial-spectral projection to generate source and target tokens.The original HSI input has dimensions 13×13×d in the described experiment.
  • Token Construction: A spatial-spectral extractor produces a 13×13×L projection, and softmax-normalized attention maps aggregate pixel information into tokens.L is set to 4 in the experiments; Conv3d, Conv2d, ReLU, and pooling blocks form the extractor.
  • Token Construction: The learnable classification token supports classification and uncertainty-based pseudolabel learning.The classification token is denoted Tcls^s/t for the source and target domains.

B. Triple-branch Encoder

The triple-branch encoder learns independent source and target adaptive spaces while modeling inter-domain correlations through coupled cross-attention and distillation.

  • Independent adaptive spaces: Source and target branches use MSA and FFN blocks to learn intra-domain token representations in independent adaptive spaces.The encoder applies self-attention to model within-domain context before producing adaptive-space representations.
  • Coupled branch: The coupled branch uses CMCA to derive coupled token representations for bidirectional inter-domain correlation modeling.The representations include target relative to source and source relative to target.
  • Cross-domain pairing: Uncertainty-filtered target predictions are used as pseudo-labels and paired with same-class source samples during training.Target samples satisfying H(q_t,i) ≤ 0.5 × log(C) are retained.
  • Bidirectional supervision: Bi-directional distillation uses coupled-branch source-to-target and target-to-source probability distributions as soft labels for both branches.The resulting losses supervise independent adaptive-space learning in the source and target domains.
  • Distribution alignment: MMD measures marginal distribution discrepancy between source and target adaptive spaces and similarity differences between coupled representations.The loss combines discrepancies for independent and coupled token representations.

C. Adaptability Reinforcement Strategy

ARS reinforces generalized intra-domain features by applying different noise to source and target inputs and enforcing teacher–student consistency, with EMA updating the teacher.

  • Noise-based reinforcement: ARS applies different noise conditions to source and target inputs before processing them with BiDA teacher and student models.The flowchart produces two semantic-token variants for each domain.
  • Consistency learning: Intra-domain consistency losses on source and target token representations update the student model.The consistency constraint is defined as mean squared error between teacher and student outputs.
  • Motivation: ARS targets domain-specific internal features that conventional domain adaptation may overlook when focusing on domain-invariant representations.Its stated purpose is to improve extraction of generalized features while reducing spectral shift.
  • Teacher–student update: The teacher model is updated by exponential moving average, while the student model is used for inference.The total loss updates the student, and EMA aggregates the teacher’s forward-time knowledge.

III. EXPERIMENTAL RESULTS AND ANALYSIS

Experiments evaluate BiDA on cross-temporal airborne and satellite datasets against transformer-based and unsupervised domain-adaptation methods using standard classification metrics.

  • Evaluation datasets: BiDA is evaluated on MFF, Houston, and HyRANK cross-temporal or cross-scene datasets.MFF and HyRANK are identified as fine-grained cross-domain classification tasks.
  • Comparison methods: The comparison includes classic transformer-based algorithms and state-of-the-art unsupervised deep domain-adaptation methods.Listed baselines include GAHT, MLUDA, MSDA, MDGTnet, TSTnet, CLDA, SCLUDA, SSWADA, and CACL.
  • Protocol and metrics: All methods use labeled source-domain data and target-domain data without target-label information.Classification Accuracy, Overall Accuracy, and Kappa are used for evaluation.

A. Experimental Data

The experiments cover airborne and satellite hyperspectral scenes with temporal or geographic domain shifts, including tree-species and cross-scene classification settings.

  • MFF cross-temporal airborne dataset: MFF uses airborne hyperspectral data from five tree species collected across multiple flight dates.The dataset is constructed for cross-temporal classification at Mengjiagang Forest Farm.
  • MFF cross-temporal airborne dataset: MFF’s repeated acquisitions produce substantial spectral differences for the same tree species despite geographic proximity.The temporal span covers four flights between May 31 and June 15, 2017.
  • Dataset splits: The MFF and Houston datasets include tabulated source and target sample counts for their experimental splits.The referenced tables report MFF SD, TD1, and TD2 counts and Houston source and target counts.
  • Houston cross-temporal satellite dataset: The Houston dataset contains 2013 and 2018 satellite scenes acquired by different sensors in different years.The scenes differ in spectral-band count while sharing the 380–1050 nm wavelength range.
  • HyRANK cross-scene satellite dataset: HyRANK provides two labeled satellite scenes, Dioni and Loukia, acquired by the Hyperion sensor.The scenes contain 176 spectral bands and differ in spatial dimensions.

B. Parameter tuning

BiDA tunes regularization and tokenization choices across multiple cross-domain HSI datasets, while semantic tokenization uses compact representations that support classification.

  • Regularization parameters: λ1 and λ2 control the contributions of domain alignment and ARS, with optimal values of 1e −1 and 1e + 0 for MFF-TD1, MFF-TD2, Houston 2018, and Loukia.The parameters are selected by evaluating OA across combinations on all experimental datasets.
  • Experimental data: The MFF dataset includes source and target regions MFF-SD, MFF-TD1, and MFF-TD2 for cross-temporal evaluation.The dataset contains labeled samples distributed across the five tree-species classes.
  • Experimental data: The five MFF classes are Larch, Mongolian pine, Korean pine, Spruce, and Broad-leaved trees.Class labels are reported consistently for the source and target regions.
  • Token number: Increasing the token count tends to degrade BiDA classification performance, indicating that a small number of tokens is sufficient for HSI representation.The semantic tokenizer incorporates spatial and spectral information from 13×13×d inputs.

C. Ablation study

Ablation studies evaluate BiDA’s losses, branches, cross-attention, noise strategy, and tokenizer, showing contributions from semantic tokenization, CMCA, and ARS.

  • Loss ablation: The BiDA backbone improves target-domain accuracy by 0.7% to 3% over other transformer-based methods, while the loss components provide additional gains.The evaluated losses include classification, MMD, bi-directional distillation, and ARS intra-domain consistency losses.
  • Branch and CMCA ablation: Adding the target branch and MMD loss improves classification accuracy by 2% to 4% across all datasets.The ablation uses unlabeled target-domain data for feature alignment.
  • Branch and CMCA ablation: Replacing MCA with CMCA improves classification accuracy by about 2% across all datasets.The reported gain is attributed to CMCA’s bi-directional interaction for acquiring domain-invariant features.
  • ARS noise ablation: Adding RandomRC and Gauss noise improves classification accuracy by 2% to 4%, whereas all three tested noise types are optimal only for Houston 2018.The three noise types are RandomRC, Gaussian noise, and radiation noise.
  • Tokenizer ablation: The semantic tokenizer exceeds the patch tokenizer by 8.34%, 7.75%, 7.3%, and 8.18% on the four target domains.The result supports token construction aligned with HSI spatial and multi-band spectral characteristics.

D. Performance on MFF cross-temporal airborne dataset

On the MFF cross-temporal airborne dataset, BiDA outperforms transformer and unsupervised domain-adaptation baselines and produces more coherent tree-species classification under spectral shifts.

  • Baseline comparison: GAHT achieves OAs of 67.38% and 68.61% on MFF-TD1 and MFF-TD2, respectively, among transformer-based methods without domain-adaptation strategies.The reported comparison includes GAHT, MLUDA, MSDA, MDGTnet, CLDA, SCLUDA, SSWADA, and CACL.
  • Quantitative comparison: 10.02% and 6.47% are BiDA’s improvements over GAHT on MFF-TD1 and MFF-TD2, respectively.BiDA achieves 77.40% on MFF-TD1 and 75.08% on MFF-TD2 in the reported classification maps.
  • Quantitative comparison: 4.6% and 2.8% are BiDA’s higher classification accuracies than MLUDA and MSDA on MFF-TD1 and MFF-TD2, respectively.The comparison identifies MLUDA and MSDA as the best-performing unsupervised deep domain-adaptation methods on this dataset.
  • Visual and feature analysis: BiDA’s classification maps show regionally coherent and accurate tree-species predictions with minimal influence from spectral shifts between flight lines.Most comparison methods show noise, prediction errors, and fragmentation in several tree-species classes.
  • Visual and feature analysis: BiDA projects source and target data into an adaptive space to address spectral shifts and improve class separability.The original MFF domains show distribution gaps and overlapping intra-domain classes before transfer learning.

E. Performance on cross-scene/temporal satellite dataset

BiDA is evaluated on Houston and HyRANK satellite datasets, with HyRANK posing greater cross-scene interpretation difficulty than the MFF airborne dataset. On Houston 2018 and Loukia, BiDA improves transfer performance over TSTnet and CACL and produces less noisy predictions.

  • Dataset characteristics: HyRANK contains 12 classes, including multiple tree species, and is more difficult for high-precision cross-scene interpretation than the MFF airborne dataset.Houston contains 7 coarse-grained land-cover classes, whereas HyRANK includes tree-species classes.
  • Quantitative results: 1.85% and 0.77% improvements over TSTnet and CACL, respectively, are reported on Houston 2018 and Loukia transfer performance.The comparison uses CA, OA, and KC results presented in Tables XIII–XIV.
  • Comparative performance: All methods using domain adaptation outperform the single-scene classification method GAHT on the evaluated satellite transfers.The comparison is reported for Houston 2018 and Loukia data.
  • Qualitative results: BiDA achieves more accurate predictions with less noise in multiple Houston 2018 regions compared with the ground-truth-referenced predictions of other methods.The qualitative comparison is shown through classification maps in Fig. 10.

IV. CONCLUSIONS

The paper proposes BiDA for cross-scene and cross-temporal HSI classification using independent source and target adaptive spaces with coupled feature alignment. Extensive experiments report improved performance across airborne and satellite datasets, including a more than 3%–5% advantage in cross-temporal tree-species classification.

  • Conclusion: BiDA uses a triple-branch transformer with source, target, and coupled branches plus a semantic tokenizer for cross-scene and cross-temporal HSI classification.The coupled branch supports interaction between source and target representations.
  • Conclusion: CMCA performs bi-directional feature alignment, while bi-directional distillation supervises independent adaptive-space learning for source and target branches.These components use inter-domain relationships to guide feature learning.
  • Conclusion: ARS addresses overlooked domain-specific generalized features within the target domain during adaptation.The strategy complements invariant-feature learning by targeting target-domain internal structure and characteristics.
  • Conclusion: More than 3%–5% higher performance than the most advanced method is reported for cross-temporal tree-species classification.The paper also reports extensive gains across three cross-temporal or cross-scene airborne and satellite datasets.
Loading 2507.02268v1…