PaperlayerView exact revision on arXiv

Source-linked AI summary

Single-source Domain Expansion Network for Cross-Scene Hyperspectral Image Classification

Yuxiang Zhang, Wei Li, Weidong Sun, Ran Tao, Qian Du

arXiv:2209.01634v1cs.CV

TL;DR

Cross-scene HSI classification often requires labeled target-domain data, but some applications allow training only on source-domain data before direct transfer. SDEnet addresses this setting with domain expansion and supervised contrastive adversarial learning, improving classification by 4% to 8% over DSAN on all target domains.

  • Problem

    Cross-scene HSI classification has not considered training with only labeled source-domain data and direct transfer to an unlabeled target domain, despite settings where target data cannot be reused for training.

  • Method

    SDEnet uses a generator with spatial-spectral semantic and morph encoders to create an extended domain, while supervised contrastive adversarial learning trains class-wise domain-invariant representations.

  • Results

    4% to 8% improvement over DSAN was achieved on all target domains in cross-scene HSI classification.

  • Takeaways & Limitations

    The experiments demonstrate the superiority of SDEnet over state-of-the-art techniques across two public HSI datasets and one additional MSI dataset.

Abstract

from arXiv · show

Currently, cross-scene hyperspectral image (HSI) classification has drawn increasing attention. It is necessary to train a model only on source domain (SD) and directly transferring the model to target domain (TD), when TD needs to be processed in real time and cannot be reused for training. Based on the idea of domain generalization, a Single-source Domain Expansion Network (SDEnet) is developed to ensure the reliability and effectiveness of domain extension. The method uses generative adversarial learning to train in SD and test in TD. A generator including semantic encoder and morph encoder is designed to generate the extended domain (ED) based on encoder-randomization-decoder architecture, where spatial and spectral randomization are specifically used to generate variable spatial and spectral information, and the morphological knowledge is implicitly applied as domain invariant information during domain expansion. Furthermore, the supervised contrastive learning is employed in the discriminator to learn class-wise domain invariant representation, which drives intra-class samples of SD and ED. Meanwhile, adversarial training is designed to optimize the generator to drive intra-class samples of SD and ED to be separated. Extensive experiments on two public HSI datasets and one additional multispectral image (MSI) dataset demonstrate the superiority of the proposed method when compared with state-of-the-art techniques.

I. INTRODUCTION

Cross-scene HSI classification must generalize from labeled source-domain data to uncertain target scenes whose spectral reflectance varies, while domain adaptation requires target-domain access during training. SDEnet addresses this single-source domain-generalization setting by expanding domains with spatial-spectral randomization, morphologically invariant features, and supervised contrastive adversarial learning.

  • Motivation: Cross-scene HSI classification suffers from poor generalization because labeled samples are costly and scene, sensor, seasonal, and weather differences alter spectral reflectance.These variations affect samples from the same land-cover classes across source and target domains.
  • Problem Setting: Domain adaptation accesses unlabeled target data during training, whereas domain generalization trains only on labeled source data before transfer to the target domain.The single-source setting is especially relevant when target data cannot be reused during training, such as on resource-limited spaceborne platforms.
  • Proposed Framework: SDEnet is proposed as a domain-generalization framework that trains with source data and directly transfers to target domains through generative adversarial learning.Its extended domain is generated from single-source data to cover target-domain shifts while retaining reliability constraints.
  • Proposed Framework: The semantic encoder applies spatial and spectral randomization, while the morph encoder extracts domain-invariant template features for effective and reliable extended-domain samples.Spatial-spectral information supplies variable domain characteristics, and morphological knowledge provides implicitly invariant information.
  • Proposed Framework: Supervised contrastive adversarial learning makes the discriminator learn class-wise domain-invariant representations while the generator produces out-of-domain data.The generator and discriminator compete iteratively to improve domain expansion and generalization capability.

II. RELATED WORK

Domain generalization learns from source-domain data without accessing the target domain during training, unlike domain adaptation. Existing approaches emphasize domain-invariant representations or data manipulation, while contrastive learning separates similar and dissimilar samples in projection space.

  • Domain Generalization: Domain generalization trains with source-domain data only and transfers the model to the target domain during inference.This differs from domain adaptation, which accesses unlabeled target-domain data during training.
  • Domain Generalization: Existing domain-generalization methods mainly learn domain-invariant representations or manipulate data through augmentation and generation.Representation methods reduce domain shift, whereas data manipulation creates or augments out-of-domain samples related to the source domain.
  • Domain-Invariant Representation: Domain-adversarial learning trains discriminators to distinguish domains while generators learn representations that fool them.This strategy is primarily associated with learning domain-invariant representations.
  • Data Generation: AdaIN maps random noise through fully connected layers to style means and variances applied to normalized feature maps for data generation.The supplied AdaIN description identifies random noise, style statistics, and normalized feature maps as the calculation components.
  • Contrastive Learning: Contrastive learning constructs positive and negative pairs so positive samples approach and negative samples separate in projection space.Similarity is typically measured with a dot product or cosine distance, regulated by temperature parameter τ.

III. PROPOSED SINGLE-SOURCE DOMAIN EXPANSION NETWORK

SDEnet combines a generator and discriminator for single-source domain expansion in hyperspectral classification. Its design constrains generated-domain effectiveness and reliability through spatial-spectral generation, morphological features, adversarial learning, supervised contrastive learning, and classification loss.

  • Network Overview: SDEnet consists of a generator and discriminator that jointly support single-source domain expansion.The generator creates extended-domain samples, while the discriminator learns representations across domains.
  • Network Overview: The generator processes 13×13×d source-domain patches through semantic and morphological encoders before mapping features back to patch size.The semantic encoder performs spatial and spectral randomization, while the morph encoder uses Dilation2D and Erosion2D convolutions.
  • Design Constraints: Spatial-spectral generation and adversarial training with supervised contrastive learning target extended-domain effectiveness.These components generate diverse samples and optimize class-wise representation learning across domains.
  • Design Constraints: Morphological knowledge supplies domain-invariant template features, while extended- and interpolated-domain classification loss supports reliability.The reliability constraints are expressed through both network structure and loss design.

A. Domain Expansion Generator

The domain-expansion generator creates extended-domain samples by separately randomizing spatial and spectral semantic information while retaining morphological template features. AdaIN-based style manipulation, spectral content replacement, and morphological operations together provide diverse but constrained generated samples.

  • Generator Overview: The generator maps a source-domain sample X to an extended-domain sample ˆX with a domain shift, and forms interpolated-domain samples between them.Interpolated-domain samples are intended to ease learning when source and extended domains differ substantially.
  • Semantic Encoder: The semantic encoder performs spatial randomization for style replacement and spectral randomization for content replacement.This division reflects spatial configuration as content and spectral information as class-related characteristics in HSI patches.
  • Semantic Encoder: Spatial randomization computes channel-wise means and standard deviations, adaptively mixes them across minibatch features, and applies the resulting style statistics.The spatial contents are retained while their style statistics are replaced through AdaIN-based processing.
  • Semantic Encoder: Spectral randomization compresses a 13×13 patch into 1×1 spectral embeddings, randomly replaces minibatch content, and reconstructs the feature map by deconvolution.AdaIN is used to maintain the style of spectral embeddings during content replacement.
  • Morph Encoder: The morph encoder reduces each hyperspectral patch to a grayscale-like map and extracts template features using Dilation2D and Erosion2D operations.These operations compute local maximum- and minimum-based combinations with learned structural elements.
  • Morph Encoder: Opening, closing, top-hat, and black-hat branches are concatenated into template features, followed by AdaIN randomization.The opening and closing branches use successive dilation and erosion operations without shared weights.

B. Domain Invariant Discriminator

The discriminator combines classification and supervised contrastive learning to extract class-wise domain-invariant representations from SD, ED, and ID. Adversarial optimization reverses the contrastive relationship for generator outputs, encouraging ED to retain labels while containing domain-specific information.

  • Discriminator architecture: The discriminator receives SD, ED, and ID through a feature extractor, classification head, and projection head for representation learning.The feature extractor uses stacked Conv2d-ReLU-MaxPool2d blocks, while the classification head outputs prediction probabilities.
  • Classification constraints: Cross-entropy losses require correct predictions for SD, ED, and ID, using SD labels because all three share the same label space.ID serves as a transition between SD and ED, reducing the pressure on the discriminator to learn ED domain features.
  • Contrastive representation learning: Supervised contrastive learning pulls same-class samples together and pushes different-class samples apart in the discriminator embedding space.Positive and negative sets are formed from class membership within the minibatch.
  • Adversarial generator objective: Adversarial contrastive optimization treats same-class SD samples as positives and same-class ED or ID samples as negatives for the generator.This separates corresponding SD and generated samples, encouraging ED to contain domain-specific information that challenges domain-invariant extraction.

C. Training Phase

SDEnet alternately optimizes the discriminator and generator with classification, supervised contrastive, and adversarial objectives. The discriminator learns invariant representations, while the generator is constrained to produce correctly labeled but domain-shifted ED samples.

  • Discriminator optimization: The discriminator is optimized using SD, ED, and ID as inputs and a weighted combination of their classification losses plus supervised contrastive loss.The objective minimizes L = LSD + LED + LID + λ1Lsupcon.
  • Generator optimization: The generator uses adversarial learning together with ED classification loss to preserve the correct label space while creating domain-specific variation.The ED classification objective acts as an auxiliary loss during generator optimization.
  • Loss balancing: The balancing coefficients λ1 and λ2 are hyper-parameters, and both are set to the same value for simplicity.They balance the supervised contrastive and adversarial learning losses, respectively.

D. Generalization Performance of SDEnet

On Houston, SDEnet produces ED samples with within-class discrimination and domain shift, while its learned features improve class separability and reduce source–target distribution discrepancy during direct TD inference.

  • Extended-domain behavior: SDEnet creates same-class ED samples with domain shift from SD while preserving discriminative information, and its discriminator reduces the feature-level shift.The Houston visualization shows better aggregation of same-class ED samples and improved class-wise representation after projection.
  • Target-domain generalization: Direct inference on Houston 2018 improves separability: inter-class distances increase and intra-class distances decrease after SDEnet feature embedding.The original target-space distributions are mixed, whereas the embedded feature space shows clearer class structure.
  • Figure interpretation: The Houston class-separability figure encodes SD with bullets, ED with crosses, and class identity with numeric labels.It compares original samples and SDEnet-derived features across the displayed class distributions.
  • Distribution discrepancy: About 0.1 reduction in mmd(SD, TD) is observed between the original distribution gap and the SDEnet feature-space gap.The comparison is reported for the Houston dataset in Table IV.

IV. EXPERIMENTAL RESULTS AND DISCUSSION

Experiments evaluate SDEnet across three cross-scene datasets and compare it with state-of-the-art domain adaptation and domain generalization methods using standard classification metrics.

  • Datasets: Experiments use the Houston, Pavia, and GID datasets to validate SDEnet across three cross-scene remote-sensing benchmarks.The GID dataset is identified as the Gaofen Image Dataset.
  • Compared methods: Comparisons include DAAN, DSAN, MRAN, HTCNN, PDEN, LDSDG, and SagNet, spanning transfer-learning and domain-generalization methods.The listed baselines include both domain adaptation and domain generalization approaches.
  • Evaluation metrics: Performance is assessed using class-specific accuracy, overall accuracy, and Kappa metrics.These metrics are used to compare the classification methods across the experimental datasets.

A. Experimental Data

Experiments use Houston, Pavia, and GID scene pairs, with parameter tuning, classification-map comparisons, and ablation evaluation reported for SDEnet.

  • Datasets: Houston, Pavia, and GID provide cross-scene datasets, while GID additionally evaluates SDEnet on multispectral imagery.Houston includes 2013/2018 scenes, Pavia includes University of Pavia/Pavia Center, and GID uses Nanchang as source and Wuhan as target.
  • Experimental design: SDEnet’s evaluation includes parameter sensitivity, ablation comparison, classification accuracy, and one-epoch execution-time measurements.The reported settings tune η, λ, and dse, while Tables XI–XV cover ablation, accuracy, and runtime.
  • Datasets: The experiments compare pseudo-color images and ground-truth maps for Houston, Pavia, and GID scene pairs.Figures 6–8 visualize the input scenes and ground-truth layouts before classification-map comparisons.
  • Classification-map evaluation: 81.76% is reported for SDEnet on Pavia Center, compared with 80.87% for PDEN and 78.94% for DSAN.Figure 10 presents the target-scene classification maps and associated percentages.
  • Implementation: SDEnet is implemented with 13×13 patches, Adam optimization, and 1e-4 ℓ2-norm weight decay.These implementation choices are stated for both generator and discriminator training.

C. Ablation Study

The ablation study removes the semantic encoder, morph encoder, supervised contrastive learning, and adversarial training to assess their contributions.

  • Ablation design: The ablation variants separately delete the semantic encoder, morph encoder, contrastive learning, or adversarial training.These variants are named SDEnet (no se), (no me), (no con), and (no adv).
  • Encoder components: Removing either generator encoder sharply reduces performance, showing that semantic and morph encoders support the effectiveness and reliability of the extended domain.The morph encoder is especially important for GID, where removing it reduces OA by 13%.
  • Contrastive learning: Removing supervised contrastive learning lowers classification performance by 1%–9% across target domains.The study attributes this component to learning class-wise domain-invariant representations by comparing source and extended-domain samples.

D. Performance on Cross-Scene HSI Classification

SDEnet is evaluated under source-only training against domain-adaptation and domain-generalization methods across Houston 2018, Pavia Center, and GID-wh.

  • Baseline comparison: DA and DG baselines perform differently by scene: DSAN leads on Houston 2018, PDEN on Pavia Center, and LDSDG on GID-wh.The reported OA differences between methods are approximately 2% in the corresponding scene comparisons.
  • Overall comparison: SDEnet improves OA by 4%–8% over DSAN on all target domains.DSAN accesses unlabeled target data during training, whereas SDEnet generates an extended domain from the source domain.
  • Overall comparison: SDEnet increases OA by 1%–4% over computer-vision DG methods including PDEN and LDSDG.The paper links this result to HSI-specific semantic and morph encoders rather than spatial-only variation modeling.
  • MSI evaluation: The spatial-spectral generation strategy remains effective on GID, an MSI dataset with only four bands.This extends the reported application beyond HSI datasets to multispectral imagery.
  • Classification maps: SDEnet produces less noisy and more accurate maps in reported areas including Houston 2018 Grass stressed and Pavia Center Brick.The figures display labeled pixels as ground truth or backgrounds depending on the target-scene visualization.
  • Computational complexity: SDEnet has lower computational cost than comparison methods because it uses shallow convolutional blocks and a low-complexity double-branch encoder.The comparison is based on one-epoch training time measured on the stated CPU/GPU platform.

V. CONCLUSIONS

SDEnet performs source-only cross-scene domain generalization through generative adversarial learning and an encoder-randomization-decoder generator.

  • Conclusion: SDEnet transfers from source to target domains using only source-domain data through generative adversarial learning.The framework generates an extended domain during training to support target-domain generalization.
  • Conclusion: Its generator combines spatial-spectral randomization in a semantic encoder with template-feature extraction in a morph encoder.Together, these components generate an extended domain while incorporating domain-invariant morphological information.
Loading 2209.01634v1…