PaperlayerView exact revision on arXiv

Source-linked AI summary

PolSAR Image Classification using a Hybrid Complex-Valued Network (HybridCVNet)

Mohammed Q. Alkhatib

arXiv:2605.31137v1cs.CV

TL;DR

HybridCVNet addresses PolSAR classification methods that overlook phase information by combining complex-valued CNNs and vision transformers. Across Flevoland and San Francisco, it outperforms the compared methods, reaching 97.39% overall accuracy on Flevoland and 93.32% average accuracy on San Francisco.

  • Problem

    Real-valued ViT-based PolSAR classifiers predominantly overlook phase information in complex-valued PolSAR data.

  • Method

    HybridCVNet combines CV-CNNs and CV-ViTs, using coherency-matrix features and complex-valued operations to exploit phase information and complementary feature maps.

  • Results

    97.39% overall accuracy on Flevoland and 93.32% average accuracy on San Francisco, outperforming the compared methods.

  • Takeaways & Limitations

    The results support combining complex-valued CNN and transformer architectures for PolSAR classification across two benchmark datasets.

Abstract

from arXiv · show

Recently, convolutional neural networks (CNNs) have become popular for image classification due to their effectiveness in computer vision tasks. Now, researchers are exploring the potential of vision transformers (ViTs) in remote sensing and Earth observation. However, traditional Real-Valued networks often overlook important phase information in Complex-Valued (CV) data like polarimetric synthetic aperture radar (PolSAR) data. To address this, new CV deep architectures have emerged. HybridCVNet, a novel hybrid network, blends CV-CNN and CV vision transformer (CV-ViT) techniques. It efficiently combines CV 3D and 2D CNNs as feature extractors, enhancing PolSAR image classification by extracting complementary information and effectively leveraging interdependencies within the data. Experimental results from widely-used PolSAR datasets show HybridCVNet outperforms other methods, achieving an overall accuracy of 97.39% on the Flevoland dataset and showing promise even with just a 1% sampling ratio, with a Kappa value of 0.972 on the San Francisco dataset. Source code is accessible through https://github.com/mqalkhatib/HybridCVNet

I. INTRODUCTION

HybridCVNet addresses limitations of real-valued PolSAR classifiers by combining complex-valued CNN and vision-transformer components. The approach is motivated by preserving phase information while modeling complementary local and global features.

  • PolSAR images support detailed Earth observation in all weather conditions and can penetrate surfaces effectively.
  • Real-valued CNNs extract spatial and polarimetric features but overlook PolSAR phase information.
  • Transformers model global image correlations, whereas convolution primarily captures local features.
  • HybridCVNet combines CV-CNNs and CV-ViTs to exploit phase information, coherency-matrix correlations, and complementary feature maps.
  • The framework was validated on two widely employed PolSAR datasets against real-valued transformer methods and related classifiers.

A. Polarimetric Data of PolSAR Image

PolSAR data are represented through the coherency matrix, whose Hermitian structure yields three real-valued and three complex-valued upper-triangular features per pixel. These six-dimensional complex vectors are then arranged into patches for model processing.

  • The scattering matrix describes how ground objects scatter polarized electromagnetic waves.
  • The coherency matrix is the most commonly used representation for PolSAR data.
  • The coherency matrix is Hermitian, with real-valued diagonal elements and complex-valued off-diagonal elements.
  • Three real-valued and three complex-valued upper-triangular elements form a 6-dimensional complex feature vector for each pixel.
  • The PolSAR image is divided into patches using sliding windows for subsequent processing.

B. Feature Extraction using CV-CNN

The feature extractor uses a hierarchical architecture in which complex-valued CNN layers process PolSAR data while retaining their complex representation. Its 3D and 2D CNN components target complementary feature types.

  • All layers in the hierarchical CNN feature extractor are converted to complex-valued operations.
  • The 3D CV-CNN extracts joint polarimetric-spatial features from PolSAR data.
  • The architecture uses CV-CNN feature extraction to address challenges posed by complex-valued PolSAR imagery.

C. Complex-Valued ViTs

HybridCVNet translates ViT layers into complex-valued layers and combines their global-context modeling with CV-CNN local feature extraction. The extracted representation is partitioned into patches before CV-ViT classification.

  • C. Complex-Valued ViTs: Complex-valued ViT layers are used to handle PolSAR data while retaining phase information.
  • C. Complex-Valued ViTs: CV-CNNs capture local spatial-polarimetric patterns, while CV-ViTs model long-range dependencies and global context through self-attention.
  • C. Complex-Valued ViTs: A 15 × 15 × 6 CV PolSAR patch is converted into 25 flattened patches of size 3×3×6 for CV-ViT processing.
  • The Flevoland classification results compare reference maps with outputs from CNN, transformer, hybrid real-valued, and HybridCVNet models.

III. EXPERIMENTAL RESULTS

HybridCVNet was evaluated on the Flevoland and San Francisco PolSAR datasets using limited training data and standardized patch-based settings.

  • Datasets: The evaluation used two widely used PolSAR datasets: Flevoland and San Francisco.Flevoland contains 15 agricultural classes, while San Francisco contains five terrain classes.
  • Datasets: The Flevoland dataset has 750 × 1024 pixels, 12-meter spatial resolution, and 15 agricultural classes.It was acquired by the NASA/JPL AIRSAR system over Flevoland in 1989.
  • Datasets: The San Francisco dataset has 900 × 1024 pixels, 10-meter spatial resolution, and five categorized terrain classes.It was acquired by L-band AIRSAR over San Francisco in 1989.
  • Experimental settings: Both datasets were represented using 15 × 15 patches, with 1% of samples for training and 99% for testing.Training used a learning rate of 1 × 10^-3, batch size 64, and 100 epochs with early stopping.

B. Classification Results

HybridCVNet achieved the strongest reported classification results on both benchmark datasets, outperforming real-valued, convolutional, and transformer-based comparison methods.

  • Flevoland results: 97.39% OA was achieved by HybridCVNet on Flevoland, exceeding 3D-CNN, WaveletCNN, ViT, Swin Transformer, PolSARFormer, and HybridRVNet.The model exceeded Swin Transformer and PolSARFormer by 3.81% and 1.65% OA, respectively.
  • Qualitative results: HybridCVNet produced a more uniform Flevoland land-cover map with reduced noise than the other evaluated models.The qualitative comparison was based on classification maps for the Flevoland region.
  • San Francisco results: 93.32% AA was achieved by HybridCVNet on San Francisco, exceeding PolSARFormer at 87.73% and HybridRVNet at 91.21%.The reported AA values for 3D-CNN, WaveletCNN, ViT, and Swin Transformer were 89.89%, 87.03%, 85.97%, and 82.51%.
  • Qualitative results: The San Francisco classification maps compared a reference map with outputs from 3D-CNN, WaveletCNN, ViT, Swin Transformer, PolSARFormer, HybridRVNet, and HybridCVNet.The figure also included a class legend.
  • Qualitative results: The zoomed comparisons covered selected white-box regions from the Flevoland and San Francisco classification figures.The San Francisco white-box region corresponded to the Mountain class and was described as easily distinguishable.

C. Ablation study

The ablation study found that combining CV-CNNs and CV-ViTs provided the strongest classification results under the 1% training setting.

  • Component combinations: 97.15% OA was obtained on Flevoland when CV-CNNs and CV-ViTs were combined.This exceeded the individual CV-CNN and CV-ViT results of 96.65% and 95.75%, respectively.
  • Component combinations: On San Francisco, combining CV-CNNs and CV-ViTs improved accuracy by 0.48% and 1.53% relative to using CV-CNNs and CV-ViTs separately.The comparison isolates the benefit of integrating both component types under the same ablation setting.
  • Component comparisons: CV-CNNs surpassed CV-ViTs in accuracy in the ablation results.The passage also notes that transformer-based classifiers require substantial training data.

D. Models Performance at Different Percentages of Training Data

HybridCVNet’s classification accuracy improves as the training-data percentage rises from 1% to 5%. At low training percentages, CNN-based models outperform transformers, but ViT surpasses 3D-CNN as training data increases, while HybridCVNet also improves across OA, AA, and Kappa.

  • Classification accuracy improves as the training-data percentage increases from 1% to 5%.The assessment uses Overall Accuracy, Average Accuracy, and Kappa index across these training ratios.
  • At low training percentages, CNN-based models outperform ViT and Swin because transformer models require more training data.
  • As training data increases, ViT surpasses 3D-CNN in Overall Accuracy, Average Accuracy, and Kappa.
  • 1.75%, 2.58%, and 1.91% are HybridCVNet’s approximate increases in OA, AA, and Kappa, respectively, across the 1%–5% training-data range.

IV. CONCLUSION

HybridCVNet combines CV-CNN and CV-ViT architectures and outperforms current state-of-the-art models on two benchmark PolSAR datasets. Its higher computational costs and longer training times arise from the use of complex-valued operations.

  • HybridCVNet combines CV-CNN and CV-ViT architectures for PolSAR image classification.
  • On Flevoland, HybridCVNet improves OA by 2.8% over PolSARFormer and by 5.34% over Swin Transformer.
  • On San Francisco, HybridCVNet attains AA of 93.32%, compared with PolSARFormer’s 87.73%.
  • Higher computational costs and longer training times result from the model’s complex-valued operations.Early stopping partly mitigated this limitation.
Loading 2605.31137v1…