PaperlayerView exact revision on arXiv

Source-linked AI summary

A Spectral-Spatial-Dependent Global Learning Framework for Insufficient and Imbalanced Hyperspectral Image Classification

Qiqi Zhu, Weihuan Deng, Zhuo Zheng, Yanfei Zhong, Qingfeng Guan, Weihua Lin, Liangpei Zhang, Deren Li

arXiv:2105.14327v1cs.CV

TL;DR

HSI classification is constrained by limited, imbalanced labels and redundant patchwise processing. SSDGL addresses these issues through global spectral-spatial modeling, balanced sampling, weighted loss, and attention, with reported ablations showing substantial component-level improvements.

  • Problem

    HSI classification faces high-dimensional data, limited labels, redundant patchwise computation, and imbalanced class distributions.

  • Method

    SSDGL combines whole-image global learning with H-B sampling, weighted softmax loss, GCL spectral dependency modeling, and GJAM attention.

  • Results

    The reported ablations show OA increasing from 58.39% to 95.69% with GCL, from 95.69% to 96.97% with GJAM, and from 96.77 to 99.63 with HB-WL.

  • Takeaways & Limitations

    SSDGL provides a global HSI classification framework designed for insufficient and imbalanced samples while avoiding pixel-patch inputs.

Abstract

from arXiv · show

Deep learning techniques have been widely applied to hyperspectral image (HSI) classification and have achieved great success. However, the deep neural network model has a large parameter space and requires a large number of labeled data. Deep learning methods for HSI classification usually follow a patchwise learning framework. Recently, a fast patch-free global learning (FPGA) architecture was proposed for HSI classification according to global spatial context information. However, FPGA has difficulty extracting the most discriminative features when the sample data is imbalanced. In this paper, a spectral-spatial dependent global learning (SSDGL) framework based on global convolutional long short-term memory (GCL) and global joint attention mechanism (GJAM) is proposed for insufficient and imbalanced HSI classification. In SSDGL, the hierarchically balanced (H-B) sampling strategy and the weighted softmax loss are proposed to address the imbalanced sample problem. To effectively distinguish similar spectral characteristics of land cover types, the GCL module is introduced to extract the long short-term dependency of spectral features. To learn the most discriminative feature representations, the GJAM module is proposed to extract attention areas. The experimental results obtained with three public HSI datasets show that the SSDGL has powerful performance in insufficient and imbalanced sample problems and is superior to other state-of-the-art methods. Code can be obtained at: https://github.com/dengweihuan/SSDGL.

I. INTRODUCTION

HSI classification is challenged by high dimensionality, limited labels, patchwise redundancy, and imbalanced classes. SSDGL addresses these issues with balanced sampling, weighted loss, spectral-spatial dependency modeling, and joint attention.

  • High-dimensional HSI data and limited labels cause redundancy and the Hughes phenomenon, challenging classification.
  • Patch-based methods incur high computational complexity because adjacent patches contain large overlapping regions.
  • FPGA exploits global spatial dependency but is poorly suited to insufficient, imbalanced samples and underweights minority classes.
  • SSDGL combines H-B sampling, weighted softmax loss, GCL, and GJAM to address sample imbalance and learn discriminative spectral-spatial representations.
  • GCL captures long short-term spectral-spatial dependencies, while GJAM reweights features through spectral and spatial attention.

A. CNN-based classification

CNN-based HSI classification learns spectral-spatial features effectively but typically processes overlapping pixel patches, creating redundant computation. Fully convolutional and global approaches instead process whole images, while sparse labels and long-tail distributions remain challenges.

  • CNN-based methods commonly generate fixed-size pixel patches with sliding windows before classification.
  • Overlapping areas between adjacent patches create redundant computation and form a bottleneck for patchwise CNN inference.
  • Fully convolutional networks replace fully connected layers with 1 × 1 convolutions and support encoder-decoder feature extraction.
  • HSI classification resembles semantic segmentation, but sparse discrete labels make direct transfer of semantic-segmentation networks infeasible.
  • Global learning processes the whole image rather than pixel patches, avoiding patch generation and accelerating inference.

III. SSDGL: SPECTRAL-SPATIAL DEPENDENT GLOBAL LEARNING FRAMEWORK

SSDGL performs whole-image HSI classification by combining spectral-spatial feature learning with hierarchical balancing and class-reweighted optimization. Its training pipeline organizes sparse labeled pixels into balanced hierarchical sequences and updates the network using weighted loss.

  • SSDGL combines spectral relationships among bands with spatial correlations across pixels using global feature learning.
  • SSDGL uses the whole image as input, applies masks to discrete labeled pixels, and optimizes encoder-decoder parameters by stochastic gradient descent.
  • A. Hierarchically balanced sampling strategy: The H-B strategy divides labeled pixels into hierarchical training sequences with balanced class representation and diverse stochastic gradients.
  • A. Hierarchically balanced sampling strategy: Each hierarchical training dataset contains all categories, while β determines the per-class minibatch size and α controls the number of hierarchical datasets.
  • The weighted loss reweights class probabilities according to labeled-sample counts and reduces the relative loss of well-classified samples.

B. GCLAM

GCLAM combines global convolutional long short-term memory with global joint attention to extract and emphasize discriminative spectral-spatial features from whole HSIs. GCL models spectral dependencies, while GJAM reweights informative spectral channels and spatial regions.

  • GCLAM: GCLAM combines GCL and GJAM to extract spectral-spatial dependencies while preserving the input spatial size.GCL captures dependencies, and GJAM estimates feature importance.
  • GCL: GCL processes whole-image channels as time steps to model long-range spectral interdependencies and enhance adjacent-channel dependencies.Its first hidden layer targets long-range features, while the second enhances adjacent-channel dependency.
  • GCL: ConvLSTMCells selectively forget, memorize, and output information, while convolutional gates transform inputs into discriminative spectral-spatial features.The convolutional phase uses gated units equivalent to convolution layers.
  • GJAM: GJAM contains global spectral and spatial attention mechanisms that reweight extracted features and emphasize informative channels and key regions.Spectral attention suppresses less-useful features, while spatial attention focuses on important local areas using global context.
  • Global spectral attention: Global spectral attention pools the whole image, processes average- and maximum-pooled spectral vectors through multilayer perceptrons, and multiplies the result with the input.The two transformed feature vectors are merged by elementwise summation before reweighting.
  • Global spatial attention: Global spatial attention pools feature maps, concatenates them, and generates a one-dimensional map that reweights spatial features without changing spatial size.Average and maximum pooling provide different spatial information before convolutional activation.

IV. EXPERIMENTS RESULTS AND ANALYSIS

The experiments compare SSDGL with established HSI classification methods across three public datasets using quantitative and qualitative evaluation.

  • Compared methods: SSDGL is compared with RBF-SVM, SS-CNN, SSRN, DBMA, MCNN-CONVLSTM, U-Net, and FPGA on three HSI datasets.The evaluation includes both quantitative metrics and qualitative classification analysis.

A. Experimental Settings

The experiments specify implementation settings, optimization details, and repeated evaluation using four standard classification metrics.

  • Model parameters: The framework uses an encoding-decoding architecture with zero padding to make input dimensions multiples of 16 and four groups in group normalization.Skip connections link encoder spatial features with decoder semantic features.
  • Optimized parameters: The GCL time step is 8 and its convolutional kernel size is 5.These values are reported as optimized parameters.
  • Optimization: Training runs for 600 epochs with gradient descent with momentum, an initial learning rate of 0.005, momentum 0.9, and weight decay 0.001.The learning-rate schedule uses power and maximum-iteration settings reported in the implementation details.
  • Metrics: Performance is evaluated using per-class accuracy, overall accuracy, average accuracy, and the kappa coefficient, with ten runs averaged for each criterion.Repeated runs reduce deviation from random training-sample selection.

B. Experiment 1: Indian Pines Dataset

Indian Pines contains 16 land-cover classes with limited labeled data, and SSDGL is evaluated against several baselines using classification maps and quantitative metrics. SSDGL improves over FPGA on overall and class-level performance under the imbalanced sampling setting.

  • Dataset: Indian Pines contains 145×145 pixels, 200 retained spectral bands, 10,249 nonbackground pixels, and 16 land-cover classes.The AVIRIS data span 0.4 to 2.5 µm with approximately 20-meter spatial resolution.
  • Experimental setting: Training uses 5% of labeled samples, with at least five samples per class, while remaining samples form the test set under H-B sampling.The dataset includes classes with strongly differing sample counts.
  • Comparison: The comparison reports results for RBF-SVM, SS-CNN, SSRN, DBMA, MCNN-CONVLSTM, U-Net, FPGA, and SSDGL using OA, AA, kappa, and per-class accuracies.Table II presents the classification results and highlights the best accuracy in each row.
  • Results: SSDGL achieves 3%–7% higher soybean accuracy and 6%–10% higher corn accuracy than FPGA.The reported gains are attributed alongside global spatial context and spectral correlation modeling.
  • Results: SSDGL improves OA, AA, and kappa by approximately 3% over FPGA on Indian Pines.The paper links this result to the proposed sampling strategy and weighted loss for long-tail distributions.

C. Experiment 2: Pavia University Dataset

On Pavia University, SSDGL achieves strong classification performance despite limited training data, with global attention improving fine-grained spatial feature extraction and overall metrics.

  • Pavia University contains 42,776 labeled pixels across nine classes after background removal.The dataset has 103 retained spectral bands and 610×340 pixels.
  • RBF-SVM produced salt-and-pepper noise because it used spectral information while ignoring adjacent-pixel spatial correlation.The passage contrasts this behavior with methods that incorporate spatial information.
  • SSDGL achieved 99.92% classification accuracy for bricks, approximately 1.5% higher than FPGA.The improvement was attributed to GJAM extracting fine-grained spatial features and attention areas.
  • SSDGL achieved higher OA, AA, and kappa than CNN-based methods and FPGA on the Pavia University dataset.The reported gains were approximately 3% over CNN-based methods and 1% over FPGA.

D. Experiment 3: Houston University Dataset

On the challenging Houston University dataset, SSDGL outperformed the compared methods, while most classes exceeded 90% accuracy but commercial remained difficult to distinguish.

  • Houston University contains 15 complex land-cover classes across 349×1,905 pixels and 144 spectral bands.The dataset was used to evaluate the framework with limited training samples.
  • SSDGL achieved the best classification performance among the compared methods on Houston University.It outperformed CNN-based methods by approximately 10–20% in overall accuracy and exceeded U-Net and FPGA on OA, AA, and kappa.
  • SSDGL increased the separation between difficult categories in the generated spectral-spatial feature space.The passage reports that categories with small differences in the original feature space became easier to distinguish.
  • Classification accuracy exceeded 90% for every category except commercial.Commercial was often misclassified as residential, road, and parking lot because their spectral-spatial features were similar.

V. DISCUSSION

The module analysis was conducted on Indian Pines, a low-spatial-resolution dataset with imbalanced sample data, to evaluate SSDGL components.

  • Component analysis experiments used Indian Pines because it has low spatial resolution and imbalanced sample data.

A. Discussion on the GCL module and GJAM module

Ablation results show that GCL, GJAM, and HB-WL each improve SSDGL performance on Indian Pines, with the combined sampling and loss strategy producing the strongest reported gains.

  • GCL module: Adding GCL increased OA from 58.39% to 95.69% on Indian Pines.The analysis varied α from 4 to 12, with the best classification accuracy reported at α = 8.
  • GJAM module: Adding GJAM increased OA from 95.69% to 96.97%.GJAM reweights feature maps and extracts attention areas.
  • H-B sampling strategy and weighted softmax loss: Adding H-B sampling and weighted softmax loss increased OA from 96.77 to 99.63 and AA from 97.65 to 99.79.H-B sampling balances class counts in hierarchical training samples, while weighted softmax recalculates category probabilities.
  • H-B sampling strategy and weighted softmax loss: SSDGL achieved its best classification performance when β was set to 10.Accuracy decreased gradually when β exceeded 20, although weighted softmax reduced the sensitivity to β.

C. Discussion on the running time

Global learning methods reduce training and inference time relative to patch-based methods by processing whole images and avoiding redundant overlapping-patch calculations. SSDGL trains longer than U-Net and FPGA but achieves improved classification accuracy.

  • Training and testing times were evaluated for five methods on the Indian Pines, Pavia University, and Houston University datasets.
  • Global learning methods train faster than patch-based deep learning methods and test 100 times faster.
  • Whole-image inference reduces redundant calculations from overlapping adjacent pixel patches, greatly decreasing model inference time.
  • SSDGL requires 2–3 times longer training than U-Net and FPGA but achieves significantly improved classification accuracy.
  • Compared with CNN-based methods, global learning greatly reduces training and inference time, broadening application prospects in HSI classification.
Loading 2105.14327v1…