A Survey: Deep Learning for Hyperspectral Image Classification with Few Labeled Samples

Sen Jia; Shuguo Jiang; Zhijie Lin; Nanying Li; Meng Xu; Shiqi Yu

Source-linked AI summary

A Survey: Deep Learning for Hyperspectral Image Classification with Few Labeled Samples

Sen Jia, Shuguo Jiang, Zhijie Lin, Nanying Li, Meng Xu, Shiqi Yu

arXiv:2112.01800v1Dec 3, 2021cs.CVcs.AIeess.IV

TL;DR

HSI deep-learning classification needs many labeled samples, but expert annotation is costly and few-label settings are common. This paper surveys small-sample methods by learning paradigm and evaluates representative approaches under consistent experimental settings. Across the reported experiments, few-shot, active, transfer-learning, and augmentation methods outperform autoencoder-based unsupervised methods, while overfitting and weak generalization remain challenges.

Problem
Deep-learning HSI classification typically requires many labeled samples, whereas expert labeling is time-consuming and expensive, leaving sufficient training data often unavailable.
Method
The paper systematically reviews few-sample HSI classification by learning paradigm and compares representative state-of-the-art methods using consistent experimental settings.
Results
Few-shot learning, active learning, transfer learning, and data augmentation outperform autoencoder-based unsupervised learning methods on limited samples in all experiments.
Takeaways & Limitations
The experiments identify few-shot learning, transfer learning, and active learning as important directions for small-sample HSI classification.
Takeaways & Limitations
Few-label classification can suffer from overfitting and weak generalization, remaining a major application challenge.

Abstract

from arXiv · show

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

1. Introduction

Hyperspectral images combine rich spectral and spatial information, supporting precise ground-object analysis, while deep learning classification is constrained by the scarcity and cost of labeled samples. This survey organizes small-sample methods by learning paradigm and evaluates representative approaches under consistent settings.

HSIs combine material-specific spectral information with spatial information about object shape, texture, and layout.These properties support applications including agriculture, land resources, gas detection, oil-spill monitoring, water-quality assessment, and vegetation-coverage analysis.
HSI classification methods use spectral, spatial, or joint spectral-spatial features, with spatial context helping improve classification accuracy and robustness.Spectral-only methods can be affected by illumination, environmental, atmospheric, and temporal conditions while ignoring spatial information.
Deep learning automatically learns patterns from data but generally requires many training samples to tune its numerous parameters.This requirement contrasts with HSI practice, where expert labeling is time-consuming and expensive.
The KSC illustration contrasts 130 labeled samples from 10 samples per class with a display containing 50% of labeled samples for deep learning.KSC contains 13 land covers and 5211 labeled samples overall.
The survey categorizes small-sample HSI methods by transfer learning, active learning, and few-shot learning, then compares state-of-the-art approaches experimentally.It focuses on few labeled samples as a central challenge and provides source code for reproducibility.
Deep-learning methods can be categorized by both model architecture and learning paradigm, and the two category types can be combined arbitrarily.

2. Deep learning models for HSI classification

The paper reviews autoencoders, CNNs, and RNNs as deep models for HSI classification, emphasizing how they extract spectral, spatial, and sequential context. It also describes lightweight classifiers, hybrid convolutions, and gated recurrent architectures used to address model and training challenges.

Autoencoder: Autoencoders use an encoder to map input x to hidden representation h and a decoder to reconstruct x̂, supporting feature extraction and dimensionality reduction.Undercomplete autoencoders reduce data dimension when h has lower dimension than x; sparse and denoising variants constrain or corrupt inputs to avoid identity mapping.
Autoencoder: For HSI classification, autoencoder pipelines separately learn spectral features from pixel vectors and spatial features from PCA-reduced image patches.The learned features can then support classification, while lightweight classifiers such as SVMs, random forests, or logistic regression reduce parameter demands.
CNN: CNNs reduce trainable parameters through local connections and shared convolutional kernels, giving them useful feature-representation and shift-invariance properties.
CNN: Hybrid 1D and 2D convolutions extract spectral and spatial features separately before fusing them for classification.The combination addresses the enlarged kernel size caused by redundant hyperspectral bands.
RNN: RNNs process sequential vectors while retaining previous state information, enabling implicit capture of contextual semantic relationships between inputs.HSI spectral images can be treated as high-dimensional data sequences, and convolutional operators may produce feature sequences for RNN input.
RNN: Classic RNNs are prone to gradient explosion or vanishing during training, so HSI classification commonly replaces them with LSTM or GRU networks.LSTM and GRU use gates to filter current input and previous state, retaining valuable context while forgetting unnecessary information.

3. Deep learning paradigms for HSI classification with few labeled samples

For HSI classification with few labeled samples, the paper organizes methods around transfer learning, active learning, and few-shot learning. These paradigms address limited labels by transferring knowledge, selecting informative samples, or learning sample relationships.

Transfer learning: Transfer learning uses knowledge from a labeled source domain to identify samples in a sparsely labeled target domain with differing data distributions.Fine-tuning trains on the source domain and adjusts the model using target-domain samples; domain adaptation maps domains into a more similar feature space.
Transfer learning: Fine-tuning is widely used because source-domain pretraining can reduce target-domain training requirements and limit overfitting from numerous model parameters.The method preserves most learned parameters while slightly tuning task-related modules on the target domain.
Transfer learning: DANN jointly performs representation learning and classification through class and domain discriminators in an end-to-end hyperspectral classification model.The surveyed result reports that DANN outperforms denoising autoencoders and PCA-based approaches.
Active learning: Active learning iteratively queries informative unlabeled samples for expert labeling, thereby reducing redundant selections when training labels are scarce.Its components include classifiers, labeled and unlabeled sets, a query function, and an expert; selected samples are added to the labeled set.
Active learning: Posterior-probability active learning selects the candidate with the smallest gap between its largest and second-largest class probabilities, representing greater uncertainty.The method is widely used in hyperspectral classification, with later variants incorporating joint spatial-spectral features or convolutional networks.
Few-shot learning: Few-shot learning studies differences among samples, while prototype-based methods represent each class through an embedding-space prototype.Hyperspectral variants integrate spatial-spectral features, use weighted prototype means, or add losses that enlarge interclass and reduce intraclass distances.

4. Experiments

The experiments compare representative small-sample HSI classifiers under identical settings across three benchmark datasets and three training-sample regimes. Results show strong performance from S-DMM and CNN HSI with few labels, while learning paradigms beyond unsupervised autoencoders generally perform better on limited samples.

Experimental results: 91.08% AA and 84.45% OA were achieved by S-DMM with 10 samples per class in the PaviaU example.At 50 samples per class, S-DMM had the highest AA at 96.47%, while CNN HSI had the highest OA at 95.21%; at 100 samples, 3DVSCNN had the highest AA at 97.13% and CNN HSI the highest OA at 97.35%.
Experimental results: Most models performed better on KSC than on PaviaU and Salinas, while similar spatial structures caused confusion in several classes.The paper attributes KSC performance to discriminating borders between surface objects and identifies difficult class pairs in PaviaU and Salinas.
Experimental results: Accuracy improved for all models as the number of labeled samples increased across the benchmark datasets.Grapes untrained had lower accuracy than other classes in Salinas.
Cross-method analysis: Few-shot learning, active learning, transfer learning, and data augmentation outperformed autoencoder-based unsupervised learning in all limited-sample experiments.The paper links these approaches to sample relationships, valuable-sample selection, cross-dataset similarity, and expanded sample diversity, whereas autoencoder features may lack task-related characteristics.
Convergence analysis: S-DMM converged fastest, with training loss stabilizing after approximately 3 epochs, while CNN HSI converged more slowly and sometimes showed turbulence.TwoCnn ranked second in performance and benefited from transfer learning for initialization; most other models stabilized after 100 epochs, while SSLstm showed severe oscillations.

5. Conclusions

The paper reviews deep learning for HSI classification with few labeled samples, evaluates representative models, and identifies learning paradigms and lightweight architectures as key directions. It highlights overfitting and weak generalization as continuing challenges while suggesting paradigm fusion and newer model families for future work.

The study reviews small-sample HSI classification frameworks and evaluates representative models on hyperspectral benchmark data sets.The experiments track performance as sample size increases and assess effectiveness and robustness using AA and OA.
Autoencoders are widely used for the sample problem, but their performance does not approach excellence.The paper directs future development toward few-shot learning, transfer learning, and active learning.
Fusing learning paradigms can combine their advantages, such as selecting valuable source samples before transferring a model to the target data set.The example is intended to avoid imbalance in class sample sizes.
RNNs are suitable for HSI classification, while combining learning paradigms with RNNs remains little explored; transformers are proposed as a parallel-processing alternative.The paper also identifies graph convolution networks as an approach for HSI classification on non-Euclidean data.
Lightweight models remain a future direction because deep networks require many labeled samples to train their numerous parameters.The paper cites group convolution as one method for reducing model weight.
Few-label classification can reduce labeling effort, but models are prone to overfitting and weak generalization.Avoiding overfitting and improving generalization are identified as major challenges for practical HSI applications.