PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Transfer Learning Network: A Unified Deep Joint Spectral-Spatial Feature Learning Model For Hyperspectral Image Classification

Cheng Deng, Yumeng Xue, Xianglong Liu, Chao Li, Dacheng Tao

arXiv:1904.02454v1cs.CV

TL;DR

HSI classification is difficult when deep networks require many labeled samples and must represent variable spectral-spatial information across related domains. The paper combines hierarchical SSAEs with active transfer learning to transfer pretrained features and selected samples, then fine-tune on limited target data. Experiments report promising performance against many state-of-the-art methods across three datasets.

  • Problem

    HSI deep networks typically require many labeled samples, while spectral-spatial variability and domain differences make generic feature learning difficult.

  • Method

    The method learns joint spectral-spatial features with hierarchical SSAEs and transfers pretrained networks and selected source samples to targets for active-learning-based fine-tuning.

  • Results

    The proposed method exhibits promising performance compared with many state-of-the-art approaches across three datasets.

  • Takeaways & Limitations

    The learned deep joint spectral-spatial representation is presented as more generic and robust than many joint spectral-spatial representations.

Abstract

from arXiv · show

Deep learning has recently attracted significant attention in the field of hyperspectral images (HSIs) classification. However, the construction of an efficient deep neural network (DNN) mostly relies on a large number of labeled samples being available. To address this problem, this paper proposes a unified deep network, combined with active transfer learning that can be well-trained for HSIs classification using only minimally labeled training data. More specifically, deep joint spectral-spatial feature is first extracted through hierarchical stacked sparse autoencoder (SSAE) networks. Active transfer learning is then exploited to transfer the pre-trained SSAE network and the limited training samples from the source domain to the target domain, where the SSAE network is subsequently fine-tuned using the limited labeled samples selected from both source and target domain by corresponding active learning strategies. The advantages of our proposed method are threefold: 1) the network can be effectively trained using only limited labeled samples with the help of novel active learning strategies; 2) the network is flexible and scalable enough to function across various transfer situations, including cross-dataset and intra-image; 3) the learned deep joint spectral-spatial feature representation is more generic and robust than many joint spectral-spatial feature representation. Extensive comparative evaluations demonstrate that our proposed method significantly outperforms many state-of-the-art approaches, including both traditional and deep network-based methods, on three popular datasets.

I. INTRODUCTION

HSI classification must handle high-dimensional spectral information, rich spatial context, and limited labeled data. The paper addresses these challenges with deep joint spectral-spatial learning combined with active and transfer learning.

  • HSIs combine hundreds of spectral bands with substantial spatial information, but high dimensionality complicates extraction of useful spectral properties.
  • Traditional spectral-spatial features are sensitive to local input changes, reducing classification accuracy.
  • Deep neural networks learn hierarchical representations that are more robust to variable spectral and spatial signatures.
  • Active learning selects informative unlabeled samples, while transfer learning propagates useful knowledge from a source domain to a target domain.
  • The proposed framework uses hierarchical SSAE networks and active transfer learning to learn generic joint spectral-spatial features with limited labeled data.

A. Sparse Autoencoder

A sparse autoencoder maps inputs through a hidden representation and reconstructs them at an output layer. Sparse penalties and overcomplete hidden layers support abstract, discriminative feature learning.

  • An autoencoder contains input, hidden, and reconstruction layers, encoding the input into a latent representation before decoding it.
  • The encoder and decoder use separate weights and biases, with an activation function calculating unit values across layers.
  • The loss function compares reconstructed data with the input and includes weight decay to reduce over-fitting.
  • An overcomplete representation uses more hidden units than input units to learn a higher-dimensional feature representation.
  • Sparse autoencoders add a penalty that discourages hidden-unit average activations from deviating from a near-zero target.

B. Stacked Sparse Autoencoder (SSAE)

A stacked sparse autoencoder builds depth by composing multiple sparse autoencoders. Each layer transforms the preceding representation into a more abstract feature.

  • SSAE stacks several sparse-autoencoder hidden layers, using each layer’s output as the next layer’s input.
  • With two layers, the first SAE learns h^(1) from the original input and the second learns h^(2) from h^(1).

III. PROPOSED METHOD

The proposed architecture separately learns deep spectral and spatial features, fuses them through another SSAE, and classifies the resulting joint representation. It uses EMAP-based spatial inputs to preserve contextual structure while avoiding direct patch vectorization.

  • The method was motivated by limited labeled data and the need for generic, robust spectral-spatial representations across datasets.
  • Separate SSAE branches extract spectral and spatial features, which are then stacked and passed to a fusion sub-network.
  • The proposed architecture combines three active-learning-based SSAEs for extracting and fusing deep spectral and spatial features.
  • Spatial information is preprocessed with EMAP attributes before entering an SSAE branch as a one-dimensional signal.
  • The fusion SSAE transforms the stacked descriptors into a deep joint spectral-spatial feature used by a softmax classifier.

B. Active Sampling Strategy for Pre-trained Network

The paper uses batch-mode active learning to select informative samples for training the SSAE network with limited labels. MCLU uncertainty guides iterative selection from an unlabeled candidate set.

  • B. Active Sampling Strategy for Pre-trained Network: Batch-mode active learning selects a small set of high-quality samples to reduce SSAE overfitting when labeled data are limited.The strategy is introduced for pre-training hierarchical SSAE networks in HSI classification.
  • B. Active Sampling Strategy for Pre-trained Network: The procedure first trains SSAE layer by layer, then trains and fine-tunes a softmax classifier using learned features and labels.
  • B. Active Sampling Strategy for Pre-trained Network: A candidate subset of unlabeled data is classified with softmax regression before active learning iteratively selects additional samples.
  • B. Active Sampling Strategy for Pre-trained Network: MCLU measures uncertainty through the difference between the largest and second-largest class-conditional probability densities.
  • B. Active Sampling Strategy for Pre-trained Network: MCLU selects samples with the minimum cdiff(x), because small differences indicate low-confidence predictions and greater information value.

C. Active Knowledge and Samples Transfer Learning

Active transfer learning transfers a source-domain SSAE and training set to a target domain, then adapts both using target-sample selection and source-sample removal. Iterative fine-tuning tailors the representation to the target distribution.

  • C. Active Knowledge and Samples Transfer Learning: The method transfers source-domain knowledge and training samples to learn flexible feature representations for related HSI target tasks.
  • C. Active Knowledge and Samples Transfer Learning: Traditional transfer learning commonly reuses lower layers pretrained on ImageNet or related images, whereas this method transfers a source-HSI network and samples.
  • C. Active Knowledge and Samples Transfer Learning: The framework initializes hierarchical SSAE networks with active-learning-selected source samples before transferring the pretrained network and training set.
  • C. Active Knowledge and Samples Transfer Learning: At each iteration, target-domain unlabeled samples are selected for labeling while source samples mismatched with the updated target distribution are removed.
  • C. Active Knowledge and Samples Transfer Learning: The updated training set fine-tunes all SSAE parameters until the loss-based stopping criterion is satisfied.

IV. EXPERIMENTS AND DISCUSSIONS

The experiments evaluate the proposed method on four widely used hyperspectral datasets, including Pavia University, and compare it with state-of-the-art alternatives. The study uses repeated experiments and reports standard classification metrics.

  • IV. EXPERIMENTS AND DISCUSSIONS: The experiments compare the proposed method with several state-of-the-art alternatives across different hyperspectral datasets.The implementations use MATLAB R2015b on a Windows 7 computer with an Intel Core i5-3470 CPU and 8GB RAM.
  • IV. EXPERIMENTS AND DISCUSSIONS: Experiments use Pavia University, Pavia Center, Salinas Valley, and Indian Pines hyperspectral datasets.The datasets are illustrated with false-color and groundtruth maps.
  • IV. EXPERIMENTS AND DISCUSSIONS: Pavia University contains 103 useful spectral bands, 610 × 340 pixels, and 9 land-cover classes.

2) Pavia Center:

The experimental setup covers Pavia Center and related transfer scenarios using multiple hyperspectral datasets, reference-data tables, and standard accuracy measures. Model sensitivity is examined through SSAE depth variations.

  • 2) Pavia Center:: Pavia Center contains 9 land-object types and has image dimensions of 1096 × 715 × 102 pixels after removing a black band.
  • 2) Pavia Center:: Reference-data tables organize class-wise samples for Pavia University, Pavia Center, and Salinas Valley.
  • 2) Pavia Center:: Each dataset is divided into training, candidate, and test sets, with r = 25, 50, 75, or 100 labeled samples selected per class for training.Experiments are repeated 10 times and averaged to reduce sampling randomness.
  • 2) Pavia Center:: Transfer experiments include cross-dataset and intra-image settings, including Pavia University–Pavia Center transfer and same-class regional transfers.
  • 2) Pavia Center:: Overall accuracy, average accuracy, and Kappa coefficient are used to assess classification performance.

B. SSAE Structure Analysis

The analysis selects two hidden layers for both SSAE feature extraction and fusion, while MCLU is chosen as the active-learning query function. Comparative experiments show that the proposed deep joint spectral-spatial representation outperforms the evaluated baselines across training-set sizes and datasets.

  • Depth Effect: Two hidden layers are selected for both feature-extraction and feature-fusion SSAEs after deeper configurations provide limited or negative gains.Increasing depth from two to three layers causes a 0.68% decline for Salinas Valley in feature-fusion results.
  • Query Function Effect: MCLU consistently outperforms random sampling and MS across all three datasets during active learning.The comparison uses 26 iterations, querying 50 samples per iteration.
  • Comparison with State-of-the-Art Methods: Deep SSAE features outperform SVM using the same features, while EMAP-based SSAE features outperform JSSAE.The results are interpreted as evidence of greater feature stability and more suitable spatial-structure learning from EMAP.
  • Comparison with State-of-the-Art Methods: The proposed method outperforms Spe-EMAP SVM, JSSAE, Spe-SSAE, EMAP-SSAE, and Spe-EMAP SSAE across three datasets and multiple training-set sizes.The study repeats each method 10 times using 5%, 10%, 15%, and 20% training sets.

D. Transferability of Active Transfer Learning Network

Active learning rapidly improves the pre-trained network with few labeled samples, and active transfer learning achieves promising target-domain classification with limited target data. Transfer performance depends on source-data size and domain similarity, with excessive source information or larger domain shifts reducing effectiveness.

  • Analysis of AL Procedure of the Pre-trained Network: Overall accuracy rises rapidly during the first four or five active-learning iterations before flattening.The procedure uses 26 iterations and selects 50 informative samples per iteration.
  • Analysis of AL Procedure of the Pre-trained Network: Active learning uses fewer than half the labeled samples of the non-active-learning method while achieving higher OA, AA, and Kappa values.The selected uncertain samples are reported to better describe the unlabeled-data distribution and avoid redundant labeling.
  • Transferability of the Pre-trained Network: Active transfer learning transfers source-domain networks and training data to target domains, then queries 80 informative target samples for fine-tuning.Pavia University and Pavia Center serve as source data for each other, while Indian Pines transfers to Salinas Valley.
  • Transferability of the Pre-trained Network: When initial source samples exceed 50 per class on Pavia datasets or 10% on Salinas Valley, overall accuracy slightly declines.The paper attributes this to excessive source information preventing the network from learning the target-domain distribution.
  • Transferability of the Pre-trained Network: Adding only 80 target samples substantially improves accuracy, but transfer is stronger between Pavia datasets than between Indian Pines and Salinas Valley.The paper relates this difference to shared land-cover classes between the Pavia datasets and differing classes across Indian Pines and Salinas Valley.

3) Computational Costs of Active Transfer Learning Network:

The proposed network requires more training time than shallow methods but offers short testing time and faster transferred-network training. Transfer training is faster because it uses few target samples and fewer iterations.

  • Computational Costs of Active Transfer Learning Network: The proposed deep network takes more training time than shallow classifiers because training requires iterative calculations.This trade-off is presented alongside the shorter test time relevant to classification tasks.
  • Computational Costs of Active Transfer Learning Network: Source-domain pre-training takes 28.62, 41.20, and 62.67 minutes on Pavia University, Pavia Center, and Salinas Valley, respectively.The corresponding testing times are 0.013, 0.039, and 0.026 minutes.
  • Computational Costs of Active Transfer Learning Network: Transferred-network training takes 21.68, 26.51, and 54.07 minutes on the three datasets, respectively.These times are shorter than source-domain pre-training because few target samples and fewer active-transfer iterations are used.

4) Domain Adaptation of the Pre-trained Network:

The domain-adaptation experiment evaluates transfer with very limited source and target samples across three datasets. Active transfer learning remains effective, particularly when the source and target distributions differ substantially.

  • Domain Adaptation of the Pre-trained Network: Domain adaptation uses 40 source samples per class for Pavia University and Salinas Valley, 30 for Pavia Center, and 20 queried target samples over 15 iterations.The evaluation covers selected classes across all three datasets.
  • Domain Adaptation of the Pre-trained Network: The domain-adaptation method obtains effective results despite relying on very limited training data.The reported conclusion is based on the performance shown in Fig. 13 and Table XIII.
  • Domain Adaptation of the Pre-trained Network: Active transfer learning is reported to take effect in domain adaptation, especially when the source and target domains have a large distribution gap.The experiment evaluates three datasets with differing source-target relationships.
Loading 1904.02454v1…