PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Transfer Learning Network: A Unified Deep Joint Spectral-Spatial Feature Learning Model For Hyperspectral Image Classification

Cheng Deng, Yumeng Xue, Xianglong Liu, Chao Li, Dacheng Tao

arXiv:1904.02454v1cs.CV

TL;DR

HSI classification is difficult when deep networks require many labeled samples and must represent variable spectral-spatial information across related domains. The paper combines hierarchical SSAEs with active transfer learning to transfer pretrained features and selected samples, then fine-tune on limited target data. Experiments report promising performance against many state-of-the-art methods across three datasets.

  • Problem

    HSI deep networks typically require many labeled samples, while spectral-spatial variability and domain differences make generic feature learning difficult.

    Source: §I · p. 1, §III · p. 3

  • Method

    The method learns joint spectral-spatial features with hierarchical SSAEs and transfers pretrained networks and selected source samples to targets for active-learning-based fine-tuning.

    Source: §I · p. 2, C. Active Knowledge and Samples… · p. 5

  • Results

    The proposed method exhibits promising performance compared with many state-of-the-art approaches across three datasets.

    Source: C. Comparison with State-of-the-Art… · p. 8, §V · p. 12

  • Takeaways & Limitations

    The learned deep joint spectral-spatial representation is presented as more generic and robust than many joint spectral-spatial representations.

    Source: §I · p. 2, §V · p. 12

Abstract

from arXiv · show

Deep learning has recently attracted significant attention in the field of hyperspectral images (HSIs) classification. However, the construction of an efficient deep neural network (DNN) mostly relies on a large number of labeled samples being available. To address this problem, this paper proposes a unified deep network, combined with active transfer learning that can be well-trained for HSIs classification using only minimally labeled training data. More specifically, deep joint spectral-spatial feature is first extracted through hierarchical stacked sparse autoencoder (SSAE) networks. Active transfer learning is then exploited to transfer the pre-trained SSAE network and the limited training samples from the source domain to the target domain, where the SSAE network is subsequently fine-tuned using the limited labeled samples selected from both source and target domain by corresponding active learning strategies. The advantages of our proposed method are threefold: 1) the network can be effectively trained using only limited labeled samples with the help of novel active learning strategies; 2) the network is flexible and scalable enough to function across various transfer situations, including cross-dataset and intra-image; 3) the learned deep joint spectral-spatial feature representation is more generic and robust than many joint spectral-spatial feature representation. Extensive comparative evaluations demonstrate that our proposed method significantly outperforms many state-of-the-art approaches, including both traditional and deep network-based methods, on three popular datasets.

I. INTRODUCTION

HSI classification must handle high-dimensional spectral information, rich spatial context, and limited labeled data. The paper addresses these challenges with deep joint spectral-spatial learning combined with active and transfer learning.

  • HSIs combine hundreds of spectral bands with substantial spatial information, but high dimensionality complicates extraction of useful spectral properties.Source: §I · p. 1
  • Traditional spectral-spatial features are sensitive to local input changes, reducing classification accuracy.Source: §I · pp. 1–2
  • Deep neural networks learn hierarchical representations that are more robust to variable spectral and spatial signatures.Source: §I · p. 2
  • Active learning selects informative unlabeled samples, while transfer learning propagates useful knowledge from a source domain to a target domain.Source: §I · p. 2
  • The proposed framework uses hierarchical SSAE networks and active transfer learning to learn generic joint spectral-spatial features with limited labeled data.Source: §I · p. 2

A. Sparse Autoencoder

A sparse autoencoder maps inputs through a hidden representation and reconstructs them at an output layer. Sparse penalties and overcomplete hidden layers support abstract, discriminative feature learning.

  • An autoencoder contains input, hidden, and reconstruction layers, encoding the input into a latent representation before decoding it.Source: A. Sparse Autoencoder · p. 2, Figure 1 · p. 2
  • The encoder and decoder use separate weights and biases, with an activation function calculating unit values across layers.Source: A. Sparse Autoencoder · p. 3
  • The loss function compares reconstructed data with the input and includes weight decay to reduce over-fitting.Source: A. Sparse Autoencoder · p. 3
  • An overcomplete representation uses more hidden units than input units to learn a higher-dimensional feature representation.Source: A. Sparse Autoencoder · p. 3
  • Sparse autoencoders add a penalty that discourages hidden-unit average activations from deviating from a near-zero target.Source: A. Sparse Autoencoder · p. 3

B. Stacked Sparse Autoencoder (SSAE)

A stacked sparse autoencoder builds depth by composing multiple sparse autoencoders. Each layer transforms the preceding representation into a more abstract feature.

III. PROPOSED METHOD

The proposed architecture separately learns deep spectral and spatial features, fuses them through another SSAE, and classifies the resulting joint representation. It uses EMAP-based spatial inputs to preserve contextual structure while avoiding direct patch vectorization.

B. Active Sampling Strategy for Pre-trained Network

The paper uses batch-mode active learning to select informative samples for training the SSAE network with limited labels. MCLU uncertainty guides iterative selection from an unlabeled candidate set.

B. Active Sampling Strategy for Pre-trained Network

C. Active Knowledge and Samples Transfer Learning

Active transfer learning transfers a source-domain SSAE and training set to a target domain, then adapts both using target-sample selection and source-sample removal. Iterative fine-tuning tailors the representation to the target distribution.

C. Active Knowledge and Samples Transfer Learning

IV. EXPERIMENTS AND DISCUSSIONS

The experiments evaluate the proposed method on four widely used hyperspectral datasets, including Pavia University, and compare it with state-of-the-art alternatives. The study uses repeated experiments and reports standard classification metrics.

IV. EXPERIMENTS AND DISCUSSIONS

  • The experiments compare the proposed method with several state-of-the-art alternatives across different hyperspectral datasets.The implementations use MATLAB R2015b on a Windows 7 computer with an Intel Core i5-3470 CPU and 8GB RAM.Source: §IV · p. 6
  • Experiments use Pavia University, Pavia Center, Salinas Valley, and Indian Pines hyperspectral datasets.The datasets are illustrated with false-color and groundtruth maps.Source: A. Datasets and Settings · p. 6
  • Pavia University contains 103 useful spectral bands, 610 × 340 pixels, and 9 land-cover classes.Source: 1) Pavia University · p. 6

2) Pavia Center:

The experimental setup covers Pavia Center and related transfer scenarios using multiple hyperspectral datasets, reference-data tables, and standard accuracy measures. Model sensitivity is examined through SSAE depth variations.

2) Pavia Center:

  • Pavia Center contains 9 land-object types and has image dimensions of 1096 × 715 × 102 pixels after removing a black band.Source: 2) Pavia Center · p. 6
  • Reference-data tables organize class-wise samples for Pavia University, Pavia Center, and Salinas Valley.Source: Table I · p. 6, Table II · p. 6
  • Each dataset is divided into training, candidate, and test sets, with r = 25, 50, 75, or 100 labeled samples selected per class for training.Experiments are repeated 10 times and averaged to reduce sampling randomness.Source: 3) Salinas Valley · p. 6
  • Transfer experiments include cross-dataset and intra-image settings, including Pavia University–Pavia Center transfer and same-class regional transfers.Source: 3) Salinas Valley · pp. 6–7
  • Overall accuracy, average accuracy, and Kappa coefficient are used to assess classification performance.Source: 3) Salinas Valley · p. 7

B. SSAE Structure Analysis

The analysis selects two hidden layers for both SSAE feature extraction and fusion, while MCLU is chosen as the active-learning query function. Comparative experiments show that the proposed deep joint spectral-spatial representation outperforms the evaluated baselines across training-set sizes and datasets.

Depth Effect

  • Two hidden layers are selected for both feature-extraction and feature-fusion SSAEs after deeper configurations provide limited or negative gains.Increasing depth from two to three layers causes a 0.68% decline for Salinas Valley in feature-fusion results.Source: 1) Depth Effect · p. 7

Query Function Effect

  • MCLU consistently outperforms random sampling and MS across all three datasets during active learning.The comparison uses 26 iterations, querying 50 samples per iteration.Source: 2) Query Function Effect · p. 8

Comparison with State-of-the-Art Methods

  • Deep SSAE features outperform SVM using the same features, while EMAP-based SSAE features outperform JSSAE.The results are interpreted as evidence of greater feature stability and more suitable spatial-structure learning from EMAP.Source: C. Comparison with State-of-the-Art… · p. 8
  • The proposed method outperforms Spe-EMAP SVM, JSSAE, Spe-SSAE, EMAP-SSAE, and Spe-EMAP SSAE across three datasets and multiple training-set sizes.The study repeats each method 10 times using 5%, 10%, 15%, and 20% training sets.Source: C. Comparison with State-of-the-Art… · p. 8

D. Transferability of Active Transfer Learning Network

Active learning rapidly improves the pre-trained network with few labeled samples, and active transfer learning achieves promising target-domain classification with limited target data. Transfer performance depends on source-data size and domain similarity, with excessive source information or larger domain shifts reducing effectiveness.

Analysis of AL Procedure of the Pre-trained Network

  • Overall accuracy rises rapidly during the first four or five active-learning iterations before flattening.The procedure uses 26 iterations and selects 50 informative samples per iteration.Source: 1) Analysis of AL Procedure of the… · p. 8
  • Active learning uses fewer than half the labeled samples of the non-active-learning method while achieving higher OA, AA, and Kappa values.The selected uncertain samples are reported to better describe the unlabeled-data distribution and avoid redundant labeling.Source: 1) Analysis of AL Procedure of the… · p. 8

Transferability of the Pre-trained Network

  • Active transfer learning transfers source-domain networks and training data to target domains, then queries 80 informative target samples for fine-tuning.Pavia University and Pavia Center serve as source data for each other, while Indian Pines transfers to Salinas Valley.Source: 2) Transferability of the Pre-trained… · pp. 8, 10
  • When initial source samples exceed 50 per class on Pavia datasets or 10% on Salinas Valley, overall accuracy slightly declines.The paper attributes this to excessive source information preventing the network from learning the target-domain distribution.Source: 2) Transferability of the Pre-trained… · p. 11
  • Adding only 80 target samples substantially improves accuracy, but transfer is stronger between Pavia datasets than between Indian Pines and Salinas Valley.The paper relates this difference to shared land-cover classes between the Pavia datasets and differing classes across Indian Pines and Salinas Valley.Source: 2) Transferability of the Pre-trained… · pp. 10–11

3) Computational Costs of Active Transfer Learning Network:

The proposed network requires more training time than shallow methods but offers short testing time and faster transferred-network training. Transfer training is faster because it uses few target samples and fewer iterations.

Computational Costs of Active Transfer Learning Network

  • The proposed deep network takes more training time than shallow classifiers because training requires iterative calculations.This trade-off is presented alongside the shorter test time relevant to classification tasks.Source: 3) Computational Costs of Active… · p. 11
  • Source-domain pre-training takes 28.62, 41.20, and 62.67 minutes on Pavia University, Pavia Center, and Salinas Valley, respectively.The corresponding testing times are 0.013, 0.039, and 0.026 minutes.Source: 3) Computational Costs of Active… · p. 11
  • Transferred-network training takes 21.68, 26.51, and 54.07 minutes on the three datasets, respectively.These times are shorter than source-domain pre-training because few target samples and fewer active-transfer iterations are used.Source: 3) Computational Costs of Active… · pp. 11–12

4) Domain Adaptation of the Pre-trained Network:

The domain-adaptation experiment evaluates transfer with very limited source and target samples across three datasets. Active transfer learning remains effective, particularly when the source and target distributions differ substantially.

Domain Adaptation of the Pre-trained Network

  • Domain adaptation uses 40 source samples per class for Pavia University and Salinas Valley, 30 for Pavia Center, and 20 queried target samples over 15 iterations.The evaluation covers selected classes across all three datasets.Source: 4) Domain Adaptation of the… · p. 12
  • The domain-adaptation method obtains effective results despite relying on very limited training data.The reported conclusion is based on the performance shown in Fig. 13 and Table XIII.Source: 4) Domain Adaptation of the… · p. 12
  • Active transfer learning is reported to take effect in domain adaptation, especially when the source and target domains have a large distribution gap.The experiment evaluates three datasets with differing source-target relationships.Source: 4) Domain Adaptation of the… · p. 12, §V · p. 12
Loading 1904.02454v1…