PaperlayerView exact revision on arXiv

Source-linked AI summary

Cascaded Recurrent Neural Networks for Hyperspectral Image Classification

Renlong Hang, Qingshan Liu, Danfeng Hong, Pedram Ghamisi

arXiv:1902.10858v1cs.CV

TL;DR

Existing HSI RNN models may underuse redundancy and complementarity across spectral bands, motivating a cascaded GRU architecture. The proposed model separates these roles across two recurrent layers, extends them with convolutional spatial features, and reports better results than compared models on two HSIs.

  • Problem

    Existing RNN-based HSI classifiers process whole spectral bands directly and may not fully exploit redundant adjacent-band and complementary non-adjacent-band information.

  • Method

    The paper proposes a cascaded GRU RNN whose first layer reduces adjacent-band redundancy, whose second layer learns non-adjacent complementarity, and whose spectral-spatial extension adds convolutional layers.

  • Results

    The proposed models achieve better classification results than compared models on two widely used hyperspectral images.

  • Takeaways & Limitations

    Cascaded recurrent processing can model redundant and complementary spectral information, while the spectral-spatial extension simultaneously learns spectral and spatial features.

Abstract

from arXiv · show

By considering the spectral signature as a sequence, recurrent neural networks (RNNs) have been successfully used to learn discriminative features from hyperspectral images (HSIs) recently. However, most of these models only input the whole spectral bands into RNNs directly, which may not fully explore the specific properties of HSIs. In this paper, we propose a cascaded RNN model using gated recurrent units (GRUs) to explore the redundant and complementary information of HSIs. It mainly consists of two RNN layers. The first RNN layer is used to eliminate redundant information between adjacent spectral bands, while the second RNN layer aims to learn the complementary information from non-adjacent spectral bands. To improve the discriminative ability of the learned features, we design two strategies for the proposed model. Besides, considering the rich spatial information contained in HSIs, we further extend the proposed model to its spectral-spatial counterpart by incorporating some convolutional layers. To test the effectiveness of our proposed models, we conduct experiments on two widely used HSIs. The experimental results show that our proposed models can achieve better results than the compared models.

I. INTRODUCTION

HSI classification must handle high-dimensional spectral signatures containing redundant adjacent-band information while also exploiting complementary spectral and spatial structure. The paper proposes cascaded GRU-based RNN models, including a spectral-spatial extension, to address these properties.

  • Motivation: HSI classification assigns candidate class labels to pixels using spectral and spatial information in a three-dimensional data cube.HSIs contain spatial dimensions recording object locations and a spectral dimension capturing material signatures across continuous bands.
  • Motivation: Dense spectral sampling creates redundant information among adjacent bands, contributing to the Hughes effect when many channels meet limited training samples.The resulting imbalance can cause a sudden drop in classification accuracy.
  • Motivation: Existing RNN models process whole spectral sequences but may not fully exploit redundant adjacent-band and complementary non-adjacent-band information.Adjacent-band redundancy can increase computational burden and may reduce accuracy by increasing within-class variance and decreasing between-class variance.
  • Contributions: The proposed cascaded GRU RNN uses two layers to reduce redundancy and learn complementarity in an end-to-end trainable model.Two adaptive strategies connect the first RNN layer with the output layer through weighted feature fusion or weighted combinations of loss functions.
  • Contributions: A spectral-spatial counterpart adds convolutional layers to learn spatial features from each band and combine them with spectral features through recurrent operators.This extension addresses the rich spatial information in HSIs without relying solely on spectral modeling.

II. METHODOLOGY

The methodology treats spectral signatures as sequences and cascades recurrent processing to model their contextual relationships. The resulting features are passed to classification outputs and trained with backpropagation through time.

  • Cascaded RNN architecture: For each pixel, the model divides spectral bands into groups, processes each group as a sequence, then recurrently combines group features before softmax classification.The first RNN learns within-group features, while the second RNN learns complementary information across groups.
  • RNN formulation: In HSI classification, each RNN input x_t corresponds to the spectral value at the t-th band.The hidden state updates recurrently from the current spectral input and the previous hidden state.
  • RNN formulation: The recurrent connection constructs contextual relationships across the sequence, with h_T intended to capture most of its time information.The hidden representation can then be used for classification.
  • Classification: For classification, the final hidden state h_T is fed to an output layer and softmax to derive class probabilities.The output transformation includes a bias vector and a hidden-to-output weight matrix.
  • Training: All recurrent and output parameters are trained using a loss function optimized by backpropagation through time.The loss compares true and predicted labels over N training samples.

B. Cascaded RNNs

The cascaded RNN divides the spectral sequence into adjacent-band subsequences, uses GRUs to summarize each subsequence, and then models complementarity across the resulting non-adjacent features.

  • Motivation: HSI spectral signatures form long sequences, making direct RNN processing difficult because gradients may vanish or explode.Each pixel has k spectral bands, so its spectral signature is treated as a sequence of length k.
  • GRU-based RNNs: GRUs use update and reset gates to control information flow inside each recurrent unit.The update gate is derived with a sigmoid function, while the reset gate controls how prior hidden information contributes to the candidate state.
  • Cascaded RNN architecture: The model partitions the spectral sequence into l subsequences of adjacent bands, with the final subsequence absorbing any remaining bands.The usual subsequence length is d = floor(k/l).
  • Cascaded RNN architecture: Parameter-sharing first-layer RNNs process the subsequences separately, and the last GRU output from each produces a feature F(1)_i.The resulting sequence F contains the first-layer features from all subsequences.
  • Cascaded RNN architecture: A second-layer RNN receives the first-layer feature sequence to learn complementary information from non-adjacent spectral regions before classification.Its final GRU output is used as F(2), which is fed to an output layer with one unit per candidate class.

C. Improvement for Cascaded RNNs

Two connection strategies strengthen cascaded RNN features by linking first-layer representations to classification during training: weighted feature concatenation or auxiliary output-level losses.

  • Design rationale: The second-layer RNN connects directly to the output layer, but first-layer performance can affect the resulting second-layer feature.The improvement strategies add relations between first-layer RNNs and the output layer.
  • Feature-level connection: The first strategy feeds first- and second-layer features to the output layer through weighted concatenation.Fusion weights for each first-layer RNN and the second-layer RNN are learned automatically from data.
  • Output-level connection: The second strategy sends each first-layer feature and the second-layer feature to separate output layers, then combines their losses using a weighted summation.The combined loss is optimized with backpropagation through time, while the first-layer output layers can be removed during prediction.
  • Prediction: The proposed architecture uses the second-layer RNN output as the final classification result after the training-only first-layer output connections are removed.This applies to the prediction phase of the output-level strategy.

D. Spectral-spatial Cascaded RNNs

The spectral-spatial cascaded RNN extends CasRNN with convolutional layers to extract spatial features alongside spectral information. It uses transfer learning because the deeper model is more difficult to train.

  • D. Spectral-spatial Cascaded RNNs: The model selects a small spatial-spectral cube around each pixel and splits it into spectral-domain matrices for processing.Each matrix corresponds to one spectral band within the cube.
  • D. Spectral-spatial Cascaded RNNs: Convolutional layers process each spectral matrix to generate spatial features before recurrent processing.The three convolutional filters are 4 × 4 × 32, 5 × 5 × 64, and 4 × 4 × 128, producing a 128-dimensional feature for each matrix.
  • D. Spectral-spatial Cascaded RNNs: The spectral-spatial cascaded RNN is deeper and more difficult to train than the cascaded RNN.
  • D. Spectral-spatial Cascaded RNNs: Transfer learning pre-trains and freezes the convolutional layers before training the two-layer RNN, followed by whole-network fine-tuning.The pre-training stage creates N × k samples by assigning each spectral slice its pixel label.

A. Data Description

The experiments use two widely used hyperspectral image datasets: Indian Pines and Pavia University. Their descriptions include spectral bands, spatial dimensions, resolutions, class information, and training/test distributions.

  • A. Data Description: The experiments are conducted on two hyperspectral images widely used for evaluating classification algorithms.
  • A. Data Description: The Indian Pines dataset contains 200 usable spectral bands and has a spatial size of 145 × 145 pixels at 20 m resolution.Four zero-valued bands and 20 noisy water-absorption bands were removed from the original 224 bands.
  • A. Data Description: Figures 5 and 6 show false-color images together with training and test data maps for Indian Pines and Pavia University.
  • A. Data Description: The Pavia University dataset uses 103 spectral bands and has an image size of 610 × 340 pixels at 1.3 m resolution.The data contain nine land-cover classes, each with more than 1000 labeled pixels.

B. Experimental Setup

The study compares the proposed models with SVM, 1D-CNN, 2D-CNN, and a GRU-based RNN. Performance is evaluated using overall accuracy, average accuracy, per-class accuracy, and Kappa coefficient.

  • B. Experimental Setup: The comparison includes SVM, 1D-CNN, 2D-CNN, the original GRU-based RNN, CasRNN, CasRNN-F, CasRNN-O, and SSCasRNN.CasRNN-F and CasRNN-O denote feature-level and output-level improvements, respectively.
  • B. Experimental Setup: SVM uses the original spectral signature with a Gaussian kernel and fivefold cross-validation for parameter selection.
  • B. Experimental Setup: The Pavia University data visualization includes false-color, training-map, and test-map views.
  • B. Experimental Setup: The 1D-CNN contains one convolutional layer, max pooling, a 100-node fully connected layer, and an output layer.
  • B. Experimental Setup: Classification performance is measured by OA, AA, per-class accuracy, and the Kappa coefficient.OA is the proportion of correctly classified test pixels, while AA averages class accuracies.

C. Parameter Analysis

Parameter analysis examines hidden-layer sizes and the number of spectral subsequences. Optimal settings vary across datasets and between CasRNN and SSCasRNN.

  • C. Parameter Analysis: The analyzed hyperparameters are subsequence number l and the hidden-layer sizes of the first- and second-layer RNNs.Hidden sizes are selected from {16, 32, 64, 128, 256, 384}, while l is selected from 2 through 20.
  • C. Parameter Analysis: For CasRNN on Indian Pines, the best OA occurs at Hidden1 = 128 and Hidden2 = 256.CasRNN performs better than other tested hidden-size settings when Hidden1 ≥32 and Hidden2 ≥128.
  • C. Parameter Analysis: For CasRNN on Pavia University, the best hidden sizes are Hidden1 = 256 and Hidden2 = 16.
  • C. Parameter Analysis: For SSCasRNN, the optimal hidden sizes are Hidden1 = 128, Hidden2 = 256 on Indian Pines and Hidden1 = 256, Hidden2 = 256 on Pavia University.
  • C. Parameter Analysis: As l increases, model OAs generally rise and then fall; SSCasRNN significantly outperforms the other three models at the same l.The optimal l is 10 for all four models on Indian Pines, while SSCasRNN uses 4 and the other models use 8 on Pavia University.

D. Performance Comparison

Across Indian Pines and Pavia University, the cascaded and spectral-spatial RNN variants outperform the original RNN and address different weaknesses of spectral-only and spatial-only classification. SSCasRNN combines spectral and spatial information, improving quantitative performance while reducing outliers and preserving object boundaries.

  • On Indian Pines, CasRNN improves RNN performance by 4 percents, outperforming SVM and 1D-CNN.The original RNN achieves OA 69.82%, AA 75.42%, and Kappa 65.87%.
  • CasRNN-F and CasRNN-O outperform CasRNN, validating both improvement strategies; CasRNN-F increases almost all class accuracies.
  • Figures 11 and 12 compare classification maps from SVM, 1D-CNN, RNN, CasRNN variants, 2D-CNN, and SSCasRNN.Different colors correspond to different classes in the visualizations.
  • On Indian Pines, SSCasRNN raises performance above 90 percents and achieves the highest accuracy in 15 classes.The authors attribute the importance of spatial information to the dataset’s large, homogeneous objects.
  • For Indian Pines maps, SSCasRNN produces fewer outliers than spectral models and retains more object-boundary details than 2D-CNN.2D-CNN performs well on homogeneous regions but can oversmooth small objects.
  • On Pavia University, SSCasRNN improves OA from 86.18% to 90.30% and produces more detailed maps, while many small objects still limit performance.2D-CNN generates more homogeneous maps but can oversmooth and performs less strongly without spectral information.

IV. CONCLUSIONS

The paper proposes cascaded RNN models for HSI classification that model redundant and complementary spectral information, add discriminative connections, and incorporate spatial features. Experiments on two widely used HSIs show improved performance over RNN, with further gains from the model modifications.

  • The cascaded RNN model explores redundant and complementary information in high-dimensional spectral signatures more fully than the original RNN.
  • Two improvement strategies connect the first-layer RNN with the output layer to generate more discriminative spectral features.
  • SSCasRNN extends the model to learn spectral and spatial features simultaneously.
  • Experiments on two widely used HSIs show higher performance than RNN, with further improvements from the model modifications.The study also evaluates hidden sizes and the number of sub-sequences.
Loading 1902.10858v1…