PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Convolutional Networks for Hyperspectral Image Classification

Danfeng Hong, Lianru Gao, Jing Yao, Bing Zhang, Antonio Plaza, Jocelyn Chanussot

arXiv:2008.02457v2cs.CV

TL;DR

Hyperspectral classification must handle rich spectral information alongside difficult sample relations and the computational burden of traditional GCNs. The paper compares CNNs and GCNs, proposes miniGCNs with mini-batch training and out-of-sample inference, and evaluates three CNN–GCN fusion strategies. Across three datasets, miniGCNs improve over GCNs or 1-D CNNs, while fused models outperform the single models.

  • Problem

    Hyperspectral classification involves difficult spectral mixing, variability, and noise, while traditional GCNs incur high costs on large graphs and require full-batch learning.

  • Method

    The paper systematically compares CNNs and GCNs, proposes supervised miniGCNs for mini-batch graph learning and direct out-of-sample inference, and combines CNN and miniGCN features using three fusion schemes.

  • Results

    Across three hyperspectral datasets, miniGCN achieves stable improvements over GCN or 1-D CNN, while FuNet outperforms the single models and FuNet-C generally performs best among the fusion strategies.

  • Takeaways & Limitations

    CNN–GCN fusion provides a stronger classification solution than either single model in the reported experiments, with miniGCNs making batch-wise combination feasible.

Abstract

from arXiv · show

To read the final version please go to IEEE TGRS on IEEE Xplore. Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification, owing to their ability to capture spatial-spectral feature representations. Nevertheless, their ability in modeling relations between samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or non-grid) data representation and analysis. In this paper, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new mini-batch GCN (called miniGCN hereinafter) which allows to train large-scale GCNs in a mini-batch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without re-training networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batch-wise network training (enabling the combination of CNNs and GCNs) we explore three fusion strategies: additive fusion, element-wise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS datasets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regards to the single CNN or GCN models. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_GCN for the sake of reproducibility.

I. INTRODUCTION

Hyperspectral image classification benefits from rich spectral information but remains difficult because spectral mixing, variability, and noise complicate discriminative feature extraction. The paper compares CNNs and GCNs, introduces miniGCNs to address large-scale GCN limitations, and investigates CNN–GCN feature fusion.

  • Motivation: Hyperspectral imagery provides detailed spectral information for distinguishing spectrally similar materials, but mixing, variability, and noise hinder discriminative feature extraction.These properties make hyperspectral classification challenging in geoscience and remote sensing.
  • CNNs and GCNs: CNNs extract spatial-spectral features effectively, whereas GCNs model relations between samples and can represent long-range spatial relations beyond CNN grid sampling.The paper positions the two architectures as complementary representations for hyperspectral data.
  • Limitations of GCNs: Traditional GCNs face high computational costs from adjacency-matrix construction and full-batch learning, causing large memory costs and slow gradient descent on large-scale data.The full-batch requirement feeds all samples into the network at once.
  • Contributions: The paper systematically analyzes CNNs and GCNs for hyperspectral classification and develops additive, element-wise multiplicative, and concatenation fusion schemes for combining their features.The fusion network is designed as an end-to-end trainable architecture.
  • Proposed miniGCNs: miniGCNs enable supervised GCN training in mini-batches and allow direct inference on large-scale out-of-sample data without retraining.The proposed approach is intended to address both scalability and deployment limitations of traditional GCNs.

A. Definition of Graph

A graph represents hyperspectral pixels as vertices connected by similarity-based edges, enabling convolution through graph spectral operations. The graph Laplacian supplies eigenvector bases, while Chebyshev approximation yields a practical GCN propagation rule.

  • Graph representation: A graph models hyperspectral pixels as vertices, with edges representing similarities between spectral signatures.The adjacency matrix A defines these vertex relationships.
  • Graph representation: The adjacency matrix A is converted into a graph Laplacian L, with degree information supplied by the diagonal matrix D.The symmetric normalized Laplacian further supports graph generalization.
  • Spectral graph convolution: Graph convolution is defined by transforming signals into the Laplacian eigenvector basis, multiplying spectral components, and applying the inverse transform.The eigenvectors of L serve as the Fourier basis for graph signals.
  • Spectral graph convolution: The graph Fourier transform uses U^⊤f, with U containing the eigenvectors of L and the inverse transform reconstructing the original signal.Orthogonality of U permits the corresponding inverse representation.
  • GCN propagation: Chebyshev polynomials approximate the graph filter to reduce convolutional computational complexity.With K = 1, the approximation leads to a simplified propagation rule for GCNs.
  • GCN propagation: GCN propagation applies renormalized graph relations, learned layer weights, and an activation function to produce each layer's output.The activation function is exemplified by ReLU.
  • Spatial relations: GCNs can capture middle-range and long-range spatial relationships, whereas CNNs tend to extract locally spatial information.The graph representation can encode both similarities and dissimilarities between samples.

III. METHODOLOGY

The methodology systematically analyzes CNNs and GCNs from four perspectives, develops miniGCNs for hyperspectral classification, and introduces an end-to-end fusion network.

  • III. METHODOLOGY: The study systematically analyzes CNNs and GCNs from four different perspectives for hyperspectral image classification.
  • III. METHODOLOGY: The paper develops miniGCNs as an improvement to existing GCNs for hyperspectral image classification.
  • III. METHODOLOGY: The methodology introduces three fusion strategies within a general end-to-end fusion network.

1) Data Preparation:

CNNs use patch-wise inputs and one-shot encoded-label outputs, whereas GCNs use pixel-wise samples and an adjacency matrix describing sample relations.

  • 1) Data Preparation:: CNNs process hyperspectral image data patch-wise and produce one-shot encoded labels.
  • 1) Data Preparation:: GCNs feed pixel-wise samples into a network whose adjacency matrix models relations between samples.The adjacency matrix must be computed before training begins.

2) Feature Representation:

CNNs capture short-range spatial-spectral information, whereas GCNs model longer-range relations but incur much higher graph-scale computational costs. miniGCNs address this bottleneck by sampling subgraphs for mini-batch training and reconstructing batch connectivity.

  • CNNs extract short-range spatial and spectral information, while GCNs model middle-range and long-range spatial relations through graph structure.
  • CNNs support mini-batch training, whereas traditional GCNs require full-batch training because all samples enter the network simultaneously.
  • GCNs have one-layer complexity O(NDP + N^2D), compared with O(NDP) for CNNs, making large graphs computationally expensive.
  • miniGCNs reduce graph training to sampled subgraphs or batches, using a node sampler with budget M where M ≪ N.
  • The miniGCN estimator aggregates features within each sampled subgraph and uses occurrence-based normalization to approximate full-batch graph convolution.
  • After sampling, each batch’s graph or adjacency matrix is reassembled according to connectivity in the full graph, and batch outputs are collected for the final layer output.

C. MiniGCNs meet CNNs: End-to-end Fusion Networks

FuNet jointly trains CNNs and miniGCNs to combine complementary spatial-spectral and relational representations. It evaluates additive, element-wise multiplicative, and concatenation fusion before classification.

  • CNNs and GCNs provide distinctive representations, so a single model may be limited by insufficient feature diversity.
  • FuNet jointly trains CNNs and miniGCNs through feature extraction and fusion modules in an end-to-end network.
  • The fusion strategies are additive, element-wise multiplicative, and concatenation fusion.
  • The operators represent element-wise addition, element-wise multiplication, and concatenation of CNN and miniGCN layer features.

IV. EXPERIMENTS

The experiments assess the proposed algorithms quantitatively and qualitatively on Indian Pines, Pavia University, and Houston2013. Dataset descriptions include land-cover classes and training/test sample distributions.

  • Three widely used hyperspectral datasets are used to evaluate the proposed algorithms quantitatively and qualitatively.
  • Indian Pines contains 16 main land-cover categories, with training and testing sample counts reported alongside their spatial distribution.

2) Pavia University Dataset:

The study evaluates the datasets, network configurations, training procedures, metrics, baselines, parameter sensitivity, fusion models, and classification maps. Pavia University is described as a 9-class ROSIS scene, while the experiments compare CNN, GCN, miniGCN, and FuNet variants.

  • Pavia University Dataset: Pavia University was acquired by ROSIS and contains 103 bands, 610 × 340 pixels, and 9 land-cover classes.
  • Data Description: Houston2013 contains 144 spectral bands across a 349 × 1905 image and 15 challenging land-cover categories.
  • Experimental Settings: Networks use TensorFlow and Adam, with training capped at 200 epochs, batch size 32, and ℓ2 regularization set to 0.001.
  • Experimental Settings: Ten 10-fold cross-validation replications split the original training set into 80% training and 20% validation data for selecting architectures and hyperparameters.
  • Evaluation: Classification performance is measured using Overall Accuracy, Average Accuracy, and Kappa Coefficient.
  • Compared Methods: Comparisons include KNN, random forest, 1-D CNN, 2-D CNN, GCN, miniGCN, and FuNet-A, FuNet-M, and FuNet-C.
  • Compared Methods: The miniGCN shares the GCN architecture but trains networks batch-wise and is intended to reach a better local optimum.
  • Fusion Networks: The FuNet configuration adds a fully connected fusion layer behind CNNs and miniGCNs to exploit diverse hyperspectral information.

C. Parameter Analysis on eA Generation

The study examines how the number of neighbors K and RBF width σ affect adjacency-matrix quality and GCN performance, then selects a stable setting for subsequent datasets.

  • C. Parameter Analysis on eA Generation: GCN and miniGCN performance is evaluated across combinations of neighbors K and RBF width σ using OA on Indian Pines.The analysis focuses on how adjacency-matrix parameters affect classification performance.
  • C. Parameter Analysis on eA Generation: The parameter combination (K, σ) = (10, 1) is selected because its performance is relatively stable across the tested range.This setting is applied to the remaining datasets for simplicity.

D. Quantitative Evaluation

Quantitative and visual evaluations compare traditional classifiers, CNNs, GCNs, miniGCNs, and fused models across three hyperspectral datasets. miniGCNs improve over single GCN and 1-D CNN models, while fused networks generally achieve the strongest results, with fusion strategy C particularly effective on two datasets.

  • D. Quantitative Evaluation: Classification scores are reported using OA, AA, κ, and individual class accuracies on Indian Pines, Pavia University, and Houston2013.Tables V–VII provide the quantitative comparison across the three datasets.
  • D. Quantitative Evaluation: Deep models outperform KNN, RF, and SVM, while 3-D CNN generally exceeds 2-D CNN but uses more parameters and can overfit with limited training samples.The comparison covers 1-D CNN, 2-D CNN, 3-D CNN, and GCN.
  • D. Quantitative Evaluation: miniGCN improves consistently over GCN and 1-D CNN, while FuNet outperforms single models; FuNet-C is especially effective on Indian Pines and Pavia University.FuNet-C shows a dramatic performance improvement relative to FuNet-A and FuNet-M on those datasets.
  • D. Quantitative Evaluation: Fused networks better identify classes with few or unbalanced samples by combining spatial-spectral CNN features with relation-augmented miniGCN features.Examples include Alfalfa, Grass Pasture Mowed, and Oats on Indian Pines, and Road and Parking Lot2 on Houston2013.
  • E. Visual Comparison: Classification maps from miniGCN are comparable to those of 2-D and 3-D CNNs, while FuNet methods produce smoother and more detailed maps.The batch-wise strategy locally preserves graph structure and can reduce error accumulation and propagation between layers.
  • V. CONCLUSION: GCNs incur high storage and computational costs for adjacency construction, may experience gradient exploding or vanishing, and require retraining for new data.These drawbacks motivate the development of miniGCNs.
  • V. CONCLUSION: miniGCNs reduce computational cost, support direct out-of-sample prediction without retraining, and enable end-to-end CNN–GCN fusion across three datasets.The paper reports FuNet superiority over single CNN and miniGCN models, while future work considers weighted fusion and other deep networks.
Loading 2008.02457v2…