PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Convolutional Subspace Clustering: A Robust Subspace Clustering Framework for Hyperspectral Image

Yaoming Cai, Zijia Zhang, Zhihua Cai, Xiaobo Liu, Xinwei Jiang, Qin Yan

arXiv:2004.10476v1cs.CVcs.LG

TL;DR

HSI clustering seeks structure in data without labels, but traditional subspace methods often ignore the data's graph structure. This paper introduces graph-convolutional subspace clustering and two efficient variants; experiments on three HSI datasets report state-of-the-art clustering performance.

  • Problem

    Traditional subspace clustering often ignores HSI data's inherent graph structure, while supervised HSI classification requires substantial labeled data.

    Source: §I · pp. 1–2

  • Method

    GCSC recasts self-representation in the non-Euclidean domain using graph convolution, with EGCSC and EKGCSC as Frobenius-norm variants.

    Source: §I · p. 2

  • Results

    Across three HSI datasets, EGCSC and EKGCSC achieve state-of-the-art clustering performance, with their OAs generally above 84% on all datasets.

    Source: 1) Quantitative Results · p. 6, §V · pp. 8–9

  • Takeaways & Limitations

    The authors present the results as bridging the performance gap between unsupervised and supervised HSI classification.

    Source: 1) Quantitative Results · p. 7

Abstract

from arXiv · show

Hyperspectral image (HSI) clustering is a challenging task due to the high complexity of HSI data. Subspace clustering has been proven to be powerful for exploiting the intrinsic relationship between data points. Despite the impressive performance in the HSI clustering, traditional subspace clustering methods often ignore the inherent structural information among data. In this paper, we revisit the subspace clustering with graph convolution and present a novel subspace clustering framework called Graph Convolutional Subspace Clustering (GCSC) for robust HSI clustering. Specifically, the framework recasts the self-expressiveness property of the data into the non-Euclidean domain, which results in a more robust graph embedding dictionary. We show that traditional subspace clustering models are the special forms of our framework with the Euclidean data. Basing on the framework, we further propose two novel subspace clustering models by using the Frobenius norm, namely Efficient GCSC (EGCSC) and Efficient Kernel GCSC (EKGCSC). Both models have a globally optimal closed-form solution, which makes them easier to implement, train, and apply in practice. Extensive experiments on three popular HSI datasets demonstrate that EGCSC and EKGCSC can achieve state-of-the-art clustering performance and dramatically outperforms many existing methods with significant margins.

I. INTRODUCTION

HSI clustering must handle unreliable similarity measurements and lacks the label information required by supervised classification. The paper introduces graph-convolutional subspace clustering and efficient variants, and reports improved clustering results on HSI datasets.

I. INTRODUCTION

  • HSI clustering faces unreliable similarity measurements from mixed pixels and redundant bands, while supervised methods require costly labels and struggle with unknown objects.The introduction also notes that HSI clustering has fewer studies and is usually less competitive in accuracy than supervised classification.Source: §I · p. 1
  • Traditional subspace clustering ignores inherent graph structure, while data noise and large errors can compromise the results; combining graph learning with subspace clustering has received little attention.Graph-regularized methods typically add a separate regularization term and tradeoff parameter.Source: §I · p. 2
  • GCSC recasts subspace clustering in the non-Euclidean domain, using graph convolution to incorporate graph and feature information into self-representation for a more robust affinity.The figure depicts two classes and misclassified points to illustrate the motivation for graph-structured data and robust embeddings.Source: Figure 1 · p. 1, §I · p. 2
  • The paper proposes EGCSC and EKGCSC with closed-form solutions and reports that the models outperform many existing HSI clustering methods.The solutions are described as easier to implement, train, and apply in practice.Source: §I · p. 2

A. Notations

  • The notation conventions distinguish vectors, matrices, scalars, and sets, and define graph Laplacians, the Frobenius norm, and trace notation.The graph Laplacian is L = D − A, with normalized version Lsym = D−1/2LD−1/2.Source: A. Notations · p. 2

B. Subspace Clustering Models

Subspace clustering represents data points using a self-expressive coefficient matrix, then uses the resulting affinity to obtain clusters. Different norm choices shape the coefficients, and low-rank formulations can also model corrupted data.

B. Subspace Clustering Models

  • The standard self-representation model seeks a coefficient matrix W satisfying XW = X while constraining its diagonal to zero to avoid trivial solutions.A norm on W specifies the model's regularization.Source: B. Subspace Clustering Models · p. 2
  • SSC uses an ℓ1-norm to encourage sparse coefficients, whereas LRSC uses a nuclear norm to enforce low rank and incorporate global data structure.The cited LRSC formulation also uses an ℓ2,1 reconstruction term.Source: B. Subspace Clustering Models · p. 3
  • Subspace clustering can model corrupted data as X = XW + N, where N represents arbitrary noise.Source: B. Subspace Clustering Models · p. 3
  • After finding W, the method constructs an affinity matrix from its coefficients and applies spectral clustering to segment the data.Source: B. Subspace Clustering Models · p. 3

C. Graph Convolutional Networks

Graph convolutions extend convolution to graph-structured data, and HSI clustering uses unsupervised methods to avoid reliance on labels. The methodology introduces GCSC, which incorporates graph embedding into subspace clustering and relates the framework to traditional models.

C. Graph Convolutional Networks

  • Graph convolutions include spectral methods that transform node representations into the spectral domain and spatial methods that use node neighborhoods.The paper uses spectral convolution; GCN approximates spectral filters with first-order Chebyshev polynomials.Source: C. Graph Convolutional Networks · p. 3

D. HSI Clustering

  • HSI clustering avoids annotation, while subspace clustering exploits data structure; related methods also combine spectral and spatial information.Centroid-based K-means and fuzzy c-means are sensitive to random initialization, while evolutionary methods can incur high computational cost.Source: D. HSI Clustering · p. 3

III. METHODOLOGY

  • The methodology presents GCSC and develops two models based on the framework: EGCSC and EKGCSC.The framework is illustrated in Fig. 2.Source: §III · p. 3

A. Graph Convolutional Subspace Clustering Framework

  • GCSC incorporates graph embedding into subspace clustering, using graph convolutional self-representation and treating traditional subspace models as special cases.Its graph embedding serves as a dictionary for affinity learning, and spectral clustering can use the resulting coefficients.Source: A. Graph Convolutional Subspace… · pp. 3–4, Figure 2 · p. 4

B. Efficient GCSC

EGCSC is a GCSC model that uses Frobenius norms and a closed-form self-representation solution. It constructs an affinity matrix for spectral clustering.

B. Efficient GCSC

  • EGCSC sets both norm choices in the GCSC framework to the Frobenius norm.The model is introduced as the first novel subspace clustering model based on the framework.Source: B. Efficient GCSC · p. 4
  • The Frobenius-norm solution yields a dense self-representation coefficient matrix without requiring diag(Z) = 0.The paper describes this as an efficient closed-form solution and attributes the result to Pan et al..Source: B. Efficient GCSC · p. 4
  • EGCSC uses an EDSC heuristic to enhance affinity-matrix block structure before spectral clustering.The authors note there is no globally accepted affinity-construction solution in the literature.Source: B. Efficient GCSC · p. 4

C. Efficient Kernel GCSC

EKGCSC extends EGCSC with the kernel trick to address nonlinear HSI data. It maps data into a higher-dimensional space, obtains a closed-form solution, and uses the resulting affinity for spectral clustering.

C. Efficient Kernel GCSC

  • EKGCSC extends EGCSC with the kernel trick because EGCSC is modeled on linear subspaces and HSI data are described as complex and nonlinear.The extension is presented as Efficient Kernel GCSC.Source: C. Efficient Kernel GCSC · p. 4
  • The model uses a positive semidefinite kernel Gram matrix and specifies the Gaussian kernel for the mapping.The kernel matrix is defined through inner products of mapped data points in a reproducing kernel Hilbert space.Source: C. Efficient Kernel GCSC · pp. 4–5
  • EKGCSC has a closed-form solution, maps data into a higher-dimensional space, and constructs an affinity matrix for spectral clustering.The paper says the mapping makes a linearly inseparable problem separable.Source: C. Efficient Kernel GCSC · p. 5

D. HSI Clustering Using The GCSC Models

The GCSC models are applied to HSI data by reducing spectral redundancy, incorporating spatial information, and constructing a graph representation. The graph is a k-nearest-neighbor graph whose neighborhoods are based on Euclidean distance.

D. HSI Clustering Using The GCSC Models

  • HSI data may contain redundant spectral bands, and the GCSC models require graph-structured data although HSI is typically Euclidean.The paper identifies these as two issues to address before applying the models.Source: D. HSI Clustering Using The GCSC Models · p. 5
  • PCA preserves the top d principal components, and 3D patches represent each pixel using its center and neighboring pixels.PCA is used to reduce redundancy and increase training efficiency; patches incorporate spectral and spatial information.Source: D. HSI Clustering Using The GCSC Models · p. 5
  • A k-nearest-neighbor graph represents data-point relationships, with neighborhoods determined by Euclidean distance.Each data point is treated as a graph node, and its k nearest neighbors define its edge relationships.Source: D. HSI Clustering Using The GCSC Models · p. 5

E. Remarks on The Proposed GCSC

GCSC uses graph convolution to embed data before self-representation, incorporating graph structure into the dictionary. Its non-Euclidean framework includes traditional subspace clustering as a Euclidean special case.

E. Remarks on The Proposed GCSC

  • In the Indian Pines visualization, corn-till, soybean-notill, and soybean-mintill are mixed in the original data but more separable and compact after graph-convolution embedding.The figure shows 100 randomly selected points per class reduced to two dimensions with t-SNE.Source: Figure 3 · p. 5, E. Remarks on The Proposed GCSC · p. 5
  • GCSC uses graph embedding Y as a self-expressive dictionary to reconstruct the original data, incorporating global structure into a clearer dictionary and more robust affinity.Graph convolution combines node features with those of nearby neighbors, making features within a cluster more similar.Source: E. Remarks on The Proposed GCSC · p. 5
  • GCSC operates in the non-Euclidean domain, where traditional subspace clustering is a Euclidean special case; it incorporates graph structure through graph convolution rather than manifold regularization.The paper characterizes graph convolution as a more straightforward way to exploit graph information.Source: E. Remarks on The Proposed GCSC · pp. 5–6

IV. EXPERIMENTS

Across three HSI datasets, EGCSC and EKGCSC achieve the best reported clustering performance on OA, NMI, and Kappa. The experiments evaluate these models against existing methods using dataset sub-scenes and standard clustering metrics.

1) Datasets and Preprocessing:

  • Evaluation uses sub-scenes from SalinasA, Indian Pines, and Pavia University; preprocessing retains at least 95% variance in four PCA bands, with neighborhood size 9 and [0, 1] scaling.The sub-scenes are selected for computational efficiency.Source: 1) Datasets and Preprocessing · p. 6

2) Evaluation Metrics:

  • Clustering is evaluated with OA, NMI, and Kappa, while running time is compared to assess computational complexity.The three accuracy metrics range from 0 to 1, with higher scores indicating more accurate clustering.Source: 2) Evaluation Metrics · p. 6

3) Compared Methods:

  • Comparisons include traditional clustering and subspace methods alongside S4C, UBL, and RMMF, with settings for EGCSC and EKGCSC reported separately.The compared traditional methods include SC, SSC, EDSC, LRSC, and ℓ2-SSC.Source: 3) Compared Methods · p. 6, Table II · p. 6

1) Quantitative Results:

  • EKGCSC outperforms EGCSC on all three datasets, with reported improvements of 0.15%, 2.78%, and 12.94%, respectively.The respective datasets are SalinasA, Indian Pines, and Pavia University.Source: 1) Quantitative Results · p. 6
  • EGCSC and EKGCSC achieve the best performance across three datasets on OA, NMI, and Kappa; their OA is generally above 84%, with EKGCSC reaching 100% on SalinasA.The comparison covers SalinasA, Indian Pines, and Pavia University.Source: 1) Quantitative Results · p. 6, Table III · p. 7
  • EKGCSC is comparable with many supervised HSI classification methods, achieving OA of 100%, 87.61%, and 97.36% on SalinasA, Indian Pines, and Pavia University, respectively.The authors describe the results as bridging the gap between unsupervised and supervised HSI classification.Source: 1) Quantitative Results · p. 7

2) Results Visualization:

The visualizations show EKGCSC’s SalinasA class map matching ground truth and its affinity matrix having a clearer block structure than EGCSC’s. Performance varies with λ, while both models are insensitive to k except that overly large neighborhoods may cause over-smoothing.

2) Results Visualization:

  • EKGCSC’s SalinasA class map completely agrees with ground truth; on Indian Pines and Pavia University, its maps are closest to ground truth, while EGCSC’s are better than other competitors.Other methods’ maps include relatively more noisy points attributed to misclassification.Source: 2) Results Visualization · p. 7

3) Visualization of The Learned Affinity Matrix :

  • Affinity-matrix rows or columns encode self-representation coefficients; both models yield sparse, block-diagonal structure, with EKGCSC showing better block structure than EGCSC.The data points are reordered by ground truth before the matrices are computed.Source: 3) Visualization of The Learned… · p. 7

4) Impact of λ and k :

  • Performance is sensitive to λ but insensitive to k for both models; excessively large k may cause over-smoothing and negatively affect clustering.The authors vary k from 5 to 40 in increments of 5.Source: 4) Impact of λ and k · p. 7

5) Impact of The Number of PCs:

The conclusion presents GCSC as a graph-convolution framework for subspace clustering, with EGCSC and EKGCSC achieving strong results on three HSI datasets. The section also reports how PC count and runtime affect the methods.

5) Impact of The Number of PCs:

  • With PCs varied from 1 to 8, clustering performance generally increases as more spectral information is included, but redundancy can reduce gains; Pavia University performs best with 4 PCs rather than 8.The authors describe dimensionality reduction as optional and as balancing computational efficiency with model performance.Source: 5) Impact of The Number of PCs · pp. 7–8

6) Comparison of Running Time:

  • Closed-form EGCSC and EKGCSC are faster than SSC, LRSC, S4C, and UBL, while graph construction and EKGCSC's kernel-matrix computation add runtime.SC, ℓ2-SSC, and RMMF run faster but do not achieve better performance than the proposed methods; both proposed methods take relatively more time than EDSC.Source: 6) Comparison of Running Time · p. 8, Table IV · p. 9

V. CONCLUSIONS

  • GCSC uses graph-convolutional self-representation to incorporate intrinsic data structure, and treats traditional subspace clustering as a special form on Euclidean data.The conclusion describes the resulting affinity as robust and identifies deep graph embedding as a possible future direction.Source: §V · pp. 8–9
  • EKGCSC achieves clustering OA of 100%, 87.61%, and 97.36% on SalinasA, Indian Pines, and Pavia University, respectively.The figure captions also report EGCSC at 99.85%, 84.83%, and 84.42% on those datasets.Source: §V · pp. 8–9, Figure 4 · p. 8, Figure 5 · p. 8

VI. APPENDIX

The appendix derives optimal-solution expressions for EGCSC and EKGCSC and includes figures on parameter and PC-count influence. The acknowledgments credit reviewers and the provider of RMMF source code.

VI. APPENDIX

  • The appendix includes figures examining the influence of λ and k and clustering OA under varying PC counts across three datasets.Source: Figure 8 · p. 9, Figure 9 · p. 10
  • The EGCSC proof differentiates its loss to derive an expression for Z and states that the relevant matrix is invertible.The appendix identifies the matrix as positive semidefinite.Source: §VI · pp. 9–10
  • The EKGCSC proof differentiates its loss and sets the derivative to zero to obtain the optimal solution for Z.Source: §VI · p. 10

ACKNOWLEGMENT

  • The acknowledgments thank the anonymous reviewers and Lefei Zhang, who provided RMMF source code.Source: ACKNOWLEGMENT · p. 10
Loading 2004.10476v1…