PaperlayerView exact revision on arXiv

Source-linked AI summary

Simultaneous Spectral-Spatial Feature Selection and Extraction for Hyperspectral Images

Lefei Zhang, Qian Zhang, Bo Du, Xin Huang, Yuan Yan Tang, Dacheng Tao

arXiv:1904.03982v1cs.CV

TL;DR

Hyperspectral classification needs a low-dimensional representation that uses spectral and spatial information without treating heterogeneous features identically or obscuring their roles. S3FSE learns a shared subspace with manifold and sparse-learning components, and experiments on three datasets show consistent classification gains over comparison methods.

  • Problem

    Feature stacking of spectral and spatial information creates high-dimensional representations, overlooks heterogeneous and complementary properties, and makes transformed features difficult to interpret.

  • Method

    S3FSE projects spectral and spatial features into a common latent subspace while using geometric modeling and row-sparse regularization to select significant original features.

  • Results

    S3FSE consistently outperforms comparison methods for hyperspectral image classification across three public datasets.

  • Takeaways & Limitations

    The learned representation is interpretable as a linear combination of a subset of significant original features.

Abstract

from arXiv · show

In hyperspectral remote sensing data mining, it is important to take into account of both spectral and spatial information, such as the spectral signature, texture feature and morphological property, to improve the performances, e.g., the image classification accuracy. In a feature representation point of view, a nature approach to handle this situation is to concatenate the spectral and spatial features into a single but high dimensional vector and then apply a certain dimension reduction technique directly on that concatenated vector before feed it into the subsequent classifier. However, multiple features from various domains definitely have different physical meanings and statistical properties, and thus such concatenation hasn't efficiently explore the complementary properties among different features, which should benefit for boost the feature discriminability. Furthermore, it is also difficult to interpret the transformed results of the concatenated vector. Consequently, finding a physically meaningful consensus low dimensional feature representation of original multiple features is still a challenging task. In order to address the these issues, we propose a novel feature learning framework, i.e., the simultaneous spectral-spatial feature selection and extraction algorithm, for hyperspectral images spectral-spatial feature representation and classification. Specifically, the proposed method learns a latent low dimensional subspace by projecting the spectral-spatial feature into a common feature space, where the complementary information has been effectively exploited, and simultaneously, only the most significant original features have been transformed. Encouraging experimental results on three public available hyperspectral remote sensing datasets confirm that our proposed method is effective and efficient.

I. INTRODUCTION

The introduction identifies shortcomings in spectral-only classification and simple feature stacking, then motivates S3FSE to select and extract complementary spectral-spatial features in an interpretable common space.

  • Spectral-only classification represents pixels independently and omits relationships among neighboring pixels.
  • Spatial information can improve classification, but many existing methods incorporate it mainly as a postprocessing step.
  • Feature stacking creates high-dimensional vectors and can trigger the curse of dimensionality when training samples are limited.
  • Stacking also treats heterogeneous features equally, misses consistent and complementary information, and makes transformed features difficult to interpret.
  • S3FSE integrates feature selection and extraction so important original features are identified while learning a lower-dimensional representation.
  • Manifold learning, co-local geometric preserving, co-graph regularization, and ℓ2,1-norm co-regularization capture feature relations and produce row-sparse projections.

II. THE PROPOSED METHOD

S3FSE uses three components: extracting spectral-spatial features, exploiting their structural information, and learning a row-sparse projection that selects and transforms informative features.

  • Spectral and spatial features are first extracted for each pixel.
  • Manifold learning and structured sparse learning then exploit the data's structure information.
  • A row-sparse projection matrix discards redundant and noisy features while transforming significant original features.

A. Notations and Problem Definition

The paper models hyperspectral data as multiple feature views and seeks a latent low-dimensional representation of training samples for classification.

  • A. Notations and Problem Definition: Matrices use uppercase notation, vectors use bold lowercase notation, and matrix rows, columns, entries, and ℓr,p-norms are defined.
  • A. Notations and Problem Definition: Each pixel set is represented in V heterogeneous spectral and spatial feature spaces, called views.
  • A. Notations and Problem Definition: Given n training samples from C classes, the objective is to learn Y ∈ Rn×d as a latent low-dimensional feature representation for classification.

B. The Objective Function of S3FSE

S3FSE jointly preserves local geometry and consensus structure across heterogeneous spectral-spatial features while learning sparse projections that select and extract informative features.

  • Co-local geometric preserving: S3FSE learns a shared low-dimensional subspace that preserves local geometric structures in spectral and spatial feature spaces.It uses manifold learning and graph-Laplacian-based modeling for heterogeneous features.
  • Co-local geometric preserving: Nonnegative view weights model the different contributions and complementary information of heterogeneous spectral and spatial features.The weights regulate each view’s contribution to preserving local geometric properties.
  • Co-graph regularization: A joint graph on all views encourages corresponding within-class samples to remain close in the common subspace.The joint graph uses label information to characterize consensus data descriptions.
  • Projection matrices co-regularization: Projection co-regularization combines feature selection and extraction so significant features receive nonzero transformations while less important features receive zeros.The projection is represented through selection and transformation matrices, with row sparsity used to avoid solving them separately.
  • Projection matrices co-regularization: The ℓ2,1-norm co-regularization makes the projection row sparse while exploiting complementary information across multiple views.Zero rows discard redundant or noisy features, and the integrated objective combines the geometric, consensus, and projection terms.

III. OPTIMIZATION

The optimization reformulates the nonsmooth objective using an auxiliary diagonal matrix and solves it iteratively through an eigenvalue problem until convergence.

  • Optimization formulation: The overall objective is convex because H1 and H2 are positive semi-definite, but its ℓ2,1-norm term is nonsmooth.The nonsmooth regularizer prevents direct easy solution despite the global-optimum property of the convex objective.
  • Optimization procedure: The method rewrites the objective using a diagonal matrix H3 that depends on the current projection matrix P.This dependence motivates alternating iterative optimization.
  • Optimization procedure: P is obtained by solving an eigenvalue problem and selecting the d eigenvectors associated with the smallest ordered eigenvalues.The selected eigenvectors form the learned projection matrix.
  • Optimization procedure: Algorithm 1 initializes P randomly, repeatedly computes H1, H2, and H3, solves the eigenvalue problem, and updates P until convergence.The procedure outputs the final projection matrix.

IV. EXPERIMENTS

The experiments evaluate S3FSE through hyperspectral image classification by projecting spectral and spatial features into a learned common subspace and classifying with an SVM.

  • Experimental protocol: Experiments use three public hyperspectral datasets to evaluate the proposed feature representation method in terms of classification.The evaluation follows previous feature-learning work.
  • Experimental protocol: Spectral and spatial features are projected into the learned low-dimensional subspace before test samples are classified with an SVM.Classification is performed in the common feature subspace.
  • Experimental protocol: The reported evaluation is designed to assess the effectiveness of the learned spectral-spatial representation through classification performance.The supplied passage does not provide numerical results.

A. Datasets description

The study uses three public hyperspectral datasets from urban and city environments, with differing spatial dimensions and retained spectral-channel counts.

  • HYDICE urban: The HYDICE urban dataset contains 187 spectral channels and measures 307 × 307 pixels after low-SNR bands were removed.It was acquired near Fort Hood, Texas, using an airborne HYDICE sensor.
  • Washington DC: The Washington DC dataset has spatial size 1280 × 307 pixels and 191 available spectral channels after water-absorption bands were deleted.It was acquired over a Washington DC mall with the HYDICE sensor.
  • Pavia city: The ROSIS Pavia city dataset contains 1400 × 512 pixels and 102 retained spectral channels after noisy channels were removed.The data were collected in Pavia, Italy, with the ROSIS instrument.

B. Experiment setup

The experiments use spectral, texture, and morphological features as separate inputs, comparing S3FSE against concatenation-based dimensionality-reduction methods and a baseline.

  • Input spectral and spatial features: Three feature types—spectral, texture, and morphological—are represented as individual feature vectors for each pixel.Texture uses Gabor wavelets, while morphological information uses Differential Morphological Profiles.
  • Input spectral and spatial features: Spectral features are formed from reflectance values across spectral channels.The spectral vector contains the reflectance values of all available channels.
  • Input spectral and spatial features: Texture features are extracted from the first principal component using 2-D Gabor wavelets across five scales and twelve directions.This produces a 60-dimensional texture feature.
  • Input spectral and spatial features: Morphological features use Differential Morphological Profiles computed from the first ten principal-component images with circular structural elements of radii 2, 4, 6, and 8.The DMP captures structural information through opening and closing operators.
  • Comparison schemes: The comparison includes a concatenation baseline and SPCA, SDA, CNFE, DNP, CoLGP, and MFC applied to the same spectral-spatial inputs.The compared methods are dimensionality-reduction approaches evaluated alongside S3FSE.
  • Implementation details: Implementation settings include k = 5 and t = 1 for CoLGP, MFC, and S3FSE, with comparison-method parameters set according to cited references or stated experimental values.CNFE uses k = 5, r1 = 2, r2 = 1, and μ = 0.75; DNP uses Ps = 1/8, Po = 1/8, and μ = 0.75.

C. Experiment 1: HYDICE urban dataset

On HYDICE urban, S3FSE combines spectral-spatial representation with simultaneous selection and extraction, yielding strong classification performance and interpretable feature contributions.

  • Dataset: The HYDICE urban dataset contains six informative land-cover classes and remains challenging because some classes have similar spectral curves.The scene has approximately 2 m spatial resolution.
  • Learned projection matrix: S3FSE learns a projection matrix whose row sparsity selects relevant original features while mapping them into a low-dimensional subspace.For HYDICE urban, overall projection sparsity is 41.59%, with spectral, texture, and DMP sparsities of 22.99%, 70.00%, and 63.75%, respectively.
  • Learned projection matrix: The spectral feature has the lowest sparsity and therefore contributes most strongly to the learned output representation.The reported sparsities are 22.99% for spectral, 70.00% for texture, and 63.75% for DMP features.
  • Classification results: Classification maps show fewer mixed pixels for S3FSE than for the baseline, SPCA, SDA, CNFE, DNP, CoLGP, and MFC representations.The S3FSE map contains only a small number of mixed pixels in the reported visual comparison.
  • Classification results: S3FSE achieves the best overall accuracy and Kappa coefficient, with the highest class accuracy in most individual classes.The results are based on mean and standard deviation of classification rates over 10 independent experiments.
  • Dimensionality and sensitivity: S3FSE performs best when the embedded subspace dimensionality exceeds five and remains stable as dimensionality increases.The dimensionality comparison covers reduced subspaces from 1 to 100.
  • Dimensionality and sensitivity: Small β values degrade classification because they retain too many redundant and noisy features, while performance is good across a wide range of α values.The sensitivity study evaluates α and β over {10^-5, 10^-4, · · · , 10^4} with k = 5 and t = 1 fixed.
  • Convergence study: The optimization converges in fewer than ten iterations, and classification overall accuracy stabilizes when the iteration count reaches five.The convergence study also compares objective-function error, classification overall accuracy, and computational cost.

D. Experiment 2: HYDICE Washington DC dataset

The Washington DC experiment evaluates spectral-spatial feature representations across seven land-cover classes using fixed training samples and classification-map, accuracy, and dimensionality analyses. S3FSE achieves the best reported visual and classification performance.

  • Dataset and setup: The HYDICE Washington DC scene contains seven land-cover classes, including spectrally similar pairs such as grass/tree and roof/road.The experiment uses 191-D spectral, 60-D texture, and 80-D DMP features per pixel.
  • Dataset and setup: Thirty samples from each class were randomly selected for training, with the remaining reference samples used for testing.
  • Classification results: S3FSE produces the visually best classification map, with very little roof mixed into road pixels compared with the other methods.The maps compare Baseline, SPCA, SDA, CNFE, DNP, CoLGP, MFC, and S3FSE.
  • Classification results: Mean and standard-deviation overall accuracies, together with OA curves across d from 1 to 100, indicate that S3FSE achieves the best classification performance.The class-specific accuracy results are reported in Table II.

E. Experiment 3: Pavia city ROSIS dataset

The Pavia city ROSIS experiment evaluates S3FSE on six urban land-cover classes with 30 training samples per class. Across classification maps, accuracy statistics, and OA-versus-dimensionality analysis, S3FSE achieves the best reported performance.

  • Dataset and setup: The Pavia city ROSIS dataset contains six classes: water, road, roof, shadow, grass, and tree.Thirty samples per class were randomly selected for training and the remainder for testing.
  • Results: S3FSE achieves the best classification performance in both visual interpretation and accuracy on the Pavia city ROSIS dataset.The evaluation uses classification maps, detailed accuracy statistics, and OA across varying embedded dimensionality d.
  • Results: Consistent results across the three hyperspectral datasets support S3FSE as an effective approach for spectral-spatial feature representation and classification.

V. CONCLUSIONS

The paper concludes that S3FSE jointly learns a common low-dimensional spectral-spatial representation while selecting significant original features. Experiments on three public datasets show consistent classification gains over comparison methods.

  • Method and contribution: S3FSE preserves the geometric structure of spectral and spatial features while exploiting their consistency and complementary information through co-manifold learning and co-graph regularization.
  • Method and contribution: An ℓ2,1-norm co-regularization simultaneously performs feature selection and extraction, discarding redundant or noisy features before transforming the significant originals.
  • Method and contribution: The learned common low-dimensional representation is interpretable as a linear combination of a subset of significant original features.
  • Conclusion: Experiments on three public hyperspectral datasets show that S3FSE consistently outperforms the comparison methods for hyperspectral image classification.
Loading 1904.03982v1…