PaperlayerView exact revision on arXiv

Source-linked AI summary

Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification

Danfeng Hong, Xin Wu, Pedram Ghamisi, Jocelyn Chanussot, Naoto Yokoya, Xiao Xiang Zhu

arXiv:1912.08847v1cs.CV

TL;DR

Local semantic changes can degrade spatial features and make the same materials harder to identify across hyperspectral scenes. The paper proposes invariant attribute profiles that combine spatial and frequency-domain invariant features, demonstrating potential and superiority for HSI classification while showing weaker discrimination for irregular classes.

  • Problem

    Local semantic changes and spectral variability can degrade spatial modeling, while spectral-only inputs are also affected by noise, labeling uncertainty, and spectral variability.

  • Method

    Invariant attribute profiles combine spatial invariant features from isotropic filtering and spatial aggregation with frequency invariant features that model invariant behaviors through continuous Fourier-based representations.

  • Results

    The resulting IAPs demonstrate potential and superiority for HSI classification, with progressively higher classification results as their components are fused.

  • Takeaways & Limitations

    IAPs provide a combined spatial-frequency representation intended to improve robustness against local semantic change in HSI classification.

  • Takeaways & Limitations

    IAPs are more discriminative for regularly structured classes than for irregular classes such as grass and tree, limiting gains for those materials.

Abstract

from arXiv · show

Up to the present, an enormous number of advanced techniques have been developed to enhance and extract the spatially semantic information in hyperspectral image processing and analysis. However, locally semantic change, such as scene composition, relative position between objects, spectral variability caused by illumination, atmospheric effects, and material mixture, has been less frequently investigated in modeling spatial information. As a consequence, identifying the same materials from spatially different scenes or positions can be difficult. In this paper, we propose a solution to address this issue by locally extracting invariant features from hyperspectral imagery (HSI) in both spatial and frequency domains, using a method called invariant attribute profiles (IAPs). IAPs extract the spatial invariant features by exploiting isotropic filter banks or convolutional kernels on HSI and spatial aggregation techniques (e.g., superpixel segmentation) in the Cartesian coordinate system. Furthermore, they model invariant behaviors (e.g., shift, rotation) by the means of a continuous histogram of oriented gradients constructed in a Fourier polar coordinate. This yields a combinatorial representation of spatial-frequency invariant features with application to HSI classification. Extensive experiments conducted on three promising hyperspectral datasets (Houston2013 and Houston2018) demonstrate the superiority and effectiveness of the proposed IAP method in comparison with several state-of-the-art profile-related techniques. The codes will be available from the website: https://sites.google.com/view/danfeng-hong/data-code.

I. INTRODUCTION

The introduction motivates invariant attribute profiles (IAPs) by showing why conventional spatial profiles can vary under local semantic changes in hyperspectral scenes. IAPs address this through spatial and frequency-domain invariant feature extraction for HSI classification.

  • HSI classification benefits from rich spectral information, but noisy pixels, labeling uncertainty, and spectral variability degrade performance when only spectral profiles are used.
  • Morphological profiles use fixed-shape structuring elements, which struggle to represent complex hyperspectral regions and higher-level spectral characteristics.
  • Attribute profiles offer flexible connected-component filtering, but geodesic reconstruction can cause leakage by merging regions belonging to different objects.
  • Conventional APs can differ substantially for the same material when local scenes shift, rotate, change composition, or exhibit illumination and atmospheric variability.
  • IAPs combine spatial invariant features from isotropic filtering and spatial aggregation with frequency invariant features from continuous Fourier-based representations in polar coordinates.
  • Experiments on challenging hyperspectral datasets report superior classification performance for IAPs compared with AP-based approaches and other state-of-the-art profile techniques.

II. METHODOLOGY

The methodology constructs invariant attribute profiles from spatially filtered and spatially aggregated hyperspectral features. These spatial invariant features use isotropic convolution, superpixel-based aggregation, and PCA-assisted SLIC segmentation.

  • Spatial invariant features: IAP spatial invariant features begin by extracting receptive-field convolutional features from each hyperspectral band with isotropic kernels.The feature set stacks bandwise convolutional responses, with K_conv aggregating local spatial information.
  • Spatial invariant features: Spatial aggregation combines filtered convolutional features with SLIC superpixels to enhance object-based edges, shapes, and feature invariance.Each pixel’s representation averages convolutional responses over the superpixel containing that pixel.
  • Spatial invariant features: Before SLIC, PCA reduces HSI spectral dimensionality while preserving spectral information, and SLIC operates on the first three principal components.This adapts SLIC from conventional color spaces to a compact hyperspectral representation.
  • Spatial invariant features: The SIF representation is formed by stacking the superpixel-aggregated responses across the extracted feature components.The cited equations describe the final stacked SIFs after spatial aggregation.

B. Invariant Feature Extraction in the Frequency Domain

Frequency-domain extraction complements spatial features by modeling rotation and other semantic changes with continuous Fourier representations in polar coordinates. The procedure constructs magnitude, absolute rotation-invariant, and relative rotation-invariant features before regional aggregation.

  • Motivation and invariance: Discrete spatial-domain HOG is sensitive to quantization artifacts, whereas continuous Fourier-based representations model rotations as smooth circular shifts.A 10° rotation produces a large discrete-histogram change but only a slight horizontal shift with largely unchanged feature shape in the Fourier-based descriptor.
  • Polar Fourier representation: Polarized Fourier analysis separates angular and radial information, allowing rotation in Cartesian coordinates to be represented as a shift in polar coordinates.The gradients are represented in polar form through magnitude and phase, providing the basis for invariant feature construction.
  • Rotation-invariant construction: Fourier-order combinations are constrained to produce rotation-invariant outputs, with the resulting convolutional representation treated as the final invariant feature.The construction includes both absolute and relative order-fitting components.
  • Pixel-wise Fourier features: The pixel-wise frequency procedure first computes Fourier-order magnitude features, then removes phase for absolute rotation invariance and couples neighboring kernel radii for relative invariance.Relative features retain richer phase information by combining adjacent convolutional Fourier representations.
  • Regionalization: The frequency-domain features are organized as pixel-wise Fourier features that are later collected and converted into regional descriptors.The regionalization step uses multi-scaled isotropic triangular convolutional kernels.

3) Regional Descriptors:

Regional descriptors aggregate pixel-wise Fourier features into multi-scale representations that capture contextual information across different receptive fields.

  • Regional descriptors: Pixel-wise Fourier features are aggregated into region-based descriptors using isotropically triangular convolutional kernels, analogous to HOG blocks.Multiple kernel scales capture semantic information from different receptive fields.
  • Regional descriptors: The resulting regional descriptors are indexed by convolutional-kernel scale and form the frequency invariant feature representation.Each descriptor corresponds to one selected convolutional kernel.

C. Invariant Attribute Profiles (IAPs)

The IAP extraction workflow combines grouped-band gradient processing, polarized Fourier features, spatial and frequency regional descriptors, and final feature assembly. The resulting IAPs concatenate SIFs with magnitude and order-fitting FIFs.

  • Step 1: HSI grouping and gradients: The workflow first groups hyperspectral bands and computes horizontal and vertical gradients from maximum-magnitude responses.The number of groups affects redundancy and coupling across bands and is selected by cross-validation on labeled training data.
  • Classification framework: The classification framework consists of HSI grouping, IAP extraction, feature stacking, and feature learning or dimensionality reduction.The described implementation uses PCA for feature learning or refinement.
  • Step 2: Fourier features: Polarized Fourier features are then extracted from the gradient-derived complex representation.Fourier transformation of the complex gradient form produces the polarized representation.
  • Step 3: Regional descriptors: Regional descriptors are constructed in both spatial and frequency domains using isotropic spatial filters and Fourier convolutional kernels.This step builds region-based representations before final profile generation.
  • Step 4: IAP generation: The final IAPs use superpixel segmentation for SIFs and magnitude plus absolute and relative order-fitting components for FIFs.The profiles are abstracted as a concatenation of spatial and frequency invariant features.

D. IAPs-based HSI Classification Framework

The IAP-based HSI classification framework combines band-wise HSI grouping, IAP extraction, feature stacking, and feature learning or dimensionality reduction.

  • The framework consists of HSI grouping, IAP extraction, feature stacking, and feature learning or dimensionality reduction.Feature learning is conducted with PCA in the described implementation.

E. Feasibility and Effectiveness Analysis of the Proposed IAPs for HSI Classification

The study evaluates spatial-spectral feature extraction for HSI classification on Pavia University, Houston2013, and Houston2018, using representative datasets and challenging train-test settings.

  • HSI classification jointly models spatial and spectral information rather than relying on pixel-wise semantic labeling alone.
  • The experiments use Pavia University, Houston2013, and Houston2018 as representative HSI classification datasets.
  • Pavia University Dataset: Pavia University contains 610 × 340 pixels, 103 spectral bands, and nine land-cover categories.The data were acquired by ROSIS at a 1.3m ground sampling distance over the Pavia University campus.
  • Houston2013 Dataset: Houston2013 contains 349 × 1905 pixels, 144 spectral channels, and 15 challenging land-cover classes.The data were acquired with the ITRES CASI-1500 sensor over the University of Houston and surrounding areas.
  • Houston2018 Dataset: Houston2018 contains 601×2384 pixels and 50 spectral bands sampled from 380nm to 1050nm at 10nm intervals.The dataset is an airborne multimodal data product acquired with the CASI-1500 sensor at a 1m ground sampling distance.
  • Challenging training pixels and fixed train-test samples are used to assess extracted features more effectively and support consistent evaluation.The passage contrasts this strategy with random selection, which usually yields very high performance.

B. Experimental Setup

The experimental setup compares IAPs with OSF and six profile-based methods using NN and RF classifiers, with OA, AA, and κ as evaluation criteria.

  • Performance is quantified with Overall Accuracy, Average Accuracy, and Kappa Coefficient using nearest neighbor and random forest classifiers.
  • The investigated MP-based and AP-based methods are categorized as unsupervised feature extraction.
  • The experimental figures visualize scene categories, false-color imagery, train-test sample distributions, and classification maps for Houston2013 and Pavia University.
  • The comparison includes the original spectral features and six advanced morphological or attribute-profile methods alongside IAPs.The compared profiles are EMPs, EAPsa, EAPss, EAPsi, EAPsall, and EEPs.
  • PCA produces the first three principal components before morphological and attribute-profile operations are applied.
  • The IAP parameters include the number and radii of spatial convolution kernels, Fourier orders, feature-learning or reduction dimension, and grouped-HSI count.Cross-validation on the available training set can determine these parameters in practical applications.

C. Results and Discussion

Across the reported comparisons, profile-based features outperform OSF, while IAPs address spectral variability and local semantic changes and achieve stronger classification results on the evaluated scenes.

  • Houston2018 Dataset: The classification maps and quantitative results are reported using OA, AA, and κ for compared methods with NN and RF classifiers.
  • Pavia University Dataset: At least 7% higher accuracy is obtained with extracted features than with OSF for both NN and RF classifiers.
  • Pavia University Dataset: Around 5% OA improvement is achieved by EAPsall over the single-attribute profile, while EEPs adds approximately 2% OA over EAPsall.
  • Earlier profile methods model spatial information effectively but are relatively weak at handling spectral variability and local scene or object semantic changes.
  • Pavia University Dataset: IAPs produce less noisy classification maps, with the Meadows class more closely approaching the manually labeled ground truth.
  • Pavia University Dataset: The IAP method extracts intrinsic invariant HSI features and dramatically outperforms the other methods in the quantitative comparison reported in Table IV.

2) Houston2013 Dataset:

On Houston2013, IAPs outperform the compared spatial feature extractors and improve classification in regions affected by local semantic changes such as shifts and rotations.

  • Houston2013 Dataset:: The experiments compare feature extraction methods using NN and RF, with classification maps and OA, AA, κ, and per-class accuracies reported.The cited figure and table jointly describe the visual and quantitative evaluation.
  • Houston2013 Dataset:: Feature extraction consistently outperforms the OSF baseline for both NN and RF classifiers.This comparison is reported for the Houston2013 classification experiments.
  • Houston2013 Dataset:: EEPs improve OAs by approximately 2% over EAPsall and 5% over EMPs, particularly with the RF classifier.The comparison concerns the reported Houston2013 results.
  • Houston2013 Dataset:: EAPsall benefits from stacking single attributes, producing a more discriminative representation than individual EAPs or EMPs.The passage attributes this improvement to fusing multiple attributes and modeling richer geometrical features.
  • Houston2013 Dataset:: IAPs outperform other feature extraction operators overall in OA, AA, and κ, with at least 10% accuracy gains for Commercial, Highway, and Parking Lot1.The resulting maps are smoother within homogeneous materials and sharper at material boundaries.

D. Parameter Sensitivity Analysis

Parameter sensitivity analysis identifies dataset-specific settings for IAPs and shows that Fourier order and post-extraction feature dimension strongly affect classification performance.

  • Parameter Sensitivity Analysis: The ns search ranges from 1 to 5, while r, m, and d are evaluated over the specified kernel-radius, Fourier-order, and dimension ranges.The dimension range extends from 10 to each dataset’s original IAP dimension at intervals of 10.
  • Parameter Sensitivity Analysis: Optimal settings are ns = 3, r = [2, 4, 6], m = [0, 1, 2, 3], d = 30 for Pavia University and Houston2013, and m = [0, 1, 2], d = 40 for Houston2018.The spatial-kernel settings are shared across the three datasets, while Fourier order and feature dimension differ for Houston2018.
  • Parameter Sensitivity Analysis: Fourier order m is important: increasing it initially causes a sharp performance decrease, after which results stabilize at a relatively poor level.This pattern distinguishes m from the spatial-kernel parameters.
  • Parameter Sensitivity Analysis: Feature dimension d reaches an optimal plateau near 30 for Pavia University and Houston2013 and 40 for Houston2018 before diminishing for RF.For NN, classification accuracy decreases dramatically when the IAP input dimension is higher.
  • Parameter Sensitivity Analysis: Dimension-reduced features generally outperform original features, but applying DR after prior reduction can degrade EsP-based methods.The paper suggests repeated reduction, such as PCA before and after extraction, may cause reuse-related degradation.

E. Ablation Studies

Ablation studies show progressively stronger representations when IAP components are fused, while feature learning or reduction is particularly important for classification performance.

  • Ablation Studies: Successively adding SIFs, FIFs, and FL or DR progressively improves classification performance and feature representation ability.The ablation evaluates the IAP components through step-wise fusion across the three datasets.
  • Ablation Studies: Applying FL or DR to original IAPs produces a great classification improvement and plays a significant role in the IAP-based framework.The paper links this to different feature spaces, feature redundancy, and the balance between feature count and training samples.
  • Ablation Studies: SIFs alone outperform FIFs alone, but combining SIFs and FIFs yields greater feature diversity and robustness.The joint IAP representation combines OSF, SIFs, and FIFs.
  • Ablation Studies: FIFs alone perform poorly, indicating that frequency information by itself is insufficiently discriminative for identifying varied materials.Its contribution is stronger when jointly used with spatial features.
  • Ablation Studies: The proposed IAP descriptor combines invariant attribute profiles from spatial and frequency domains to improve robustness against local semantic changes.The conclusion identifies this combination as the paper’s central feature-extraction design.
Loading 1912.08847v1…