Source-linked AI summary
3DSS-Mamba: 3D-Spectral-Spatial Mamba for Hyperspectral Image Classification
Yan He, Bing Tu, Bo Liu, Jun Li, Antonio Plaza
TL;DR
HSI classification needs global spectral-spatial modeling without the limited receptive fields of CNNs or the quadratic costs of Transformers, while conventional Mamba remains difficult to adapt to 3D hyperspectral data. 3DSS-Mamba addresses this with 3D token generation and spectral-spatial selective scanning across five route priorities. Experiments on three public datasets report superior classification performance over competing methods.
Problem
CNNs and Transformers face receptive-field or quadratic-complexity limitations, while Mamba's adaptation to high-dimensional 3D hyperspectral classification remains underexplored.
Method
3DSS-Mamba converts HSI cubes into 3D spectral-spatial tokens and applies pixel-wise 3DSS scanning through stacked 3DMB blocks.
Results
Experiments on three public hyperspectral datasets indicate that 3DSS-Mamba outperforms competing state-of-the-art methods.
Takeaways & Limitations
The framework provides a feasible HSI classification solution combining global spectral-spatial contextual modeling with linear computational complexity.
Abstract
from arXiv · showhide
Hyperspectral image (HSI) classification constitutes the fundamental research in remote sensing fields. Convolutional Neural Networks (CNNs) and Transformers have demonstrated impressive capability in capturing spectral-spatial contextual dependencies. However, these architectures suffer from limited receptive fields and quadratic computational complexity, respectively. Fortunately, recent Mamba architectures built upon the State Space Model integrate the advantages of long-range sequence modeling and linear computational efficiency, exhibiting substantial potential in low-dimensional scenarios. Motivated by this, we propose a novel 3D-Spectral-Spatial Mamba (3DSS-Mamba) framework for HSI classification, allowing for global spectral-spatial relationship modeling with greater computational efficiency. Technically, a spectral-spatial token generation (SSTG) module is designed to convert the HSI cube into a set of 3D spectral-spatial tokens. To overcome the limitations of traditional Mamba, which is confined to modeling causal sequences and inadaptable to high-dimensional scenarios, a 3D-Spectral-Spatial Selective Scanning (3DSS) mechanism is introduced, which performs pixel-wise selective scanning on 3D hyperspectral tokens along the spectral and spatial dimensions. Five scanning routes are constructed to investigate the impact of dimension prioritization. The 3DSS scanning mechanism combined with conventional mapping operations forms the 3D-spectral-spatial mamba block (3DMB), enabling the extraction of global spectral-spatial semantic representations. Experimental results and analysis demonstrate that the proposed method outperforms the state-of-the-art methods on HSI classification benchmarks.
I. INTRODUCTION
HSI classification combines rich spectral and spatial information but remains challenged by limited receptive fields, quadratic attention costs, and uncertain Mamba adaptation to 3D hyperspectral data. 3DSS-Mamba addresses this gap with 3D tokenization and spectral-spatial selective scanning, with experiments reporting strong performance across three public datasets.
- HSI classification distinguishes ground-object categories using hundreds of continuous spectral bands alongside spatial information.
- CNNs provide local receptive fields, whereas Transformers model long-range dependencies but introduce substantial efficiency and memory overhead.
- Mamba offers linear-complexity long-distance sequence modeling, but its suitability for high-dimensional 3D hyperspectral classification requires further exploration.
- 3DSS-Mamba converts HSI cubes into 3D spectral-spatial tokens and uses 3DSS pixel-wise scanning across spectral and spatial dimensions within stacked 3DMB blocks.
- Experiments on three public hyperspectral datasets report the proposed method's effectiveness and superiority over competing approaches.
II. THE PROPOSED NETWORK
The proposed network is built from State Space Model foundations and organizes the HSI classifier around spectral-spatial token processing, stacked Mamba blocks, and prediction.
- The framework combines State Space Model preliminaries with a 3DSS mechanism, 3DMB blocks, and an overall HSI classification architecture.
- SSMs map one-dimensional inputs to outputs through an intermediate hidden state using state-transition and projection matrices.
- Zero-order hold discretization converts continuous SSM parameters for integration into discrete sequence-based deep models.
- Convolution accelerates the linear recurrence, while S6 introduces input-dependent projection matrices to enable sequence-state interactions.
B. 3D-Spectral-Spatial Selective Scanning
3DSS extends selective scanning from spatial sequences to 3D hyperspectral tokens by flattening spectral and spatial dimensions, scanning with S6, and restoring 3D outputs. Five route priorities support comparison of alternative spectral-spatial orderings.
- 3DSS performs pixel-wise sequential scanning of 3D hyperspectral tokens along both spectral and spatial dimensions to model global relationships.
- The mechanism flattens each 3D token into forward sequences, adds flipped sequences for contextual coverage, and sends them to S6 for selective scanning.
- Five routes test dimension prioritization: spectral-priority, spatial-priority, cross spectral-spatial, cross spatial-spectral, and parallel spectral-spatial.
- Spectral-priority unfolds spectral bands before arranging spatial positions, whereas spatial-priority organizes locations before stacking bands.
- After scanning, one-dimensional mappings are reshaped into 3D structures and merged into output tokens.
C. 3D-Spectral-Spatial Mamba Block
The 3D-Spectral-Spatial Mamba Block uses 3DSS as its core unit to capture global spectral-spatial semantic information with linear computational complexity.
- 3DMB uses the 3DSS mechanism as its core computing unit for global spectral-spatial semantic information extraction.
- The block normalizes inputs, processes parallel embedding branches for gating and 3D convolution, then applies 3DSS, normalization, gating, a linear layer, and residual connection.
- Linear computational complexity allows 3DMB to be stacked more extensively than Transformer blocks under similar computational budgets.
D. 3D-Spectral-Spatial Mamba: Overview
The 3DSS-Mamba overview combines PCA-based patch preparation, spectral-spatial token generation, stacked 3DMB blocks, and classification to process HSI cubes.
- PCA reduces the hyperspectral data dimension before the modified image is divided into 3D patch cubes.
- Patch labels are determined by the central pixel of each input patch.
- SSTG uses a 3D convolution block and embedding operation to project each HSI patch cube into spectral-spatial tokens.
- The generated tokens pass through stacked 3DMB blocks for spectral-spatial semantic extraction, producing successive block outputs.
- After 3DMB modeling, average pooling and a multilayer-perceptron classifier produce the final classification results.
A. Datasets Description
The study evaluates 3DSS-Mamba on three publicly available hyperspectral image databases: Pavia University, Indian Pines, and Houston 2013.
- Three publicly available HSI databases—Pavia University, Indian Pines, and Houston 2013—are used for comprehensive evaluation.
1) Pavia University:
The experimental setup combines multiple HSI datasets, standard accuracy metrics, repeated trials, and sensitivity analyses of scanning routes, kernels, patches, and 3DMB configurations.
- A. Datasets Description: The datasets include Pavia University, Indian Pines, and Houston 2013, with dataset-specific spectral, spatial, and sample characteristics reported.
- B. Experimental Settings: Evaluation uses overall classification accuracy (OA), average classification accuracy (AA), and Kappa under identical experimental conditions.
- B. Experimental Settings: Results are averaged over five consecutive experiments to support fair comparison.
- E. Ablation Study: The study examines five scanning routes in 3DSS to assess the effect of spectral-spatial dimension priority.
C. Ablation Study
The ablation study evaluates scanning-route choices and key input settings for 3DSS-Mamba. The parallel spectral-spatial route and dataset-dependent patch-size analysis guide subsequent experiments.
- Scanning-route ablation: Parallel spectral-spatial scanning outperforms Spectral-priority on Pavia University by 1.0% OA, 1.83% AA, and 1.31% Kappa.Five routes are evaluated: Spectral-priority, Spatial-priority, Cross spectral-spatial, Cross spatial-spectral, and Parallel spectral-spatial.
- Input patch-size analysis: A 13×13 patch size produces the maximum peak on Indian Pines and Houston 2013, while Pavia University accuracy decreases as patch size increases to 11.The tested patch sizes range from 9×9 to 17×17 with an interval of 2.
- Visualization: The classification maps compare 3DSS-Mamba with SVM, CNN, and Transformer-based methods on Pavia University.The figure includes reference, SVM, 1D-CNN, 2D-CNN, 3D-CNN, VIT(Pixel), VIT(Patch), SF, HSI-BERT, and 3DSS-Mamba maps.
3) Different Embedding Dims and Depths for 3DMB:
The study selects a lightweight 3DSS-Mamba configuration while comparing classification behavior across datasets and competing architectures. Results include dataset-specific gains, visual quality, and computational considerations.
- 3DMB structure: Embedding dimension 32 and stacked depth 1 provide a lightweight configuration balancing accuracy and computational burden.Lower dimensions can underfit, whereas larger dimensions and deeper networks provide limited accuracy improvements with added computational cost.
- Pavia University Dataset: On Pavia University, improvements over the suboptimal approach reach 0.86% OA, 0.81% AA, and 1.14% Kappa.The proposed method is reported to provide relatively stable and superior performance against Transformer-based architectures.
- Visualization: The Pavia University classification map for 3DSS-Mamba shows clearer category boundaries and less noise than the compared methods.The comparison particularly notes noise in SVM and misclassifications in 1D-CNN and VIT(Pixel).
2) Indian Pines Dataset:
Across Indian Pines, Houston 2013, and computational analyses, 3DSS-Mamba is reported to deliver strong classification performance, efficiency, and robustness to training-sample proportions.
- Indian Pines Dataset: On Indian Pines, 3DSS-Mamba achieves the highest recognition performance among the compared methods using 10% of the reference samples.It shows improvements in categories including Corn-notill, Cornmintil, and Soybean-notill, with smooth and clear classifications across most regions.
- Houston 2013 Dataset: On Houston 2013, 3DSS-Mamba achieves the highest OA, AA, and Kappa, improving over the suboptimal method by 1.6%, 1.34%, and 1.73%.The method also provides the most precise prediction details despite the dataset’s discrete and local sample targets.
- Computational complexity: 3DSS-Mamba balances classification effectiveness and computational efficiency through linear sequential modeling.VIT(Patch) and SF have almost 10 times the model parameters of 3DSS-Mamba, while CNNs are constrained by local receptive fields and Transformers incur higher resource consumption.
- Robustness assessment: 3DSS-Mamba accuracy steadily increases with larger training-sample proportions and generally outperforms competing methods across most tested percentages.Training proportions range from 1.0% to 10.0% for Indian Pines and Houston 2013, and from 0.5% to 5.0% for Pavia University.
- Conclusion: The paper concludes that 3DSS-Mamba combines global spectral-spatial contextual modeling with linear computational complexity for HSI classification.The conclusion describes the framework as a feasible solution and identifies broader hyperspectral scalability as future work.