PaperlayerView exact revision on arXiv

Source-linked AI summary

Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A Comprehensive Review

Minghua Wang, Danfeng Hong, Zhu Han, Jiaxin Li, Jing Yao, Lianru Gao, Bing Zhang, Jocelyn Chanussot

arXiv:2205.06407v1cs.CVcs.LGeess.IV

TL;DR

The growing volume and high-order structure of hyperspectral data challenge processing methods built for 2-D matrices. This review surveys tensor decomposition across hyperspectral processing tasks and synthesizes task-specific results and open challenges.

Abstract

from arXiv · show

Owing to the rapid development of sensor technology, hyperspectral (HS) remote sensing (RS) imaging has provided a significant amount of spatial and spectral information for the observation and analysis of the Earth's surface at a distance of data acquisition devices, such as aircraft, spacecraft, and satellite. The recent advancement and even revolution of the HS RS technique offer opportunities to realize the full potential of various applications, while confronting new challenges for efficiently processing and analyzing the enormous HS acquisition data. Due to the maintenance of the 3-D HS inherent structure, tensor decomposition has aroused widespread concern and research in HS data processing tasks over the past decades. In this article, we aim at presenting a comprehensive overview of tensor decomposition, specifically contextualizing the five broad topics in HS data processing, and they are HS restoration, compressed sensing, anomaly detection, super-resolution, and spectral unmixing. For each topic, we elaborate on the remarkable achievements of tensor decomposition models for HS RS with a pivotal description of the existing methodologies and a representative exhibition on the experimental results. As a result, the remaining challenges of the follow-up research directions are outlined and discussed from the perspective of the real HS RS practices and tensor decomposition merged with advanced priors and even with deep neural networks. This article summarizes different tensor decomposition-based HS data processing methods and categorizes them into different classes from simple adoptions to complex combinations with other priors for the algorithm beginners. We also expect this survey can provide new investigations and development trends for the experienced researchers who understand tensor decomposition and HS RS to some extent.

I. INTRODUCTION

Hyperspectral images carry rich spatial and spectral information, but matrix-based processing can destroy their spatial-spectral completeness. This review surveys tensor-decomposition methods for HS processing and introduces notation and decomposition preliminaries.

I. INTRODUCTION

  • Tensor decompositions preserve HS images’ spatial-spectral completeness, addressing a limitation of matrix methods that reshape spectral bands into vectors.The review notes that tensor methods have excellent performance with a tolerable increase in computational complexity.Source: §I · p. 2
  • The article presents a comprehensive survey of tensor-decomposition methods for HS processing, reviewing more than 100 publications, most published in the last five years.It presents representative works by decomposition category, discusses variants incorporating HS priors, and uses experimental examples to evaluate methods and identify challenges.Source: §I · p. 2
  • The review’s accompanying table summarizes publication years, method descriptions, prior information, and available codes for repeatability and further study.The authors state that the material is intended to benefit both beginners and experienced readers.Source: §I · p. 2

II. NOTATIONS AND PRELIMINARIES

  • The preliminaries define tensor operations and notation, then introduce Tucker, CP, BT, t-SVD, TT, and TR decompositions, illustrated for third-order tensors.The section also defines special tensors used in describing t-SVD.Source: §II · pp. 2, 4, 5, 6

III. HS RESTORATION

HS restoration estimates a clean image from degraded observations that may include blur, missing data, sparse errors, or additive noise. Because recovery is ill-posed, the models impose constraints reflecting desired image properties.

III. HS RESTORATION

  • Noise, blur, and missing data can degrade image quality and the efficiency of subsequent applications, making restoration a preprocessing step.The cited applications include HS classification for crops and wetlands and refinement of spectral information for target detection.Source: §III · p. 7
  • Restoration estimates the recovered HS image X from observed data T modeled as T = M(X) + S + N.M represents a degradation operator, S sparse error, and N additive noise.Source: §III · p. 7
  • The degradation operator distinguishes restoration tasks: blur yields deblurring, a binary mask yields inpainting, and identity degradation yields destriping or denoising.Denoising may model Gaussian noise alone or mixed sparse and additive noise.Source: §III · p. 7
  • Because restoration is ill-posed, recovery requires constraints on X, including priors such as non-local similarity and spatial and spectral smoothness.The model also uses regularizations on the recovered image and sparse component, with regularization parameters.Source: §III · p. 7

A. HS Denoising

HS denoising addresses mixed noise using tensor-decomposition models and prior constraints that exploit spatial and spectral information.

A. HS Denoising

  • HS images may contain mixed noise, including Gaussian, salt-and-pepper, and dead-line noise.The paper illustrates these noise types in Fig. 5.Source: A. HS Denoising · p. 7
  • Low-rank structure, sparse representation, non-local similarity, and total variation are among the prior constraints used to extract HS spatial and spectral information.Source: A. HS Denoising · p. 7
  • The section introduces different low-rank tensor-decomposition models for denoising and combines them with one or two additional prior constraints.Source: A. HS Denoising · p. 7

1) LR Tensor Decomposition:

The section surveys low-rank tensor models for hyperspectral processing, from factorization and rank minimization to combinations with spatial, spectral, and non-local priors. These added priors address limitations of low-rank-only methods and support denoising while retaining image detail.

1) LR Tensor Decomposition:

  • Low-rank tensor methods are grouped into factorization-based approaches, which require predefined ranks and factor updates, and rank-minimization approaches, which update low-rank tensors directly.Rank bounds are often unavailable in noisy hyperspectral scenes, motivating direct minimization.Source: 1) LR Tensor Decomposition · p. 7
  • Tucker and CP factorization model low-rank structure, but Tucker requires manually set ranks and computing the minimum number of CP rank-1 factors is costly.Examples include LRTA and PARAFAC models for Gaussian-noise denoising.Source: 1) LR Tensor Decomposition · p. 7
  • Pure low-rank tensor decomposition uses hyperspectral low-rank priors but is described as ineffective against mixed noise without other information.Tensor rank minimization is non-convex and NP-hard, so nuclear norms are used as convex surrogates.Source: 1) LR Tensor Decomposition · p. 8

2) Other priors regularized LR Tensor Decomposition:

  • Non-local methods group similar full-band patches into 4-D tensor groups to jointly exploit spatial patch similarity and spectral correlation.Tensor dictionary learning is described as the first model to combine non-local similarity with the low-rank property of these groups.Source: 2) Other priors regularized LR Tensor… · p. 8
  • Spatial and spectral smoothness priors include 2DTV, 3DTV, and SSTV, while LRTDTV incorporates weighted 3DTV into low-rank tensor decomposition.The review notes that isotropic TV can leave blurring artifacts and discusses anisotropic TV for denoising.Source: 2) Other priors regularized LR Tensor… · pp. 9–10
  • TLR-l0TV combines low-rank tensor structure with l0 gradient minimization to preserve edges; it is reported to improve mixed-noise denoising and post-denoising classification accuracy.The review contrasts l0 gradient minimization with l1 TV, which it describes as a relaxation of the l0 gradient.Source: 2) Other priors regularized LR Tensor… · p. 10
  • Subspace methods represent spectral data using a low-dimensional basis and coefficients, with SNLRSF adding non-local constraints to the subspace coefficients.The spectral vectors are assumed to lie in a subspace whose dimension is much smaller than the number of bands.Source: 2) Other priors regularized LR Tensor… · pp. 10–11

F + ||Li||TTN

The described framework combines a spectral low-rank model with non-local spatial factorization. It also specifies how reduced-dimensional coefficient images are formed into patch groups.

F + ||Li||TTN

  • The workflow reshapes the reduced-dimensional coefficient image into a tensor, segments it into overlapping patches, then clusters nearby similar patches by Euclidean distance.The coefficient image has dimensions l × hv; patches have dimensions p × p × l.Source: F + ||Li||TTN · p. 11
  • The approach combines a spectral low-rank tensor model, based on spectral signatures lying in a low-dimensional subspace, with non-local factorization along the spatial direction.Related methods also use spatial-difference group sparsity and spectral continuity constraints.Source: F + ||Li||TTN · p. 11
  • A difference-continuous basis is used to define a non-local tensor subspace, and the passage reports that difference regularization promotes continuity of restored hyperspectral data.Source: F + ||Li||TTN · p. 11

3) Experimental results and analysis:

The experiments compare tensor-decomposition methods for Gaussian and mixed-noise removal and hyperspectral deblurring. NGmeet leads the reported Gaussian-noise metrics, TLR-L0TV leads mixed-noise removal, and OLRT is evaluated across blur cases.

3) Experimental results and analysis:

  • NGmeet obtains the highest reported metric values among the Gaussian-noise denoising methods, while LRTA is the fastest and ITSReg, TDL, and LLRT outperform LRTA.LRTA is described as fastest because it considers only spectral correlation.Source: 3) Experimental results and analysis · p. 11, Table IV · p. 11
  • TLR-L0TV achieves the highest reported performance for mixed-noise removal, preserving texture and edges while removing most noise; LRTR and LRTDGS are the two fastest methods.LRTR and LRTDGS nevertheless leave some dead-lines in this complex mixed-noise case.Source: 3) Experimental results and analysis · p. 11, Table V · p. 11

B. HS Deblurrring

  • OLRT is evaluated on light and heavy Gaussian blur and uniform blur, with the review describing recovery of structural information and image edges.The quantitative evaluation is reported in Table VI, but the supplied passages do not state its comparative values.Source: B. HS Deblurrring · pp. 12–13, Table VI · p. 12

C. HS Inpainting

The section reviews tensor completion for missing hyperspectral data and tensor models for destriping. TVWTR gives the strongest reported inpainting results, while LRTD outperforms WLRTR on the destriping indices described.

C. HS Inpainting

  • Inpainting estimates missing image data from observed values and applies to hyperspectral, multispectral, and multi-temporal remote-sensing images.Examples of missing data include dead pixels, thick clouds, and cloud shadows.Source: C. HS Inpainting · p. 13, Figure 12 · p. 13
  • TVWTR performs best in random-missing and text-removal inpainting, recovering details with fewer artifacts than HaLRTC and LRTC.The experiments use a Houston2013 subimage; under an 80% missing ratio, HaLRTC and LRTC leave artifacts, and HaLRTC and LRTC are the fastest and second-fastest methods.Source: C. HS Inpainting · p. 13, Figure 13 · p. 14, Figure 14 · p. 14

D. HS Destriping

  • Stripe noise is associated with non-uniform responses of neighboring detectors, and stripes tend to occur along the sensor scanning direction.Source: D. HS Destriping · p. 13
  • LRTD scores higher than WLRTR on all four reported destriping indices, and its spatial and spectral smoothness prior helps preserve clearer edges and details.In the example, both methods remove nonperiodic stripes by considering non-local similarity and spectral correlation.Source: D. HS Destriping · p. 15, Table VIII · p. 15

E. Future challenges

The review identifies unresolved restoration challenges and surveys tensor-decomposition approaches to hyperspectral compressed sensing (CS), including reconstruction models that combine tensor structure with additional priors.

E. Future challenges

  • A unified framework combining non-local similarity, spatial and spectral smoothness, and subspace representation remains a key restoration challenge.The review also flags manual adjustment of regularization parameters, blind estimation of degradation locations, and the computational cost of high-dimensional tensors.Source: E. Future challenges · p. 15

IV. HS CS

  • HS CS reconstructs a hyperspectral tensor from compressive measurements, but direct inversion is ill-posed and requires additional prior information.The measurement operator can use random downsampling, a random permutation, and a Walsh-Hadamard transform; the regularization term can encode spectral correlation, smoothness, or non-local similarity.Source: §IV · p. 15

A. Tensor decomposition-based HS CS reconstruction methods

  • Tucker-based CS methods combine low-rank structure with priors such as total variation, non-local similarity, and spectral correlation.JTenRe3-DTV uses weighted 3-D total variation, while other approaches group similar patches or use tensor-ring decomposition in a low-dimensional subspace.Source: A. Tensor decomposition-based HS CS… · pp. 15–17

B. HS Kronecker CS methods

The section describes multidimensional Kronecker sampling and tensor-based reconstruction, then reports comparisons of CS methods and introduces tensor approaches to hyperspectral anomaly detection.

B. HS Kronecker CS methods

  • Kronecker CS uses sensing matrices across the three hyperspectral dimensions to reduce measurement dimensionality, and tensor-based methods exploit multidimensional block sparsity during reconstruction.The described methods include NBOMP, BOSE, and TBR; TBR uses a low-complexity technique to decouple hyperparameters.Source: B. HS Kronecker CS methods · p. 17

C. Experimental results and analysis

  • At lower sampling ratios, SLNTCS obtains poorer reconstruction results than JTenRe3DTV; at 10%, JTenRe3DTV gives a more acceptable result with fewer artifacts in the enlarged area.The comparison uses the Reno data set and a randomly permuted Hadamard transform; Fig. 17 shows a representative band at 10%.Source: C. Experimental results and analysis · p. 17, Figure 17 · p. 17

D. Future challenges

  • The review identifies finding more appropriate tensor decompositions and improving robustness to noise as open challenges for hyperspectral CS.Tucker decomposition has been applied successfully, but the review calls for exploring newer decompositions and handling noise in sampling and reconstruction.Source: D. Future challenges · pp. 17–18

V. HS AD

  • Hyperspectral anomaly detection seeks manmade objects that occupy small areas and differ spectrally from their surrounding background, often framing detection as unsupervised two-class classification.The review focuses primarily on Tucker decomposition-based methods because Tucker was the first and an essential tensor-decomposition approach used for this task.Source: §V · p. 18

A. Tensor decomposition-based HS AD methods

  • Tucker-based anomaly-detection methods model the background as low-rank and anomalies as sparse, with variants using clustering, reconstruction error, or endmember extraction.Other tensor approaches include PTA, which constrains the background with truncated nuclear norm and spatial total-variation regularization.Source: A. Tensor decomposition-based HS AD… · p. 18

B. Experimental results and analysis

The review illustrates PTA anomaly detection on three airport data sets and reports AUC values above 0.9. It also notes unresolved challenges involving background complexity and the limited exploration of decomposition types.

B. Experimental results and analysis

C. Future challenges

  • Other tensor-decomposition frameworks remain rarely investigated for hyperspectral anomaly detection, despite the development of Tucker-based models.Complex backgrounds and few targets also increase detection difficulty, while roads and roofs can remain among the detected background pixels.Source: C. Future challenges · p. 19
  • The review identifies a need for consolidated optimization that accounts for both spatial sparsity and the 3-D structure of anomalies.Its described modeling convention represents background and anomalies as the low-rank and sparse parts, respectively.Source: C. Future challenges · p. 19

VI. HS-MS FUSION

Hyperspectral–multispectral fusion uses a high-spatial-resolution multispectral image to improve hyperspectral spatial resolution while retaining spectral resolution. The review surveys coupled CP and Tucker approaches that model the two images jointly.

VI. HS-MS FUSION

  • HS–MS fusion seeks a high-spatial-resolution hyperspectral image with its original spectral resolution, using a high-spatial-resolution multispectral image as assistance.The low-resolution hyperspectral input is modeled as spatially downsampled and blurred, while the multispectral input is spectrally downsampled; the degradation operators are assumed known in advance.Source: §VI · pp. 19–20

1) CP Decomposition Model:

  • Coupled CP methods model the fused image through latent factors and degradation matrices, with identifiability guaranteed under relaxed conditions but equal treatment of dimensions noted as potentially unsuitable for real HS super-resolution.Later approaches include SCUBA, which combines CP decomposition and matrix factorization, and a method adding nonlocal tensor extraction.Source: 1) CP Decomposition Model · p. 20

2) Tucker Decomposition Model:

  • Tucker-based fusion methods estimate mode-wise dictionaries and a sparse core, with variants adding non-local structure, total variation, or manifold regularization.Other cited approaches include coupled Tucker approximation, nonnegative Tucker decomposition, and models admitting additive variability.Source: 2) Tucker Decomposition Model · pp. 20–21

3) BT Decomposition Model:

The review surveys BT, TT, TR, and tensor-rank-minimization approaches to hyperspectral super-resolution, describing how these decompositions and added priors model spatial-spectral structure.

3) BT Decomposition Model:

  • NN-CBCTD applies a coupled nonnegative BT decomposition to HS super-resolution, addressing the lack of physical meaning in latent factors under CP and Tucker models.The journal version adds recoverability analysis and a more flexible LL1 framework solved by block coordinate descent; Graph Laplacian was also introduced into CBTD.Source: 3) BT Decomposition Model · p. 21

4) TT Decomposition Model:

  • Dian et al. proposed LTTR for HS SR, learning correlations among spatial, spectral, and non-local modes of 4-D FBP patches; Li et al. presented NLRSR, using TT-rank constraints on 4-D non-local patches.Source: 4) TT Decomposition Model · p. 21

5) TR Decomposition Model:

  • CTRF couples TR factorization for super-resolution, while NCTRF adds nuclear-norm regularization to the third, spectral TR factor.Related extensions use graph-Laplacian regularization, factor smoothing, or low-rank properties of non-local patches and TR factors.Source: 5) TR Decomposition Model · p. 21

6) Tensor Rank Minimization for SR:

  • LTMR reconstructs HR-HS images from a spectral subspace and coefficients of grouped patches; related methods add truncation, non-local sparsity, or noise robustness.FLTMR speeds LTMR using a truncation value; NPTSR models non-local patch similarities, while TV-TLMR and TTN-based methods address mixed noise and striping, respectively.Source: 6) Tensor Rank Minimization for SR · pp. 21–22

B. Experimental results and analysis

The review compares tensor-based HS-MS fusion methods and surveys related fusion challenges and linear spectral unmixing models. It also describes how tensor representations and additional priors are used to model abundance and endmember structure.

B. Experimental results and analysis

  • The fusion evaluation uses a simulated University of Houston image and five metrics, with smaller SAM indicating better performance and CC ranging from 0 to 1.The reference sub-image is 400 × 400 × 46; the LR-HS input is produced with a Gaussian blur kernel of size five.Source: B. Experimental results and analysis · pp. 22–23
  • CSTF leads in MPSNR and ERGAS, while SC-LL1 leads in SAM, RMSE, and CC; LTMR and LTTR show poor spectral reconstruction despite good spatial results.The comparison covers CSTF, Blind-STEREO, LTTR, SC-LL1, and LTMR.Source: B. Experimental results and analysis · p. 23

C. Future challenges

  • Pixel-level HS-MS fusion requires registered inputs, and fusion quality depends heavily on registration accuracy; non-registered fusion remains a challenge.The review notes that current methods emphasize fusion more than registration between modalities captured by different platforms and times.Source: C. Future challenges · p. 23

VII. HS SU

  • HS spectral unmixing separates observed spectra into endmembers and fractional abundances, with linear mixing assuming each incident light interacts with one material.The review contrasts this assumption with intimate and multilayered nonlinear mixtures, which involve interactions at microscopic or macroscopic scales.Source: §VII · p. 23, A. Linear Mixing Model · p. 23

A. Linear Mixing Model

  • A matrix-vector tensor factorization represents an HS data tensor as sums of endmember vectors paired with abundance matrices.This factorization is linked to the linear mixing model, with noise represented as an additional component.Source: 2) BT Decomposition Model · p. 24
  • Tensor unmixing methods add priors such as total variation, sparsity, low rank, and nonlocal similarity to model abundance structure.Examples include MVNTF-TV, SPLRTF, and nonlocal tensor-based sparse unmixing.Source: 2) BT Decomposition Model · p. 24, 3) Mode-3 Tensor Representation Model · p. 25

B. NonLinear Mixing Model

The review describes tensor-based nonlinear unmixing and compares several methods on Urban hyperspectral data. It identifies evaluation without ground truth, complex mixture modeling, computational cost, and broader model applicability as open concerns.

B. NonLinear Mixing Model

  • The tensor nonlinear model represents the observed cube using linear endmember contributions, bilinear interaction abundances, interaction endmembers, and noise.The method described uses low-rank properties of abundance and nonlinear interaction maps.Source: B. NonLinear Mixing Model · p. 26

C. Experimental results and analysis

  • LR-NTF outperforms the competing tensor-based methods on Urban data by combining a nonlinear mixing model with low-rank abundance regularization.The comparison also reports improvements from TV regularization and SeCoDe’s convolutional treatment of spectral variability over MVNTF.Source: C. Experimental results and analysis · p. 26

D. Future challenges

  • RMSE and SAD support quantitative unmixing comparisons only when reference abundances and endmembers exist, leaving real-scene evaluation metrics an open need.Source: D. Future challenges · p. 27
  • Tensor-based nonlinear unmixing needs models for complex intimate and multilayered interactions, while high computational costs constrain large images and many endmembers.The review calls for more efficient approaches as image size and endmember count increase.Source: D. Future challenges · p. 27

VIII. CONCLUSION

  • The review identifies model applicability, parameter adjustment, computational efficiency, methodological feasibility, and multi-mission applications as challenges for tensor methods.The paper also notes that suitable handcrafted priors vary by scenario and suggests integrating deep learning to learn features implicitly.Source: §VIII · p. 27

Related papers

Loading 2205.06407v1…