PaperlayerView exact revision on arXiv

Source-linked AI summary

Hyperspectral Unmixing Based on Nonnegative Matrix Factorization: A Comprehensive Review

Xin-Ru Feng, Heng-Chao Li, Rui Wang, Qian Du, Xiuping Jia, Antonio Plaza

arXiv:2205.09933v1cs.CVeess.IV

TL;DR

Hyperspectral unmixing seeks endmembers and abundances from mixed pixels, and this survey reviews NMF-based approaches that exploit HSI information. Its experiments report benefits from sparsity, spatial information, robust constraints, and multilayer or deep architectures, alongside runtime differences and a limitation in Road estimates.

  • Problem

    Hyperspectral unmixing must estimate endmembers and abundances from mixed HSI pixels, while spectral variability and nonlinear interactions pose challenges for unmixing.

    Source: §I · p. 2, §II · p. 3

  • Method

    The survey uses NMF as a baseline and reviews methods that exploit spectral, spatial, and structural information through constrained, structured, and generalized formulations.

    Source: §I · p. 2

  • Results

    Methods generally achieve satisfactory mean SAD for most materials, and the survey reports benefits from sparsity, spatial information, robust constraints, and multilayer or deep architectures.

    Source: §VI · pp. 16, 18

  • Takeaways & Limitations

    On Jasper Ridge, L1/2-NMF is fastest among the compared methods, while more complex factorizations require more time.

    Source: §VI · p. 18

    Road estimates are undesirable for all methods, possibly because few Road pixels cause VCA to fail during initialization.

    Source: §VI · p. 17

Abstract

from arXiv · show

Hyperspectral unmixing has been an important technique that estimates a set of endmembers and their corresponding abundances from a hyperspectral image (HSI). Nonnegative matrix factorization (NMF) plays an increasingly significant role in solving this problem. In this article, we present a comprehensive survey of the NMF-based methods proposed for hyperspectral unmixing. Taking the NMF model as a baseline, we show how to improve NMF by utilizing the main properties of HSIs (e.g., spectral, spatial, and structural information). We categorize three important development directions including constrained NMF, structured NMF, and generalized NMF. Furthermore, several experiments are conducted to illustrate the effectiveness of associated algorithms. Finally, we conclude the article with possible future directions with the purposes of providing guidelines and inspiration to promote the development of hyperspectral unmixing.

I. INTRODUCTION

Hyperspectral unmixing estimates endmembers and abundances from mixed pixels; this section motivates NMF for the task and presents its baseline model and main development directions.

I. INTRODUCTION

  • Unmixing decomposes mixed HSI pixels into endmembers and their corresponding abundances.Mixed spectral responses arise because pixels can contain multiple ground objects.Source: §I · p. 1
  • Geometrical methods may struggle without pure signatures, while sparse regression is limited by coherent libraries and varying imaging conditions.Geometrical methods may also struggle with highly mixed data.Source: §I · p. 2
  • NMF is attractive because its nonnegativity and interpretability suit joint estimation of endmembers and abundances; multilayer and deep variants retain interpretability while exploring hidden information.Current deep-learning approaches may require many training samples and network parameters for satisfactory unmixing.Source: §I · p. 2
  • The survey organizes NMF improvements into constrained, structured, and generalized forms, and reports experiments intended to demonstrate associated algorithms.Constraints add terms such as sparsity, smoothness, or graph penalties; structured NMF modifies the cost function, while generalized NMF extends the decomposition form.Source: §I · p. 2

II. CLASSICAL NMF FOR HYPERSPECTRAL UNMIXING

  • Classical NMF approximates the HSI data matrix as the product of nonnegative endmember and abundance matrices, consistent with the linear mixture model.The abundance matrix is subject to nonnegativity and sum-to-one constraints.Source: §II · p. 3
  • To impose the abundance sum-to-one constraint, the observation and signature matrices are redefined using a parameter that controls the constraint's impact.The survey also describes random, VCA, and ATGP initialization for endmembers, FCLS initialization for abundances, and accelerated optimization methods.Source: §II · p. 3
  • Because NMF's cost function is non-convex and can reach local optima, the survey reviews improvements through added constraints, cost-function structure, and extended decomposition forms.These directions address spectral and spatial information, structural details, and characteristics such as nonlinearities and hidden information.Source: §II · p. 3

III. CONSTRAINED NMF FOR HYPERSPECTRAL UNMIXING

Constrained NMF adds endmember and abundance regularizers to the factorization, using HSI spectral and spatial information to improve unmixing. Endmember constraints include simplex volume, compactness, smoothness, prior spectra, and semantic information.

III. CONSTRAINED NMF FOR HYPERSPECTRAL UNMIXING

  • The constrained NMF framework adds endmember and abundance regularization terms, balanced by nonnegative parameters α and β.The review then organizes endmember constraints separately from abundance constraints.Source: §III · p. 3

A. Endmember constraints

  • Endmember constraints also promote spectral smoothness, match known spectral signatures, or use clustering to incorporate high-level semantic information.The reviewed methods use different smoothness metrics, prior-spectrum discrepancies, and clustering-guided regularization.Source: A. Endmember constraints · pp. 3, 5, 6
  • Minimum-volume constraints avoid the pure-pixel assumption, but some methods require dimensionality reduction to calculate simplex volume.The simplex vertices correspond to endmembers; alternatives include Gram determinant, logarithm Gram determinant, and nuclear-norm regularizers.Source: A. Endmember constraints · p. 4
  • Endmember distance constrains simplex compactness by summing distances from each endmember to the centroid; this constraint is simpler and convex.MDC-NMF uses this endmember distance constraint.Source: A. Endmember constraints · p. 4
  • Other endmember constraints address intra-class variability or nonlinear mixtures, including class-specific spectra and BMM-based component projections.The BMM-based method projects pixels into approximate linear mixture components to reduce collinearity.Source: A. Endmember constraints · p. 5

B. Sparsity constraints

Abundance sparsity constraints range from elementwise penalties to collaborative and spatial-group structures. The review compares their sparsity behavior and notes trade-offs in optimization, noise sensitivity, and how spatial structure is represented.

B. Sparsity constraints

  • Lq regularizers encourage sparse abundances, with 0 < q < 1 producing sparser results than L1 but creating nonconvex optimization challenges.L1 is popular but is not compatible with the abundance sum-to-one constraint; L1/2 is widely used, while data-guided methods adapt q to pixel mixedness.Source: B. Sparsity constraints · p. 6
  • GMCA models each abundance row as a combination of sparsely represented morphological components, naturally incorporating spatial information through abundance-map morphology.A concatenated dictionary discriminates among components, and their coefficients are constrained by an L1 norm.Source: B. Sparsity constraints · p. 7
  • Transform-domain sparsity, correntropy-based adaptive penalties, and GMC regularization offer alternatives to direct abundance sparsity penalties.The GMC penalty is nonconvex and nonseparable, and is described as avoiding systematic underestimation of high components of sparse vectors.Source: B. Sparsity constraints · p. 7
  • Collaborative sparsity imposes row sparsity across pixels and can remove redundant endmembers when their number is overestimated.It does not itself explore spatial information; weighted L2,1 penalties on local blocks are one reviewed extension.Source: B. Sparsity constraints · p. 7
  • Spatial group sparsity uses superpixel groups to encourage shared sparse structure, while group-based methods can also represent spectral variability.The review notes that effective grouping of abundances remains an open issue.Source: B. Sparsity constraints · p. 7
  • L1/2 can yield sparser solutions than L1, while DGC-NMF uses L1/2 for highly sparse pixels and L2 for smoother, low-sparsity pixels.The review also notes that L1/2 benefits from combination with other constraints to represent intrinsic HSI structure.Source: B. Sparsity constraints · pp. 6–7
  • Weighted L1 penalties target abundance sparsity in spectral or spatial domains, but the double-reweighted approach is sensitive to noise corruption.Local-coordinate and subblock penalties additionally incorporate local spatial information.Source: B. Sparsity constraints · pp. 6–7

C. Smooth constraints

Smoothness constraints use spatial relationships to regularize abundance maps, from local neighborhoods and adaptive weights to total variation and nonlocal methods. Regular windows can overlook edges and local detail, motivating adaptive irregular neighborhoods.

C. Smooth constraints

  • Abundance smoothness methods use neighboring pixels because they are more likely to contain the same materials.The review describes smooth constraints as a way to preserve spatial-contextual information.Source: C. Smooth constraints · pp. 7–8
  • ASSNMF uses surrounding elements of each abundance map and a linear transform at low computational cost, extending beyond adjacent-element smoothness.The review cautions that pixel pairs at equal spatial distances do not necessarily have equal smoothness levels.Source: C. Smooth constraints · p. 8
  • Weighted NMF and adaptive local-neighborhood constraints encode pixel contributions while combining spectral and spatial information; separation constraints can prevent over-smoothing.The weight for a neighboring pixel characterizes how much it contributes to the considered pixel.Source: C. Smooth constraints · p. 8
  • TV regularization captures piecewise smoothness in abundance maps and is combined with sparsity or endmember constraints in reviewed methods.Nonlocal TV and log-sum regularization extend from piecewise to nonlocal smoothness.Source: C. Smooth constraints · p. 8
  • Predefined regular-shaped neighborhoods can ignore edges and local spatial details, whereas irregular shapes may adapt more reasonably.The review identifies this as a limitation of the neighborhood choices in the methods discussed.Source: C. Smooth constraints · p. 8

D. Manifold constraints

Manifold constraints encode the low-dimensional structure of hyperspectral data through graph regularizers and learned relationships. Reviewed approaches construct affinities from spectral, joint spectral-spatial, region-based, or locally linear relationships.

D. Manifold constraints

  • Graph regularization exploits local geometric structure, with affinity matrices used to define the graph relationships.The review motivates manifold methods by noting that data may lie on a low-dimensional submanifold of the high-dimensional space.Source: D. Manifold constraints · p. 8
  • Spectral-similarity graphs use nearest neighbors and heat-kernel affinities; related approaches include L1-based and Hessian graphs.The heat-kernel weight is larger for similar pixels; the Hessian graph is described as more stable than the Laplacian graph and as capturing nonlinear-mapping relationships.Source: D. Manifold constraints · p. 8
  • Spectral-spatial graphs combine spatial distance and spectral similarity to preserve edges by avoiding links across dissimilar pixels.Reviewed alternatives include bilateral-filter weights and hypergraphs that model high-order relations among multiple pixels.Source: D. Manifold constraints · pp. 8–9
  • Region-based methods distinguish consistent spectra within regions from differing structures across regions, with separate regularizers for homogeneous and detail regions.R-NMF explores consistent within-region distributions; another approach assigns manifold and L1/2 regularization to homogeneous regions and L1 to detail regions.Source: D. Manifold constraints · p. 9
  • Other learned graph weights use local linear embedding or subspace clustering to represent latent HSI structure.The subspace-clustering method learns a structure matrix by minimizing reconstruction error and retains the largest k values in each column.Source: D. Manifold constraints · p. 9

E. Low-rank constraints

Spatial correlation in hyperspectral images can be represented as low-rank abundance structure. The reviewed methods use local low-rank constraints and spatial priors to capture that structure.

E. Low-rank constraints

  • High spatial correlation in HSIs can be represented as low rank in abundance matrices.Source: E. Low-rank constraints · p. 9
  • DSPLR-NMF combines local low-rank and sparse structures in abundance matrices across non-overlapping subimages.The local low-rank constraint is applied to each subimage’s abundance matrix.Source: E. Low-rank constraints · p. 9
  • Superpixel-based low-rank penalties incorporate spatial priors by constraining superpixels generated from the HSI with SLIC.Source: E. Low-rank constraints · p. 9
  • Low-rank constraints can also construct a self-representation matrix for learning subspace structure.Source: E. Low-rank constraints · p. 9

F. Abundance separation constraints

Spatial constraints use abundance relationships and spatial context to improve unmixing, though local correlation can oversmooth differences between pixels. Weighted NMF addresses imbalanced pixel representation, while other constraints model global spatial structure or adapt weights.

F. Abundance separation constraints

  • Spatial correlation can improve unmixing and noise robustness, but local windows may oversmooth dissimilar pixels; separation constraints target differences between endmembers or within a material class.The reviewed approaches use KL-divergence-based abundance separation, same-material separation, or label-informed dissimilarity regularization.Source: F. Abundance separation constraints · p. 9

G. Other spatial constraints

  • Subspace, substance-dependence, and spectral-spatial joint constraints represent global spatial structure, while clustering and label information support alternative spatial regularizers.S2-NMF combines global spatial information with local spectral information.Source: G. Other spatial constraints · p. 9

A. Weighted NMF

  • CW-NMF addresses imbalanced pixel counts across endmembers by using clustering-based weights that increase minority-pixel influence and reduce majority-pixel influence.The review also reports weighted NMF as effective for improving NMF robustness.Source: A. Weighted NMF · pp. 9–10
  • Self-paced NMF uses weighted least-squares losses and a model-age-dependent function to learn weights adaptively, with extensions for pixel and element weighting.Source: A. Weighted NMF · p. 10

B. Projection-based NMF

The reviewed NMF variants alter endmember representation or the loss to address artificial endmembers and noise. Projection-based and convex approaches constrain how endmembers relate to spectral libraries or observed pixels, while robust models target varied noise and outliers.

B. Projection-based NMF

  • Projection-based NMF projects spectral-library spectra into a subspace to form endmembers related to the HSI, with subspace dimension controlling the number of endmembers.The projection is defined by A = UV, where U is the library and V the transformation matrix.Source: B. Projection-based NMF · p. 10

C. Convex NMF

  • Convex NMF constrains each extracted endmember to be a nonnegative linear combination of observed pixels, avoiding artificial endmembers in the solution space.A = XΞ expresses this relationship.Source: C. Convex NMF · p. 10

D. Robust NMF

  • Least-squares NMF is sensitive to the varied noise found in HSIs, motivating robust losses and regularizers to improve unmixing.Source: D. Robust NMF · p. 10
  • An L2,1 residual regularizer models outliers and nonlinear effects, while L1,2-based modeling explicitly addresses sparse band noise and supports robustness to band noise by rows.The review also describes L2,1 methods for handling noisy pixels by columns.Source: D. Robust NMF · p. 10
  • Correntropy-based methods target noisy bands, pixels, or elements, and the review reports their effectiveness for non-Gaussian and impulsive noise.CSsRS-NMF adaptively weights noisy pixels; CENMF targets noisy bands.Source: D. Robust NMF · pp. 10–11
  • General robust losses downplay large noise and include L2, Cauchy, and Welsch losses; truncated Cauchy loss cuts off large errors to suppress outliers.The review reports that truncated Cauchy loss can suppress outliers effectively.Source: D. Robust NMF · pp. 10–11

V. GENERALIZED NMF

Generalized NMF extends the linear model to represent known and unknown endmembers or second-order light scattering. Kernelized NMF uses feature mappings for nonlinearities, with online and incremental variants addressing computational demands.

A. Linear-quadratic NMF

  • Linear-quadratic NMF represents the image as linear and bilinear components, supporting either known and unknown endmembers or second-order light-scattering interactions.The corresponding matrices represent endmembers and abundances, or endmembers and interaction abundances, depending on the formulation.Source: A. Linear-quadratic NMF · p. 11

B. Kernelized NMF

  • Constrained kernel NMF maps observed pixels and endmembers into a feature space where the data are linearly separable, enabling nonlinear unmixing without estimating a nonlinear mixture model.The mapping is denoted by φ, and a Gaussian kernel is generally used.Source: B. Kernelized NMF · p. 11
  • Kernel-based methods face computational burden, so online and incremental KNMF variants use SGD, mini-batches, averaged SGD, or incremental updates to address efficiency and accuracy.Source: B. Kernelized NMF · p. 11

C. Transform-domain based NMF

Transform-domain methods use wavelets or curvelets to represent hyperspectral data compactly and impose priors in the transformed domain. Tensor factorization preserves intrinsic structure and can link tensor decomposition to the linear mixing model, while spatial-information constraints and other extensions address its limitations.

C. Transform-domain based NMF

  • Wavelet-domain NMF represents hyperspectral data compactly and integrates volume, spatial-smoothness, and sparsity priors to address ill-posedness.Curvelet transforms can impose sparse abundance regularization to enhance sparsity and diversity.Source: C. Transform-domain based NMF · p. 11

D. Nonnegative tensor factorization (NTF)

  • Nonnegative tensor factorization preserves intrinsic HSI structure, but an early CPD unmixing method lacked an explicit link to the linear mixing model.The later MV-NTF method uses block term decomposition to factor the HSI tensor into component tensors.Source: D. Nonnegative tensor factorization… · p. 11
  • MV-NTF links tensor factorization with the linear mixing model by representing each component with an endmember and low-rank abundance factors.Its model also includes a noise term.Source: D. Nonnegative tensor factorization… · pp. 11–12
  • MV-NTF's strict rank constraint leaves local spatial information underused, motivating graph and total-variation regularizers and coupling with pixel-level NMF.Other extensions add sparsity, minimum-volume, and nonlinearity constraints or low-rank tensor regularization.Source: D. Nonnegative tensor factorization… · p. 12

E. Multilayer/Deep extensions

Multilayer and deep NMF extensions seek hierarchical feature extraction while retaining interpretability, using different factorization architectures and training strategies. Their practical development still faces choices about model parameters, loss functions, optimization, and initialization.

E. Multilayer/Deep extensions

  • MLNMF hierarchically factorizes the coefficient matrix across layers, producing endmembers from the product of basis matrices and abundances from the final coefficient matrix.The architecture repeatedly factorizes the preceding layer's coefficient matrix.Source: E. Multilayer/Deep extensions · p. 12
  • Constrained multilayer variants add sparsity and geometric structure, while CMLNMF instead factorizes the basis matrix with minimum-volume and sparsity constraints.Further variants incorporate spectral-spatial total variation, graph, or homogeneous-region regularizers.Source: E. Multilayer/Deep extensions · p. 12
  • A deep NMF model uses layerwise pretraining followed by fine-tuning to reduce total reconstruction error, addressing layerwise objectives that do not reduce that total error.A sparsity-constrained deep NMF variant was also proposed for unmixing.Source: E. Multilayer/Deep extensions · p. 12
  • Deep extensions also include orthogonality-constrained NMF, multilayer tensor decompositions, and deep unfolding into interpretable neural-network-like architectures.The cited deep unfolding approach is described as having strong learning power and little computational cost.Source: E. Multilayer/Deep extensions · pp. 12–13
  • Deep convex NMF factorizes through data-point weights, while deep nsNMF inserts a smoothing matrix at each layer.DANMF uses encoder and decoder components to learn hierarchical mappings in community detection.Source: E. Multilayer/Deep extensions · pp. 12–13
  • Determining inner ranks, layer count, loss function, efficient optimization algorithms, and initial conditions remains challenging for multilayer NMF.These extensions combine interpretability with hierarchical feature extraction.Source: E. Multilayer/Deep extensions · p. 13

VI. EXPERIMENTAL RESULTS AND DISCUSSION

Across three real hyperspectral datasets, the experiments compare seven NMF-based methods using spectral-angle results and visual comparisons of signatures and abundance maps. The reported findings highlight benefits from spatial information, sparse-noise modeling, and multilayer/deep architectures, alongside differences in runtime.

VI. EXPERIMENTAL RESULTS AND DISCUSSION

  • Experimental results were averaged over ten independent runs, and tests used three real hyperspectral scenes: Cuprite, Samson, and Jasper Ridge.Cuprite has 188 bands and 12 minerals; Samson has 156 bands and three endmembers; Jasper Ridge has 198 bands and four endmembers.Source: §VI · pp. 13–14
  • Across Tables II–IV, all methods achieve satisfactory mean SAD for most materials; SGSNMF, TV-RSNMF, and MV-NTF-TV have lower mean SAD than L1/2-NMF, and SSRDMF outperforms MLNMF.SGSNMF often provides the best standard-deviation results, with its endmember initialization based on segmentation and region-based VCA.Source: §VI · p. 16
  • Across Cuprite, Samson, and Jasper Ridge, estimated signatures are generally correlated with reference spectra, while the abundance maps are compared with scene imagery because ground-truth abundance maps are unavailable.Samson maps correlate with geological maps; on Jasper Ridge, Road estimates are undesirable for all methods, possibly because few Road pixels lead VCA to miss the signature during initialization.Source: Figure 5 · p. 15, Figure 6 · p. 15, §VI · pp. 16–17
  • The abundance-map comparisons show sparse maps from L1/2-NMF, noise robustness from L1/2-RNMF, and local spatial and global spectral-spatial structure in MV-NTF-TV results.The paper also reports that MLNMF and SSRDMF abundance maps align with ground truth; sparse regularization, spectral-spatial information, and multilayer/deep architectures are described as beneficial.Source: §VI · pp. 17–18
  • On Jasper Ridge, average runtimes were 4.99 s for L1/2-NMF, 60.81 s for SGSNMF, 25.19 s for TV-RSNMF, 43.31 s for L1/2-RNMF, 107.50 s for MV-NTF-TV, 19.38 s for MLNMF, and 136.54 s for SSRDMF.L1/2-NMF was fastest; the paper attributes longer times to spatial-structure learning, explicit sparse-noise modeling, or complex factorization, depending on the method.Source: §VI · p. 18

VII. DISCUSSION AND FUTURE DIRECTIONS

The review describes constrained, structured, and generalized NMF as ways to use HSI information, while noting practical drawbacks including parameter tuning, computation time, and initialization. It identifies future work in joint spectral-spatial modeling, endmember variability, nonlinear mixtures, robustness, and faster processing.

VII. DISCUSSION AND FUTURE DIRECTIONS

  • Constrained NMF can improve endmember and abundance estimates with spectral and spatial constraints, while structured and generalized NMF accommodate richer structures and characteristics.The review also notes extensive parameter tuning for constrained NMF, time costs for generalized NMF, and reliance on proper guidance or initialization across NMF-based methods.Source: §VII · p. 18
  • Joint use of spatial and spectral information is identified as a direction for more reliable unmixing.The review says spectral-spatial joint methods have shown considerable potential.Source: §VII · p. 18
  • Further NMF work should describe endmember variability more fully, including through mathematical treatments such as a 4D endmember tensor [59].Source: §VII · p. 18
  • Most current NMF algorithms rely on the LMM, while nonlinear spectral mixtures from multipath scattering motivate further nonlinear unmixing research, –.Source: §VII · p. 18
  • Robustness research should address sensitivity to initialization choices and tunable parameters, in addition to different noise types.Source: §VII · p. 18
  • Computationally demanding graph-regularized, tensor, and multilayer or deep methods motivate faster alternatives for real- or near-real-time HSI processing.Source: §VII · p. 18
Loading 2205.09933v1…