Source-linked AI summary
Nonlinear unmixing of hyperspectral images: models and algorithms
Nicolas Dobigeon, Jean-Yves Tourneret, Cédric Richard, José C. M. Bermudez, Stephen McLaughlin, Alfred O. Hero
TL;DR
The paper addresses the limitations of the linear mixing model for hyperspectral unmixing when nonlinear interactions occur. It surveys recent nonlinear models and algorithms, including physics-based and machine-learning-inspired approaches. The review highlights nonlinear-subspace evidence and identifies spatial interactions and computational cost as continuing challenges.
Problem
The linear mixing model may be invalid for hyperspectral data affected by multi-scattering effects or intimate interactions, motivating nonlinear models.
Method
The paper provides an overview of recent advances in nonlinear unmixing modeling, covering physics-based, parametric, and nonparametric approaches.
Results
The investigated dataset is better represented in a nonlinear subspace, with ARE reduced to 7.9 × 10−3 using GPLVM.
Takeaways & Limitations
Detection can separate linearly and nonlinearly mixed pixels, enabling linear pixels to benefit from established linear-unmixing methods while focusing nonlinear methods on the remainder.
Takeaways & Limitations
The reviewed physics-based models do not account for spatial interactions among materials present in the image.
Abstract
from arXiv · showhide
When considering the problem of unmixing hyperspectral images, most of the literature in the geoscience and image processing areas relies on the widely used linear mixing model (LMM). However, the LMM may be not valid and other nonlinear models need to be considered, for instance, when there are multi-scattering effects or intimate interactions. Consequently, over the last few years, several significant contributions have been proposed to overcome the limitations inherent in the LMM. In this paper, we present an overview of recent advances in nonlinear unmixing modeling.
I. MOTIVATION FOR NONLINEAR MODELS
Hyperspectral unmixing commonly relies on the LMM, but nonlinear interactions can make it unsuitable and require more complex models and algorithms. The paper reviews recent advances addressing this challenge.
- Problem setup: Spectral unmixing estimates endmembers and abundances by inverting a mixing model that describes how endmembers form measured spectra.The model’s abundances parameterize the formation process.
- Why nonlinear models are needed: The LMM is useful when it approximates actual mixing, but practical situations involving nonlinear effects violate its assumptions.The LMM assumes macroscopic mixing and photons interacting with only one material.
- Modeling challenge: Physics-based nonlinear models can capture important effects, but their nonlinear and integral formulations hinder practical unmixing implementations.The paper notes that such models remain difficult to apply automatically to hyperspectral images.
- Modeling challenge: Nonlinear unmixing remains challenging because the problem is ill-posed, transformations may be partly or totally unknown, and effective algorithms require innovative approaches.The paper discusses both supervised and unsupervised algorithmic needs.
- Paper scope: The paper provides an updated review of nonlinear unmixing techniques introduced during the decade after an earlier comprehensive review.It focuses on advances that followed a period when few algorithmic solutions were available.
II. NON LINEAR MODELS
Linear mixing is appropriate under macroscopic, single-material photon interactions; when either assumption fails, nonlinear effects may arise and motivate distinct nonlinear model families.
- Linear mixing assumptions: Linear mixtures assume a macroscopic mixing scale and photons interacting with only one material before reaching the sensor.Checkerboard-type scenes illustrate the single-material interaction assumption.
- Nonlinear effects: When either linear-mixture assumption does not hold, different nonlinear effects may occur.The paper introduces two families of nonlinear models after identifying these assumption failures.
A. Intimate mixtures
Intimate mixtures arise at microscopic scales where photon paths interact with multiple constituents, requiring physically motivated models that relate spectra to mixture properties and abundances. Their inversion is difficult because models depend on experiment-specific parameters and may require recovering unknown material signatures.
- Physical setting: Intimate mixtures occur when constituent spatial scales are smaller than the photon path length, producing microscopic interactions.Such mixtures are associated with relative mass fractions.
- Physics-based models: Radiative-transfer frameworks, especially Hapke-based approaches, model intimate-mixture reflectances using mass fractions and particle or scattering properties.Relevant parameters include constituent density, particle size, and single-scattering albedo.
- Practical limitations: Approximations such as discrete-dipole and Shkuratov models remain strongly dependent on parameters inherent to the experiment.They require knowledge of the sensor’s geometric positioning relative to the observed sample.
- Practical limitations: This dependence makes estimating mass fractions difficult, especially in unsupervised settings where material signatures must also be recovered.The inversion is described as very difficult to implement.
- Choice of components: The appropriate mixture scale depends on the desired definition of pure components and the spatial or spectral resolution of the application.Different analyses may target whole materials or their constituent subcomponents.
- Choice of components: Defining finer components requires mixtures that are not macroscopic, while sensor-resolution-scale instances can serve as practical pure components.A spatially homogeneous sand patch is given as an example of one pure component.
B. Bilinear models
Bilinear models extend linear mixing by adding interaction terms between material spectra, with coefficients that vary by model and may preserve LMM-like geometry under weak nonlinearities.
- General bilinear formulation: Bilinear models augment the linear contribution with pairwise products mi ⊙mj that model nonlinear interactions between materials.The coefficient βi,j,p controls the interaction amount for components mi and mj in pixel p.
- Nascimento model: The Nascimento model represents products mi ⊙mj as virtual endmembers with corresponding abundances, making it interpretable as an expanded linear mixture.It reduces to the LMM when the virtual-endmember abundances are zero.
- Fan model: The Fan model sets βi,j,p = ai,paj,p, linking nonlinear interactions directly to the abundances of the interacting materials.This relation implies no interaction involving an endmember absent from the pixel.
- Geometrical implications: Synthetic mixtures showed that FM and GBM preserve pure-component spectra as cluster vertices, unlike NM, under the illustrated conditions.Largest-volume simplex endmember extraction may therefore remain valid for FM and GBM when nonlinear interactions are weak.
- Model extensions: Bilinear models omit within-component products mi ⊙mi, whereas the linear-quadratic model includes these quadratic terms.The LQM was derived using radiative-transfer modeling for a canyon-like urban scene.
C. Other approximating physics-based models
Other physics-based approximations broaden nonlinear modeling through macroscopic, microscopic, and post-nonlinear formulations, while geometrical methods can remain applicable when nonlinear mixtures preserve extremal endmembers.
- Dual model: A dual model was introduced to describe both macroscopic and microscopic mixtures.Its microscopic component models intimate mixtures using the average single-scattering albedo mapped into the reflective domain.
- Geometrical approaches: Geometrical algorithms search for endmembers as extremal points rather than explicitly assuming linear pixel mixtures.With pure pixels, this property can support their use for nonlinear mixtures such as FM and GBM.
- Polynomial post-nonlinear model: The polynomial post-nonlinear model applies a pixel-specific nonlinear transformation gp to a linear mixture of endmember spectra.The transformation is a second-order polynomial parameterized by the single nonlinearity parameter bp.
- Polynomial post-nonlinear model: PPNM reduces to the standard LMM when bp = 0 and includes both bilinear cross-products and quadratic self-products.It has been shown sufficiently flexible to describe most bilinear models introduced in the section.
D. Limitations a pixel-wise nonlinear SU
Pixel-wise nonlinear unmixing simplifies inversion by modeling interactions within each pixel, but excludes neighborhood effects and remains challenging when endmembers and abundances must both be inferred.
- Model limitation: The reviewed physics-based models ignore spatial interactions from materials in neighboring pixels.Their bilinear interactions are restricted to components present in the targeted pixel.
- Model limitation: This locality assumption enables abundance and nonlinear-coefficient estimation independently for each pixel.The paper characterizes the assumption as a strong simplification; adjacency effects have been addressed separately in an unmixing context.
- Algorithmic settings: Nonlinear unmixing algorithms may be physics-based or rely on mild assumptions, and may operate in supervised or unsupervised settings.Supervised methods estimate abundances when endmembers are known, whereas unsupervised methods jointly estimate endmembers and abundances.
- Algorithmic settings: Unsupervised unmixing is more challenging because jointly estimating endmembers and abundances requires solving a blind source separation problem.Supervised methods require that pure spectral signatures have been identified previously.
A. Model-based parametric nonlinear unmixing algorithms
Model-based nonlinear unmixing formulates abundance and nonlinearity estimation under specified physical models, using constrained optimization, Bayesian inference, or geometrical transformations.
- Radiative-transfer models: Radiative-transfer approaches transform measured reflectance into single-scattering-albedo mixtures so standard linear unmixing can estimate material mass fractions.Neural networks can instead learn the nonlinear inversion from reflectance to single-scattering albedo.
- Supervised estimation: Supervised nonlinear unmixing estimates abundance coefficients and nonlinearity parameters by minimizing reconstruction error under a known endmember matrix.The resulting constrained nonlinear regression is difficult because the criterion is nonlinear and the parameters must satisfy mixing-model constraints.
- Optimization strategies: Optimization strategies include Taylor-linearized FCLS, constrained gradient descent, and Bayesian Monte Carlo methods with priors encoding parameter constraints.These approaches target bilinear and post-nonlinear models, while Bayesian formulations incorporate constraints through prior distributions.
- Joint and unsupervised estimation: When endmembers must also be identified, iterative SPICE, NMF, unsupervised Bayesian PPNM, and manifold-based geodesic methods extend model-based unmixing.Geometrical methods compute distances or maximum-volume simplices on nonlinear manifolds induced by models such as the GBM.
- Scope and challenges: Model-based methods depend on choosing an appropriate nonlinear model and become more demanding when endmembers, abundances, and nonlinearity parameters are all unknown.The survey describes supervised and unsupervised algorithms for this progressively harder setting.
B. Model-free nonlinear unmixing algorithms
Model-free methods avoid committing to a fixed nonlinear mixing law by learning flexible spectral relationships through kernels, Gaussian processes, or manifold-based representations.
- Kernel methods: Kernel methods replace spectral inner products with kernel functions, modeling nonlinear distortions independently of material interactions.Such distortions can improve detectability or separability, but they have limited physical interest for mixtures governed by material interactions.
- Interaction-aware kernels: Interaction-aware kernel methods incorporate nonlinear relationships between known endmembers through a functional optimization framework.The K-HYPE algorithm combines an abundance-parameterized LMM with a nonparametric RKHS term and can mimic the PPNM with degree-2 polynomial functions.
- Kernel selection: The RKHS must be selected carefully: an unsuitable functional space can cause the interaction-aware strategy to fail, whereas an appropriate kernel can capture broad nonlinear relationships.The kernel controls the balance between fitting and regularity in the learned function.
- Gaussian-process and manifold methods: GPLVM-based unmixing learns a smooth mapping from fractional abundances to observed pixels while estimating kernel parameters, endmembers, and abundances without a known mixing model.Only the number of endmembers is assumed known in the described unsupervised formulation.
- Experimental comparison: Prior knowledge of the true mixing model improves abundance estimation; with model uncertainty, PPNM is flexible, while model-free K-HYPE is an alternative.Using an inappropriate model-based algorithm can produce poor unmixing results.
IV. DETECTING NONLINEAR MIXTURES
Nonlinear detection can improve unmixing accuracy by identifying when nonlinear effects affect the observations, while avoiding unnecessary computational cost on linearly mixed pixels.
- Motivation: Nonlinear mixing effects can yield more accurate endmember and abundance identification than linear modeling.The benefit is motivated by scenes involving nonlinear interactions or scattering effects.
- Motivation: Nonlinear unmixing generally has higher computational complexity than LMM-based approaches.This cost motivates screening pixels before applying nonlinear algorithms.
- Detection-guided unmixing: A detection stage can route linearly mixed pixels to linear unmixing and reserve nonlinear methods for pixels where they are necessary.The survey presents nonlinear-mixing detection as a preprocessing step for unmixing.
A. Detection using a polynomial post-nonlinear model (PPNM)
The PPNM-based detector tests whether each pixel follows an LMM or a nonlinear post-nonlinear model by estimating its pixel-specific nonlinearity parameter.
- Hypothesis test: The detector compares H0, in which a pixel follows the LMM, with H1, in which it follows the PPNM.The PPNM uses a polynomial nonlinearity and can represent different nonlinear relationships between endmembers and observations.
- Parameter estimation: Under the PPNM, the pixel-specific parameter bp characterizes nonlinearity and is estimated jointly with abundances and noise variance.The maximum-likelihood estimator of bp is used to approximate the test statistic's distribution.
- Decision rule: The generalized likelihood ratio test accepts the nonlinear hypothesis when the statistic T exceeds threshold η and the linear hypothesis when it falls below it.The threshold can be related explicitly to false-alarm and detection probabilities because T is approximately normally distributed under both hypotheses.
- Assumptions: The detection strategy assumes prior knowledge of the variances under the competing hypotheses.A modified test strategy has been proposed to address this practical requirement.
- Illustration: The method is illustrated by detecting linear and nonlinear pixels generated under the LMM, FM, GBM, and PPNM.The figure distinguishes detected linear pixels with red crosses from nonlinear pixels with blue dots.
B. Robust model-free detection
The detector tests whether observed pixels follow the LMM or a general nonlinear model by measuring their distance from the LMM hyperplane. Its threshold can be calibrated statistically, while unknown noise variance requires estimation.
- Limitations: The detector assumes a specific nonlinear mixing model, so actual mixtures that do not obey that model may challenge the alternative hypothesis.The nonlinear residual may depend on the endmember matrix and abundance vector, and analogous models can incorporate group-sparse constraints.
- Detection formulation: The procedure frames nonlinear-mixture detection as binary testing between the LMM and a general nonlinear model.Under the LMM, observations lie in the hyperplane defined by the endmembers; under the nonlinear alternative, a residual component moves them away from it.
- Detection formulation: The test statistic uses the squared Euclidean distance between an observed pixel and the LMM hyperplane.The distance distinguishes the hypotheses because the LMM mean lies in the hyperplane whereas the nonlinear-model mean includes a component outside it.
- Statistical test: When noise variance is unknown, replacing σ2 with an estimate produces T*, whose performance approaches T as variance estimation improves.The variance estimate can be obtained from the smallest eigenvalues of a sample covariance matrix, but its accuracy depends on how many eigenvalues are used.
V. CONCLUSIONS AND OPEN CHALLENGES
The paper reviews nonlinear unmixing approaches and identifies remaining challenges involving physical-model integration, heterogeneous media, spatial information, computational cost, and unknown model complexity.
- V. CONCLUSIONS AND OPEN CHALLENGES: Nonlinear unmixing methods include parametric algorithms based on physical models and nonparametric techniques that avoid rigid nonlinear assumptions.These approaches address interactions such as multiple scattering and intimate mixtures.
- V. CONCLUSIONS AND OPEN CHALLENGES: Nonlinear modeling can increase computational complexity and degrade performance on large hyperspectral images, while the number of endmembers is usually unknown.Manifold learning and dimensionality estimation are identified as promising approaches for handling this complexity.
- V. CONCLUSIONS AND OPEN CHALLENGES: Detection strategies can identify regions where nonlinear models may outperform linear ones, allowing linear pixels to use established linear-unmixing methods.Statistical outlier approaches are another option when only a few nonlinear subregions are present.
- V. CONCLUSIONS AND OPEN CHALLENGES: Integrating algorithmic approaches with physical models could improve nonlinear unmixing by accounting for scattering, dispersion, and beam-interaction depth.Physical models can guide the selection of simplified mathematical and statistical models.
- V. CONCLUSIONS AND OPEN CHALLENGES: Heterogeneous media remain a challenge because regions may combine linear, weakly nonlinear, and strongly nonlinear pixels.The paper notes that in situ measurements coupled with simulation tools have produced preliminary results.
- V. CONCLUSIONS AND OPEN CHALLENGES: Flexible unsupervised nonlinear unmixing remains difficult, particularly when exploiting spatial information rather than relying on pixel-by-pixel analysis.This is framed as a major challenge in nonlinear blind source separation.