PaperlayerView exact revision on arXiv

Source-linked AI summary

Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review

Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard, Jocelyn Chanussot, Lucas Drumetz, Jean-Yves Tourneret, Alina Zare, Christian Jutten

arXiv:2001.07307v4eess.IVeess.SP

TL;DR

Traditional spectral unmixing assumes fixed endmember signatures, limiting abundance accuracy when atmospheric, illumination, or environmental conditions cause spectral variability. The paper reviews variability-aware unmixing and spectral-library strategies, finding improved abundance reconstruction over FCLS across evaluated methods while identifying theoretical and parameter-selection challenges.

  • Problem

    Fixed endmember signatures can limit abundance-estimation accuracy when spectral variability occurs, while reliable abundance ground truth for real images is difficult to collect.

  • Method

    The paper reviews variability-aware unmixing methods and spectral-library construction, augmentation, and pruning strategies.

  • Results

    All evaluated methods that modeled spectral variability achieved better abundance reconstruction than the FCLS baseline, with library-based methods performing very well in the reported scene.

  • Takeaways & Limitations

    Improving parameter-selection robustness and using metadata for informed adjustment could make spectral-unmixing methods more practical as out-of-the-box solutions.

  • Takeaways & Limitations

    A deeper theoretical understanding of endmember variability is still needed to identify conditions under which spectral unmixing can or cannot be solved exactly.

Abstract

from arXiv · show

The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout the whole unmixing process and compromises the quality of its results. Therefore, large efforts have been recently dedicated to mitigate the effects of spectral variability in SU. This resulted in the development of algorithms that incorporate different strategies to allow the EMs to vary within a hyperspectral image, using, for instance, sets of spectral signatures known a priori, Bayesian, parametric, or local EM models. Each of these approaches has different characteristics and underlying motivations. This paper presents a comprehensive literature review contextualizing both classic and recent approaches to solve this problem. We give a detailed evaluation of the sources of spectral variability and their effect in image spectra. Furthermore, we propose a new taxonomy that organizes existing works according to a practitioner's point of view, based on the necessary amount of supervision and on the computational cost they require. We also review methods used to construct spectral libraries (which are required by many SU techniques) based on the observed hyperspectral image, as well as algorithms for library augmentation and reduction. Finally, we conclude the paper with some discussions and an outline of possible future directions for the field.

I. INTRODUCTION

Hyperspectral pixels mix materials, and endmember spectra can vary across an image because of illumination, atmospheric, and environmental conditions. This variability undermines fixed-signature unmixing, motivating models and a practitioner-oriented taxonomy for selecting methods.

  • I. INTRODUCTION: Hyperspectral cameras sample hundreds of contiguous wavelength intervals, enabling precise material identification but requiring a spatial-resolution and signal-to-noise trade-off.Large target-to-sensor distances can produce low-spatial-resolution images.
  • I. INTRODUCTION: Spectral unmixing decomposes each mixed pixel into endmember signatures and fractional abundance proportions.The Linear Mixing Model represents pixels as combinations of endmembers plus additive noise, with nonnegative abundances summing to one.
  • I. INTRODUCTION: The traditional LMM assumes identical endmember signatures across pixels, simplifying computation but potentially reducing abundance accuracy when spectra vary.Spectral variability can cause proportion indeterminacy, erroneous abundances, and selection of too many endmembers.
  • I. INTRODUCTION: Spectral variability commonly arises from illumination, atmospheric, growing, and environmental conditions, and can affect signatures of the same material across an image.A variable endmember matrix M_n generalizes the fixed-matrix LMM for pixel-specific signatures.
  • I. INTRODUCTION: SU with spectral variability addresses abundance recovery and, in some methods, recovery of pixel-specific endmember signatures.All methods must address abundance effects, whereas estimating varying signatures adds difficulty and is not always attempted.
  • I. INTRODUCTION: Existing approaches use Bayesian, parametric, spatially localized, or library-based models with different computational costs, accuracy, and supervision requirements.The review organizes these alternatives in a practitioner-oriented taxonomy and also covers library extraction, pruning, and evaluation resources.

II. ORIGINS OF SPECTRAL VARIABILITY AND THEIR EFFECTS

Spectral variability arises from atmospheric, illumination, topographic, and intrinsic material effects, altering observed spectra and degrading unmixing. Atmospheric correction can reduce these effects, but its accuracy depends on difficult-to-obtain atmospheric parameters, especially for spatially variable aerosols.

  • Sources of spectral variability: Atmospheric, illumination, topographic, and intrinsic material conditions are major sources of spectral variability that affect endmember signatures and unmixing.Understanding these effects supports physically informed models of spectral variability.
  • A. Atmospheric effects: Atmospheric gases and aerosols absorb and scatter radiation, producing sensor radiances that differ from desired ground reflectance in wavelength-dependent ways.Gas absorption is strongly wavelength dependent, while aerosol absorption varies more smoothly across spectra.
  • A. Atmospheric effects: Physics-based atmospheric correction estimates ground reflectance from at-sensor reflectance using atmospheric parameters, including aerosol properties and per-pixel water vapor.Water vapor can be estimated with three-band ratios near strong absorption wavelengths before solving for ground reflectance.
  • Limitations of atmospheric correction: Physics-based models require precise atmospheric properties that are difficult to obtain because aerosol scattering and absorption vary spatially and temporally.Parameter inaccuracies introduce significant, spectrally non-uniform errors in retrieved surface reflectance.
  • Effects on image spectra: Correction errors can reach 1.7% in retrieved reflectance, 5% in NDVI, and 30% in biomass estimates when aerosol optical thickness varies spatially.Errors are non-uniform across bands, with gas or water absorption spikes and aerosol-related errors concentrated at 450 nm-750 nm.
  • Effects on unmixing: Atmospheric-parameter uncertainty can degrade abundance estimation by up to 30% under high scattering conditions, more severely than reflectance estimation.Water-vapor uncertainty had a stronger effect than aerosol optical-thickness uncertainty, while aerosol uncertainty strongly affected abundance maps for spectrally similar endmembers.

B. Illumination and topographic effects

Illumination, atmospheric, topographic, and intrinsic material differences produce distinct forms of spectral variability. Constant scaling can fit some illumination changes, whereas shadows and intrinsic variability often require more complex models.

  • Illumination variability mainly arises from changing terrain topography and occlusion of the light source, producing shaded areas.
  • 30 Pavia red-roof pixels had an average Pearson correlation of about 0.993 with their closest scaled mean spectrum.This supports constant scaling as a good model for illumination-driven variability in that example.
  • Shadow signatures depend on diffuse illumination, neighboring image areas, and cloud cover, so a single dark endmember is generally insufficient.
  • Topographic effects can alter soil, vegetation, and rock reflectance by changing the relative illumination and viewing angles.Hapke’s model represents reflectance as a function of incidence angles, single scattering albedo, and photometric parameters.
  • Intrinsic variability reflects material differences caused by environmental, compositional, biological, and physical factors, including vegetation conditions, soil moisture, and mineral impurities.Its wavelength-dependent behavior means simple scaling is usually insufficient for vegetation and other intrinsic variations.
  • Analytical endmember models constrain variability to a low-dimensional manifold, providing information that can reduce the ill-posedness of unsupervised spectral unmixing.

III. UNMIXING METHODS THAT USE SPECTRAL LIBRARIES

Spectral-library methods represent material variability using bundles of reference signatures and select or combine signatures during unmixing. Their families differ in supervision, computational complexity, interpretability, and abundance quality.

  • Library-based methods are conceptually simple and easy to interpret, but their unmixing quality depends strongly on the available spectral library.
  • Spectral libraries contain multiple signatures for each material, allowing each pixel’s endmember to be represented by an unknown bundle element.
  • Library representations may select one signature per material or use sparse or convex combinations of signatures for greater flexibility.
  • Library-based unmixing methods are grouped into MESMA, sparse unmixing, machine learning, and spectral-transformation formulations.
  • These method families differ in required user supervision, computational complexity, and abundance-estimation quality.

A. MESMA and variants for small spectral libraries

MESMA and related methods allow endmembers to vary per pixel by searching spectral-library combinations, but exhaustive optimization becomes costly and potentially ill-posed for large bundles. Variants reduce complexity, add flexibility, or quantify uncertainty with different trade-offs.

  • MESMA searches endmember and abundance combinations from spectral bundles to minimize each pixel’s reconstruction error.
  • MESMA’s computational cost scales with the product of the individual library sizes, making exhaustive unmixing unrealistic for large libraries.
  • Large bundles can make MESMA ill-posed because different material combinations may produce very similar reconstruction errors.
  • Approximate MESMA variants reduce complexity through early stopping, unconstrained least squares, angle minimization, or library pruning.
  • These approximations can negatively affect abundance reconstruction, making thresholds and tolerances practically important.
  • Flexible variants use sparse non-negative combinations, while fuzzy methods summarize abundance uncertainty through average, maximum, and minimum cover fractions.

C. Machine Learning Algorithms

Machine-learning approaches formulate unmixing as supervised or probabilistic regression from mixed spectra to abundances and can incorporate endmember variability through varied training signatures. Their flexibility is balanced by high computational demands, dependence on representative training data, and limited physical interpretability.

  • Machine-learning methods are flexible in principle, but may have high computational complexity or lack a clear physical motivation.
  • Unmixing quality depends on training-data representativeness, and many methods do not return an endmember spectrum for each pixel.
  • Supervised machine-learning SU learns mappings from mixed pixels to abundance fractions using known-proportion training data and models such as neural networks, random forests, and SVMs.
  • Training data can incorporate spectral variability by using multiple signatures for each endmember when generating synthetic mixtures.
  • Large spectral libraries usually produce extremely large training sets, leaving training computationally expensive despite acceleration strategies.
  • Extended SVMs and multi-task Gaussian processes use pure-pixel libraries or posterior abundance distributions to address variability with reduced joint variation in training data.

D. Spectral transformations

Spectral transformations reduce endmember variability by transforming observed spectra and spectral libraries before unmixing. Approaches range from simple user-defined preprocessing to automated library-based selection, weighting, and discriminative transformations.

  • Transformation framework: Spectral transformations apply affine operations to observed pixels and spectral-library elements to reduce endmember variability before spectral unmixing.The transformation uses Wλ and bn and is applied consistently to image spectra and library elements.
  • User-defined spectral transformations: User-defined transformations normalize illumination or brightness, emphasize spectral features, or use derivatives and wavelet representations.Examples include band subtraction, mean removal, convex-hull normalization, first- or second-order derivatives, and wavelets.
  • User-defined spectral transformations: Band selection restricts unmixing to wavelength intervals with little within-material variability, making the approach simple, interpretable, and potentially dimension-reducing.Many such methods depend on expert knowledge of the application.
  • Library-based spectral transformations: Library-based transformations automate spectral-band selection, weighting, or more general transformations using information from spectral libraries.These methods avoid delegating transformation choices entirely to the user.
  • Band selection: Stable-zone methods select bands using residual variance or instability measures based on intra-class and inter-class endmember variability.Extensions also reduce correlation between signatures of different endmember classes, while class-subset variants support MESMA.
  • General spectral transformations: Fisher-discriminant transformations seek a feature space with low within-class variance and high between-class separability, but depend on representative covariance estimates.FDA has been applied with a priori or image-derived libraries and improved MESMA for urban surfaces.

IV. UNMIXING METHODS THAT ESTIMATE THE ENDMEMBERS FROM THE IMAGE

Blind unmixing methods estimate endmembers and abundances directly from the hyperspectral image without requiring prior spectral libraries. The reviewed families trade supervision, modeling flexibility, computational cost, and estimation quality differently.

  • Overview: Local, parametric, EM-model-free, and Bayesian methods estimate endmembers and abundances directly from the image without prior spectral libraries.Their main trade-offs concern user supervision, computational complexity, and abundance-estimation quality.
  • A. Local unmixing methods: Local methods are conceptually simple and computationally efficient but require supervision, while local endmember extraction and aggregation into global results remain difficult.Determining the number of local endmembers is also a central issue.
  • A. Local unmixing methods: Local unmixing assumes endmember signatures are constant within small regions, where spectral variability is often negligible.The workflow segments the image, extracts local endmembers, performs local unmixing, and combines results globally.
  • B. Parametric models: Parametric models represent each endmember as a function of a reference matrix and pixel-dependent parameters, recovering those parameters jointly with abundances.Their low-dimensional parameterization supports flexible, physically motivated models and incorporation of prior information.
  • B. Parametric models: Parametric models can be difficult to design and estimate because flexible parsimonious functions are challenging to construct and optimization may be non-convex or initialization-sensitive.Deep generative models learn the parametric function from image-extracted pure pixels, but still recover latent representations and abundances.

C. EM-model-free methods

EM-model-free methods mitigate spectral variability without imposing a specific endmember model. They modify reconstruction metrics, add structured residuals, learn projections, or exploit low-rank tensor structure, often at increased optimization difficulty.

  • Overview: EM-model-free methods address spectral variability without assuming a specific model for endmember signatures.They use modifications to the unmixing cost function or flexible representations instead of explicit parametric endmember models.
  • Robust metrics: Robust reconstruction metrics can reduce sensitivity to spectral-amplitude variations, but they produce nonlinear and possibly non-convex optimization problems.Projected gradient descent has been used with the spectral angle mapper metric.
  • Residual models: Structured additive residual models capture spectral variability and other unmodeled effects using smooth or low-coherence bases alongside the linear mixing model.One approach uses discrete-cosine-transform columns for spectrally smooth residuals; another combines per-pixel scaling with a non-parametric additive term.
  • Subspace methods: Projection-based methods jointly estimate a low-dimensional spectral subspace and abundances by penalizing the projection operator's nuclear norm.Unmixing minimizes reconstruction error in the projected space.
  • Tensor methods: Tensor-factorization methods model pixel-dependent endmember and abundance tensors as approximately low-rank, yielding parsimonious non-convex formulations without application-specific parametric representations.The formulation uses non-negative tensor factorization.

D. Bayesian Methods

Bayesian methods represent endmembers as random vectors from statistical distributions and estimate abundances and endmembers statistically. They can require little user supervision, but distributional assumptions, hyperparameters, and computational cost remain important constraints.

  • Overview: Bayesian approaches can require very little user supervision, although hierarchical models estimate distribution parameters jointly at higher computational cost.Some methods require distribution parameters or hyperparameters to be set a priori.
  • Overview: Bayesian methods model endmember spectra with statistical distributions and formulate unmixing as estimation of abundances and endmember realizations.Distribution parameters encode the assumed statistical model for each endmember and pixel.
  • The Normal Compositional Model: The Normal Compositional Model uses multivariate Gaussian endmember distributions with mean vectors and covariance matrices because of its mathematical tractability.These parameters may be estimated from image-derived pure pixels or homogeneous image regions.
  • The Normal Compositional Model: Expectation-maximization approaches can converge to poor local minima, motivating more robust but costly Markov chain Monte Carlo methods.The convergence issue limited the large-scale applicability of expectation-maximization-based methods.
  • Bayesian extensions: Bayesian extensions model shared atmospheric effects, spectral smoothness, spatial homogeneity, abundance sparsity, or piecewise-convex mixtures through additional statistical structure.Examples include correlated endmembers, sparse Dirichlet priors, spatial classes, and mixtures of Gaussian distributions.
  • Other endmember distributions: Alternative distributions address limitations of Gaussian models, including bounded or skewed reflectances with Beta distributions and multimodal spectra with Gaussian mixtures.These alternatives can require approximate or staged parameter-estimation procedures.

V. SPECTRAL LIBRARIES

Spectral libraries support variability-aware spectral unmixing, but their construction is difficult when scene conditions differ from available measurements. The review organizes image-based and model-based strategies that reduce supervision or synthesize representative endmember signatures.

  • V. SPECTRAL LIBRARIES: Library quality strongly affects spectral-unmixing performance, while laboratory or in situ measurements may be costly, unavailable, or mismatched to scene conditions.Existing libraries acquired under different conditions can introduce errors into the unmixing process.
  • V. SPECTRAL LIBRARIES: The review identifies three directions: extracting and clustering pure pixels, generating libraries with radiative-transfer models, and interpolating signatures using spatial locations.These approaches correspond to image-based extraction, physics-based synthesis, and spatial interpolation of image-derived signatures.
  • Image based library construction: Image-based extraction produces signatures under the image’s conditions and can use expert knowledge, but depends on pure pixels and may include mixed pixels or difficult material-class assignments.The image should not be too small, and clustering extracted signatures into correct material classes remains challenging.
  • Image based library construction: Supervised image-based methods select pixels using manually chosen endmembers, expert screening, spectral distances, or iterative updates of reference signatures.Later approaches also differentiate similar materials across spectral scales, wavelet coefficients, sparse representations, and auxiliary classification information.
  • V. SPECTRAL LIBRARIES: Physics-based library generation can represent endmember variability mathematically, but it requires accurate knowledge of the physical reflectance process.Data augmentation offers an alternative strategy for synthesizing additional signatures from small pre-existing libraries.

B. Library pruning techniques

Library pruning removes signatures that are redundant, irrelevant, or poorly matched to the observed image before unmixing. The reviewed techniques trade computational savings against the difficulty of identifying scene-representative signatures.

  • B. Library pruning techniques: Large spectral libraries increase computational complexity and can make unmixing more ill-posed, motivating library reduction, endmember selection, and same-class pruning.These groups target different forms of redundancy or mismatch before or during spectral unmixing.
  • Library reduction techniques: Library reduction removes redundant signatures without using the observed image, generally improving computational complexity but not necessarily unmixing quality.Representative subsets may be chosen using squared error, average spectral angle, or count-based selection metrics.
  • Endmember selection methods: Endmember selection removes entire material classes absent from each pixel using classification maps or complementary data such as road-network density and LIDAR height.The approach leverages the observation that pixels usually contain only a small number of materials.
  • Pruning libraries within the same class: Identifying signatures acquired under conditions different from the observed image is difficult, limiting reliable scene-specific pruning.This difficulty is explicitly identified as a limitation of same-class endmember pruning.
  • Pruning libraries within the same class: Same-class pruning retains signatures representative of the observed image, using spectral distances, wavelet-domain comparisons, eigenvector distances, or iterative abundance-based removal.Distance-based pruning may discard relevant signatures when many pixels are mixed, while iterative pruning depends on early unmixing accuracy.

VI. EXPERIMENTAL EVALUATION

The review evaluates unmixing methods with spectral variability through synthetic-data generation, software resources, and a tutorial-style simulation. Synthetic data are emphasized because real images rarely provide reliable abundance ground truth.

  • VI. EXPERIMENTAL EVALUATION: The evaluation section discusses synthetic-data generation, practitioner-oriented software packages, and a tutorial simulation using reviewed unmixing techniques.The tutorial follows method selection with the decision tree in Fig. 2.

A. Generating Synthetic Data

Synthetic hyperspectral data are generated by constructing abundances, varying endmember signatures, and applying a mixing model. The review uses physical and simplified models to simulate variability from material properties, viewing geometry, and atmospheric compensation.

  • A. Generating Synthetic Data: Synthetic-data generation has three steps: generate abundances, generate per-pixel endmember signatures, and apply a mixing model to create image pixels.The reviewed example uses the linear mixing model and can add noise for a desired signal-to-noise ratio.
  • A. Generating Synthetic Data: Abundances can be sampled from a Dirichlet distribution or generated with spatial contextual information.Dirichlet sampling allows control over the amount of pure pixels and supports large Monte Carlo simulations.
  • A. Generating Synthetic Data: Realistic endmember variability is difficult because material spectra depend jointly on physical, chemical, and environmental parameters.Radiative-transfer models describe processes governing vegetation spectra, mineral interactions, and atmospheric effects.
  • A. Generating Synthetic Data: Well-calibrated radiative-transfer models generate simulated scenes that strongly agree with reference ground truth collected under the same circumstances.Such simulations can support studies of nonlinear mixtures and endmember variability effects.
  • A. Generating Synthetic Data: Simplified models vary vegetation or material reflectance through viewing geometry, scattering assumptions, atmospheric compensation parameters, and material properties.The examples include PROSPECT-D, a simplified Hapke model, and an atmospheric compensation model; the Hapke simplification becomes nonlinear at large albedo values.
  • A. Generating Synthetic Data: The synthetic scene includes generated signatures for vegetation, dirt, and water before mixed pixels are formed.Per-pixel endmembers are sampled from the synthetically generated signature sets.

B. Available Software Resources

Software resources range from established packages implementing classical variability-aware methods to standalone Matlab or Python prototypes. An illustrative simulation compares selected algorithm families using abundance, reconstruction, endmember, and runtime measures.

  • Available software resources: VIPER tools and RStoolbox provide implementations of MESMA and related classical techniques, while newer methods are commonly distributed as Matlab or Python prototypes.VIPER tools are available as plug-ins for ENVI and QGIS, and MESMA is also implemented in RStoolbox.
  • Simulation setup: The illustrative simulation generated a 50×50-pixel synthetic image containing vegetation, dirt, and water with spatially correlated abundances.Synthetic endmember signatures used L = 198 bands, and white Gaussian noise produced a signal-to-noise ratio of 30 dB.
  • Algorithm selection and setup: The comparison selected library-based, Bayesian, parametric, and EM-model-free methods, using FCLS with VCA-extracted endmembers as a baseline.VCA endmembers also initialized or served as reference or mean signatures for several methods.
  • Quantitative and visual results: All variability-aware methods improved abundance reconstruction over FCLS; library-based methods performed especially well, with fractional sparse SU slightly outperforming MESMA.The library-based result was attributed to image-extracted spectra accurately representing the scene’s typical endmember variability.
  • Quantitative and visual results: ELMM and DeepGUn substantially improved over FCLS, whereas RUSAL improved less; regularization-parameter sensitivity could reduce parametric and EM-model-free performance in Monte Carlo simulations.BCM also improved considerably without parameter adjustment, while NCM-E was virtually identical to FCLS and its isotropic Gaussian hypothesis may be unsuitable for this dataset.
  • Computational cost: Library-based methods ran fastest, parametric methods required intermediate time, RUSAL was very fast, and Bayesian methods were slowest, with NCM-E taking significantly longer.The results indicate that spectral-library construction can substantially affect the runtime of library-based techniques.

VII. DISCUSSION, CONCLUSIONS AND FUTURE DIRECTIONS

The review identifies persistent practical and theoretical challenges in spectral-variability-aware spectral unmixing, including computational cost, parameter sensitivity, dependence on spectral libraries, limited evaluation data, and incomplete theory. It highlights future directions centered on more robust tuning, stronger data resources, broader mixture models, and representations that combine application insight with mathematical flexibility.

  • Robustness and supervision: Parameter selection remains closely tied to unmixing quality, motivating robust or informed adjustment methods for practical deployment.Metadata such as external classification maps is suggested as one possible source for informed adjustment.
  • Spectral libraries: Most spectral-variability methods depend on spectral libraries or reference endmembers, making robust selection of this data important for reliable performance.References may be known a priori or extracted from the observed hyperspectral image.
  • Mixture modeling: Most reviewed methods use the LMM despite the prevalence of nonlinear mixtures, although spatially varying linear models can address nonlinear unmixing to some extent.The review notes that the relationship does not eliminate the need to consider nonlinear mixing.
  • Evaluation: Evaluation is hindered by limited datasets with reliable abundance ground truth and the absence of an agreed protocol for realistic synthetic data.The review calls for larger publicly available datasets to strengthen validation.
  • Modeling spectral variability: Existing variability models range from application-specific constrained forms to mathematically flexible generic representations, suggesting value in combining practical insight with rigorous mathematical treatment.The review presents this combination as a route toward improved scene-specific representations.
  • Computational challenges: Many methods rely on approximate solutions to complex non-convex optimization problems, while some algorithms are computationally expensive and scale poorly to large images.More reliable optimization and faster alternatives could clarify attainable accuracy and support large-scale processing.
Loading 2001.07307v4…