PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Generative Endmember Modeling: An Application to Unsupervised Spectral Unmixing

Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez

arXiv:1902.05528v3cs.CV

TL;DR

Endmember variability is common, can propagate estimation errors through unmixing, and remains difficult to model with both flexibility and low-dimensional representation. The paper learns generative endmember models from observed pure pixels and uses them in an alternating unmixing procedure. On synthetic and real data, the approach reports more accurate abundance estimates than standard methods accounting for variability, at a small computational-cost increase.

  • Problem

    Endmember variability can propagate estimation errors through unmixing, while existing models struggle to combine flexible representation with low-dimensional structure.

    Source: A. EM variability and learning-based… · pp. 1–2

  • Method

    DeepGUn learns a generative model of each endmember from observed pure pixels, then alternates estimation of abundances and latent endmember representations.

    Source: B. Proposed methodology · p. 2

  • Results

    On synthetic and real data, DeepGUn yields more accurate abundance estimates than standard state-of-the-art methods accounting for endmember variability, with a small computational-cost increase.

    Source: B. Proposed methodology · p. 2

  • Takeaways & Limitations

    The approach produces a low-dimensional, physically accurate model intended to describe the endmember variability present in a scene.

    Source: B. Proposed methodology · p. 2

    The improved accuracy comes with a small increase in computational cost compared with the best competing strategies.

    Source: B. Proposed methodology · p. 2

Abstract

from arXiv · show

Endmember (EM) spectral variability can greatly impact the performance of standard hyperspectral image analysis algorithms. Extended parametric models have been successfully applied to account for the EM spectral variability. However, these models still lack the compromise between flexibility and low-dimensional representation that is necessary to properly explore the fact that spectral variability is often confined to a low-dimensional manifold in real scenes. In this paper we propose to learn a spectral variability model directly form the observed data, instead of imposing it \emph{a priori}. This is achieved through a deep generative EM model, which is estimated using a variational autoencoder (VAE). The encoder and decoder that compose the generative model are trained using pure pixel information extracted directly from the observed image, what allows for an unsupervised formulation. The proposed EM model is applied to the solution of a spectral unmixing problem, which we cast as an alternating nonlinear least-squares problem that is solved iteratively with respect to the abundances and to the low-dimensional representations of the EMs in the latent space of the deep generative model. Simulations using both synthetic and real data indicate that the proposed strategy can outperform the competing state-of-the-art algorithms.

I. INTRODUCTION

The introduction frames endmember variability as a source of unmixing error that existing models struggle to represent with both flexibility and low-dimensional structure. It presents DeepGUn, which learns generative endmember models from observed pure pixels and reports more accurate abundance estimates on synthetic and real data.

A. EM variability and learning-based SU methods

  • Endmember variability is common and can propagate significant estimation errors through unmixing; it may arise from environmental, illumination, atmospheric, or temporal changes.Spectral-unmixing methods estimate both material spectra and their proportions within image pixels.Source: A. EM variability and learning-based… · p. 1, §I · p. 1
  • Existing parametric models struggle to balance flexible representation of complex endmember variability with low-dimensionality, risking either restrictive models or ill-posed estimation.Real-scene variability is often confined to low-dimensional manifolds because spectra can depend on only a few material properties.Source: A. EM variability and learning-based… · pp. 1–2
  • Prior supervised learning approaches require large training sets, while some autoencoder methods lack a direct physical mixing connection or use many parameters.The introduction also notes that labeled-data requirements can make supervised learning computationally infeasible.Source: A. EM variability and learning-based… · p. 2

B. Proposed methodology

  • DeepGUn learns a generative neural model of each material's spectral manifold from pure pixels in the observed image, then alternates estimates of abundances and latent endmember representations.The pure-pixel training supports an unsupervised formulation within an extended linear mixing model.Source: B. Proposed methodology · p. 2
  • On synthetic and real data, DeepGUn yields more accurate abundance estimates than standard state-of-the-art methods accounting for endmember variability, with a small computational-cost increase.Qualitative abundance-map analysis confirms the results.Source: B. Proposed methodology · p. 2

II. LINEAR MIXING MODELS

The section describes the linear mixing model and its fixed-endmember assumption, then motivates learning the variability function instead of prescribing its form.

II. LINEAR MIXING MODELS

  • The LMM represents each pixel as a mixture of endmember spectra, with nonnegative abundances summing to one.Its fixed spectra across pixels can undermine abundance accuracy when endmembers vary within a scene.Source: §II · p. 3
  • Extended models describe pixel-specific endmembers with a parametric function of a reference endmember matrix and variability parameters.Additive and multiplicative variability factors are examples of proposed functional forms.Source: §II · p. 3
  • Prescribed additive or multiplicative forms can be too restrictive or produce ill-posed optimization, so the paper proposes learning the parametric function with a generative model.The stated goal is a better balance between low-dimensional representation and flexibility for complex variability.Source: §II · p. 3

III. GENERATIVE MODELS

Generative models learn a data distribution in a way that allows new samples to be generated. The section explains latent-variable mappings and contrasts VAE and GAN properties, motivating the paper's VAE choice.

III. GENERATIVE MODELS

  • Generative models estimate a data distribution from observations so they can generate new samples resembling the observed data.The section distinguishes this goal from density estimation by requiring sampling from the estimated model.Source: §III · p. 3
  • A low-dimensional latent variable can be mapped through a neural network to approximate a high-dimensional data distribution and generate samples.This approach exploits the possibility that high-dimensional distributions are supported on low-dimensional manifolds.Source: §III · p. 3
  • VAEs model data through a latent-variable graphical model and learn its generator by maximizing a lower bound on data log-likelihood.The encoder distribution approximates the intractable posterior; greater capacity can bring the lower bound closer to the true log-likelihood.Source: §III · p. 3
  • GANs learn through an adversarial game and can approximate complex distributions sharply, but are harder to train than VAEs.VAEs also provide an encoder for latent representations and offer stable training, motivating their use in this work.Source: §III · p. 3

IV. A DEEP GENERATIVE ENDMEMBER MODEL

The paper models endmember spectral variability with a deep generative model, exploiting the low-dimensional manifolds often associated with real-scene variability. It learns the models from pure pixels in the observed image and uses latent representations to estimate abundances and endmembers.

IV. A DEEP GENERATIVE ENDMEMBER MODEL

  • Deep generative models represent endmember spectral variability on a low-dimensional manifold, addressing the need to capture complex variability compactly.The paper relates this structure to spectral signatures depending on a few photometric or chemical properties.Source: §IV · p. 4

A. The steps of the proposed SU method

  • The method divides unmixing into learning generative and encoder models from observed-image pure pixels, then estimating each pixel’s latent endmembers and abundances.The pure pixels provide information for estimating the endmember models and can reduce ill-posedness.Source: A. The steps of the proposed SU method · p. 4

B. Learning the generative and encoder models Gθp and Dφp

  • A VAE learns each material’s generative and encoder models from its extracted pure pixels, provided those pixels represent the material’s variability.The learned generators then describe the endmember variability manifolds.Source: B. Learning the generative and encoder… · p. 4
  • A reference endmember’s latent code regularizes estimates toward pure-pixel representations, helping prevent the generator from representing mixed pixels.This robustness strategy relies on having correctly identified reference signatures.Source: B. Learning the generative and encoder… · pp. 4–5

C. Extracting sets of pure pixels from the observed HI

The method selects training pixels by their spectral-angle proximity to reference endmembers, then alternates abundance and latent-representation estimation. Regularizers constrain abundances and keep latent endmember codes near reference codes.

C. Extracting sets of pure pixels from the observed HI

  • For each endmember, the method selects the S_p image pixels with the smallest spectral angle to its reference signature; success depends on a reasonably accurate reference matrix.The authors found this strategy more robust and easier to adjust than the cited alternative.Source: C. Extracting sets of pure pixels from… · p. 5

V. THE UNMIXING ALGORITHM

  • Abundance regularization promotes spatial smoothness, positivity, and sum-to-one constraints.Its spatial terms use horizontal and vertical first-order gradients.Source: §V · p. 5
  • Latent-code regularization keeps estimated endmember representations close to reference codes, but performance depends on reference signatures representing their material classes adequately.The regularization weight λZ controls this term’s contribution to the cost function.Source: §V · p. 5
  • DeepGUn first trains generative and encoder models on extracted pure pixels, then alternates abundance and latent-representation estimation.The joint problem is non-smooth and non-convex when solved simultaneously.Source: §V · p. 5

A. Optimization with respect to Z

The latent-representation update decomposes into per-pixel regularized nonlinear least-squares problems solved with a quasi-Newton algorithm. The paper describes favorable recovery geometry under assumptions on the generator, while noting non-smoothness can affect optimization.

A. Optimization with respect to Z

  • The algorithm initializes reference endmembers and abundances, extracts pure pixels, trains the generative and encoder models, and encodes the reference signatures.The algorithm then iterates until its stopping criterion is satisfied and generates endmembers from the latent codes.Source: A. Optimization with respect to Z · p. 6
  • Each pixel’s latent representation is estimated by solving a regularized nonlinear least-squares problem with a quasi-Newton algorithm.The per-pixel problems have dimensionality K × P.Source: A. Optimization with respect to Z · pp. 5–6
  • Under suitable assumptions on generator eG, latent recovery has no stationary points outside a small neighborhood of the solution and its negative scalar multiple.The authors describe this as favorable global geometry for the optimization problem.Source: A. Optimization with respect to Z · p. 6
  • Because eG may be non-differentiable, quasi-Newton optimization is applied with line search; convergence is generally observed when line search avoids non-differentiable points,.The cited discussion characterizes these algorithms as efficient for approximate solutions to non-smooth problems.Source: A. Optimization with respect to Z · p. 6

B. Optimization with respect to A

The abundance-dependent objective is restated from the terms of (7), while the latent-variable update uses a quasi-Newton procedure. With latent variables fixed, abundance estimation is an edge-preserving spatially regularized unmixing problem solved using ADMM.

B. Optimization with respect to A

  • The abundance-dependent objective is obtained by retaining from (7) only terms that depend on A.The passage introduces this restatement but does not provide its expression.Source: B. Optimization with respect to A · p. 6
  • The quasi-Newton procedure updates the latent variables with a Wolfe-condition line search until its stopping criterion is met.Its input includes abundances, observations, a regularization parameter, and an initial latent representation; it outputs updated latent variables.Source: B. Optimization with respect to A · p. 6
  • With latent variables fixed, the abundance problem uses a pixel-dependent endmember matrix and edge-preserving spatial regularization, and ADMM solves the nonseparable problem.The ADMM solution is omitted as described elsewhere.Source: B. Optimization with respect to A · p. 6

VI. NEURAL NETWORK ARCHITECTURE

A VAE learns generative and encoder models from pure pixels, using a two-dimensional latent space selected as sufficient to capture each material’s variability. The network uses three hidden ReLU layers and is trained with Adam for 50 epochs.

VI. NEURAL NETWORK ARCHITECTURE

  • The VAE learns generative and encoder models from pure pixels, with latent dimension K = 2 experimentally found sufficient to capture each material’s variability.The authors note that VAE training is simpler and more stable than GAN training and that VAEs naturally provide an encoder approximating the posterior.Source: §VI · p. 6
  • The encoder and decoder use three hidden layers with ReLU activations, and Table II details the decoder architecture.Layer and neuron counts were selected according to the autoencoder implementation cited in the paper.Source: §VI · p. 6, Table II · p. 7
  • Training used Adam for 50 epochs, with mini-batches one third the size of each endmember’s training set.The authors report that this configuration produced spectrally smooth signatures and generalized effectively with small training samples.Source: §VI · p. 6

VII. EXPERIMENTAL RESULTS

The experiments compare DeepGUn with FCLS, PLMM, ELMM, and GLMM using common initialization procedures. Performance is evaluated with abundance, endmember, and reconstructed-image error metrics, plus spectral angle for endmembers.

VII. EXPERIMENTAL RESULTS

  • DeepGUn is compared with FCLS, PLMM, ELMM, and GLMM, using VCA reference endmembers and FCLS-initialized abundance maps.Pure-pixel training sets use the 100 pixels with smallest spectral angles to the reference endmembers; alternating optimization stops after at most 10 iterations or when relative changes fall below 10^-3.Source: §VII · p. 7
  • Evaluation uses NRMSE for abundance maps, endmember matrices, and reconstructed images.For reconstructed images, the estimate is formed by multiplying the estimated pixel-dependent endmember matrix by the estimated abundances.Source: §VII · p. 7
  • The estimated endmembers are also evaluated with Spectral Angle Mapper using the true and estimated signatures for each material and pixel.The iterative latent-variable procedure stops when the relative change in each latent vector is below 10^-3.Source: §VII · p. 7

A. Synthetic data

On the four synthetic datasets, DeepGUn gives the best reported abundance and endmember reconstruction results, while image reconstruction error is not uniformly best. Its computational complexity lies between GLMM and PLMM, with a reported reasonable computational cost.

A. Synthetic data

  • The synthetic cubes contain three or five 224-band endmembers, spatially correlated abundances, and 30 dB SNR, with variability modeled in four different ways.DC1–DC3 use three endmembers and DC4 uses five; DC4 uses pure pixels manually extracted from a real hyperspectral image.Source: A. Synthetic data · p. 7
  • DeepGUn outperformed competing algorithms in NRMSEA on all four datasets, and its abundance maps were closer to the true maps than those of other methods.Figure 2 displays the abundance maps for DC1–DC4, with abundance values encoded from blue (0) to red (1).Source: Table III · p. 7, A. Synthetic data · p. 8, Figure 2 · p. 8
  • Algorithm parameters were selected by dataset-specific grid searches, while DeepGUn fixed λZ = 0.1 and searched λA over three values.Pure-pixel sets for each DeepGUn endmember used the 100 pixels closest to the reference materials.Source: A. Synthetic data · p. 7
  • DeepGUn achieved the best NRMSEM and SAMM results for all four data cubes.The authors state that these results indicate precise material identification from the observed scenes.Source: A. Synthetic data · p. 8
  • DeepGUn's NRMSEY was comparable to FCLS and significantly higher than GLMM's, whose greater degrees of freedom are noted as a reason.The authors caution that image reconstruction error is not directly linked to abundance reconstruction error.Source: A. Synthetic data · p. 8
  • DeepGUn's computational complexity falls between GLMM and PLMM, with superior unmixing performance, easier parameter tuning, and reasonable computational cost.The paper reports a small computational-cost increase compared with the best competing strategies.Source: A. Synthetic data · p. 8

B. Influence of the latent dimension K

The latent dimension K controls how much endmember variability DeepGUn can represent, but larger K can worsen abundance estimation. The experiments show that this effect differs between DC1 and DC2.

B. Influence of the latent dimension K

C. Real data

On real images, DeepGUn produces abundance maps with improvements in several scene-specific comparisons, although its reconstruction errors are higher than those of competing variability models. The authors conclude that simulations on synthetic and real data indicate significant improvements in abundance estimation accuracy.

C. Real data

  • Real-image tests used Houston, Samson, and Jasper Ridge AVIRIS data with 188, 156, and 198 retained bands, respectively.The images originally had 224 bands; water-absorption and low-SNR bands were removed.Source: C. Real data · p. 9
  • DeepGUn maps better show Houston’s strong vegetation and concrete abundances at the stadium and stands, and asphalt in the parking lot.The Samson and Jasper Ridge maps also show improvements, including less Water–Soil confusion in Samson and stronger river Water abundance in Jasper Ridge.Source: C. Real data · p. 9, Figure 5 · p. 9, Figure 6 · p. 9
  • DeepGUn has higher reconstruction errors than other variability methods, but reconstruction error is not a reliable measure of real-image abundance estimation.The authors attribute the higher errors to DeepGUn’s K × P degrees of freedom per pixel, compared with more than L × P for ELMM, GLMM, and PLMM; execution times were comparable.Source: C. Real data · pp. 9–10

VIII. CONCLUSIONS

  • Synthetic and real-data simulations indicate that DeepGUn can significantly improve abundance estimation accuracy.The conclusion describes excellent performance despite a simple strategy for selecting training data for the generative model.Source: §VIII · p. 10
Loading 1902.05528v3…