PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Band Image Fusion Based on Spectral Unmixing

Qi Wei, Jose Bioucas-Dias, Nicolas Dobigeon, Jean-Yves Tourneret, Marcus Chen, Simon Godsill

arXiv:1603.08720v1cs.CV

TL;DR

High-spectral-resolution images are limited by acquisition devices’ spatial resolution, motivating fusion of complementary images. The paper jointly estimates endmember signatures and abundance maps using spectral-mixture structure and abundance constraints; experiments report competitive comparisons with two state-of-the-art methods and improved estimation performance.

  • Problem

    High-spectral-resolution multi-band images generally have limited spatial resolution, motivating reconstruction of an image with both high spatial and spectral resolution.

    Source: §I · p. 2

  • Method

    The method jointly estimates endmember signatures and abundances, regularizing through the mixture model’s low-rank representation and abundance non-negativity and sum-to-one constraints, with ADMM updates.

    Source: §I · p. 4

  • Results

    Experiments report competitive performance against two state-of-the-art methods, with improved estimation performance.

    Source: §VI · p. 30

  • Takeaways & Limitations

    The proposed method combines multi-band fusion and spectral unmixing, using abundance constraints to regularize the ill-posed problem.

    Source: §I · p. 4

Abstract

from arXiv · show

This paper presents a multi-band image fusion algorithm based on unsupervised spectral unmixing for combining a high-spatial low-spectral resolution image and a low-spatial high-spectral resolution image. The widely used linear observation model (with additive Gaussian noise) is combined with the linear spectral mixture model to form the likelihoods of the observations. The non-negativity and sum-to-one constraints resulting from the intrinsic physical properties of the abundances are introduced as prior information to regularize this ill-posed problem. The joint fusion and unmixing problem is then formulated as maximizing the joint posterior distribution with respect to the endmember signatures and abundance maps, This optimization problem is attacked with an alternating optimization strategy. The two resulting sub-problems are convex and are solved efficiently using the alternating direction method of multipliers. Experiments are conducted for both synthetic and semi-real data. Simulation results show that the proposed unmixing based fusion scheme improves both the abundance and endmember estimation comparing with the state-of-the-art joint fusion and unmixing algorithms.

I. INTRODUCTION

High-spectral-resolution images generally have limited spatial resolution, motivating reconstruction of a high-spatial and high-spectral image from complementary observations. This paper frames fusion as an ill-posed inverse problem and jointly estimates endmember signatures and abundance maps using spectral-mixture structure and physical constraints.

I. INTRODUCTION

  • High-spectral-resolution images are limited by acquisition devices’ spatial resolution, while reconstructing a high-spatial and high-spectral image from complementary observations is a challenging inverse problem.The paper describes fusion as restoring a three-dimensional data cube from two degraded data cubes.Source: §I · p. 2
  • The averaging assumption used by earlier approaches can fail when spatial blur makes a low-resolution pixel’s corresponding high-resolution area arbitrarily large while the downsampling ratio remains fixed.The assumption treats low-resolution pixels as averages of high-resolution pixels in an area whose size depends on the downsampling ratio.Source: §I · p. 3
  • CNMF does not use HS-image abundance estimates or MS-image endmember signatures, leaving spectral and spatial information in both images not fully exploited.The paper also notes a related framework that replaces CNMF’s alternating NMF steps with alternating proximal forward-backward steps.Source: §I · p. 3
  • The proposed fusion problem uses the linear spectral mixture model’s low-rank representation and abundance non-negativity and sum-to-one constraints as regularization.The constraints are motivated by the intrinsic physical properties of abundances.Source: §I · p. 4
  • The method jointly estimates endmember signatures and abundances by solving constrained linear regression subproblems with ADMM.The two optimization subproblems are described as constrained linear regression problems.Source: §I · p. 4

II. PROBLEM STATEMENT

The section represents the target image through a linear mixture of endmember signatures and models the observed images as spectrally or spatially degraded versions of that target, with additive noise.

II. PROBLEM STATEMENT

  • Each target-image column is a spectral pixel, while each row gathers pixel values from one spectral band.The image is rearranged as an mλ × n matrix, with n equal to the number of pixels per band.Source: §II · p. 4

A. Linear Mixture Model

B. Forward Model

  • The two complementary observations are linear spectral and spatial degradations of the full-resolution target image, with additive terms for modeling errors and sensor noise.The model assumes pixel-independent noise and diagonal row-covariance matrices.Source: B. Forward Model · pp. 5–6

C. Composite Fusion Model

The paper combines the linear mixture and observation models to formulate fusion as joint estimation of endmembers and abundances. It uses physical-property priors and a joint likelihood for the two observations.

C. Composite Fusion Model

  • The target image is modeled by combining the linear mixture model with the forward observation model.The endmember matrix can be estimated jointly with the abundance matrix.Source: C. Composite Fusion Model · pp. 6–7

D. Statistical Methods

  • The inference problem estimates the unknown endmember and abundance matrices from the two high-spectral and high-spatial resolution observations.The inference is challenging because of the target matrix's large size and the downsampling operator, which prevents direct Fourier diagonalization of the blurring operator.Source: D. Statistical Methods · p. 7
  • Joint estimation of the endmembers and abundances is ill-posed, and the paper proposes priors based on their intrinsic physical properties.The problem is described as non-convex and without a unique solution, motivating constraints or prior information for regularization.Source: D. Statistical Methods · pp. 7–8

A. Likelihoods (Data Fidelity Term)

  • The two observations are modeled as conditionally independent given the unobserved scene, yielding a joint likelihood equal to the product of their likelihoods.The observations have matrix Gaussian distributions under the stated noise model.Source: A. Likelihoods (Data Fidelity Term) · p. 8

B. Priors (Regularization Term)

The paper encodes physical constraints on abundances and endmembers as priors, then maximizes the joint posterior through alternating optimization of the two matrices.

1) Abundances:

  • Abundance vectors satisfy non-negativity and sum-to-one constraints, representing physically constrained proportions across endmembers.These constraints are also represented as a uniform distribution over the feasible abundance region.Source: 1) Abundances · p. 9

2) Endmembers:

  • Endmember values are constrained to lie between zero and one, and the constraints are converted into a uniform distribution over the feasible region.The passage associates the bound with the reflectance values represented by the matrix.Source: 2) Endmembers · pp. 9–10

C. Posteriors (Constrained Optimization)

  • The fusion problem is formulated as maximizing the joint posterior of the abundance and endmember matrices, equivalently minimizing its negative logarithm under the abundance constraints.The formulation includes two data-fidelity terms, so both images contribute to estimating endmembers and high-resolution abundance maps.Source: C. Posteriors (Constrained Optimization) · p. 10

IV. ALTERNATING OPTIMIZATION SCHEME

  • The objective is non-convex jointly, so FUMI alternates between abundance and endmember optimizations, each a convex subproblem solved with ADMM.The stated convergence guarantee for each conditional ADMM optimization holds under mild conditions.Source: §IV · p. 11

A. Convergence Analysis

The section examines convergence and solution guarantees for the alternating procedure and its ADMM subproblems, including the lack of general uniqueness for BCD minimizers. It also considers the fixed-endmember case, where the problem is convex and a more efficient algorithm guarantees a global optimum.

A. Convergence Analysis

  • Under the BCD theorem’s assumptions, every limit point is stationary; although uniqueness is not guaranteed, small quadratic terms ensure unique minima, and convergence was observed without them.The target function is continuously differentiable; the quadratic terms make the objective strictly convex, allowing the theorem to be invoked.Source: A. Convergence Analysis · pp. 11–12

B. Optimization w.r.t. the Abundance Matrix A (M fixed)

  • For fixed M, the abundance ADMM sub-iterations converge to a solution when their component functions are closed, proper, and convex.The updates alternate between minimizing over A and V and updating the dual variable; the V step projects onto the simplex.Source: B. Optimization w.r.t. the Abundance… · p. 13

C. Optimization w.r.t. the Endmember Matrix M (A fixed)

  • For fixed A, the endmember ADMM is guaranteed to converge to a solution when its component functions are closed, proper, and convex and H^T H has full column rank.The iterations update M, T, and the dual variable.Source: C. Optimization w.r.t. the Endmember… · p. 16

A. Convergence Analysis

  • When endmember signatures are fixed a priori (M is known), unsupervised unmixing and fusion degenerate to supervised unmixing and fusion by not updating M; the alternating scheme is unnecessary.Source: 1) Updating M · p. 18
  • With a priori fixed endmembers M, problem (10) becomes convex and guarantees a global optimum rather than only a stationary point.The resulting problem can be solved much more efficiently.Source: 1) Updating M · pp. 18–19

V. EXPERIMENTAL RESULTS

The proposed unmixing-based fusion method is applied to multi-band images associated with synthetic and semi-real data. The algorithms were implemented in MATLAB R2014A on an Intel Core i7-2600 CPU.

  • The proposed method is applied to multi-band images associated with both synthetic and semi-real data.Source: §V · p. 19
  • All algorithms were implemented using MATLAB R2014A.Source: §V · p. 19
  • The implementation ran on an Intel Core i7-2600 CPU at 3.40 GHz.Source: §V · p. 19

A. Quality metrics

The section evaluates fusion quality using five image quality measures.

A. Quality metrics

1) Fusion quality:

Fusion quality is assessed with five quantitative measures, each with a stated direction for better fusion. The paper also turns to assessing unmixing-result quality.

1) Fusion quality:

2) Unmixing quality:

The section defines the unmixing quality measures and describes synthetic tests of the fusion method. Its implementation uses subspace reduction and reports that varying selected ADMM parameters has little effect on convergence.

2) Unmixing quality:

  • The smaller NMSE and absolute SAM indicate better unmixing quality and less spectral distortion, respectively.Overall SAM averages the endmember SAM values and is expressed in degrees.Source: 2) Unmixing quality · p. 19

B. Synthetic data

  • Synthetic mixtures use five USGS-library endmembers and Dirichlet-generated abundances satisfying non-negativity and sum-to-one constraints.The endmembers have 224 bands spanning 383–2508 nm, and the considered abundance matrix covers a 100 × 100 image.Source: B. Synthetic data · p. 20
  • SISAL initializes the endmember matrix and does not require pure pixels in the image.The algorithm can theoretically use any endmember extraction algorithm for initialization.Source: B. Synthetic data · p. 20
  • SVD-based subspace projection preserves the original formulation while greatly reducing matrix dimensions.The method learns the projection from the high-spectral-resolution data and projects the HS observations, noise covariance, and spectral response.Source: B. Synthetic data · p. 20
  • ADMM runs for 30 iterations, and varying the selected parameters does not substantially affect overall algorithm convergence.The stopping rule compares successive objective updates against a relative-difference threshold of 10^-4.Source: B. Synthetic data · p. 21

1) HS and MS image fusion:

Across synthetic and semi-real fusion tests, FUMI performs strongly on fusion quality, while unmixing results distinguish endmember from abundance estimation. On Moffett, the FUMI variants fuse better but take more computation time.

1) HS and MS image fusion:

  • FUMI outperforms the other two methods in synthetic HS+MS fusion, with competitive time complexity.Berne’s method and FUMI perform similarly for endmember estimation, both better than Yokoya’s; FUMI gives the best abundance estimation.Source: 1) HS and MS image fusion · p. 21, Table II · p. 22

2) HS and PAN image fusion:

  • FUMI performs best on all fusion quality measures and uses the least CPU time for synthetic HS+PAN fusion.Berne’s method gives the best endmember estimates, while FUMI gives the best abundance estimates.Source: Table IV · p. 22, 2) HS and PAN image fusion · p. 22

C. Semi-real data

  • The Moffett test uses an HS reference and simulated degraded HS and PAN observations; the scattered data show no pure pixel.The PAN image is formed by averaging the first 50 HS bands, and the HS observation is filtered and downsampled.Source: C. Semi-real data · p. 23, 1) Moffett dataset · p. 23

1) Moffett dataset:

  • S-FUMI and UnS-FUMI produce better-looking Moffett fused images than the other methods, with smaller errors in the corresponding RMSE maps.The figure compares HS, MS, and reference images with Berne’s, Yokoya’s, S-FUMI, and UnS-FUMI outputs; the bottom row shows the corresponding RMSE maps.Source: 1) Moffett dataset · p. 24, Figure 1 · p. 25
  • On Moffett, S-FUMI and UnS-FUMI outperform the other methods on all fusion metrics, but cost more computation time.Their higher cost is attributed mainly to alternating endmember and abundance updates and the ADMM updates within them.Source: Table VI · p. 24, 1) Moffett dataset · p. 24
  • For Moffett unmixing, FUMI has competitive endmember estimates and much better abundance estimates than Berne’s and Yokoya’s methods.S-FUMI and UnS-FUMI have similar fusion results, but UnS-FUMI estimates endmembers better than S-FUMI; the paper attributes this to jointly updating endmembers with abundances using complementary spectral and spatial information.Source: Table VII · p. 24, 1) Moffett dataset · p. 24

2) Pavia dataset:

The Pavia evaluation compares fusion and unmixing with prior methods, while the conclusion describes an alternating joint fusion-and-unmixing algorithm. Experiments report competitive performance against two state-of-the-art methods, and UnS-FUMI outperforms S-FUMI in endmember and abundance estimation.

2) Pavia dataset:

  • The Pavia reference image is a 100 × 100 × 93 hyperspectral image; the original 115 bands were reduced to 93 after water-vapor absorption bands were removed.The image was acquired over Pavia, Italy, by ROSIS.Source: 2) Pavia dataset · p. 26
  • UnS-FUMI outperforms S-FUMI in Pavia endmember and abundance estimation, attributed to alternating updates of endmembers and abundances.The Pavia results also describe S-FUMI and UnS-FUMI as visually and quantitatively competitive and much better than the other methods.Source: 2) Pavia dataset · pp. 26–27, Figure 7 · p. 29, Figure 8 · p. 30

VI. CONCLUSION

  • Abundance updates solve a Sylvester matrix equation and project onto a simplex; the equation is solved analytically using R-FUSE without iterative updates.The method alternately updates endmembers and abundances using ADMM.Source: §VI · pp. 28–29
  • Endmember updates use least-squares regression followed by thresholding, while future work will incorporate spatial and spectral degradation into the estimation framework.The conclusion characterizes the two endmember-update steps as not computationally intensive.Source: §VI · pp. 30–31
  • The joint fusion-and-unmixing algorithm compared competitively with two state-of-the-art methods and improved performance.The conclusion reports the overall comparison without specifying the performance metric in these passages.Source: §VI · p. 30

Related papers

Loading 1603.08720v1…