Bayesian Fusion of Multi-Band Images

Qi Wei; Nicolas Dobigeon; Jean-Yves Tourneret

Source-linked AI summary

Bayesian Fusion of Multi-Band Images

Qi Wei, Nicolas Dobigeon, Jean-Yves Tourneret

arXiv:1307.5996v2Jul 23, 2013cs.CVphysics.data-anstat.ME

TL;DR

The paper addresses recovery of a high-spatial- and high-spectral-resolution scene from images degraded by sensor-specific spatial and spectral transformations. It uses hierarchical Bayesian fusion with a geometrically informed prior and MCMC sampling incorporating HMC, and simulations show competition with state-of-the-art MS–HS fusion techniques.

Problem
Recovering a high-spatial- and high-spectral-resolution scene from multiple degraded observations requires fusing spatial and spectral information in an ill-posed problem.
Method
The method uses a hierarchical Bayesian model with physical sensor transformations, a geometrically informed prior, and an HMC-based hybrid Gibbs sampler to generate posterior samples.
Results
Simulations on pseudo-real data showed that the proposed method competed with state-of-the-art techniques for fusing MS and HS images.
Takeaways & Limitations
The framework supports Bayesian fusion of multiple multi-band images with varied physical transformations and produces a high-resolution hyperspectral reconstruction from low-spatial-resolution HS and MS data.

Abstract

from arXiv · show

In this paper, a Bayesian fusion technique for remotely sensed multi-band images is presented. The observed images are related to the high spectral and high spatial resolution image to be recovered through physical degradations, e.g., spatial and spectral blurring and/or subsampling defined by the sensor characteristics. The fusion problem is formulated within a Bayesian estimation framework. An appropriate prior distribution exploiting geometrical consideration is introduced. To compute the Bayesian estimator of the scene of interest from its posterior distribution, a Markov chain Monte Carlo algorithm is designed to generate samples asymptotically distributed according to the target distribution. To efficiently sample from this high-dimension distribution, a Hamiltonian Monte Carlo step is introduced in the Gibbs sampling strategy. The efficiency of the proposed fusion method is evaluated with respect to several state-of-the-art fusion techniques. In particular, low spatial resolution hyperspectral and multispectral images are fused to produce a high spatial resolution hyperspectral image.

I. INTRODUCTION

HS–MS fusion combines complementary spatial and spectral information to recover a high-resolution image, but its high dimensionality makes the problem challenging. The paper addresses this challenge with a Bayesian model using sensor-aware degradations, a geometrically informed prior, and MCMC sampling with an HMC step.

The high dimensionality of HS–MS data makes fusion challenging and motivates the development of innovative methods.
HS–MS fusion can exploit complementary spatial and spectral information to obtain a high spatial and spectral resolution image.
Existing pansharpening methods, including component substitution, relative spectral contribution, and high-frequency injection, can be inapplicable or inefficient for HS+MS fusion.
The proposed Bayesian model explicitly exploits sensor acquisition processes and specifications to design spatial and spectral degradations for the recovered image.
The prior assumes that the high-resolution HS image lies in a lower-dimensional subspace, suitable when scenes contain finitely many macroscopic materials.
The MMSE estimator is computed from MCMC-generated samples, with an HMC step introduced because conventional MCMC is difficult in high-dimensional spaces.
HMC exploits Hamiltonian dynamics to propose higher-acceptance states and reduce correlation between successive samples.

II. PROBLEM FORMULATION

The paper models multiple sensor observations as degraded versions of a common high-spatial- and high-spectral-resolution scene. It formulates Bayesian inference using sensor-specific observation models and estimates the scene from its posterior distribution.

A. Notations and observation model: The observation model represents multiple images from potentially heterogeneous sensors observing the same scene.
A. Notations and observation model: Observed images may differ in spatial and spectral resolutions, and are treated as degraded versions of the high-resolution scene.
A. Notations and observation model: The model includes additive error terms representing both observation noise and mismodeling.
A. Notations and observation model: Each observed image is vectorized into a measurement vector with total size Np = nx,pny,pnλ,p, using a BIP-like scheme for convenience.
A. Notations and observation model: Noise is modeled as band-dependent Gaussian, with sensor-dependent variance reflecting differing signal-to-noise ratios.
A. Notations and observation model: The degradation operator can represent spatial decimation, spatial blurring, and other spatial or spectral transformations.
A. Notations and observation model: The fusion task recovers the high-spectral- and high-spatial-resolution scene by combining information from all observed images.
B. Bayesian estimation of x: Bayesian estimation infers the fused image through a posterior formed from the likelihood and prior, with the paper focusing on the posterior mean or MMSE estimator.

A. Likelihood function

The model represents observed sensor vectors with Gaussian noise and formulates fusion around a reduced-dimensional spectral representation of the scene. A prior on projected scene vectors regularizes the ill-posed recovery problem while modeling spectral-band correlations.

Likelihood model: Each observed vector z_p is modeled as normally distributed with mean F_p x and covariance Λ_p.
Likelihood model: The likelihood provides a data-fitting term for each observed vector z_p.
Likelihood model: Measurements may come from heterogeneous sensors, so observed vectors are assumed conditionally independent given the scene and noise covariances.
Unknown parameters: The unknowns include the scene x and observation-specific noise covariance matrices Λ.
Scene prior: Each hyperspectral vector x_i is projected into a lower-dimensional subspace because its dimension is usually much smaller than the number of spectral bands.
Scene prior: A prior on projected vectors u_i regularizes the ill-posed problem, with covariance Σ_u modeling correlations among spectral bands after projection.
Scene prior: The Gaussian prior is motivated by prior fusion work and conjugacy with the Gaussian likelihood, yielding simpler posterior-based estimators; the method is also reported as robust to non-Gaussian images.

2) Noise variance priors:

The model assigns conjugate priors to noise variances and hierarchical priors to scene covariance and noise-level hyperparameters. Posterior estimators are then approximated with an MCMC strategy that combines Gibbs sampling with Metropolis-Hastings and uses burn-in samples.

Noise variance priors: Inverse-gamma distributions are assigned to noise variances, enabling closed-form conditional distributions for variance updates.
Noise variance priors: The inverse-gamma shape is controlled by two parameters; ν is fixed while γ is estimated from the data.
Hyperparameter priors: The parameter priors are treated as a priori independent, and the hyperparameter vector includes μ̄_u, Σ̄_u, Σ_u, and γ.
Hyperparameter priors: Σ_u receives an inverse-Wishart prior, with hyper-hyperparameters Ψ and η fixed to provide a non-informative prior.
Hyperparameter priors: A non-informative Jeffreys’ prior is assigned to γ because prior knowledge of the mean noise level is absent.
Posterior inference: The posterior of the projected image u is obtained by marginalizing hyperparameters, but its complexity prevents closed-form MMSE and MAP estimators.
Posterior inference: An MCMC algorithm generates asymptotically posterior-distributed samples using a Metropolis-within-Gibbs sampler, from which the MMSE estimate is computed after burn-in.
Posterior inference: The recovered highly resolved hyperspectral image is reconstructed from the estimated projected scene using x̂_MMSE = V^T û_MMSE.

IV. HYBRID GIBBS SAMPLER

The hybrid Gibbs sampler uses HMC to draw high-dimensional image samples efficiently while preserving asymptotic targeting of the posterior distribution.

Hybrid Gibbs sampler: The sampler uses Gibbs updates for conditional distributions associated with the target distribution.These conditional distributions are generally easier to simulate than the joint target distribution.
Sampling the high-resolution image: Direct Gaussian sampling is impractical because it requires inversion of an fM × fM matrix.Coordinate-wise Gibbs sampling would require fM moves and can mix poorly while being time demanding.
Hamiltonian Monte Carlo: Hamiltonian Monte Carlo generates vectors directly and improves sampler mixing, especially in high-dimensional problems.HMC exploits distribution gradients and introduces auxiliary momentum variables.
Hamiltonian Monte Carlo: The HMC proposal uses leapfrog iterations in the joint image–momentum space, followed by an accept/reject step.The move is controlled by stepsize ε and the number of leapfrog iterations.
Convergence: The accept/reject procedure makes the simulated vectors asymptotically distributed according to the distribution of interest.The proposed HMC strategy ensures asymptotic convergence to the posterior distribution.
Generality: The HMC strategy can also be extended to non-Gaussian posterior distributions, unlike the cited alternative methods.Other approaches may require additional optimization schemes and produce only approximately target-distributed samples.

D. Complexity Analysis

The proposed MCMC sampler is computationally costly, with complexity increasing sharply with the number of spectral bands; projection reduces this burden while retaining important information.

Complexity: The MCMC method can be computationally costly compared with optimization methods.The complexity is mainly due to the Hamiltonian Monte Carlo component.
Complexity: O((f mλ)^3)+O((f mλmxmy)^2) complexity becomes highly expensive as mλ increases.The number of pixels mxmy generally cannot be reduced significantly.
Complexity reduction: Projecting mλ-dimensional spectral vectors into a lower-dimensional subspace decreases complexity while keeping most important information.The projection forms f mλ × 1 vectors for the sampler.

V. SIMULATION RESULTS

The experiments reconstruct a high-spatial- and high-spectral-resolution hyperspectral image from lower-resolution hyperspectral and multispectral observations using an AVIRIS scene.

Dataset: The reference image is a 1994 AVIRIS hyperspectral image acquired over Moffett Field, California, reduced from 224 to 177 bands.It serves as the high-spatial- and high-spectral-resolution target.
Observation generation: The reconstruction uses a spatially blurred hyperspectral image and a spectrally mixed multispectral image derived from the reference.The HS image uses a 5 × 5 averaging filter, while the MS image averages adjacent bands using realistic spectral responses.
Observation model: The HS and MS observation models are perfectly known, and both observations contain zero-mean additive Gaussian noise.The simulations use SNR1 values of 35dB and 30dB across HS bands, and SNR2 = 30dB for all MS bands.
Subspace learning: Principal-component subspace learning selects eigenvectors associated with the largest covariance eigenvalues.For the Moffett Field image, 10 eigenvectors contain 99.93% of the information.
Results: The proposed algorithm performs competitively with state-of-the-art methods on the AVIRIS fusion results.The comparison is shown in Fig. 2 alongside the HS image, MS image, MAP, and Wavelet MAP results.

2) Hyper-hyperparameters selection:

The implementation uses largely non-informative prior settings, adaptively tunes HMC stepsizes, and evaluates fusion quality with complementary image-quality measures.

Hyper-hyperparameters selection: The identity matrix for Ψ and inverse-gamma parameters η = emλ + 3 are chosen to produce non-informative priors.The paper notes that parameter ν disappears after integrating the joint posterior over the relevant variable.
HMC tuning: HMC performance is governed mainly by stepsize ε and the number of leapfrog steps NL.A large stepsize lowers acceptance, whereas a small stepsize increases computational complexity.
HMC tuning: The stepsize is adapted using a statistical acceptance ratio over a counting window, with diminishing adaptation intended to preserve stationarity.The experiment uses a counting window NW = 50.
HMC tuning: The leapfrog trajectory length is selected by trial and error, with NL uniformly randomized between Nmin = 50 and Nmax = 55.Randomization is used to avoid potential resonance.
Quality measures: Fusion quality is assessed using RSNR, SAM, UIQI, ERGAS, and DD.RSNR rewards larger values, while SAM, ERGAS, and DD favor smaller distortion values; UIQI reaches 1 for identical images.
Quality measures: UIQI measures correlation, luminance, and contrast similarity, while ERGAS and DD quantify spectral or overall distortion.For multiband images, UIQI is computed band-by-band and averaged.

D. Comparison with other Bayesian models

The proposed Bayesian model differs from earlier Bayesian models through hierarchical inference, a block-diagonal covariance structure, and an explicit multispectral forward model. It performs competitively with state-of-the-art methods while also estimating uncertainty-related quantities.

Model differences: The hierarchical model infers the target image, noise variances s2, and hyperparameter Σu jointly.This structure is described as more general and flexible than previous Bayesian models.
Model differences: The block-diagonal covariance matrix Σu exploits correlations between spectral bands.
Model differences: The forward model F2 incorporates the relationship between the multispectral image and the target image.
Comparison results: The HMC method provides slightly better image-restoration results than the other compared methods.The comparison uses RSNR, UIQI, SAM, ERGAS, and DD on the AVIRIS dataset.
Uncertainty estimation: The proposed method also estimates the covariance matrix and noise variances from MCMC samples.The generated samples can additionally be used to compute confidence intervals for estimators.
Noise-variance estimation: The estimated noise powers track their variations within tolerable discrepancy.The estimates are obtained from samples generated by the Gibbs sampler and illustrated for HS and MS images.

2. The corresponding

The method remains robust when the spectral response in the forward model is imperfect and is also evaluated on ROSIS data and HS+PAN pansharpening. The paper concludes that it competes with state-of-the-art fusion methods while supporting broader multi-band fusion settings.

Robustness to F2 uncertainty: As FSNR decreases, uncertainty about F2 increases and the proposed algorithm’s performance decreases.FSNR is used to control the uncertainty in the spectral response.
Robustness to F2 uncertainty: Above 8dB FSNR, the proposed method outperforms the MAP and wavelet-based MAP methods.This result is reported for reconstruction performance under imperfect knowledge of F2.
HS+PAN pansharpening: The proposed algorithm is extended to fuse hyperspectral and panchromatic images, the HS+PAN extension of conventional MS+PAN pansharpening.The PAN image is formed by averaging all high-resolution HS bands and has SNR 30dB.
Conclusions: The hierarchical Bayesian model fuses multiple multi-band images with different spectral and spatial resolutions through a fully unsupervised algorithm.
Conclusions: Simulations on pseudo-real data show competition with state-of-the-art MS–HS fusion techniques and robustness to forward-model misspecification.The posterior is sampled using a Hamiltonian Monte Carlo algorithm.