Source-linked AI summary
Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations
Lina Zhuang, Jose M. Bioucas-Dias
TL;DR
HSI restoration is challenged by degradation and costly existing algorithms. This paper introduces FastHyDe and FastHyIn, which combine learned low-rank subspaces with self-similar eigen-image denoising for denoising and inpainting. Experiments report similar or better performance than state-of-the-art methods for additive and Poissonian noise, with the methods also fast, robust to subspace overestimation, and user-friendly.
Problem
Existing HSI denoising and inpainting algorithms are often time-consuming because of large image sizes and iterative spatial-spectral estimation.
Method
FastHyDe and FastHyIn exploit low-rank subspace representations, self-similar eigen-images, sparse patch representations, and plug-and-play denoisers.
Results
FastHyDe and FastHyIn achieve similar or better performance than state-of-the-art methods for additive and Poissonian noise.
Takeaways & Limitations
The methods are fast, robust to subspace overestimation, and user-friendly, requiring no parameters that are hard to tune.
Abstract
from arXiv · showhide
This paper introduces two very fast and competitive hyperspectral image (HSI) restoration algorithms: fast hyperspectral denoising (FastHyDe), a denoising algorithm able to cope with Gaussian and Poissonian noise, and fast hyperspectral inpainting (FastHyIn), an inpainting algorithm to restore HSIs where some observations from known pixels in some known bands are missing. FastHyDe and FastHyIn fully exploit extremely compact and sparse HSI representations linked with their low-rank and self-similarity characteristics. In a series of experiments with simulated and real data, the newly introduced FastHyDe and FastHyIn compete with the state-of-the-art methods, with much lower computational complexity.
I. INTRODUCTION
HSI restoration is motivated by severe, varied degradation and the computational burden of existing methods. The paper introduces FastHyDe and FastHyIn using low-rank, self-similar, and sparse HSI structure.
- HSIs provide high spectral resolution for material identification but suffer degradation from electronic, Poissonian, quantization, stripe, and atmospheric noise.
- Poissonian noise is increasingly important as spectral bandwidth decreases, while variance-stabilizing transformations can approximately convert sufficiently strong photon counts into additive Gaussian noise.
- Existing HSI methods exploit spatial, spectral, temporal, or low-rank coherence, while single-band approaches use self-similarity and non-local patch processing.
- Many published HSI denoising and inpainting algorithms are time-consuming because of large data sizes and iterative spatial-spectral estimation.
- FastHyDe and FastHyIn exploit low-rank subspace representations, self-similar eigen-images, and sparse representations of similar non-local patches.
- The paper extends prior work with a detailed FastHyDe treatment, the FastHyIn inpainting algorithm, and extensive experiments and comparisons.
II. FORMULATION AND PROPOSED DENOISER
The proposed denoiser projects HSI data into a learned low-dimensional subspace, denoises self-similar eigen-images independently, and reconstructs the clean image. FastHyDe uses plug-and-play denoisers, while FastHyIn extends the approach to known missing observations.
- II. FORMULATION AND PROPOSED DENOISER: HSI spectral vectors are modeled in a k-dimensional subspace with nb ≫ k, yielding the representation X = EZ.
- II. FORMULATION AND PROPOSED DENOISER: The basis E can be learned with HySime or SVD, while the rows of Z are called eigen-images.
- II. FORMULATION AND PROPOSED DENOISER: Eigen-images are self-similar because they are linear combinations of HSI bands sharing the same spatial self-similarity structure.
- A. Additive Gaussian i.i.d. noise: For additive Gaussian noise, FastHyDe exploits subspace projection and denoises eigen-images with non-local patch-based methods.
- A. Additive Gaussian i.i.d. noise: The objective combines Gaussian-noise data fidelity with a regularizer expressing prior information for self-similar images.
- A. Additive Gaussian i.i.d. noise: Plug-and-play replaces explicitly designed proximity operators with state-of-the-art self-similarity denoisers, and this work selects BM3D for its speed and performance.
- A. Additive Gaussian i.i.d. noise: FastHyDe learns E, computes noisy eigen-images, denoises them componentwise, and reconstructs the clean HSI.
- A. Additive Gaussian i.i.d. noise: FastHyDe remains extremely robust to subspace-dimension estimation errors provided the dimension is not underestimated.
B. Additive Gaussian non-i.i.d. noise
The non-i.i.d. Gaussian case is converted to an i.i.d. problem by whitening observations with the noise covariance, after which the denoising procedure is applied in a learned subspace and transformed back.
- B. Additive Gaussian non-i.i.d. noise: The model assumes zero-mean additive Gaussian noise that is pixelwise independent, with spectral covariance Cλ that is positive definite.Positive definiteness ensures that the covariance is non-singular and can be used for whitening.
- B. Additive Gaussian non-i.i.d. noise: Whitening uses the positive-definite noise covariance Cλ to convert spectrally correlated noise into an i.i.d. noise model.The i.i.d. case has Cλ = σ2I, whereas the non-i.i.d. case may have a non-diagonal covariance.
- B. Additive Gaussian non-i.i.d. noise: The whitened denoising problem is formulated using an orthonormal basis eE learned from the whitened observations eY.The resulting clean whitened data is estimated by following the same steps used by Algorithm 1.
- B. Additive Gaussian non-i.i.d. noise: The estimated clean data in the original domain is recovered from the denoised whitened estimate eX.This completes the transformation back from the whitened representation.
III. FAST INPAINTING
FastHyIn formulates hyperspectral inpainting as recovery of a vectorized clean image from incomplete noisy observations selected by a user-provided mask, while considering multiple noise models.
- III. FAST INPAINTING: The vectorized clean HSI x contains t = nb × n components, while y contains q ≤ t observed noisy measurements.The noise vector n has the same q-dimensional observation space as y.
- III. FAST INPAINTING: The mask M selects a subset of x’s components and is a binary matrix formed from rows of the identity matrix.M has one 1 per row and is assumed to be supplied by the user.
- III. FAST INPAINTING: The method considers Gaussian i.i.d. noise first, followed by Gaussian non-i.i.d. and Poissonian noise.
A. Additive Gaussian i.i.d. noise
For Gaussian i.i.d. noise, FastHyIn exploits the subspace model to estimate coefficients at incomplete pixels, denoises the resulting eigen-images, and reconstructs the clean HSI without iterative inversion of the full operator.
- A. Additive Gaussian i.i.d. noise: Assuming X = EZ, FastHyIn estimates each incomplete pixel’s subspace coefficients from its observed components before reconstruction.The observed vector at pixel i contains ni ≤ nb components and corresponding noise components.
- A. Additive Gaussian i.i.d. noise: FastHyIn avoids iterative solvers for the non-diagonal inpainting operator by using a suboptimal solution described as very fast and effective.The approach operates pixelwise through submatrices Mi of the mask.
- A. Additive Gaussian i.i.d. noise: The coefficient estimate is a least-squares solution requiring ET MT_i MiE to be non-singular, which is ensured here by ni ≥ k.Thus, each incomplete pixel must provide at least as many observed components as the subspace dimension.
- A. Additive Gaussian i.i.d. noise: After estimating coefficients, FastHyIn reconstructs each pixel and solves a denoising problem on the recovered observations.The reconstructed observations are vectorized as by before the denoising formulation is applied.
- A. Additive Gaussian i.i.d. noise: FastHyIn applies FastHyDe to the coefficient-denoising optimization and reduces to FastHyDe when the inpainting mask is M = I.Algorithm 2 learns the subspace, computes eigen-images, denoises them component by component, and reconstructs bX = EbZ.
- A. Additive Gaussian i.i.d. noise: Coefficient computation is needed only for incompletely observed pixels because fully observed pixels directly provide bzi = ET yi.When ni = nb, Mi is diagonal and the estimate equals the vector used in FastHyDe.
B. Additive Gaussian non-i.i.d. noise
For non-i.i.d. Gaussian inpainting, FastHyIn accounts for band correlations through covariance-aware coefficient estimation, then whitens the observations and applies eigen-image denoising in the whitened subspace.
- B. Additive Gaussian non-i.i.d. noise: Non-i.i.d. coefficient estimation uses the pixel-specific covariance Cλ,i = MiCλMT_i in a maximum-likelihood formulation.This accounts for bandwise dependence while retaining pixelwise independence.
- B. Additive Gaussian non-i.i.d. noise: The non-i.i.d. algorithm computes coefficients only at incomplete pixels, replaces their observations with reconstructed values, and then whitens the data.These operations are summarized in Algorithm 3.
- B. Additive Gaussian non-i.i.d. noise: After whitening, FastHyIn relearns a basis for the whitened data, denoises the eigen-images one by one, and reconstructs the clean HSI.The reconstruction is bX = √CλeEbZ.
- B. Additive Gaussian non-i.i.d. noise: Algorithm 3 covers pixelwise-independent Gaussian noise with possibly non-diagonal band covariance Cλ.A diagonal Cλ permits a simplified procedure that whitens incomplete observed pixels directly.
A. Denoising experiments
Denoising experiments evaluate FastHyDe on semi-synthetic Pavia Centre and Washington DC Mall HSIs under Gaussian and Poissonian noise. FastHyDe achieves the best reported quantitative performance across noise types and remains robust when the subspace dimension is overestimated.
- Datasets and setup: The experiments use Pavia Centre and Washington DC Mall subscenes projected onto signal subspaces of dimension 8 after removing 23 low-SNR water-vapor absorption bands.The resulting projected images are treated as clean data before Gaussian or Poissonian noise is added.
- Noise models: Three additive-noise cases are considered: Gaussian i.i.d., Gaussian non-i.i.d. with diagonal covariance, and Poissonian noise at 15 dB SNR.Non-i.i.d. and Poissonian observations are transformed to approximately additive i.i.d. noise before comparison methods are applied.
- Evaluation: FastHyDe is compared with BM3D, BM4D, PCA+BM4D, and NAILRMA using mean PSNR and mean SSIM across bands.The competing methods receive noise estimates from HySime, while FastHyDe parameters include the noise covariance matrix and signal subspace.
- Results: FastHyDe yields uniformly the best performance across the different noise types, with gains increasing as the noise level increases.The paper evaluates performance using PSNR and SSIM-derived mean measures reported for both datasets.
- Computational cost: Denoising only the eigen-images reduces FastHyDe complexity relative to competing methods because their number is much smaller than the number of spectral bands.The implementation uses MATLAB R2010 on a desktop with eight Intel Core i7-4970 CPUs and 16 GB of RAM.
- Robustness: FastHyDe PSNR is practically constant when the estimated subspace dimension is not underestimated, demonstrating robustness to subspace overestimation.Figure 2 varies the estimated dimension around the true subspace dimension of 8 and plots signal power, noise power, and FastHyDe PSNR.
B. Inpainting experiments
Inpainting experiments evaluate FastHyIn on corrupted Washington DC Mall and Pavia Centre images containing stripes under Gaussian and Poissonian noise. FastHyIn removes noise while inpainting and reports the best MPSNR and MSSIM, while comparison methods show dataset- or noise-specific shortcomings.
- Experimental setup: FastHyIn is compared with PDE, UBD, and LRTV for restoring corrupted hyperspectral images with simulated stripes.The corrupted inputs reuse Gaussian i.i.d., Gaussian non-i.i.d., and Poissonian noise cases, with stripes introduced in four bands.
- Experimental setup: The missing-value locations are known beforehand and supplied to PDE, LRTV, and FastHyIn as an input variable.This establishes a known-mask setting for the inpainting comparisons.
- Visual results: For Washington DC Mall, almost all methods inpaint the dead lines visually except LRTV in Case 1.The paper reports these comparisons in Figures 7 and 8.
- Noise removal: Unlike PDE, FastHyIn, UBD, and LRTV remove substantial noise while performing inpainting.PDE functions only as an inpainter, whereas the other methods also suppress noise in these experiments.
- Quantitative results: FastHyIn yields uniformly the best MPSNR and MSSIM results, while remaining user-friendly with parameters computed by HySime.Robustness is illustrated by using subspace dimension k = 10 although the true value is k = 8.
V. EVALUATION WITH REAL DATA
Real-data experiments evaluate FastHyDe and FastHyIn on noisy and corrupted hyperspectral scenes, including denoising, destriping, and inpainting. FastHyDe qualitatively recovers more information in noisy bands, while FastHyIn produces better inpainting results than the other inpainters.
- FastHyDe was applied to the 145 × 145-pixel, 220-channel Indian Pines scene, whose noise was assumed to be non-i.i.d. and estimated with HySime.
- FastHyDe was compared with BM3D, BM4D, PCA+BM4D, and NAILRMA on Indian Pines, using subspace dimensions of 18 for PCA+BM4D and 25 for FastHyDe.The FastHyDe setting reflects its reported robustness to subspace-dimension overestimation.
- Denoising: The denoising figures assess band 70 under Gaussian i.i.d., Gaussian non-i.i.d., and Poissonian noise for Washington DC Mall and Pavia Centre.
- Destriping: The destriping figures assess band 60 under Gaussian i.i.d., Gaussian non-i.i.d., and Poissonian noise for Washington DC Mall and Pavia Centre.
- Real-data denoising: On Indian Pines, FastHyDe qualitatively recovers more information in noisy bands than BM3D, BM4D, PCA+BM4D, and NAILRMA.
- Real-data inpainting: On Urban data, FastHyIn produces qualitatively better inpainting results than PDE, UBD, and LRTV for bands containing almost no useful information.
VI. CONCLUSIONS
The paper proposes FastHyDe for hyperspectral denoising and FastHyIn for inpainting. Both exploit low-dimensional and self-similar HSI representations to achieve similar or better performance with much lower computational complexity.
- FastHyDe and FastHyIn exploit low-dimensional subspaces and self-similar eigen-images, enabling compact and sparse representations for restoration.
- FastHyDe and FastHyIn achieve similar or better performance than state-of-the-art methods for additive and Poissonian noise, with much lower computational complexity.
- The methods are reported to be robust to subspace overestimation and require no parameters that are hard to tune.