PaperlayerView exact revision on arXiv

Source-linked AI summary

FastHyMix: Fast and Parameter-free Hyperspectral Image Mixed Noise Removal

Lina Zhuang, Michael K. Ng

arXiv:2109.08879v1eess.IVcs.CV

TL;DR

Narrow spectral bands reduce SNR and the reliability of information measured from hyperspectral images, while mixed noise can combine Gaussian, impulse, and stripe-like components. FastHyMix models that noise with a Gaussian mixture and uses subspace representation and a deep image prior; the authors report similar or better performance for complex mixed noise with much shorter running time.

  • Problem

    Narrower spectral bands reduce SNR and the reliability of HSI features, while mixed noise includes Gaussian, impulse, and deadline or stripe components.

    Source: §I · p. 1, B. Contributions · p. 2

  • Method

    FastHyMix uses a Gaussian mixture model to estimate Gaussian-noise intensity and sparse-noise locations, then exploits spectral low-rankness and spatial correlation through subspace representation and an FFDNet-derived deep image prior.

    Source: B. Contributions · p. 2

  • Results

    The authors report similar or better performance for complex mixed noise than state-of-the-art algorithms, with much shorter running time.

    Source: §VII · p. 13

  • Takeaways & Limitations

    FastHyMix adapts its parameters to noise statistics and is reported to handle high- and low-intensity noise and different stripe values.

    Source: F. Robustness of FastHyMix to… · p. 12, §VII · p. 13

Abstract

from arXiv · show

Hyperspectral imaging with high spectral resolution plays an important role in finding objects, identifying materials, or detecting processes. The decrease of the widths of spectral bands leads to a decrease in the signal-to-noise ratio (SNR) of measurements. The decreased SNR reduces the reliability of measured features or information extracted from HSIs. Furthermore, the image degradations linked with various mechanisms also result in different types of noise, such as Gaussian noise, impulse noise, deadlines, and stripes. This paper introduces a fast and parameter-free hyperspectral image mixed noise removal method (termed FastHyMix), which characterizes the complex distribution of mixed noise by using a Gaussian mixture model and exploits two main characteristics of hyperspectral data, namely low-rankness in the spectral domain and high correlation in the spatial domain. The Gaussian mixture model enables us to make a good estimation of Gaussian noise intensity and the location of sparse noise. The proposed method takes advantage of the low-rankness using subspace representation and the spatial correlation of HSIs by adding a powerful deep image prior, which is extracted from a neural denoising network. An exhaustive array of experiments and comparisons with state-of-the-art denoisers were carried out. The experimental results show significant improvement in both synthetic and real datasets. A MATLAB demo of this work will be available at https://github.com/LinaZhuang for the sake of reproducibility.

I. INTRODUCTION

Narrow spectral bands reduce photon capture and SNR, weakening the reliability of HSI information, while real mixed noise varies in type and distribution. FastHyMix addresses this with adaptive noise modeling, spectral subspace representation, and a deep image prior.

I. INTRODUCTION

  • Narrower spectral bands mean fewer photons and lower SNR, reducing the reliability of features or information extracted from HSIs.The paper frames HSI denoising as fundamental preprocessing for subsequent applications.Source: §I · p. 1

A. Related work

  • Prior approaches exploit HSI low-rankness or sparse representations, but the paper identifies limited robustness to mixed noise and difficulty separating low-rank clean HSIs from low-rank stripes.It also motivates adaptive noise modeling because real noise distributions and noise types vary across imaging conditions.Source: §I · pp. 1–2
  • FastHyMix estimates the spectral subspace and HSI coefficients separately, avoiding the poor subspace estimates associated with their joint estimation under severe mixed noise.The paper contrasts its separate estimation strategy with iterative joint estimation approaches.Source: A. Related work · p. 2

B. Contributions

  • FastHyMix fits mixed noise with a Gaussian mixture model to estimate per-band Gaussian noise intensity and locate sparse noise.The method targets Gaussian noise, impulse noise, and deadlines or stripes, whose type and distribution can vary with imaging conditions.Source: §I · pp. 1–2, B. Contributions · p. 2
  • FastHyMix is described as parameter-free and user-friendly, with parameters adapted to image-specific noise statistics.The proposed framework combines traditional machine-learning techniques with a neural-network-derived prior.Source: B. Contributions · p. 2
  • The method embeds an FFDNet-derived deep image prior to regularize HSI eigen-images, and experiments report improved clean-HSI estimation accuracy.The authors motivate this choice partly by the lack of real HSIs for training hyperspectral denoisers and use a network trained on RGB images.Source: B. Contributions · p. 2, A. Related work · p. 2

II. PROBLEM FORMULATION

The formulation models an observed HSI as a clean image plus additive mixed noise, and represents the clean image through a low-dimensional spectral subspace. The following noise-estimation procedure uses bandwise coarse-noise analysis and a Gaussian mixture model when a Gaussian fit is inadequate.

II. PROBLEM FORMULATION

  • The observation model assumes additive noise comprising Gaussian noise, stripes, deadlines, and impulse noise.The paper notes that real-HSI Gaussian noise may be pixel-wise independent but band-wise dependent.Source: §II · p. 3
  • A clean HSI is represented in a low-dimensional spectral subspace, with an orthogonal basis and coefficient tensor whose mode-3 slices are called eigen-images.The representation follows from the high spectral correlation of hyperspectral vectors.Source: §II · p. 3
  • FastHyMix models complex mixed noise with a Gaussian mixture model, then uses its noise statistics in an adaptive removal algorithm based on subspace representation and deep image prior.The paper describes the Gaussian mixture as a flexible approximation for a wide range of noise distributions.Source: §II · p. 3

III. HYPERSPECTRAL MIXED NOISE ESTIMATION

  • Noise-statistics estimation proceeds band by band, beginning with a coarse noise estimate from linear regression that exploits spectral correlation.The procedure is introduced as estimating Gaussian noise intensity and sparse-noise locations.Source: §III · p. 3
  • A normality test checks whether each band’s coarse-noise histogram is adequately modeled by a Gaussian distribution.Bands with an approximately Gaussian histogram are treated as containing Gaussian noise.Source: §III · p. 3
  • When a coarse-noise histogram is not well modeled by a Gaussian, the method fits a two-component Gaussian mixture to partition Gaussian and sparse noise.The partition supports estimating per-band Gaussian-noise standard deviation and sparse-noise locations.Source: §III · p. 3

A. Estimation of noise statistics

The method estimates band-wise noise statistics by predicting each spectral band from the others and analyzing the resulting regression errors. A Gaussian fit and normality checks distinguish Gaussian noise from mixed noise.

A. Estimation of noise statistics

  • Highly correlated spectral bands are modeled by predicting each band as a linear combination of the others, with regression error serving as a coarse estimate of uncorrelated noise.The regression coefficients are estimated by least squares.Source: A. Estimation of noise statistics · p. 3

B. Standard deviation estimation of Gaussian noise

  • A band is treated as Gaussian when its coarse-noise histogram is Gaussian; skewness below 3 and kurtosis below 10 in absolute value are the stated normality checks.For a Gaussian band, noise standard deviation is estimated from the coarse-noise vector.Source: B. Standard deviation estimation of… · p. 3
  • Two real-HSI bands are treated as Gaussian-noise examples because their coarse-noise histograms fit Gaussian distributions.The cited passage does not identify the bands or datasets by name.Source: B. Standard deviation estimation of… · p. 4

C. Detection of pixels corrupted by mixed noise

When a band’s coarse-noise histogram is not well modeled by a Gaussian distribution, the method treats it as mixed Gaussian and sparse noise. It fits a two-component Gaussian mixture model and uses posterior probabilities to separate the noise groups.

C. Detection of pixels corrupted by mixed noise

  • A poor Gaussian fit to coarse noise signals a mixture of Gaussian and sparse noise, for which the method fits a Gaussian mixture model.The paper describes the mixture model as capable of modeling complex noise distributions.Source: C. Detection of pixels corrupted by… · p. 4

1) Gaussian mixture model fitted to mixed noise:

2) Cluster using Gaussian mixture model:

The method partitions coarse-noise elements into Gaussian and sparse groups by their estimated posterior probabilities.

2) Cluster using Gaussian mixture model:

D. Estimation of C and M

FastHyMix estimates Gaussian-noise statistics and sparse-noise locations from a Gaussian mixture clustering of coarse noise. It uses these estimates to guide noise whitening and mixed-noise removal.

D. Estimation of C and M

  • The Gaussian-noise standard deviation is estimated separately for each band from its assigned coarse-noise elements, yielding a diagonal covariance estimate.The passage defines the per-band estimate as the standard deviation of the set of Gaussian-noise elements and the covariance diagonal as the squared estimates.Source: D. Estimation of C and M · p. 6
  • The estimated mask marks each observation as Gaussian-only or mixed-noise, based on clustering each coarse-noise element.A value of 1 denotes Gaussian-only corruption; 0 denotes mixed noise.Source: D. Estimation of C and M · p. 7

IV. FASTHYMIX: FAST HSI MIXED NOISE REMOVAL

  • FastHyMix begins mixed-noise removal with a whitening step based on the estimated Gaussian-noise correlation matrix and sparse-noise locations.Source: §IV · p. 7

A. Gaussian noise-whitening transformation

FastHyMix whitens the HSI before learning its spectral subspace, then estimates subspace coefficients and denoises the resulting eigen-images. Its eigen-image estimation combines sparse-noise recovery with a deep image prior for spatially correlated data.

A. Gaussian noise-whitening transformation

  • Whitening addresses different Gaussian-noise intensities across bands while retaining a low-dimensional spectral subspace.Source: A. Gaussian noise-whitening… · p. 7
  • The spectral subspace is estimated by SVD of Gaussian-only observed pixels, and the method is robust to dimension errors provided the subspace dimension is not underestimated.The selected dimension P is the spectral-subspace dimension.Source: A. Gaussian noise-whitening… · p. 7

B. Estimation of eigen-images

  • Eigen-image estimation uses data fidelity for Gaussian noise and a regularizer tailored to spatially correlated eigen-images.Source: B. Estimation of eigen-images · p. 7
  • A fast approximate solution first estimates sparse-noise-corrupted components by least squares, requiring at least P Gaussian-only bands at each pixel, then removes remaining mainly Gaussian noise.The least-squares matrix is stated to be nonsingular when the number of Gaussian-only components is at least the subspace dimension.Source: B. Estimation of eigen-images · p. 7
  • After projection decouples the eigen-image problem across mode-3 slices, FastHyMix plugs in FFDNet with λ = 1 for whitened Gaussian noise.The paper selects FFDNet for speed after training and its ability to handle varying noise levels without retraining.Source: B. Estimation of eigen-images · pp. 7–8

C. Inverse noise-whitening transformation

FastHyMix applies an inverse noise-whitening transformation to obtain its clean-HSI estimate. The algorithm also specifies the sequence of estimation, denoising, reconstruction, and output steps.

C. Inverse noise-whitening transformation

V. EXPERIMENTS WITH SIMULATED IMAGES

The experiments use two simulated and two real hyperspectral datasets. Figure 6 shows the HSIs used in the experiments.

V. EXPERIMENTS WITH SIMULATED IMAGES

  • Experiments were conducted on two simulated and two real hyperspectral datasets.Source: §V · p. 8
  • The experiments therefore include both simulated and real HSI data.Source: §V · p. 8
  • Figure 6 presents the hyperspectral images used in the experiments.Source: Figure 6 · p. 8, §V · p. 8

A. Simulation of noisy datasets and comparisons

The experiments compare six denoisers on simulated Washington DC Mall and Pavia University data corrupted by four additive-noise cases. FastHyMix performs best across the tested noise types and runs in the shortest time.

A. Simulation of noisy datasets and comparisons

  • The tests use Washington DC Mall and Pavia University subregions, with noise cases spanning Gaussian noise alone, stripes, salt-and-pepper noise, and their combination.The compared methods include NG-meet, LRMR, NonRLRS, KBR, NMoG, and LRTF-DFR.Source: A. Simulation of noisy datasets and… · p. 8
  • Performance is assessed using per-band PSNR, SSIM, FSIM, and SAD, with their band-averaged forms reported for denoised images.The same subspace dimension is used for methods sharing that parameter, and regularization parameters are fine-tuned for the comparisons.Source: A. Simulation of noisy datasets and… · p. 9

B. Mixed noise removal

  • FastHyMix yields the best denoising performance in the shortest time across the two simulated datasets and four noise cases.The visual comparisons show comparable results for FastHyMix, KBR, and NMoG; heavy stripes remain after NG-meet denoising in mixed-noise cases.Source: B. Mixed noise removal · p. 9
  • NG-meet works well for Gaussian-only noise but not mixed noise, indicating that a Gaussian denoiser alone does not address the tested mixtures.The authors note that comparing NG-meet on mixed-noise cases is unfair because it was designed for Gaussian noise.Source: B. Mixed noise removal · p. 9

C. Estimation of mixed noise

FastHyMix estimates Gaussian noise intensity and sparse-noise locations, then uses deep image priors and subspace representations. Its reported tests show robustness to subspace overestimation and varied noise intensities and stripe values.

C. Estimation of mixed noise

  • A two-component Gaussian mixture model estimates per-band Gaussian noise intensity and identifies sparse-noise locations; its intensity estimates are closer to the truth than HySime’s in mixed-noise cases.Both methods estimate well in the Gaussian-only case, while FastHyMix accurately identifies sparse-noise pixels across the tested cases.Source: C. Estimation of mixed noise · p. 9, Figure 10 · p. 12, Figure 11 · p. 12

D. Deep prior for eigen-images

  • Filtering eigen-images with FFDNet considerably increases MPSNR, supporting the contribution of a deep image prior even though the network was trained on grayscale images.The authors relate its applicability to shared local and non-local similarity and piece-wise smoothness in grayscale images and HSIs.Source: D. Deep prior for eigen-images · p. 9, Figure 12 · p. 12

E. Robustness of FastHyMix to subspace overestimation

  • FastHyMix remains robust when the estimated subspace dimension is not underestimated, with MPSNR nearly constant as the dimension increases.The figure compares MPSNR across estimated dimensions for Washington DC Mall and Pavia University, whose true subspace dimension is 8.Source: E. Robustness of FastHyMix to subspace… · p. 11, Figure 13 · p. 12

F. Robustness of FastHyMix to different noise intensities and noise types

  • Across 12 mixed-noise cases, FastHyMix yields the best results under high- and low-intensity noise; it also outperforms comparison methods across maximum-, minimum-, and random-valued stripes.Cases 5–16 vary Gaussian, stripe, and salt-and-pepper noise intensity; cases 8, 17, and 18 vary stripe value.Source: F. Robustness of FastHyMix to… · pp. 11–12

VI. EXPERIMENTS WITH REAL IMAGES

Real-image tests report stripe and deadline removal on Tiangong-1 and Hyperion Cuprite data. The paper concludes that FastHyMix achieves similar or better performance for complex mixed noise with much shorter running time.

A. Tiangong-1 dataset

  • On Tiangong-1 data, FastHyMix alleviates stripe noise in the displayed bands and is reported to give the best qualitative result in the shortest time.The image is a 351 × 253 subregion; bands 29, 30, 31, and 59 contain obvious stripes, while LRTF-DFR shows incorrect illumination in bands 29 and 59.Source: A. Tiangong-1 dataset · p. 12

B. Hyperion Cuprite dataset

  • On Hyperion Cuprite data, FastHyMix and NMoG restore severe stripes well, while visible stripes remain with other methods; FastHyMix also runs much faster than the comparison methods.The tested subregion has 177 channels after water-vapor bands are removed; band 143 is mainly affected by deadlines.Source: B. Hyperion Cuprite dataset · p. 12

VII. CONCLUSION

  • FastHyMix achieves similar or better performance for complex mixed noise with much shorter running time than state-of-the-art algorithms.The paper also describes its regularization parameters as adaptively set to noise statistics.Source: §VII · p. 13
Loading 2109.08879v1…