PaperlayerView exact revision on arXiv

Source-linked AI summary

Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration

Yi Chang, Luxin Yan, Houzhang Fang, Sheng Zhong, Zhijun Zhang

arXiv:1709.00192v1cs.CV

TL;DR

HSI restoration must address multiple degradations while preserving 3-D spectral-spatial structure, but prior methods often focus on one task or use vector and matrix models. The paper introduces unified weighted low-rank tensor recovery, including WLRTR-RPCA for stripe noise, and reports consistent state-of-the-art performance across restoration tasks.

  • Problem

    HSIs undergo noise, blur, and spatial or spectral downsampling, while existing methods often target one task and fail to fully exploit 3-D spectral-spatial correlation.

  • Method

    WLRTR constructs third-order tensors capturing spatial sparsity, non-local spectral-spatial similarity, and spectral correlation, with weighted singular values and an RPCA extension for stripe noise.

  • Results

    The proposed WLRTR models consistently outperform state-of-the-art methods across denoising, destriping, deblurring, and super-resolution experiments.

  • Takeaways & Limitations

    A unified low-rank tensor framework can support diverse HSI restoration tasks while preserving intrinsic spectral-spatial structure.

  • Takeaways & Limitations

    Joint HSI destriping and deblurring is outside the scope of the work.

Abstract

from arXiv · show

Hyperspectral imaging, providing abundant spatial and spectral information simultaneously, has attracted a lot of interest in recent years. Unfortunately, due to the hardware limitations, the hyperspectral image (HSI) is vulnerable to various degradations, such noises (random noise, HSI denoising), blurs (Gaussian and uniform blur, HSI deblurring), and down-sampled (both spectral and spatial downsample, HSI super-resolution). Previous HSI restoration methods are designed for one specific task only. Besides, most of them start from the 1-D vector or 2-D matrix models and cannot fully exploit the structurally spectral-spatial correlation in 3-D HSI. To overcome these limitations, in this work, we propose a unified low-rank tensor recovery model for comprehensive HSI restoration tasks, in which non-local similarity between spectral-spatial cubic and spectral correlation are simultaneously captured by 3-order tensors. Further, to improve the capability and flexibility, we formulate it as a weighted low-rank tensor recovery (WLRTR) model by treating the singular values differently, and study its analytical solution. We also consider the exclusive stripe noise in HSI as the gross error by extending WLRTR to robust principal component analysis (WLRTR-RPCA). Extensive experiments demonstrate the proposed WLRTR models consistently outperform state-of-the-arts in typical low level vision HSI tasks, including denoising, destriping, deblurring and super-resolution.

1 Introduction

HSI restoration is challenged by diverse degradations and methods that lose 3-D spectral-spatial structure or target only individual tasks. The paper proposes a unified weighted low-rank tensor framework covering multiple restoration problems.

  • HSIs suffer random noise, stripe noise, blurring, and low spatial resolution, while hardware improvements are impractical for quality enhancement.
  • The model represents denoising, destriping, mixed-noise removal, deblurring, and super-resolution through different degradation operators and error settings.
  • Vector- and matrix-based restoration methods can lose spectral-spatial correlation, while some tensor methods are heuristic and difficult to extend across tasks.
  • The proposed framework uses a unified tensor low-rank prior for denoising, destriping, deblurring, and super-resolution.
  • The constructed tensor jointly models spatial sparsity, non-local spectral-spatial similarity, and spectral correlation.
  • WLRTR assigns different weights to core-tensor singular values and provides an analytical solution, while WLRTR-RPCA models stripe noise as gross error using the L211 norm.

2 Related work

The related work covers HSI denoising, destriping, deblurring, and super-resolution, highlighting limitations in existing priors, dimensional representations, and tensor formulations.

  • HSI Denoising: Prior HSI denoising methods separately model spectral correlation or nonlocal self-similarity, while tensor approaches that combine them remain heuristic or task-specific.The paper introduces reweighting to encode fine-grained sparsity in core tensor coefficients and improve flexibility across restoration tasks.
  • HSI Destriping: Existing destriping methods either treat stripes as Gaussian-like noise or estimate stripe components separately, while most matrix-based methods miss directional structure.The related work motivates tensor RPCA with a directional induced L211 norm for jointly estimating the clean image and stripe component.
  • HSI Deblurring: Most HSI deblurring methods use spatial priors such as gradients, Huber-Markov, TV, or GMM, without exploiting HSI nonlocal self-similarity.The paper focuses on non-blind deblurring and incorporates spectral correlation and nonlocal information.
  • HSI Super-resolution: HSI super-resolution fuses low-spatial, high-spectral HSI with high-spatial, low-spectral panchromatic or multispectral imagery, commonly using sparse representation or matrix factorization.The related work describes tensor-based alternatives, including NLSTF, but notes limitations in auxiliary RGB use and flexibility.
  • HSI Super-resolution: Unlike NLSTF, the proposed WLRTR uses HOSVD with a weighted low-rank tensor prior and supports auxiliary high-resolution RGB information.This combination is presented as more flexible and as refining the underlying high-resolution HSI prior.

3 Weighted Low-rank Tensor Recovery Model

This section develops a tensor-based restoration framework that preserves multidimensional HSI structure, constructs low-rank tensors from similar cubic patches, and applies weighted singular-value regularization. The resulting model is motivated by the sparsity of tensor singular values and shows improved recovery over matrix alternatives.

  • Notations and Preliminaries: Tensor decomposition represents an HSI as a core tensor multiplied by orthogonal factor matrices along each mode.The core tensor captures interactions among components, while factor matrices represent principal components in each mode.
  • Why Low-rank Tensor Recovery: Matrix recovery models exploit singular-value sparsity, but vectorizing or matricizing HSI data can lose spectral-spatial structural correlation.Clean patches exhibit sparser, exponentially decaying singular values than noisy patches, motivating low-rank recovery.
  • Low-rank Tensor Construction: The constructed tensor groups a key m × m × b cubic with k nearest non-local cubics, producing dimensions m^2 × (k + 1) × b.This construction jointly uses spatial local sparsity, non-local similarity, and spectral correlation; setting b = 0 or k = 0 yields matrix cases.
  • Low-rank Tensor Construction: Using a 4-order tensor slightly degrades denoising because short spatial patches hinder HOSVD from extracting intrinsic spatial subspace bases.The paper therefore motivates the 3-order construction despite the potential for richer spatial relationships.
  • Weighted Low-rank Tensor Recovery Model: Weighted tensor recovery penalizes smaller singular values more heavily while penalizing larger values less to preserve major data components.The weighted nuclear norm assigns non-negative weights to singular values, and the paper provides an analytical solution.

4 HSI Restoration With WLRTR Model

The paper develops WLRTR-based procedures for denoising, destriping, deblurring, and super-resolution by combining tensor low-rank priors with task-specific degradation and noise models.

  • HSI denoising: HSI denoising combines an identity degradation operator, random-noise fidelity, and the WLRTR prior in a minimization problem.
  • HSI denoising: Alternating minimization updates low-rank tensors, reconstructs the whole image from overlapping cubics, and iterates until convergence.The reconstruction subproblem has a closed-form solution, and the process is summarized in Algorithm 1.
  • HSI destriping: WLRTR-RPCA models stripe noise as a sparse error using a direction-discriminative L2,1-norm extended to the third-order tensor case.The column-sparsity assumption corresponds to broken and intact detectors.
  • HSI destriping: Mode-1 tensor unfolding preserves the directional characteristic needed for estimating stripe errors, whereas mode-2 and mode-3 unfolding may lose it.
  • HSI deblurring: For deblurring, ADMM separates image deconvolution from denoising, with 3-D FFTs used to compute the convolutional subproblem efficiently.The image deconvolution step has a closed-form frequency-domain solution.
  • HSI super-resolution: Super-resolution handles spatial blurring and downsampling together with spectral downsampling, using ADMM auxiliary variables to decouple data-fidelity terms from regularization.

5 Experimental results

The experiments evaluate the proposed methods against competing state-of-the-art HSI restoration methods using simulated and real benchmark datasets.

  • Experiments compare competing state-of-the-art HSI restoration methods and report results on simulated and real benchmark datasets.

5.1 Experimental Setting

The evaluation spans hyperspectral, color-image, and MRI datasets, with standardized preprocessing, parameter settings, and multiple quantitative quality indexes.

  • Benchmark Datasets: The study tests HSIs, three-channel color images, and MRIs across CAVE, BSD, HHD, PMRI, Salinas, and Gulf Wetlands datasets.
  • Pre-processing: All original images are converted to an 8-bit scale, while band-averaged cubics support nonlocal matching with lower computational load.
  • Evaluation Indexes: Evaluation uses PSNR, SSIM, ERGAS, and SAM to assess denoising performance.

5.2 HSI Denoising

On simulated and real HSI denoising, WLRTR produces clearer reconstructions and preserves fine texture, while quantitative evaluations show consistent superiority across tested comparisons.

  • Visual Results: WLRTR reconstructs clearer regions and tiny hair texture, while competing methods produce more artifacts at high noise levels.
  • Quantitative Results: WLRTR achieves the best PSNR, SSIM, ERGAS, and SAM results on CAVE, with its advantage over BM4D almost exceeding 4.3dB at σ = 100.
  • Quantitative Results: WLRTR obtains the highest average PSNR for each CAVE scene and significantly higher PSNR across all bands of the example image toy.
  • Visual Results: At noise levels σ = 30 and σ = 50 on CAVE images, WLRTR is compared visually with nine competing denoising methods.

5.3 Color Image Denoising

WLRTR jointly processes RGB channels to exploit cross-channel spectral correlation, producing better detail preservation and denoising quality than competing methods on BSD.

  • WLRTR jointly processes RGB channels instead of restoring each channel separately, exploiting spectral correlation between channels.
  • Under noise level σ = 40 on BSD, WLRTR preserves more texture and edge details than WNNM.
  • WLRTR better preserves image details with less chrominance color artifacts than the other competing color-image denoising methods.
  • Joint RGB multichannel processing improves color-image denoising performance, as reflected by PSNR and SSIM comparisons on BSD.

5.4 HSI Destriping

WLRTR-RPCA is evaluated for mixed random-and-stripe noise and real multiplicative stripe noise, where it removes stripes while preserving image structure and avoiding artifacts.

  • Under heavy mixed noise on Salinas, residual stripes remain in several competing methods’ results.
  • WLRTR-RPCA preserves detailed image structure without introducing noticeable artifacts in mixed-noise removal.
  • For real multiplicative stripe noise, only PARAFAC, LRMR, and WLRTR-RPCA work well, while the other methods fail to remove the stripes.
  • Unlike PARAFAC and LRMR, WLRTR-RPCA decouples stripe noise as a sparse error component from image components without the reported detail damage or false artifacts.

5.5 HSI Deblurring

DB-WLRTR is compared with single-image and HSI deblurring methods on CAVE under known Gaussian blur, showing an advantage across tested blur levels.

  • HSI deblurring has received relatively little research attention compared with other HSI restoration tasks.
  • On CAVE, DB-WLRTR is compared with HL, FPD, and SSTV under different Gaussian blur levels using a known point spread function.
  • DB-WLRTR has an overwhelming advantage over the competing methods under the tested Gaussian blur levels.

5.6 HSI Super-resolution

SR-WLRTR is evaluated on CAVE under Gaussian and uniform blur with scaling factor s = 8, outperforming matrix- and tensor-based competitors in spatial-spectral reconstruction.

  • SR-WLRTR is compared with CNMF, NSSR, and NLSTF under Gaussian and uniform blur at scaling factor s = 8.
  • SR-WLRTR outperforms competing methods in both spatial and spectral aspects, especially under Gaussian blur.
  • At 700nm, SR-WLRTR achieves the smallest reconstruction errors, particularly around sharp edges.
  • Compared with matrix-based methods, SR-WLRTR better preserves spatial-spectral structures with improved spatial details and less spectral distortion.
  • Compared with NLSTF, SR-WLRTR produces more visually pleasing results, while NLSTF exhibits obvious gridding artifacts.

5.7 Discussion

The discussion examines how band count, non-local similarity cubics, band consistency, and convergence affect WLRTR restoration, alongside comparisons with competing methods. Results indicate performance gains from more bands and cubics up to practical limits, preserved spectral information under inconsistency, and rapid convergence.

  • Band number: PSNR values increase with band count, rapidly below 20 bands and more slowly afterward, while memory and computation grow rapidly.The experiments therefore use fewer than 40 bands; a 512*512*31 image takes about 23 minutes on the reported MATLAB implementation.
  • Band consistency: WLRTR preserves interframe spectral information while removing noise from PMRI images with large spectral differences between frames.Reflectance spectra after denoising retain the interframe information with slight differences attributed to noise reduction.
  • Non-local similarity cubics: PSNR reaches its highest level with 150 to 200 non-local similarity cubics, while performance deteriorates slightly beyond 230.The authors attribute the deterioration to insufficient similarity between the target cubic and searching cubics and set the number between 100 and 200.
  • Convergence: Functional energy decreases rapidly and monotonically while PSNR rapidly reaches a stable value, with convergence typically occurring in 3 or 4 steps.This empirical convergence analysis uses the evolution of functional energy and PSNR across iterations.
  • Overall comparison: Across HSI restoration tasks and other multispectral images, WLRTR and its extensions obtain better visual results with fewer artifacts than compared methods.The reported explanation is that the constructed 3-order tensor captures spatial sparsity, non-local cubic redundancy, band consistency, and weighted core-tensor sparsity.

6 Conclusion

The conclusion presents WLRTR as a unified weighted low-rank tensor framework for HSI restoration that models spatial-spectral structure and supports multiple tasks. Experiments report consistent advantages over state-of-the-art methods, while future work targets computational acceleration.

  • Contribution: WLRTR models spatial sparsity, non-local spatial-spectral cubic redundancy, and spectral consistency through high-order low-rank properties of constructed 3-order subtensors.The framework is intended to preserve spatial-spectral correlation across HSI restoration problems.
  • Unified optimization: Variable splitting unifies diverse HSI restoration problems and transforms them into easier subproblems with closed-form solutions.The weighted strategy is introduced for the low-rank tensor prior subproblem, whose closed-form solution is analyzed.
  • Stripe noise: WLRTR-RPCA incorporates the structural and directional properties of common HSI stripe noise into the recovery framework.This extension treats stripe noise within the proposed robust recovery formulation.
  • Experimental conclusion: Extensive simulated and real experiments report consistent outperformance of state-of-the-art methods across HSI restoration tasks in quantitative and visual assessments.The strongest reported advantages are in destriping, deblurring, and super-resolution, where few tensor methods had been proposed.
  • Future work: Future work will speed WLRTR through parallel implementation and reduced computational complexity.The conclusion also identifies compressed sensing, unmixing, and video applications as possible extensions.
Loading 1709.00192v1…