Source-linked AI summary
Non-local Meets Global: An Iterative Paradigm for Hyperspectral Image Restoration
Wei He, Quanming Yao, Chao Li, Naoto Yokoya, Qibin Zhao, Hongyan Zhang, Liangpei Zhang
TL;DR
HSI restoration is an ill-posed problem, and existing non-local methods face increasing computational burden with spectral dimension while prior methods do not fully balance spatial and spectral structure. The paper proposes a unified iterative paradigm that learns a global spectral basis and applies non-local low-rank denoising to a reduced image, with alternating minimization and rank adaptation. Experiments on simulated and real data report best results across denoising, compressed reconstruction, and inpainting.
Problem
HSI restoration is ill-posed, while balancing spectral correlation with non-local similarity remains a problem.
Method
The method jointly updates a global orthogonal spectral basis and reduced image, using alternating minimization, non-local denoising, and rank adaptation.
Results
The unified model achieves the best results on denoising, compressed HSI reconstruction, and inpainting using simulated and real data.
Takeaways & Limitations
The paradigm combines spatial non-local similarity with global spectral low-rank structure while reducing the computational burden of high-dimensional non-local processing.
Takeaways & Limitations
The noise-reestimation result depends on an assumption; if it fails, performance can deteriorate substantially.
Abstract
from arXiv · showhide
Non-local low-rank tensor approximation has been developed as a state-of-the-art method for hyperspectral image (HSI) restoration, which includes the tasks of denoising, compressed HSI reconstruction and inpainting. Unfortunately, while its restoration performance benefits from more spectral bands, its runtime also substantially increases. In this paper, we claim that the HSI lies in a global spectral low-rank subspace, and the spectral subspaces of each full band patch group should lie in this global low-rank subspace. This motivates us to propose a unified paradigm combining the spatial and spectral properties for HSI restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity from the low-rank orthogonal basis exploration. An efficient alternating minimization algorithm with rank adaptation is developed. It is done by first solving a fidelity term-related problem for the update of a latent input image, and then learning a low-dimensional orthogonal basis and the related reduced image from the latent input image. Subsequently, non-local low-rank denoising is developed to refine the reduced image and orthogonal basis iteratively. Finally, the experiments on HSI denoising, compressed reconstruction, and inpainting tasks, with both simulated and real datasets, demonstrate its superiority with respect to state-of-the-art HSI restoration methods.
1 INTRODUCTION
HSI restoration is needed for degraded hyperspectral data but must balance spatial non-local similarity, global spectral low-rank structure, and computational cost. The paper proposes a unified iterative paradigm with adaptive optimization and evaluates it across three restoration tasks.
- Motivation: HSI degradations from noise, undersampling, and missing data substantially affect subsequent processing, making restoration a fundamental initial step.HSI restoration is framed as an ill-posed inverse problem requiring suitable regularization.
- Motivation: Previous methods separately emphasize spatial non-local similarity or spectral correlation, leaving the balance between these properties unresolved.Non-local methods achieve strong results but can incur high computational cost, while spectral low-rank regularization alone lacks spatial regularization.
- Proposed paradigm: The proposed paradigm integrates spatial non-local similarity with a global spectral low-rank property through a jointly updated orthogonal basis and reduced image.The method first learns a global spectral basis, then applies non-local processing to the projected HSI.
- Optimization: An efficient alternating minimization algorithm with convergence analysis and adaptive orthogonal-basis dimension prediction is developed.The flow includes latent HSI estimation, basis optimization, non-local similarity estimation, noise estimation, and rank adaptation.
- Evaluation: The unified model is evaluated on simulated and real data for denoising, compressed HSI reconstruction, and inpainting.The authors report best results across all three restoration tasks compared with state-of-the-art methods.
2 RELATED WORK
Related work models HSI restoration with non-local similarity and low-rank priors. Existing approaches either incur computational burdens, underuse spectral and spatial structure, or reshape the HSI for direct spectral low-rank approximation.
- Prior models: HSI restoration is ill-posed, so prior knowledge is introduced through non-local similarity and low-rank regularizers.These regularizers constrain the solution using spatial and spectral structure.
- Non-local similarity: Non-local methods group full-band patches and apply low-rank tensor approximation to exploit spatial similarity and spectral structure.Proposed variants include Tucker, weighted nuclear norm, tensor ring, and other low-rank models.
- Non-local similarity: Previous methods may underuse spectral and local spatial correlation, leaving the balance between spectral correlation and non-local similarity unresolved.Some methods emphasize non-local priors to reduce computational burden because spectral and local low-rank properties may be weak in patch groups.
- Spectral low-rank methods: Global spectral low-rank methods reshape the HSI into a matrix and apply PCA, robust PCA, or low-rank matrix factorization.The motivation is that the spectral subspace dimension can be far smaller than the original image dimension.
3 PROPOSED NGMEET PARADIGM
NGmeet unifies global spectral low-rank modeling with spatial non-local similarity for HSI restoration. It alternates latent-image estimation, orthogonal-basis and reduced-image learning, non-local denoising, noise re-estimation, and rank adaptation across iterations.
- Unified restoration paradigm: The paradigm combines a global spectral low-rank prior with a spatial non-local regularizer applied to the reduced image M.Non-local groups extract corresponding patches from M for spatial denoising.
- Unified restoration paradigm: NGmeet accommodates denoising, compressed reconstruction, and inpainting by changing the degradation operator h in a common observation model.These tasks use identity, compressed-measurement, and sampling operators, respectively.
- Unified restoration paradigm: NGmeet models the recovered HSI as a low-dimensional spectral representation with an orthogonal basis A and reduced image M, where K ≪ B.The basis captures a common subspace across spectra, while M contains the reduced spectral representation.
- Alternating optimization: The orthogonal constraint keeps basis representations distinct and preserves the noise distribution needed for noise estimation and Gaussian non-local denoising.The basis is constrained by A^⊤A = I.
- Alternating optimization: Alternating minimization first estimates a latent input image for data consistency, then optimizes the orthogonal basis and reduced image before non-local denoising reconstructs the output.The auxiliary-variable reformulation supports this alternating optimization structure.
- Rank adaptation: Real-data efficiency is supported by iterative group re-matching, noise re-estimation, and rank adaptation, with K updated as K = min(K + δ × i, B).The method increases the spectral dimension across iterations while handling estimated non-local groups and noise levels.
- Spectral orthogonal optimization: SVD provides an efficient closed-form solution for optimizing A under arbitrary K, whereas an alternative optimization is limited to the first iteration when the basis size is expanded.The two optimization approaches yield nearly identical restoration results according to the cited analysis.
Step 5: Non-local denoising on ¯ Mi
Step 5 denoises the reduced image by matching non-local groups, applying low-rank WNNM, and reconstructing the reduced image for HSI recovery. Iterative re-matching and rank adaptation refine restoration while balancing noise removal, detail preservation, and computational cost.
- Non-local group processing: The method transfers high-dimensional HSI denoising to low-dimensional reduced-image denoising, but overlapping patches require an approximate three-step solution.The steps include searching non-local groups, denoising each group, and reconstructing the reduced image.
- Iterative refinement: Re-matching non-local groups during iterations can improve restoration performance as reduced-image noise decreases and k-NN matching becomes more accurate.The approach therefore does not keep the same groups fixed throughout all iterations.
- Low-rank group denoising: WNNM solves each non-local patch-group subproblem through matrix reshaping and singular value decomposition.The denoised group tensors directly reconstruct the reduced image, and Xi = Mi ×3 Ai gives the recovered HSI at iteration i.
- Noise modeling: The reduced-image noise is assumed to remain Gaussian with unchanged variance after orthogonal projection, enabling existing spatial denoising methods.If the assumed univariate Gaussian noise model fails, performance can deteriorate substantially.
- Rank adaptation: Increasing the adaptive rank K trades stronger noise removal at lower ranks for better detail preservation at higher ranks.K starts small for low-quality latent inputs and increases as noise variance decreases, allowing later bases to capture more image details.
- Convergence analysis: The algorithm optimizes an approximated objective, with a constant error caused by approximating overlapping non-local regularizers.Without re-matching and rank adaptation, its generated sequences have limit points that are critical points of the approximated objective under coercivity.
4 APPLICATION EXAMPLES
NGmeet applies one restoration paradigm to denoising, compressed HSI reconstruction, and inpainting by adapting the degradation operator and latent-image update. Its spectral basis reduces non-local processing cost while retaining spatial denoising, yielding a favorable complexity trade-off.
- Unified formulation: NGmeet implements denoising, compressed reconstruction, and inpainting within a common restoration framework.The three tasks are represented through different degradation operators in the same optimization model.
- Denoising: For denoising, the degradation operator is identity and the noise is modeled as Gaussian; the latent input update uses known or estimated noise variance.The real-dataset variance is estimated using a multiple-regression-based approach.
- Compressed reconstruction: Compressed HSI reconstruction recovers images from compressed measurements, using a latent-image optimization solved efficiently by preconditioned conjugate gradients.The measurement operator may be poorly conditioned, making initialization important for predicting a satisfactory latent input.
- Inpainting: Inpainting treats the degradation as sampling and uses the observed-pixel projection to recover missing HSI information.The sampling set Ω identifies observed pixels, while unobserved entries are updated from the current reconstruction.
- Complexity: Each iteration combines stage-A SVD, O(MNB^2), with stage-B non-local denoising, O(Tn^2Kp^2).Compared with methods performing only stage B, NGmeet adds stage-A cost but is at least B/K times faster in stage B.
5 EXPERIMENTS
The experiments evaluate NGmeet across denoising, compressed reconstruction, and inpainting using simulated and real HSI settings. The supplied passage establishes the experimental scope and implementation environment.
- Experimental scope: Experiments cover HSI denoising, compressed HSI reconstruction, and inpainting.The implementations were programmed in Matlab on a Core i7-7820HK computer with 64G memory.
5.1 HSI denoising experiments
The denoising experiments evaluate NGmeet against spectral low-rank and spatial non-local methods on simulated and real data. NGmeet achieves the best results in almost all quantitative cases and preserves visual quality and spectral details.
- Experimental setup: NGmeet is compared with spectral low-rank methods, spatial non-local methods, and other restoration baselines using PSNR, SSIM, and SAM.Lower SAM indicates higher spectral similarity between the original and restored images.
- Simulated-data results: NGmeet achieves the best results in almost all simulated denoising cases across the evaluated datasets and noise levels.The comparison also shows that non-local methods can outperform spectral low-rank methods for MSI, whereas the reverse can occur for HSI.
- Simulated-data results: On the CAVE-toy image with noise variance 100, FastHyDe, LLRT, and NGmeet outperform the remaining methods, while NGmeet produces the best visual quality.FastHyDe and LLRT show artifacts in enlarged regions.
- Real-data results: On real Indian Pines and Urban data, NGmeet removes noise while retaining spectral details, unlike methods that produce excessive smoothing, color changes, or stripes.Although designed for Gaussian noise, NGmeet achieves the best results on these real datasets.
5.2 Compressed HSI reconstruction experiments
The compressed reconstruction experiments test NGmeet on simulated CAVE, PaC, and WDC data and on CASSI-coded imagery. NGmeet achieves the best reported quantitative and visual results across these evaluations.
- Simulated-data experiments: The simulated reconstruction experiments use CAVE, PaC, and WDC data with sampling ratios of 2%, 5%, 10%, 15%, and 20%.The CAVE subset has size 300 × 300 × 31, and inputs are normalized to [0, 255].
- Simulated-data experiments: NGmeet achieves the best visual results in the simulated compressed reconstruction comparisons.The cited visual comparison includes CAVE-Toy results at a sampling ratio of 0.02 and reconstructed coded-Toy images.
- CASSI reconstruction: For CASSI compressive imaging, NGmeet achieves the best quantitative and visual results compared with GAP-TV and DeSCI.The hardware compressed operator is known in advance in this setting.
5.3 HSI inpainting experiments
The inpainting experiments evaluate NGmeet on simulated missing-pixel data and real Landsat-7 stripe gaps. NGmeet achieves the strongest reported quantitative and visual reconstruction results across these settings.
- Simulated-data experiments: The simulated inpainting experiments use CAVE and PaC images with observation ratios of 5%, 10%, and 20%.The PaC image has size 200 × 200 × 80, and inputs are normalized to [0, 255].
- Simulated-data experiments: NGmeet achieves the best results in almost all simulated inpainting cases compared with AWTC, Tmac, HaLRTC, T-SVD, TVTR, LLRTC, and NLLRTC.On the Pavia dataset at a 5% sampling ratio, NGmeet has the highest PSNR and SSIM values for almost all bands.
- Real-data experiments: On real Landsat-7 stripe inpainting, NGmeet achieves the best visual performance, while NLLRTC and NGmeet correct missing information left unreconstructed by LRRTC.The Dubai time-series image is reshaped to 300 × 300 × 48 after merging six spectral and eight temporal dimensions.
5.4 Ablation study
The ablation and efficiency studies examine NGmeet’s empirical settings, adaptive rank dimension, runtime, convergence, denoiser choice, and band scalability. Results support rank adaptation, stable convergence, and nearly band-independent runtime relative to prior non-local methods.
- Ablation study: The ablation study evaluates empirical subproblem solutions, non-local group re-matching, and rank adaptation within the denoising objective.Denoising is used because the restoration tasks share the same objective model.
- Rank adaptation: HySime initialization of K yields the best results among the tested initializations, and updating K during iterations improves performance.The algorithm increases K linearly with stepsize δ; the experiments fix δ = 2.
- Efficiency: NGmeet’s reduced-image projection adds little time, while its spatial non-local denoising stage provides the main computational advantage over KBR and LLRT.The comparison separates stage A spectral projection from stage B spatial non-local denoising.
- Efficiency: As the number of bands increases, KBR and LLRT require substantially more time, whereas NGmeet’s runtime remains almost unchanged and its performance remains best.The comparison uses WDC data with noise variance 100 and reports computational time and SSIM.
- Convergence: NGmeet converges to a stable PSNR value very quickly at different noise levels on the WDC dataset.The convergence analysis tracks PSNR as the iteration count increases.
- Denoiser choice: Replacing WNNM with WSNM produces slightly higher evaluation results, indicating that the paradigm is compatible with alternative non-local denoisers.The authors state that a better reduced-image denoiser could further improve results.
- Band dimensionality: Processing more bands simultaneously yields better denoising results for NGmeet in the comparison with HSI-DeNet and its reduced-band variant.The reduced-band variant uses 10 bands, matching HSI-DeNet’s training setting.
6 CONCLUSION
The paper presents a unified spatial-spectral paradigm for HSI restoration, combining spatial non-local similarity with global spectral low-rank structure. Experiments on three restoration tasks using simulated and real datasets confirm high performance.
- The proposed paradigm integrates spatial non-local similarity through low-dimensional orthogonal bases and global spectral low-rank structure through reduced-image denoising.
- NGmeet uses an alternating minimization method to solve its optimization problem.
- Experiments on denoising, compressed HSI reconstruction, and inpainting with simulated and real datasets confirmed the method’s high performance.
- WNNM is not required in the unified spatial-spectral paradigm, while future work will explore CNN-based non-local similarity and automated machine learning for configuration.