Source-linked AI summary
Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising
Shuai Hu, Feng Gao, Xiaowei Zhou, Junyu Dong, Qian Du
TL;DR
HSI denoising must handle mixed noise while preserving the rich spectral information needed for analysis, yet jointly modeling global and local features remains challenging. HCANet combines convolution and attention through CAFM and uses MSFN for multi-scale feature extraction. Experiments report strong quantitative and visual denoising performance, including effective removal of complex noise while preserving local details.
Problem
HSI denoising is important for preserving image quality and downstream analysis, but simultaneously modeling global and local features remains underexplored and challenging.
Method
HCANet combines convolution and attention through CAFM to model global and local features, and uses MSFN to extract features at multiple scales.
Results
HCANet outperforms other methods across reported metrics for single-type and complex noise cases and preserves local details in qualitative comparisons.
Takeaways & Limitations
HCANet provides effective HSI denoising across challenging noise conditions while maintaining visual image details.
Abstract
from arXiv · showhide
Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. However, simultaneously modeling global and local features is rarely explored to enhance HSI denoising. In this letter, we propose a hybrid convolution and attention network (HCANet), which leverages both the strengths of convolution neural networks (CNNs) and Transformers. To enhance the modeling of both global and local features, we have devised a convolution and attention fusion module aimed at capturing long-range dependencies and neighborhood spectral correlations. Furthermore, to improve multi-scale information aggregation, we design a multi-scale feed-forward network to enhance denoising performance by extracting features at different scales. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet. The proposed model is effective in removing various types of complex noise. Our codes are available at \url{https://github.com/summitgao/HCANet}.
I. INTRODUCTION
HSI denoising is important because mixed sensor noise can degrade image quality and downstream analysis, while effective joint modeling of global and local features remains challenging. HCANet addresses these challenges with convolution-attention fusion and multi-scale feature extraction, achieving strong denoising performance on benchmark datasets.
- HSI denoising is critical because mixed sensor noise degrades image quality and can hinder subsequent analysis, interpretation, detection, and classification.
- Jointly modeling global and local information remains challenging because convolutions capture local features while attention captures long-range dependencies.
- HCANet combines convolution and attention through CAFM to capture long-range dependencies and neighborhood spectral correlations.
- Experiments on two real-world benchmark datasets report that HCANet is superior to other state-of-the-art competitors.
- MSFN extracts features at different scales to improve feed-forward-network information aggregation and suppress noise from multiple scales.
II. METHODOLOGY
HCANet is a U-shaped HSI denoising network built from CAMixing blocks that combine convolution-attention fusion with multi-scale feed-forward processing. It predicts a noise residual from the noisy HSI and reconstructs the clean image through residual addition.
- HCANet uses a U-shaped network with CAMixing blocks, each containing a CAFM and an MSFN.
- For noisy HSI I, HCANet predicts a residual map I_N with the input shape and reconstructs the clean image as Î = I + I_N.
- The network begins with a 3 × 3 × 3 convolution for low-level features and is trained with reconstruction loss plus a global gradient regularizer.
- CAFM contains local and global branches, using convolution and channel shuffling for local features and attention for long-range dependencies.
A. Convolution and Attention Fusion Module
CAFM combines local convolution and global attention branches to model complementary neighborhood and long-range information in HSI features.
- A. Convolution and Attention Fusion Module: CAFM combines local convolution and global attention branches to model both local features and long-range dependencies.The design treats convolution and attention as complementary mechanisms for global and local feature modeling.
- A. Convolution and Attention Fusion Module: The local branch adjusts channels, shuffles channel information, and applies depth-wise separable and 3 × 3 × 3 convolutions for feature extraction.Its formulation identifies Fconv as the local-branch output and CS as channel shuffle.
- A. Convolution and Attention Fusion Module: The global branch generates query, key, and value tensors, computes channel-wise attention, and uses a learnable scaling parameter before softmax.The attention map is formed from reshaped query and key tensors, with α controlling the matrix-multiplication magnitude.
- A. Convolution and Attention Fusion Module: The CAFM output is obtained by combining the global attention output Fatt with the local convolution output Fconv.This fusion produces the module output after the two branches process the input feature.
B. Multi-Scale Feed-Forward Network
MSFN replaces single-scale feed-forward aggregation with parallel gated paths that extract multi-scale features for richer nonlinear transformation.
- B. Multi-Scale Feed-Forward Network: MSFN aggregates multi-scale features after each CAMixing block to enhance nonlinear information transformation beyond the original single-scale FFN.The module is motivated by the limited information captured through single-scale aggregation.
- B. Multi-Scale Feed-Forward Network: Two 1 × 1 convolutions expand channels with ratio γ = 2, while parallel paths use gating through element-wise feature products.The parallel design separates feature processing before recombining information through the gating mechanism.
- B. Multi-Scale Feed-Forward Network: The upper path uses 3 × 3 dilated convolutions with dilation rates 2 and 3 for multi-scale feature extraction.These dilated operations provide the explicitly described multi-scale receptive fields.
- B. Multi-Scale Feed-Forward Network: MSFN has a distinct role from CAFM by enriching features with contextual information through multi-scale processing.The paper distinguishes MSFN’s contextual enrichment from CAFM’s convolution-attention fusion.
C. Loss Function
The network is trained with reconstruction loss and a global gradient regularizer to improve denoising quality while constraining the reconstructed HSI.
- C. Loss Function: The reconstruction loss uses an L1 objective between the estimated noise-free HSI and the noisy HSI.The paper identifies Î as the estimated noise-free image and I as the noisy HSI.
- C. Loss Function: A global gradient regularizer constrains the reconstructed HSI to enhance denoising quality and curtail redundancy.The gradient operator is applied along horizontal, vertical, and spectral axes.
- C. Loss Function: The total training objective combines reconstruction loss and gradient regularization, with λ governing the regularizer weight.The weight parameter is set to 0.01 based on empirical evidence to balance the loss terms.
A. Experiment Setup
The experiments train HCANet on ICVL and evaluate it on Pavia under Gaussian, blind Gaussian, and complex real-world noise settings against model-driven and deep-learning baselines.
- Benchmark datasets: Experiments train on ICVL and evaluate on Pavia, using 31 spectral bands and 128×128×31 training cubes.The ICVL images have resolution 1392 × 1300, and training data are augmented with rotation and scaling.
- Noise setting: Testing covers Gaussian noise with σ = 30 to σ = 70, blind Gaussian noise, and complex sensor-related noise.The complex-noise evaluation targets noise types commonly found in hyperspectral data from real spaceborne sensors.
- Noise setting: Complex-noise cases combine Gaussian noise with stripe, deadline, and other band-level corruptions.Stripe noise affects 5% ∼15% of columns, while deadline noise is added to the third of the spectral bands.
- Baselines and metrics: Table II evaluates the proposed model and comparison methods across five complex-noise cases.The table provides quantitative evaluation for the HSI denoising task under the stated complex-noise settings.
- Baselines and metrics: HCANet is compared with BM4D, LRMR, LRTV, Restormer, and MAFNet using PSNR, SSIM, and SAM.PSNR and SSIM are interpreted as better when larger, while SAM is also used to quantify denoising performance.
B. Experimental Analysis
HCANet outperforms the compared methods across Gaussian and complex noise, preserves local details qualitatively, and achieves strong denoising with moderate complexity.
- Quantitative and qualitative comparison: HCANet achieves the best performance across all metrics for single-type Gaussian noise and all complex-noise cases.Table I covers varying Gaussian-noise intensities, while Table II covers complex noise; the supplied passage reports HCANet as best in both settings.
- Quantitative and qualitative comparison: HCANet removes most noise while preserving local details and restoring original image features in qualitative comparisons.MAFNet tends to oversmooth, while Restormer performs poorly on complex noise; HCANet is described as effective in contrast.
- Ablation study: HCANet with all components performs best in the ablation study, with performance improving as the local branch, 3D convolution, and MSFN are added.The ablation is conducted on ICVL under σ = 30 and 50 epochs.
- Computational complexity: HCANet achieves optimal denoising performance while maintaining a relatively moderate number of parameters and computational complexity.The comparison is conducted under blind Gaussian noise after 50 epochs.
IV. CONCLUSION
The conclusion presents HCANet as a hybrid HSI denoising network combining global and local feature modeling with multi-scale feature extraction. Experiments show effectiveness against state-of-the-art methods in quantitative and visual reconstruction quality.
- IV. CONCLUSION: HCANet combines CAFM for global-local feature fusion with MSFN for multi-scale feature extraction in HSI denoising.The conclusion identifies these modules as the network’s principal design contributions.
- IV. CONCLUSION: Experiments on challenging HSI datasets demonstrate effectiveness against state-of-the-art denoising methods in quantitative metrics and reconstructed-image visual quality.The conclusion reports strong performance across both quantitative evaluation and visual reconstruction results.