NBNet: Noise Basis Learning for Image Denoising with Subspace Projection

Shen Cheng; Yuzhi Wang; Haibin Huang; Donghao Liu; Haoqiang Fan; Shuaicheng Liu

Source-linked AI summary

NBNet: Noise Basis Learning for Image Denoising with Subspace Projection

Shen Cheng, Yuzhi Wang, Haibin Huang, Donghao Liu, Haoqiang Fan, Shuaicheng Liu

arXiv:2012.15028v2Dec 30, 2020cs.CV

TL;DR

Image denoising must recover clean signals when noise and image content are difficult to separate, particularly in weak-texture or low-SNR regions. NBNet uses SSA to learn image-adaptive signal subspaces and project inputs for denoising, achieving state-of-the-art benchmark performance with lower computational cost.

Problem
Image denoising seeks to recover clean signals from noisy observations, while weak textures and low-SNR regions make local signal-noise separation difficult.
Method
NBNet combines a UNet architecture with SSA, which learns signal-subspace bases and performs non-local image projection end-to-end.
Results
NBNet achieves state-of-the-art PSNR and SSIM on popular benchmarks, with reported gains over MIRNet on SIDD using substantially lower computational cost and fewer parameters.
Takeaways & Limitations
Subspace basis generation and projection can introduce global structure into denoising while preserving local details and improving efficiency relative to adding convolutional blocks.

Abstract

from arXiv · show

In this paper, we introduce NBNet, a novel framework for image denoising. Unlike previous works, we propose to tackle this challenging problem from a new perspective: noise reduction by image-adaptive projection. Specifically, we propose to train a network that can separate signal and noise by learning a set of reconstruction basis in the feature space. Subsequently, image denosing can be achieved by selecting corresponding basis of the signal subspace and projecting the input into such space. Our key insight is that projection can naturally maintain the local structure of input signal, especially for areas with low light or weak textures. Towards this end, we propose SSA, a non-local subspace attention module designed explicitly to learn the basis generation as well as the subspace projection. We further incorporate SSA with NBNet, a UNet structured network designed for end-to-end image denosing. We conduct evaluations on benchmarks, including SIDD and DND, and NBNet achieves state-of-the-art performance on PSNR and SSIM with significantly less computational cost.

1. Introduction

Image denoising requires recovering clean signals from noisy observations, especially where local filtering struggles with weak textures or low signal-to-noise ratios. NBNet addresses this through learned non-local subspace projection and achieves state-of-the-art benchmark performance with efficient computation.

Motivation: Image denoising recovers a clean signal x from a noisy observation y containing additive noise n.Both the image and noise terms are unknown, making the problem ill-posed.
Motivation: Local convolutional responses can confuse noise and signal in weak-texture or low-SNR regions without additional global structure information.This limitation motivates incorporating non-local image information into denoising.
Subspace Projection: NBNet learns image basis vectors and reconstructs inputs in their signal subspace to preserve image information while suppressing noise irrelevant to the basis.The approach treats natural images as lying in a low-rank signal subspace.
Method: SSA learns subspace bases and image projection end-to-end within a UNet-style NBNet for projection-based image denoising.SSA is designed as a simple, efficient module that can also be plugged into conventional CNNs.
Results: NBNet achieves state-of-the-art PSNR and SSIM on popular denoising benchmarks, including SIDD and DND.The authors also report that projection-based denoising is a promising direction for further study.

2. Related Works

Image denoising has progressed from handcrafted priors to learned CNN architectures, while recent work also addresses realistic noise synthesis. NBNet instead introduces SSA to learn subspace bases and perform image projection for noise reduction.

Traditional Methods: Classical denoising methods use image priors such as non-local means, sparse coding, and BM3D, but often have high complexity and limited generalization.
CNN-based Methods: CNN-based denoising has gained attention through end-to-end learning and increasingly advanced network architectures.
Subspace Projection: SSA differs from prior attention modules by learning subspace bases and image projection rather than selecting regions or features.
Noise Modeling: Synthetic-noise research explores Gaussian-Poisson, in-camera process, GMM, and GAN-generated noise to reduce reliance on difficult real paired datasets.
Subspace Projection: NBNet studies subspace basis generation and improves noise reduction through projection, unlike approaches focused on noise modeling.

3. Method

NBNet performs image-adaptive denoising by generating signal-subspace bases from feature maps and projecting features before fusion in a UNet architecture. Its SSA module implements differentiable basis generation and projection, preserving useful structure while suppressing irrelevant noise.

3.1.1. Basis Generation: SSA generates K basis vectors from paired feature maps and transforms one feature map into their signal subspace.The basis-generation function takes X1 and X2, produces V ∈ R^N×K, and is trained end-to-end.
Visual Results: On SIDD and DND examples, the method is reported to preserve weak textures, textures, and sharpness.The cited visual examples emphasize dots and line patterns in weak-texture regions and texture sharpness on DND.
3.1.2. Projection: Orthogonal projection reconstructs feature maps within the signal subspace using a normalization term for non-orthogonal generated bases.The projection operations are linear, differentiable matrix manipulations with appropriate reshaping.
3.2. NBNet Architecture and Loss Function: NBNet uses a four-stage UNet with residual convolutional blocks, skip connections, and a final convolution producing a global residual for denoising.The denoising result is formed by applying the global residual to the noisy input.
3.2. NBNet Architecture and Loss Function: NBNet places SSA modules in skip connections, projecting low-level features under guidance from upsampled high-level features before fusion.This distinguishes NBNet from conventional UNets that directly fuse low-level and high-level features.
3.2. NBNet Architecture and Loss Function: The network is trained end-to-end on clean-noisy image pairs with an L1 loss between the clean image and NBNet output.The clean image, noisy image, and network are denoted by x, y, and G(·), respectively.

4. Evaluation and Experiments

The evaluation compares NBNet with competing denoising methods across synthetic and real datasets, including AWGN cases. The supplied passages specify the comparison scope but provide no numerical outcomes from Table 4.

4. Evaluation and Experiments: Experiments compare NBNet with previous methods on synthetic and real datasets and include ablation studies of the proposed method.The evaluation plan covers implementation details, five real-image datasets, and ablations.
4. Evaluation and Experiments: Table 4 reports PSNR results for competing methods on AWGN noise cases across three test datasets.The supplied passage identifies the table’s metric, noise setting, and dataset count but not the individual values.

4.1. Training Settings

NBNet is trained end-to-end without pre-training, using a fixed subspace size and an iterative optimization schedule with cropped training patches.

4.1. Training Settings: NBNet requires no pre-training and sets the subspace size K to 16 for all modules.The architecture is trained through an end-to-end strategy.
4.1. Training Settings: Training uses Adam with momentum terms (0.9, 0.999), an initial learning rate of 2 × 10^-4, cosine annealing, and 700,000 minibatch iterations.Training instances are 128 × 128 patches, with 32 instances per mini-batch and random rotation, cropping, and flipping augmentation.

4.2. Results on Synthetic Gaussian Noise

NBNet is evaluated on synthetic non-i.i.d. and additive Gaussian noise using datasets generated under controlled mask and noise settings. It outperforms VDN across test cases and by approximately 0.3 dB in PSNR on AWGN.

The synthetic training set combines 432 BSD images, 400 ImageNet validation images, and 4,744 Waterloo images, while testing uses Set5, LIVE1, and BSD68.
Non-i.i.d. Gaussian noise uses a spatially variant mask, with one mask for training and three for testing to assess generalization.
NBNet outperforms VDN on every non-i.i.d. Gaussian-noise test case despite using purely blind noise reduction.
∼0.3 dB: NBNet surpasses VDN by this average PSNR margin on AWGN at σ = 15, 25, and 50.
The method achieves the best evaluation results without explicitly relying on a prior distribution of noise data.

4.3. Results on SIDD Benchmark

On the real-world SIDD benchmark, NBNet is compared with prior denoising methods using quantitative, visual, parameter, and computational evaluations. It slightly improves MIRNet’s PSNR and SSIM while using substantially fewer resources.

SIDD contains about 30,000 noisy images from 10 scenes captured under different lighting conditions with five smartphone cameras, including 1,280 validation images.
NBNet is compared with VDN, DANet, and MIRNet on SIDD using quantitative and visual results, alongside parameter and computational-cost comparisons.
39.75 PSNR: NBNet exceeds MIRNet’s 39.72 while using 11.2% of its computational cost and 41.82% of its parameters.
0.969 SSIM: NBNet improves over MIRNet’s 0.959 on the SIDD benchmark.

4.4. Results on DND Benchmark

NBNet is evaluated on the real-noise DND benchmark, which provides 50 test images and no training data. Using training data from SIDD and Renoir, NBNet achieves better PSNR than MIRNet with lower computational and parameter costs while preserving textures and sharpness.

DND contains 50 real noisy-image and ground-truth pairs captured with consumer-grade cameras having differing sensor sizes.
The DND test set provides 20 patches from each image, totaling 1,000 patches, but supplies no training data.
NBNet is trained using SIDD and Renoir data for DND evaluation, with the same model selected for best SIDD validation performance submitted to DND.
NBNet achieves better PSNR than MIRNet’s 39.88 dB with a fraction of MIRNet’s computational cost and parameter count.
Visual comparisons show clean output images while preserving textures and sharpness.

4.5. Ablation Study

The ablation studies examine SSA placement, subspace dimensionality, and projection choices. Results favor SSA, K = 16, and projections whose basis uses both X1 and X2 while projecting X1.

Ablation Study: The ablation study evaluates SSA, the signal-subspace dimension K, and alternative projection options.
SSA Ablation: DnCNN + Concat improves over DnCNN by about 0.2 dB, while DnCNN + SSA is evaluated as the SSA-based alternative.
Influence about Different k Values: K = 32 fails to converge because the basis dimension equals the first-stage channel dimension, while larger subspaces can make fitting unstable.
Influence about Different k Values: K = 1 causes significant information loss, whereas K = 8 and K = 16 achieve comparable performance; the best choice is K = 16.
Options about Projection: Basis generation from only X1 causes non-convergence, while generation from only X2 is trainable but produces unsatisfactory results.
Options about Projection: 39.75 dB: projecting X1 onto a basis generated from both X1 and X2 produces the best PSNR.

4.6. Basis Visualization and Discussion

SSA’s learned basis vectors capture non-local texture patterns, helping NBNet reconstruct weak textures more consistently than without SSA or conventional local filtering.

SSA recovers weak dotted texture in dark regions more consistently with surrounding image content than the version without SSA.The SSA-enabled result avoids the blurring observed when SSA is disabled.
The same qualitative behavior is reported in Fig. 6, where NBNet outperforms other methods in weak-textured regions.
Many of the 16 learned basis channels contain the dotted pattern across the image patch.This shared pattern supports reconstructing weak texture using information from other image regions.
SSA’s non-local correlation lets similar texture elsewhere support reconstruction of weak texture through globally determined coefficients.The projection combines the learned basis with coefficients determined from the image.
Conventional convolutional filtering can blur weak-texture areas when local responses are insignificant and downsampled features provide coarse information.

5. Conclusion

The paper frames image denoising as subspace projection rather than relying on complex architectures or accurate noise modeling. Learned basis generation and projection introduce global structure information while preserving local detail.

Subspace projection offers a new perspective on denoising by introducing global structure information into the denoising process.
SSA learns subspace basis generation and projection end-to-end within a UNet-style denoising architecture.
The proposed approach is intended to achieve better local detail preservation without relying on complicated network architectures or accurate noise modeling.