Source-linked AI summary
Spatial-Adaptive Network for Single Image Denoising
Meng Chang, Qi Li, Huajun Feng, Zhihai Xu
TL;DR
Image denoising methods can oversmooth details because rigid local convolutions cannot adapt to textures and edges, while larger receptive fields increase computational cost. SADNet uses spatially adaptive deformable sampling within a multiscale encoder-decoder to remove noise from coarse to fine. It achieves state-of-the-art performance on synthetic and real noisy images with moderate or relatively small computational overhead.
Problem
Rigid local convolutions can use irrelevant neighboring features, causing oversmoothing and detail loss, while deeper or non-local networks require more memory and computation.
Method
SADNet combines residual spatial-adaptive blocks with deformable feature sampling, an encoder-decoder, and a context block for multiscale coarse-to-fine denoising.
Results
SADNet achieves state-of-the-art performance on both synthetic and real noisy images with moderate running time and relatively small computational overhead.
Takeaways & Limitations
Spatially adaptive capability can restore richer details in complex scenes under heavy noise while maintaining efficient denoising.
Takeaways & Limitations
At excessively small resolutions, image structures can be destroyed and information lost, hindering feature reconstruction.
Abstract
from arXiv · showhide
Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but more computational overhead is needed. In this paper, we propose a novel spatial-adaptive denoising network (SADNet) for efficient single image blind noise removal. To adapt to changes in spatial textures and edges, we design a residual spatial-adaptive block. Deformable convolution is introduced to sample the spatially correlated features for weighting. An encoder-decoder structure with a context block is introduced to capture multiscale information. With noise removal from the coarse to fine, a high-quality noisefree image can be obtained. We apply our method to both synthetic and real noisy image datasets. The experimental results demonstrate that our method can surpass the state-of-the-art denoising methods both quantitatively and visually.
1 Introduction
Image denoising remains challenging because fixed local convolutions can oversmooth textures and edges, while deeper or non-local networks increase computational costs. SADNet addresses these issues with spatial adaptation, multiscale processing, and coarse-to-fine noise removal, achieving strong performance with relatively small overhead.
- Motivation: Traditional CNNs use fixed local neighborhoods that can introduce irrelevant features, oversmooth edges, and lose image details.Increasing depth or adding non-local modules expands the receptive field but increases memory and time consumption.
- Proposed method: SADNet uses residual spatial-adaptive blocks to adapt feature processing to changes in spatial textures and edges.The blocks introduce deformable convolution to sample spatially relevant features for weighting.
- Proposed method: An encoder-decoder structure with residual blocks and a context block captures multiscale information and removes noise from coarse to fine.The context block is applied at the coarsest scale to enlarge the receptive field.
- Results: Experiments on multiple synthetic and real noisy image datasets demonstrate state-of-the-art performance with relatively small computational overhead.The reported evaluation covers both synthetic and real noisy images.
2 Related works
Image denoising research spans model-based priors and learning-based methods, with deep networks becoming increasingly prominent. Recent work has also shifted toward realistic, spatially variant and channel-dependent noise using real noisy-image datasets.
- Model-based methods: Model-based denoising methods use priors such as local smoothing, sparsity, non-local self-similarity, and external statistics with optimization algorithms.Non-local self-similarity methods including NLM, BM3D, and WNNM exploit repeated structures within images.
- Learning-based methods: Learning-based methods increasingly use deep neural networks, including approaches that combine natural priors with CNNs and end-to-end residual learning.Examples include TRND, NLNet, DnCNN, and FFDNet, which introduces a noise level map as input.
- Real-image denoising: Recent research focuses on realistic noise and real noisy images rather than only synthetic additive white Gaussian noise.Real noisy datasets and methods such as N3Net, CBDNet, PD, and RIDNet support this shift.
3 Framework
SADNet combines spatial-adaptive feature sampling with a multiscale encoder-decoder to remove noise while preserving image structures. Its RSAB and context block address fixed-location convolution and limited receptive-field constraints.
- 3 Framework: SADNet processes noisy features through a multiscale encoder-decoder and reconstructs the denoised image from coarse to fine.Initial features enter the encoder-decoder, while long residual learning lets the network learn the noise component.
- 3.1 Residual spatial-adaptive block: Modulated deformable convolution resamples spatially relevant features and adjusts their amplitudes using learnable offsets and modulation scalars in [0, 1].The design targets texture and edge changes that fixed-location convolution cannot accommodate reliably.
- 3.1 Residual spatial-adaptive block: The residual spatial-adaptive block replaces its first convolution with modulated deformable convolution and uses leaky ReLU.A traditional convolution follows the deformable convolution, with a short skip connection for local residual learning.
- 3.1 Residual spatial-adaptive block: Offsets and modulation scalars are transferred from coarse to fine, using upsampling and current-scale features to estimate more accurate feature locations.Bilinear interpolation is used to upsample the previous offset maps and modulation scalars.
- 3.2 Context block: The context block enlarges the receptive field and captures multiscale information with differently sized dilated convolutions instead of further downsampling.This avoids the structural damage and information loss associated with overly small spatial resolutions while not increasing parameter count.
4 Experiments
Experiments evaluate SADNet on synthetic and real noisy-image datasets, using ablations, visual comparisons, benchmark metrics, and efficiency measurements. Results support the roles of spatial adaptivity, multiscale context, and coarse-to-fine denoising across varied noise conditions.
- Experimental setup: Training uses synthetic and real-noise datasets, with augmentation and 128 × 128 patches optimized using ADAM.Synthetic data are generated from DIV2K with added noise; real-noise training uses SIDD, RENOIR, and Poly.
- Ablation study: Replacing RSAB with an original ResBlock substantially decreases performance, while introducing the context block improves performance.The ablation study is conducted on Kodak24 at σ = 50; offset transfer is also separately evaluated.
- Kernel visualization: Learned kernels distribute sampling locations along edges while remaining approximately uniform in smooth or homogeneous textured regions.This visualization indicates that RSAB sampling locations change with image content across coarse-to-fine scales.
- Synthetic noisy images: On synthetic grayscale images, SADNet achieves the highest PSNR on most datasets and tested noise levels, while requiring less computational overhead than RNAN.On synthetic color images, it outperforms the state-of-the-art methods on all datasets and tested noise levels, with larger improvements at higher noise levels.
- Synthetic noisy images: Visual comparisons on BSD68 and Kodak24 show that SADNet restores difficult feathers and clothing textures without the oversmoothing artifacts seen in competing methods.The comparisons use challenging examples with noise level σ = 50.
- Real noisy images: On DND, SIDD, and Nam, SADNet achieves stronger quantitative or visual results than the compared real-image denoisers, including both PSNR/SSIM improvements and texture recovery.On Nam with JPEG compression, SADNet gains 1.88, 1.83, and 1.61 dB over RIDNet, PD, and CBDNet*, respectively; its FLOPs and running time are also reported as low.
5 Conclusion
SADNet uses multiscale spatial adaptation to sample relevant features for denoising, restoring richer details in complex scenes under heavy noise while maintaining moderate running time.
- SADNet uses multiscale residual spatial-adaptive blocks to sample relevant features for weighting based on image content and textures.
- A context block captures multiscale information, while offset transfer improves the accuracy of estimated sampling locations.
- Spatially adaptive capability restores richer details in complex scenes under heavy noise.
- SADNet achieves state-of-the-art performance on both synthetic and real noisy images with a moderate running time.