Source-linked AI summary
Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis
Kai Zhang, Yawei Li, Jingyun Liang, Jiezhang Cao, Yulun Zhang, Hao Tang, Deng-Ping Fan, Radu Timofte, Luc Van Gool
TL;DR
Deep denoising models often rely on simplified noise assumptions, leaving general-purpose blind denoising for real images unresolved. This paper combines a swin-conv UNet with a synthesized degradation pipeline covering diverse noise and resizing operations. Experiments report state-of-the-art AWGN removal and significantly improved practicability for real-image denoising.
Problem
Existing denoising methods commonly rely on simple noise assumptions, while general-purpose blind denoising for real images remains unsolved because real degradations are more varied.
Method
The paper combines swin-conv blocks in a UNet with a practical degradation-synthesis pipeline using diverse noise types, resizing, random shuffling, and double degradation.
Results
The new architecture achieves state-of-the-art performance for AWGN removal, while the proposed degradation model significantly improves practicability for real images.
Takeaways & Limitations
The resulting model is presented as a strong baseline for synthetic Gaussian denoising and practical blind image denoising.
Abstract
from arXiv · showhide
While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved. In this paper, we attempt to solve this problem from the perspective of network architecture design and training data synthesis. Specifically, for the network architecture design, we propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block, and then plug it as the main building block into the widely-used image-to-image translation UNet architecture. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise (including Gaussian, Poisson, speckle, JPEG compression, and processed camera sensor noises) and resizing, and also involves a random shuffle strategy and a double degradation strategy. Extensive experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance and the new degradation model can help to significantly improve the practicability. We believe our work can provide useful insights into current denoising research.
1. Introduction
Image denoising recovers a clean image from noisy observations, but existing assumptions often fail on real images. The paper addresses this gap through a swin-conv UNet and a broader noise-synthesis strategy for practical blind denoising.
- Motivation: Image denoising recovers a latent clean image from a noisy observation and supports image restoration and other vision tasks.It can also evaluate image priors and optimization algorithms and serve as a component in solving deblurring and super-resolution.
- Motivation: AWGN is widely used because it is mathematically convenient, but models trained with it often perform poorly on real images because of noise-assumption mismatch.JPEG compression, Poisson, and camera sensor noise are also common assumptions.
- Motivation: Realistic degradation modeling improves real-image denoising, as models trained with accurate degradation can outperform AWGN-trained models on the sRGB DND dataset.The cited work uses real noisy and almost noiseless references from consumer cameras.
- Contributions: The proposed swin-conv block combines residual convolutional local modeling with Swin Transformer non-local modeling inside a multiscale UNet.The architecture is evaluated on benchmark PSNR performance for AWGN removal.
- Contributions: The proposed hand-designed synthesis model combines varied noise types and degradation operations to train a general-purpose blind denoising model.The introduction identifies Gaussian, Poisson, speckle, JPEG compression, processed camera sensor noise, resizing, random shuffling, and double degradation.
- Contributions: The trained blind denoising model significantly improves practicability for real images and provides a strong baseline for synthetic Gaussian and practical blind denoising.The authors also state that code will be released upon acceptance.
2. Related work
Related work develops denoising methods that exploit non-local image structure through model-based algorithms and deep network architectures. These approaches include explicit nearest-neighbor, recurrent, and transformer-based non-local processing.
- Non-Local Prior: Model-based denoising methods commonly exploit non-local self-similarity, where local patches have similar patches elsewhere in the image.Representative methods include BM3D, LSSC, and WNNM.
- Deep Architectures: Some deep methods incorporate non-local operations through gradient-based MRFs, unrolled networks, neural nearest-neighbor blocks, and recurrent architectures.These methods seek to model correlations among non-local patches within learned or unrolled processing structures.
- Deep Architectures: Transformer-based methods such as IPT exploit non-local modeling, but IPT uses a fixed image patch size and can encounter boundary-related issues.The passage continues by noting limitations associated with its design.
3. Method
The method improves blind denoising through both inference architecture and synthesized training degradations. It combines MAP-inspired learning with a multiscale Swin-Conv-UNet and a degradation pipeline covering diverse, signal-dependent and channel-correlated noise.
- Problem formulation: Deep blind denoising learns network parameters from paired noisy-clean images to capture both degradation-process knowledge and clean-image priors.The formulation treats the deep model as a compact unrolled inference of MAP estimation, while training data implicitly defines the degradation process.
- Swin-Conv-UNet: SCUNet inserts swin-conv blocks into a four-scale UNet with strided-convolution downscaling, transposed-convolution upscaling, and residual connections.The architecture uses channels of 64, 128, 256 and 512 across its four scales.
- Swin-Conv block: Each swin-conv block splits features between SwinT and residual convolution branches, then concatenates them through a 1×1 convolution with a residual connection.The split, branch processing, concatenation, and residual addition are defined by Eqs. (4)–(6).
- Swin-Conv block: The hybrid block combines local RConv modeling with non-local SwinT modeling, while multiscale processing further enhances both and grouped operations reduce computation and parameters.The paper describes SCUNet as a hybrid CNN-Transformer network.
- Training data synthesis: The synthesis pipeline creates paired data by applying diverse degradations, resizing, reverse-forward tone mapping, random ordering, and double degradation before cropping patches.The pipeline aims to avoid the laborious construction of large real noisy/clean image-pair collections.
- Training data synthesis: The noise model includes correlated generalized Gaussian noise, signal-dependent Poisson noise, and grayscale Poisson variants for training blind denoising models.The generalized Gaussian model uses a 3×3 RGB covariance matrix, while Poisson noise is sampled with α uniformly from 2 to 4.
4. Discussion
The proposed blind-denoising pipeline targets practical real-image noise, while SCUNet is evaluated against established methods using PSNR and qualitative comparisons. The authors report improved generalization to varied unseen noise and strong grayscale denoising results.
- The training pipeline is designed for deep blind image denoising and resizes high-quality images to produce corresponding clean images.
- Table 1 compares average PSNR(dB) across Set12, BSD68, and Urban100 at noise levels 15, 25, and 50, highlighting the best and second-best results.
- 27.76/0.8222, 27.60/0.8243, 28.29/0.8420, 28.16/0.8423, 28.41/0.8480, and 28.62/0.8560 are reported for NLRN, FOCNet, DAGL, DRUNet, SwinIR, and SCUNet, respectively.
- The authors report that practical blind denoising addresses more varied real-image noise than blind Gaussian or camera-sensor-noise removal, which can generalize poorly to unseen noise.
- The proposed model uses a larger degradation space and is reported to outperform Noise Clinic, making it the first deep model readily applicable to real applications.
5. Experiments
Experiments show that SCUNet improves synthetic Gaussian denoising while its broad degradation synthesis supports practical blind denoising on real images. Resizing, varied noise types, and architectural design each contribute to the reported results.
- Synthetic Gaussian Denoising: SCUNet achieves significantly better PSNR than other methods across grayscale Gaussian denoising noise levels and datasets.It surpasses DnCNN and FFDNet by average gains of 0.6dB on Set12, 0.3dB on BSD68, and 1.6dB on Urban100.
- Synthetic Gaussian Denoising: SCUNet recovers repetitive fine structures and preserves sharpness better than competing methods in qualitative Gaussian-denoising comparisons.On “Barbara,” it produces more visually pleasant results, while other methods miss some repetitive lines or introduce smoothness.
- Synthetic Gaussian Denoising: SCUNet produces the best overall color Gaussian denoising performance across CBSD68, Kodak24, McMaster, and Urban100.Average PSNR gains over DnCNN, FFDNet, and DSNet are 0.5dB, 0.7dB, 1.1dB, and 1.6dB on the four datasets, respectively.
- Practical Blind Image Denoising: SCUNet achieves the best trade-off among FLOPs, runtime, and parameter count compared with DRUNet and SwinIR.Its lower FLOPs result from combining UNet with the swin-conv block, while efficient implementation can further reduce runtime.
- Practical Blind Image Denoising: SCUNet and SCUNetG achieve strong visual real-image denoising results, including removal of processed camera sensor noise and recovery of underlying edges.The models also handle real images with unknown noise types and levels, including likely processed sensor noise and Gaussian-like noise.
- Practical Blind Image Denoising: Resizing during training-data synthesis improves generalization to resized noisy images compared with training without resizing.The model trained without resizing fails to completely remove Gaussian and unknown processed camera sensor noise from bicubically upsampled images.
6. Conclusion
The paper addresses practical blind image denoising through a network architecture for stronger local and non-local prior modeling and a data-synthesis pipeline for realistic degradations. Experiments support the architecture for Gaussian denoising and the trained model’s practicality on real noisy images.
- Conclusion: The paper proposes a swin-conv block in a UNet and a synthesis pipeline combining varied noise, resizing, random shuffling, and double degradation.The block combines residual convolutional and Swin Transformer modeling abilities.
- Conclusion: Extensive experiments demonstrate the architecture’s effectiveness for Gaussian denoising and the blind model’s practicability on real noisy images.