Source-linked AI summary
When AWGN-based Denoiser Meets Real Noises
Yuqian Zhou, Jianbo Jiao, Haibin Huang, Yang Wang, Jue Wang, Honghui Shi, Thomas Huang
TL;DR
The paper addresses the poor transfer of denoisers trained on pixel-independent synthetic noise to spatially and channel-wise correlated, variant real noise. It trains a noise-estimator-plus-denoiser model with mixed AWGN and RVIN, then uses Pixel-shuffle Down-sampling to adapt it. The resulting method achieves state-of-the-art performance on real sRGB images in the DND benchmark among models trained with synthetic noises.
Problem
Denoisers trained on pixel-independent AWGN generalize poorly to real noises that are spatially and channel-wise correlated and variant.
Method
The method trains an explicit noise estimator and conditional denoiser with mixed AWGN and RVIN, then adapts them to real noise using Pixel-shuffle Down-sampling.
Results
The method achieves state-of-the-art performance on the DND benchmark and other real noisy RGB images among models trained only with synthetic noises.
Takeaways & Limitations
Pixel-shuffle Down-sampling enables the synthetic-noise-trained model to process real noise without explicit real-noise modeling or real-noise dataset images.
Takeaways & Limitations
The approach assumes that RGB real noise can be locally approximated as AWGN and that pixel-shuffling can break correlated patterns into pixel-independent ones suitable for adaptation.
Abstract
from arXiv · showhide
Discriminative learning-based image denoisers have achieved promising performance on synthetic noises such as Additive White Gaussian Noise (AWGN). The synthetic noises adopted in most previous work are pixel-independent, but real noises are mostly spatially/channel-correlated and spatially/channel-variant. This domain gap yields unsatisfied performance on images with real noises if the model is only trained with AWGN. In this paper, we propose a novel approach to boost the performance of a real image denoiser which is trained only with synthetic pixel-independent noise data dominated by AWGN. First, we train a deep model that consists of a noise estimator and a denoiser with mixed AWGN and Random Value Impulse Noise (RVIN). We then investigate Pixel-shuffle Down-sampling (PD) strategy to adapt the trained model to real noises. Extensive experiments demonstrate the effectiveness and generalization of the proposed approach. Notably, our method achieves state-of-the-art performance on real sRGB images in the DND benchmark among models trained with synthetic noises. Codes are available at https://github.com/yzhouas/PD-Denoising-pytorch.
Introduction
Real-image denoising remains difficult because models trained on pixel-independent synthetic noise generalize poorly to spatially and channel-wise complex real noise. The paper proposes mixed AWGN–RVIN training and Pixel-shuffle Down-sampling to adapt such denoisers to real noise.
- Problem: Synthetic-noise denoisers often degrade rapidly on real images because real noise is spatially and channel-wise variant and correlated rather than pixel-independent AWGN.FFDNet can produce over-smoothed results with manually adjusted high noise-level maps, while blind real-image denoising lacks accurate real-noise modeling.
- Method: The proposed basis network combines an explicit noise estimator with a conditional denoiser trained on mixed AWGN and RVIN.The model is designed for blind and non-blind denoising and can handle pixel-independent spatially and channel-wise variant noise.
- Method: Pixel-shuffle Down-sampling adapts the trained denoiser by breaking spatially correlated real noise into pixel-independent noise components.This divide-and-conquer strategy is intended to reduce the domain gap between synthetic training noise and real noise.
- Results: The method achieves state-of-the-art performance on the DND benchmark and other real noisy RGB images among models trained only with synthetic noises.The method uses no images or prior metadata from real-noise datasets, and PD can also boost some existing denoising models.
Related Work
Prior denoising research includes discriminative CNN-based methods for synthetic Gaussian noise and approaches tailored to the complexity of real camera noise. Real-noise methods use realistic noise modeling, noise profiling, and data augmentation.
- Synthetic-noise denoising: CNN-based denoisers have achieved strong performance on synthetic Gaussian noise, including methods using MLPs, trainable reaction diffusion, and deep CNN architectures.Representative designs use residual learning, batch normalization, dilated convolution, skip connections, and ResNet structures.
- Real-noise denoising: Real camera noise arises from multiple noise sources and in-camera processing, producing spatial and channel correlation, variance, and signal dependence.Photon noise, read-out noise, demosaicing, color transformation, and gamma transformation contribute to these characteristics.
- Real-noise denoising: Existing real-image denoising attempts use realistic noise modeling, noise profiling, or data augmentation to address real-noise complexity.CBDNet models realistic noise with an in-camera pipeline and trains an explicit noise estimator with a larger penalty for under-estimation.
Methodology
The methodology trains a flexible denoiser on mixed AWGN-RVIN and adapts it to real noise with Pixel-shuffle Down-sampling, which addresses spatial/channel correlation and variance. The model combines explicit noise estimation with blind and non-blind denoising, while refinement merges texture details with flat regions.
- Basis Noise Model: Mixed AWGN-RVIN training forms the basis noise model, with RVIN targeting defective pixels such as dead pixels and long-exposure artifacts.
- Basis Model Structure: The basis model uses a noise estimator E followed by a non-blind denoiser R to jointly learn the image residual.E predicts noise-level maps, which are concatenated with the noisy image before denoising.
- Basis Model Structure: The network is supervised by noise-estimation, blind-denoising, and non-blind-denoising objectives combined as a weighted sum with equal loss weights.The noise maps encode AWGN standard deviations and RVIN corrupted-pixel ratios, normalized to [0,1].
- Pixel-shuffle Down-sampling (PD) Adaptation: Pixel-shuffle Down-sampling adapts the synthetic-noise-trained model by preserving noisy pixels while breaking spatially correlated real noise into pixel-wise independent patterns.The adaptation selects the smallest stride whose changing factor falls below τ = 0.008; real-noise experiments show an abrupt drop at stride s = 2.
- Pixel-shuffle Down-sampling (PD) Adaptation: PD refinement denoises a pixel-shuffled mosaic, refills and reprocesses subimages to obtain texture details, then combines them with over-smoothed flat regions.The flat-region branch uses refined noise maps and pixel-shuffle upsampling before the final combination kF + (1 − k)T.
Experiments
Experiments evaluate the model on synthetic mixed and spatially variant noises, then test PD adaptation on real DND images. Results show gains from mixed AWGN-RVIN training, pixel-shuffle refinement, and the full refinement pipeline.
- Synthetic Noise Evaluation: The proposed model achieves the best reported performance on mixed AWGN-RVIN removal in Set20 by explicitly disentangling the different noise types.Its conditional denoiser can differentiate mixed noises from other noise types.
- Synthetic Noise Evaluation: Fully convolutional models trained with spatially invariant AWGN can handle spatially variant Gaussian noise, while pixel-wise estimation improves flexibility for real-noise adaptation.The proposed network explicitly estimates a pixel-wise noise map, unlike scalar estimators used by BM3D and FFDNet.
- Real RGB Noise Evaluation: PD produces spatially variant denoising that smooths backgrounds while preserving texture details in full-blind DND results.The comparisons highlight texture details and background or edge regions separately.
- Real RGB Noise Evaluation: +0.83 dB: the AWGN-RVIN model with PD refinement outperforms CBDNet on the DND benchmark among models trained only with synthetic noise.Compared with its base model, the proposed adaptation improves real-noise performance by 5.8 dB without using DND prior data.
- Ablation Study on Real RGB Noise: Stride s = 2 breaks spatial correlation in real noise, whereas larger strides can destroy textural structures and details.The fabric texture becomes invisible when s > 2.
- Ablation Study on Real RGB Noise: The full refinement pipeline has the best visual quality because it removes artifacts produced by intermediate refinement variants I and DI.The pipeline combines adaptation, denoising, refilling, repeated denoising, averaging, and flat-region refinement.
Conclusions
The paper revisits real-image blind denoising by training on AWGN-RVIN noise and adapting the model to real noise with Pixel-shuffle Down-sampling. Results indicate that this strategy handles spatial and channel variance and correlation while preserving details.
- The proposed network combines an explicit multi-type, multi-channel noise estimator with an adaptive conditional denoiser trained on AWGN-RVIN noise.
- Figure 9 presents an ablation study of the merging factor k with s = 2.
- Pixel-shuffle Down-sampling is used as a refinement strategy to generalize the AWGN-RVIN-trained network to real noises.
- The method processes spatially and channel-variant, correlated real noises without explicit noise modeling.
- The study reports qualitative improvements in spatially variant denoising and detail preservation after applying Pixel-shuffle Down-sampling.