Source-linked AI summary
Toward Convolutional Blind Denoising of Real Photographs
Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, Lei Zhang
TL;DR
Deep CNN denoisers often overfit AWGN and therefore struggle with real-world photographs whose noise is shaped by signal dependence and camera processing. CBDNet combines realistic noise modeling, synthetic and real noisy-clean training pairs, and asymmetric noise estimation, and it achieves state-of-the-art quantitative and visual results across three real-world datasets. The noise estimation design also enables interactive denoising, though a single blind CNN performs comparably at lower noise levels and worse under heavy noise.
Problem
Deep CNN denoisers trained on simplified AWGN generalize poorly to real-world photographs with more complicated noise.
Method
CBDNet combines heteroscedastic Gaussian and ISP-based noise modeling, synthetic and real noisy-image training, and a noise-estimation subnetwork with asymmetric loss.
Results
CBDNet achieves state-of-the-art quantitative metrics and visual quality on three real-world noisy image datasets.
Takeaways & Limitations
Realistic noise modeling, mixed synthetic and real training data, and noise estimation improve CBDNet's applicability to real-world noisy photographs and enable interactive denoising.
Takeaways & Limitations
A single blind CNN performs on par with CBDNet at lower noise levels but is inferior for images with heavy noise.
Abstract
from arXiv · showhide
While deep convolutional neural networks (CNNs) have achieved impressive success in image denoising with additive white Gaussian noise (AWGN), their performance remains limited on real-world noisy photographs. The main reason is that their learned models are easy to overfit on the simplified AWGN model which deviates severely from the complicated real-world noise model. In order to improve the generalization ability of deep CNN denoisers, we suggest training a convolutional blind denoising network (CBDNet) with more realistic noise model and real-world noisy-clean image pairs. On the one hand, both signal-dependent noise and in-camera signal processing pipeline is considered to synthesize realistic noisy images. On the other hand, real-world noisy photographs and their nearly noise-free counterparts are also included to train our CBDNet. To further provide an interactive strategy to rectify denoising result conveniently, a noise estimation subnetwork with asymmetric learning to suppress under-estimation of noise level is embedded into CBDNet. Extensive experimental results on three datasets of real-world noisy photographs clearly demonstrate the superior performance of CBDNet over state-of-the-arts in terms of quantitative metrics and visual quality. The code has been made available at https://github.com/GuoShi28/CBDNet.
1. Introduction
Real-world photographic noise differs substantially from AWGN because camera noise sources and ISP processing complicate its distribution. CBDNet addresses this mismatch with realistic noise modeling, mixed synthetic and real training data, and asymmetric noise estimation, achieving state-of-the-art results on three real-world datasets.
- Real camera noise combines multiple sources and is altered by demosaicing, Gamma correction, and compression, making it unlike AWGN.
- Deep CNN denoisers trained for AWGN generalize poorly to real photographs, while non-blind models may smooth image details.
- CBDNet models real noise with heteroscedastic Gaussian components and the in-camera processing pipeline, yielding over 5 dB PSNR gain over AWGN on DND.
- CBDNet combines synthetic noisy images with real noisy photographs to better characterize real-world noise and improve denoising performance.
- Its noise estimation subnetwork uses asymmetric loss to penalize under-estimated noise more heavily and supports interactive adjustment of the noise level map.
- Experiments on NC12, DND, and Nam show state-of-the-art quantitative metrics and visual quality for CBDNet.
2. Related Work
Prior work largely targets Gaussian denoising, whereas real-image noise requires richer statistical and cross-channel models and often separate estimation and denoising stages. CBDNet builds on these directions with learned noise estimation and non-blind denoising for real photographs.
- Deep denoising CNNs improved Gaussian denoising, including blind AWGN models, but real-image noise remains more complex than AWGN.
- CBDNet uses a noise estimation subnetwork and asymmetric loss to suppress noise-level under-estimation during blind denoising.
- Poisson-Gaussian models represent signal-dependent photon noise and stationary disturbances, while camera response and quantization can further affect raw-noise modeling.
- In-camera processing can create spatially and cross-channel correlated noise, motivating models beyond channel-independent assumptions.
- Blind denoising commonly separates noise estimation from non-blind denoising, with prior methods using statistical, Bayesian, and low-rank approaches.
3. Proposed Method
CBDNet combines a realistic, signal-dependent noise model with a two-subnetwork architecture for blind denoising of real photographs. It uses asymmetric noise-estimation loss, interactive noise-map adjustment, and alternating synthetic/real-image training.
- 3.1. Realistic Noise Model: Poisson-Gaussian noise is modeled as signal-dependent and stationary components, while demosaicing and Gamma correction capture spatial and chromatic correlations introduced by the ISP.The resulting synthetic noise is channel- and space-dependent.
- 3.2. Network Architecture: CBDNet contains a noise-estimation subnetwork that predicts a noise-level map and a non-blind denoising subnetwork that uses the map with the noisy image.The estimated map can be scaled before denoising for interactive adjustment.
- 3.3. Asymmetric Loss and Model Objective: Asymmetric loss penalizes noise under-estimation more heavily because non-blind denoisers are sensitive to under-estimated noise but robust to over-estimated noise.The parameter setting 0 < α < 0.5 increases the penalty for under-estimation.
- 3.3. Asymmetric Loss and Model Objective: The model objective combines reconstruction, asymmetric noise-estimation, and total-variation losses, with perceptual loss added for qualitative visual evaluation.The asymmetric and TV terms are weighted by λasymm and λTV.
- 3.4. Training with Synthetic and Real Noisy Images: The noise model can generate unlimited synthetic images, while real nearly noise-free training targets require averaging hundreds of images from static scenes.The real-image data address noise-model mismatch but impose scene constraints.
- 3.4. Training with Synthetic and Real Noisy Images: CBDNet is trained with alternating batches of synthetic and real noisy images, using 400 BSD500, 1600 Waterloo, 1600 MIT-Adobe FiveK, and 120 RENOIR images.Synthetic batches use all losses, whereas real-image batches omit noise-level supervision unavailable for those images.
4. Experimental Results
Experiments on NC12, DND, and Nam evaluate CBDNet against blind and non-blind denoisers, noise-model variants, training-data choices, and interactive noise adjustment. CBDNet achieves strong real-photograph denoising while preserving structures, with results influenced by realistic noise modeling and combined synthetic-real training.
- Datasets: Three real-world datasets—NC12, DND, and Nam—are used for qualitative and quantitative evaluation, with NC12 lacking clean ground truth.DND provides 50 noisy/nearly noise-free pairs, while Nam uses 11 static scenes and 500 JPEG noisy images per scene to form nearly noise-free means.
- Comparison with State-of-the-arts: On NC12, CBDNet removes noise while preserving salient image structures, whereas competing methods remain limited in dark regions.
- Comparison with State-of-the-arts: CBDNet achieves the highest DND PSNR/SSIM results, slightly exceeding MCWNNM and TWSC, while balancing noise removal and structure preservation.CBDNet processes a 512 × 512 image in about 0.4s; the comparison also reports poor generalization for CDnCNN-B and limited performance for several AWGN denoisers.
- Comparison with State-of-the-arts: CBDNet(JPEG) outperforms CBDNet by approximately 1.3 dB in PSNR and achieves the best comparison performance on Nam.
- Ablation Studies: Heterogeneous Gaussian modeling improves PSNR by about 0.8–1 dB over Gaussian modeling without ISP, while the full model combines heterogeneous noise with ISP processing.The ablation compares Gaussian, heterogeneous Gaussian, and ISP-augmented variants on DND and Nam.
- Ablation Studies: CBDNet(All) outperforms synthetic-only and real-only training, and α = 0.3 improves generalization to unknown real noise by penalizing under-estimation more heavily.Users can also tune the estimated noise map interactively; γ = 0.7 and γ = 1.3 are reported for two DND images to adjust texture preservation and noise removal.
5. Conclusion
CBDNet’s conclusion identifies realistic noise modeling and mixed synthetic-real training as central to blind denoising of real photographs, with noise estimation enabling asymmetric-loss training and interactive adjustment.
- Realistic noise modeling, including heterogeneous Gaussian noise and the ISP pipeline, is critical for applying synthetic-image models to real photographs.
- Combining synthetic and real noisy images can boost denoising performance during training.
- A noise estimation subnetwork enables asymmetric loss, improving generalization to real-world noise and supporting convenient interactive denoising.