Source-linked AI summary
Variational Denoising Network: Toward Blind Noise Modeling and Removal
Zongsheng Yue, Hongwei Yong, Qian Zhao, Lei Zhang, Deyu Meng
TL;DR
Blind image denoising is difficult because real noise is heterogeneous, unknown in advance, and hard to simulate comprehensively. VDN uses a neural variational posterior to jointly infer clean images and noise variances within a Bayesian framework, and experiments report superiority on blind denoising, including complicated non-i.i.d. real noise. Its scope is bounded by the challenges of generalizing classical methods and the non-i.i.d. Gaussian noise assumption used in its model.
Problem
Blind denoising remains challenging because real noise is difficult to pre-know for MAP methods and comprehensively simulate for deep-learning methods.
Method
VDN parameterizes a variational posterior with neural networks, treating the clean image and noise variances as latent variables conditioned on the noisy input.
Results
VDN is reported to outperform previous blind-denoising methods and estimate and remove complicated non-i.i.d. noise, including types absent from training data.
Takeaways & Limitations
VDN provides joint blind denoising and noise estimation with a generative distribution that supports interpretability and test-time estimation of noise.
Takeaways & Limitations
The model assumes pixel-wise non-i.i.d. Gaussian noise, while real noise can exhibit spatially variant and signal-dependent characteristics.
Abstract
from arXiv · showhide
Blind image denoising is an important yet very challenging problem in computer vision due to the complicated acquisition process of real images. In this work we propose a new variational inference method, which integrates both noise estimation and image denoising into a unique Bayesian framework, for blind image denoising. Specifically, an approximate posterior, parameterized by deep neural networks, is presented by taking the intrinsic clean image and noise variances as latent variables conditioned on the input noisy image. This posterior provides explicit parametric forms for all its involved hyper-parameters, and thus can be easily implemented for blind image denoising with automatic noise estimation for the test noisy image. On one hand, as other data-driven deep learning methods, our method, namely variational denoising network (VDN), can perform denoising efficiently due to its explicit form of posterior expression. On the other hand, VDN inherits the advantages of traditional model-driven approaches, especially the good generalization capability of generative models. VDN has good interpretability and can be flexibly utilized to estimate and remove complicated non-i.i.d. noise collected in real scenarios. Comprehensive experiments are performed to substantiate the superiority of our method in blind image denoising.
1 Introduction
Blind image denoising seeks to recover clean images from real noisy observations whose complicated, unknown distributions challenge both model-driven and data-driven methods. VDN addresses this by jointly inferring clean images and noise distributions in one Bayesian variational framework.
- 1 Introduction: Real-image noise accumulates from multiple sources, making its information difficult to access accurately during clean-image recovery.Sources include capturing instruments, transmission media, and image quantization.
- 1 Introduction: Classical Bayesian methods use fidelity and regularization terms but can overfit known noise types and generalize poorly to unknown complicated noise.Their design depends on understanding the data-generation mechanism and suitable image priors.
- 1 Introduction: Blind denoising remains challenging because real noise is difficult to pre-specify for MAP models and comprehensively simulate for deep-learning models.
- 1 Introduction: VDN jointly infers the clean image and noise distribution from a noisy image using a variational posterior conditioned on that input.The clean image and noise variances are treated as latent variables.
- 1 Introduction: The method provides explicit posterior hyper-parameters, enabling efficient blind denoising with automatic noise estimation on test images.
- 1 Introduction: VDN simultaneously performs noise estimation and blind denoising within one Bayesian framework while modeling spatially relevant non-i.i.d. noise.The paper also presents it as a generative approach with interpretability and generalization to complicated unseen noise.
2 Related Work
Prior image-denoising research divides mainly into model-driven Bayesian methods that encode image priors and data-driven deep networks learned from noisy-clean image pairs. Later methods incorporate non-local structure, spatially varying noise handling, and camera-image generation modeling.
- 2 Related Work: Model-driven methods construct MAP objectives with fidelity and regularization terms encoding pre-known image priors.Examples include total variation, anisotropic diffusion, wavelet coring, non-local means, and CBM3D.
- 2 Related Work: Data-driven methods learn deep neural denoisers directly from collections of noisy-clean image pairs instead of pre-setting an image prior.Early approaches included convolutional networks and auto-encoders, followed by deeper memory-based architectures.
- 2 Related Work: Recent networks embed non-local image properties into deep models to facilitate denoising.NLRN, N3Net, and UDNet are cited examples.
- 2 Related Work: FFDNet improves flexibility for spatially variant noise by pre-evaluating the noise level and supplying it with the noisy image.
- 2 Related Work: Other methods simulate image-generation processes in cameras to address real-image noise.Guo et al. and Brooks et al. are cited as examples.
3 Variational Denoising Network for Blind Noise Modeling
VDN formulates blind denoising as Bayesian inference over a latent clean image and pixel-wise noise variances, with neural networks predicting the posterior parameters from the noisy input. Its explicit posterior supports joint denoising and noise estimation, while the generative formulation targets complicated non-i.i.d. noise and improved generalization.
- Bayesian model: VDN approximates the posterior of the latent clean image and noise variances conditioned on each observed noisy image.The training objective uses noisy/clean image pairs to construct a variational parametric approximation that can infer both quantities at test time.
- Bayesian model: The observation model uses pixel-wise Gaussian noise with spatially varying variances instead of a conventional i.i.d. noise assumption.This design is intended to represent spatially variant and signal-dependent characteristics of real noise.
- Variational posterior: D-Net predicts posterior parameters for the clean image, while S-Net predicts posterior parameters for the noise variances from the noisy image.The two networks share parameters across training examples and provide explicit functions for test-time inference.
- Optimization: The network parameters are optimized by minimizing the negative variational lower bound, whose terms represent noisy-image likelihood and posterior-prior discrepancies.The analytically differentiable objective supports stochastic-gradient training through backpropagation.
- Inference: At test time, D-Net directly outputs the denoising result through µ, while S-Net infers per-pixel noise variance from the inverse-Gamma posterior mode.The D-Net uses a depth-4 U-Net with residual learning for the clean-image parameter.
- Discussion: The proposed generative formulation combines interpretability with explicit prediction functions and is designed to estimate and remove complicated non-i.i.d. noise.The discussion connects this design to efficient inference and generalization beyond the training noise distribution.
4 Experimental Results
Experiments evaluate VDN on synthetic non-i.i.d. noise, AWGN, and real-world benchmarks, including SIDD and DND. VDN consistently performs strongly, estimates noise variance maps, and remains effective across noise settings and hyper-parameter choices.
- Experimental setup: VDN is evaluated on synthetic and real datasets in sRGB space, using comparisons with classical, discriminative, and blind-denoising methods.The comparison set includes CBM3D, WNNM, NCSR, MLP, DnCNN-B, MemNet, FFDNet, UDNet, and CBDNet.
- Synthetic non-i.i.d. noise: Testing uses spatially varying non-i.i.d. Gaussian noise, with training and testing maps deliberately differentiated to assess robustness and generalization.Three test cases are generated from maps different from the training map.
- Synthetic non-i.i.d. noise: VDN outperforms competing methods across all non-i.i.d. test cases and exceeds FFDNet by about 0.25dB on average despite automatically inferring the noise distribution.The reported comparison also attributes apparent weaknesses of several discriminative methods to overfitting training noise bias.
- AWGN: On AWGN with σ = 15, 25, 50, VDN achieves the best or at least comparable performance to FFDNet.This result is reported despite VDN being designed and trained for non-i.i.d. noise.
- Real-world noise: On SIDD, VDN outperforms retrained DnCNN-B by 0.87 PSNR and CBDNet by 0.60dB under identical training conditions.The paper also reports that VDN outperforms available competing methods on the SIDD benchmark.
- Real-world noise: On DND, VDN surpasses all competing methods, with the paper attributing its advantage over CBDNet mainly to deep variational inference optimization.CBDNet uses the same optimized network structure, including S-Net for noise estimation and D-Net for denoising.
- Noise estimation: S-Net predicts image-content-related noise variance maps on real benchmarks, consistent to some extent with the signal dependence of real noise.These maps provide evidence that the learned noise model captures spatially varying noise structure.
- Hyper-parameter analysis: VDN performs consistently well across the tested values of local-window size p, where each noise-variance estimate is formed in a p × p neighborhood.The mode of the inverse gamma distribution provides the local estimate for σ²_i.
5 Conclusion
The paper proposes VDN, a variational inference algorithm that jointly performs blind image denoising and noise estimation within one Bayesian framework. Experiments demonstrate its superiority for blind image denoising, with potential application to other low-level vision tasks.
- VDN learns an approximate posterior conditioned on the noisy input, with the clean image and noise variances as latent variables.This posterior enables both denoising and noise estimation within a unified Bayesian formulation.
- VDN is a generative method that estimates the noise distribution directly from the input data.
- Comprehensive experiments demonstrate VDN’s superiority over previous works on blind image denoising.
- VDN can support super-resolution and deblurring by enabling fidelity terms based on estimated non-i.i.d. noise distributions rather than traditional i.i.d. Gaussian noise.