PaperlayerView exact revision on arXiv

Source-linked AI summary

Blind Universal Bayesian Image Denoising with Gaussian Noise Level Learning

Majed El Helou, Sabine Süsstrunk

arXiv:1907.03029v2cs.CVeess.IV

TL;DR

Knowing the noise level before training or at test time limits practical denoising, motivating blind universal models. The paper derives fusion denoising and adapts it to real images; its networks generalize to unseen noise levels and improve real-image denoising performance.

  • Problem

    Requiring the exact noise level before training or at test time is a practical limitation, and denoising is challenging as an ill-posed inverse problem.

    Source: §I · p. 1, §II · p. 2

  • Method

    The authors derive optimal fusion denoising under a Gaussian image-prior assumption, then adapt the network with a learned fusion function for real images.

    Source: §I · p. 1

  • Results

    Fusion Net generalizes better to higher unseen noise levels, while BUIFD improves real-image denoising performance on both training and unseen noise levels.

    Source: §V · p. 12

  • Takeaways & Limitations

    BUIFD’s improvements on unseen noise levels address a setting where needing noise-level knowledge at test time is a practical limitation.

    Source: §II · p. 2, §V · p. 12

    The theoretical fusion-denoising derivation assumes a Gaussian image prior, which does not necessarily apply to real-world images.

    Source: §I · p. 1

Abstract

from arXiv · show

Blind and universal image denoising consists of using a unique model that denoises images with any level of noise. It is especially practical as noise levels do not need to be known when the model is developed or at test time. We propose a theoretically-grounded blind and universal deep learning image denoiser for additive Gaussian noise removal. Our network is based on an optimal denoising solution, which we call fusion denoising. It is derived theoretically with a Gaussian image prior assumption. Synthetic experiments show our network's generalization strength to unseen additive noise levels. We also adapt the fusion denoising network architecture for image denoising on real images. Our approach improves real-world grayscale additive image denoising PSNR results for training noise levels and further on noise levels not seen during training. It also improves state-of-the-art color image denoising performance on every single noise level, by an average of 0.1dB, whether trained on or not.

I. INTRODUCTION

Denoising is an ill-posed restoration task, while practical use often cannot rely on knowing the noise level before training or at test time. The paper derives fusion denoising and develops networks that generalize to unseen noise levels, including an adaptation for real images.

I. INTRODUCTION

  • A single blind universal model addresses the practical difficulty of knowing noise levels before training or at test time.The paper describes denoising as an ill-posed inverse problem.Source: §I · p. 1
  • Fusion denoising is derived under a Gaussian image-prior assumption and fuses the noisy input with a prior weighted by the signal-to-noise ratio.The paper reports that DnCNN models the optimal function at training noise levels, while Fusion Net generalizes better to unseen levels.Source: §I · p. 1
  • BUIFD improves state-of-the-art denoising on trained noise levels for grayscale and color BSD68/CBSD68 images, and unseen-level grayscale BSD68 results improve by multiple PSNR points.The synthetic Fusion Net also far outperforms DnCNN at unseen noise levels.Source: §I · p. 1
  • Because the Gaussian-prior assumption may not hold for real images, BUIFD adapts Fusion Net by learning a fusion function for real-image denoising.The authors also describe synthetic experiments assessing network optimality and generalization against the Bayesian solution.Source: §I · pp. 1–2

II. RELATED WORK

The related work frames denoising around image priors and the practical need for blind, universal models. The theoretical framework derives an SNR-weighted fusion estimate under Gaussian image and additive-noise assumptions.

II. RELATED WORK

  • Denoising methods rely on image priors, which can be specified through assumptions, sparsity, self-similarity, or learned feature representations.The proposed network uses learned feature representations as its prior.Source: §II · p. 2
  • Blind, universal denoising avoids requiring known noise levels and multiple stored models, including when noise varies spatially.The framework assumes additive white Gaussian noise, while noting that this model does not cover every practical noise setting.Source: §II · p. 2
  • Some restoration approaches require 25 denoising networks below noise level 50, while DnCNN is described as both blind and universal.The related-work discussion contrasts these approaches with BUIFD and BM3D as evaluation comparators.Source: §II · p. 2

A. Theoretical Framework

  • The theoretical framework assumes an additive, independent, identically distributed Gaussian noise model and a Gaussian image prior centered at the prior mean.The authors note that the Gaussian prior is later modified for real-world images.Source: A. Theoretical Framework · pp. 2–3
  • Under these assumptions, the posterior is Gaussian, and its MAP estimate is also its conditional expected value.The derivation obtains posterior parameters by matching Gaussian expressions.Source: A. Theoretical Framework · p. 3
  • Fusion denoising combines the noisy image with the prior using an SNR-based weight.The paper identifies the operation as fusion denoising and illustrates the Fusion Net in Fig. 1(b).Source: A. Theoretical Framework · p. 3, Figure 1 · p. 3

B. Fusion Net Architecture

Fusion Net adapts the blind DnCNN design to learn the prior and SNR weighting separately, then uses them to implement the theoretically derived fusion denoising function.

B. Fusion Net Architecture

  • The predicted fusion factor is inversely proportional to SNR and proportional to noise level, controlling the prior’s weight in the fusion.The prior CNN predicts the prior mean; the SNR CNN predicts the factor used for fusion.Source: B. Fusion Net Architecture · p. 3
  • Fusion Net retains the DnCNN architecture design while separately predicting the image prior and SNR function for weighted fusion.Its parallel CNNs use convolution, ReLU, and batch-normalization blocks.Source: B. Fusion Net Architecture · pp. 3–4
  • Unlike DnCNN’s noise prediction and subtraction, Fusion Net learns the optimal fusion function that combines the noisy input with the predicted prior.The architecture keeps a residual-learning connection while preserving parallel prior- and SNR-prediction networks.Source: B. Fusion Net Architecture · pp. 3–4

C. Fusion Net Feature Disentangling

Fusion Net separates prior and SNR-function predictions and uses explicit SNR supervision alongside reconstruction loss to encourage these intermediate outputs to match their theoretical roles.

C. Fusion Net Feature Disentangling

  • With near-zero reconstruction loss, coefficient matching in the fusion equation makes intermediate outputs a and b approximately equal to the prior and SNR function.The extension beyond training data assumes the training dataset is sufficiently general.Source: C. Fusion Net Feature Disentangling · p. 4
  • The loss combines denoised-image reconstruction with a penalty for predicting the SNR function.The denoised output is expressed as a·b + y·(1−b), with α weighting the reconstruction term.Source: C. Fusion Net Feature Disentangling · p. 4
  • Fusion Net minimizes denoised-image reconstruction loss while separately learning the image prior and SNR function, and also uses explicit SNR information.This differs from DnCNN’s use of ground-truth noise-free images alone during training.Source: C. Fusion Net Feature Disentangling · p. 4

D. Denoising Non-Gaussian Images

BUIFD adapts the theoretical fusion strategy for real images by learning a data-dependent fusion function from prior and noise-level predictions. Its evaluation considers training and unseen noise levels.

D. Denoising Non-Gaussian Images

  • BUIFD targets real-image denoising at both training noise levels and levels not seen during training.The approach is evaluated on real images, where the simple Gaussian image-prior assumption does not apply.Source: D. Denoising Non-Gaussian Images · p. 4
  • BUIFD replaces theoretical optimal fusion with pixel-wise product fusion followed by three trainable convolution layers.The layers learn a data-dependent fusion function.Source: D. Denoising Non-Gaussian Images · pp. 4–5
  • The predicted noise level controls the balance between observation and prior: lower noise gives the observation more weight, while higher noise gives the prior more.The prior predictor estimates the expected value of the real-world intensity distribution for each pixel.Source: D. Denoising Non-Gaussian Images · p. 4
  • The fusion receives the noisy image, prior prediction, noise-level prediction, and two additional pixel-wise products that reduce the convolution layers’ learning burden.The noise-level prediction is normalized to [0, 1].Source: D. Denoising Non-Gaussian Images · pp. 4–5

E. Relation With the Bayesian Framework

BUIFD borrows Bayesian fusion while adapting beyond the Gaussian image-prior assumption, predicting the prior mean rather than a full pixel-intensity distribution. The paper connects its theoretical gain at unseen noise level 70 with an approximately 3 dB gain on real BSD68 images at noise level 75.

E. Relation With the Bayesian Framework

  • BUIFD's product fusion weights the noisy input and learned prior using SNR, with three linear convolutional layers intended to preserve the Bayesian relation while allowing local smoothing.The fusion layers have no nonlinearities.Source: E. Relation With the Bayesian Framework · p. 5
  • BUIFD predicts the expected mean of the unknown pixel-intensity distribution, not the distribution itself, and the paper notes that even noise-free images cannot reveal that mean exactly.The authors say this makes it difficult to assess how closely the real-image network respects their distinctive prior definition.Source: E. Relation With the Bayesian Framework · p. 5
  • The real-image BSD68 experiment showed about 3 dB improvement at noise level 75, paralleling the theoretical improvement of 3.30 dB at unseen noise level 70.Source: E. Relation With the Bayesian Framework · p. 5
  • The authors hope this methodology motivates theoretical experiments with known optimal solutions and deep-network designs inspired by Bayesian solutions.Source: E. Relation With the Bayesian Framework · p. 5

A. Fusion Net Experimental Setup

The Fusion Net experiments use synthetic images and noise generated under a Gaussian image-prior assumption. Networks train on randomly sampled Gaussian noise levels from 5 to 25 and are evaluated at specified noise levels.

A. Fusion Net Experimental Setup

  • Training and test images are generated synthetically under the Gaussian image-prior assumption, using over 200k training patches and 256 test images of size 256 × 256.Training patches are 40 × 40 pixels; pixel intensities are sampled from N(127, 25²), then normalized to [0, 1].Source: A. Fusion Net Experimental Setup · p. 5
  • Noise standard deviations are sampled uniformly from [5, 25] for training and resampled at each epoch, while test images use specified Gaussian noise levels.Training runs for 50 epochs with batch size 128 and Adam; the learning rate starts at 0.001 and is divided by 10 every 30 epochs, and α in Eq. (10) is 0.1.Source: A. Fusion Net Experimental Setup · p. 5

B. Fusion Net Evaluation

In the theoretical evaluation, Fusion Net generalizes better than DnCNN to unseen noise levels, while both perform similarly on training levels. The supplied real-image setup passages describe training and evaluation configurations but do not provide the benchmark table values.

C. Real-Image Experimental Setup

  • The grayscale BSD68 table compares PSNR/SSIM for non-blind and blind BM3D, DnCNN, and BUIFD, with network variants trained up to specified noise levels.Source: Table II · p. 6

B. Fusion Net Evaluation

  • At training noise levels, DnCNN and Fusion Net perform similarly and close to the optimal upper bound; the T-test null hypothesis of similar expected PSNR holds across those configurations.At noise level 25, the null hypothesis would be rejected only above significance level 0.053.Source: B. Fusion Net Evaluation · pp. 5–6
  • Fusion Net generalizes better to unseen noise levels than DnCNN, performing close to optimal up to noise level 60, with the gap growing as noise increases.Both networks perform well at training levels; the comparison uses the optimal mathematical solution as an upper bound.Source: B. Fusion Net Evaluation · p. 6

C. Real-Image Experimental Setup

  • The real-image networks use the DnCNN prior-predictor architecture, a shallower five-block noise-level network, and three 16-channel convolutional fusion layers.The noise-level output is normalized to [0, 1]; the training setup uses 50 × 50 patches with stride 10 and batch size 128.Source: C. Real-Image Experimental Setup · pp. 6–7
  • The real-image figures include qualitative comparisons at seen noise level 25 and unseen level 75, while the color benchmark table compares PSNR/SSIM on CBSD68.The supplied captions identify the comparisons but do not provide table cell values.Source: Figure 3 · p. 7, Table IV · p. 8
  • BUIFD and DnCNN are trained with the same hyperparameters and simulated noise distributions, using 400 Berkeley grayscale images or 432 color images.The architectures are retained for both grayscale and color networks.Source: C. Real-Image Experimental Setup · p. 8

D. Real-Image Evaluation

On BSD68 grayscale images, BUIFD improves PSNR over DnCNN across tested noise levels, including unseen levels; it also performs well on color images and spatially varying noise. Visual comparisons show a trade-off between noise removal and detail preservation.

D. Real-Image Evaluation

  • At high noise levels, random variation in local noise means across small patches produces smudging-like or wave-like artifacts around low-frequency regions.The denoised image is formed by fusing the noisy input with a network-derived prior and noise-level values.Source: D. Real-Image Evaluation · p. 8
  • BUIFD improves grayscale PSNR over DnCNN by up to 0.7 dB and an average of 0.36 dB, including at unseen noise levels.The approach also outperforms non-blind BM3D by an average of 0.25 dB when trained across all noise levels, and performs just as well when trained up to level 55.Source: D. Real-Image Evaluation · p. 8
  • Without retraining, BUIFD handles row-wise spatially varying noise and outperforms DnCNN across the tested noise setups.Neither the prior-predicting nor noise-level-predicting branch was trained for spatially varying noise.Source: D. Real-Image Evaluation · p. 8
  • BUIFD outperforms CDnCNN at every tested color noise level, with an average PSNR improvement of about 0.1 dB.On average, the networks also outperform non-blind CBM3D by about 0.5 dB for CDnCNN and 0.6 dB for CBUIFD.Source: D. Real-Image Evaluation · p. 9
  • BUIFD balances detail preservation and noise removal, while non-blind BM3D blurs details and DnCNN leaves more noise in some smooth regions.For the grayscale example at noise level 45, BUIFD preserves details and outperforms non-blind BM3D in PSNR.Source: D. Real-Image Evaluation · pp. 9–10, Figure 5 · p. 10

E. Extended Benchmark Comparisons

The extended comparisons evaluate blind denoising on several image benchmarks and noise levels, using both traditional and learning-based methods. The section also describes a MemNet-based extension of the fusion approach and includes example visual results.

E. Extended Benchmark Comparisons

  • The evaluation covers blind luminance denoising on BSD68, Set5, Set14, Sun_Hays80, Urban100, and Manga109, with noise levels 10 to 80 in steps of 10.All learning-based methods in this section are trained up to noise level 55.Source: E. Extended Benchmark Comparisons · pp. 11–12
  • Fig. 8 shows a Set14 example, placing traditional non-blind methods in the top row and learning methods in the bottom row.The non-blind examples use noise level 25.Source: Figure 8 · p. 12
  • The comparisons include blind BM3D, EPLL, KSVD, WNNM, DnCNN, BUIFD, MemNet, and BUIFD(M), reporting PSNR and SSIM.BUIFD(M) applies the fusion technique with MemNet replacing DnCNN as the prior-predicting CNN.Source: E. Extended Benchmark Comparisons · p. 12

V. CONCLUSION

The conclusion presents fusion denoising as a theoretically derived optimal solution and reports improved generalization on synthetic data. It then describes BUIFD, which learns a data-dependent fusion function for real images and improves denoising performance on seen and unseen noise levels.

V. CONCLUSION

  • Fusion Net integrates a theoretically derived optimal fusion-denoising solution into a deep learning architecture for additive Gaussian noise removal.The approach is compared with the optimal mathematical solution and blind universal denoiser DnCNN.Source: §V · p. 12
  • On synthetic data, Fusion Net generalizes substantially better to higher unseen noise levels.The paper attributes improved generalization toward the optimum to integrating an auxiliary fusion loss into Fusion Net.Source: §V · p. 12
  • For real images, BUIFD learns a data-dependent fusion function and improves state-of-the-art denoising performance on training and unseen noise levels.Source: §V · p. 12
Loading 1907.03029v2…