PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, Lei Zhang

arXiv:1608.03981v1cs.CV

TL;DR

Image denoising methods can be computationally costly or require separate models for fixed noise levels. This paper proposes DnCNN, which combines residual learning and batch normalization to train a single model for multiple denoising tasks, achieving strong results including a 0.6 dB average PSNR gain over BM3D on BSD68 across three noise levels.

  • Problem

    Existing denoising methods can involve costly testing optimization or separate models for fixed noise levels, limiting efficient handling of unknown and non-Gaussian noise.

  • Method

    DnCNN predicts residual noise instead of the clean image and integrates batch normalization with residual learning in a single deep CNN.

  • Results

    0.6 dB average PSNR gain over BM3D was reported for DnCNN-S across three BSD68 noise levels, while DnCNN-B also outperformed competing methods with known noise levels.

  • Takeaways & Limitations

    A single DnCNN model can address blind Gaussian denoising, multi-scale single-image super-resolution, and JPEG deblocking.

Abstract

from arXiv · show

Discriminative model learning for image denoising has been recently attracting considerable attentions due to its favorable denoising performance. In this paper, we take one step forward by investigating the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image denoising. Specifically, residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance. Different from the existing discriminative denoising models which usually train a specific model for additive white Gaussian noise (AWGN) at a certain noise level, our DnCNN model is able to handle Gaussian denoising with unknown noise level (i.e., blind Gaussian denoising). With the residual learning strategy, DnCNN implicitly removes the latent clean image in the hidden layers. This property motivates us to train a single DnCNN model to tackle with several general image denoising tasks such as Gaussian denoising, single image super-resolution and JPEG image deblocking. Our extensive experiments demonstrate that our DnCNN model can not only exhibit high effectiveness in several general image denoising tasks, but also be efficiently implemented by benefiting from GPU computing.

I. INTRODUCTION

The paper frames image denoising as recovering a clean image from noisy observations while addressing the computational and modeling limitations of prior-based methods. It proposes DnCNN, a feed-forward residual-learning CNN with batch normalization, and extends it across several denoising tasks.

  • Problem setting: Image denoising recovers a clean image x from an observation y = x + v, commonly assuming additive white Gaussian noise with standard deviation σ.The paper notes that image-prior modeling is central when the likelihood is known.
  • Limitations of prior-based methods: Prior-based methods often require complex test-time optimization and manually chosen parameters, limiting computational efficiency and potentially compromising performance.These models are generally non-convex, adding further flexibility in parameter selection.
  • Motivation: The paper treats denoising as plain discriminative learning that separates noise from noisy images using feed-forward CNNs, motivated by deep architectures, improved training methods, and GPU parallelism.CNNs provide greater capacity and flexibility for exploiting image characteristics while supporting efficient computation.
  • Proposed method: DnCNN predicts the residual noise v rather than directly estimating the clean image, while batch normalization stabilizes and enhances training.Residual learning implicitly removes the latent clean image through hidden-layer operations.
  • Contributions and results: DnCNN achieves better Gaussian denoising than BM3D, WNNM, and TNRD at certain noise levels, and a single model can handle blind Gaussian denoising, SISR, and JPEG deblocking.The paper reports that blind Gaussian denoising can outperform methods trained for a specific noise level and that the model extends to three general denoising tasks.

II. RELATED WORK · A. Deep Neural Networks for Image Denoising · B. Residual Learning and Batch Normalization

Related work established deep neural networks as viable image-denoising models, while advances in CNN training—including batch normalization and residual learning—provided techniques for improving model construction and optimization.

  • A. Deep Neural Networks for Image Denoising: CNNs were proposed for image denoising and reported to have representation power similar to or better than MRF models.
  • A. Deep Neural Networks for Image Denoising: Multi-layer perceptrons were successfully applied to image denoising.
  • A. Deep Neural Networks for Image Denoising: Stacked sparse denoising auto-encoders were used for Gaussian noise removal and achieved comparable results to an unspecified baseline.
  • B. Residual Learning and Batch Normalization: Large-scale datasets and advances in deep learning have contributed to CNN success across various vision tasks.
  • B. Residual Learning and Batch Normalization: Representative CNN-training advances include ReLU, depth–width tradeoffs, parameter initialization, and gradient-based optimization algorithms.
  • B. Residual Learning and Batch Normalization: Batch normalization and residual learning were identified as representative advances in training CNN models.

1) Residual Learning: … A. Network Depth

DnCNN combines residual learning and batch normalization to enable deep, stable, high-performing image-denoising CNNs, while its architecture uses VGG-style 3 × 3 convolutions without pooling and selects depth through receptive-field considerations. The model adopts a single residual unit and sets depth 17 for a 35 × 35 receptive field in Gaussian denoising at a fixed noise level.

  • 1) Residual Learning:: Residual learning addresses performance degradation with increasing CNN depth by learning residual mappings that are easier to optimize than original mappings.This strategy enables extremely deep CNNs to be trained while improving accuracy.
  • 1) Residual Learning:: DnCNN predicts the residual image with a single residual unit, unlike residual networks that use many residual units with identity shortcuts.The formulation is connected to TNRD and extended to several general image-denoising tasks.
  • 2) Batch Normalization:: Batch normalization alleviates internal covariate shift by normalizing and then scaling and shifting each layer’s inputs before nonlinearity.Mini-batch SGD training efficiency is reduced when internal nonlinearity-input distributions change during training.
  • 2) Batch Normalization:: Integrating residual learning with batch normalization produces fast, stable training and better denoising performance.The paper presents this integration as a previously unstudied combination for CNN-based image denoising.
  • III. THE PROPOSED DENOISING CNN MODEL: DnCNN design separates task-specific CNN development into network architecture design and model learning from training data.The architecture modifies VGG to suit image denoising and sets depth according to effective patch sizes.
  • A. Network Depth: DnCNN uses 3 × 3 convolutional filters and no pooling layers, giving a depth-d receptive field of (2d+1)×(2d+1).Larger receptive fields capture context from larger image regions, but depth must balance performance and efficiency.
  • A. Network Depth: At noise level σ = 25, receptive-field size is used to guide DnCNN depth because denoising methods’ effective patch sizes correlate with receptive fields and increase with noise level.The analysis compares effective patch sizes of leading denoising methods, including BM3D.
  • A. Network Depth: 35 × 35 is the selected DnCNN receptive-field size for Gaussian denoising at a fixed noise level, corresponding to depth 17 and approximating EPLL’s 36×36 effective patch size.The paper tests whether this EPLL-sized receptive field can compete with leading denoising methods.

B. Network Architecture

DnCNN learns a residual mapping from noisy observations to noise rather than directly predicting the clean image, then reconstructs the image by subtraction. Its architecture uses depth-dependent convolutional layers with ReLU and batch normalization, beginning with 3 × 3 filters tailored to the number of image channels.

  • Residual learning: DnCNN trains a residual mapping R(y) ≈ v from noisy input y = x + v and estimates the clean image as x = y − R(y).This replaces direct clean-image prediction with residual learning.
  • Residual learning: The trainable parameters are learned using averaged mean squared error between desired and estimated residual images over noisy-clean training pairs.The training set contains N noisy-clean image or patch pairs.
  • Layer architecture: For depth D, DnCNN contains three layer types, including an initial Conv+ReLU layer and Conv+BN+ReLU layers from layers 2 through D − 1.The supplied architecture description identifies the layer types by color in Fig. 1.
  • Layer architecture: The first layer uses 64 filters of size 3 × 3 × c, where c = 1 for grayscale images and c = 3 for color images.The filters generate 64 feature maps followed by ReLU nonlinearity.
  • Layer architecture: Layers 2 through D − 1 use 64 filters of size 3 × 3 × 64 together with batch normalization and ReLU.This specifies the repeated intermediate-layer configuration described in the passage.

1) Deep Architecture: · 2) Reducing Boundary Artifacts: · C. Integration of Residual Learning and Batch Normalization for Image Denoising

DnCNN combines residual learning, batch normalization, convolution, and ReLU to separate image structure from noise while improving training and denoising. It also uses zero padding to preserve feature-map dimensions and reduce boundary artifacts.

  • 1) Deep Architecture:: DnCNN adopts residual learning to predict the noise residual and uses batch normalization to accelerate training and improve denoising performance.Convolution with ReLU gradually separates image structure from the noisy observation through hidden layers.
  • 2) Reducing Boundary Artifacts:: Zero padding before convolution preserves the spatial size of middle-layer feature maps and is used to address boundary artifacts.This differs from symmetric padding at preprocessing or repeated padding before every stage in other methods.
  • C. Integration of Residual Learning and Batch Normalization for Image Denoising: Residual learning is more suitable for denoising because the noisy observation is closer to the clean image than to the residual image.The residual mapping is easier to optimize when the original mapping resembles an identity mapping.
  • C. Integration of Residual Learning and Batch Normalization for Image Denoising: Residual learning converges faster and more stably than original mapping under the same optimization and architecture settings.The comparison uses SGD with momentum and Adam, with average PSNR evaluated across training epochs.
  • C. Integration of Residual Learning and Batch Normalization for Image Denoising: Residual learning and batch normalization are both associated with Gaussian distributions, suggesting mutual benefits for Gaussian denoising.The paper presents this as a likelihood that is subsequently analyzed.
  • C. Integration of Residual Learning and Batch Normalization for Image Denoising: Residual learning benefits from batch normalization, as residual learning without batch normalization converges quickly but performs worse than the combination.Batch normalization offers CNN benefits such as alleviating internal covariate shift.
  • C. Integration of Residual Learning and Batch Normalization for Image Denoising: Batch normalization benefits from residual learning, whereas batch normalization without residual learning can adversely affect convergence.Together, they speed training and improve performance; the experiment trains four combinations at noise level 25 on 68 Berkeley images.
  • C. Integration of Residual Learning and Batch Normalization for Image Denoising: Their integration speeds up and stabilizes training while boosting denoising performance.The stated observation concerns the combined use of residual learning and batch normalization.

D. Connection with TNRD

DnCNN generalizes one-stage TNRD by recasting its diffusion step as a feed-forward CNN and extending the architecture with ReLU, greater depth, and batch normalization. This residual formulation supports a single model across multiple denoising tasks, including unknown-noise Gaussian denoising, SISR, and JPEG deblocking.

  • Connection with TNRD: DnCNN generalizes one-stage TNRD by interpreting its diffusion iteration as a two-layer feed-forward CNN that estimates the residual between degraded and clean images.The first diffusion stage is a gradient-descent inference step, and the resulting CNN predicts residual image v1.
  • Connection with TNRD: The proposed architecture replaces TNRD influence functions with ReLU, increases CNN depth, and incorporates batch normalization to improve training and modeling capacity.These changes are presented as three architectural generalizations of one-stage TNRD.
  • Connection with TNRD: A single CNN model can address Gaussian denoising with unknown noise levels, SISR with multiple upscaling factors, and JPEG deblocking with different quality factors.The formulation is stated to hold for multiple noise distributions, including generalized Gaussian noise, and is assumed applicable to SISR and JPEG compression artifacts.
  • Connection with TNRD: DnCNN estimates residual images by progressively removing the latent clean image from degraded observations in hidden layers, enabling robustness to complex noise distributions.The paper interprets the CNN operations as removing latent clean image x from degraded observation y to estimate residual v.

E. Extension to General Image Denoising

DnCNN is extended from fixed-level Gaussian denoising to blind Gaussian denoising using one model trained across noise levels, avoiding explicit noise-level estimation. A single model is also learned for blind Gaussian denoising, SISR, and JPEG deblocking.

  • Motivation: Existing discriminative Gaussian denoisers train separate models for fixed noise levels, while unknown-noise denoising commonly depends on noise-level estimation.Their results are therefore affected by estimation accuracy, and they cannot handle non-Gaussian noise distributions.
  • Blind Gaussian denoising: A single DnCNN trained on noise levels σ ∈[0, 55] denoises test images within that range without estimating their noise level.Training uses noisy images spanning a wide range of Gaussian noise levels.
  • Multiple denoising tasks: One DnCNN is trained for blind Gaussian denoising, SISR, and JPEG deblocking using varied noise levels, upscaling factors, and JPEG quality factors.Experimental results show that this single model yields excellent results across the considered tasks.

IV. EXPERIMENTAL RESULTS · A. Experimental setting · 1) Training and Testing Data:

The experiments train DnCNN variants on cropped patches from Berkeley-related image datasets across known-noise, blind Gaussian, color denoising, and three-task settings. Evaluation uses separate natural-image test sets and task-specific synthetic degradations.

  • 1) Training and Testing Data:: 400 images of size 180 × 180 are used to train Gaussian denoising models with known or unknown noise levels.Larger training datasets yielded only negligible improvements.
  • 1) Training and Testing Data:: DnCNN-S is trained at σ = 15, 25 and 50 using 128 × 1, 600 patches of size 40 × 40.The model targets Gaussian denoising with a known specific noise level.
  • 1) Training and Testing Data:: DnCNN-B is trained for blind Gaussian denoising with σ ∈[0, 55] using 128 × 3, 000 patches of size 50 × 50.It is a single model covering the specified noise-level range.
  • 1) Training and Testing Data:: Testing uses BSD68 with 68 natural images and a separate dataset of 12 images, none included in training.Both datasets are widely used for evaluating Gaussian denoising methods.
  • 1) Training and Testing Data:: CDnCNN-B is trained on 432 Berkeley color images and tested on color BSD68, using σ ∈[0, 55] and 128 × 3, 000 patches of size 50×50.This extends blind denoising to color images.
  • 1) Training and Testing Data:: The three-task model uses 91 images from and 200 Berkeley training images, with Gaussian noise levels from [0, 55].Its inputs also include bicubic downsampling followed by upsampling for SISR at factors 2, 3 and 4.

2) Parameter Setting and Network Training: · B. Compared Methods

The study specifies distinct DnCNN depths and a residual-learning training setup, then compares DnCNN against established non-local, generative, and discriminative denoising methods. Training uses MatConvNet with SGD on GPU, with model-specific runtimes reported.

  • 2) Parameter Setting and Network Training:: DnCNN-S uses depth 17, whereas DnCNN-B and DnCNN-3 use depth 20 to capture sufficient spatial information for denoising.The models learn the residual mapping R(y) to predict the residual v.
  • 2) Parameter Setting and Network Training:: Training uses the Eqn. (1) residual-mapping loss, weights initialized by, SGD, weight decay 0.0001, momentum 0.9, and mini-batches of 128.The loss learns R(y) for predicting the residual v.
  • 2) Parameter Setting and Network Training:: DnCNN models are trained for 50 epochs while the learning rate decays exponentially from 1e−1 to 1e−4.This schedule applies to the 50-epoch training process.
  • 2) Parameter Setting and Network Training:: The proposed models are trained with MatConvNet in Matlab (R2015b) on a PC with an Intel(R) Core(TM) i7-5820K CPU 3.30GHz and an Nvidia Titan X GPU.The stated environment is used unless otherwise specified.
  • 2) Parameter Setting and Network Training:: GPU training takes about 6 hours for DnCNN-S, one day for DnCNN-B/CDnCNN-B, and three days for DnCNN-3.These are reported training times on the Nvidia Titan X GPU.
  • B. Compared Methods: DnCNN is compared with BM3D and WNNM, two non-local similarity methods; EPLL, a generative method; and MLP, CSF, and TNRD, discriminative training methods.The implementation codes were downloaded from the authors’ websites with default parameter settings.

C. Quantitative and Qualitative Evaluation

DnCNN-S and DnCNN-B achieve leading PSNR results on BSD68, while DnCNN-B/CDnCNN-B consistently outperform BM3D/CBM3D across noise levels. Qualitatively, the models preserve sharp details, avoid artifacts, and improve color naturalness.

  • Quantitative evaluation: DnCNN-S outperforms BM3D by 0.6dB on all three noise levels on BSD68.The paper notes that few methods outperform BM3D by more than 0.3dB on average.
  • Quantitative evaluation: DnCNN-S yields the highest PSNR on most of 12 test images, outperforming competing methods by 0.2dB to 0.6dB on most images.It fails to achieve the best result on only “House” and “Barbara”.
  • Qualitative evaluation: DnCNN-S and DnCNN-B preserve sharp edges and fine details while producing visually pleasant results in smooth regions.BM3D, WNNM, EPLL, and MLP tend to oversmooth edges and textures, whereas TNRD can generate smooth-region artifacts.
  • Qualitative evaluation: CDnCNN-B recovers more natural colors, details, and sharper edges than CBM3D, which generates false color artifacts in some regions.The comparison concerns color image denoising.
  • Quantitative evaluation: DnCNN-B/CDnCNN-B consistently outperform BM3D/CBM3D by a large margin across a wide range of noise levels.This demonstrates the feasibility of training a single DnCNN-B model for blind Gaussian denoising within that range.

D. Run Time · E. Experiments on Learning a Single Model for Three General Image Denoising Tasks

DnCNN combines GPU-efficient testing with a single DnCNN-3 model that addresses blind Gaussian denoising, SISR, and JPEG deblocking. Across these tasks, DnCNN-3 outperforms or matches established methods while producing sharp, visually pleasant outputs.

  • D. Run Time: DnCNN denoises a 512 × 512 image in 60ms on GPU with unknown noise level, offering a computational-efficiency advantage over TNRD.The GPU implementation uses Nvidia cuDNNv5 to accelerate computation.
  • E. Experiments on Learning a Single Model for Three General Image Denoising Tasks: A single DnCNN-3 model is trained for blind Gaussian denoising, SISR, and JPEG image deblocking.The paper states that existing methods had not been reported to handle all three tasks with one model.
  • E. Experiments on Learning a Single Model for Three General Image Denoising Tasks: For Gaussian denoising, DnCNN-3 is compared with BM3D and TNRD on BSD68, assuming known noise levels for BM3D and TNRD.The comparison uses state-of-the-art methods, while DnCNN-3 addresses blind Gaussian denoising.
  • E. Experiments on Learning a Single Model for Three General Image Denoising Tasks: For SISR, DnCNN-3 is evaluated against TNRD and VDSR on Set5, Set14, BSD100, and Urban100 across upscaling factors 2, 3, and 4.TNRD uses a specific model per factor, whereas VDSR uses one model for all three factors.
  • E. Experiments on Learning a Single Model for Three General Image Denoising Tasks: For JPEG image deblocking, DnCNN-3 is compared with AR-CNN and TNRD on Classic5 and LIVE1 across quality factors 10, 20, 30, and 40.AR-CNN and TNRD use multiple quality-factor-specific models.
  • E. Experiments on Learning a Single Model for Three General Image Denoising Tasks: DnCNN-3 outperforms BM3D and TNRD for Gaussian denoising, surpasses TNRD for SISR, and is on par with VDSR.For JPEG deblocking, it exceeds AR-CNN by about 0.3dB in PSNR and TNRD by about 0.1dB across quality factors.
  • E. Experiments on Learning a Single Model for Three General Image Denoising Tasks: DnCNN-3 and VDSR produce sharp edges and fine details in SISR, while DnCNN-3 recovers straighter lines than AR-CNN and TNRD in JPEG deblocking.An additional example shows visually pleasant output despite distortions with different levels in different regions.

V. CONCLUSION

The paper proposes DnCNN for image denoising, combining residual learning and batch normalization to separate noise from noisy observations, accelerate training, and improve performance. A single model can perform blind Gaussian denoising with unknown noise levels.

  • V. CONCLUSION: DnCNN integrates residual learning and batch normalization to separate noise from noisy observations, accelerate training, and boost denoising performance.Residual learning enables noise-focused processing, while batch normalization and residual learning jointly speed training and improve denoising.
  • V. CONCLUSION: A single DnCNN model handles blind Gaussian denoising with unknown noise levels, unlike models trained for specific noise levels.The model avoids requiring separate discriminative models for each known noise level.
Loading 1608.03981v1…