PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Deep CNN Denoiser Prior for Image Restoration

Kai Zhang, Wangmeng Zuo, Shuhang Gu, Lei Zhang

arXiv:1704.03264v1cs.CV

TL;DR

Integrating fast discriminative denoisers into flexible model-based restoration remained underexplored. The paper trains CNN denoisers and integrates them into model-based optimization, reporting a flexible, fast, effective framework across restoration tasks.

  • Problem

    How to integrate fast discriminative denoisers with flexible model-based restoration remained underexplored, despite the approaches' complementary strengths.

    Source: §1 · p. 2

  • Method

    The paper trains a set of CNN denoisers and incorporates them as priors into model-based optimization using variable splitting.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    Experiments report a flexible, fast, effective framework across image restoration tasks, including deblurring and super-resolution.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The learned denoisers can serve as image priors for model-based restoration of inverse problems beyond denoising.

    Source: §1 · p. 2

    Further study is needed to reduce the number of denoisers and iterations required by the framework.

    Source: §5 · p. 8

Abstract

from arXiv · show

Model-based optimization methods and discriminative learning methods have been the two dominant strategies for solving various inverse problems in low-level vision. Typically, those two kinds of methods have their respective merits and drawbacks, e.g., model-based optimization methods are flexible for handling different inverse problems but are usually time-consuming with sophisticated priors for the purpose of good performance; in the meanwhile, discriminative learning methods have fast testing speed but their application range is greatly restricted by the specialized task. Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e.g., deblurring). Such an integration induces considerable advantage when the denoiser is obtained via discriminative learning. However, the study of integration with fast discriminative denoiser prior is still lacking. To this end, this paper aims to train a set of fast and effective CNN (convolutional neural network) denoisers and integrate them into model-based optimization method to solve other inverse problems. Experimental results demonstrate that the learned set of denoisers not only achieve promising Gaussian denoising results but also can be used as prior to deliver good performance for various low-level vision applications.

1. Introduction

The paper trains fast CNN denoisers and plugs them into model-based optimization, aiming to combine discriminative learning’s speed with model-based methods’ flexibility across inverse problems. It evaluates the denoisers for Gaussian denoising and their use in deblurring and super-resolution.

1. Introduction

  • Model-based optimization can address varied restoration tasks but is often time-consuming, whereas discriminative learning is fast and effective but typically specialized to particular tasks or noise levels.Image restoration covers denoising, deblurring, and super-resolution, represented by different degradation operators.Source: §1 · pp. 1–2
  • Variable splitting with ADMM or HQS separates fidelity and regularization, making the regularization step a denoising subproblem that can use discriminatively learned denoisers.The paper identifies integration with discriminative denoisers as a gap in prior study.Source: §1 · p. 2
  • The work trains a set of fast, effective CNN denoisers as image priors for model-based optimization.The networks use ReLU, batch normalization, Adam, and dilated convolution; variable splitting enables the denoisers to contribute a strong image prior.Source: §1 · p. 2
  • The learned denoisers are used as modular components for other inverse problems, with experiments demonstrating the integration for deblurring and super-resolution.The denoisers also provide performance for image denoising.Source: §1 · p. 2

2. Background

Prior work uses denoiser priors within iterative optimization by separating fidelity and regularization, and HQS illustrates how a denoiser can serve as a modular prior across restoration problems.

2.1. Image Restoration with Denoiser Prior

  • Prior methods used denoiser priors with Nash equilibrium, augmented Lagrangian, ADMM, HQS, or primal-dual optimization for deblurring, super-resolution, denoising, and inpainting.Examples include IDDBM3D [19], a CBM3D-based SISR method, and plug-and-play approaches; the SISR method improved PSNR over SRCNN.Source: §2.1 · p. 2
  • These approaches decouple fidelity and regularization, typically alternating between a fidelity-related subproblem and a denoising subproblem.HQS is used as an example because of its simplicity, but other suitable optimization methods can incorporate denoiser priors.Source: §2.1 · p. 2

2.2. Half Quadratic Splitting (HQS) Method

  • HQS introduces an auxiliary variable and iteratively minimizes a cost with a non-descending penalty parameter, separating fidelity and regularization into distinct subproblems.The fidelity subproblem is a quadratic regularized least-squares problem with fast solutions for different degradation matrices.Source: §2.2 · p. 3
  • The HQS regularization subproblem corresponds to Gaussian denoising at noise level λ/µ, allowing Gaussian denoisers to act as modular priors.This lets restoration methods use gray or color denoisers without an explicit image prior, combine complementary denoisers, and extend the approach to methods such as ISTA and FISTA.Source: §2.2 · p. 3

3. Learning Deep CNN Denoiser Prior

The section motivates an efficient, strong denoiser prior and presents a deep CNN denoiser designed for restoration. Its design uses dilated convolutions, batch normalization, residual learning, small training patches, and a bank of noise-level-specific models.

3.1. Why Choose CNN Denoiser?

  • CNN denoisers are chosen for GPU-efficient inference, deep prior capacity, complementary external priors, and learned joint RGB modeling instead of a hand-designed color pipeline.The section notes that modern images are commonly RGB and that jointly handling correlated color channels can outperform treating them independently.Source: §3.1 · p. 3

3.2. The Proposed CNN Denoiser

  • The proposed seven-layer denoiser uses 3×3 dilated convolutions with dilation factors 1, 2, 3, 4, 3, 2, 1, and 64 feature maps in each middle layer.Its first block is Dilated Convolution+ReLU, its five middle blocks add Batch Normalization, and its final block is Dilated Convolution.Source: §3.2 · p. 4, Figure 1 · p. 4
  • At noise level 25, the dilated model reached 29.15dB average PSNR on BSD68, versus 28.94dB for a seven-layer traditional model and near 29.20dB for a 16-layer model.The dilated design obtains a 33×33 receptive field with seven layers; the alternatives have either a 15×15 receptive field at seven layers or require 16 layers for a 33×33 field.Source: §3.2 · p. 4
  • Batch normalization and residual learning enable fast, stable training, can improve denoising performance, and facilitate transfer between models for different noise levels.The paper adopts their combination and reports empirically that it enables fast transfer across noise levels.Source: §3.2 · p. 4
  • Zero padding lets the CNN model image boundaries, while cropping large patches into smaller non-overlapping patches augments boundary information; larger patches did not improve performance, but patches smaller than the receptive field reduced it.The fourth layer's dilation factor of 4 pads four zeros at feature-map boundaries.Source: §3.2 · pp. 4–5
  • The authors trained 25 denoisers per gray and color prior, at noise levels from 0 to 50 in steps of 2, which they report is sufficient for varied restoration problems.The iterative framework can use inexact but fast subproblem solutions, so the bank is much smaller than training separate models for each degradation.Source: §3.2 · p. 5

4. Experiments

Experiments show that the learned CNN denoisers perform strongly on Gaussian denoising and can serve as priors for deblurring and super-resolution. The evaluations also examine their speed and performance across different degradation settings.

4.1. Image Denoising

  • Training uses 400 BSD images, 400 selected ImageNet validation images, and 4,744 Waterloo Exploration Database images; the larger dataset slightly improves results on some test images but not BSD68 PSNR.Images are cropped into 35×35 patches, with 256×4,000 patches selected for training.Source: §4.1 · p. 5
  • On gray BSD68, the CNN denoiser gains about 0.2 dB PSNR over WNNM, MLP, and TNRD; on color BSD68, it consistently outperforms CBM3D by a large margin.The passage reports that WNNM, MLP, and TNRD each exceed BM3D by about 0.3 dB; it attributes the color result to CNN capacity for modeling color-image priors.Source: Table 1 · p. 5, Table 2 · p. 5, §4.1 · p. 5
  • For noise level 25, the 24-feature-map denoiser achieves 28.94 dB PSNR, comparable to TNRD, at faster speed; the proposed denoiser is competitive in CPU and GPU implementations.The authors attribute its speed-performance compromise to simple 3×3 convolutions and ReLU, fluent information flow across layers, and batch normalization.Source: Table 3 · p. 6, §4.1 · p. 6

4.2. Image Deblurring

  • Deblurring evaluations use circular boundaries and FFT-based implementation across a Gaussian kernel of standard deviation 1.6 and two real blur kernels, with varied Gaussian noise levels.The method plugs color denoisers into HQS; its denoiser noise level is decayed over 30 iterations.Source: §4.2 · p. 6
  • The denoiser-prior method produces promising deblurring PSNR results and recovers sharpness and naturalness, whereas IDDBM3D, NCSR, and MLP tend to smooth edges and create color artifacts.The illustrated Leaves comparison uses a Gaussian blur kernel with standard deviation 1.6 and noise level σ=2.Source: Table 4 · p. 6, §4.2 · p. 6, Figure 3 · p. 7

4.3. Single Image Super-Resolution

  • SISR tests cover bicubic downsampling at scales 2 and 3 and 7×7 Gaussian blur with standard deviation 1.6 at scale 3; the method supports different blur kernels without retraining.The procedure uses 30 main iterations, five back-projection updates before each denoising step, and denoiser noise levels decaying from 12×sf to sf.Source: §4.3 · p. 7
  • On the Butterfly image, the method takes 0.5 seconds on GPU and 12 seconds on CPU, compared with 198 seconds on CPU for NCSR.These are example testing times reported alongside the Set5 and Set14 super-resolution evaluation.Source: §4.3 · p. 7
  • On Set5 and Set14, ProposedG and ProposedC handle bicubic and Gaussian blur well and outperform SRBM3DG and SRBM3DC, while bicubic-trained SRCNN and VDSR deteriorate on Gaussian blur.The comparison includes both gray- and color-channel PSNR results; SRCNN and VDSR are included on Gaussian blur to illustrate their limitations despite being trained for bicubic blur.Source: §4.3 · p. 7, Table 5 · p. 8

5. Conclusion

The paper trains fast CNN denoisers and plugs them into HQS via variable splitting, combining discriminative denoising with model-based optimization for image restoration. It also identifies open directions for reducing model cost, expanding task coverage, combining priors, and informing CNN design.

5. Conclusion

  • The learned CNN denoiser prior, plugged into HQS using variable splitting, supports image deblurring and super-resolution within a flexible, fast, and effective restoration framework.The paper presents expressive CNN denoiser priors as an alternative to modeling image priors with sophisticated conventional methods.Source: §5 · p. 8
  • The framework is proposed as a flexible alternative to task-specialized discriminative methods and time-consuming model-based methods, while producing favorable results across restoration tasks.The conclusion highlights the potential benefits of integrating flexible model-based optimization with fast discriminative learning.Source: §5 · p. 8
  • Future work includes reducing the number of CNN denoisers and iterations, extending HQS to inpainting and blind deblurring, and combining complementary priors.The authors also suggest using the HQS-as-MAP perspective to inform task-specific CNN design, while cautioning that the best architecture need not be MAP-inspired.Source: §5 · p. 8

Related papers

Loading 1704.03264v1…