PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Deep CNN Denoiser Prior for Image Restoration

Kai Zhang, Wangmeng Zuo, Shuhang Gu, Lei Zhang

arXiv:1704.03264v1cs.CV

TL;DR

Model-based optimization is flexible but slow, whereas discriminative methods are fast but task-specific, leaving integration with fast discriminative denoiser priors underexplored. The paper trains CNN denoisers and plugs them into model-based optimization through variable splitting. The resulting framework achieves promising denoising performance and supports deblurring, super-resolution, and other restoration applications.

  • Problem

    Fast discriminative denoiser priors had not been sufficiently integrated with flexible model-based optimization methods for inverse problems.

  • Method

    The paper trains a set of CNN denoisers and integrates them as modular priors into model-based optimization using variable splitting.

  • Results

    The learned CNN denoisers achieve promising Gaussian denoising results and support good performance across image restoration applications.

  • Takeaways & Limitations

    Combining CNN denoisers with model-based optimization yields a flexible, fast, and effective framework for various image restoration tasks.

  • Takeaways & Limitations

    Further work is needed to reduce the number of CNN denoisers and whole iterations and to extend the framework to additional inverse problems.

Abstract

from arXiv · show

Model-based optimization methods and discriminative learning methods have been the two dominant strategies for solving various inverse problems in low-level vision. Typically, those two kinds of methods have their respective merits and drawbacks, e.g., model-based optimization methods are flexible for handling different inverse problems but are usually time-consuming with sophisticated priors for the purpose of good performance; in the meanwhile, discriminative learning methods have fast testing speed but their application range is greatly restricted by the specialized task. Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e.g., deblurring). Such an integration induces considerable advantage when the denoiser is obtained via discriminative learning. However, the study of integration with fast discriminative denoiser prior is still lacking. To this end, this paper aims to train a set of fast and effective CNN (convolutional neural network) denoisers and integrate them into model-based optimization method to solve other inverse problems. Experimental results demonstrate that the learned set of denoisers not only achieve promising Gaussian denoising results but also can be used as prior to deliver good performance for various low-level vision applications.

1. Introduction

Image restoration recovers a clean image from degraded observations, but existing approaches trade flexibility against speed. This paper combines CNN denoisers with model-based optimization to retain advantages of both paradigms.

  • Problem setting: Image restoration recovers latent clean images from degraded observations, with different degradation matrices defining denoising, deblurring, and super-resolution tasks.The observation model is y = Hx + v, where H specifies the degradation and v is additive white Gaussian noise.
  • Existing approaches: Model-based optimization methods flexibly handle different inverse problems but typically require time-consuming iterative inference and sophisticated priors.Their flexibility comes from specifying the degradation matrix H.
  • Existing approaches: Discriminative learning methods provide fast testing and promising performance but are usually restricted to specialized degradation tasks.They learn from degraded-clean image pairs and use compact inference functions or learned prior parameters.
  • Proposed direction: Variable splitting techniques decouple fidelity and regularization terms, allowing denoiser priors to serve as modular components within model-based optimization.This integration permits the use of discriminatively learned denoisers while preserving an optimization-based framework.
  • Contributions: The paper trains fast CNN denoisers and plugs them into model-based optimization methods to address denoising and other inverse problems.The network uses CNN techniques including ReLU, batch normalization, Adam, and dilated convolution.

2. Background

Denoiser priors can be inserted into iterative optimization by separating data fidelity from regularization. HQS provides a simple framework in which Gaussian denoisers replace an explicit image prior and can support multiple inverse problems.

  • Denoiser-prior optimization: Prior work incorporated denoiser priors into iterative schemes for deblurring and super-resolution by alternating fidelity-related and denoising subproblems.The common mechanism is decoupling the fidelity term from the regularization term.
  • Half Quadratic Splitting: HQS introduces an auxiliary variable to reformulate the restoration objective, then minimizes a penalized cost function with an iteratively varying parameter µ.The resulting optimization separates the fidelity and regularization terms.
  • Half Quadratic Splitting: The fidelity subproblem is a quadratic regularized least-squares problem with fast solutions tailored to different degradation matrices.This subproblem enforces consistency with the observation model while remaining compatible with varied inverse problems.
  • Half Quadratic Splitting: The regularization subproblem is equivalent to Gaussian denoising of the updated image at noise level λ/µ.This equivalence enables Gaussian denoisers to function as modular priors in the optimization procedure.
  • Denoiser prior: Replacing the explicit prior with a denoiser prior permits gray or color denoisers, unknown explicit priors, and complementary denoisers to be used across inverse problems.The text notes that the same property can also be used with other optimization methods.

3. Learning Deep CNN Denoiser Prior

The paper develops a fast CNN denoiser prior designed for integration into model-based optimization, addressing limitations of existing denoiser priors and color-image modeling. Its architecture uses dilated convolutions, batch normalization, ReLU, residual learning, and carefully chosen training patches and noise levels.

  • Motivation: Existing denoiser priors, including TV, GMM, K-SVD, non-local means, and BM3D, have respective drawbacks for restoration.The choice of denoiser prior is important because it serves as the regularization term in model-based restoration.
  • Motivation: Jointly modeling correlated RGB channels is favored because modern camera and internet images are predominantly color images.The paper notes that existing methods mainly focus on gray-image priors, with relatively few color-image approaches.
  • Why Choose CNN Denoiser?: CNNs are selected for efficient GPU inference, powerful deep prior modeling, complementary external priors, and advances in CNN training and design.The paper specifically highlights complementarity with internal priors such as BM3D.
  • The Proposed CNN Denoiser: The seven-layer denoiser uses dilated-convolution blocks with ReLU and batch normalization, employing dilation factors 1, 2, 3, 4, 3, 2, and 1.The middle layers contain 64 feature maps, while the first and last blocks omit batch normalization in the stated configurations.
  • The Proposed CNN Denoiser: 29.15dB average PSNR on BSD68 at noise level 25 exceeds the 28.94dB seven-layer traditional-convolution model and approaches the 29.20dB sixteen-layer model.The comparison uses three models trained with the same settings.
  • The Proposed CNN Denoiser: Training patches smaller than the network receptive field reduce performance, whereas larger patches did not empirically improve it.Cropping 70×70 patches into four non-overlapping 35×35 patches largely augments boundary information.
  • The Proposed CNN Denoiser: The iterative framework uses a set of denoisers trained across noise levels [0, 50], balancing model count against matching the current optimization noise level.The paper motivates inexact but fast subproblem solutions and notes that the denoiser in the optimization framework need not target traditional Gaussian denoising directly.

4. Experiments

Experiments evaluate the proposed CNN denoiser on Gaussian denoising, image deblurring, and single-image super-resolution. Results show strong denoising quality and competitive restoration performance, including robustness to blur kernels beyond those used by specialized discriminative models.

  • 4.1. Image Denoising: The proposed CNN denoiser improves gray-image denoising over BM3D, WNNM, MLP, and TNRD on BSD68, while also outperforming CBM3D for color images.The reported gray-image gain is about 0.2dB over WNNM, MLP, and TNRD; color results consistently exceed CBM3D by a large margin.
  • 4.1. Image Denoising: The denoiser is competitive in CPU and GPU runtime, and its 24-feature-map variant reaches 28.94dB PSNR while running faster than TNRD.Runtime comparisons use 256×256, 512×512, and 1024×1024 images at noise level 25.
  • 4.2. Image Deblurring: Deblurring experiments use Gaussian and real blur kernels, additive Gaussian noise, circular boundaries, and FFT-based implementation across six testing images.The six images are Cameraman, House, Lena, Monarch, Leaves, and Parrots; one setting uses a Gaussian kernel with standard deviation 1.6 and noise level σ=2.
  • 4.2. Image Deblurring: The CNN denoiser prior achieves very promising deblurring PSNR results and preserves sharpness and naturalness better than methods that smooth edges or create color artifacts.The comparison includes IDDBM3D, NCSR, MLP, and the proposed CNN-prior optimization method.
  • 4.3. Single Image Super-Resolution: For single-image super-resolution, ProposedG and ProposedC handle all tested blur cases well, whereas SRCNN and VDSR deteriorate seriously when the kernel is not bicubic.The evaluation reports average PSNR(dB) on Set5 and Set14 and compares against CNN, statistical, model-based, and denoiser-prior methods.

5. Conclusion

The paper integrates fast CNN denoisers into a model-based HQS optimization framework, yielding a flexible and effective approach for image denoising, deblurring, and super-resolution. It also identifies several directions for extending and improving the framework.

  • The authors trained fast CNN denoisers and plugged them into HQS to solve image deblurring and super-resolution.The integration uses variable splitting to incorporate the learned denoiser prior into model-based optimization.
  • The resulting framework is described as flexible, fast, and effective across various image restoration tasks.It combines the task flexibility of model-based optimization with the efficient CNN denoising prior.
  • The framework addresses limitations of conventional model-based and discriminative methods by combining task flexibility with fast CNN inference.Model-based methods are described as time-consuming, whereas discriminative methods are specialized for certain restoration tasks.
  • Future work includes reducing the number of CNN denoisers and iterations and extending the framework to inpainting and blind deblurring.The authors also suggest combining complementary priors and exploring CNN architectures for task-specific discriminative learning.
  • The paper notes that the best CNN architecture is not necessarily inspired by MAP inference despite HQS being interpretable as MAP inference.This qualifies the implications of using MAP-based insight for CNN design.
Loading 1704.03264v1…