PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning a Discriminative Prior for Blind Image Deblurring

Lerenhan Li, Jinshan Pan, Wei-Sheng Lai, Changxin Gao, Nong Sang, Ming-Hsuan Yang

arXiv:1803.03363v2cs.CV

TL;DR

Blind image deblurring is ill-posed and existing hand-crafted priors may not generalize across image scenarios. The paper learns a CNN binary classifier as a discriminative prior, embeds it in a MAP framework, and solves the resulting model with half-quadratic splitting and gradient descent. The method performs favorably against state-of-the-art and domain-specific approaches across natural, text, face, and low-illumination images, with extension to non-uniform deblurring.

  • Problem

    Blind deblurring is ill-posed, while hand-crafted image priors may not generalize well beyond natural images to text, face, and low-illumination scenarios.

  • Method

    The method learns a deep CNN binary classifier distinguishing blurred from clear images, embeds it as a MAP regularizer, and optimizes it using half-quadratic splitting and gradient descent.

  • Results

    The method performs favorably against state-of-the-art methods on natural-image benchmarks and domain-specific deblurring tasks, including natural, text, face, and low-illumination images.

  • Takeaways & Limitations

    A discriminative prior learned from blurred-versus-clear classification supports blind deblurring across multiple image scenarios and can be extended to non-uniform deblurring.

  • Takeaways & Limitations

    The intensity-trained classifier is less effective with significant noise and outliers; salt-and-pepper noise can prevent correct discrimination and degrade restoration.

Abstract

from arXiv · show

We present an effective blind image deblurring method based on a data-driven discriminative prior.Our work is motivated by the fact that a good image prior should favor clear images over blurred images.In this work, we formulate the image prior as a binary classifier which can be achieved by a deep convolutional neural network (CNN).The learned prior is able to distinguish whether an input image is clear or not.Embedded into the maximum a posterior (MAP) framework, it helps blind deblurring in various scenarios, including natural, face, text, and low-illumination images.However, it is difficult to optimize the deblurring method with the learned image prior as it involves a non-linear CNN.Therefore, we develop an efficient numerical approach based on the half-quadratic splitting method and gradient decent algorithm to solve the proposed model.Furthermore, the proposed model can be easily extended to non-uniform deblurring.Both qualitative and quantitative experimental results show that our method performs favorably against state-of-the-art algorithms as well as domain-specific image deblurring approaches.

1. Introduction

Blind deblurring is ill-posed because both the latent image and blur kernel are unknown, motivating a general learned prior that favors clear images over blurred ones. The proposed CNN prior is embedded in MAP deblurring and optimized with an efficient splitting-based algorithm.

  • The blur model is ill-posed because both the latent image and blur kernel are unknown, yielding infinitely many solutions without additional constraints.
  • Hand-crafted priors work well on natural images but generalize poorly to text, face, and low-illumination images.
  • The method learns a deep-CNN binary classifier that distinguishes blurred from clear images and uses it as a latent-image regularizer in the MAP framework.
  • Global average pooling and multi-scale training support varying image sizes, while half-quadratic splitting with gradient descent optimizes the nonlinear-CNN deblurring model.
  • The method performs favorably against state-of-the-art methods on natural-image benchmarks and domain-specific deblurring tasks, and extends to non-uniform deblurring.

2. Related Work

Prior deblurring methods use optimization-based hand-crafted priors or learning-based networks, but their effectiveness can be limited across image domains and large blur kernels. This work learns a CNN-based discriminative prior and embeds it in a conventional MAP framework.

  • Hand-crafted priors are effective for natural images but are not able to handle text, face, and low-illumination inputs reliably.
  • Most existing priors are designed from limited observations, motivating a data-driven prior that favors clear images over blurred images across scenarios.
  • Learning-based deblurring methods include end-to-end CNNs and networks that estimate blur kernels before conventional non-blind deconvolution.
  • The binary-classifier architecture uses global average pooling rather than fully connected layers to accommodate different input sizes.
  • Deep CNN deblurring performance still falls behind conventional optimization-based approaches for large blur kernels.
  • The proposed approach combines CNN discriminative ability with a coarse-to-fine MAP framework for blind deblurring.

3. Learning a Data-Driven Image Prior

The paper learns a CNN-based binary classifier as a data-driven prior that distinguishes blurred from clear images and embeds it in MAP deblurring.

  • 3. Learning a Data-Driven Image Prior: The learned classifier serves as a latent-image prior in the MAP model, alongside an L0 gradient regularizer.
  • 3.2. Network architecture: Global average pooling lets the network process different image sizes and reduces parameters before the sigmoid output.
  • 3.3. Loss function: The method trains a deep CNN to predict whether an input image is blurred, using blurred images as label 1 and clear images as label 0.The classifier outputs the probability that an image is blurred.
  • 3.4. Implementation details: Training uses 500 clear images, synthetic blur from 200 kernels, Gaussian noise with σ = 0.01, and 100,000 blurred images.
  • 3.4. Implementation details: Multi-scale training randomly resizes inputs between [0.25, 1] to improve robustness to different image sizes.

4. Blind Image Deblurring

The deblurring model alternates latent-image and kernel estimation, then uses half-quadratic splitting and gradient descent to optimize the CNN-regularized objective efficiently.

  • 4.1. Blind image deblurring: The model solves for the latent image and blur kernel alternately while balancing the learned prior, L0 gradient prior, and data term.
  • 4.1. Solving I: Half-quadratic splitting introduces auxiliary variables for the image and its horizontal and vertical gradients, avoiding direct minimization of non-convex terms.
  • 4.1. Solving I: With fixed auxiliary variables, latent-image estimation becomes a least-squares problem with a closed-form Fourier-domain solution.
  • 4.1. Solving I: The auxiliary image variable is updated iteratively by gradient descent, using back-propagation to compute the CNN-prior derivative.
  • 4.2. Solving k: Kernel estimation uses image gradients, FFT, nonnegativity projection, normalization, and a coarse-to-fine image pyramid.

5. Extension to Non-Uniform Deblurring

The discriminative prior extends to non-uniform motion blur by representing observations as weighted sums of transformed latent images and alternating image and motion-weight estimation.

  • 5. Extension to Non-Uniform Deblurring: Non-uniform blur is modeled as a weighted sum of latent clear images transformed by camera-pose homographies.
  • 5. Extension to Non-Uniform Deblurring: Figure 3 reports quantitative evaluations on benchmark datasets, including the natural-image and Sun et al. datasets.
  • 5. Extension to Non-Uniform Deblurring: The pose weights are nonnegative and sum to one, while each homography is applied to the latent image using bilinear interpolation.
  • 5. Extension to Non-Uniform Deblurring: The non-uniform problem is solved by alternately minimizing with respect to the latent image and pose-weight vector.

6. Experimental Results

Experiments across natural, real-world, text, low-illumination, face, and non-uniform deblurring show that the proposed method generally produces clear results with fewer artifacts and competitive quantitative performance.

  • Natural images: The method achieves the highest average PSNR on the Köhler natural-image dataset and produces clearer images with less ringing artifacts.The dataset contains 4 latent images and 12 blur kernels, with comparisons against five generic deblurring methods.
  • Natural images: On the Sun et al. dataset, the proposed method performs competitively against six optimization-based methods and one learning-based method under the error-ratio metric.The evaluation uses the same non-blind deconvolution method for restoring latent images.
  • Domain-specific images: For real blurred images, the method generates clearer results with fewer artifacts than several methods and is comparable to the dark-channel approach.The comparison includes methods and method.
  • Domain-specific images: On text images, the proposed method performs favorably against generic deblurring algorithms, although the specialized text method achieves the highest PSNR.The proposed method also generates sharper results with clearer characters, and Table 1 summarizes the quantitative comparison.
  • Domain-specific images: For low-illumination images, the method produces results comparable to the specialized method, while face-image results show less ringing artifacts than two state-of-the-art methods.The method also produces comparable non-uniform deblurring results with sharp edges and clear textures.

7. Analysis and Discussion

The analysis evaluates the CNN prior across image sizes, domains, optimization behavior, and limitations. Multi-scale training improves size robustness, the learned prior strengthens edge recovery, and the method converges within 50 iterations but remains vulnerable to salt-and-pepper noise.

  • 7. Analysis and Discussion: The method produces sharper text results, comparable low-illumination results, visually pleasing face results, and comparable non-uniform deblurring results against relevant baselines.The text comparison is against a state-of-the-art text method, while the low-illumination comparison is against Hu et al.
  • 7.1. Effectiveness of the proposed image prior: Multi-scale training makes the classifier more robust to different input sizes, improving its suitability for the coarse-to-fine MAP framework.Without it, accuracy drops significantly when test images are downscaled by more than 4×.
  • 7.1. Effectiveness of the proposed image prior: The classifier responds strongly across blurred images but mostly weakly on clear images except smooth regions, supporting its intended clear-versus-blurred discrimination.Figure 10 evaluates classification accuracy and the ablation study on separate datasets.
  • 7.2. Relation with L0-regularized priors: The learned discriminative prior restores sharper intermediate edges than L0-based alternatives, improving blur-kernel estimation when strong edges are otherwise difficult to recover.The L0 gradient prior alone cannot reconstruct strong edges well, while integrating the CNN prior reaches state-of-the-art performance in the ablation study.
  • 7.3. Runtime and convergence property: The optimization runs competitively with state-of-the-art methods and converges well within 50 iterations at the finest image scale.Runtime is reported for three image sizes, while convergence is assessed using kernel similarity and the objective function.
  • 7.4. Limitations: Salt-and-pepper noise limits the learned intensity-based prior, and median prefiltering improves deblurring but does not preserve recovered details well.The authors identify joint deblurring and denoising as future work.

8. Conclusions

The paper learns a data-driven discriminative prior with a binary CNN classifier and embeds it into a coarse-to-fine MAP framework. Combined with efficient optimization, the approach performs favorably across several image types and extends to non-uniform deblurring.

  • 8. Conclusions: The method learns a CNN-based binary classifier that favors clear images over blurred images, then uses it as a prior in a coarse-to-fine MAP framework.Global average pooling and multi-scale training improve robustness to different image sizes.
  • 8. Conclusions: The proposed half-quadratic splitting algorithm supports efficient blur-kernel estimation and extension to non-uniform deblurring.The conclusion describes the optimization as efficient and applicable across the reported scenarios.
  • 8. Conclusions: Quantitative and qualitative comparisons show favorable performance against generic and domain-specific blind deblurring algorithms on natural, text, face, and low-illumination images.The conclusion also states that the prior is effective across these image types.
Loading 1803.03363v2…