Source-linked AI summary
Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution
Jiawei Zhang, Jinshan Pan, Wei-Sheng Lai, Rynson Lau, Ming-Hsuan Yang
TL;DR
Non-blind deconvolution is ill-posed, while existing priors and deep methods can be computationally expensive or require retraining for each blur kernel. The paper addresses this with an iterative FCNN that denoises gradients and uses them to guide deconvolution, achieving favorable quality and efficiency against state-of-the-art methods.
Problem
Non-blind deconvolution is ill-posed, and existing image priors can require expensive optimization while deep methods may need retraining for different blur kernels.
Method
The method decomposes non-blind deconvolution into gradient-domain denoising and image deconvolution, cascading an FCNN in a multi-stage architecture to iteratively restore images.
Results
The method performs favorably against state-of-the-art methods on synthetic and real-world images and is more efficient through GPU-parallel FCNN denoising.
Takeaways & Limitations
The learned image priors preserve image details while the iterative FCNN suppresses noises and ringing artifacts during non-blind deconvolution.
Takeaways & Limitations
Many existing image priors remain heavily based on empirical natural-image statistics and often lead to highly non-convex optimization problems requiring expensive computation.
Abstract
from arXiv · showhide
In this paper, we propose a fully convolutional networks for iterative non-blind deconvolution We decompose the non-blind deconvolution problem into image denoising and image deconvolution. We train a FCNN to remove noises in the gradient domain and use the learned gradients to guide the image deconvolution step. In contrast to the existing deep neural network based methods, we iteratively deconvolve the blurred images in a multi-stage framework. The proposed method is able to learn an adaptive image prior, which keeps both local (details) and global (structures) information. Both quantitative and qualitative evaluations on benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art algorithms in terms of quality and speed.
1. Introduction
Single-image non-blind deconvolution seeks a sharp image from a blurred image and known blur kernel, but the inverse problem is ill-posed and existing approaches face artifact, generalization, and efficiency limitations. The paper proposes an iterative FCNN that learns image priors from gradient-domain denoising and applies them to deconvolution.
- Non-blind deconvolution recovers a sharp latent image from a blurred image and known blur kernel, but solving for the image is ill-posed because noise is unknown.
- Conventional deconvolution methods suffer ringing artifacts and handle large motion and outliers less effectively.
- Hand-designed image priors rely heavily on natural-image statistics, often produce highly non-convex optimization, and require expensive computation.
- Existing deep neural methods require retraining for different blur kernels, limiting their practicality in real-world scenarios.
- The proposed FCNN learns image priors through gradient-domain denoising and uses them for iterative image deconvolution without retraining across blur kernels.
2. Related Work
Prior non-blind deblurring methods constrain an ill-posed problem with statistical or learned image priors, but many require complex, time-consuming optimization. Deep methods address restoration with neural networks, while kernel-specific initialization can still require per-kernel fine-tuning.
- Non-blind deblurring requires assumptions or prior knowledge to constrain its ill-posed solution space.
- Wiener deconvolution assumes Gaussian pixel values, whereas Hyper-Laplacian priors model the heavy-tailed distribution of natural-image gradients.
- Field-of-experts methods can produce complex optimization problems that are usually time-consuming to solve.
- GMM-based gradient and patch priors achieve good results but require heavy computational loads during optimization.
- Deep learning has been applied to denoising, super-resolution, and edge-preserving filtering.
- A prior CNN-based deblurring method uses SVD-based initialization but requires fine-tuning the network for every blur kernel.
3. Proposed Algorithm
The proposed algorithm alternates denoising and deconvolution: an FCNN learns gradient-domain priors, which guide iterative image restoration through a deconvolution module.
- Algorithm overview: The method decomposes non-blind deconvolution into denoising and deconvolution, using learned gradient information as an image prior.The denoising output guides the subsequent image restoration step.
- Optimization framework: Half-quadratic splitting separates the auxiliary-variable denoising problem from the image deconvolution problem.Once the auxiliary variable is obtained, the clear image can be solved efficiently with FFT.
- Network architecture: The network takes a blurry image and its blur kernel, then applies deconvolution followed by convolutions on horizontal and vertical gradients.The denoised gradients are used as priors for deconvolution in the next iteration.
- Gradient denoising: The FCNN removes noise and ringing artifacts from image gradients while preserving image textures and details.Vertical and horizontal gradients share network weights by transposing the vertical gradient during training.
- Training: FCNN weights are updated iteratively, while deconvolution hyper-parameters are trained end-to-end with fixed FCNN weights.The loss uses the learned denoising mapping on horizontal and vertical gradients together with the ground-truth image.
- Iterative refinement: Iterative gradient-domain processing reduces noise more effectively across iterations than intensity-domain processing in the illustrated comparison.The figure compares initial and refined outputs generated in the two domains.
4. Analysis and Discussion
The analysis examines iterative FCNN deconvolution, gradient-domain denoising, and the L1 loss. Iteration reduces artifacts, while gradient processing and L1 reconstruction improve restoration quality.
- Effect of FCNN for iterative deconvolution: Iteratively solving the deconvolution and denoising stages produces high-quality results, whereas one iteration leaves artifacts and lower PSNR.The iterative FCNN reduces artifacts and produces clearer images with higher PSNR than a one-iteration network.
- Gradient Domain v.s. Intensity Domain: The FCNN denoises vertical and horizontal gradients, whose outputs guide image deconvolution in subsequent iterations.The method applies the learned gradient prior to iterative restoration.
- Gradient Domain v.s. Intensity Domain: Gradient-domain reconstruction contains fewer noises and artifacts than intensity-domain reconstruction.The image gradient models details and structures, motivating its use for denoising and restoration.
- Effect of the Loss Function: L1 reconstruction loss converges better than L2 loss and removes noise and artifacts more effectively.The comparison uses the same settings for the first iteration; the L2 converged training loss is higher than the L1 loss.
- Training setup: Training uses randomly generated blur kernels to support the learned deconvolution process.Examples of these kernels are shown in Figure 6.
5. Experimental Results
Experiments evaluate the method on benchmark datasets across noise levels, estimated kernels, visual quality, and runtime. The iterative FCNN generally outperforms several baselines, preserves textures, and runs efficiently, while EPLL can slightly exceed it on some metrics at 1% noise.
- Quantitative comparisons: On 1% Gaussian noise, the proposed method outperforms HL, MLP, and CSF in PSNR and SSIM, while EPLL performs slightly better.EPLL requires complex optimization and tends to smooth details, whereas the proposed method produces clearer textures.
- Iterative evaluation: The iterative FCNN outperforms its one-iteration version on PSNR and SSIM at 1% noise.This comparison supports the effectiveness of the iterative design.
- Kernel estimation: With estimated blur kernels, the proposed method still performs well and can improve restored results from blind deblurring methods.The evaluation uses kernels estimated by Pan et al.
- Quantitative comparisons: At 3% and 5% Gaussian noise, the proposed methods outperform HL, MLP, and CSF and generate clearer images with fine textures.Tables 3 and 4 report the quantitative comparisons, and Figure 7 provides visual results.
- Running time: The proposed method runs more efficiently than other state-of-the-art methods in the reported runtime comparison.The comparison uses CPU execution for HL and EPLL and GPU execution for MLP, CSF, and the proposed method.
- Real blurry images: On one real blurry image with added 3% Gaussian noise, the three-iteration network has comparable performance to EPLL.CSF leaves noise in flat regions, while HL produces an overly blurry result.
6. Conclusions
The method combines FCNN-based denoising with iterative deconvolution to remove noise and ringing while preserving image details. It achieves favorable performance on synthetic and real-world images with improved efficiency through GPU-parallel denoising.
- The proposed algorithm combines a deconvolution part with an FCNN-based denoising part.The learned FCNN features assist the deconvolution process.
- The iterative FCNN deconvolution is designed to remove noise and ringing artifacts while preserving image details.
- The method performs favorably against state-of-the-art methods on synthetic and real-world images.
- GPU-parallel FCNN denoising makes the approach more efficient.