Source-linked AI summary
A Neural Approach to Blind Motion Deblurring
Ayan Chakrabarti
TL;DR
Blind motion deblurring must recover sharp images from observations with unknown camera-shake blur, a more ill-posed problem than non-blind deconvolution. The paper predicts Fourier-domain deconvolution filters for overlapping patches, aggregates their restorations, and estimates a global kernel for final deconvolution. It reports performance close to state-of-the-art iterative methods with substantially faster kernel estimation on a GPU.
Problem
Blind deconvolution must recover sharp natural images when the motion blur kernel is unknown, making it significantly more ill-posed than non-blind deconvolution.
Method
A discriminatively trained neural network predicts complex Fourier coefficients of patch deconvolution filters, whose overlapping outputs initialize sharp-image estimation before global-kernel non-blind deconvolution.
Results
The method achieves performance close to state-of-the-art iterative methods while providing a significant speed advantage.
Takeaways & Limitations
The approach offers a practically useful trade-off between reliability and speed for blind image deconvolution.
Takeaways & Limitations
The method relies on training kernels being translated canonically by centering each kernel’s center of mass, and assumes additive i.i.d. Gaussian noise.
Abstract
from arXiv · showhide
We present a new method for blind motion deblurring that uses a neural network trained to compute estimates of sharp image patches from observations that are blurred by an unknown motion kernel. Instead of regressing directly to patch intensities, this network learns to predict the complex Fourier coefficients of a deconvolution filter to be applied to the input patch for restoration. For inference, we apply the network independently to all overlapping patches in the observed image, and average its outputs to form an initial estimate of the sharp image. We then explicitly estimate a single global blur kernel by relating this estimate to the observed image, and finally perform non-blind deconvolution with this kernel. Our method exhibits accuracy and robustness close to state-of-the-art iterative methods, while being much faster when parallelized on GPU hardware.
1 Introduction
The paper proposes a neural approach to blind motion deblurring that predicts patch-level deconvolution filters, aggregates overlapping-patch restorations, and then estimates a global blur kernel. It targets the accuracy of iterative methods while avoiding their long runtimes through discriminative, non-iterative processing.
- Motivation: Blind motion deblurring seeks to recover sharp photographs degraded by camera shake, but existing methods trade efficiency against robustness.Heuristic methods can fail on many blurred images, whereas iterative approaches achieve stronger performance but require much longer runtimes.
- Approach: The method uses the locally restored image to infer one global blur kernel, followed by non-blind deconvolution.This global fitting step improves on the initial estimate formed by averaging independent patch outputs.
- Positioning: The work tests whether discriminatively trained neural networks can approach generative-prior methods without multiple iterative refinements.Earlier neural approaches addressed more restricted settings, including text images or discrete blur-kernel choices, while another general method mimicked iterative refinement.
- Approach: The proposed method uses a neural network to restore individual patches from observations blurred by arbitrary camera-shake kernels.The network is applied to overlapping patches and its outputs are composed into an initial sharp-image estimate.
- Approach: Rather than directly regress intensities or iteratively estimate blur kernels, the network predicts complex Fourier coefficients of a deconvolution filter.A multi-resolution frequency representation with locality-limited early connectivity reduces the number of learned weights.
2 Patch-wise Neural Deconvolution
The method restores blurred images patch by patch by predicting deconvolution filters in the Fourier domain, then combines these local estimates into sharp-image patches. Its architecture uses multi-resolution frequency inputs and is trained end-to-end against sharp-patch intensities.
- The observed image is modeled as a sharp image convolved with an unknown blur kernel and additive Gaussian noise.
- The network receives a 65 × 65 observed patch and predicts restoration-filter DFT coefficients for its central 33 × 33 patch.The larger input accounts for dependencies outside the restored patch, especially near its boundaries.
- The predicted filter is applied in the Fourier domain, followed by an inverse DFT and cropping to obtain the restored central patch.Real-valued images and a unit-sum kernel impose conjugate symmetry and set the zero-frequency coefficient to 1.
- Training minimizes mean squared error between predicted and true sharp-patch intensities, while gradients pass through the linear filtering and inverse-DFT operations.This makes the filter coefficients optimal with respect to final restored intensities rather than an intermediate target.
- Network Architecture: The input is encoded with multi-resolution frequency bands, using lower-resolution sampling for higher frequencies to reduce the representation size.DFTs at 17 × 17, 33 × 33, and 65 × 65 scales provide band-pass and high-frequency components, alongside a low-pass band.
- Network Architecture: A seven-hidden-layer feed-forward network predicts the filter coefficients from this encoding, with early connectivity restricted to adjacent frequency bands.The network is trained on synthetic motion-blurred Pascal VOC patches with Gaussian noise matched to the evaluation benchmark.
3 Whole Image Restoration
The method restores whole images by averaging neural estimates from overlapping patches, estimating one global motion kernel, and applying non-blind deconvolution. Kernel estimation uses positivity and unit-sum constraints, gradient selection, L1 regularization, and a single global-kernel optimization.
- Neural patch restoration: The network restores every overlapping blurry-image patch, and Hanning-weighted averaging combines the patch estimates into an initial sharp-image estimate.Each pixel is averaged across estimates from patches containing it.
- Global kernel estimation: The method estimates one global blur kernel by relating the observed image to the neural-average sharp-image estimate.This incorporates the fact that the entire image shares the same motion kernel.
- Global kernel estimation: Kernel estimation enforces k[n] > 0 and Σ_n k[n] = 1 within a fixed 51 × 51 support, without assuming the kernel size is known.The support size follows the benchmark convention, while the effective kernel size remains unknown.
- Global kernel estimation: Only strong gradients participate in kernel estimation: derivative-filter responses are retained at the two percent pixel locations with highest magnitudes.The method uses first- and second-order derivatives at eight orientations.
- Final restoration: Unlike iterative methods that repeatedly refine both estimates, this method estimates the global kernel once from the neural-average output.The one-time estimate is followed by non-blind deconvolution using EPLL.
- Final restoration: The kernel objective is optimized with L1-regularized versions over a small range of λ values, using an efficient Fourier-domain half-quadratic-splitting procedure.Candidate kernels are clipped to positivity, sparsified, normalized, and selected by the original unregularized cost before final EPLL deconvolution.
4 Experiments
Experiments show that the network produces content-dependent patch restorations, while the full method achieves accuracy and robustness near state-of-the-art methods with substantially lower kernel-estimation time.
- Benchmark: The benchmark contains 640 blurred images generated from 80 natural images using 8 real motion-blur kernels.
- Local Network Predictions: Predicted restoration filters vary across patches from the same blurred image, reflecting image-content-dependent frequency restoration.The network restores attenuated frequency components without amplifying noise.
- Local Network Predictions: Patch-based restoration can recover frequency components even when corresponding kernel components are impossible to estimate from content-free frequency ranges.This motivates estimating restoration filters rather than blur kernels from individual patches.
- Local Network Predictions: Averaging overlapping patch estimates provides an initial sharp-image estimate, while fitting a global kernel to these outputs supports the final restoration.The locally restored frequencies are coherent with the same true blur kernel.
- Performance Evaluation: The full method performs close to the two state-of-the-art methods, with near-identical success rate to them and better outlier performance than Sun et al..It significantly outperforms the remaining evaluated algorithms across metrics and remains more consistent across kernel sizes than the previous neural approach.
- Performance Evaluation: 65 seconds of GPU kernel estimation substantially undercuts the 91-minute and 38-minute runtimes reported for the two state-of-the-art methods.The initial neural-average estimate accounts for 45 seconds, and batched patch processing maps naturally to GPU parallelism.
- Performance Evaluation: The method offers a practically useful trade-off between reliability and speed compared with faster heuristic or prior-based methods.Those simpler methods achieve lower running times but are less robust and have lower success rates.
5 Conclusion
The paper presents a neural blind-deconvolution method that achieves performance comparable to state-of-the-art iterative methods while offering substantial speed advantages. It also identifies local processing as a basis for future spatially varying blur applications.
- 5 Conclusion: A neural network restores individual blurry patches, whose outputs are averaged before estimating a global blur kernel.The resulting kernel supports final whole-image restoration through non-blind deconvolution.
- 5 Conclusion: The approach delivers comparable performance to state-of-the-art iterative blind deblurring methods with significant speed advantages.Its efficiency is especially relevant for practical image restoration workflows.
- 5 Conclusion: Because the network processes local regions independently, the authors expect it may support spatially varying blur from defocus and subject motion.They also propose exploring information sharing across patches through new architectures and pooling strategies.