PaperlayerView exact revision on arXiv

Source-linked AI summary

MC-Blur: A Comprehensive Benchmark for Image Deblurring

Kaihao Zhang, Tao Wang, Wenhan Luo, Boheng Chen, Wenqi Ren, Bjorn Stenger, Wei Liu, Hongdong Li, Ming-Hsuan Yang

arXiv:2112.00234v3cs.CV

TL;DR

Image deblurring must handle blur caused by multiple factors, yet existing datasets and methods often target specific scenarios or rely on imperfect blur synthesis. The paper constructs MC-Blur with real-world and synthesized multi-cause blur, then benchmarks state-of-the-art methods across diverse settings. The resulting analyses compare method behavior, efficiency, and cross-dataset performance while identifying strengths and limitations across scenarios.

  • Problem

    Existing deblurring datasets and methods do not fully represent images containing multiple blur causes, while low-frame-rate averaging can create unnatural motion blur.

  • Method

    The paper constructs MC-Blur from high-frame-rate motion data, large-kernel UHD blur, heavy defocus, and real-world blurry images, then benchmarks recent methods across scenarios.

  • Results

    Benchmarking reveals different method behavior across motion, UHD, and defocus settings and provides a comprehensive view of current methods’ strengths and limitations.

  • Takeaways & Limitations

    MC-Blur supplies a multi-cause benchmark for studying deblurring methods in diverse and real-world scenarios.

Abstract

from arXiv · show

Blur artifacts can seriously degrade the visual quality of images, and numerous deblurring methods have been proposed for specific scenarios. However, in most real-world images, blur is caused by different factors, e.g., motion and defocus. In this paper, we address how different deblurring methods perform in the case of multiple types of blur. For in-depth performance evaluation, we construct a new large-scale multi-cause image deblurring dataset (called MC-Blur), including real-world and synthesized blurry images with mixed factors of blurs. The images in the proposed MC-Blur dataset are collected using different techniques: averaging sharp images captured by a 1000-fps high-speed camera, convolving Ultra-High-Definition (UHD) sharp images with large-size kernels, adding defocus to images, and real-world blurry images captured by various camera models. Based on the MC-Blur dataset, we conduct extensive benchmarking studies to compare SOTA methods in different scenarios, analyze their efficiency, and investigate the built dataset's capacity. These benchmarking results provide a comprehensive overview of the advantages and limitations of current deblurring methods, and reveal the advances of our dataset.

I. INTRODUCTION

Image deblurring restores sharp images from blurry inputs, but existing datasets and methods do not fully represent real-world blur caused by multiple factors. MC-Blur addresses this gap with four complementary subsets and extensive benchmarking across diverse scenarios.

  • Image deblurring restores a sharp image from an observed blurry input and is used in medical imaging, computational photography, and video enhancement.
  • Existing datasets can produce unnatural motion blur when low-frame-rate images are averaged, motivating more realistic data construction.
  • MC-Blur combines motion, UHD motion, heavy defocus, and real-world mixed blur through four complementary subsets.RHM uses high-speed-camera frames; UHDM uses large kernels; LSD captures heavy defocus; RMBQ contains real blurry images without sharp ground truth.
  • RHM uses sharp frames captured at 250, 500, and 1000 fps with varied devices to mimic real-world motion blur.
  • UHDM targets single-image UHD deblurring, while LSD and RMBQ address heavy defocus and qualitative real-world evaluation, respectively.LSD contains heavier defocus than existing datasets, and RMBQ includes images from varied mobile devices without sharp ground truth.
  • The study benchmarks recent state-of-the-art methods, analyzes efficiency, and evaluates cross-dataset learning to compare performance across scenarios.

II. RELATED WORK

The related-work section surveys datasets commonly used for image deblurring and reviews existing deblurring methods.

  • The paper surveys commonly used image deblurring datasets before reviewing existing image deblurring methods.

A. Image Deblurring Datasets

Prior deblurring datasets use frame averaging, kernel convolution, or paired cameras, but differ in scale, alignment accuracy, and blur coverage.

  • Existing datasets synthesize motion blur by averaging frames or convolving images with uniform or non-uniform kernels.
  • The GoPro, HIDE, and REDS datasets generate blur by averaging continuous sharp frames over a time window.
  • Two-camera image pairs provide realistic blur but require alignment, which can produce imprecise correspondences.
  • Existing datasets lack either defocus or UHD images, while some real-blur datasets remain small for deep-learning-based deblurring.

B. Deblurring Methods

Image deblurring methods include conventional prior-based approaches and deep networks, with conventional methods challenged by complex dynamic-scene blur.

  • Deblurring methods are broadly divided into conventional approaches and deep learning-based approaches.
  • Conventional Deblurring Methods: Conventional methods impose constraints on blur kernels or latent images using priors such as sparse gradients, dark channels, and structure.
  • Conventional Deblurring Methods: Conventional methods often struggle with dynamic scenes containing camera motion, object motion, and scene-depth variation.
  • Deep Deblurring Methods: Deep methods train neural networks end-to-end by comparing restored outputs with ground-truth sharp images.
  • Deep Deblurring Methods: Deep deblurring architectures include multi-scale, adversarial, patch-level, and blur-kernel-estimation approaches.

III. MC-BLUR DATASET

MC-Blur is a large-scale dataset spanning multiple blur causes, resolutions, capture conditions, and evaluation settings.

  • MC-Blur contains uniform blur, frame-averaged motion blur, heavy defocus blur, and real-world blur across more than 1000 diverse scenes.
  • RHM Subset: RHM synthesizes motion blur from high-frame-rate sharp images, including a 1000-fps setting with 30,000 training and 7,500 testing images.
  • UHDM Subset: UHDM targets 4K+ restoration with large blur kernels because UHD deblurring requires recovering more details than low-resolution restoration.
  • LSD Subset: Existing defocus data are limited because the Dual-Pixel dataset contains only 500 blurry-sharp pairs and primarily targets the dual-pixel problem.
  • LSD Subset: LSD provides 2,250 training and 550 testing sharp-blurry pairs at resolutions of at least 3600 × 2400 pixels.

IV. BENCHMARKING AND ANALYSIS

The benchmark evaluates established deblurring methods across MC-Blur subsets using quantitative, qualitative, efficiency, and cross-dataset analyses.

  • The study benchmarks deblurring methods across blurry-image categories, analyzes UHD efficiency, and examines cross-dataset learning.
  • Nine state-of-the-art methods are evaluated using PSNR and SSIM, with qualitative comparisons on synthesized and real blurry images.
  • NIQE and SSEQ are additionally used to evaluate performance on real-world blurry images.

B. Benchmarking on the MC-Blur Dataset

Benchmarking across MC-Blur subsets shows that deblurring performance depends strongly on blur type and image resolution. MIMO-UNet and Restormer excel on motion blur, MPRNet leads UHD deblurring, and MIMO-UNet performs best on defocus and qualitative real-world tests.

  • RHM Subset: MIMO-UNet and Restormer consistently outperform other methods on the RHM motion-blur subset.The paper attributes this result to their multi-scale schemes and transformer blocks.
  • UHDM Subset: Large-size kernels make UHDM more difficult, with all methods achieving lower PSNR and SSIM than on RHM.UHD deblurring also requires recovering more details, while the evaluated networks were designed for non-UHD images.
  • UHDM Subset: MPRNet is the top performer on UHDM, while MIMO-UNet and Restormer also show strong performance.The paper links MPRNet’s result to its multi-stage architecture.
  • LSD Subset: Defocus deblurring performs significantly worse than motion deblurring because defocus causes depth-dependent blur and loss of vital image information.MIMO-UNet achieves the strongest performance on the LSD subset.
  • RMBQ Subset: MIMO-UNet exhibits superior qualitative performance on real-world RMBQ images, recovering sharp text or image structure.The methods were trained on RHM and tested on RMBQ without ground-truth sharp images.

C. Efficiency Analysis on UHD Images

The efficiency analysis evaluates runtime and model overhead on UHD images. Several methods require more than ten seconds per image, whereas lightweight models process UHD images in a few seconds.

  • Runtime Analysis: More than ten seconds are required to process one UHD image with DeepDeblur, SRN, DMPHN, DBGAN, MPRNet, and Restormer.The measurements use a standard platform with a P40 GPU.
  • Runtime Analysis: DeblurGAN, DeblurGAN-v2, and MIMO-UNet take 2.46, 3.63, and 2.45 seconds, respectively, per UHD image.
  • Efficiency Analysis: The evaluation provides runtime and parameter-overhead benchmarks for assessing the practical applicability of deblurring methods on high-resolution images.

D. Cross-dataset Learning for Motion Deblurring

Cross-dataset experiments show that motion-deblurring performance depends on the relationship between training and testing sources. RHM provides stronger single-source generalization, and adding RHM improves multi-source training.

  • Single-source Training: A deblurring network trained and tested on the same source is expected to achieve the best performance.The study trains an identical network under five combinations of training sets and evaluates it across testing sets.
  • Single-source Training: RHM-trained models consistently achieve the best performance across different test datasets among single-source training settings.
  • Multi-source Training: Adding RHM as an additional training source significantly improves performance in multi-source training.
  • Real-world Testing: RHM-trained networks perform better on the real-world RMBQ and RWBI datasets, which lack ground-truth sharp images.

E. The Effectiveness of The LSD Set

The LSD subset targets heavier defocus blur than DPDD, and cross-dataset experiments show that training data severity and distribution affect defocus-deblurring performance. Models trained on one blur type also transfer poorly to other blur types.

  • Motivation: LSD contains heavier defocus blur than DPDD, motivating a separate evaluation of its effectiveness.
  • Cross-dataset Defocus Learning: Models trained on LSD perform better on heavy-defocus images, whereas DPDD-trained models perform better on non-heavy-defocus images.
  • Cross-dataset Defocus Learning: A model trained on one defocus dataset cannot perform well on the other despite both datasets containing real defocus blur.
  • Cross-factor Generalization: Models trained on RHM motion blur perform poorly when recovering sharp images from LSD and UHDM blur.The experiment evaluates MPRNet, Restormer, and MIMO-UNet trained on RHM and tested on other blur factors.
  • Cross-factor Generalization: Recovering sharp images across varied blur types remains an open topic.

G. Discussion

Benchmarking on MC-Blur shows that deblurring performance and efficiency vary substantially across blur types and image resolutions. GAN-based methods underperform on motion blur, UHD blur remains more difficult and slower to process, while defocus results are closer across architectures.

  • GAN-based networks achieve lower PSNR and SSIM than non-GAN methods on motion-blurred images, with fewer differences on defocus images.The comparison suggests that whole-image attention may be more useful for defocus deblurring than pixel-level losses alone.
  • Current deep deblurring networks generate high-quality results for motion blur but struggle more with large-kernel UHD blurry images.The paper identifies UHD image deblurring as a meaningful direction for further study as UHD capture becomes more common.
  • Deblurring a UHD image takes significantly longer than deblurring a non-UHD image, making high-rate UHD restoration with maintained performance an open problem.The cited discussion states that current methods can process a non-UHD image in two seconds, whereas UHD processing is substantially slower.
  • Cross-dataset benchmarking is used to examine the advantages of MC-Blur and the limitations of current state-of-the-art deblurring methods across diverse scenarios.The dataset contains multiple blur causes and supports benchmarking across its distinct subsets.
Loading 2112.00234v3…