Source-linked AI summary
DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
Orest Kupyn, Tetiana Martyniuk, Junru Wu, Zhangyang Wang
TL;DR
Single-image blind motion deblurring must handle unknown, spatially varying blur and other real-world artifacts while balancing restoration quality against efficiency. DeblurGAN-v2 combines an FPN-based generator, a relativistic double-scale discriminator, and interchangeable backbones to navigate that trade-off. Across benchmarks, it reports competitive quality and substantially faster inference, including lightweight models aimed at real-time video deblurring.
Problem
Single-image blind motion deblurring involves unknown, spatially varying blur kernels complicated by noise and artifacts, motivating improvements in quality and efficiency.
Method
DeblurGAN-v2 uses a conditional GAN with an FPN generator, a relativistic global-and-local discriminator, hybrid losses, and interchangeable sophisticated or lightweight backbones.
Results
DeblurGAN-v2 achieves competitive objective and subjective deblurring quality across benchmarks, while MobileNet-DSC is 11 times faster than DeblurGAN, over 100 times faster than two other competitors, and has a 4 MB model size.
Takeaways & Limitations
Interchangeable backbones provide a practical quality-efficiency spectrum, with lightweight variants supporting the possibility of real-time video deblurring and the architecture extending to general image restoration.
Abstract
from arXiv · showhide
We present a new end-to-end generative adversarial network (GAN) for single image motion deblurring, named DeblurGAN-v2, which considerably boosts state-of-the-art deblurring efficiency, quality, and flexibility. DeblurGAN-v2 is based on a relativistic conditional GAN with a double-scale discriminator. For the first time, we introduce the Feature Pyramid Network into deblurring, as a core building block in the generator of DeblurGAN-v2. It can flexibly work with a wide range of backbones, to navigate the balance between performance and efficiency. The plug-in of sophisticated backbones (e.g., Inception-ResNet-v2) can lead to solid state-of-the-art deblurring. Meanwhile, with light-weight backbones (e.g., MobileNet and its variants), DeblurGAN-v2 reaches 10-100 times faster than the nearest competitors, while maintaining close to state-of-the-art results, implying the option of real-time video deblurring. We demonstrate that DeblurGAN-v2 obtains very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency. Besides, we show the architecture to be effective for general image restoration tasks too. Our codes, models and data are available at: https://github.com/KupynOrest/DeblurGANv2
1. Introduction
DeblurGAN-v2 targets single-image blind motion deblurring with a flexible GAN framework designed to improve both restoration quality and inference efficiency. Its FPN-based generator and backbone choices support a range from high quality to compact, fast models.
- Single-image blind motion deblurring is difficult because real-world blur kernels are unknown, spatially varying, and complicated by noise and other artifacts.
- GANs are used in image restoration because they can produce sharper and more plausible textures than classical feed-forward encoders.
- DeblurGAN-v2 introduces a conditional GAN framework with an FPN generator and a relativistic least-squares discriminator operating at global and local scales.
- Inception-ResNet-v2 targets state-of-the-art quality, whereas MobileNet and MobileNet-DSC target compact size and fast inference.
- 11 times faster than DeblurGAN, over 100 times faster than two earlier competitors, and 4 MB in model size are reported for MobileNet-DSC.These efficiency results are presented alongside experiments on three benchmarks and the possibility of real-time video deblurring.
2. Related work
Prior deblurring methods range from handcrafted-prior and kernel-based approaches to end-to-end CNNs and GANs. DeblurGAN-v2 builds on this progression while addressing the ill-posedness and computational demands of blind motion deblurring.
- Blind deblurring is more realistic than non-blind deblurring but highly ill-posed, while handcrafted natural-image priors poorly capture complicated real-image blur variations.
- Earlier deep methods estimate blur kernels or motion flow, whereas end-to-end CNNs restore clean images directly from blurry inputs.
- Multi-scale CNN and Scale-Recurrent CNN methods process different blur levels using image pyramids or recurrent multi-scale structures, but multi-scale image processing is time-consuming and memory-demanding.
- GAN-based deblurring treats restoration as image-to-image translation and includes DeblurGAN, which used Wasserstein GAN with gradient penalty and perceptual loss.
- Least-Squares GANs address saturating logarithmic losses with smoother gradients, while relativistic GANs estimate whether real data are more realistic than sampled fake data.
3. DeblurGAN-v2 Architecture
DeblurGAN-v2 combines an FPN-based generator, flexible backbones, a relativistic least-squares discriminator, and global-local discrimination for efficient motion deblurring. Its training uses multi-scale feature aggregation, hybrid losses, and interpolated training data to improve restoration quality and support different efficiency targets.
- Feature Pyramid Deblurring: The generator uses an FPN backbone to aggregate features across scales instead of processing multiple input-scale images.Five feature maps are upsampled, concatenated, and followed by additional upsampling and convolutional layers to restore image resolution.
- Flexible Backbones: Backbone swapping lets the architecture navigate deblurring quality and inference efficiency.Inception-ResNet-v2 targets strong performance, while MobileNet V2 and depthwise-separable convolutions reduce complexity; the switch is implemented as a one-line command.
- Relativistic Least-Squares Discriminator: The discriminator replaces WGAN-GP with a relativistic least-squares objective that is observed to train faster and more stably.The paper also reports sharper outputs and higher perceptual quality with this objective.
- Double-Scale Discriminator: A double-scale discriminator combines a local patch branch with a global full-image branch.The global branch supplies spatial context for highly non-uniform blurs and complex object motion, while the local branch captures patch-level features.
- Overall Loss Function: Training uses a hybrid three-term loss combining pixel-space, perceptual, and adversarial objectives.The pixel-space MSE term is included to help correct color and texture distortions, while adversarial losses include global and local discriminator terms.
- Training Datasets: Training data are generated by interpolating 240-fps videos to 3840 fps before averaging frames.This preparation produces smoother, more continuous synthetic blurs than directly averaging consecutive frames, without noticeably changing PSNR or SSIM.
4. Experimental evaluation
DeblurGAN-v2 is evaluated across benchmark datasets using multiple backbones, objective and subjective quality measures, and inference-efficiency comparisons. Results show a strong quality–efficiency trade-off, with lightweight variants approaching competitive quality while enabling much faster deblurring.
- Benchmark comparisons: The evaluation compares DeblurGAN-v2 with traditional and deep-learning methods using PSNR, SSIM, and average single-GPU inference time.The experiments cover GoPro, Kohler, DVD, and Lai-related evaluations.
- GoPro evaluation: DeblurGAN-v2 with Inception-ResNet-v2 ranks among the top PSNR/SSIM performers while taking 78% less inference time than SRN.It has slightly lower PSNR than SRN but higher SSIM.
- GoPro evaluation: MobileNet and MobileNet-DSC achieve SSIM values of 0.925 and 0.922, respectively, while being up to 100 times faster than competing deep deblurring methods.These scores are comparable to DeblurGAN’s 0.927 and DeepDeblur’s 0.916.
- Efficiency: MobileNet-DSC processes one image in 0.04s, corresponding to near real-time deblurring for 25-fps video.The model is also described as simultaneously achieving reasonably high performance and high inference efficiency.
- Kohler evaluation: On Kohler, Inception-ResNet-v2 nearly ties SRN in PSNR/SSIM while using one-fifth of SRN’s inference complexity, and both lightweight variants outperform DeblurGAN on both metrics.Visual examples show restored edges and textures without noticeable artifacts, whereas SRN exhibits color artifacts in the cited example.
- DVD evaluation: On the DVD single-frame setting, MobileNet outperforms WFA and DVD while being at least 17 times faster, despite being tested at a higher resolution than DVD.The authors describe this as evidence of potential for video deblurring, while identifying video-specific optimization as future work.
- Subjective evaluation: On the Lai real-image benchmark, Inception-ResNet-v2 achieves the highest subjective quality, while MobileNet variants remain preferred to DeepDeblur and DeblurGAN despite 2–3 orders-of-magnitude greater speed.Because the images lack sharp counterparts, the study compares score rankings rather than absolute subjective scores.
- Ablation study: The ablation study finds that adding FPN, global-scale discrimination, and revised losses steadily improves PSNR and SSIM, with FPN contributing most significantly.MSE or perceptual loss also benefits training stability and final results.
5. Conclusion
DeblurGAN-v2 is presented as an efficient and flexible image-deblurring framework with promising quantitative and qualitative results. Its backbone choices support tradeoffs between performance and efficiency, with planned extensions to video enhancement and mixed degradations.
- DeblurGAN-v2 is introduced as a powerful and efficient image-deblurring framework with promising quantitative and qualitative results.
- Different backbones enable flexible tradeoffs between deblurring performance and efficiency.
- The authors plan to extend DeblurGAN-v2 to real-time video enhancement and improved handling of mixed degradations.