Source-linked AI summary
Scale-recurrent Network for Deep Image Deblurring
Xin Tao, Hongyun Gao, Yi Wang, Xiaoyong Shen, Jue Wang, Jiaya Jia
TL;DR
Single-image deblurring is ill-posed, especially when real blur is more complex than traditional models assume. The paper proposes SRN-DeblurNet, a scale-recurrent coarse-to-fine network with shared weights, recurrent hidden states, and encoder-decoder ResBlocks. Experiments report more efficient training and higher-quality restoration than existing methods, quantitatively and qualitatively.
Problem
Real-world image deblurring is ill-posed because complex motion and processing effects exceed simplified blur models, while traditional solutions require costly tuning and computation.
Method
SRN-DeblurNet shares network weights across coarse-to-fine scales and uses recurrent hidden states within an encoder-decoder ResBlock architecture.
Results
The method produces higher-quality deblurring results than existing methods both quantitatively and qualitatively, while using approximately one-quarter of Nah et al.'s training time for similar restoration.
Takeaways & Limitations
The proposed scale-recurrent structure reduces parameters and training difficulty while supporting state-of-the-art image deblurring results.
Abstract
from arXiv · showhide
In single image deblurring, the "coarse-to-fine" scheme, i.e. gradually restoring the sharp image on different resolutions in a pyramid, is very successful in both traditional optimization-based methods and recent neural-network-based approaches. In this paper, we investigate this strategy and propose a Scale-recurrent Network (SRN-DeblurNet) for this deblurring task. Compared with the many recent learning-based approaches in [25], it has a simpler network structure, a smaller number of parameters and is easier to train. We evaluate our method on large-scale deblurring datasets with complex motion. Results show that our method can produce better quality results than state-of-the-arts, both quantitatively and qualitatively.
1. Introduction
Image deblurring is ill-posed because real blur is complex, while traditional methods rely on costly assumptions and tuning. The paper proposes SRN, which shares weights across scales and combines recurrent modules with an encoder-decoder ResBlock network.
- Real blur from camera shake, object motion, and defocus makes deblurring an ill-posed recovery of sharp edges and details.
- Traditional methods use blur-model constraints and image priors but require intensive tuning, expensive computation, and simplified assumptions that hinder real-world performance.
- SRN explores a more effective network structure for multi-scale image deblurring and addresses two important issues in CNN-based systems.
- Scale-recurrent Structure: Sharing network weights across scales reduces training difficulty and provides stability benefits compared with independent parameters at each scale.
- Encoder-decoder ResBlock Network: The recurrent structure uses hidden states to carry useful information across scales, while the encoder-decoder ResBlock network provides a large receptive field for large-motion deblurring.
- Experiments report approximately one-quarter of Nah et al.'s training time for similar restoration, fewer than one-third as many trainable parameters, and higher-quality results.
2. Related Work
Prior deblurring methods use handcrafted priors, learned components, or end-to-end CNNs, often following coarse-to-fine restoration. CNN image-processing architectures include encoder-decoder, multi-scale, cascaded, and dilated designs.
- Image/Video Deblurring: Traditional deblurring methods combine natural-image priors with coarse-to-fine optimization, while frequency-domain methods form an exception with limited applicability.
- Image/Video Deblurring: Learning-based methods range from replacing traditional modules with learned parameters to end-to-end networks for image and video deblurring.
- CNNs for Image Processing: Encoder-decoder networks increase regression ability through a U-Net-like structure and are widely used across image-processing tasks.
- CNNs for Image Processing: Multi-scale CNNs and cascaded refinement networks simplify training by progressively refining outputs from a very small scale.
- CNNs for Image Processing: Dilated convolutional networks use increasing dilation rates to approximate increasing kernel sizes.
3. Network Architecture
SRN-DeblurNet performs coarse-to-fine deblurring with a recurrent network that processes blurry images across scales and propagates intermediate information. Its encoder-decoder design uses residual blocks, ConvLSTM-based hidden states, and large convolution kernels to handle severe motion.
- Scale-recurrent Network: SRN-DeblurNet takes downsampled blurry images at multiple scales and produces corresponding sharp images, with the full-resolution image as final output.
- Scale-recurrent Network: At each scale, the network combines the blurry image with an upsampled initial deblurred result from the previous scale to estimate a sharper image.
- Scale-recurrent Network: Shared recurrent processing passes hidden-state features across scales, capturing image structures and kernel information from coarser scales.
- Encoder-decoder ResBlock: The encoder-decoder transforms inputs into smaller spatial feature maps with more channels, then decodes them back toward the input shape.
- Model configuration and losses: The model uses three scales, with each successive coarser scale having half the spatial size, and applies Euclidean loss at every scale.The ground truth is downsampled to each scale using bilinear interpolation; scale weights are empirically set to 1.0.
4. Experiments
Experiments evaluate training setup, scale-recurrent and encoder-decoder ResBlock designs, and comparisons with prior deblurring methods. Results show that multi-scale processing, shared recurrent structure, and ResBlocks improve restoration quality while maintaining efficiency.
- Experimental Setup: The GOPRO dataset provides 3,214 blurry/clear pairs, with 2,103 used for training and 1,111 for evaluation.
- Experimental Setup: The proposed method trains for about 72 hours and processes a 720 × 1280 image in around 1.6 seconds.
- Multi-scale Strategy: 29.98dB versus 28.40dB PSNR shows that the multi-scale SRN outperforms the same-size single-scale model SS.
- Multi-scale Strategy: Two scales improve results, while three scales improve them further with only minor additional gains.
- Multi-scale Strategy: Shared-weight models outperform the independently parameterized scale-cascaded model under the fixed-data and fixed-epoch training setting.
- Multi-scale Strategy: ConvLSTM achieves the best results among the tested recurrent modules, outperforming vanilla RNN and the no-recurrence variant.
- Encoder-decoder ResBlock Network: The SR-EDRB models perform best among encoder-decoder baselines, with quality improving as the number of ResBlocks increases.
- Comparisons: On complex benchmark blur, the proposed method produces sharper structures, clearer details, and fewer artifacts than the compared methods, including on extreme motion.
5. Conclusion
The paper identifies a suitable coarse-to-fine network structure for image deblurring and proposes a scale-recurrent network with encoder-decoder ResBlocks. The resulting model uses fewer parameters, is easier to train, and achieves state-of-the-art qualitative and quantitative results.
- The proposed scale-recurrent network combines shared recurrent processing across scales with encoder-decoder ResBlocks.
- Compared with previous multi-scale deblurring methods, the new structure has fewer parameters and is easier to train.
- The method produces state-of-the-art results both qualitatively and quantitatively.