BlurDM: A Blur Diffusion Model for Image Deblurring

1 Introduction

• Dynamic scene deblurring must recover sharp details from a single image despite directional, non-uniform blur.The task is described as highly ill-posed.Source: §1 · p. 1
• Regression-loss deblurring can produce over-smoothed results with limited high-frequency detail, while standard diffusion is not designed specifically for deblurring.Directly applying standard diffusion to deblurring networks is described as limiting their potential.Source: §1 · p. 1
• Motion blur accumulates along motion trajectories during continuous exposure, unlike the random perturbations modeled by conventional diffusion.BlurDM uses this distinction to motivate a diffusion process that progressively introduces structured blur.Source: §1 · p. 2
• BlurDM progressively adds both noise and blur, implicitly modeling blur residuals without ground-truth residual labels, then removes both to restore sharp images.It uses dedicated noise and blur residual estimators in the reverse process.Source: §1 · p. 2
• BlurDM is reported to significantly and consistently improve four deblurring models on four benchmark datasets.The paper positions BlurDM as a prior-generation network for enhancing existing deblurring models.Source: §1 · p. 2

2.1 Image Deblurring

• CNN approaches use recurrent multi-scale, multi-patch, and multi-temporal designs, while Transformer approaches model long-range dependencies but can require substantial data and memory.Regression-loss methods are also described as prone to oversmoothing and loss of high-frequency detail.Source: §2.1 · p. 3

2.2 Diffusion Models

• Earlier diffusion work also uses stochastic refinement, multi-scale guidance, and task-specific priors for restoration.These approaches establish diffusion’s use in low-level vision and deblurring.Source: §2.2 · p. 3
• Diffusion has been used for restoration and deblurring, including latent-space prior generation to reduce latency, but these methods overlook intrinsic blur formation.The paper contrasts blur formation as convolutional with residual diffusion’s direct subtraction of sharp and blurred images.Source: §2.2 · p. 3

3 Proposed Method

• BlurDM integrates blur diffusion with noise diffusion to incorporate blur formation into a diffusion framework for image deblurring.The framework’s three-stage integration learns and uses a diffusion prior in a deblurring network.Source: §3 · pp. 3–4, Figure 2 · p. 4

3.1 Dual Noise and Blur Diffusion Process

• The forward process progressively adds noise and blur to a sharp image, with blur accumulation representing continuous exposure.Because datasets lack ground-truth blur residuals, the formulation implicitly represents them; each transition is Gaussian with a deterministic mean shift from the blur residual.Source: §3.1 · pp. 4–5

3.2 Dual Denoising and Deblurring Process

• The reverse process estimates and removes blur and noise residuals from a terminal observation sampled around the fully blurred input.Dedicated blur and noise estimators approximate the respective residuals at each step.Source: §3.2 · p. 5

3.3 Latent BlurDM

• Latent BlurDM uses a three-stage strategy: train a sharp prior with the Sharp Encoder and Prior Fusion Module, learn a diffusion prior from blurred inputs, then jointly optimize the components.The Prior Fusion Module modulates decoder features with the sharp prior; the final inference model uses the Blur Encoder, BlurDM, and deblurring network.Source: §3.3 · p. 6

4.1 Experimental Setup

• The evaluation uses MIMO-UNet, Stripformer, FFTformer, and LoFormer, with GoPro, HIDE, and both RealBlur subsets.GoPro and RealBlur-J/R include training and test pairs; HIDE is used only for testing.Source: §4.1 · p. 7

4.2 Experimental Results

• 0.53 dB average PSNR gain, 0.004 SSIM gain, and 0.0028 LPIPS reduction are reported across backbones and datasets.Average PSNR improvements by dataset are 0.31 dB on GoPro, 0.32 dB on HIDE, 0.78 dB on RealBlur-J, and 0.69 dB on RealBlur-R.Source: §4.2 · p. 7
• BlurDM-enhanced models produce sharper, more visually appealing outputs than their baselines on GoPro, HIDE, and RealBlur-J.The method is integrated in the deblurring network’s latent space to use learned image priors while preserving fidelity to sharp-image contents.Source: §4.2 · p. 7, Figure 4 · p. 8, Figure 5 · p. 8

4.3 Ablation studies

• Noise-only and blur-only estimator variants each improve over baseline, while the complete BlurDM design achieves the best result through complementary gains.The ablation compares a conventional noise-estimator design, a blur-residual-only variant, and the full design.Source: §4.3 · p. 8, Table 2 · p. 9
• Decoded latent outputs progress from blurred I5 to sharp I0, and GoPro performance improves with two iterations and peaks at five.BlurDM was trained with T = 5 steps; eight iteration settings were tested on GoPro.Source: §4.3 · p. 9, Figure 7 · p. 10
• BlurDM adds averages of 4.16G FLOPs, 3.33M parameters, and 9 milliseconds across the four deblurring models.The overhead is measured for 256 × 256 images on an NVIDIA GeForce RTX 3090; parameter counts vary with decoder channel dimensions.Source: §4.3 · p. 9, Table 4 · p. 10
• BlurDM outperforms DDPM and RDDM; DDPM’s gain is comparable to an MLP prior, while BlurDM explicitly integrates blur formation into diffusion.The comparison reports higher PSNR and SSIM for BlurDM than the diffusion-based alternatives, with comparable parameter counts and FLOPs.Source: §4.3 · pp. 8–9
• The full three-stage training pipeline achieves the highest PSNR among the tested settings.The comparisons include an ideal sharp-image prior upper bound and a jointly optimized, purely data-driven setting.Source: §4.3 · p. 10, Table 6 · p. 10

5 Limitations

• BlurDM is designed to handle blur caused by camera motion and moving objects.Its design follows the motion-blur formation process.Source: §5 · p. 10
• Defocus blur is depth-dependent and arises from optical aberrations, unlike motion blur's temporal accumulation.The authors say BlurDM may not be well-suited to defocus blur.Source: §5 · p. 10
• The authors leave defocus deblurring open, suggesting depth estimation or optical defocus modeling as possible directions.Source: §5 · p. 10

6 Conclusion

• BlurDM effectively improves deblurring performance across four deblurring models and four datasets.Source: §6 · p. 10
• In the forward process, BlurDM progressively degrades sharp images by adding both noise and blur.Source: §6 · p. 10
• In the reverse process, BlurDM removes noise and blur residuals to restore the image.Source: §6 · p. 10
• BlurDM generates a prior in existing deblurring networks' latent spaces and integrates it into decoder blocks through the Prior Fusion Module.Source: §6 · p. 10

NeurIPS Paper Checklist

• The checklist marks the abstract and introduction as accurately reflecting the paper's contributions and scope.Source: NeurIPS Paper Checklist · p. 14
• The checklist guidance says claims should match theoretical and experimental results and reflect expected generalization scope.Source: NeurIPS Paper Checklist · p. 14

2. Limitations

• The authors state that their work's limitations are discussed in Section 5.Source: §2 · p. 14
• The experiments report no error bars or statistical significance tests because the authors say these would be too computationally expensive.The authors point to consistent PSNR and SSIM improvements across four benchmark datasets and four deblurring models, and describe Table 1 results as stable gains.Source: §2 · p. 16
• Despite the lack of formal statistical analysis, the authors report consistent PSNR and SSIM improvements across GoPro, HIDE, RealBlur-J, and RealBlur-R and four deblurring models.They characterize the Table 1 gains as stable and clear.Source: §2 · p. 16

• The submission points to Appendix A.10 for its broader-impact discussion and says core method development does not use LLMs as an important, original, or non-standard component.Source: §10 · pp. 17, 20
• The work uses publicly available datasets and existing models, which the authors say are cited and used under respected licenses.Source: §10 · p. 18
• The authors report that their code, inference results, and model weights are well documented.Source: §10 · p. 19
• The paper reports no human-subject or crowdsourcing research and says no IRB approval was required.Source: §10 · p. 19

• Blur residuals accumulated across exposure intervals combine with the clean-image component to represent the fully blurred image.Source: §A.1 · p. 21
• Independent per-step Gaussian noises combine into a Gaussian term with zero mean and the derived aggregate variance.Source: §A.1 · p. 21
• The resulting one-step Gaussian transition is mathematically equivalent to the full forward process and more computationally efficient.It captures accumulated blur and noise in a single transition.Source: §A.1 · p. 21

A.2 ELBO and Optimization for BlurDM

• BlurDM’s ELBO-based objective learns blur-residual and noise estimators, while trajectory-level reconstruction supervision trains intermediate predictions without per-step blur-residual ground truth.Its terminal prior is a noise-perturbed blurred observation aligned with the forward marginal.Source: §A.2 · pp. 22–24

A.3 Deterministic Implicit Sampling for BlurDM

• The deterministic reverse sampler preserves consistency with the original forward-process distribution at every timestep.Source: §A.3 · p. 25

A.4 Theoretical justification of latent BlurDM.

• A first-order encoder expansion indicates that image-space blur residuals and stochastic noise project into latent space with the same coefficients.Source: §A.4 · p. 26

A.5 Training cost of the three-stage training strategy

• 8% more training time yields a 0.42 dB PSNR improvement for BlurDM 3 over Baseline 2 on GoPro.The reported comparison uses MIMO-UNet and the three-stage training strategy.Source: §A.5 · p. 26, Table 7 · p. 26

A.6 Performance comparison of the number of steps used in RDDM and BlurDM

• 32.28 dB is BlurDM’s peak PSNR at step 5, versus RDDM’s 32.08 dB at step 4; BlurDM’s best LPIPS is 0.0113 at step 10, versus RDDM’s 0.0121.Source: §A.6 · p. 26, Table 8 · p. 27

A.7 Visualizations of Blur Residual; A.8 Comparison between BlurDM and RDDM in Blur Residuals.

• As exposure time increases, GoPro images and residuals evolve; unlike RDDM’s scaled subtraction, BlurDM models nonlinear spatial accumulation of blur.Source: §A.7 · p. 26, Figure 8 · p. 27, §A.8 · p. 27

A.9 Deblurred Results on Real-world Datasets

• Qualitative RealBlur-J test comparisons show outputs from MIMO-UNet, Stripformer, FFTformer, and LoFormer with BlurDM and their baselines.Source: §A.9 · p. 27, Figure 10 · p. 29, Figure 11 · p. 30

A.10 Broader Impacts

• The authors identify potential benefits in autonomous driving, medical imaging, and historical-media restoration, while noting privacy risks from reconstructing blurred faces or sensitive content.They recommend responsible use with appropriate privacy safeguards and user consent.Source: §A.10 · p. 27