Source-linked AI summary
Controllable blind deblurring with diffusion models
Imane Si Salah, Emile Cribelier, Thomas Veit, Wolf Hauser, Arthur Leclaire
TL;DR
Blind deblurring must recover plausible high-frequency details from degraded images without a known blur kernel, while retaining fidelity to the input. SuperSharpen uses latent diffusion conditioned on the degraded image and a user-adjustable blur measure, and compares ControlNet conditioning with full finetuning. Across synthetic and real-world blur, finetuning provides better fidelity with fewer hallucinated details, while the blur measure controls restoration strength.
Problem
Blind deblurring seeks to restore high-frequency information from unknown blur, but generative restoration must balance photo-realistic detail against fidelity to the degraded input.
Method
SuperSharpen is a latent-diffusion blind deblurring method that conditions on the degraded image and blur measure, comparing ControlNet conditioning with full diffusion-denoiser finetuning.
Results
Finetuning achieves better fidelity to the degraded input with fewer hallucinated details, while SupS achieves detail generation and fidelity on synthetic and real-world blur.
Takeaways & Limitations
Users can adjust restoration strength through the blur measure, and the method generalizes to real-world blur from optical imperfections and RAW processing.
Takeaways & Limitations
The method is restricted to spatially uniform blur and may produce artifacts on sensitive content such as text and faces.
Abstract
from arXiv · showhide
Image acquisition with a camera involves several degradations due to the optical system, sensor, or low-level processing steps. We address blind deblurring in professional photography: we aim to invert unknown isotropic blur without knowledge of the degradation kernel.For such inverse problems,where some high-frequency information is lost, it is challenging to use generative models to produce details that are both photo-realistic and faithful to the input. We propose SuperSharpen, a diffusion-based blind deblurring method offering explicit control over restoration strength through a blur measure. We compare two conditioning strategies: a ControlNet-style adapter on a frozen backbone, and full finetuning of the diffusion prior. Our experiments show that finetuning achieves better fidelity with fewer hallucinated details. We validate our approach on synthetic and real-world blur, demonstrating improved perceptual quality and controllable restoration strength.
1. INTRODUCTION
SuperSharpen targets blind restoration of mild, spatially invariant blur while addressing the difficulty of generating details that remain faithful to degraded images. It introduces blur-measure conditioning and compares ControlNet-based conditioning with full diffusion-backbone finetuning.
- Mild spatially invariant blur from defocus and denoising or ISP processing is addressed, while motion and non-uniform blur are excluded.
- Pixel-wise learning methods can produce overly smoothed outputs, while real-world performance can suffer from distribution shift between training and deployment data.
- The blur measure provides explicit inference-time control over restoration strength by representing the desired amount of blur removal.
- SuperSharpen is a latent-diffusion blind deblurring model that conditions restoration on the degraded image and a blur measure.
- The blur measure is computed by fitting an isotropic Gaussian to averaged impulse responses from multiple spatial locations and using its standard deviation.
2. BLIND DEBLURRING NETWORK
SuperSharpen conditions latent diffusion on a degraded image to perform blind deblurring, comparing ControlNet conditioning with full backbone finetuning. It also introduces a blur measure that lets users adjust restoration strength during inference.
- Experimental comparison: SupS-FT achieves the best LPIPS, while SupS-CN achieves the highest no-reference scores; SupS-FT has lower PSNR and SSIM than PASD.The comparison reports PSNR, SSIM, and LPIPS values for the synthetic-blur evaluation and attributes the fidelity difference to dependence on a frozen prior.
- Experimental comparison: The SupS models produce sharp, natural results on synthetic and real blur, with SupS-FT yielding more faithful details and fewer high-frequency artifacts than competing approaches.On real blurry images, the models balance detail generation and fidelity to the input; on synthetic blur, SupS-FT produces more consistent textures than SupS-CN.
- Diffusion model conditioning: SuperSharpen conditions a Stable Diffusion backbone on the degraded image latent and noisy latent to generate a restored image.The degraded image is encoded into a latent representation and reinjected during reverse diffusion.
- Diffusion model conditioning: SupS-CN trains a ControlNet adapter while keeping the pretrained diffusion backbone frozen.The conditioning branch concatenates the degraded and noisy latents and injects features through residual connections.
- Diffusion model conditioning: SupS-FT directly integrates the conditioning input into the diffusion backbone and finetunes the entire diffusion denoiser.Its U-Net is extended to accept the concatenation of the degraded and noisy latents.
- Blur measure estimation: The blur measure is estimated during training and supplied as a user-adjustable conditioning signal at inference.It is derived by fitting an isotropic Gaussian kernel to the effective impulse response of the degradation pipeline; higher values produce stronger detail generation.
3. EXPERIMENTS
Experiments compare two conditioning strategies for SuperSharpen on synthetic and real spatially uniform blur. Full finetuning provides stronger fidelity, while blur measure values control restoration strength on real images.
- Experimental setup: SupS-FT finetunes the SD2.1 U-Net, whereas SupS-CN freezes it and trains ControlNet adapter layers.Both methods additionally condition diffusion on the degraded input and blur measure.
- Experimental setup: Training uses LSDIR images with paired degradations generated by a Real-ESRGAN-like pipeline and blur-measure computation.Gaussian-blur augmentation is reduced to 10% of samples later in training to encourage sharp outputs.
- Synthetic evaluation: Synthetic evaluation uses Urban100 images with isotropic Gaussian blur and additive Gaussian noise, comparing SuperSharpen with DiffBIR, PASD, SeeSR, and ResShift.The Gaussian blur standard deviation ranges from 0.7 to 7.5, and noise standard deviation ranges from 0 to 7.
- Synthetic evaluation: SupS-FT achieves the best LPIPS, while PASD outperforms it on PSNR and SSIM; SupS-CN obtains the highest no-reference scores.The authors associate finetuning with better fidelity to the degraded input than ControlNet because it depends less on a frozen prior.
- Synthetic evaluation: Qualitatively, SupS-FT produces textures more consistent with HQ images and fewer high-frequency artifacts than DiffBIR.SupS-CN reaches similar sharpness but less accurate textures.
- Real-data evaluation: On real Adobe5K crops without sharp ground truth, SuperSharpen generates details while preserving input fidelity, unlike methods that remain blurry or hallucinate unfaithful textures.The benchmark targets approximately spatially uniform blur from camera capture and RAW processing.
- Real-data evaluation: Higher BM values produce stronger correction and sharper real-image outputs, allowing users to adjust the balance between fidelity and generated content.The perceptually optimal BM need not equal the physical blur-kernel size.
4. CONCLUSION
SuperSharpen is a diffusion-based blind deblurring method with controllable restoration strength through a blur measure. Finetuning improves fidelity relative to ControlNet, while the method remains limited to spatially uniform blur and can produce artifacts or fail to preserve identity at near-zero BM.
- Conclusion: SuperSharpen performs blind deblurring with restoration strength controlled by a blur measure.The method is evaluated on real-world blur from optical imperfections and RAW processing.
- Conclusion: Finetuning provides better fidelity to the degraded input, while ControlNet produces equally sharp but less faithful results.
- Conclusion: The method is restricted to spatially uniform blur and may produce artifacts on sensitive content such as text and faces.Identity is not yet preserved when the blur measure is close to zero.