Source-linked AI summary
Unsupervised Domain-Specific Deblurring via Disentangled Representations
Boyu Lu, Jun-Cheng Chen, Rama Chellappa
TL;DR
Domain-specific single-image deblurring is challenging because blur restoration is ill-posed, paired data is costly, and generic or strongly supervised methods may generalize poorly to real specialized images. The paper addresses this with an unsupervised disentangled representation framework that separates content from blur and regularizes the learned mappings. Experiments on face and text deblurring report competitive performance against state-of-the-art methods, with evaluations also covering semantic recovery.
Problem
Domain-specific deblurring lacks reliable solutions because generic priors do not generalize well, paired data is expensive, and strong supervision may generalize poorly to real images.
Method
The method disentangles content and blur features, regularizes blur attributes with KL divergence, and combines blurring, adversarial, cycle-consistency, and perceptual losses for unpaired training.
Results
Experiments on face and text deblurring achieve competitive performance compared with state-of-the-art methods and evaluate semantic recovery using face verification and OCR.
Takeaways & Limitations
The framework provides an unsupervised domain-specific deblurring approach that produces promising quantitative and visual results across face and text tasks.
Takeaways & Limitations
For text deblurring, the perceptual loss is omitted because it does not help and can sometimes hurt performance, possibly due to text-image intensity distributions differing from ImageNet images.
Abstract
from arXiv · showhide
Image deblurring aims to restore the latent sharp images from the corresponding blurred ones. In this paper, we present an unsupervised method for domain-specific single-image deblurring based on disentangled representations. The disentanglement is achieved by splitting the content and blur features in a blurred image using content encoders and blur encoders. We enforce a KL divergence loss to regularize the distribution range of extracted blur attributes such that little content information is contained. Meanwhile, to handle the unpaired training data, a blurring branch and the cycle-consistency loss are added to guarantee that the content structures of the deblurred results match the original images. We also add an adversarial loss on deblurred results to generate visually realistic images and a perceptual loss to further mitigate the artifacts. We perform extensive experiments on the tasks of face and text deblurring using both synthetic datasets and real images, and achieve improved results compared to recent state-of-the-art deblurring methods.
1. Introduction
Domain-specific deblurring remains difficult because generic priors and paired-data learning do not handle specialized blur or real images reliably. The paper proposes an unsupervised disentangled framework and reports competitive results on face and text deblurring, including semantic-recovery evaluations.
- Motivation: Generic deblurring priors struggle to generalize from natural images to domain-specific face and text images, while existing specialized methods handle limited blur types and often require examples.These limitations motivate domain-specific deblurring methods for applications affected by blur.
- Motivation: Paired training data is expensive to collect, generated blur does not cover all real-world variants, and strong supervision can reduce generalization to real images.Learning-based methods can model complex blur, but their data requirements and supervision remain problematic.
- Proposed Method: The framework combines content and blur encoders with blurred and sharp-image generators, while adversarial, KL, cycle-consistency, and perceptual losses constrain the learned mappings.The overview distinguishes blurring and deblurring branches and assigns the losses to their corresponding outputs.
- Proposed Method: The method disentangles content and blur features, regularizes blur attributes with KL divergence, and uses shared content representations to support unsupervised domain-specific deblurring.A blurring generator, adversarial and cycle-consistency losses, and perceptual loss further support realistic outputs, content preservation, and artifact reduction.
- Results: Experiments on face and text deblurring achieve competitive performance against state-of-the-art methods and also evaluate recovery of semantic information through face verification and OCR.The introduction reports both deblurring comparisons and downstream semantic evaluations.
2. Related Works
Prior work spans hand-designed priors, CNN-based deblurring, domain-specific models, and disentangled representations. These approaches address different aspects of the ill-posed problem but leave challenges in generalization across specialized domains and factor separation.
- Generic Methods: Single-image blind deblurring is highly ill-posed, motivating natural-image and kernel priors that regularize the latent sharp-image solution space.Reported priors include heavy-tailed gradients, sparse kernels, l0 gradients, normalized sparsity, and dark channels.
- Generic Methods: CNN-based methods estimate blur kernels or deconvolution filters, combining learned representations with conventional deblurring formulations.The cited approaches operate in image or frequency domains to predict kernels or filters.
- Domain-Specific Methods: Generic natural-image methods are difficult to generalize to face and text images, prompting domain-specific priors, end-to-end text models, and exemplar-guided face deblurring.These methods target specialized image domains rather than relying solely on generic natural-image assumptions.
- Disentangled Representations: Disentangled-representation research separates factors such as pose and identity or identity and attributes for face recognition and synthesis.These precedents motivate using factor separation for image deblurring.
3. Proposed Method
The framework separates content and blur representations, then uses adversarial, cycle-consistency, perceptual, and KL-based objectives to train with unpaired sharp and blurred images. At test time, content and blur features from a blurred input are passed to the deblurring generator.
- Framework: Content encoders, a blur encoder, generators, and discriminators form the framework for translating between unpaired sharp and blurred image domains.The sharp generator combines blurred-image content and blur features, while the blur generator combines sharp-image content and blur features.
- Content–blur disentanglement: Shared weights in the content encoders guide blurred-image content extraction, while the KL loss regularizes blur features toward N(0, 1) to suppress content information.The blur encoder is also trained through generation of blurred versions of sharp images, discouraging it from encoding the blurred input's content.
- Adversarial loss: Adversarial losses make generated images resemble real samples in both domains, with discriminators distinguishing generated images from real sharp or blurred images.The sharp-domain discriminator distinguishes deblurred images from real sharp images, while the generator minimizes the loss to produce realistic sharp results.
- Cycle-consistency loss: Cycle consistency requires deblurred images to reblur into the original blurred samples and blurred translations to return to the original sharp domain.This constraint addresses the absence of paired supervision by preserving content across the two translation directions.
- Perceptual loss: A perceptual loss uses conv3,3 features from ImageNet-pretrained VGG-19 to reduce artifacts, but it is omitted for text deblurring because it can hurt performance.The loss is balanced carefully because perceptual features are sensitive to blur, and the text-image intensity distribution differs from ImageNet images.
- Testing and objective: At test time, the blurring branch is removed and the deblurring generator produces a sharp image from the test blur's extracted content and blur features.The overall objective combines adversarial, KL, cycle-consistency, and perceptual losses with empirically selected weights.
4. Experimental Results
Experiments evaluate the method on face and text deblurring using synthetic and real images, standard restoration metrics, perceptual similarity, OCR, and face verification. Ablations and comparisons indicate benefits from cycle consistency, KL-regularized disentanglement, and perceptual loss, with competitive results against state-of-the-art methods.
- Datasets and Metrics: Experiments use CelebA, BMVC Text, and CFP, with PSNR, SSIM, feature distance, OCR character error rate, and face-verification accuracy as evaluation measures.Natural-image generalization is also tested qualitatively.
- Ablation Study: Ablations show that the blurring branch improves deblurring, KL regularization enables better content–blur separation, and perceptual loss reduces visible artifacts.The blurring branch especially improves perceptual distance, while perceptual loss improves visual realism.
- Face Results: The method outperforms other unsupervised algorithms on CelebA and achieves results comparable to supervised face deblurring on conventional and perceptual metrics.The comparison uses PSNR, SSIM, and VGG-Face distance on CelebA.
- Face Results: The proposed method improves blurred-image face verification, outperforms CycleGAN on both protocols, and is comparable to supervised deblurring methods.Evaluation covers frontal-to-frontal and frontal-to-profile protocols on CFP.
- Face Results: On real blurred faces, the method recovers sharp facial structure and background textures while competing methods often under-deblur, over-deblur, or introduce artifacts.The comparison is shown in Figure 5.
- Text Results: On BMVC Text, the method matches supervised text deblurring on perceptual distance and OCR error rate despite substantially worse PSNR.The method sometimes changes text fonts because it uses unpaired training data.
5. Conclusions
The proposed unsupervised method disentangles content and blur features, uses regularization and consistency losses, and performs promisingly on face and text deblurring against state-of-the-art approaches.
- The method disentangles content and blur features, using KL divergence to discourage content information in blur representations.
- A blurring branch and cycle-consistency loss preserve the original images' content structure during unsupervised training.
- Adversarial and perceptual losses help produce realistic deblurred images while reducing unrealistic artifacts.
- Experiments on face and text deblurring show promising quantitative and visual performance compared with other state-of-the-art approaches.