Source-linked AI summary
Exploiting Diffusion Prior for Real-World Image Dehazing with Unpaired Training
Yunwei Lan, Zhigao Cui, Chang Liu, Jialun Peng, Nian Wang, Xin Luo, Dong Liu
TL;DR
Real-world image dehazing is limited by weak feature representation and insufficient use of physical priors. Diff-Dehazer combines diffusion priors, physical guidance, and text-aware guidance in an unpaired CycleGAN framework, outperforming other methods across real-world datasets while sometimes misaligning under severe haze.
Problem
Real-world dehazing methods have limited generalization because they lack sufficient feature representation and use of real-world physical priors.
Method
Diff-Dehazer integrates diffusion priors, physical-aware guidance, and text-aware guidance into an unpaired, cycle-consistent dehazing framework.
Results
The method outperforms other methods across multiple real-world datasets, producing more realistic images with richer details and textures.
Takeaways & Limitations
Diff-Dehazer demonstrates strong generalization for real-world hazy images with varied haze types and densities without retraining on OHAZE.
Takeaways & Limitations
The diffusion-based fine-tuning paradigm may produce images misaligned with severely hazy inputs, although performance remains stable in most cases.
Abstract
from arXiv · showhide
Unpaired training has been verified as one of the most effective paradigms for real scene dehazing by learning from unpaired real-world hazy and clear images. Although numerous studies have been proposed, current methods demonstrate limited generalization for various real scenes due to limited feature representation and insufficient use of real-world prior. Inspired by the strong generative capabilities of diffusion models in producing both hazy and clear images, we exploit diffusion prior for real-world image dehazing, and propose an unpaired framework named Diff-Dehazer. Specifically, we leverage diffusion prior as bijective mapping learners within the CycleGAN, a classic unpaired learning framework. Considering that physical priors contain pivotal statistics information of real-world data, we further excavate real-world knowledge by integrating physical priors into our framework. Furthermore, we introduce a new perspective for adequately leveraging the representation ability of diffusion models by removing degradation in image and text modalities, so as to improve the dehazing effect. Extensive experiments on multiple real-world datasets demonstrate the superior performance of our method. Our code https://github.com/ywxjm/Diff-Dehazer.
Introduction
Real-world image dehazing is limited by scene-dependent physical priors, paired-data requirements, and insufficient representation from limited training images. Diff-Dehazer addresses these challenges by combining unpaired training with diffusion priors, physical-prior guidance, and text-modality semantics.
- Motivation: Hazy conditions degrade image quality, cause information loss, and restrict downstream tasks such as object detection.Image dehazing aims to restore clear images from hazy inputs.
- Limitations of Existing Methods: Hand-crafted physical priors can produce over-saturated results because they are not universally compatible with all scenes.Conventional methods estimate atmospheric light and transmission to reverse the Atmospheric Scattering Model.
- Unpaired Training: Unpaired training seeks to learn mappings between real-world hazy and clear images while imposing structure consistency despite misaligned information.This paradigm is introduced to overcome the bottleneck of paired training.
- Limitations of Existing Methods: Existing unpaired methods can suffer sub-optimal performance because limited training images constrain effective representation, motivating diffusion-model pre-training and fine-tuning.CycleDehaze, D4, and ODCR are cited as methods with promising real-world dehazing performance.
- Diff-Dehazer: Diff-Dehazer combines diffusion prior with unpaired training and incorporates physical properties of real-world hazy scenes to improve dehazing.The framework is presented as an effective paradigm for real-world image dehazing.
- Contributions: The method uses pre-trained stable diffusion in CycleGAN, introduces Physics-Aware Guidance, and leverages high-level semantics from text modality.These components are listed among the paper’s four contributions.
Related Works
Related work progresses from physical-prior and deep-learning dehazing methods to weakly supervised and unpaired frameworks, while diffusion models have gained traction in vision. However, existing approaches remain limited for real-world hazy images because they rely heavily on synthetic data and insufficiently exploit real-world information.
- Physical-prior methods: Early methods use natural-image physical priors with the atmospheric scattering model, including DCP for transmission and atmospheric-light estimation and BCCR for transmission optimization.DCP estimates the transmission map and atmospheric light using the dark channel prior, while BCCR combines boundary constraints with L1-norm contextual regularization.
- Deep-learning methods: Deep-learning methods include circular-learning, Transformer-based, and other network designs, but struggle in real-world scenarios because they depend on synthetic paired data.Examples include C2PNet and methods based on the Transformer architecture.
- Weakly supervised and unpaired methods: Weakly supervised and unpaired approaches such as RefineDNet, D4, and ODCR improve dehazing, yet often fail to demonstrate effectiveness on real-world images.RefineDNet uses a two-stage framework, D4 builds on CycleGAN to decompose the atmospheric scattering model, and ODCR introduces orthogonal decoupling contrastive regularization.
- Diffusion models: Diffusion models have achieved strong results in vision, including text-to-image and conditional image generation, while their application to low-level vision is expanding.The cited applications include StableSR and other diffusion-based low-level vision methods.
- Diffusion models: Diffusion-feature methods still insufficiently address image dehazing because they rely on synthetic data and neglect vital real-world information.This limitation prevents effective handling of real-world hazy images.
Method
Diff-Dehazer builds an unpaired hazing-dehazing CycleGAN framework using diffusion backbones, cycle-consistent training, and a dehazing-only inference path. It further enhances dehazing with text-aware and physics-aware guidance based on refined captions and real-world physical priors.
- Cycle-consistent framework: Diff-Dehazer forms hazing-dehazing cycles between real hazy and clear images and trains them with cycle-consistent constraints.A hazy image is transformed into a fake clear image and back into a cycle hazy image, with the clear-image path applied in reverse.
- Cycle-consistent framework: After training, inference obtains a clear image from a hazy image using only the dehazing process.The inference path corresponds to the blue arrows in the method overview.
- Diffusion backbones: SD Turbo v2.1 supplies the hazing and dehazing backbones because its distilled diffusion model generates many high-quality images in one step.The backbone contains a VAE encoder, VAE decoder, and U-Net.
- Text-Aware Guidance: Text-Aware Guidance removes haze-related terms from image captions and uses positive and negative prompts through classifier-free guidance.The method integrates textual information to enhance dehazing in the text modality.
- Physics-Aware Guidance: Physics-Aware Guidance integrates DCP and BCCR, reconstructs hazy images with the atmospheric scattering model, and introduces a physical loss.DCP- and BCCR-dehazed images are treated as clear images for hazy-image reconstruction, while a fusion model refines their complementary information.
Loss Function
The proposed Diff-Dehazer follows CycleGAN by combining cycle consistency, physical-prior, identity, and GAN losses. Its physical loss reconstructs hazy images with the ASM and measures reconstruction using L1 and LPIPS distances.
- Loss Function: The training objective combines cycle consistency, physical-prior, identity, and GAN losses, weighted by λ_phy, λ_idt, and λ_GAN.The cycle, identity, and GAN losses follow CycleGAN and are detailed in the supplementary materials.
- Loss Function: The physical loss uses the ASM to reconstruct a hazy image as I_phy = J_ref t_ref + A(1 − t_ref).I denotes the overall notation for real and fake hazy images.
- Loss Function: The reconstruction term L_rec combines L1 distance and LPIPS distance.LPIPS is attributed to Zhang et al. (2018).
Experiments and Discussions
Experiments across multiple real-world dehazing datasets show that Diff-Dehazer produces more natural, detailed, and faithful results than competing methods, with strong quantitative performance and generalization. Ablations further support the contributions of its network components, text prompts, and physical priors.
- Experimental Setup: Diff-Dehazer is evaluated qualitatively and quantitatively on URHI, RTTS, Haze2020, OHAZE, NHAZE, and the Fattal dataset.The comparisons include physical-prior, weakly supervised, and fully supervised methods, using full-reference metrics when ground truth is available.
- RTTS and Haze2020: Diff-Dehazer produces more realistic, natural, detailed, and high-contrast images than competing methods on real-world hazy images.Qualitative comparisons report that competing unpaired methods and all-in-one restoration methods are less effective, while Diff-Dehazer restores textures and preserves fidelity.
- Quantitative Results: Diff-Dehazer achieves superior RTTS FID, NIQE, and MUSIQ scores, ranks lower on CLIPIQA, and surpasses other methods across various Haze2020 metrics.The lower RTTS CLIPIQA ranking is attributed to physical-prior integration intended to constrain diffusion stochasticity and preserve natural outputs.
- OHAZE: On OHAZE, Diff-Dehazer achieves superior visualization and quantitative metrics without retraining, indicating generalization across real-world haze types and densities.The method produces high-quality results while using the same trained network for evaluation on this dataset.
- Ablation Studies: Ablations show that the full model performs best, positive prompts reduce mismatched details, negative prompts further improve dehazing, and jointly using DCP and BCCR yields superior results.The ablations examine the backbone, PAG, TAG, prompt settings, guidance scale, and physical-prior configurations.
Conclusion
Diff-Dehazer is an unpaired real-world image dehazing framework that exploits stable diffusion and physical priors derived from natural images. It also performs dehazing in text and image modalities using enriched high-level semantics, with experiments validating its superiority.
- Conclusion: Diff-Dehazer exploits stable diffusion and physical priors derived from natural images within an unpaired framework for real-world image dehazing.The framework is explicitly named Diff-Dehazer.
- Conclusion: The method performs dehazing in text and image modalities by leveraging enriched high-level semantics.The multimodal design is intended to obtain more qualified results.
- Conclusion: Extensive experiments validated the superiority of Diff-Dehazer.