Source-linked AI summary
RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priors
Rui-Qi Wu, Zheng-Peng Duan, Chun-Le Guo, Zhi Chai, Chong-Yi Li
TL;DR
Real image dehazing lacks sufficiently aligned paired data and robust priors for diverse real scenes. The paper introduces realistic multi-factor haze synthesis and RIDCP, which uses VQGAN codebook priors with controllable matching. Extensive experiments report superior real-image dehazing performance, while remaining domain gaps can still cause incorrect prior matching and low saturation.
Problem
Real dehazing is limited by scarce large-scale aligned pairs, synthetic data that omits degradation factors, and priors that do not cover diverse real scenes.
Method
RIDCP combines a phenomenological multi-factor degradation pipeline with VQGAN-derived high-quality codebook priors and controllable HQP matching.
Results
Extensive experiments show the proposed paradigm's superiority, and RIDCP effectively processes wild hazy images while producing adjustable results.
Takeaways & Limitations
The paper presents a real-scene dehazing paradigm that couples more realistic paired-data synthesis with high-quality priors and user-controllable enhancement.
Takeaways & Limitations
Even after the synthesis pipeline reduces the domain gap, it cannot cover all real-world hazy conditions, making correct HQP matching difficult and sometimes lowering color saturation.
Abstract
from arXiv · showhide
Existing dehazing approaches struggle to process real-world hazy images owing to the lack of paired real data and robust priors. In this work, we present a new paradigm for real image dehazing from the perspectives of synthesizing more realistic hazy data and introducing more robust priors into the network. Specifically, (1) instead of adopting the de facto physical scattering model, we rethink the degradation of real hazy images and propose a phenomenological pipeline considering diverse degradation types. (2) We propose a Real Image Dehazing network via high-quality Codebook Priors (RIDCP). Firstly, a VQGAN is pre-trained on a large-scale high-quality dataset to obtain the discrete codebook, encapsulating high-quality priors (HQPs). After replacing the negative effects brought by haze with HQPs, the decoder equipped with a novel normalized feature alignment module can effectively utilize high-quality features and produce clean results. However, although our degradation pipeline drastically mitigates the domain gap between synthetic and real data, it is still intractable to avoid it, which challenges HQPs matching in the wild. Thus, we re-calculate the distance when matching the features to the HQPs by a controllable matching operation, which facilitates finding better counterparts. We provide a recommendation to control the matching based on an explainable solution. Users can also flexibly adjust the enhancement degree as per their preference. Extensive experiments verify the effectiveness of our data synthesis pipeline and the superior performance of RIDCP in real image dehazing.
1. Introduction
Real image dehazing is limited by scarce aligned real pairs, diverse degradations, and insufficiently robust priors. RIDCP addresses these issues with phenomenological haze synthesis and controllable high-quality codebook priors.
- Real-world dehazing is ill-posed, while handcrafted statistical priors fail to cover diverse scenes.
- Large-scale, perfectly aligned real hazy-clean pairs are difficult to collect, so models mainly train on synthetic data based on a limited physical model.Available paired real datasets are small, insufficiently diverse, and unlike wild hazy images.
- A VQGAN trained on high-quality images supplies fixed high-quality priors, while RIDCP learns to match and fuse them for dehazing.The network includes an encoder, a new decoder, and normalized feature alignment for better feature fusion.
- Controllable HQPs matching replaces nearest-neighbor matching with weighted feature-to-prior distances, enabling adjustable enhancement and an explainable control parameter.The parameter can shift HQP activation distributions and users can adjust the dehazing result.
- RIDCP is reported to process wild hazy images effectively and generate adjustable results, unlike the cited DAD and PSD methods.
- RIDCP introduces a phenomenological degradation pipeline that simulates multiple factors in real hazy images.
2. Related Work
Related work spans handcrafted priors, deep networks, real-haze data generation, and discrete codebooks. RIDCP combines realistic degradation modeling with high-quality codebook priors to target real-scene generalization.
- Image Dehazing: Handcrafted dehazing priors can fail in diverse scenes, such as sky regions where the dark channel assumption is unavailable.RIDCP instead pre-trains a discrete codebook on large-scale high-quality data.
- Image Dehazing: Deep methods estimate physical degradation parameters or directly predict haze-free images, but their performance drops on real-world data.
- Real Image Dehazing: GAN-based approaches generate haze-domain data or unpaired training frameworks, but generated artifacts can harm model training.
- Codebook Priors: VQ-VAE and VQGAN learn discrete codebooks, with VQGAN improving perceptual reconstruction through adversarial supervision.Discrete codebooks have also supported face restoration and super-resolution.
3. Data Preparation for Real Image Dehazing
The proposed preparation pipeline synthesizes paired hazy data by combining physical haze formation with low-light, colorful-haze, noise, depth, and JPEG degradation factors.
- The pipeline reformulates haze generation as P(J(x), t(x), A) and incorporates multiple degradation factors to reduce the synthetic-real domain gap.
- Equation (2) applies brightness adjustment and Gaussian noise before haze formation, simulating poor lighting in hazy weather.The brightness factor γ ranges from 1.5 to 3.0.
- Depth estimation supplies the transmission map, while β ∈[0.3, 1.5] controls haze density.
- A three-channel atmosphere-light bias ∆A ∈[−0.025, 0.025] simulates colorful haze, with A ranging from 0.25 to 1.0.
- JPEG compression is applied to final outputs because dehazing can amplify JPEG artifacts.
- Five hundred clean images form the paired dataset, with hazy samples generated on the fly; low light and JPEG compression each occur with 50% probability.
4. Methodology
RIDCP combines a VQGAN-derived high-quality codebook with feature alignment and controllable matching to dehaze real images despite residual synthetic–real domain gaps.
- Latent Codebook for High-quality Priors: RIDCP pre-trains VQGAN on high-quality images to obtain a discrete codebook and decoder, then trains the dehazing network on synthesized hazy data.The framework uses the codebook as high-quality priors during real-image dehazing.
- Controllable HQPs Matching Operation: Voronoi simulations illustrate that distance recalculation moves features from originally unfavorable cells to cells associated with better HQPs.The method aims to address divergent code activation caused by the remaining synthetic–real domain gap.
- Image Dehazing via Feature Matching: The pre-trained VQGAN can remove thin haze and recover vivid color without fine-tuning, but incorrect code matching and quantization information loss introduce distorted textures.These limitations motivate learning an encoder and decoder specialized for dehazing.
- Latent Codebook for High-quality Priors: Nearest-neighbor matching maps encoder features to codebook priors, and the VQGAN decoder reconstructs the resulting representation.The matching operation selects the codebook vector minimizing Euclidean distance for each latent feature.
- Image Dehazing via Feature Matching: The decoder uses deformable-convolution warping and normalized addition to align VQGAN features with dehazing features and balance their contributions.The normalization addresses unstable relative magnitudes between warped VQGAN features and dehazing features.
- Controllable HQPs Matching Operation: CHM recalculates feature-to-codebook distances with frequency-based weights controlled by α, shifting assignments toward better high-quality priors.The weighting function is designed to preserve unchanged frequencies while making adjustment strength monotonic in α.
- Controllable HQPs Matching Operation: The recommended α is obtained by minimizing forward Kullback–Leibler divergence between clean-image and hazy-image code activation distributions.This provides an explainable control strategy for real-domain adaptation.
5. Experiments
Experiments evaluate RIDCP on real hazy datasets, visual comparisons, quantitative metrics, user preference, and component ablations. Results indicate strong perceptual dehazing, adjustable enhancement, and benefits from the proposed data and feature-processing components.
- Datasets: RTTS contains over 4,000 real hazy images, while Fattal’s dataset provides 31 classic real hazy cases for visual comparison.
- Quantitative and qualitative comparison: RIDCP produces the best perceptual results in brightness, colorfulness, and haze residue among the compared methods.PSD produces bright results but limited dehazing, whereas DAD removes haze but shows color bias and dark tone.
- Adjustment-parameter ablation: Figure 7 varies enhancement from α = −60.0 to α = 60.0, with α = 21.25 recommended for the most natural results.The controllable parameter provides a linear relationship between α and image enhancement effect.
- NFA ablation: NFA preserves useful HQP-reconstructed information while reducing distorted textures caused by feature matching.Without fusion, direct addition, or warping variants leave dark regions, thin haze, or non-homogeneous fog residues in comparisons.
- Data-generation ablation: Dehamer and MSBDN trained on the proposed synthetic data reduce haze and increase brightness but still lag RIDCP in color recovery.This comparison supports the contribution of HQPs and the real-domain adaptation strategy.
6. Discussion
The discussion reports broad effectiveness of the proposed real-image dehazing paradigm while identifying unresolved limitations in haze handling, enhancement-based restoration, and evaluation.
- Conclusion: The phenomenological degradation pipeline synthesizes more realistic hazy data and achieves significant gains in haze removal.RIDCP progressively leverages VQGAN high-quality priors to form the proposed real-image dehazing network.
- Limitations: RIDCP and existing dehazing methods cannot process non-homogeneous haze well.
- Limitations: Enhancement-based dehazing is limited for extremely dense haze, motivating generative ability for recovery.
- Limitations: Fair quantitative benchmarking remains difficult because robust metrics for dehazing quality are still needed.
A.1. Detailed Architecture
RIDCP uses an encoder–decoder architecture whose encoder downsamples through convolution and residual layers, while the decoder upsamples and reconstructs through convolution and residual layers.
- Encoder: Each encoder layer contains a stride-2 downsampling convolution followed by two residual layers.
- Decoder: Each decoder layer contains upsampling, convolution, and two residual layers.
A.2. Traning Objectives
VQGAN training combines reconstruction, perceptual, adversarial, codebook, and semantic-guided objectives, while RIDCP uses pretrained VQGAN features to identify codes for hazy inputs.
- VQGAN objectives: VQGAN reconstruction loss combines L1 reconstruction, perceptual loss, and adversarial loss.
- Codebook objective: Codebook optimization includes feature commitment terms and semantic-guided regularization using a convolutional layer and pretrained VGG19 features.The stated coefficients are β = 0.25 and γ = 0.1.
- VQGAN objectives: The total VQGAN loss is defined as the sum of reconstruction and codebook losses.
- RIDCP objective: For a hazy image, RIDCP’s encoder extracts features that are used with pretrained VQGAN guidance to find the corresponding code.
B. Experiments Results
Because real image dehazing lacks ground truth, quantitative metrics may not reflect true algorithm performance. The paper therefore emphasizes qualitative results to assess RIDCP and its key components.
- Real image dehazing lacks ground truth, making quantitative metrics difficult to interpret.
- The paper provides extensive qualitative results to demonstrate RIDCP’s performance.
- Qualitative evaluation also examines the effectiveness of each key component.
B.1. More Visual Results
RIDCP performs well on the authors’ daytime and nighttime scenes and maintains stable dehazing ability on densely hazy, color-biased RTTS images.
- RIDCP performs well on both daytime and nighttime scenes in the authors’ captured data.
- On RTTS, RIDCP achieves satisfactory performance across visual comparisons with state-of-the-art methods.
- RIDCP maintains stable dehazing ability under dense haze and heavy color bias.
B.2. Ablation Study
The ablations indicate that HQPs improve brightness and reduce color bias, while the phenomenological degradation pipeline significantly improves dehazing capability. The method may also extend to industrial applications and other dehazing algorithms.
- Ablation Study: HQPs help the network generate results with better brightness and lower color bias.
- Ablation Study: The phenomenological degradation pipeline brings significant improvement in dehazing capability.
- Ablation Study: The pipeline can generally boost the performance of dehazing algorithms beyond RIDCP.
- Ablation Study: RIDCP performs well on real-world hazy scenes and could potentially support automatic driving and computational photography.