Source-linked AI summary
UCL-Dehaze: Towards Real-world Image Dehazing via Unsupervised Contrastive Learning
Yongzhen Wang, Xuefeng Yan, Fu Lee Wang, Haoran Xie, Wenhan Yang, Mingqiang Wei, Jing Qin
TL;DR
Real-world dehazing lacks practical paired hazy/clean training data, while synthetic data create domain-shift concerns. UCL-Dehaze addresses this with unsupervised contrastive and adversarial learning over unpaired real-world images, and reports favorable results against state-of-the-art methods using 1,800 images. Its performance remains limited on synthetic datasets relative to supervised methods and on heavily hazy scenes.
Problem
Paired real-world hazy/clean images are difficult or impossible to obtain, while synthetic training data create a domain shift that degrades real-world dehazing.
Method
UCL-Dehaze uses adversarial training together with patch-wise and pixel-wise contrastive losses, treating clean images as positives and hazy images as negatives without paired training data.
Results
UCL-Dehaze performs favorably against state-of-the-art dehazing approaches, including in evaluations using 1,800 unpaired real-world training images.
Takeaways & Limitations
Unpaired real-world positive and negative images can support an unsupervised dehazing paradigm that performs favorably across quantitative, qualitative, and human-subjective evaluations.
Takeaways & Limitations
UCL-Dehaze has a slight performance gap on synthetic datasets relative to supervised state-of-the-art methods and is not very robust in heavily hazy scenes.
Abstract
from arXiv · showhide
While the wisdom of training an image dehazing model on synthetic hazy data can alleviate the difficulty of collecting real-world hazy/clean image pairs, it brings the well-known domain shift problem. From a different yet new perspective, this paper explores contrastive learning with an adversarial training effort to leverage unpaired real-world hazy and clean images, thus bridging the gap between synthetic and real-world haze is avoided. We propose an effective unsupervised contrastive learning paradigm for image dehazing, dubbed UCL-Dehaze. Unpaired real-world clean and hazy images are easily captured, and will serve as the important positive and negative samples respectively when training our UCL-Dehaze network. To train the network more effectively, we formulate a new self-contrastive perceptual loss function, which encourages the restored images to approach the positive samples and keep away from the negative samples in the embedding space. Besides the overall network architecture of UCL-Dehaze, adversarial training is utilized to align the distributions between the positive samples and the dehazed images. Compared with recent image dehazing works, UCL-Dehaze does not require paired data during training and utilizes unpaired positive/negative data to better enhance the dehazing performance. We conduct comprehensive experiments to evaluate our UCL-Dehaze and demonstrate its superiority over the state-of-the-arts, even only 1,800 unpaired real-world images are used to train our network. Source code has been available at https://github.com/yz-wang/UCL-Dehaze.
1 Introduction
Real-world image dehazing is difficult because paired hazy/clean data are impractical to obtain and synthetic training introduces domain shift. UCL-Dehaze instead uses unpaired real-world images with contrastive and adversarial learning.
- Haze reduces visibility and contrast in outdoor images, degrading traffic monitoring, object detection, and outdoor surveillance.
- Image dehazing is ill-posed, while hand-crafted priors may fail on arbitrary real-world images and require parameter tuning.
- Paired real-world hazy/clean data are difficult or impossible to obtain, and synthetic-to-real domain shift degrades real-world dehazing.
- UCL-Dehaze casts real-world dehazing as unsupervised image-to-image translation using contrastive learning and adversarial training.
- UCL-Dehaze performs favorably against state-of-the-art methods using only 1,800 unpaired real-world training images.
- The self-contrastive perceptual loss pulls restored images toward clean positives and pushes them away from hazy negatives in representation space.
2 Related Work
Prior-based methods depend on hand-crafted assumptions, while learning-based methods increasingly address dehazing through CNNs, GANs, semi-supervision, and unpaired translation. UCL-Dehaze differs by combining adversarial training with pixel-wise and patch-wise contrastive learning on unpaired real-world data.
- Prior-based: Prior-based dehazing methods restore images using hand-crafted assumptions derived from the atmospheric scattering model.
- Prior-based: The performance of prior-based methods is limited by the accuracy of their hand-crafted priors across real-world scenarios.
- Learning-based: Learning-based methods use CNNs or end-to-end networks to estimate atmospheric components or directly generate haze-free images.
- Unsupervised and semi-supervised methods: Semi-supervised methods combine synthetic data and real-world images but remain dependent on the quality and quantity of synthetic data.
- Unsupervised and semi-supervised methods: Unpaired image-to-image translation methods can generalize well to real-world scenes, but most use clean images as positives while underusing hazy images.
- Contrastive learning: UCL-Dehaze uses unpaired real-world hazy and clean images as negative and positive data, training pixel-wise and patch-wise with adversarial learning.
3 UCL-Dehaze
UCL-Dehaze frames real-world image dehazing as unsupervised image-to-image translation using unpaired hazy and clean images. Its generator combines adversarial training with patch-wise and pixel-wise contrastive objectives to restore haze-free images.
- Overview: UCL-Dehaze learns a mapping from real-world hazy images to haze-free images using unpaired data rather than paired synthetic images.The approach addresses the synthetic-to-real gap by training on unpaired real-world images.
- Network Architecture: The generator is a UNet-like encoder-decoder with nine residual blocks, skip connections, spectral normalization, and self-calibrated convolutions.The architecture uses downsampling, low-resolution residual processing, upsampling, and multi-scale feature extraction.
- Adversarial Training: Adversarial training uses a PatchGAN discriminator and LSGAN loss to distinguish real clean images from generator-produced restored images.The discriminator guides the generator toward more realistic dehazed outputs.
- Patch-wise Contrastive Learning: Patch-wise contrastive learning matches corresponding patches of the input and restored images while pushing other input patches away.Features are mapped to K-dimensional vectors and trained through an (N +1)-way classification objective.
- Pixel-wise Contrastive Learning: The pixel-wise self-contrastive perceptual loss uses VGG-16 features to pull restored images toward clean positives and away from hazy negatives.Clean and restored images form positive pairs, while hazy and restored images form negative pairs.
4 Experiments
Experiments evaluate UCL-Dehaze on synthetic and real-world settings, using unpaired real-world training images and comparisons with prior, supervised, and unsupervised methods. UCL-Dehaze achieves strong quantitative and qualitative results, improves downstream object detection, runs efficiently, and remains limited on heavily hazy scenes.
- Implementation Details: Training uses 1,800 real-world hazy images and 1,800 clean images from RESIDE, with the two sets unpaired.Hazy images come from RTTS and URHI, while clean images come from ITS.
- Comparison with State-of-the-arts: UCL-Dehaze achieves the highest PSNR and SSIM by a large margin on both the SOTS outdoor and HSTS synthetic datasets.The comparison includes 18 state-of-the-art dehazing methods.
- Comparison with State-of-the-arts: UCL-Dehaze outperforms other methods on CIEDE2000, Contrast gain, and σ, while ranking second on visibility.These metrics assess color difference, contrast, visibility, and saturation, respectively.
- Comparison with State-of-the-arts: Qualitative comparisons report clearer, more natural, and perceptually pleasing results on synthetic and real-world hazy images.Compared methods are described as producing color distortion, haze residuals, darkening, or unnaturally bright outputs in different cases.
- Ablation Study: The self-contrastive perceptual loss contributes especially strongly, producing a 3.3dB PSNR gain over variant V2 in the ablation study.The study reports that each component contributes to image dehazing.
- Efficiency and Downstream Tasks: UCL-Dehaze processes one HSTS hazy image in about 0.08s on average and outperforms other algorithms for object detection after dehazing.The associated detection experiment uses a pre-trained YOLOv4 detector.
- Limitations and Discussion: On synthetic datasets, UCL-Dehaze still has a slight performance gap relative to state-of-the-art supervised methods.The paper contrasts this synthetic-dataset gap with stronger performance in real-world scenes.
- Limitations and Discussion: UCL-Dehaze is not very robust in heavily hazy scenes, where overall scenes and object edges are difficult to recover naturally.The paper presents two typical failure cases and identifies this as a limitation shared with several other approaches.
5 Conclusion
UCL-Dehaze addresses real-world image dehazing without paired training data by using unpaired hazy and clean images in an adversarial contrastive framework. Its self-contrastive perceptual loss pulls restored images toward clean images and away from hazy images, supporting real-world generalization.
- UCL-Dehaze avoids paired training data by leveraging unpaired real-world hazy and clean images.
- Adversarial training and unpaired positive/negative data support generalization in real-world hazy scenarios.
- The self-contrastive perceptual loss encourages restored images to approach clean images while remaining distant from hazy images.