Source-linked AI summary
You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network
Boyun Li, Yuanbiao Gou, Shuhang Gu, Jerry Zitao Liu, Joey Tianyi Zhou, Xi Peng
TL;DR
Single-image dehazing traditionally relies on hazy-clean pairs or image collections, which are difficult to obtain and can introduce domain shift from synthetic haze. YOLY instead disentangles each observed hazy image into scene radiance, transmission, and atmospheric-light layers and recomposes them self-supervised; experiments report promising quantitative and qualitative performance, while the method remains only comparable to state-of-the-art supervised approaches.
Problem
Single-image dehazing needs methods that avoid ground-truth clean images and image collections, because collecting suitable paired data is difficult and synthetic haze can create domain shift.
Method
YOLY uses three joint subnetworks to disentangle an observed hazy image into scene radiance, transmission-map, and atmospheric-light layers, then reconstructs it self-supervised.
Results
Experiments on synthetic and real-world datasets demonstrate promising quantitative and qualitative dehazing performance.
Takeaways & Limitations
YOLY provides an unsupervised, untrained dehazing approach and a learning-based way to synthesize new hazy images through haze transfer.
Takeaways & Limitations
YOLY remarkably outperforms most unsupervised shallow and deep methods but is only comparable to state-of-the-art supervised approaches.
Abstract
from arXiv · showhide
In this paper, we study two challenging and less-touched problems in single image dehazing, namely, how to make deep learning achieve image dehazing without training on the ground-truth clean image (unsupervised) and a image collection (untrained). An unsupervised neural network will avoid the intensive labor collection of hazy-clean image pairs, and an untrained model is a ``real'' single image dehazing approach which could remove haze based on only the observed hazy image itself and no extra images is used. Motivated by the layer disentanglement idea, we propose a novel method, called you only look yourself (\textbf{YOLY}) which could be one of the first unsupervised and untrained neural networks for image dehazing. In brief, YOLY employs three jointly subnetworks to separate the observed hazy image into several latent layers, \textit{i.e.}, scene radiance layer, transmission map layer, and atmospheric light layer. After that, these three layers are further composed to the hazy image in a self-supervised manner. Thanks to the unsupervised and untrained characteristics of YOLY, our method bypasses the conventional training paradigm of deep models on hazy-clean pairs or a large scale dataset, thus avoids the labor-intensive data collection and the domain shift issue. Besides, our method also provides an effective learning-based haze transfer solution thanks to its layer disentanglement mechanism. Extensive experiments show the promising performance of our method in image dehazing compared with 14 methods on four databases.
1 Introduction
Single-image dehazing is difficult because existing learning-based methods generally require hazy-clean pairs or large training datasets, while real-world data collection creates domain-shift concerns. YOLY addresses this gap with an unsupervised, untrained, layer-disentangling network that reconstructs the observed hazy image and supports haze transfer.
- Motivation: Haze reduces visibility, contrast, and visual detail, degrading downstream vision tasks such as object detection.Image dehazing is therefore used as both preprocessing and visual enhancement.
- Existing methods: Prior- and learning-based methods commonly use the atmospheric scattering model, but deep methods learn its parameters from large-scale training data.DehazeNet, for example, estimates transmission using a neural network trained on a large-scale dataset.
- Motivation: Large hazy-clean datasets are difficult to collect, and synthetic haze may be inconsistent with real haze, creating a domain-shift issue.The paper identifies simultaneous unsupervised and untrained dehazing as a relatively underexplored goal.
- Proposed method: YOLY uses three joint subnetworks to disentangle a hazy image into scene radiance, transmission-map, and atmospheric-light layers.The subnetworks are J-Net, T-Net, and A-Net.
- Proposed method: The disentangled layers are recomposed to reconstruct the observed hazy image using self-supervision derived only from that image.Thus, YOLY does not train on hazy-clean pairs or an image collection.
- Additional capability: YOLY also enables learning-based haze transfer by extracting haze components from one image and applying them to another clean image.This avoids handcrafting physical-model parameters for haze synthesis.
2 Related Work
Prior-based dehazing relies on handcrafted image priors, whereas learning-based methods use data-driven neural networks. YOLY differs from these approaches by operating unsupervised and untrained on each individual hazy image while also disentangling layers for haze transfer.
- 2.1 Prior-based methods: Prior-based methods estimate transmission and atmospheric light from handcrafted properties such as texture, contrast, chromatic aberration, or dark-channel statistics.These priors are derived from assumed intrinsic properties of haze-free images.
- 2.1 Prior-based methods: Prior-based performance can be undesirable when the adopted prior is inconsistent with changing environments.Their shallow-model design may also limit capacity for complex data.
- 2.2 Learning-based methods: Learning-based methods use a data-driven approach to learn transmission maps and/or atmospheric light, with neural networks achieving state-of-the-art performance.Unlike prior-based methods, they depend on training data to learn atmospheric-scattering parameters.
- 2.2 Learning-based methods: YOLY is unsupervised and does not require hazy-clean image pairs, unlike existing deep learning-based dehazing methods.This distinguishes it from the supervised training paradigm described for those methods.
- 2.3 Unsupervised deep image enhancement methods: YOLY is untrained: it handles each single hazy image directly without training on a dataset, avoiding the associated data-collection and domain-shift issues.This differs from N2N, which requires a corrupted image collection with the same noise distribution.
- 2.3 Unsupervised deep image enhancement methods: YOLY differs from other unsupervised methods through layer disentanglement, variational inference for atmospheric light, and color attenuation used as a clean-image supervisor.DDIP instead uses a U-Net-like fitting structure and early-stopping strategy.
3 Proposed Method
YOLY disentangles a single hazy image into clean image, transmission, and atmospheric-light layers, then reconstructs the input through the atmospheric scattering model. Its subnetworks and losses are designed for self-supervised, single-image learning without clean-image ground truth.
- Architecture: YOLY feeds one hazy image into J-Net, T-Net, and A-Net to predict clean image, transmission map, and atmospheric light layers.The outputs are combined to reconstruct the observed hazy image through the atmospheric scattering model.
- Self-supervision: The reconstruction loss constrains the complete network to reproduce the hazy input after layer disentanglement, incorporating the haze-generation process.This self-supervision guides both layer disentanglement and layer composition.
- J-Net: J-Net predicts the clean image using a non-degenerate architecture without downsampling, while its loss uses the brightness–saturation relationship to avoid clean-image ground truth.The output is normalized to [0, 1] with a sigmoid in the final layer.
- T-Net: T-Net uses a one-channel output for transmission estimation and is guided by self-supervision back-propagated from YOLY’s top reconstruction layer rather than an explicit loss.Its structure is otherwise similar to J-Net because both depend on the input image.
- A-Net: A-Net models atmospheric light with a variational auto-encoder, combining hint, variational-inference, and regularization losses.The regularizer is applied to A-Net outputs to encourage smooth atmospheric-light estimates, while the VAE resamples latent codes before decoding.
- Regularization: The atmospheric-light regularization uses neighborhood averaging, whereas applying the same regularization to J-Net could remove high-frequency details from the recovered haze-free image.The neighborhood is second-order, and the paper interprets this regularization as mean filtering.
4 Experiments
YOLY is evaluated on two synthetic and two real-world datasets against 14 baselines using PSNR, SSIM, and visual comparisons. It consistently performs strongly against unsupervised methods, with reported advantages on SOTS and HSTS, while also supporting real-world dehazing and haze transfer.
- Experimental setting: Experiments cover two synthetic and two real-world datasets, comparing YOLY with 14 baseline methods using PSNR and SSIM.The baselines include learning-based, prior-based, and unsupervised deep image enhancement methods.
- Synthetic indoor results: On SOTS, YOLY exceeds the best unsupervised deep method by 2.44 PSNR and 0.1180 SSIM, and gains 0.55 SSIM over the best prior-based method.YOLY surpasses all supervised methods except DehazeNet quantitatively and shows the best visual results in the reported comparisons.
- Synthetic outdoor results: On HSTS, YOLY outperforms the best unsupervised deep method by 2.91 PSNR and 0.0283 SSIM, and the best prior-based method by 4.9 PSNR and 0.0941 SSIM.Although quantitatively worse than DehazeNet, YOLY is reported to show better recovery in an example visual comparison.
- Synthetic indoor results: Unsupervised deep baselines generally perform poorly in the SOTS comparisons because they were not specifically designed for dehazing, whereas DDIP and YOLY perform better.The comparison is presented as evidence of the difficulty of unsupervised single-image dehazing.
- Real-world results: YOLY produces the best visual results in almost all reported real-world scenes, preserving details and avoiding distortions or detail loss observed in several baselines.The comparison discusses detailed areas, background color distortions, and low-light regions.
- Haze transfer: YOLY also provides haze transfer through layer disentanglement, generating haze that is lighter near the camera and heavier farther away than handcrafted haze.The transferred haze is described as more realistic than the handcrafted haze, while outdoor depth estimation remains difficult.
5 Conclusion
YOLY is an unsupervised and untrained dehazing network that separates hazy images into scene radiance, transmission, and atmospheric-light layers. Experiments show promising quantitative and qualitative performance, while results remain only comparable to state-of-the-art supervised approaches.
- YOLY separates each observed hazy image into scene radiance, transmission map, and atmospheric light layers without using information beyond that image.This gives the method both unsupervised and untrained characteristics.
- The method also supports transferable haze synthesis by extracting haze from a given image in a learning- and unsupervised fashion.
- Experiments on two synthesis datasets and one real-world dataset demonstrate promising quantitative and qualitative performance.
- YOLY remarkably outperforms most existing unsupervised shallow and deep methods but is only comparable to state-of-the-art supervised dehazing approaches.The authors identify further performance improvement as a future direction.