Source-linked AI summary
Gated Fusion Network for Single Image Dehazing
Wenqi Ren, Lin Ma, Jiawei Zhang, Jinshan Pan, Xiaochun Cao, Wei Liu, Ming-Hsuan Yang
TL;DR
Single-image dehazing is challenged by ill-posed transmission estimation and errors in atmospheric-light modeling. The paper proposes a multi-scale gated fusion network that directly combines three derived inputs using learned confidence maps, and reports favorable performance against state-of-the-art methods on synthetic and real-world images. Its main scope boundary is failure on images with very large fog.
Problem
Transmission estimation from a hazy image is severely ill-posed, and transmission errors can degrade recovered images and atmospheric-light estimation.
Method
GFN directly fuses white-balanced, contrast-enhanced, and gamma-corrected inputs with learned confidence maps in a multi-scale encoder-decoder network.
Results
The proposed method performs favorably against state-of-the-art dehazing methods on synthetic haze concentrations and challenging real-world images.
Takeaways & Limitations
GFN restores dehazed results without relying on explicit transmission and atmospheric-light estimates.
Takeaways & Limitations
The method cannot handle corrupted images with very large fog when the atmospheric light is not constant and the hazy model does not hold.
Abstract
from arXiv · showhide
In this paper, we propose an efficient algorithm to directly restore a clear image from a hazy input. The proposed algorithm hinges on an end-to-end trainable neural network that consists of an encoder and a decoder. The encoder is exploited to capture the context of the derived input images, while the decoder is employed to estimate the contribution of each input to the final dehazed result using the learned representations attributed to the encoder. The constructed network adopts a novel fusion-based strategy which derives three inputs from an original hazy image by applying White Balance (WB), Contrast Enhancing (CE), and Gamma Correction (GC). We compute pixel-wise confidence maps based on the appearance differences between these different inputs to blend the information of the derived inputs and preserve the regions with pleasant visibility. The final dehazed image is yielded by gating the important features of the derived inputs. To train the network, we introduce a multi-scale approach such that the halo artifacts can be avoided. Extensive experimental results on both synthetic and real-world images demonstrate that the proposed algorithm performs favorably against the state-of-the-art algorithms.
1. Introduction
Single-image dehazing is ill-posed because transmission and atmospheric light are unknown, while transmission errors can create artifacts. The proposed GFN avoids explicit estimation by fusing three derived inputs through learned confidence maps and multi-scale training.
- Single-image dehazing estimates a clean image from one hazy or foggy image for downstream scene-understanding tasks.
- Transmission estimation is severely ill-posed, and inaccurate estimates can produce haze artifacts or interfere with atmospheric-light estimation.
- The proposed end-to-end network avoids explicit transmission and atmospheric-light estimation through a fusion strategy.
- Three derived inputs address color cast and visibility: white balancing, contrast enhancement, and gamma correction each target different hazy regions.
- Learned confidence maps gate the derived inputs, while multi-scale training is used to eliminate halo artifacts.
2. Related Work
Prior dehazing methods use multiple images, hand-crafted priors, or data-driven transmission estimation, each relying on restrictive assumptions or potentially complex processing. GFN instead directly fuses three derived inputs without estimating transmission or atmospheric light.
- Related approaches: Dehazing research includes multi-image, hand-crafted-prior, and data-driven approaches.
- Multi-image aggregation: Multi-image methods assume multiple observations of the same scene, although most specific scenes have only one image available.
- Hand-crafted priors: Hand-crafted-prior methods can perform poorly when their assumptions inadequately describe real-world images, producing artifacts such as color distortions.
- Data-driven methods: CNN-based methods learn haze-related features but commonly still require accurate transmission and atmospheric-light estimation.
- Gated fusion: GFN directly fuses three derived images with learned confidence maps instead of estimating transmission and atmospheric light or using complex luminance, chromatic, and saliency blending.
3. Gated Fusion Network
GFN restores hazy images by learning confidence maps that gate three derived inputs—white-balanced, contrast-enhanced, and gamma-corrected—within an encoder-decoder network. A multi-scale coarse-to-fine design is trained across image resolutions to reduce halo artifacts and refine the final output.
- Gated Fusion Network: GFN learns confidence maps to combine the original hazy image and three derived inputs while retaining their most significant features.The derived inputs are white-balanced, contrast-enhanced, and gamma-corrected versions of the hazy image.
- Derived Inputs: White balancing targets atmospheric-light color casts, while contrast enhancement and gamma correction improve visibility and address dark regions.Gamma correction is used to overcome the dark limitation of the contrast-enhanced input.
- Network Architecture: The network uses an encoder-decoder architecture with skip connections, early fusion of the four inputs, and dilation to enlarge receptive fields without losing local details.The final layer reconstructs confidence maps for gating the derived inputs.
- The multi-Scale Refinement: A multi-scale coarse-to-fine procedure is introduced because strong confidence-map transitions can produce halo artifacts.Finer stages receive the previous stage’s sharp image, and the finest scale restores the high-resolution image.
- Training: Training applies mean squared error at every pyramid level and combines multi-scale content loss with adversarial loss.The network is optimized using the multi-scale content and adversarial objectives.
4. Experimental Results
The proposed algorithm is evaluated on synthetic and real-world hazy images against state-of-the-art methods, using quantitative metrics and qualitative comparisons. It achieves favorable results, with visually closer reconstructions, fewer color distortions, and competitive performance across haze concentrations and datasets.
- Evaluation setup: The proposed algorithm is quantitatively evaluated on synthetic and real-world hazy photographs against state-of-the-art methods.The experiments compare accuracy and visual effects across the reported datasets.
- Training data: Training uses 1,400 NYU Depth clean images with seven synthesized haze concentrations per image and 1% Gaussian noise.Atmospheric light and scattering coefficients are sampled during synthesis, with β ranging from 0.5 to 1.5.
- Synthetic dataset: On synthetic images, prior-based methods produce darker results and color distortions, while learning-based methods retain some haze.The proposed method produces results close to the ground-truth haze-free images.
- Quantitative results: The paper reports that qualitative improvements on synthetic images are reflected in PSNR and SSIM, and that the method performs favorably on the RESIDE SOTS dataset.Table 1 reports synthetic-dataset metrics, while Table 2 covers RESIDE SOTS.
- Synthetic dataset: The proposed method performs favorably across β = 0.8, 1.0, and 1.2, with PSNR exceeding AOD-Net by up to 2dB, especially under heavy haze.The comparison reports performance across multiple haze concentrations and notes that adversarial loss improves SSIM in some cases.
- Real-world images: On real-world images, competing methods exhibit over-enhancement, color distortions, darkness, or remaining haze, whereas the proposed results are clear with moderately enhanced details.The real-image comparison includes seven state-of-the-art dehazing algorithms.
5. Analysis and Discussions
The multi-scale network refines dehazed results by using coarser-scale information, reducing halo artifacts and producing cleaner edges. The gated fusion network preserves visible regions through pixel-wise confidence maps, while very large fog remains a failure case.
- Effectiveness of Multi-Scale Network: Coarser-scale outputs provide additional information to the finer-scale network, improving the final dehazed results.The comparison contrasts finest-scale-only results with the proposed multi-scale approach.
- Effectiveness of Gated Fusion Network: Figure 8 compares hazy inputs, ungated results, unfused results, and the gated fusion network.The figure presents these settings on two real-world examples.
- Effectiveness of Multi-Scale Network: The multi-scale approach produces cleaner edges than the finest-scale network, which shows halo artifacts around the person’s head.The halo appears in confidence maps and carries into the final dehazed result.
- Effectiveness of Gated Fusion Network: Pixel-wise confidence maps gate the derived inputs to preserve regions with good visibility during fusion.The fusion strategy retains useful information through single-pixel operations.
- Limitations: The method cannot handle corrupted images with very large fog because heavy haze interferes with the atmospheric-light assumption.The paper identifies this as a limitation and illustrates a case where the method does not produce a clear image.
6. Conclusions
The paper addresses single-image dehazing with a multi-scale gated fusion network that learns confidence maps for derived inputs. It avoids explicit transmission and atmospheric-light estimation, and experiments on synthetic and real-world images demonstrate the approach’s effectiveness.
- Conclusions: The proposed GFN is a fusion-based encoder-decoder that learns confidence maps for derived inputs.It generates a final dehazed result by combining the derived inputs according to the estimated confidence maps.
- Conclusions: The approach applies white balance and contrast enhancement before using GFN to estimate each derived input’s confidence map.The confidence maps and derived inputs are then used to render the final dehazed result.
- Conclusions: Experiments on synthetic and real-world images demonstrate the effectiveness of the proposed approach.