Source-linked AI summary
GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing
Xiaohong Liu, Yongrui Ma, Zhihao Shi, Jun Chen
TL;DR
Single-image dehazing is difficult because the haze model leaves the problem underdetermined and preprocessing choices may be heuristic. GridDehazeNet uses an end-to-end trainable CNN with learned inputs and attention-based multi-scale estimation on a grid network, avoiding dependence on the atmosphere scattering model. It achieves superior or highly competitive performance against state-of-the-art methods on synthetic and real-world dehazing settings.
Problem
The atmosphere scattering model makes dehazing underdetermined without atmospheric light and transmission, while hand-selected preprocessing may be poorly suited to the problem.
Method
GridDehazeNet is an end-to-end trainable CNN that generates learned inputs and performs attention-based multi-scale estimation on a grid network.
Results
GridDehazeNet outperforms state-of-the-art methods on synthetic and real-world images, with highly competitive performance on an additional synthetic dataset.
Takeaways & Limitations
The results suggest that image dehazing need not rely on the atmosphere scattering model and motivate reconsidering the physical model's role in restoration algorithm design.
Abstract
from arXiv · showhide
We propose an end-to-end trainable Convolutional Neural Network (CNN), named GridDehazeNet, for single image dehazing. The GridDehazeNet consists of three modules: pre-processing, backbone, and post-processing. The trainable pre-processing module can generate learned inputs with better diversity and more pertinent features as compared to those derived inputs produced by hand-selected pre-processing methods. The backbone module implements a novel attention-based multi-scale estimation on a grid network, which can effectively alleviate the bottleneck issue often encountered in the conventional multi-scale approach. The post-processing module helps to reduce the artifacts in the final output. Experimental results indicate that the GridDehazeNet outperforms the state-of-the-arts on both synthetic and real-world images. The proposed hazing method does not rely on the atmosphere scattering model, and we provide an explanation as to why it is not necessarily beneficial to take advantage of the dimension reduction offered by the atmosphere scattering model for image dehazing, even if only the dehazing results on synthetic images are concerned.
1. Introduction
Image dehazing seeks to recover clear images from haze, but the problem is underdetermined and raises broader questions about physical models, preprocessing, and multi-scale estimation. GridDehazeNet addresses these issues with an end-to-end trainable CNN that avoids reliance on the atmosphere scattering model and uses learned preprocessing with attention-based multi-scale estimation.
- Image dehazing recovers a clear image from a hazy image and supports robust visual analysis under environmental distortion.
- The atmosphere scattering model makes dehazing underdetermined without the atmospheric light A and transmission map t(x).
- Physical-model-based algorithms may suffer performance loss on real-world images because of model mismatch.
- Hand-selected preprocessing methods are heuristic and may not be best suited to the dehazing problem.
- Multi-scale estimation reduces a high-dimensional statistical model to a low-dimensional one, but this creates a bottleneck issue in image restoration.
- GridDehazeNet is an end-to-end trainable CNN that avoids the atmosphere scattering model, learns preprocessing, and applies attention-based multi-scale estimation on a grid network.
2. Related Work
Single-image dehazing without side information is difficult, motivating methods that estimate haze-related quantities, use priors, or learn from synthetic data. Later approaches increasingly reduce reliance on hand-crafted priors, with some methods bypassing or abandoning the atmosphere scattering model.
- Early dehazing methods required multiple images of the same scene or side information from other sources.
- Single-image dehazing without side information commonly estimates the transmission map t(x) and atmospheric light A before inverting the haze model.
- Representative prior-based methods use contrast maximization, albedo analysis, or the Dark Channel Prior to remove haze.
- Deep-learning methods increasingly use synthetic datasets and reduce reliance on hand-crafted priors, including CNNs for transmission estimation.
- AOD-Net reformulates the haze equation to bypass transmission and atmospheric-light estimation, while later work explicitly abandons the atmosphere scattering model.
3. GridDehazeNet
GridDehazeNet avoids reliance on the atmosphere scattering model and combines trainable preprocessing, attention-based grid multi-scale estimation, and post-processing for single-image dehazing. Its design addresses constrained-search, preprocessing, bottleneck, and artifact issues in image restoration.
- No reliance on the atmosphere scattering model: GridDehazeNet avoids the atmosphere scattering model because constraining transmission maps can create obstructive local minima and heterogeneous loss surfaces.Figure 2 illustrates a constrained local minimum even when the unconstrained loss surface has a better global minimum.
- Trainable preprocessing: The trainable preprocessing module generates learned inputs that expose diverse and pertinent features instead of relying on hand-selected preprocessing methods.It converts one hazy image into several variants, while trainable transforms can identify more suitable feature domains.
- Attention-based multi-scale estimation: Attention-based multi-scale estimation uses a grid network with dense cross-scale connections to alleviate bottlenecks in hierarchical architectures.The backbone uses three rows and six columns, with rows representing scales and columns connecting them through learned upsampling and downsampling blocks.
- Network architecture: The network contains preprocessing, backbone, and post-processing modules, with the latter reducing artifacts in images produced directly by the backbone.The preprocessing module generates 16 learned inputs, and the post-processing structure is symmetrical to preprocessing.
- Attention mechanism: Channel-wise attention assigns trainable weights to feature channels so GridDehazeNet can adjust the contributions of different scales during fusion.The authors report that performance can improve substantially with only a small number of trainable attention weights.
- Network flexibility: GridDehazeNet can be pruned through suitable attention weights to recover structures resembling encoder-decoder and conventional multi-scale networks.Removing the exchange branches produces a structure resembling the conventional multi-scale network.
4. Experimental Results
The experiments evaluate GridDehazeNet on synthetic and real-world dehazing data, using quantitative and qualitative comparisons against existing methods. The setup relies on large-scale synthetic training data and end-to-end training without pre-training submodules.
- GridDehazeNet is evaluated on synthetic and real-world datasets using quantitative dehazing results and qualitative visual effects.
- RESIDE supplies synthetic hazy images from indoor and outdoor scenarios for training and testing.
- The network is trained end-to-end without pre-training submodules, using 240×240 RGB patches and Adam optimization.
4.3. Synthetic Dataset
On synthetic data, GridDehazeNet is compared with prior-based and data-driven state-of-the-art methods using qualitative and quantitative assessments. It achieves stronger visual dehazing and remains highly competitive on an additional synthetic dataset.
- The synthetic evaluation compares GridDehazeNet with DCP, DehazeNet, MSCNN, AOD-Net, and GFN using qualitative and quantitative assessments.SOTS is assessed with average PSNR and SSIM.
- DCP, DehazeNet, MSCNN, AOD-Net, and GFN each exhibit distinct residual haze, color distortion, halo artifacts, or limited thick-haze removal in the comparisons.
- GridDehazeNet produces the best qualitative balance of haze removal and artifact or distortion suppression on SOTS.Its outputs are described as free of major artifacts and visually most similar to haze-free counterparts.
- GridDehazeNet outperforms the state-of-the-arts by a wide margin on average PSNR and SSIM values reported for SOTS.
- On a new synthetic dataset generated from 500 Sun RGB-D clear images with depth maps, GridDehazeNet remains fairly robust and highly competitive.The haze uses the atmosphere scattering model with β ∈[0.04, 0.2] and A ∈[0.8, 1.0].
4.4. Real-World Dataset
On the real-world dataset, evaluation is qualitative because haze-free counterparts are unavailable. The results are largely consistent with the synthetic comparisons, including incomplete haze removal and distortions from competing methods.
- The real-world dataset is evaluated qualitatively because haze-free counterparts of its hazy images are unavailable.
- The real-world results are largely consistent with the synthetic results.
- DCP suffers severe color distortions, while DehazeNet, MSCNN, and AOD-Net leave haze visibly incomplete; GFN struggles with dense haze and some color distortions.
4.5. Atmosphere Scattering Model
The study compares direct dehazing with an atmosphere-scattering-model-based strategy that first estimates transmission and atmospheric light. Although the model reduces estimated parameters, incorporating it degrades performance.
- The indirect strategy estimates a transmission map and atmospheric light before computing the dehazed image through the atmosphere scattering model.
- Reducing the number of parameters with the atmosphere scattering model nevertheless incurs performance degradation relative to direct estimation.
- Table 2 compares different estimation strategies for image dehazing.
4.6. Learned Inputs
The trainable pre-processing module generates multiple learned inputs that expose different aspects of a hazy image. These learned representations are evaluated against RGB/zero inputs and hand-selected derived inputs.
- Four illustrated learned inputs, among 16 total, each enhance a particular aspect of the hazy image.One learned input highlights texture that is not evident in the original hazy image.
- The learned-input ablation replaces the first three inputs with RGB channels and the remaining inputs with zero feature maps.
- The comparison against hand-selected inputs uses variants from hazy, white-balanced, contrast-enhanced, gamma-corrected, and grayscale images.
- Table 3 reports SOTS comparisons for different input types.
4.7. Ablation Study
Ablations examine the backbone’s grid dimensions and remove individual GridDehazeNet components. The reported results attribute performance contributions to the full design and its constituent modules.
- The backbone ablation varies the number of rows r and columns c in the grid network.Rows represent scales, while columns bridge information exchange across scales.
- Tables 4 and 5 report comparisons for backbone configurations and GridDehazeNet variants on SOTS.
- Additional variants remove exchange branches, attention-based channel-wise fusion, post-processing, perceptual loss, or grid-network structure.
- Each component contributes to performance, supporting the overall GridDehazeNet design.
4.8. Runtime Analysis
GridDehazeNet’s unoptimized implementation dehazes an SOTS image in about 0.22 seconds on average. Among the compared methods, it ranks second in runtime.
- 0.22s is the average time for GridDehazeNet to dehaze one SOTS image using unoptimized code.
- GridDehazeNet ranks second among the dehazing methods compared by average runtime.
5. Conclusion
GridDehazeNet achieves competitive single-image dehazing performance, while its generic components suggest broader applicability to image restoration and motivate rethinking physical-model use.
- GridDehazeNet is an end-to-end trainable CNN for single-image dehazing.
- Its generic building components are expected to apply across a wide range of image restoration problems.
- The work examines the puzzling role of the atmosphere scattering model in image dehazing.
- The authors suggest rethinking how physical models are used when designing image restoration algorithms.