PaperlayerView exact revision on arXiv

Source-linked AI summary

DehazeNet: An End-to-End System for Single Image Haze Removal

Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, Dacheng Tao

arXiv:1601.07661v2cs.CV

TL;DR

Single-image haze removal is difficult because transmission depends on unknown, spatially varying depth. DehazeNet learns medium transmission end to end with a purpose-designed CNN, achieving superior benchmark performance while remaining efficient and easy to use.

  • Problem

    Single-image haze removal is difficult because haze transmission depends on unknown depth that varies across image positions.

  • Method

    DehazeNet is a trainable end-to-end CNN that learns medium transmission from hazy images for subsequent haze-free image recovery.

  • Results

    DehazeNet achieves superior performance over existing methods on benchmark images while remaining efficient and easy to use.

  • Takeaways & Limitations

    The lightweight architecture provides an effective and efficient approach to single-image dehazing.

  • Takeaways & Limitations

    Atmospheric light α is not modeled as a global constant learned jointly with medium transmission in a unified network.

Abstract

from arXiv · show

Single image haze removal is a challenging ill-posed problem. Existing methods use various constraints/priors to get plausible dehazing solutions. The key to achieve haze removal is to estimate a medium transmission map for an input hazy image. In this paper, we propose a trainable end-to-end system called DehazeNet, for medium transmission estimation. DehazeNet takes a hazy image as input, and outputs its medium transmission map that is subsequently used to recover a haze-free image via atmospheric scattering model. DehazeNet adopts Convolutional Neural Networks (CNN) based deep architecture, whose layers are specially designed to embody the established assumptions/priors in image dehazing. Specifically, layers of Maxout units are used for feature extraction, which can generate almost all haze-relevant features. We also propose a novel nonlinear activation function in DehazeNet, called Bilateral Rectified Linear Unit (BReLU), which is able to improve the quality of recovered haze-free image. We establish connections between components of the proposed DehazeNet and those used in existing methods. Experiments on benchmark images show that DehazeNet achieves superior performance over existing methods, yet keeps efficient and easy to use.

I. INTRODUCTION · II. RELATED WORKS

Single-image haze removal is difficult because transmission varies with unknown scene depth, motivating DehazeNet, an end-to-end CNN that estimates transmission maps for haze-free image recovery. The introduction also positions the method against enhancement, multi-image, depth-based, and prior-driven approaches, while proposing BReLU for image restoration.

  • I. INTRODUCTION: Haze scatters light and reduces subject contrast, making haze removal desirable in terrestrial and consumer photography.Haze can obscure distant subjects when dense atmospheric light penetration is needed.
  • I. INTRODUCTION: Unknown, position-varying scene depth makes single-image haze removal challenging.The haze transmission depends on depth that varies across image positions.
  • I. INTRODUCTION: Prior approaches include histogram-, contrast-, and saturation-based enhancement, alongside polarization and multi-constraint methods using multiple images.Depth-based methods additionally require user-provided depth information or known 3D models.
  • I. INTRODUCTION: Single-image methods have improved through assumptions and priors but can remain less effective on some images because they depend on heuristic haze cues.The cited progress includes local-contrast maximization based on a Markov Random Field.
  • I. INTRODUCTION: DehazeNet is a trainable end-to-end CNN that maps hazy image patches to medium transmission maps, which are then used for pixel-wise haze-free image recovery.The introduction identifies accurate medium transmission estimation as central to haze removal, apart from estimating global atmospheric light magnitude.
  • I. INTRODUCTION: DehazeNet’s architecture is specially designed to embody established image-dehazing principles, and it introduces Bilateral Rectified Linear Unit as a nonlinear activation.BReLU extends ReLU and is presented as improving image-restoration accuracy.
  • I. INTRODUCTION: The paper notes that an almost identical adjustable bounded rectifier was proposed earlier, while motivating BReLU specifically for image restoration rather than recognition.The manuscript dates the comparison to December 2015 and the related arXiv proposal to November 2015.
  • II. RELATED WORKS: Related work emphasizes the atmospheric scattering model as the basic underlying model of image dehazing and reviews assumptions for computing haze-relevant features.The section surveys important image-dehazing methods organized around these modeling and feature-computation contributions.

A. Atmospheric Scattering Model · B. Haze-relevant features

The paper models hazy-image formation with atmospheric scattering and identifies accurate medium-transmission estimation as central to recovering a clean scene. Because dehazing is ill-posed, existing methods use assumptions or priors to derive haze-relevant features for removal.

  • A. Atmospheric Scattering Model: The atmospheric scattering model represents a hazy image as I(x) = J(x)t(x) + α(1 − t(x)).The model is attributed to McCartney and later developments by Narasimhan and Nayar.
  • A. Atmospheric Scattering Model: I(x) is the observed hazy image, J(x) the recoverable scene, t(x) the medium transmission, and α the global atmospheric light.The pixel index x refers to pixels in the observed hazy image.
  • A. Atmospheric Scattering Model: The scene J(x) can be recovered after estimating the two atmospheric quantities α and t(x).The scattering equation contains three unknowns before these estimates are obtained.
  • A. Atmospheric Scattering Model: The medium transmission map t(x) describes the portion of light that is not scattered and reaches the camera.Its behavior depends on scene-point distance d(x) and atmospheric scattering coefficient β; as d(x) approaches infinity, t(x) approaches zero.
  • A. Atmospheric Scattering Model: In practical distant-view imaging, transmission is represented by a low value t0 rather than an infinite-distance limit.The global atmospheric light α is estimated more stably using a rule instead of relying directly on the limiting equation.
  • A. Atmospheric Scattering Model: Accurate medium-transmission estimation is the key step for recovering a clean scene and achieving haze removal.This follows from the atmospheric scattering model’s dependence on estimated transmission and atmospheric light.
  • B. Haze-relevant features: Image dehazing is inherently ill-posed, so existing methods use empirical assumptions or prior knowledge to compute intermediate haze-relevant features.Final haze removal is then achieved based on these features.

1) Dark Channel: · 2) Maximum Contrast: · 3) Color Attenuation:

The paper presents three haze-related priors for transmission estimation: dark-channel statistics, local maximum contrast, and color attenuation. These features relate haze to low channel intensities, reduced contrast, and changes in saturation and brightness or scene depth.

  • 1) Dark Channel:: In most haze-free outdoor patches, at least one color channel contains pixels with very low intensities, often near zero.The dark channel is defined as the minimum of all pixel colors within a local patch.
  • 1) Dark Channel:: The dark channel correlates strongly with haze amount and is used to estimate medium transmission as t(x) ∝ 1 − D(x).
  • 2) Maximum Contrast:: Haze reduces image contrast through the transmission of the atmospheric scattering process.
  • 2) Maximum Contrast:: Local contrast is measured from pixel-intensity variance in an s×s patch, then maximized locally over a surrounding r×r region.
  • 2) Maximum Contrast:: The contrast feature visibly correlates with medium transmission, motivating local-contrast maximization to enhance image visibility.
  • 3) Color Attenuation:: Under haze, patch saturation decreases sharply while brightness increases, so their difference estimates haze concentration.
  • 3) Color Attenuation:: In HSV color space, brightness and saturation are expressed from RGB channel extrema, while color attenuation is proportional to scene depth and aids transmission estimation.

4) Hue Disparity: · III. THE PROPOSED DEHAZENET · A. Layer Designs of DehazeNet

The paper identifies hue disparity as a haze-related cue whose inverse relationship with medium transmission supports transmission estimation. DehazeNet then learns hazy-image-to-transmission mappings through four sequential operations implemented by specialized network layers.

  • 4) Hue Disparity:: Hue disparity compares an image with its semi-inverse to detect haze-related changes across color channels.For haze-free images, the semi-inverse channels do not all flip, producing large hue changes relative to the original image.
  • 4) Hue Disparity:: According to the defined hue-disparity feature, medium transmission t(x) varies inversely with H(x).
  • III. THE PROPOSED DEHAZENET: DehazeNet explicitly learns the mapping from raw hazy images to their associated medium transmission maps.The design targets transmission estimation because transmission is the most important quantity for recovering a haze-free image under the atmospheric scattering model.
  • A. Layer Designs of DehazeNet: The network uses cascaded convolutional and pooling layers with nonlinear activations placed after selected layers.
  • A. Layer Designs of DehazeNet: The architecture implements the four operations with three convolution layers, max-pooling, a Maxout unit, and a BReLU activation function.
  • A. Layer Designs of DehazeNet: Prior work found multi-scale features effective for haze removal by densely computing input-image features at multiple spatial scales.
  • A. Layer Designs of DehazeNet: DehazeNet organizes transmission estimation into feature extraction, multi-scale mapping, local extremum, and nonlinear regression.These are the four sequential operations specified for the network’s layer designs.

1) Feature Extraction:

DehazeNet extracts haze-relevant features through learned convolutional operations designed around established dehazing assumptions. Its architecture uses Maxout units for automatic feature extraction and parallel convolutions at multiple spatial scales.

  • Feature Extraction: Existing haze-relevant features can be densely extracted by convolving a hazy image with appropriate filters followed by nonlinear mappings.Examples include dark channel, hue disparity, and color attenuation.
  • Feature Extraction: Maxout units map kn1-dimensional vectors into n1-dimensional outputs and learn haze-relevant features automatically rather than using heuristic rules.The first layer contains n1 output feature maps, with convolution filters operating on the three input image channels.
  • Feature Extraction: The second layer uses parallel convolutions with filter sizes 3 × 3, 5 × 5, and 7 × 7 for multi-scale feature extraction.This design is motivated by the benefits of multi-scale convolutional operations in architectures such as GoogLeNet.

2) Multi-scale Mapping: · 3) Local Extremum:

The multi-scale mapping organizes second-layer parameters into three groups, while the local-extremum operation performs dense neighborhood-based integration without reducing feature-map resolution. These design choices support spatial invariance and preserve resolution for image restoration, alongside the ReLU/BReLU activation comparison.

  • 2) Multi-scale Mapping:: The second layer’s n2 parameter pairs are divided into three groups, with n2 also denoting its output dimension.The index i ranges from 1 to n2 across the output feature maps.
  • 2) Multi-scale Mapping:: The parameter grouping uses (p,q)=(1,1) and partitions n2 pairs of parameters into three groups.The passage specifies upward integer rounding and remainder operations in this grouping.
  • 3) Local Extremum:: Spatial invariance is motivated by integrating linear-feature responses through pooling operations in visual-cortex complex cells.The passage relates this integration to the classical CNN use of neighborhood maxima for overcoming local sensitivity.
  • 3) Local Extremum:: The local-extremum operation evaluates a neighborhood maximum under each pixel to address local sensitivity.The passage frames this operation as consistent with the stated dehazing assumption.
  • 3) Local Extremum:: The local-extremum neighborhood is Ω(x), an f3 × f3 region centered at x, and the third-layer output dimension satisfies n3 = n2.Unlike conventional CNN max-pooling, the operation is described as densely applied.
  • 3) Local Extremum:: The section includes a comparison of Rectified Linear Unit (ReLU) and Bilateral Rectified Linear Unit (BReLU).This comparison is identified in Figure 3.
  • 3) Local Extremum:: Unlike max-pooling, the local extremum is applied densely to every feature-map pixel and preserves resolution for image restoration.The passage explicitly contrasts this behavior with max-pooling’s usual reduction of feature-map resolutions.

4) Non-linear Regression: … 1) Training Data:

DehazeNet combines a CNN-based end-to-end architecture with a regression-oriented BReLU activation and layers connected to established dehazing features. Because paired natural-scene data are scarce, training uses synthesized hazy/haze-free patches generated under physical-model assumptions.

  • 4) Non-linear Regression:: Sigmoid can suffer vanishing gradients, while ReLU provides sparse representations but is not perfectly suited to image-restoration regression.The passage contrasts these standard activations in terms of optimization and task suitability.
  • 4) Non-linear Regression:: BReLU uses marginal values tmin = 0 and tmax = 1 in DehazeNet.These values specify the activation’s output bounds in this paper.
  • 4) Non-linear Regression:: Four cascaded layers form a trainable CNN end-to-end system whose convolutional filters and biases are learned network parameters.The architecture is explicitly connected to expertise from existing image-dehazing methods.
  • B. Connections with Traditional Dehazing Methods: The first DehazeNet layer extracts haze-relevant features, and suitable filters can reproduce established dehazing features.The passage introduces dark-channel-related feature extraction in the first layer.
  • B. Connections with Traditional Dehazing Methods: Opposite filters with unit bias make Maxout outputs equivalent to minimum color-channel values, resembling the dark channel.Round filters similarly relate to maximum contrast, while combinations of all-pass and opposite filters extend these connections.
  • 1) Training Data:: Natural-scene hazy/haze-free pairs or corresponding transmission maps are not massively available, making labelled-data collection difficult.The training procedure therefore resorts to synthesized data based on the physical haze formation model.
  • 1) Training Data:: Training pairs are synthesized by assuming image content is independent of transmission and transmission is locally constant within small patches.These assumptions permit assigning an arbitrary transmission to an individual image patch.
  • 1) Training Data:: Haze-free Internet images are randomly sampled into 16 × 16 patches spanning daily-life, natural, and city-landscape content.The stated goal is for this varied training-sample content to be learned into DehazeNet’s filters.

2) Training Method: · IV. EXPERIMENTS

DehazeNet is trained through supervised learning to map RGB haze observations to medium transmission using synthetic paired patches and stochastic gradient descent. Experiments assess convergence and comparative performance while demonstrating CPU-only processing at approximately 1.5 seconds per 640 × 480 image.

  • 2) Training Method:: Supervised training learns the mapping from RGB values to medium transmission by minimizing loss between hazy training patches and corresponding ground-truth transmissions.The network parameters are optimized from paired hazy patches and medium transmissions synthesized from haze-free patches.
  • 2) Training Method:: SGD trains the DehazeNet implementation in Caffe, which contains 3 convolutional layers and 1 max-pooling layer with Maxout and BReLU activations.Maxout follows the first convolutional operation, while BReLU follows the last convolutional operation.
  • 2) Training Method:: The architecture configuration and parameter settings are summarized in Table I.The table documents the proposed DehazeNet model architecture used for training.
  • IV. EXPERIMENTS: DehazeNet’s convergence is analyzed and its performance is compared with FVR, DCP, BCCR, ATM, RF, BPNN, and CAP.The cited passage frames these comparisons as verification of the DehazeNet architecture.
  • IV. EXPERIMENTS: 100,000 synthetic patches are generated by sampling 10,000 haze-free patches and applying 10 uniformly sampled transmissions t ∈(0, 1) to each patch.Filter weights are initialized from a Gaussian distribution with µ = 0 and σ = 0.001, while biases are set to [text truncated in passage].
  • IV. EXPERIMENTS: 1.5 seconds is the approximate CPU-only processing time for a 640 × 480 image in MATLAB 2014A on an Intel i7 3770 at 3.4GHz.The lightened CNN-based network is described as supporting realtime performance without GPUs.

A. Model and performance · 1) Maxout unit in feature extraction F1: · 2) BReLU in non-linear regression F4:

DehazeNet focuses on specially designed feature-extraction and nonlinear-regression layers for transmission estimation, with fewer parameters than the two three-layer CNN baselines. Maxout accelerates and improves feature-extraction convergence, while BReLU improves convergence and precision in nonlinear regression over ReLU and Sigmoid.

  • A. Model and performance: DehazeNet uses specially designed feature-extraction F1 and nonlinear-regression F4 layers for transmission estimation.SRCNN and CNN-L with the same three-layer depth serve as traditional CNN baselines.
  • A. Model and performance: 8,240 parameters are used by DehazeNet, compared with 18,400 for SRCNN and 67,552 for CNN-L.These models are compared as three-layer baselines.
  • 1) Maxout unit in feature extraction F1:: F1 uses nonlinear dimension reduction to approximate traditional haze-relevant feature extraction and discover principal attributes while reducing pattern noise.The passage contrasts this design with classical PCA and LDA mappings and ReLU-based nonlinear sparse mapping.
  • 1) Maxout unit in feature extraction F1:: Maxout converges faster than ReLU and linear units, with performance improved by approximately 0.30e-2 over both alternatives.Maxout provides the equivalent function of almost all haze-relevant features.
  • 2) BReLU in non-linear regression F4:: BReLU combines bilateral restraint with local linearity to constrain the solution space, reduce gradient vanishing, and improve precision in image restoration and reconstruction.The contrast experiment replaces BReLU with ReLU and Sigmoid in F4.
  • 2) BReLU in non-linear regression F4:: BReLU converges faster than ReLU and Sigmoid, especially during the first 50,000 iterations.The passage reports the comparison during training in the nonlinear regression layer F4.
  • 2) BReLU in non-linear regression F4:: 0.05e-2 improvement over ReLU and 0.20e-2 over Sigmoid are achieved by BReLU in convergent precision.The same evaluation plots predicted transmission against ground-truth transmission on test patches.

B. Filter number and size · C. Quantitative results on synthetic patches

DehazeNet’s filter-width and filter-size studies identify a compact architecture that balances accuracy, speed, structural representation, robustness, and overfitting. On synthetic patches, DehazeNet achieves the best reported transmission-estimation MSE, outperforming RF and CAP-based learning methods.

  • B. Filter number and size: The filter-number and filter-size experiments compare training and testing MSEs under progressively modified DehazeNet parameter settings.All variants are trained on the same dataset.
  • B. Filter number and size: Increasing the number of filters improves performance, but a smaller network is preferred when fast dehazing is required.The adopted lightened network still performs better than other popular methods.
  • B. Filter number and size: Larger F2 filters capture richer structural information, while multi-scale 3/5/7 filters are adopted in F2.The multi-scale mapping combines filter sizes of 3, 5, and 7.
  • B. Filter number and size: The best F3/F4 setting is 5-M-7-6 because larger F4 filters improve fitting but risk overfitting, whereas F3 local extrema improve testing robustness.F3 and F4 filter sizes are adjusted simultaneously while preserving the same receptive field.
  • C. Quantitative results on synthetic patches: The synthetic-patch comparison evaluates DehazeNet against learning-based haze-removal methods using predicted-versus-ground-truth transmission error.The compared approaches include CAP’s linear model, BPNN, and RF.
  • C. Quantitative results on synthetic patches: 1.19e-2 is DehazeNet’s best state-of-the-art MSE on synthetic testing patches.Table III reports MSE between predicted transmissions and ground-truth transmissions.
  • C. Quantitative results on synthetic patches: 0.07e-2 is the MSE difference between DehazeNet and the next state-of-the-art result, RF.RF sorts patch feature values to break correlation between haze-relevant features and image content, whereas DehazeNet uses content information for sky regions and white objects.

D. Quantitative results on synthetic images

On synthetic hazy images generated from Middlebury stereo pairs with known depth, DehazeNet is evaluated against six methods using MSE, SSIM, PSNR, and WPSNR. It achieves the best overall evaluation performance and shows robustness to varying scattering coefficients, atmospheric airlight, image scale, and noise.

  • Synthetic-image evaluation: DehazeNet is tested on synthesized hazy images from 12 Middlebury stereo-image pairs with known depth and compared against DCP, FVR, BCCR, ATM, CAP, and RF.The synthesized images are restored to haze-free images by DehazeNet.
  • Synthetic-image evaluation: The evaluation uses mean square error (MSE), structural similarity (SSIM), peak signal-to-noise ratio (PSNR), and weighted peak signal-to-noise ratio (WPSNR).One-pass evaluation (OPE) denotes running the conventional method with standard parameters.
  • One-pass evaluation: DehazeNet achieves the best performance across MSE, SSIM, PSNR, and WPSNR despite being optimized with an MSE loss function.This result is reported for β = 1 and pure-white atmospheric airlight α = 1.
  • Robustness evaluations: DehazeNet shows excellent robustness to changes in scattering coefficient, atmospheric airlight, and image scale.The robustness evaluations vary β ∈{0.75, 1.0, 1.25, 1.5}, atmospheric airlight α, and four scale coefficients from 0.4 to 1.0; multi-scale mapping in F2 supports scale robustness.
  • Noise robustness evaluation: DehazeNet performs more robustly under additive white Gaussian noise than the other methods, benefiting from Maxout suppression in F1 and local extrema in F3.The noise robustness evaluation uses standard deviations σ ∈{10, 15, 20, 25}.

E. Qualitative results on real-world images · V. CONCLUSION

DehazeNet is evaluated qualitatively on five challenging real-world images, where it restores scene details but can distort difficult sky regions. The conclusion presents its trainable transmission-estimation design while identifying joint atmospheric-light and scattering-model learning as future work.

  • E. Qualitative results on real-world images: The qualitative comparison focuses on five challenging real-world images with large white or gray regions that are difficult for existing dehazing methods.The paper notes that algorithms perform well on general outdoor images, making visual ranking difficult.
  • E. Qualitative results on real-world images: DehazeNet removes much of the haze in the first three examples while restoring scene and object details.The results are shown alongside qualitative comparisons and depth-map restorations.
  • E. Qualitative results on real-world images: The first three examples still show significant sky-region over-enhancement, including overly dark, oversaturated, or distorted skies.Clouds and haze have similar atmospheric-scattering behavior, making sky regions especially challenging.
  • E. Qualitative results on real-world images: Prior-based transmission estimates become unreliable when scene objects resemble atmospheric light, as in the fair-skinned complexion and white marble failure cases.The cited priors include DCP, BCCR, and FVR; bright dark-channel values near such objects undermine the estimates.
  • V. CONCLUSION: The paper reformulates medium transmission estimation as a trainable end-to-end system with specially designed feature-extraction and nonlinear-regression layers.The approach is inspired by traditional haze-relevant features and dehazing methods, while distinguishing these layers from classical CNNs.
  • V. CONCLUSION: In DehazeNet’s first layer F1, the Maxout unit is described as similar to prior-based methods and more effective to learn.The supplied conclusion passage presents this as a key architectural connection and finding.
  • V. CONCLUSION: Future work includes learning atmospheric light α jointly with medium transmission because α cannot be treated as a global constant.The paper also proposes learning the atmospheric scattering model in a deeper neural network for direct haze-to-haze-free mapping without explicit medium-transmission estimation.
Loading 1601.07661v2…