PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint Transmission Map Estimation and Dehazing using Deep Networks

He Zhang, Vishwanath Sindagi, Vishal M. Patel

arXiv:1708.00581v2cs.CV

TL;DR

Single-image dehazing is ill-posed, and many existing methods assume constant atmospheric light and separate transmission estimation from dehazing. This paper relaxes that assumption in an end-to-end network that jointly estimates transmission and dehazes images; reported visual comparisons include strong results on synthetic images, while some specific sky regions remain over-exposed.

  • Problem

    Single-image dehazing is ill-posed, and many methods assume constant atmospheric light and handle transmission estimation and dehazing as separate tasks.

    Source: §I · p. 1

  • Method

    The end-to-end CNN relaxes the constant atmospheric-light assumption and jointly learns transmission maps and dehazed images using transmission-guided features and task-specific losses.

    Source: §I · p. 2

  • Results

    On synthetic images, the proposed method and Berman et al. achieve the best visual performance among the compared methods, while Berman et al. show color shift.

    Source: D. Comparison with state-of-the-art… · p. 7

  • Takeaways & Limitations

    Joint learning implicitly learns atmospheric light, avoiding the need for manual calculation.

    Source: §I · p. 2

    Some specific images have over-exposed central sky regions, which the authors primarily attribute to the rarity of similar training samples.

    Source: E. Failure Cases · p. 9

Abstract

from arXiv · show

Single image haze removal is an extremely challenging problem due to its inherent ill-posed nature. Several prior-based and learning-based methods have been proposed in the literature to solve this problem and they have achieved superior results. However, most of the existing methods assume constant atmospheric light model and tend to follow a two-step procedure involving prior-based methods for estimating transmission map followed by calculation of dehazed image using the closed form solution. In this paper, we relax the constant atmospheric light assumption and propose a novel unified single image dehazing network that jointly estimates the transmission map and performs dehazing. In other words, our new approach provides an end-to-end learning framework, where the inherent transmission map and dehazed result are learned directly from the loss function. Extensive experiments on synthetic and real datasets with challenging hazy images demonstrate that the proposed method achieves significant improvements over the state-of-the-art methods.

I. INTRODUCTION

The introduction frames single-image dehazing as ill-posed and proposes jointly estimating transmission and dehazing while relaxing the constant atmospheric-light assumption.

I. INTRODUCTION

  • Many methods assume constant atmospheric light and treat transmission estimation and dehazing as separate tasks, despite the ill-posed nature of single-image dehazing.Earlier methods use priors or additional information such as depth or multiple images to address the ill-posed problem.Source: §I · p. 1
  • Transmission-map learning uses adversarial, gradient, and pixel-wise Euclidean losses.The authors propose the GAN framework for learning transmission maps.Source: §I · p. 2
  • The dehazing network combines transmission maps with hazy-image features and optimizes perceptual and pixel-wise Euclidean losses.The stated aim is perceptually better results with high-quality details.Source: §I · p. 2
  • The proposed end-to-end CNN jointly learns transmission maps and dehazed images, relaxing the constant atmospheric-light assumption.The network implicitly learns the transformations from hazy input to transmission map and from transmission map to dehazed image.Source: §I · p. 2
  • The authors report conducting experiments on synthetic and real image datasets to demonstrate the method’s effectiveness.Source: §I · p. 2

II. RELATED WORK

Prior-based and CNN-based methods address single-image dehazing, but many CNN methods still separate transmission estimation from dehazing and assume constant atmospheric light. The proposed method instead estimates a transmission map and dehazed image jointly.

A. Single Image Dehazing

  • Earlier prior-based methods estimate haze-free images using assumptions about patch contrast, scene layers, dark channels, or color clusters.Examples include dark-channel and non-local color priors.Source: A. Single Image Dehazing · p. 2
  • Many CNN methods still use a two-step procedure and assume constant atmospheric light, limiting learning to the mapping from hazy input to transmission map.The proposed approach relaxes this assumption to learn the mapping onward to a dehazed image.Source: A. Single Image Dehazing · p. 2

II. RELATED WORK

  • The proposed method has transmission-map estimation, hazy-feature extraction, and guided image-dehazing modules.Estimated transmission maps are concatenated with hazy-image features before guided dehazing.Source: Figure 2 · p. 3

B. Loss Functions

Pixel-wise losses can blur outputs and miss perceptual detail, motivating adversarial and perceptual losses. The paper applies adversarial loss to transmission-map learning and VGG-16-based perceptual loss to dehazing.

B. Loss Functions

  • Pixel-wise L1 or L2 losses have limited ability to capture sharp edges and contours and tend to produce blurred results.The paper uses adversarial loss for transmission-map learning and perceptual loss for dehazing.Source: B. Loss Functions · p. 3

1) Adversarial loss:

  • Adversarial loss trains a generator to produce samples resembling the training distribution while a discriminator distinguishes real from synthesized samples.The proposed work uses GAN as an additional loss to guide transmission-map learning.Source: 1) Adversarial loss · p. 3

2) Perceptual loss:

  • Perceptual loss compares high-level features from a pre-trained convolutional network, and this work uses VGG-16-based perceptual loss for dehazing.Source: 2) Perceptual loss · p. 3

III. PROPOSED METHOD

  • The proposed network combines transmission-map estimation, hazy-image feature extraction, and transmission-guided dehazing.The modules estimate transmission, extract haze-relevant features, and combine those features with the map to dehaze.Source: §III · p. 3

A. Transmission Map Estimation

The method estimates transmission maps with adversarial, pixel-wise, and gradient losses, then combines the maps with hazy-image features to produce dehazed images. Joint end-to-end learning relaxes the constant atmospheric-light assumption and trains both tasks with image-space losses.

A. Transmission Map Estimation

  • The transmission-map estimator uses a conditional GAN with a generator and discriminator, alongside pixel-wise Euclidean loss.A gradient-based loss also enforces consistency between estimated and target transmission-map gradients.Source: A. Transmission Map Estimation · p. 4

B. Hazy Feature Extraction and Guided Image Dehazing

  • Hazy-image features are concatenated with the estimated transmission map and passed to a guided dehazing module that maps them to a dehazed image.The fusion network is trained with VGG-16-based perceptual loss and pixel-wise Euclidean loss.Source: B. Hazy Feature Extraction and Guided… · p. 4
  • The network jointly learns a nonlinear mapping from hazy input and transmission map to dehazed image, implicitly learning atmospheric light and avoiding its manual estimation.Source: B. Hazy Feature Extraction and Guided… · p. 4

C. Training Loss

  • The transmission-map loss combines L2 error, adversarial error, and gradient loss to address blurred estimates associated with minimizing L2 error alone.The gradient loss compares target and estimated gradients in horizontal and vertical directions.Source: 1) Transmission map loss Lt · pp. 4–5
  • Gradient loss eliminates halo artifacts near complicated edges, while the GAN framework sharpens local reconstructions and preserves object structure.These qualitative ablation findings are reported for transmission estimation.Source: Figure 3 · p. 5
  • The dehazing loss combines pixel-wise Euclidean and perceptual losses, and its gradient is also propagated to the transmission estimator.The perceptual loss uses high-level features from relu3_1 in VGG-16.Source: 2) Dehazing loss Ld · p. 5

D. Discussion

Relaxing the constant atmospheric-light condition enables end-to-end learning of the mappings from hazy image to transmission map and from transmission map to dehazed image, with added model complexity.

D. Discussion

  • Relaxing constant atmospheric light enables end-to-end learning but introduces more model complexity.Source: D. Discussion · p. 5
  • The method uses the transmission map because it preserves information about the portion of light that reaches the camera without scattering.Source: D. Discussion · p. 5
  • Adversarial and gradient losses provide strong regularization for transmission-map estimation.Source: D. Discussion · p. 5

IV. EXPERIMENTS

The experiments evaluate the proposed network on synthetic and real datasets, examine its components through ablation studies, and compare it with recent state-of-the-art methods.

IV. EXPERIMENTS

  • The experiments cover synthetic and real datasets containing a variety of hazy conditions.Source: §IV · p. 6
  • Ablation studies examine improvements associated with the proposed network’s different modules.Source: §IV · p. 6
  • The proposed network is compared with recent state-of-the-art methods.Source: §IV · p. 6

A. Datasets

The evaluation uses synthetic data generated from NYU Depth images and a separate set of real-world hazy images. The synthetic training and test images use varied atmospheric light and scattering conditions.

A. Datasets

  • 2,400 synthetic training images are generated from 600 NYU Depth images, with four randomly sampled atmospheric-light and scattering-coefficient settings per source image.Source: A. Datasets · p. 6
  • The synthetic test set contains 300 images, with no training images appearing in the test set.Source: A. Datasets · p. 6
  • Atmospheric light A(x) is sampled from [0.5, 1.2] and scattering coefficient β from [0.4, 1.6] to vary haze conditions.Source: A. Datasets · p. 6
  • Although the synthetic images come from an indoor dataset with relatively shallow depths, varying β is used to vary haze concentration for outdoor dehazing.Source: A. Datasets · p. 6
  • A separate real-world test dataset comprises 30 hazy images downloaded from the Internet.Source: A. Datasets · p. 6

B. Training Details

The network is trained in stages, first for transmission estimation and then jointly for dehazing and transmission estimation. Ablations examine transmission-map losses, map-guided dehazing, perceptual loss, and their runtime trade-offs.

B. Training Details

  • The network is trained first on transmission-map estimation loss, then fine-tuned using both transmission-estimation and dehazing losses.Training uses ADAM with a learning rate of 2 × 10−3, batch size 10, and 256 × 256 resized samples.Source: B. Training Details · p. 6

C. Ablation Study

  • Adding gradient loss eliminates halo artifacts near complicated edges, while adding the discriminator refines local regions and sharpens transmission-map reconstructions.Table I reports quantitative SSIM results for transmission-map Ablation 1 on synthetic datasets.Source: C. Ablation Study · p. 6, Table I · p. 6
  • Without transmission-map estimation, dehazed outputs can show color distortion; adding the map branch improves image quality, and perceptual loss adds high-quality detail.Table II reports quantitative SSIM results for dehazed-image Ablation 2 on synthetic datasets.Source: C. Ablation Study · p. 6, Table II · p. 7, Figure 4 · p. 7
  • The Ablation 2 variants are also compared by average running time, and multi-task learning brings a slight increase in training and inference complexity.The loss functions increase training time, but do not affect inference time.Source: C. Ablation Study · pp. 6–7, Table III · p. 7

D. Comparison with state-of-the-art Methods

On synthetic and real-world images, the proposed method is reported to produce strong dehazing and transmission-map results compared with prior approaches. It achieves the best synthetic-dataset SSIM and is reported to generalize to real-world conditions despite training on synthetic data.

D. Comparison with state-of-the-art Methods

  • On a synthetic test image, the proposed method is reported to dehaze across varied haze contents and produce higher-quality transmission maps than recent methods.The comparison describes other methods as producing incomplete haze removal or over-correction; Fig. 5 presents estimated transmission maps in its first row and dehazed images in its last row.Source: D. Comparison with state-of-the-art… · p. 7, Figure 5 · p. 8
  • On synthetic images used by prior methods,, Berman et al., and the proposed method are reported to perform best visually, while Berman et al. show color shift.Source: D. Comparison with state-of-the-art… · p. 7, Figure 6 · p. 8
  • The proposed method achieves the best SSIM performance among five state-of-the-art methods on the synthetic dataset.Parameters for other methods were fine-tuned and held consistent; transmission estimates for and are not included because their released code cannot estimate them.Source: D. Comparison with state-of-the-art… · p. 7, Table IV · p. 8
  • On real-world examples, prior methods are described as leaving haze, producing artifacts or oversaturation, or darkening outputs; the proposed method is reported to achieve near-complete dehazing without those effects.The text attributes partial dehazing to and, and darker outputs under heavy haze to Berman et al.,.Source: D. Comparison with state-of-the-art… · p. 8, Figure 7 · p. 9
  • Although trained on synthetic data, the proposed method is reported to generalize to real-world conditions and process 512×512 images at 18 frames per second.A particularly difficult real-world example shows improved recovery of Mandarin characters hidden by haze.Source: D. Comparison with state-of-the-art… · pp. 8–9

E. Failure Cases

The method generalizes to most outdoor cases but can saturate regions in specific images. The paper also notes a synthetic-to-real performance gap and calls for better ways to leverage synthetic data.

E. Failure Cases

  • Saturation in certain regions limits the method on specific images despite its generalization to most outdoor cases.The authors describe this as a problem common to existing methods.Source: E. Failure Cases · p. 9
  • Rarely seen training examples primarily explain why the central sky remains unrecovered and looks over-exposed in the cited failure case.The example is shown in Fig. 11.Source: E. Failure Cases · p. 9
  • A performance gap between synthetic and real-world images illustrates limitations of learning from synthetic data and motivates exploring better ways to leverage it for real-world generalization.Source: E. Failure Cases · p. 9

V. CONCLUSION

The paper presents an end-to-end CNN that jointly estimates transmission maps and dehazes images, relaxing the constant atmospheric light assumption. It reports experiments on synthetic and real datasets, comparisons with recent methods, and ablation studies.

V. CONCLUSION

  • The paper's main contribution is a multi-task end-to-end CNN that jointly estimates transmission maps and dehazes images, relaxing the constant atmospheric light assumption.Source: §V · pp. 9–10
  • The accompanying material includes real-world qualitative comparisons, proposed-method dehazing results, a labeled failure-case figure, and a synthesized-dataset running-time table.The supplied captions do not state the table values or comparative outcomes.Source: Figure 9 · p. 10, Figure 10 · p. 10, Figure 11 · p. 10
  • The network estimates a transmission map and uses it for subsequent dehazing, retaining the standard image degradation model.Source: §V · p. 10
  • Experiments span multiple synthetic and real datasets, compare results with recent methods, and include detailed ablation studies.Source: §V · p. 10

Related papers

Loading 1708.00581v2…