PaperlayerView exact revision on arXiv

Source-linked AI summary

Nighttime Dehazing with a Synthetic Benchmark

Jing Zhang, Yang Cao, Zheng-Jun Zha, Dacheng Tao

arXiv:2008.03864v3cs.CVcs.LGeess.IV

TL;DR

Nighttime dehazing lacks realistic large-scale benchmarks because active illumination and haze are spatially varying and intertwined. The paper introduces 3R to synthesize nighttime haze, OS-MRP to separate color correction from haze removal, and a CNN baseline; experiments report superiority in image quality and runtime, with stated limitations from depth and prior assumptions.

  • Problem

    Nighttime dehazing is difficult because uneven artificial illumination and haze jointly reduce visibility, while existing synthetic daytime data and benchmarks are inadequate.

  • Method

    3R models scene geometry, real-world light colors, illumination, reflectance, and haze, while OS-MRP and an encoder-decoder CNN address dehazing.

  • Results

    The proposed methods show superiority over state-of-the-art methods in both image quality and runtime on synthetic and real-world images.

  • Takeaways & Limitations

    The paper provides a realistic nighttime-hazy benchmark and efficient methods for evaluating and performing nighttime dehazing.

  • Takeaways & Limitations

    Synthetic images may contain depth-boundary artifacts, and OSFD can produce color artifacts or residual haze in large monochromatic or dense-haze regions.

Abstract

from arXiv · show

Increasing the visibility of nighttime hazy images is challenging because of uneven illumination from active artificial light sources and haze absorbing/scattering. The absence of large-scale benchmark datasets hampers progress in this area. To address this issue, we propose a novel synthetic method called 3R to simulate nighttime hazy images from daytime clear images, which first reconstructs the scene geometry, then simulates the light rays and object reflectance, and finally renders the haze effects. Based on it, we generate realistic nighttime hazy images by sampling real-world light colors from a prior empirical distribution. Experiments on the synthetic benchmark show that the degrading factors jointly reduce the image quality. To address this issue, we propose an optimal-scale maximum reflectance prior to disentangle the color correction from haze removal and address them sequentially. Besides, we also devise a simple but effective learning-based baseline which has an encoder-decoder structure based on the MobileNet-v2 backbone. Experiment results demonstrate their superiority over state-of-the-art methods in terms of both image quality and runtime. Both the dataset and source code will be available at https://github.com/chaimi2013/3R.

1 INTRODUCTION

Nighttime haze removal is difficult because spatially varying artificial illumination and haze jointly degrade visibility, while limited realistic training data impedes deep-learning progress. The paper addresses these challenges with a synthetic benchmark, OS-MRP, and a CNN baseline.

  • Spatially varying artificial lights create uneven illumination and color casts, while haze further reduces visibility through absorption and scattering.
  • Nighttime models trained on synthetic daytime haze do not generalize well because nighttime illumination differs and is entangled with haze effects.
  • 3R synthesizes realistic nighttime hazy images from daytime clear images by modeling empirical light colors, scene geometry, light rays, reflectance, and haze.
  • The proposed benchmark evaluates state-of-the-art nighttime dehazing methods comprehensively.
  • OS-MRP sequentially separates color correction from haze removal, while the CNN baseline targets strong performance with high computational efficiency.
  • Extensive experiments report superiority over state-of-the-art methods in both image quality and runtime.

2 RELATED WORK

Nighttime imaging combines spatially varying illumination, color cast, reflectance, and haze, making standard daytime priors and synthesis assumptions inadequate. Prior work uses statistical priors or supervised learning, while this paper models real-world light-color distributions and scene geometry for more realistic synthesis.

  • The nighttime imaging model represents the observed hazy image using illuminance, color cast, reflectance, and haze transmission.
  • Recovering the nighttime clear image is ill-posed because latent illumination, color cast, and transmission must be estimated.
  • Daytime statistical priors such as DCP cannot be directly applied because active-light color cast biases nighttime image statistics.
  • Supervised dehazing methods require large-scale paired training samples, whereas existing nighttime synthesis assumptions limit the sampling space.
  • The proposed synthesis method samples empirical real-world light colors and incorporates scene geometry to render spatially varying color cast and haze simultaneously.

3 SYNTHETIC NIGHTTIME HAZY IMAGES

The 3R pipeline synthesizes nighttime hazy images by learning realistic light-color statistics, reconstructing scene geometry, simulating illumination, and rendering haze. It produces visually realistic images and a configurable benchmark from Cityscapes data.

  • Light-color prior: Over 1,300 real-world nighttime images were used to derive a prior distribution of artificial light colors.
  • Light-color prior: The light-color sampling procedure covers 98.68% of observed samples and produces visually realistic colors.
  • Scene and ray simulation: 3R reconstructs scene geometry and uses inverse-square and Lambertian illumination laws based on distance, incident direction, and surface normals.
  • Scene and ray simulation: Virtual roadside lights are placed using semantic labels, with colors randomly sampled from the empirical prior.
  • Haze rendering: Haze is rendered from scene-depth-dependent transmission and integrated with illumination to form the nighttime hazy image.
  • Results and benchmark: 3R produces more realistic illumination than the compared method, while haze especially reduces contrast in distant regions.
  • Results and benchmark: The Nighttime Hazy Cityscapes dataset contains 2,750 images generated from 550 Cityscapes images under light, medium, and dense haze settings.

4 OS-MRP FOR NIGHTTIME DEHAZING

OS-MRP estimates nighttime color cast using a pixel-adaptive optimal scale, then sequentially performs color correction and dehazing. Its multiscale fusion is designed to balance reflectance coverage, localization, and computational efficiency.

  • 4.1 Optimal-scale maximum reflectance prior: Figure 4 maps the selected optimal scale, with hot colors indicating larger scales.The map shows small optimal scales for distinct-color or bright pixels and large scales for monochromatic pixels.
  • 4.1 Optimal-scale maximum reflectance prior: Small scales suit distinct-color or bright pixels, whereas monochromatic pixels require larger scales to obtain sufficient reflectance evidence.The optimal scale is sufficiently large to reach the highest reflectance probability without unnecessary loss of localization.
  • 4.1 Optimal-scale maximum reflectance prior: OS-MRP defines maximum reflectance across multiple patch scales and selects an optimal scale for each pixel.The prior addresses cases where a fixed local patch may fail, such as monochromatic regions.
  • 4.2 Initial multiscale fusion: The proposed pipeline first dehazes through multiscale fusion, then refines the clear image and color cast using optimal-scale fusion.The optimal-scale fusion uses the scale selected independently at each pixel.
  • 4.2 Initial multiscale fusion: OS-MRP computes color-cast estimates at multiple scales, averages them for an initial estimate, and refines the result with a fast guided filter.The method then removes color cast, re-estimates illuminance with MRP, estimates transmission with DCP, and recovers the haze-free image.
  • 4.3 Computational complexity analysis: OSFD reduces patch-operation costs to O(N), while total optimal-scale processing remains O(|S| N) and is faster after downsampling.The implementation uses overlapping sliding windows, summed-area tables, and downsampled images for scale-specific calculations.

5 A CNN-BASED BASELINE

ND-Net is a lightweight learning-based nighttime dehazing baseline using a MobileNet-v2 encoder and convolutional decoder. It combines pixel-level and perceptual supervision while leaving alternative backbones and specialized dehazing blocks for future work.

  • 5.1 Network structure: ND-Net adopts an encoder-decoder architecture with a MobileNet-v2 encoder and fully convolutional decoder for computational efficiency.The MobileNet-v2 backbone is selected because it has lightweight parameters and lower complexity.
  • 5.1 Network structure: The authors leave exploration of other backbones and specialized dehazing blocks for future work.The stated baseline is intended as a feasible learning-based solution and a validation of the synthetic benchmark.
  • 5.1 Network structure: The decoder contains five convolutional blocks and a convolutional prediction layer with residual and bottleneck branches.Each branch uses convolutional operations, with the bottleneck branch reducing and restoring feature dimensions.
  • 5.2 Training objectives: Training combines Mean Square Error and VGG-based perceptual losses against the ground-truth haze-free image.The perceptual term compares VGG feature maps using the L1 norm, while MSE uses the L2 norm.

6 EXPERIMENTS

Experiments benchmark nighttime dehazing and color-cast removal on synthetic and real-world datasets, showing that OSFD and ND-Net improve image quality while OSFD also offers efficient runtime. The study also documents method-specific artifacts and limitations under difficult conditions.

  • Experimental settings: The evaluation uses synthetic benchmark datasets, 150 real-world nighttime hazy images, and 1,500 real-world daytime clear images with PSNR, SSIM, and CIEDE2000 metrics.The datasets support nighttime dehazing and color-removal evaluation.
  • Nighttime dehazing: As haze density increases, removing color cast and haze becomes more difficult, and all compared methods degrade.NDIM achieves the worst scores across all metrics in the reported comparisons.
  • Nighttime dehazing: OSFD achieves the best scores among prior-based methods and outperforms MRP by about 0.05 SSIM on NHC-L, NHC-M, and NHC-D.OSFD also removes haze, recovers details, and preserves local structures efficiently.
  • Nighttime dehazing: ND-Net achieves the best scores across all metrics, while its NHR-trained model shows good generalization on the NHC datasets.Its visual results contain fewer color, illumination, and noise artifacts, although residual haze remains.
  • Color cast removal: OSFD achieves the best color-removal performance by selecting an optimal scale for estimating color cast across diverse local statistics.This matters because the dark channel prior fails when color cast biases the dark channel.
  • Runtime: NDIM, GS, MRPF, MRP, OSFD, and ND-Net process a 512x512 image in 5.63s, 22.52s, 0.236s, 1.769s, 0.576s, and 0.0074s, respectively.OSFD is reported as 3x faster than MRP and 10x faster than NDIM and GS.
  • Limitations: Synthetic images may contain depth-induced artifacts, while OSFD may produce color artifacts or residual haze in monochromatic, dense-haze, or residual-color-cast regions.The reported causes include inaccurate depth, prior failure, inaccurate scale selection, and disentangled dehazing affected by residual color cast.

7 CONCLUSION

The paper introduces 3R for realistic nighttime-hazy image synthesis and proposes OS-MRP-based OSFD alongside a CNN baseline. Experiments report superiority over state-of-the-art methods in image quality and runtime.

  • Contributions: 3R uses scene geometry and real-world light colors to generate realistic nighttime hazy images and construct a benchmark for haze removal, color correction, and runtime.The benchmark evaluates state-of-the-art nighttime dehazing methods across these dimensions.
  • Contributions: OS-MRP adapts to varying local image statistics, enabling sequential color correction and haze removal through a computationally efficient dehazing method.The prior selects an optimal scale for local statistics.
  • Contributions: The CNN-based baseline shows good dehazing and generalization ability, while the proposed methods outperform state-of-the-art methods in image quality and runtime.The conclusion reports this result across the paper’s extensive experiments.

8.1 More synthetic results generated by the proposed 3R

Additional experiments show that 3R produces more varied and controllable nighttime haze, illumination, and color conditions than the compared synthesis approach. Its geometry-aware rendering and parameter controls support benchmark generation and potential training use.

  • Visual comparison: 3R produces more realistic light-color variation than the compared method by sampling colors from a real-world prior distribution.Its images can vary from light yellow to warm red, with different color casts across image areas.
  • Visual comparison: 3R reduces the strong vignetting associated with a single central light source by calculating illuminance using scene geometry and Lambert’s cosine law.The calculation uses surface normals, incident light direction, and light-path distance.
  • Dataset applicability: The method can synthesize nighttime hazy images on datasets such as RESIDE and Virtual KITTI, although Virtual KITTI results are excluded from the benchmark because its rendered textures are not sufficiently realistic.Virtual KITTI also lacks nighttime hazy imaging conditions in the original dataset.
  • Real-world visual results: On NHM and NHR, OSFD outperforms the compared methods visually, while ND-Net generally achieves the best performance on both datasets.Some NHM images still produce overdehazed results with more saturated colors because of shallow depth of field and colorful objects.
  • Controllable synthesis: Changing β_t, η, and β_l enables controllable synthesis of haze density, light colors, and illuminance intensity.Increasing β_t produces denser haze, changing sampled colors produces diverse color casts, and increasing β_l makes images darker.

8.2 Comparison with state-of-the-art dehazing methods

The color-removal comparison shows that OSFD’s adaptive scale selection handles monochromatic regions and preserves intrinsic colors more effectively than MRP in the presented visual results.

  • Visual comparison: OSFD handles monochromatic areas such as cars and trees while avoiding MRP’s whitish effect or residual color cast.Daytime clear images without color cast also show OSFD retaining intrinsic image colors.

8.3 Hyper-parameter settings

The paper studies ND-Net training settings and OSFD’s number of scales, selecting defaults based on dehazing quality, color-cast removal, efficiency, and result stability.

  • ND-Net settings: Larger input images, longer training, and perceptual loss improve ND-Net dehazing results, so the final-row configuration is used by default.The study varies training epochs, batch size, input image size, and loss choice.
  • OSFD scales: More OSFD scales lower CIEDE2000 scores, with the largest improvement from 1–5 scales before performance saturates.The study uses the NHC-L dataset and selects 10 scales for later experiments.
  • OSFD scales: OSFD uses 10 scales in subsequent experiments because this setting balances computational efficiency with reducing annoying color artifacts.The choice follows the observed saturation in color-cast removal performance.
  • Evaluation stability: NHC evaluation scores are stable across random light-color changes, with low standard deviations averaged over five augmented tests and 550 images.Cityscapes was augmented five times, and each evaluation used all 550 images.
Loading 2008.03864v3…