PaperlayerView exact revision on arXiv

Source-linked AI summary

FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network

Jing Zhang, Dacheng Tao

arXiv:1906.04334v2cs.CVcs.LG

TL;DR

Single-image dehazing is an ill-posed restoration problem important for downstream vision tasks, while existing models may be inefficient or limited in representation capacity. FAMED-Net uses a lightweight three-scale, point-wise end-to-end architecture to learn haze-free images directly. Experiments report superiority over representative models across complexity, efficiency, restoration accuracy, and cross-set generalization.

  • Problem

    Single-image dehazing remains ill-posed, while existing models can be overcomplex, computationally inefficient, or limited in representation capacity.

  • Method

    FAMED-Net combines three-scale encoders and a fusion module with densely connected point-wise convolutions and pooling layers to learn haze-free images directly.

  • Results

    FAMED-Net outperforms representative state-of-the-art models on synthetic benchmarks and real-world hazy images across model complexity, computational efficiency, restoration accuracy, and cross-set generalization.

  • Takeaways & Limitations

    FAMED-Net provides a fast, lightweight, and accurate deep architecture for single-image dehazing.

  • Takeaways & Limitations

    Transmission estimates can be incorrect in sky regions, producing under-dehazed artifacts; incorporating high-level semantics is left for future work.

Abstract

from arXiv · show

Single image dehazing is a critical image pre-processing step for subsequent high-level computer vision tasks. However, it remains challenging due to its ill-posed nature. Existing dehazing models tend to suffer from model overcomplexity and computational inefficiency or have limited representation capacity. To tackle these challenges, here we propose a fast and accurate multi-scale end-to-end dehazing network called FAMED-Net, which comprises encoders at three scales and a fusion module to efficiently and directly learn the haze-free image. Each encoder consists of cascaded and densely connected point-wise convolutional layers and pooling layers. Since no larger convolutional kernels are used and features are reused layer-by-layer, FAMED-Net is lightweight and computationally efficient. Thorough empirical studies on public synthetic datasets (including RESIDE) and real-world hazy images demonstrate the superiority of FAMED-Net over other representative state-of-the-art models with respect to model complexity, computational efficiency, restoration accuracy, and cross-set generalization. The code will be made publicly available.

I. INTRODUCTION

Single-image dehazing is a challenging ill-posed restoration problem that matters for downstream vision tasks. FAMED-Net addresses this with a lightweight multi-scale end-to-end architecture and outperforms representative state-of-the-art models across synthetic and real-world images.

  • Haze causes low-contrast, blurry, and noisy images that can hinder object detection and semantic segmentation.
  • Single-image dehazing is practically useful because it requires no additional scene depth, polarization, or image sequence, but remains ill-posed.
  • FAMED-Net uses three-scale encoders and a fusion module to directly learn the haze-free image end-to-end.
  • Fully point-wise convolutions, densely reused features, and pooling layers make FAMED-Net lightweight and computationally efficient.
  • FAMED-Net outperforms representative state-of-the-art models on synthetic benchmarks and real-world hazy images.

II. RELATED WORK

Prior-based dehazing estimates haze structure using assumptions such as transmission, atmospheric light, and dark-channel statistics, while learning-based methods learn mappings from data. End-to-end models increasingly learn scene radiance directly rather than estimating transmission first.

  • The atmospheric scattering model represents a hazy image as attenuated scene radiance plus scattered atmospheric light.
  • Transmission denotes the fraction of scene radiance reaching the camera and depends on scene depth; under homogeneous haze, it is controlled by attenuation coefficient and depth.
  • AOD-Net reformulates transmission and atmospheric light into a single variable K and learns a direct mapping from a raw hazy image to scene radiance.
  • The dark channel prior uses local minimum channel values to estimate transmission, but its assumptions can fail for bright objects resembling atmospheric light.
  • Deep CNN dehazing methods either estimate transmission or directly learn scene radiance end-to-end, with FAMED-Net belonging to the latter category.

C. Multi-scale pyramid architecture

The paper frames dehazing through multi-scale feature representation and a probabilistic formulation, using point-wise convolutions to learn statistical regularities efficiently.

  • Multi-scale representation: Pyramid architectures support multiresolution and multi-scale feature representations in computer vision.
  • Deep supervision: Deep supervision adds intermediate dehazed-image supervision at each scale to facilitate multi-scale and multi-level feature learning.
  • Probabilistic formulation: The dehazing formulation models the observed degraded image as y = x + z, where x is haze-free and z is an intermediate degradation-related variable.
  • Probabilistic formulation: Using L2 supervision corresponds to modeling reconstruction error with a normal distribution, while priors constrain latent degradation-related variables.
  • Point-wise modeling: The model learns statistical regularities with compact point-wise convolutions, motivating an end-to-end fully point-wise CNN for single-image dehazing.

B. The Single-scale FAMED-Net: FAMED-Net-SS

FAMED-Net-SS learns a latent K variable with a fully point-wise CNN, reusing features across receptive fields and aggregating multi-level statistics through pooling.

  • Network design: FAMED-Net-SS uses five point-wise convolutional layers, with the first four forming the K-encoder and the fifth serving as the decoder.
  • Network design: Dense connections reuse features from different receptive fields throughout the network.
  • Network design: Each block combines convolution, batch normalization, ReLU, and pooling to map densely connected feature inputs.
  • Multi-level aggregation: Pooling kernels with sizes r_l × r_l, where r_l = 2l−1 for l ∈ [1, 4], aggregate multi-level statistics within receptive fields.
  • Training: The network is trained with L2 loss using estimated scene radiance and weight decay regularization.

C. The Multi-scale Variants of FAMED-Net: FAMED-Net-GP and FAMED-Net-LP

FAMED-Net extends the single-scale design with Gaussian and Laplacian pyramid variants that combine multi-scale K estimates before recovering scene radiance.

  • FAMED-Net-GP: The Gaussian-pyramid variant processes the hazy image at original, 1/2, and 1/4 scales using separate K-encoders without shared weights.
  • FAMED-Net-GP: Coarse-scale K maps are interpolated to the original scale, concatenated, and fused by a 1 × 1 convolution followed by ReLU.
  • FAMED-Net-GP: The fused K estimate is used to recover scene radiance, defining the Gaussian-pyramid structure FAMED-Net-GP.
  • Training: Multi-scale training uses L2 supervision for each scale and for the fused scene-radiance estimate, with all loss weights set to 1.
  • FAMED-Net-LP: In the Laplacian-pyramid variant, an interpolated coarse K map is added to the finer-scale encoder output, enforcing residual learning at finer scales.

D. Model Complexity Analysis

FAMED-Net is designed to remain lightweight and computationally efficient through fully point-wise convolutions, while supporting arbitrary image sizes and fixed-size testing.

  • Complexity: FAMED-Net-SS contains 5,987 learnable parameters and 9.39x107 FLOPs.
  • Complexity: Moving to FAMED-Net-GP triples the parameter count while increasing FLOPs by about 30%.
  • Scalability: The fully convolutional structure permits arbitrary image sizes, with computational cost increasing linearly with image size.
  • Testing strategy: Fixed-size testing resizes the longest image side to 360, then restores the estimated K map to the original size by bilinear interpolation and refinement.
  • Comparison: Table II compares FAMED-Net and state-of-the-art models using parameters, model size, and runtime.

IV. EXPERIMENTS

Experiments evaluate FAMED-Net against prior-based and deep CNN methods on RESIDE, using synthetic benchmark scores, real-world comparisons, and ablations. Training uses RESIDE’s large synthetic datasets and 128×128 patches, with implementation settings and architecture variants documented.

  • Evaluation setup: RESIDE evaluation compares FAMED-Net with image prior-based and deep CNN-based dehazing methods.The benchmark reports PSNR and SSIM on SOTS and includes subjective evaluation on real-world hazy images.
  • Datasets: RESIDE provides 110,500 synthetic indoor and 313,950 synthetic outdoor hazy training images, while SOTS contains 500 indoor and 500 outdoor test images.
  • Training protocol: FAMED-Net training uses 128×128 randomly cropped patches, with depth-level statistics analyzed across the training patches.
  • Training protocol: FAMED-Net is trained for 400,000 iterations, using a batch size of 48 and a learning rate reduced after 200,000 and 320,000 iterations.
  • Ablation design: The experiments include basic-architecture ablations, training-data and iteration studies, and variant comparisons on RESIDE TestSet-S.

1) Ablations on the Basic Architecture:

Ablations show that wider features, batch normalization, multi-scale processing, and sufficient training data improve FAMED-Net, while added structural convolutions improve accuracy at modest computational cost. The Gaussian-pyramid model is selected as the accuracy–complexity compromise.

  • Basic architecture: Batch normalization substantially improves FAMED-Net-FD4 over FAMED-Net-NoBN and also accelerates convergence.
  • Feature channels: FAMED-Net-S gains 0.3 dB and 0.024 SSIM over FAMED-Net-FD4, and 1.5 dB and 0.06 SSIM over FAMED-Net-NoBN.
  • Multi-scale design: Adding a down-scale branch improves PSNR by 0.2 dB with only a marginal SSIM decrease, while the three-scale FAMED-Net-GP is the best architecture.
  • Training data and iterations: Training FAMED-Net-GP on all 424,450 ITS and OTS images yields gains of 2.14 dB and 0.0425 SSIM over the smaller training setup.
  • Pyramid variants: FAMED-Net-LP performs comparably to FAMED-Net-GP, with marginally lower PSNR and marginally higher SSIM, so FAMED-Net-GP remains the default.

3) The Effectiveness of Max Pooling:

Max pooling substantially improves the multi-scale model while preserving its lightweight design, and FAMED-Net achieves the strongest reported SOTS comparison alongside favorable qualitative results. The architecture uses point-wise convolutions, multi-scale processing, and dense feature reuse.

  • Max pooling: FAMED-Net-GP-MaxP gains 0.83 dB and 0.0091 SSIM over average pooling and outperforms FAMED-Net-GP-3x3 by 0.57 dB with nearly identical SSIM.
  • Efficiency: FAMED-Net-GP-MaxP contains 17,991 parameters and 1.24×10^8 FLOPs, and is selected as the representative FAMED-Net model.
  • Quantitative comparison: FAMED-Net surpasses the second-best SOTS methods by 3.6 dB and 0.05 SSIM.
  • Quantitative comparison: CNN-based methods generally outperform image prior-based methods, while architecture choices such as point-wise convolution and end-to-end learning affect performance.
  • Design conclusions: The authors attribute the compact model to point-wise convolutions, layer-by-layer feature aggregation, end-to-end modeling, multi-scale design, and dense feature reuse.
  • Subjective evaluation: On synthetic hazy images, competing methods exhibit residual haze, over-saturation, color distortions, or noise, whereas FAMED-Net produces comparatively cleaner results.
  • Subjective evaluation: On real-world images, FAMED-Net produces better or comparable results with clear details and fewer color artifacts than representative state-of-the-art methods.

3) Cross-set Generalization:

FAMED-Net is evaluated for cross-set generalization without fine-tuning and for learned statistical regularities on haze-free images. It generalizes better than GFN and DCPDN and avoids the over-dehazing tendency associated with AOD-Net’s learned statistics.

  • Cross-set generalization: Using pretrained models without fine-tuning, FAMED-Net shows better cross-set generalization than GFN and DCPDN on RESIDE SOTS and TestA.
  • Learned regularities: On haze-free images, FAMED-Net learns a more effective statistical regularity than DCP, FPCNet, and AOD-Net.
  • Learned regularities: AOD-Net’s statistics remain far from zero on haze-free images, implying that it implicitly assumes residual haze and produces over-dehazed artifacts.
  • Transmission visualization: The transmission visualization uses warm colors for high transmission, corresponding to near-camera regions with small depth.

5) Runtime Analysis:

FAMED-Net is evaluated as a lightweight, efficient dehazing architecture, while the discussion identifies limitations in its learned statistical regularity and current handling of heterogeneous illumination and haze.

  • FAMED-Net is described as outperforming prior methods while using a more efficient architecture.The paper frames this advantage in the context of its limitations discussion.
  • FAMED-Net implicitly learns locally statistical regularity for dehazing, similarly to prior- and learning-based methods.
  • Transmission estimates are incorrect in sky regions, producing under-dehazed artifacts.The authors suggest incorporating high-level semantics and jointly modeling enhancement with high-level understanding as future work.
  • The authors propose extending FAMED-Net to non-uniform atmospheric light and heterogeneous haze using illumination modeling and region-based techniques.Examples include artificial ambient light in nighttime haze and haze-density-aware segmentation.
  • Experiments on benchmark datasets and real-world hazy images support FAMED-Net as a fast, lightweight, and accurate dehazing architecture.

A. Illumination Balancing Network

IBNet adapts FAMED-Net to estimate balanced illumination for dehazed images, with experiments reporting accurate restoration and visually improved illumination, detail, and comparisons against competing methods.

  • Motivation: Dehazed images can appear dim because scene radiance is usually less bright than atmospheric light, motivating illumination balancing.The stated goals are more pleasing appearance and support for subsequent high-level tasks.
  • Network design: IBNet estimates a balanced illumination map and replaces the input image’s unbalanced illumination distribution with that estimate.It is constructed from FAMED-Net with a one-channel output and without the recovery module.
  • Dataset preparation: IBNet is trained and evaluated on illumination-unbalanced RESIDE images generated by applying image-specific cubic mappings to clear-image illumination channels.
  • Results: IBNet achieved good restoration accuracy on RESIDE TestSet-S for illumination balancing.The supplied passage references PSNR and SSIM results in Table VIII but does not provide their values.
  • Comparisons: On real-world hazy images, FAMED-Net produced better or comparable results to state-of-the-art methods, with clear details, fewer color artifacts, and high sky-region fidelity.
  • Comparisons: Comparisons on haze-free images indicate more effective learned statistical regularity than DCP, FPCNet, and AOD-Net.
Loading 1906.04334v2…