PaperlayerView exact revision on arXiv

Source-linked AI summary

DEA-Net: Single image dehazing based on detail-enhanced convolution and content-guided attention

Zixuan Chen, Zewei He, Zhe-Ming Lu

arXiv:2301.04805v1cs.CV

TL;DR

Single-image dehazing must recover clean images from hazy observations, while CNN feature learning remains limited by conventional convolutions and incomplete attention to haze non-uniformity. DEA-Net addresses this with prior-informed DEConv, channel-specific CGA, and CGA-based fusion. Across benchmark comparisons, DEA-Net variants achieve leading restoration results, including 39.16 dB PSNR and 0.9921 SSIM for DEA-Net-S on SOTS-indoor.

  • Problem

    Single-image dehazing remains challenging because vanilla convolutions underuse useful priors and existing attention can overlook channel-wise haze differences.

  • Method

    DEA-Net combines detail-enhanced convolution, coarse-to-fine content-guided channel-specific attention, and CGA-based mixup fusion in an encoder-decoder architecture.

  • Results

    39.16 dB PSNR and 0.9921 SSIM were achieved by DEA-Net-S on SOTS-indoor, while DEA-Net-CR ranked first across comparisons on SOTS and Haze4K.

  • Takeaways & Limitations

    The combined design provides a dehazing network that reports accurate haze-free reconstruction with faster inference and competitive benchmark performance.

Abstract

from arXiv · show

Single image dehazing is a challenging ill-posed problem which estimates latent haze-free images from observed hazy images. Some existing deep learning based methods are devoted to improving the model performance via increasing the depth or width of convolution. The learning ability of convolutional neural network (CNN) structure is still under-explored. In this paper, a detail-enhanced attention block (DEAB) consisting of the detail-enhanced convolution (DEConv) and the content-guided attention (CGA) is proposed to boost the feature learning for improving the dehazing performance. Specifically, the DEConv integrates prior information into normal convolution layer to enhance the representation and generalization capacity. Then by using the re-parameterization technique, DEConv is equivalently converted into a vanilla convolution with NO extra parameters and computational cost. By assigning unique spatial importance map (SIM) to every channel, CGA can attend more useful information encoded in features. In addition, a CGA-based mixup fusion scheme is presented to effectively fuse the features and aid the gradient flow. By combining above mentioned components, we propose our detail-enhanced attention network (DEA-Net) for recovering high-quality haze-free images. Extensive experimental results demonstrate the effectiveness of our DEA-Net, outperforming the state-of-the-art (SOTA) methods by boosting the PSNR index over 41 dB with only 3.653 M parameters. The source code of our DEA-Net will be made available at https://github.com/cecret3350/DEA-Net.

I. INTRODUCTION

The paper targets limitations of conventional CNN feature learning and attention for single-image dehazing by combining prior-informed convolution, channel-specific attention, and feature fusion in DEA-Net.

  • Motivation: Single-image dehazing recovers clean scenes from hazy images whose contrast and color distortions can degrade subsequent vision tasks.The task is framed as a fundamental low-level restoration problem and potential preprocessing step.
  • Challenges: Existing dehazing CNNs often use vanilla convolutions without explicit priors, while transformer alternatives require complex training and substantial computation.The paper identifies embedding designed priors into CNNs as a target direction.
  • Challenges: Haze non-uniformity occurs across image regions and feature channels, motivating attention that models spatial importance beyond a single shared map.The stated problem includes uneven haze distribution at both image and feature levels.
  • Proposed Method: DEConv combines parallel vanilla and difference convolutions to encode prior information, then re-parameterizes them as one convolution without extra parameters or computational cost.The design uses four difference convolutions and one vanilla convolution during feature extraction.
  • Proposed Method: CGA generates channel-specific spatial importance maps in a coarse-to-fine manner, using input features to emphasize significant regions in each channel.The attention mechanism first produces coarse maps and then refines them according to individual feature channels.
  • Proposed Method: DEA-Net combines DEConv, CGA, and CGA-based mixup fusion in a U-Net-like architecture for reconstructing high-quality haze-free images.The fusion scheme joins low-level encoder features with corresponding high-level features.

A. Single Image Dehazing

Single-image dehazing methods generally follow prior-based or data-driven approaches, while difference convolutions derive feature representations from local pixel differences.

  • Method Categories: Single-image dehazing methods are mainly categorized as prior-based methods or data-driven methods.Prior-based methods manually generalize statistical discrepancies between hazy and haze-free images, whereas data-driven methods learn mappings from large-scale datasets.
  • Difference Convolutions: Difference convolutions originate from local binary pattern representations of pixel differences and were later incorporated into convolutional feature extraction.The related-work passage connects LBP, local binary convolution, and central difference convolution.

III. METHODOLOGY

DEA-Net uses a three-level encoder-decoder-like architecture centered on detail-enhanced convolutions and content-guided attention. Its feature fusion scheme addresses receptive-field mismatch while re-parameterization preserves vanilla-convolution inference cost.

  • Overall Architecture: DEA-Net has encoder, feature-transform, and decoder parts, using DEBs in the encoder and decoder and stacked DEABs in the feature-transform part.The hierarchical network contains three levels and restores a haze-free image from a hazy input.
  • Detail-enhanced Convolution: DEConv combines one vanilla convolution with four difference convolutions to encode intensity- and gradient-level information.The difference convolutions are CDC, ADC, HDC, and VDC; the latter two incorporate horizontal and vertical gradient priors.
  • Detail-enhanced Convolution: Re-parameterization converts the five parallel convolutions into one vanilla convolution, retaining richer features with no extra parameters or inference computation.During forward propagation, corresponding kernel weights are added to form the converted kernel.
  • Content-guided Attention: CGA generates coarse spatial importance maps and refines them with input-feature guidance to assign a unique map to each channel.This channel-specific refinement emphasizes significant regions and useful feature information.
  • CGA-based Mixup Fusion Scheme: The CGA-based mixup scheme adaptively fuses encoder low-level features with corresponding high-level features using learned spatial weights.It targets receptive-field mismatch before fusion and uses skip connections to ease gradient propagation.

D. Overall Architecture

DEA-Net combines DEConv, CGA-based feature fusion, and a three-level encoder-decoder architecture to recover haze-free images. It uses multiscale feature processing, skip-style fusion, and L1 training loss.

  • D. Overall Architecture: DEA-Net combines DEConv, CGA, and CGA-based mixup fusion in a three-level U-Net-like encoder-decoder architecture.The network contains encoder, feature-transform, and decoder parts with two down-sampling and two up-sampling operations.
  • D. Overall Architecture: Feature extraction is deployed across all three levels, using DEB at levels 1 and 2 and DEAB at level 3.This differs from methods that transform features only in low-resolution space, which the paper associates with information loss in detail-sensitive dehazing.
  • D. Overall Architecture: The predicted haze-free image is trained against ground truth with pixel-wise L1 loss, or mean absolute error.The loss minimizes the pixel-wise difference between the prediction J and corresponding ground truth GT.

A. Datasets and Metrics

DEA-Net is evaluated on synthetic and real-captured dehazing data, including RESIDE subsets and Haze4K. Performance is measured with PSNR and SSIM on uncropped RGB images.

  • A. Datasets and Metrics: DEA-Net is trained on RESIDE ITS and OTS and tested on the corresponding SOTS-indoor and SOTS-outdoor subsets.Each SOTS subset contains 500 testing images; ITS contains 1,399 indoor clean images with 10 simulated hazy images per clean image, while around 296K OTS images are selected for training.
  • A. Datasets and Metrics: Haze4K provides an additional evaluation setting with 3,000 synthetic training images and 1,000 synthetic testing images.Real-captured hazy images are also used to further verify performance on real scenes.
  • A. Datasets and Metrics: PSNR and SSIM are used to evaluate dehazing image quality on RGB images without cropping pixels.The metrics are computed for fair comparison across methods.

B. Implementation Details

The implementation uses PyTorch on one RTX3080Ti GPU, with specified block counts, Adam optimization, and fixed training hyperparameters.

  • B. Implementation Details: DEA-Net is implemented in PyTorch and trained on a single NVIDIA RTX3080Ti GPU.DEB, DEB, and DEAB are deployed at levels 1, 2, and 3, respectively.
  • B. Implementation Details: The block counts across stages [N1, N2, N3, N4, N5] are set to.The model uses Adam with default β1, β2, ε values of 0.9, 0.999, and 1e-8.
  • B. Implementation Details: The initial learning rate is 1e-4 and the batch size is 16.The passage also specifies cosine-related training settings, but does not provide the remaining schedule details.

C. Ablation Study

The ablation study separately examines DEConv, CGA, and the CGA-based mixup fusion scheme to assess their effects in DEA-Net.

  • C. Ablation Study: The ablation experiments investigate the designs and effects of DEConv, CGA, and the CGA-based mixup fusion scheme.The study analyzes the contribution of each component separately.
  • C. Ablation Study: The ablation study is structured to evaluate the contribution of each proposed component.The supplied passage identifies contribution analysis as the purpose of the ablation experiments.

1) DEConv:

DEConv replaces selected vanilla convolutions with parallel difference-convolution branches that incorporate local descriptors, then re-parameterizes them into a standard convolution without added inference cost.

  • DEConv: DEConv replaces the first vanilla convolution in RB and FAB blocks with the proposed detail-enhanced convolution.The resulting variants are named RBw/ DEConv and FABw/ DEConv.
  • DEConv: The ablation comparisons use the same six cascaded blocks, fusion scheme, training budget, and SOTS-Indoor test dataset for fair evaluation.The reported ablation settings use 500K iterations and an initial learning rate of 2e−4.
  • DEConv: 0.27 and 0.6 dB PSNR improvements result from replacing vanilla convolution with DEConv in RB and FAB, respectively.The comparison uses the corresponding baseline blocks on SOTS-Indoor.
  • DEConv: Re-parameterization converts the parallel DEConv structure into a standard 3 × 3 convolution by adding learned kernel weights at corresponding positions.This transformation is evaluated for parameters, FLOPs, and inference time in Model FAB D.
  • DEConv: DEConv uses parallel convolution layers, including difference convolutions, to extract richer features by embedding traditional prior information.The explored designs show that adding a parallel vanilla branch can reduce performance, whereas adding a parallel CDC branch improves it.

2) CGA:

CGA generates channel-specific spatial importance maps so each feature channel can emphasize regions relevant to its own content, unlike single-map attention.

  • CGA: CGA refines coarse spatial importance maps using input-feature content to produce a distinct map for every channel.The mechanism uses spatial and channel attention to generate initial maps before channel-wise refinement.
  • CGA: Single-channel spatial attention in FAM and CBAM neglects feature-channel specificity, whereas CGA models channel-wise haze distribution differences.This comparison motivates CGA’s channel-specific spatial weighting.
  • CGA: Model DEAB exceeds Model FAB D and Model FAB D CBAM by 1.5 dB and 1.01 dB PSNR, respectively.These results are reported in the right three columns of Table I.
  • CGA: Visual comparisons show that CGA learns more accurate spatial weights than FAM, whose one-channel map can mix contour patterns into haze-region estimates.Figure 8 displays eight randomly selected CGA maps alongside the corresponding processing results.

3) CGA-based Mixup Fusion Scheme:

The CGA-based mixup fusion scheme combines low-level encoder features with corresponding high-level features, and its ablation isolates the fusion choice from other components.

  • CGA-based Mixup Fusion Scheme: The ablation compares CGA-based mixup with AECR-Net mixup and element-wise addition within Model DEAB.The models are referred to as Model DEAB, Model DEAB A, and Model DEAB C.
  • CGA-based Mixup Fusion Scheme: Element-wise addition achieves performance very similar to the baseline mixup scheme, differing by 0.06 dB in PSNR.The supplied passage reports this comparison on SOTS-Indoor.
  • CGA-based Mixup Fusion Scheme: Adding residual blocks in levels 1 and 2 improves PSNR by 2.52 dB, indicating a substantial contribution from high-resolution feature processing.The resulting simplified DEA-Net-S reaches 39.16 dB PSNR and 0.9921 SSIM.

D. Comparisons with SOTA Methods

DEA-Net is compared with earlier and recent dehazing methods on SOTS-Indoor, SOTS-Outdoor, and Haze4K using restoration quality, efficiency, and visual results.

  • Quantitative Analysis: DEA-Net variants achieve leading quantitative results across SOTS and Haze4K, with DEA-Net-CR ranking first in all reported comparisons.DEA-Net-S reaches 39.16 dB PSNR and 0.9921 SSIM on SOTS-Indoor; DEA-Net and DEA-Net-CR obtain the best Haze4K SSIM values of 0.9869 and 0.9885.
  • Quantitative Analysis: DEA-Net variants run fastest among recent SOTA methods while maintaining an acceptable parameter and FLOP budget.Any DEA-Net variant ranks second best in both parameter count and FLOPs; FLOPs and runtime use 256 × 256 color images.
  • Qualitative Analysis: On synthetic SOTS-Indoor, DEA-Net produces sharper contours and edges with fewer haze residuals than previous SOTA methods.The visual comparison is presented in Figure 9.
  • Qualitative Analysis: On SOTS-Outdoor, DEA-Net outputs are described as closest to ground truth, while real-image results show fewer haze remnants, color deviations, and artifacts.The outdoor and real-world comparisons are shown in Figures 10 and 11.
  • Conclusion: The conclusion attributes DEA-Net’s state-of-the-art quantitative and qualitative results to DEConv, channel-specific CGA, and CGA-based feature fusion.DEConv is re-parameterized without extra parameters or computational cost.
Loading 2301.04805v1…