PaperlayerView exact revision on arXiv

Source-linked AI summary

MixDehazeNet : Mix Structure Block For Image Dehazing Network

LiPing Lu, Qian Xiong, DuanFeng Chu, BingRong Xu

arXiv:2305.17654v1cs.CV

TL;DR

Existing dehazing methods can overlook image multi-scale characteristics with large kernels, while prior attention designs do not adequately handle uneven haze distribution. MixDehazeNet uses a five-stage U-net with mix blocks that pair multi-scale parallel large kernels and enhanced parallel attention. Across RESIDE-IN, RESIDE-OUT, and RESIDE-6k, MixDehazeNet-L outperforms previous methods in PSNR and SSIM and is reported as the first to exceed 42 dB PSNR on RESIDE-IN.

  • Problem

    Existing dehazing methods can overlook image multi-scale characteristics with large kernels, while prior attention designs do not adequately handle uneven haze distribution.

    Source: §1 · p. 2

  • Method

    MixDehazeNet uses a five-stage U-net with mix blocks that pair multi-scale parallel large kernels and enhanced parallel attention.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Across RESIDE-IN, RESIDE-OUT, and RESIDE-6k, MixDehazeNet-L outperforms previous methods in PSNR and SSIM and is reported as the first to exceed 42 dB PSNR on RESIDE-IN.

    Source: §4.3 · p. 7, §5 · p. 9

  • Takeaways & Limitations

    MixDehazeNet-L outperforms all previous state-of-the-art methods in PSNR and SSIM on RESIDE-IN, RESIDE-OUT, and RESIDE-6k.

    Source: §4.3 · p. 7

Abstract

from arXiv · show

Image dehazing is a typical task in the low-level vision field. Previous studies verified the effectiveness of the large convolutional kernel and attention mechanism in dehazing. However, there are two drawbacks: the multi-scale properties of an image are readily ignored when a large convolutional kernel is introduced, and the standard series connection of an attention module does not sufficiently consider an uneven hazy distribution. In this paper, we propose a novel framework named Mix Structure Image Dehazing Network (MixDehazeNet), which solves two issues mentioned above. Specifically, it mainly consists of two parts: the multi-scale parallel large convolution kernel module and the enhanced parallel attention module. Compared with a single large kernel, parallel large kernels with multi-scale are more capable of taking partial texture into account during the dehazing phase. In addition, an enhanced parallel attention module is developed, in which parallel connections of attention perform better at dehazing uneven hazy distribution. Extensive experiments on three benchmarks demonstrate the effectiveness of our proposed methods. For example, compared with the previous state-of-the-art methods, MixDehazeNet achieves a significant improvement (42.62dB PSNR) on the SOTS indoor dataset. The code is released in https://github.com/AmeryXiong/MixDehazeNet.

1. Introduction

The introduction identifies two gaps in dehazing—large kernels can overlook image multi-scale structure, and existing attention designs do not adequately address uneven haze—and introduces MixDehazeNet to address them. It reports state-of-the-art results across multiple datasets.

1. Introduction

  • Haze degrades human recognition and can reduce the performance of high-level vision models, motivating single-image dehazing as a possible preprocessing step.The stated goal of single-image dehazing is to recover a clear scene from its matching hazy image.Source: §1 · p. 1
  • The authors identify two gaps: large receptive fields can overlook image multi-scale characteristics, while prior separate or serial attention designs do not adequately address uneven haze.They associate channel attention with global atmospheric light A and pixel attention with location-dependent transmission t(x).Source: §1 · p. 2
  • MSPLCK combines multi-scale parallel large kernels and large receptive fields, using large convolutions for hazy regions and small convolutions to restore texture detail.The authors also describe parallel dilated convolution as providing large receptive fields and long-distance modeling.Source: §1 · p. 2
  • EPA jointly extracts shared global and location-dependent local information through parallel channel and pixel attention to address uneven haze distribution.The module fuses simple pixel attention, channel attention, and pixel attention through a multi-layer perception.Source: §1 · p. 2
  • MixDehazeNet achieves state-of-the-art results on multiple dehazing datasets, and MixDehazeNet-L is the first model to exceed 42 dB PSNR on SOTS indoor.Figure 1 compares dehazing methods on SOTS indoor; circle size represents parameter count and MACs use a logarithmic axis.Source: Figure 1 · p. 1, §1 · p. 2

2. Related Work

The related work reviews prior-based and learned dehazing methods, then discusses large-kernel designs and their computational considerations.

Prior-based image dehazing

  • Prior-based methods estimate haze information from image priors, but restoration can be distorted when the prior assumptions are not met.Examples include DCP and Rank-one Prior.Source: §2 · pp. 2–3

Learning-based image dehazing

  • Learned dehazing methods progressed from estimating transmission and atmospheric light to directly predicting haze-free images, as in GridDehazeNet.DehazeNet and MSCNN estimate transmission while using prior-based methods for atmospheric light.Source: §2 · p. 3
  • Learned methods also use channel and pixel attention in FFA-Net, contrastive learning in AECR-Net [40], density maps in PMNet [41], and uncertainty modules in UDN [17].Source: §2 · p. 3
  • Transformer-based dehazing offers global modeling and large receptive fields but is described as parameter-heavy, costly to train, and high-latency.The authors therefore focus on CNNs and use large dilated convolutions for receptive fields and long-distance modeling.Source: §2 · p. 3

Large convolutional kernels

  • RepLKNet proposes using a few large kernels instead of stacks of small ones; the Visual Attention Network decomposes large convolution into spatial-local, spatial-long-range, and channel components.The cited decomposition is presented as a way to address computational overhead and parameter count.Source: §2 · p. 3

3. MixDehazeNet

MixDehazeNet is a five-stage U-net whose Mix Structure Blocks combine multi-scale parallel large convolutions with enhanced parallel attention. The network also uses SK Fusion for skip and main branches and soft reconstruction in place of a global residual.

3. MixDehazeNet

  • MixDehazeNet uses a five-stage U-net with Mix Structure Blocks, SK Fusion for skip/main branches, and soft reconstruction instead of a global residual.The network predicts a dehazed image from the hazy input.Source: §3 · p. 3

3.1. Multi-Scale Parallel Large Convolution Kernel

  • MSPLCK normalizes features, combines three parallel dilated convolution scales, and uses a two-pointwise-convolution GELU MLP with an identity shortcut.Large and medium branches model long-distance features and large haze regions; the small branch targets small haze areas and texture details.Source: §3.1 · pp. 3–4

3.2. Enhanced Parallel Attention

  • EPA runs simple pixel, channel, and pixel attention in parallel, joining their outputs before an MLP and residual addition.Channel attention extracts shared global information associated with atmospheric A, while pixel attention extracts location-dependent information associated with t(x).Source: §3.2 · p. 5

3.3. Mix structure block

  • The transformer-style Mix Structure Block replaces multi-head self-attention with MSPLCK and the feed-forward module with EPA.The block combines large-haze-area capture and texture restoration with parallel extraction of shared global and location-dependent local information.Source: §3.3 · p. 5

3.4. Training Loss

  • Training predicts dehazed bJ from a hazy/clear image pair and combines L1 loss with contrastive loss.The contrastive term uses features from layers of a fixed pretrained model, with a weight balancing it against L1 loss.Source: §3.4 · p. 5

4. Experiments

Experiments evaluate MixDehazeNet on RESIDE datasets, compare it with prior methods, and test the contributions of its modules and inference-time trade-offs.

4.1. Datasets

  • Experiments used RESIDE-IN, RESIDE-OUT, and RESIDE-6K; the first two trained on 13,990 and 313,950 pairs, respectively, and were tested on corresponding 500-pair SOTS indoor and outdoor sets.RESIDE-6K contains 6,000 pairs; 3,000 ITS and 3,000 OTS pairs are used for training.Source: §4.1 · p. 6

4.2. Implementation Details

  • Models were trained on four RTX-3090 cards using 256 × 256 crops, with AdamW and cosine annealing for the learning rate.The paper provides small, basic, and large variants and describes a ResNet-152 feature-based training loss.Source: §4.2 · p. 6

4.3. Comparison with state-of-the-art methods

  • MixDehazeNet-L is the first method to exceed 42dB PSNR on RESIDE-IN and outperforms prior methods on PSNR and SSIM across three datasets.The comparison includes RESIDE-IN, RESIDE-OUT, and RESIDE-6K.Source: Table 1 · p. 6, §4.3 · p. 7
  • On RESIDE-IN and RESIDE-OUT, MixDehazeNet restorations are reported as clearer and closer to ground truth than prior methods’ results.The reported comparisons note artifacts from DCP, GridDehazeNet, and FFA-Net on RESIDE-IN, and haze residue or color distortion from these methods on RESIDE-OUT.Source: §4.3 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7

4.4. Ablation Study

  • On RESIDE-IN, adding MSPLCK and EPA together increased PSNR by 3.96 dB over MixDehazeNet-S-Base; individually, they increased it by 1.76 dB and 1.89 dB.The ablations used MixDehazeNet-S and tested on RESIDE-IN.Source: §4.4 · p. 8, Table 3 · p. 8
  • MSPLCK’s multi-scale kernels performed better than same-scale parallel kernels, while larger receptive fields yielded better haze removal.The comparisons tested parallel kernels with sizes 7, 13, and 19.Source: §4.4 · p. 8, Table 5 · p. 8
  • Ablations indicate that EPA’s parallel attention mechanism is more suitable for image dehazing than the tested serial attention alternatives.The comparisons used serial channel and pixel attention, with and without simple pixel attention.Source: §4.4 · p. 8, Table 6 · p. 8

4.5. Inference Time

  • MixDehazeNet is reported to deliver higher PSNR than prior methods at similar inference times, including a 6.23dB increase over FFA-Net.On RESIDE-IN, the paper reports similar inference times for MixDehazeNet-S and MSBDN, and for MixDehazeNet-B and AECR-Net or PMNet.Source: Table 1 · p. 6, §4.5 · pp. 8–9

5. Conclusion

MixDehazeNet combines multi-scale parallel large convolutions with enhanced parallel attention to address image dehazing. The authors report that it is the first method to exceed 42dB PSNR on RESIDE-IN.

5. Conclusion

  • The authors report MixDehazeNet is the first method to exceed 42dB PSNR on the RESIDE-IN dataset.Source: §5 · p. 9
  • MixDehazeNet’s mix structure block combines a multi-scale parallel large convolution kernel module with enhanced parallel attention.Source: §5 · p. 9
  • The large-kernel module provides multi-scale large receptive fields, while enhanced parallel attention addresses uneven haze and lets useful features pass through the backbone.Source: §5 · p. 9

Related papers

Loading 2305.17654v1…