Source-linked AI summary
Multi-Scale Boosted Dehazing Network with Dense Feature Fusion
Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, Ming-Hsuan Yang
TL;DR
Image dehazing is an ill-posed restoration problem requiring designs beyond simply increasing network depth or width. The paper proposes MSBDN, combining an SOS boosted decoder with back-projection-based Dense Feature Fusion, and reports favorable performance against state-of-the-art methods. Its boosting rationale is supported under a stated axiom about performance across images with different haze levels.
Problem
Image dehazing is an ill-posed inverse problem, while simply stacking or widening layers is inefficient for substantial performance gains.
Method
MSBDN combines an SOS boosted decoder for progressive restoration with back-projection-based Dense Feature Fusion for spatial and multi-level feature integration.
Results
The proposed model performs favorably against state-of-the-art dehazing methods, while its boosted decoder exceeds other boosting strategies by at least 0.36 dB.
Takeaways & Limitations
Boosting and Dense Feature Fusion are reported as effective design principles for image dehazing, improving restoration without extra layers in the MSBDN ablation.
Takeaways & Limitations
The boosting claim relies on an axiom concerning relative PoH performance for images of the same scene with different haze levels.
Abstract
from arXiv · showhide
In this paper, we propose a Multi-Scale Boosted Dehazing Network with Dense Feature Fusion based on the U-Net architecture. The proposed method is designed based on two principles, boosting and error feedback, and we show that they are suitable for the dehazing problem. By incorporating the Strengthen-Operate-Subtract boosting strategy in the decoder of the proposed model, we develop a simple yet effective boosted decoder to progressively restore the haze-free image. To address the issue of preserving spatial information in the U-Net architecture, we design a dense feature fusion module using the back-projection feedback scheme. We show that the dense feature fusion module can simultaneously remedy the missing spatial information from high-resolution features and exploit the non-adjacent features. Extensive evaluations demonstrate that the proposed model performs favorably against the state-of-the-art approaches on the benchmark datasets as well as real-world hazy images.
1. Introduction
Image dehazing restores clean scene radiance from hazy inputs, but its ill-posed nature makes generic network scaling insufficient. The paper proposes MSBDN, combining boosting and error feedback through an SOS decoder and Dense Feature Fusion.
- Motivation: Image dehazing is an ill-posed inverse problem in which small measurement errors can cause dramatic changes.The paper argues that specialized priors or network designs are therefore needed.
- Proposed approach: The proposed MSBDN combines boosting and error feedback mechanisms for image restoration.These principles are adapted from denoising and super-resolution, respectively.
- Proposed approach: The SOS boosting strategy is incorporated into the decoder to progressively restore the haze-free image.The decoder is interpreted as an image restoration module.
- Proposed approach: Dense Feature Fusion uses back-projection to preserve high-resolution spatial information and exploit non-adjacent features.This module addresses information compression caused by U-Net downsampling.
- Evaluation: Extensive evaluations report favorable performance against state-of-the-art dehazing methods on benchmark datasets and real-world hazy images.The contribution summary also reports that both boosting and DFF improve dehazing networks.
2. Related Work
Prior dehazing methods use scene assumptions, estimated physical quantities, or learned mappings to recover clean images. Their effectiveness can depend on whether those assumptions or intermediate estimates are accurate for the input scene.
- Prior-based methods: Hand-crafted dehazing priors target quantities such as transmission, atmospheric light, or scene radiance.Examples include surface shading, contrast, dark-channel, and non-local path priors.
- Prior-based methods: Prior-based methods are less effective when their scene-specific or atmospheric assumptions do not hold.The dark channel prior, for example, performs poorly for images without zero-intensity pixels.
- Deep learning methods: Deep learning methods often estimate transmission maps and then restore images, but inaccurate atmospheric-light or transmission estimates can produce color distortions.This difficulty is associated with the airlight-albedo ambiguity.
- Deep learning methods: End-to-end dehazing networks directly restore clean radiance scenes instead of relying on separate transmission-map estimation.The related-work passage presents these networks as an alternative to two-stage approaches.
3. Proposed Method
The proposed MSBDN uses SOS boosting in a U-Net decoder to progressively restore haze-free images, while DFF applies back-projection feedback to recover spatial information and fuse non-adjacent features.
- Multi-Scale Boosted Dehazing Network: Under Axiom 1, SOS boosting improves dehazing performance by reducing the Portion of Haze across iterations.The axiom assumes better dehazing results on images of the same scene with less haze; the proposition states PoH( Ĵn+1) < PoH( Ĵn).
- Multi-Scale Boosted Dehazing Network: The multi-scale boosted decoder incorporates SOS boosting to progressively refine features and reconstruct the estimated haze-free image.Each level upsamples the previous feature, strengthens it with encoder information, and applies a trainable refinement unit.
- Multi-Scale Boosted Dehazing Network: The proposed SOS module is selected over diffusion, twicing, pyramid, and original U-Net decoder alternatives because it obtains the best results in the reported evaluation.The alternatives differ in how upsampled decoder features and encoder latent features are fused or refined.
- Dense Feature Fusion Module: DFF addresses U-Net limitations by using back-projection feedback to recover missing spatial information and connect features from non-adjacent levels.The module progressively projects and back-projects feature differences before updating the enhanced feature.
- Dense Feature Fusion Module: At decoder level n, DFF progressively fuses preceding enhanced features one at a time to enhance the current boosted feature.The preceding features are drawn from all earlier DFF modules in the decoder, and the enhanced output is connected across levels.
4. Experimental Results
The proposed method is evaluated on benchmark, real-world, and downstream detection settings, with ablations isolating the contributions of boosted decoding and dense feature fusion. It performs strongly against competing dehazing methods and improves qualitative restoration and detection results.
- Performance Evaluation: The proposed algorithm is evaluated against state-of-the-art methods on RESIDE, HazeRD, and NTIRE18-Dehazing settings.Evaluations use benchmark datasets and compare against handcrafted-prior and deep-learning methods with PSNR, SSIM, and CIEDE2000.
- Performance Evaluation: The proposed method outperforms competing algorithms in both PSNR and SSIM on the SOTS dataset.The paper attributes this result to the architecture’s boosting and back-projection design for dehazing.
- Performance Evaluation: The method achieves better CIEDE2000 and SSIM results than other methods on HazeRD and comparable results with MsPPN on NTIRE18-Dehazing.MsPPN is described as specially designed for ultra-high-resolution datasets with extra pre-processing and post-processing.
- Performance Evaluation: The dehazed images produced by the method are visually clearer, with fewer color distortions, haze residuals, and artifacts across SOTS and real-world examples.The real-image result is described as sharper and brighter, while the SOTS examples are restored with fewer visible artifacts.
- Performance Evaluation: The proposed method obtains the highest detection accuracy on the KITTI Haze dataset after dehazing images for object detection.The qualitative detection results indicate improved recognition of cars and pedestrians.
- Ablation Study and Analysis: The SOS boosted decoder outperforms other boosting strategies by at least 0.36 dB without adding extra layers.Networks using boosting modules also improve substantially over the PFFNet baseline without boosting.
- Ablation Study and Analysis: MSBDN-S improves average PSNR by 0.17 dB over MSBDN, demonstrating benefits from exploiting non-adjacent preceding features.The DFF design progressively fuses preceding features and remedies missing spatial information.
- Ablation Study and Analysis: MSBDN-DFF exceeds MSBDN-M by 0.55 dB without extra layers, while PFFNet-DFF improves over PFFNet by 3.73 dB.These comparisons support the effectiveness and portability of the proposed dense feature fusion module.
5. Conclusions
The proposed MSBDN-DFF combines SOS boosting and dense feature fusion for image dehazing. Ablation studies support the effectiveness of its modules, while evaluations show favorable performance against state-of-the-art methods on dehazing datasets.
- MSBDN uses an encoder-decoder architecture with a boosted decoder based on the SOS boosting strategy.
- The DFF module uses back-projection to preserve spatial information and exploit features from non-adjacent levels.
- Ablation studies demonstrate that the proposed modules are effective for image dehazing.
- Extensive evaluations show favorable performance against state-of-the-art methods on image dehazing datasets.