PaperlayerView exact revision on arXiv

Source-linked AI summary

Half Wavelet Attention on M-Net+ for Low-Light Image Enhancement

Chi-Mao Fan, Tsung-Jung Liu, Kuan-Hsien Liu

arXiv:2203.01296v1eess.IVcs.AIcs.CVcs.MM

TL;DR

Conventional M-Net can lose spatial information during downsampling and incur decoder burden from feature fusion. HWMNet combines an improved M-Net+ with half wavelet attention, and the authors report state-of-the-art performance on two low-light enhancement datasets in quantitative and visual quality, with less computational complexity.

  • Problem

    Conventional M-Net uses downsampling that can cause spatial-information loss and decoder feature fusion that increases network burden.

    Source: §3.2 · p. 2

  • Method

    HWMNet combines an improved hierarchical M-Net+ with a half wavelet attention block for wavelet-domain features and SKFF for feature fusion.

    Source: §1 · p. 1

  • Results

    HWMNet achieves state-of-the-art performance on two low-light enhancement datasets in quantitative and visual quality, with less computational complexity.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The authors conclude that HWMNet achieves competitive performance on low-light image enhancement.

    Source: §5 · p. 4

Abstract

from arXiv · show

Low-Light Image Enhancement is a computer vision task which intensifies the dark images to appropriate brightness. It can also be seen as an ill-posed problem in image restoration domain. With the success of deep neural networks, the convolutional neural networks surpass the traditional algorithm-based methods and become the mainstream in the computer vision area. To advance the performance of enhancement algorithms, we propose an image enhancement network (HWMNet) based on an improved hierarchical model: M-Net+. Specifically, we use a half wavelet attention block on M-Net+ to enrich the features from wavelet domain. Furthermore, our HWMNet has competitive performance results on two image enhancement datasets in terms of quantitative metrics and visual quality. The source code and pretrained model are available at https://github.com/FanChiMao/HWMNet.

1. INTRODUCTION

Image enhancement is framed as an image-restoration task, while learning-based methods have recently surpassed conventional prior-based methods in inference time and enhancement performance. The paper proposes HWMNet, combining M-Net+ with half wavelet attention, and reports state-of-the-art enhancement results on two datasets.

1. INTRODUCTION

  • Image enhancement is treated as a restoration task in which a low-quality image results from degradation of a high-quality image and additive noise.The paper groups enhancement methods into algorithm-based and learning-based approaches, with algorithm-based methods grounded in image priors.Source: §1 · p. 1
  • The paper situates its approach amid recent learning-based methods, which it says surpass conventional prior-based methods in inference time and enhancement performance.It notes the prominence of convolutional neural networks in computer vision, including image enhancement.Source: §1 · p. 1
  • The proposed HWMNet combines an improved hierarchical M-Net+ with a half wavelet attention block to enrich features from the wavelet domain.The approach also uses selective kernel feature fusion (SKFF) to combine features at different resolutions.Source: §1 · p. 1
  • HWMNet achieves state-of-the-art performance on two low-light enhancement datasets in quantitative and visual quality, with less computational complexity.The authors report these results as a contribution of the proposed model.Source: §1 · p. 1

2. RELATED WORK

The related work reviews conventional image-enhancement approaches and the hierarchical M-Net architecture. It presents M-Net as a means of enriching contextual information across image resolutions.

2.1. Image enhancement

  • Conventional enhancement methods use image priors, while recent CNN-based learning methods are reported to almost surpass prior-based approaches in performance.Prior-based approaches include histogram equalization, retinex, and dehazing methods.Source: §2.1 · p. 1

2.2. M-Net

  • M-Net extends U-Net with additional gatepost feature paths in the encoder and decoder to enrich contextual information across resolutions and address spatial loss from sampling.The cited related work also applies the framework to fingerprint image denoising.Source: §2.2 · p. 2

3. PROPOSED METHOD

HWMNet uses a hierarchical M-Net+ backbone with half wavelet attention blocks and end-to-end Charbonnier-loss training. M-Net+ changes downsampling and feature fusion to address spatial-information loss and decoder burden in conventional M-Net.

3.1. Half Wavelet attention M-Net (HWMNet)

  • HWMNet applies shared 3 × 3 convolutions and half wavelet attention blocks across its hierarchical input paths, with pixel unshuffle downsampling in the U-Net trunk.The model is trained end-to-end with Charbonnier loss, a variant of L1 loss; ε is set to 10^-3.Source: §3.1 · p. 2

3.2. M-Net+

  • M-Net+ addresses conventional M-Net's spatial-information loss and decoder fusion burden using pixel unshuffle in the U-Net path, bilinear downsampling in the gatepost path, and SKFF fusion.These choices provide more diverse multi-scale cascading features and target the parameter and computational burden of high-dimensional decoder features.Source: §3.2 · pp. 2–3

3.3. Half Wavelet Attention Block

  • HWAB splits input features into an identity path and a wavelet path, applies attention to the transformed features, then combines the paths through convolution and a shortcut.The wavelet path uses DWT, DAU attention, and inverse wavelet transformation before the output feature is formed.Source: §3.3 · p. 3

4. EXPERIMENT

Experiments evaluate HWMNet on LOL and MIT-Adobe FiveK. HWMNet is runner-up on all reported LOL metrics and achieves state-of-the-art PSNR and SSIM on MIT-Adobe FiveK.

4.1. Experiment Setup

  • Training uses 256 × 256 patches, batch size 2, and 1 × 10^5 iterations; evaluation reports PSNR, SSIM, and LPIPS.PSNR and SSIM are better when higher, while lower LPIPS indicates restoration closer to human perception.Source: §4.1 · p. 3

4.2. Experiment Datasets

  • LOL provides 485 training and 15 testing image pairs; MIT-Adobe FiveK contains 5,000 images, with the first 4,500 used for training and the last 500 for testing.For FiveK, enhanced images from expert C are used as ground truth.Source: §4.2 · pp. 3–4

4.3. Image Enhancement Performance

  • On LOL, HWMNet ranks runner-up for PSNR, SSIM, and LPIPS; on MIT-Adobe FiveK, it achieves state-of-the-art PSNR and SSIM.The reported FLOPs on a 400 × 592 color image are 2.84T for MIRNet, 1.05T for LLFlow, and 0.92T for HWMNet.Source: Table 1 · p. 3, §4.3 · p. 4, Table 2 · p. 4

5. CONCLUSION

HWMNet combines M-Net+ with wavelet-domain attention for low-light image enhancement. The authors report competitive performance and identify denoising and deblurring as future extensions.

5. CONCLUSION

  • HWMNet achieves competitive performance on low-light image enhancement, with M-Net+ enriching features at different resolutions and HWAB enriching wavelet-domain semantic information.The authors identify image denoising and deblurring as future restoration tasks for the model.Source: §5 · pp. 4–5

Related papers

Loading 2203.01296v1…