PaperlayerView exact revision on arXiv

Source-linked AI summary

LYT-NET: Lightweight YUV Transformer-based Network for Low-light Image Enhancement

A. Brateanu, R. Balmez, A. Avram, C. Orhei, C. Ancuti

arXiv:2401.15204v7cs.CVeess.IV

TL;DR

Low-light images can lose detail and contrast, making enhancement while correcting distortions a challenging task. LYT-Net separates luminance and chrominance processing in a lightweight transformer-based architecture. It consistently outperforms state-of-the-art methods across LOL dataset versions in PSNR and SSIM while maintaining high computational efficiency.

  • Problem

    Low-light capture degrades image detail and contrast, while enhancement must correct distortions without introducing artifacts or color imbalance.

    Source: §I · p. 1

  • Method

    LYT-Net uses a lightweight dual-path transformer that processes luminance and chrominance separately with MHSA, CWD, and MSEF blocks.

    Source: §I · p. 1

  • Results

    LYT-Net consistently outperforms state-of-the-art methods on all LOL dataset versions in PSNR and SSIM while maintaining high computational efficiency.

    Source: §V · p. 4

  • Takeaways & Limitations

    LYT-Net is reported to enhance visibility in low-contrast or poorly lit areas while eliminating noise without introducing artifacts.

    Source: §III · p. 4

    On high-resolution SDSD images, LYT-Net shows limitations attributed to its significantly low parameter count.

    Source: §III · p. 3

Abstract

from arXiv · show

This letter introduces LYT-Net, a novel lightweight transformer-based model for low-light image enhancement (LLIE). LYT-Net consists of several layers and detachable blocks, including our novel blocks--Channel-Wise Denoiser (CWD) and Multi-Stage Squeeze & Excite Fusion (MSEF)--along with the traditional Transformer block, Multi-Headed Self-Attention (MHSA). In our method we adopt a dual-path approach, treating chrominance channels U and V and luminance channel Y as separate entities to help the model better handle illumination adjustment and corruption restoration. Our comprehensive evaluation on established LLIE datasets demonstrates that, despite its low complexity, our model outperforms recent LLIE methods. The source code and pre-trained models are available at https://github.com/albrateanu/LYT-Net

I. INTRODUCTION

Low-light images lose detail and contrast, while enhancement must also correct distortions without adding artifacts or color imbalance. LYT-Net uses a lightweight dual-path transformer, and the paper reports competitive results despite its low complexity.

I. INTRODUCTION

  • LLIE aims to improve clarity and contrast while correcting dark-scene distortions without unwanted artifacts or color imbalance.The introduction notes that low-light capture can degrade image quality and affect imaging systems.Source: §I · p. 1
  • Earlier LLIE spans Retinex and CNN methods and GAN approaches; ViT use in low-level vision remains underexplored, and diffusion models have emerged for LLIE.The cited approaches include Retinex-Net, Diff-Retinex, and EnlightenGAN.Source: §I · p. 1
  • On the LOL dataset, LYT-Net performs comparably to and often outperforms more complex recent deep-learning LLIE methods.The comparison evaluates performance against model complexity.Source: Figure 1 · p. 1, §I · p. 2
  • LYT-Net separates luminance and chrominance, applying convolution and pooling plus MHSA to Y, and CWD to U/V before MSEF fusion.The recombined channels pass through final convolutional layers to produce the restored image.Source: §I · pp. 1–2, Figure 2 · p. 2

II. OUR APPROACH

LYT-Net uses separate luminance and chrominance paths, then fuses their features to produce an enhanced image.

II. OUR APPROACH

  • The dual-path design treats luminance Y and chrominance U/V separately to handle illumination adjustment and corruption restoration.Y features are extracted with convolution and pooling, then enhanced by MHSA; U/V pass through CWD to reduce noise while preserving detail.Source: §II · p. 2
  • After CWD processing, the enhanced chrominance channels are recombined and processed by the MSEF block.The approach uses detachable CWD, MHSA, and MSEF blocks.Source: §II · p. 2
  • The U, V, and Y channels are concatenated and passed through final convolutional layers to produce the output image.Source: §II · p. 2

A. Channel-wise Denoiser Block

The Channel-wise Denoiser combines multiscale convolution, an attention bottleneck, and skip-connected reconstruction, with design choices intended to reduce computational and parameter costs.

A. Channel-wise Denoiser Block

  • CWD uses a U-shaped network with an MHSA bottleneck, convolutional layers, and skip connections for feature capture and denoising.The attention bottleneck captures long-range dependencies, while upsampling and skip connections recover spatial resolution.Source: A. Channel-wise Denoiser Block · p. 2
  • The block uses four conv3×3 layers: the first has stride 1, and the remaining three use stride 2 to capture features at different scales.Source: A. Channel-wise Denoiser Block · p. 2
  • Interpolation-based upsampling cuts CWD parameters by more than half while preserving performance.The block applies MHSA on a feature map with reduced spatial dimensions to improve computational efficiency.Source: A. Channel-wise Denoiser Block · p. 2

B. Multi-headed Self-attention Block

The MHSA block projects input features into query, key, and value representations, applies self-attention across multiple heads, and returns features in the original embedding size and spatial dimensions.

B. Multi-headed Self-attention Block

C. Multi-stage Squeeze & Excite Fusion Block

MSEF recalibrates features using a squeeze-and-excitation sequence, then adds a residual connection to produce its output. It performs efficient feature extraction on self-attended and denoised chrominance features with a minor parameter-count increase.

C. Multi-stage Squeeze & Excite Fusion Block

D. Loss Function

LYT-Net trains with a hybrid loss that combines pixel, perceptual, histogram, signal-quality, color, and structural terms. The components are intended to address different aspects of image enhancement.

D. Loss Function

  • The hybrid loss is L = L_S + α1L_Perc + α2L_Hist + α3L_PSNR + α4L_Color + α5L_MS-SSIM, with α1–α5 balancing the constituent loss functions.Source: D. Loss Function · p. 3
  • Smooth L1 penalizes prediction errors, while perceptual loss compares VGG-extracted feature maps and histogram loss aligns pixel-intensity distributions.Source: D. Loss Function · p. 3
  • PSNR loss penalizes mean squared error, color loss minimizes channel-mean differences, and multiscale SSIM loss evaluates structural similarity across scales.Source: D. Loss Function · p. 3

III. RESULTS AND DISCUSSION

LYT-Net outperforms current state-of-the-art methods on all LOL dataset versions in PSNR and SSIM while using 3.49G FLOPS and 0.045M parameters. Results are less favorable on high-resolution SDSD images; the qualitative comparisons and ablation also examine restoration and block choices.

Implementation and Evaluation

  • LYT-Net was trained and evaluated on LOL-v1, LOL-v2-real, and LOL-v2-synthetic, with random 256 × 256 crops, flips, and rotations and batch size 1.Training used ADAM for 1000 epochs, with a cosine-annealed learning rate decaying to 1 × 10−6.Source: §III · p. 3

Quantitative Results

  • Across LOL dataset versions, LYT-Net outperforms current state-of-the-art methods in PSNR and SSIM, using 3.49G FLOPS and 0.045M parameters; SDSD high-resolution images expose limitations.The authors attribute the SDSD limitation to the model's low parameter count and expect a deeper variant to improve performance.Source: §III · p. 3, Table I · p. 4

Qualitative Results

  • On LOL and LIME comparisons, prior methods show color distortion, over- or under-exposure, or contrast loss, while LYT-Net enhances visibility and poorly lit areas while eliminating noise without artifacts.Fig. 3 covers LOL; Fig. 4 covers LIME.Source: §III · pp. 3–4, Figure 4 · p. 4

Ablation Study

  • On LOLv1, applying CWD to luminance retains lighting artifacts, whereas CWD on U and V preserves detail without introducing noise; MSEF raises PSNR by 0.16, 0.24, and 0.26 dB across CWD combinations for 546 additional parameters.Pooling and interpolation-based upsampling smooth the illumination for more uniform lighting.Source: §III · p. 4

V. CONCLUSIONS

LYT-Net is a lightweight transformer-based LLIE model that separates luminance and chrominance processing through a dual-path framework. The authors report that it outperforms state-of-the-art methods across LOL versions on PSNR and SSIM while maintaining computational efficiency.

V. Conclusions

  • LYT-Net uses a dual-path framework that processes luminance and chrominance separately to manage illumination adjustment and restore corrupted regions.Source: §V · p. 4
  • Table II reports an ablation study of the performance and parameter impact of CWD and MSEF blocks.Source: Table II · p. 4
  • The model combines CWD and MSEF blocks with a ViT block using MHSA.Source: §V · p. 4
  • The conclusion reports consistent PSNR and SSIM outperformance on all LOL dataset versions while maintaining high computational efficiency.Source: §V · p. 4

Related papers

Loading 2401.15204v7…