Source-linked AI summary
Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method
Tao Wang, Kaihao Zhang, Tianrun Shen, Wenhan Luo, Bjorn Stenger, Tong Lu
TL;DR
Existing low-light enhancement datasets and methods are limited for modern 4K and 8K imagery, motivating a dedicated UHD benchmark and algorithms. The paper constructs UHD-LOL, benchmarks LLIE methods, and introduces LLFormer, which achieves state-of-the-art performance across the evaluated datasets while addressing UHD computational demands.
Problem
Existing LLIE datasets primarily contain images at 1K resolution or less, limiting evaluation and training for modern 4K and 8K images.
Method
The paper builds the UHD-LOL benchmark and introduces LLFormer, a hierarchical transformer using axis-based attention and cross-layer attention fusion for UHD-LLIE.
Results
LLFormer achieves state-of-the-art performance on LOL, MIT-Adobe FiveK, and UHD-LOL, outperforming other state-of-the-art methods in extensive experiments.
Takeaways & Limitations
UHD-LOL and LLFormer provide a benchmark and transformer-based baseline for low-light enhancement of ultra-high-definition images.
Abstract
from arXiv · showhide
As the quality of optical sensors improves, there is a need for processing large-scale images. In particular, the ability of devices to capture ultra-high definition (UHD) images and video places new demands on the image processing pipeline. In this paper, we consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method. The core components of LLFormer are the axis-based multi-head self-attention and cross-layer attention fusion block, which significantly reduces the linear complexity. Extensive experiments on the new dataset and existing public datasets show that LLFormer outperforms state-of-the-art methods. We also show that employing existing LLIE methods trained on our benchmark as a pre-processing step significantly improves the performance of downstream tasks, e.g., face detection in low-light conditions. The source code and pre-trained models are available at https://github.com/TaoWangzj/LLFormer.
Introduction
The paper targets low-light enhancement for 4K and 8K images, where existing low-resolution datasets and methods face efficiency and visual-quality challenges. It introduces the UHD-LOL benchmark, benchmarks existing methods, and proposes LLFormer, which achieves state-of-the-art performance across evaluated datasets.
- Existing LLIE datasets such as LOL and SID mainly contain images of 1K resolution or less.
- UHD-LOL provides 4K and 8K low-light/normal-light image pairs for training and evaluating UHD-LLIE algorithms.UHD-LOL4K contains 8,099 pairs, while UHD-LOL8K contains 2,966 pairs.
- LLFormer combines an efficient Axis-based Transformer Block with a Cross-layer Attention Fusion Block for UHD-LLIE.Its hierarchical structure is designed to alleviate the computational bottleneck of UHD processing.
- The paper benchmarks existing LLIE algorithms on UHD-LOL to characterize their performance and limitations in the UHD setting.
- LLFormer achieves state-of-the-art quantitative and qualitative performance on LOL, MIT-Adobe FiveK, and the UHD-LOL benchmark.
Related Work
Prior low-light enhancement research includes datasets built from real captures and synthetic degradation, alongside traditional and data-driven enhancement methods. Traditional approaches commonly use histogram, Retinex, or dehazing-based formulations.
- Earlier LLIE datasets include small real-image collections and larger datasets created by synthesizing low-light conditions.
- LLIE methods are broadly divided into non-learning-based and data-driven approaches.
- Traditional LLIE methods commonly rely on histogram-based, Retinex-based, or dehazing-based techniques.
Benchmark and Methodology
The paper constructs UHD-LOL for 4K and 8K low-light enhancement and proposes LLFormer, a hierarchical transformer designed to reduce the cost of processing UHD images. Its axis-based attention, gated feed-forward network, and cross-layer fusion integrate efficient spatial modeling with adaptive hierarchical feature aggregation.
- Benchmark Dataset: UHD-LOL contains paired 4K and 8K images for benchmarking ultra-high-definition low-light image enhancement.The subsets use 3,840 × 2,160 and 7,680 × 4,320 resolutions and include diverse indoor and outdoor scenes.
- LLFormer Architecture: LLFormer uses a hierarchical encoder-decoder that extracts, downsamples, reconstructs, and fuses features before image reconstruction.Weighted skip connections combine encoder and decoder features, while cross-layer fusion is used near the network head and tail.
- Axis-based Transformer Block: Axis-based multi-head self-attention computes attention sequentially along height and width axes, reducing complexity from the standard quadratic dependence on spatial resolution to linear.Depth-wise convolutions supply local information before attention, while the dual gated feed-forward network filters less informative features and enriches local context.
- Axis-based Transformer Block: An axis-based transformer block combines axis-based attention, a dual gated feed-forward network, and two normalization layers.The attention output is passed to width-axis attention, and the feed-forward network uses parallel gated paths with element-wise products.
- Cross-layer Attention Fusion Block: The cross-layer attention fusion block adaptively fuses hierarchical features using learnable correlations among layers.It is placed symmetrically at the network head and tail to capture long-distance dependencies during feature extraction and image reconstruction.
Experiments and Analysis
Experiments evaluate LLFormer on UHD and public datasets, compare it with existing methods, test downstream face detection, and analyze architectural components and complexity.
- UHD-LOL4K Benchmark: 0.42 dB improvement in PSNR over Restormer makes LLFormer the best-performing method on UHD-LOL4K among the compared approaches.LLFormer also achieves the best UHD-LOL4K scores in SSIM, LPIPS, and MAE.
- UHD-LOL8K Benchmark: LLFormer outperforms other methods on UHD-LOL8K, where transformer-based methods rank highest across pixel-wise and perceptual metrics.The 8K images are partitioned into four 4K patches for evaluation, and LLFormer produces more detailed visual results.
- Downstream Face Detection: 71.2% AP improvement is obtained for RetinaFace after LLFormer preprocessing on 300 DARK FACE images, exceeding Uformer and Restormer preprocessing.The corresponding AP improvements are 67.06% for Uformer and 68.11% for Restormer.
- Public-Dataset Evaluation: LLFormer achieves higher performance than Restormer on LOL and the best results on all metrics for MIT-Adobe FiveK among the compared methods.Table 2 evaluates PSNR, SSIM, LPIPS, and MAE, using expert C images for MIT-Adobe FiveK.
- Ablation Studies: Each cross-layer attention fusion and weighted skip-connection component improves results, while 1 × 1 convolutions provide only a minor gain.The ablation study reports results in terms of PSNR and SSIM.
- Complexity Analysis: 36.20/0.9867 performance with 22.52G operations, 24.52M parameters, and 0.063s speed provides the best tradeoff among wider and deeper variants.The comparison varies network width and depth and evaluates overhead, performance, and speed.
Conclusion
The paper establishes a large-scale UHD benchmark and introduces LLFormer as a transformer-based baseline for UHD low-light enhancement. Extensive experiments report that LLFormer significantly outperforms other state-of-the-art methods, while the dataset and model support future benchmarking.
- UHD-LOL comprises UHD-LOL4K and UHD-LOL8K subsets for evaluating UHD low-light image enhancement.
- LLFormer is presented as the first transformer-based baseline network for the UHD-LLIE task.
- Extensive experiments show that LLFormer significantly outperforms other state-of-the-art methods.
- The UHD-LOL dataset and LLFormer are intended to serve as benchmarks for LLIE and UHD-LLIE research.