Source-linked AI summary
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, Yulun Zhang
TL;DR
Low-light enhancement methods must address corruption overlooked by the Retinex model, while prior approaches face multi-stage training and limited long-range modeling. Retinexformer introduces a one-stage Retinex framework with an illumination-guided Transformer, and experiments report superior performance across thirteen datasets plus practical value in user studies and low-light detection.
Problem
Low-light Retinex methods do not model corruptions hidden in dark scenes or introduced during light-up, and CNN-based approaches have limited long-range dependency modeling.
Method
Retinexformer combines ORF, which estimates illumination and restores corruption in one stage, with IGT, which uses illumination representations to guide non-local self-attention.
Results
Retinexformer significantly outperforms state-of-the-art methods on thirteen datasets, with user-study and low-light detection results also indicating practical value.
Takeaways & Limitations
The reported results support Retinexformer as a low-light enhancement method with performance and practical evidence spanning image enhancement and low-light detection.
Takeaways & Limitations
Directly applying original vision Transformers can be computationally unaffordable because self-attention complexity is quadratic in input spatial size.
Abstract
from arXiv · showhide
When enhancing low-light images, many deep learning algorithms are based on the Retinex theory. However, the Retinex model does not consider the corruptions hidden in the dark or introduced by the light-up process. Besides, these methods usually require a tedious multi-stage training pipeline and rely on convolutional neural networks, showing limitations in capturing long-range dependencies. In this paper, we formulate a simple yet principled One-stage Retinex-based Framework (ORF). ORF first estimates the illumination information to light up the low-light image and then restores the corruption to produce the enhanced image. We design an Illumination-Guided Transformer (IGT) that utilizes illumination representations to direct the modeling of non-local interactions of regions with different lighting conditions. By plugging IGT into ORF, we obtain our algorithm, Retinexformer. Comprehensive quantitative and qualitative experiments demonstrate that our Retinexformer significantly outperforms state-of-the-art methods on thirteen benchmarks. The user study and application on low-light object detection also reveal the latent practical values of our method. Code, models, and results are available at https://github.com/caiyuanhao1998/Retinexformer
1. Introduction
Low-light enhancement must improve visibility while addressing corruptions such as noise, artifacts, and color distortion. Retinexformer proposes a one-stage Retinex framework and illumination-guided Transformer to address limitations of prior methods.
- Low-light enhancement targets poor visibility and contrast while restoring noise, artifacts, and color distortion that affect perception and downstream vision tasks.
- Retinex-based methods model illumination but can introduce noise and local color distortion because they assume corruption-free images.
- CNN-based enhancement methods either learn brute-force mappings or use Retinex formulations with multi-stage training pipelines.
- CNN-based methods have limitations in capturing long-range dependencies, while full vision Transformers incur quadratic complexity with input spatial size.
- Retinexformer introduces ORF, which models corruption with perturbation terms, estimates illumination, restores corruption, and supports end-to-end one-stage training.
- IG-MSA uses illumination information to guide self-attention and model long-range interactions between regions with different exposure levels.
- Retinexformer outperforms state-of-the-art methods on thirteen datasets, with user-study and low-light detection results suggesting practical value.
2. Related Work
Prior low-light enhancement research includes direct amplification, Retinex-based conventional methods, and Transformer applications. These approaches motivate a framework that better accounts for illumination, corruption, and non-local dependencies.
- Plain Methods: Plain methods such as histogram equalization and gamma correction amplify visibility and contrast but barely consider illumination factors.
- Traditional Cognition Methods: Traditional cognition methods use Retinex theory and illumination factors, but their assumptions can conflict with real under-exposed scenes.
- Vision Transformer: Transformers have been applied across high-level and low-level computer vision tasks, including image restoration and synthesis.
- Vision Transformer: Retinexformer’s overview combines an illumination estimator with an IGT corruption restorer, whose IG-MSA directs self-attention using illumination representations.
3. Method
Retinexformer uses a one-stage Retinex framework that estimates illumination, lights up the input, and restores corruption. Its Illumination-Guided Transformer uses illumination features to guide efficient long-range attention across differently lit regions.
- One-stage Retinex-based Framework: The Retinex model is extended with perturbations to reflectance and illumination to represent corruptions hidden in dark scenes or introduced during light-up.These corruptions include noise, artifacts, under-/over-exposure, and color distortion.
- One-stage Retinex-based Framework: ORF estimates illumination information to produce a lit-up image and feature, then feeds both into a corruption restorer that outputs the enhanced image.The illumination estimator takes the low-light image and its illumination prior map as inputs.
- One-stage Retinex-based Framework: The illumination estimator fuses the image with a channel-mean illumination prior and models interactions between differently lit regions to generate light-up features and a three-channel light-up map.Well-exposed regions provide semantic contextual information for under-exposed regions.
- Illumination-Guided Transformer: IGT serves as the corruption restorer in a three-scale U-shaped architecture, producing a residual image that is added to the lit-up image.Downsampling, skip connections, and upsampling form the restoration path, with Ien = Ilu + Ire.
- Illumination-Guided Transformer: IG-MSA reshapes image and illumination features into tokens and uses illumination-guided representations to direct self-attention across regions with different lighting conditions.The guidance emphasizes contextual information from better-lit regions that can help restore darker, more corrupted regions.
- Illumination-Guided Transformer: O(IG-MSA) is linear in spatial size, unlike the quadratic O(G-MSA), enabling illumination-guided attention in each basic unit of the network.This lower complexity addresses the computational burden that limits global Transformer layers in prior CNN-Transformer hybrids.
4. Experiment
Experiments across enhancement benchmarks, visual comparisons, user studies, and low-light detection evaluate Retinexformer’s effectiveness and efficiency. The method consistently improves image quality and practical vision performance while using fewer computational resources than key baselines.
- Quantitative Results: Retinexformer significantly outperforms state-of-the-art algorithms on eight quantitative enhancement datasets.
- Quantitative Results: 0.55, 1.32, 1.53, 1.57, 0.66, 0.33, 1.18, and 1.13 dB improvements over SNR-Net are reported on LOL-v1, LOL-v2-real, LOL-v2-synthetic, SID, SMID, SDSD-indoor, SDSD-outdoor, and FiveK.The comparison uses the listed datasets in that order.
- Efficiency: The method uses 40% of SNR-Net’s parameters and 59% of its FLOPS, while requiring only 1.4% and 6.2% of the parameters of IPT and Restormer.The SNR-Net comparison reports 1.61/4.01 parameters and 15.57/26.35 FLOPS.
- Quantitative Results: Retinexformer achieves 4.30–8.54 dB gains over selected Retinex-based methods and 1.95–4.26 dB gains over selected Transformer-based restoration methods.The reported comparisons cover seven benchmarks.
- Qualitative Results: Visual comparisons show that Retinexformer suppresses noise, exposure errors, color distortion, blur, black spots, and unnatural artifacts while restoring details and textures.These comparisons span SID, SMID, SDSD, and additional no-ground-truth datasets.
- User Study: Human subjects favored Retinexformer most on five datasets and second most on LOL-v1 and SDSD-indoor.The study used 23 human subjects evaluating exposure, color distortion, noise, and artifacts.
- Low-light Object Detection: The low-light detection experiment uses ExDark images, YOLO-v3 trained from scratch, and enhancement methods as fixed preprocessing modules.ExDark contains 7363 annotated under-exposed images, split into 5890 training and 1473 testing images.
- Ablation Study: Applying ORF and IG-MSA individually improves the baseline by 1.45 and 2.39 dB, while jointly applying them yields a 3.37 dB gain.The ablation study is conducted on SDSD-outdoor.
5. Conclusion
The paper presents Retinexformer as a Transformer-based low-light enhancement method built from a one-stage Retinex framework and illumination-guided restoration. Experiments across thirteen datasets, user studies, and low-light detection support its reported effectiveness and practical value.
- Retinexformer combines the One-stage Retinex-based Framework with an Illumination-Guided Transformer for low-light image enhancement.
- ORF models hidden and light-up-induced corruptions, estimates illumination, and restores noise, artifacts, exposure errors, and color distortion.
- Experiments report that Retinexformer outperforms state-of-the-art methods on thirteen datasets, with user-study and low-light detection results indicating practical value.