PaperlayerView exact revision on arXiv

Source-linked AI summary

Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement

Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, Yulun Zhang

arXiv:2303.06705v3cs.CV

TL;DR

Low-light enhancement methods must address corruption overlooked by the Retinex model, while prior approaches face multi-stage training and limited long-range modeling. Retinexformer introduces a one-stage Retinex framework with an illumination-guided Transformer, and experiments report superior performance across thirteen datasets plus practical value in user studies and low-light detection.

  • Problem

    Low-light Retinex methods do not model corruptions hidden in dark scenes or introduced during light-up, and CNN-based approaches have limited long-range dependency modeling.

    Source: §1 · pp. 1–2, §3.1 · p. 3

  • Method

    Retinexformer combines ORF, which estimates illumination and restores corruption in one stage, with IGT, which uses illumination representations to guide non-local self-attention.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Retinexformer significantly outperforms state-of-the-art methods on thirteen datasets, with user-study and low-light detection results also indicating practical value.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The reported results support Retinexformer as a low-light enhancement method with performance and practical evidence spanning image enhancement and low-light detection.

    Source: §1 · p. 2, §5 · p. 8

    Directly applying original vision Transformers can be computationally unaffordable because self-attention complexity is quadratic in input spatial size.

    Source: §1 · p. 2

Abstract

from arXiv · show

When enhancing low-light images, many deep learning algorithms are based on the Retinex theory. However, the Retinex model does not consider the corruptions hidden in the dark or introduced by the light-up process. Besides, these methods usually require a tedious multi-stage training pipeline and rely on convolutional neural networks, showing limitations in capturing long-range dependencies. In this paper, we formulate a simple yet principled One-stage Retinex-based Framework (ORF). ORF first estimates the illumination information to light up the low-light image and then restores the corruption to produce the enhanced image. We design an Illumination-Guided Transformer (IGT) that utilizes illumination representations to direct the modeling of non-local interactions of regions with different lighting conditions. By plugging IGT into ORF, we obtain our algorithm, Retinexformer. Comprehensive quantitative and qualitative experiments demonstrate that our Retinexformer significantly outperforms state-of-the-art methods on thirteen benchmarks. The user study and application on low-light object detection also reveal the latent practical values of our method. Code, models, and results are available at https://github.com/caiyuanhao1998/Retinexformer

1. Introduction

Low-light enhancement must improve visibility while addressing corruptions such as noise, artifacts, and color distortion. Retinexformer proposes a one-stage Retinex framework and illumination-guided Transformer to address limitations of prior methods.

  • Low-light enhancement targets poor visibility and contrast while restoring noise, artifacts, and color distortion that affect perception and downstream vision tasks.Source: §1 · p. 1
  • Retinex-based methods model illumination but can introduce noise and local color distortion because they assume corruption-free images.Source: §1 · p. 1
  • CNN-based enhancement methods either learn brute-force mappings or use Retinex formulations with multi-stage training pipelines.Source: §1 · p. 1
  • CNN-based methods have limitations in capturing long-range dependencies, while full vision Transformers incur quadratic complexity with input spatial size.Source: §1 · p. 2
  • Retinexformer introduces ORF, which models corruption with perturbation terms, estimates illumination, restores corruption, and supports end-to-end one-stage training.Source: §1 · p. 2
  • IG-MSA uses illumination information to guide self-attention and model long-range interactions between regions with different exposure levels.Source: §1 · p. 2
  • Retinexformer outperforms state-of-the-art methods on thirteen datasets, with user-study and low-light detection results suggesting practical value.Source: §1 · p. 2

2. Related Work

Prior low-light enhancement research includes direct amplification, Retinex-based conventional methods, and Transformer applications. These approaches motivate a framework that better accounts for illumination, corruption, and non-local dependencies.

Plain Methods

  • Plain methods such as histogram equalization and gamma correction amplify visibility and contrast but barely consider illumination factors.Source: §2.1 · p. 2

Traditional Cognition Methods

  • Traditional cognition methods use Retinex theory and illumination factors, but their assumptions can conflict with real under-exposed scenes.Source: §2.1 · p. 2

Vision Transformer

  • Transformers have been applied across high-level and low-level computer vision tasks, including image restoration and synthesis.Source: §2.2 · p. 2
  • Retinexformer’s overview combines an illumination estimator with an IGT corruption restorer, whose IG-MSA directs self-attention using illumination representations.Source: Figure 2 · p. 3

3. Method

Retinexformer uses a one-stage Retinex framework that estimates illumination, lights up the input, and restores corruption. Its Illumination-Guided Transformer uses illumination features to guide efficient long-range attention across differently lit regions.

One-stage Retinex-based Framework

  • The Retinex model is extended with perturbations to reflectance and illumination to represent corruptions hidden in dark scenes or introduced during light-up.These corruptions include noise, artifacts, under-/over-exposure, and color distortion.Source: §3.1 · p. 3
  • ORF estimates illumination information to produce a lit-up image and feature, then feeds both into a corruption restorer that outputs the enhanced image.The illumination estimator takes the low-light image and its illumination prior map as inputs.Source: §3.1 · pp. 3–4, §3 · p. 3
  • The illumination estimator fuses the image with a channel-mean illumination prior and models interactions between differently lit regions to generate light-up features and a three-channel light-up map.Well-exposed regions provide semantic contextual information for under-exposed regions.Source: §3.1 · p. 4

Illumination-Guided Transformer

  • IGT serves as the corruption restorer in a three-scale U-shaped architecture, producing a residual image that is added to the lit-up image.Downsampling, skip connections, and upsampling form the restoration path, with Ien = Ilu + Ire.Source: §3.2 · p. 4
  • IG-MSA reshapes image and illumination features into tokens and uses illumination-guided representations to direct self-attention across regions with different lighting conditions.The guidance emphasizes contextual information from better-lit regions that can help restore darker, more corrupted regions.Source: §3.2 · p. 4
  • O(IG-MSA) is linear in spatial size, unlike the quadratic O(G-MSA), enabling illumination-guided attention in each basic unit of the network.This lower complexity addresses the computational burden that limits global Transformer layers in prior CNN-Transformer hybrids.Source: §3.2 · pp. 4–5

4. Experiment

Experiments across enhancement benchmarks, visual comparisons, user studies, and low-light detection evaluate Retinexformer’s effectiveness and efficiency. The method consistently improves image quality and practical vision performance while using fewer computational resources than key baselines.

Quantitative Results

  • Retinexformer significantly outperforms state-of-the-art algorithms on eight quantitative enhancement datasets.Source: §4.2 · p. 5, Table 1 · p. 6
  • 0.55, 1.32, 1.53, 1.57, 0.66, 0.33, 1.18, and 1.13 dB improvements over SNR-Net are reported on LOL-v1, LOL-v2-real, LOL-v2-synthetic, SID, SMID, SDSD-indoor, SDSD-outdoor, and FiveK.The comparison uses the listed datasets in that order.Source: §4.2 · p. 6

Efficiency

  • The method uses 40% of SNR-Net’s parameters and 59% of its FLOPS, while requiring only 1.4% and 6.2% of the parameters of IPT and Restormer.The SNR-Net comparison reports 1.61/4.01 parameters and 15.57/26.35 FLOPS.Source: §4.2 · p. 6

Quantitative Results

  • Retinexformer achieves 4.30–8.54 dB gains over selected Retinex-based methods and 1.95–4.26 dB gains over selected Transformer-based restoration methods.The reported comparisons cover seven benchmarks.Source: §4.2 · p. 6

Qualitative Results

  • Visual comparisons show that Retinexformer suppresses noise, exposure errors, color distortion, blur, black spots, and unnatural artifacts while restoring details and textures.These comparisons span SID, SMID, SDSD, and additional no-ground-truth datasets.Source: §4.2 · pp. 6–7, Figure 4 · p. 6, Figure 5 · p. 7

User Study

  • Human subjects favored Retinexformer most on five datasets and second most on LOL-v1 and SDSD-indoor.The study used 23 human subjects evaluating exposure, color distortion, noise, and artifacts.Source: §4.2 · p. 7

Low-light Object Detection

  • The low-light detection experiment uses ExDark images, YOLO-v3 trained from scratch, and enhancement methods as fixed preprocessing modules.ExDark contains 7363 annotated under-exposed images, split into 5890 training and 1473 testing images.Source: §4.3 · pp. 7–8, §4.2 · p. 7

Ablation Study

  • Applying ORF and IG-MSA individually improves the baseline by 1.45 and 2.39 dB, while jointly applying them yields a 3.37 dB gain.The ablation study is conducted on SDSD-outdoor.Source: §4.4 · p. 8

5. Conclusion

The paper presents Retinexformer as a Transformer-based low-light enhancement method built from a one-stage Retinex framework and illumination-guided restoration. Experiments across thirteen datasets, user studies, and low-light detection support its reported effectiveness and practical value.

  • Retinexformer combines the One-stage Retinex-based Framework with an Illumination-Guided Transformer for low-light image enhancement.Source: §5 · p. 8
  • ORF models hidden and light-up-induced corruptions, estimates illumination, and restores noise, artifacts, exposure errors, and color distortion.Source: §5 · p. 8
  • Experiments report that Retinexformer outperforms state-of-the-art methods on thirteen datasets, with user-study and low-light detection results indicating practical value.Source: §5 · p. 8

Related papers

Loading 2303.06705v3…