PaperlayerView exact revision on arXiv

Source-linked AI summary

MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space

Jiangwei Weng, Zhiqiang Yan, Ying Tai, Jianjun Qian, Jian Yang, Jun Li

arXiv:2405.16105v1cs.CVcs.AI

TL;DR

Low-light enhancement must address poor global visibility and local degradation, while existing SSMs can model long-range information but have limited flexibility for local detail. MambaLLIE combines implicit Retinex-aware guidance with a global-then-local state space design, and the paper reports consistent best performance across benchmarks, low-light object detection, and a user study.

  • Problem

    Low-light images can have poor global visibility and local degradation, while vision SSMs' long-range modeling can leave local information insufficiently captured.

    Source: §1 · pp. 1–2

  • Method

    MambaLLIE combines implicit Retinex-aware guidance with a global-then-local state space block integrating local-enhanced state space and selective-kernel modules.

    Source: §1 · p. 2

  • Results

    MambaLLIE is reported to achieve the best performance across benchmarks, low-light object detection, and a user study; it averages 0.2 dB higher PSNR than RetinexFormer on benchmark datasets.

    Source: §4.2 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    The proposed GLSSB combines global and local information through implicit Retinex-aware selective kernels in a global-then-local state space design.

    Source: §6 · p. 9

    The method requires a reasonable illumination prior whose design relies on prior experience.

    Source: §5 · p. 9

Abstract

from arXiv · show

Recent advances in low light image enhancement have been dominated by Retinex-based learning framework, leveraging convolutional neural networks (CNNs) and Transformers. However, the vanilla Retinex theory primarily addresses global illumination degradation and neglects local issues such as noise and blur in dark conditions. Moreover, CNNs and Transformers struggle to capture global degradation due to their limited receptive fields. While state space models (SSMs) have shown promise in the long-sequence modeling, they face challenges in combining local invariants and global context in visual data. In this paper, we introduce MambaLLIE, an implicit Retinex-aware low light enhancer featuring a global-then-local state space design. We first propose a Local-Enhanced State Space Module (LESSM) that incorporates an augmented local bias within a 2D selective scan mechanism, enhancing the original SSMs by preserving local 2D dependency. Additionally, an Implicit Retinex-aware Selective Kernel module (IRSK) dynamically selects features using spatially-varying operations, adapting to varying inputs through an adaptive kernel selection process. Our Global-then-Local State Space Block (GLSSB) integrates LESSM and IRSK with LayerNorm as its core. This design enables MambaLLIE to achieve comprehensive global long-range modeling and flexible local feature aggregation. Extensive experiments demonstrate that MambaLLIE significantly outperforms state-of-the-art CNN and Transformer-based methods. Project Page: https://mamballie.github.io/anon/

1 Introduction

Low-light images suffer from poor global visibility and local degradation, while existing approaches face trade-offs between local detail, global context, and computational cost. MambaLLIE combines a global-then-local state space block with implicit Retinex-aware guidance to model both global and local dependencies.

1 Introduction

  • Insufficient lighting and sensor degradation produce poor global visibility and local color distortion and noise, which can affect human perception and high-level tasks such as object detection.Source: §1 · p. 1
  • CNNs aggregate local information but have limited receptive fields, while Transformer self-attention captures longer dependencies at quadratic computational cost.Source: §1 · p. 1
  • MambaLLIE’s ERF visualization is described as combining outward global perception with a large local receptive field, unlike the cited comparison methods.Source: Figure 1 · p. 2
  • Vision SSMs offer global information modeling with linear complexity, but their long-range focus can leave local interactions under-refined.Source: §1 · p. 2
  • MambaLLIE combines implicit Retinex-aware guidance with a global-then-local state space framework for low-light enhancement.Source: §1 · p. 2
  • Its global-then-local block integrates a local-enhanced state space module and an implicit Retinex-aware selective kernel to capture global and local dependencies.The selective kernel guides deeper representations without complex structural design or constraints for estimating physical priors.Source: §1 · p. 2

2 Related work

Related work spans end-to-end learning and Retinex-based enhancement, while vision SSMs offer global receptive fields but continue to face challenges in local information modeling.

Low Light Image Enhancement

  • End-to-end methods rely on training-data distributions, whereas physical-prior methods such as ZeroDCE [13] and RUAS [24] lack ideal reference guidance and can lag supervised models.Source: §2 · p. 2
  • Supervised Retinex methods decompose illumination and reflectance, with Retinex-Net [43], KinD [52], and URetinex-Net using staged or multi-branch designs; balancing complexity and efficiency remains challenging.Source: §2 · p. 3

Vision State Space Model

  • Vision SSM work includes global-receptive-field approaches such as VMamba [25] and Vim [53], while existing models are described as giving insufficient attention to local information.Other cited approaches target local scanning, lightweight global/local features, or enhanced Mamba representations.Source: §2 · p. 3

3 Methodology

MambaLLIE combines state-space modeling with Retinex-aware guidance in a global-then-local design. Its U-shaped pipeline integrates modules for global context, local 2D dependencies, and illumination-guided feature selection.

3.1 Preliminaries

  • Retinex methods decompose images into reflectance and illumination maps, but pixel-wise illumination adjustment misses noise and artifacts, while map restoration requires multiple branches and training constraints.Reflectance represents a static property of captured objects; illumination represents the illumination map.Source: §3.1 · p. 3
  • SSMs model long-range dependencies, but visual scan strategies can overlook neighborhood structure; MambaLLIE uses global perception before local detail to address this gap.The paper describes SSMs as continuous linear time-invariant systems and notes that fixed scanning can neglect local invariants.Source: §3.1 · pp. 3–4, §3.3 · p. 5

3.2 Overall Pipeline

  • MambaLLIE uses a U-shaped encoder-decoder with skip connections, feeding augmented inputs into GLSSB blocks and adding the input image to the final projected output.The augmented input includes the low-light image and maximum and mean priors.Source: §3.2 · pp. 4–5

3.3 Global-then-Local State Space Block

  • LESSM adds an augmented local bias to the state-space computation to preserve local 2D dependencies alongside long-range modeling.The bias is independent of the hidden state; the module processes two branches and aggregates their features element-wise.Source: §3.3 · p. 5
  • IRSK selects among depth-wise convolution kernels with different receptive fields using spatial selection guided by illumination priors.Its Retinex-aware maps are concatenated with input features through residual connections before further convolution and activation.Source: §3.3 · pp. 5–6

4 Experiments

The experiments evaluate MambaLLIE on paired low-light benchmarks, real-world detection and user perception, and ablations of its Retinex-aware and global-then-local components. The reported results favor MambaLLIE across benchmark metrics and real-world evaluations, while ablations support the combined design.

4.1 Benchmark Datasets and Implementation Details

  • Evaluation uses five paired low-light datasets; training uses 128 × 128 crops, batch size 8, and 1.5 × 10^5 iterations.The datasets are LOL-V2-real, LOL-V2-syn, SMID, SDSD-indoor, and SDSD-outdoor.Source: §4.1 · p. 6

4.2 Main Results on Benchmarks.

  • With roughly similar parameters, MambaLLIE reports an average 0.2 dB improvement over RetinexFormer on benchmark datasets and an average 1 dB PSNR improvement over SNR-Net across all datasets.It achieves comparable SSIM to SOTA methods on LOL-V2-real and SDSD-outdoor.Source: §4.2 · p. 7
  • MambaLLIE enhances brightness, preserves colors, and restores details where prior methods show insufficient illumination, color distortion, or degradation.The qualitative comparisons cover LOL-V2-real, LOL-V2-syn, SMID, SDSD-indoor, and SDSD-outdoor.Source: §4.2 · p. 7, Figure 3 · p. 7, Figure 4 · p. 7

4.3 Real World Experimental Evaluation

  • On ExDark, MambaLLIE achieves the best average detection result among adopted models and the best results on Car, Chair, Cup, and Table.With MambaLLIE preprocessing, YOLOv3 detects two persons and a chair in extreme-dark regions where other methods fail.Source: §4.3 · pp. 7–8, Table 2 · p. 8, Figure 5 · p. 8
  • MambaLLIE receives the best user-study scores across overall quality, local detail, and color distortion/noise ratings.The evaluation uses seven low-light images and ratings from 70 participants.Source: §4.3 · p. 8, Table 3 · p. 8, Figure 5 · p. 8

4.4 Ablation Study

  • 1.25 dB PSNR improvement over Baseline-1 and 1.00 dB over Baseline-2 support the implicit Retinex-aware framework.Baseline-1 removes Retinex guidance; Baseline-2 estimates an illumination map and brightens the image by element-wise multiplication.Source: §4.4 · p. 8, Table 4 · p. 9
  • LESSM improves PSNR by 0.33 dB over Baseline-1 and 0.08 dB over Baseline-2; combining it with IRSK yields the full model's highest PSNR and SSIM.The ablations compare the modules with vanilla state-space blocks and the LESSM-only variant.Source: §4.4 · pp. 8–9, Table 4 · p. 9
  • IRSK learns complementary positive and negative illumination features; small-first, larger-deeper kernels focus locally before broader feature fusion.The authors note that image enhancement can suffer from edge padding, especially after upsampling, while larger receptive fields support globally consistent results.Source: §4.4 · p. 9, Figure 6 · p. 9, Table 5 · p. 9

5 Limitation and Discussion

The approach requires a reasonably designed illumination prior based on prior experience. The authors also note that its MSE focus and PSNR/SSIM evaluation can have inherent biases, which they address with additional real-world evaluations.

5 Limitation and Discussion

  • The method requires a reasonable illumination prior, whose design relies on prior experience.The authors contrast this requirement with end-to-end methods.Source: §5 · p. 9
  • The authors note that enhancement models, including theirs, primarily focus on mean square error and use PSNR and SSIM, which have inherent biases.Source: §5 · p. 9
  • Additional real-world experiments were conducted to mitigate metric bias and further validate the approach.Source: §5 · p. 9

6 Conclusion

The conclusion presents MambaLLIE as a state-space model whose GLSSB combines global and local information through implicit Retinex-aware selective kernels. The authors report best performance across benchmarks, low-light object detection, and a user study, and identify low-light video enhancement as future work.

6 Conclusion

  • MambaLLIE consistently achieves the best performance on benchmarks, low-light object detection, and a user study.Source: §6 · p. 9
  • GLSSB combines global and local information through implicit Retinex-aware selective kernels in a global-then-local state-space framework.Source: §6 · p. 9
  • Future work will address local redundancy and global dependencies in low-light video enhancement using efficient state-space modeling.Source: §6 · p. 9

A Broader Impact.

The broader-impact discussion describes low-light image enhancement as improving degraded-image quality and sees potential for the proposed approach to influence other low-level tasks and state-space research. It also cautions that training-data distribution deviations may be associated with color deviation in real-world results.

A Broader Impact

  • Low-light image enhancement is described as improving the quality of degraded images.Source: A Broader Impact · p. 13
  • The authors suggest the method may advance other low-level tasks and inspire future research in state-space models.Source: A Broader Impact · p. 13
  • Deviations in the training-data distribution may leave generated real-world results with color deviation.Source: A Broader Impact · p. 13

B More Results.

The appendix provides additional qualitative comparisons with state-of-the-art methods, including comparisons for object detection; the captions do not state comparative outcomes.

B More Results

  • Figure 7 presents additional qualitative comparisons with state-of-the-art methods.Source: Figure 7 · p. 13
  • Figure 8 presents additional qualitative comparisons with state-of-the-art methods.Source: Figure 8 · p. 14
  • Figure 9 presents qualitative comparisons for object detection against state-of-the-art methods.Source: Figure 9 · p. 14

Related papers

Loading 2405.16105v1…