Source-linked AI summary
LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models
Hai Jiang, Ao Luo, Xiaohong Liu, Songchen Han, Shuaicheng Liu
TL;DR
Low-light enhancement remains challenging because existing methods can overfit, generalize poorly, or depend on known degradation modes. LightenDiffusion combines latent-space Retinex decomposition with diffusion-based unsupervised restoration using unpaired data. It outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while generalizing better across scenes.
Problem
Low-light enhancement is difficult because learning-based methods can overfit and zero-shot diffusion methods are limited by known degradation modes.
Method
LightenDiffusion decomposes unpaired low-light and normal-light features into latent-space reflectance and illumination maps, then uses them for guided diffusion restoration with self-constrained consistency.
Results
LightenDiffusion outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while offering better generalization abilities.
Takeaways & Limitations
The framework provides visually favorable unsupervised low-light enhancement from extensive unpaired real-world data across various scenes.
Takeaways & Limitations
Learning-based competitors can overfit and generalize poorly, while zero-shot diffusion methods remain limited by known degradation modes.
Abstract
from arXiv · showhide
In this paper, we propose a diffusion-based unsupervised framework that incorporates physically explainable Retinex theory with diffusion models for low-light image enhancement, named LightenDiffusion. Specifically, we present a content-transfer decomposition network that performs Retinex decomposition within the latent space instead of image space as in previous approaches, enabling the encoded features of unpaired low-light and normal-light images to be decomposed into content-rich reflectance maps and content-free illumination maps. Subsequently, the reflectance map of the low-light image and the illumination map of the normal-light image are taken as input to the diffusion model for unsupervised restoration with the guidance of the low-light feature, where a self-constrained consistency loss is further proposed to eliminate the interference of normal-light content on the restored results to improve overall visual quality. Extensive experiments on publicly available real-world benchmarks show that the proposed LightenDiffusion outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while being more generalizable to various scenes. Our code is available at https://github.com/JianghaiSCU/LightenDiffusion.
1 Introduction
Low-light enhancement is difficult because illumination varies and the task is ill-posed, while existing learning and diffusion approaches face generalization or degradation-mode limitations. LightenDiffusion addresses these issues with latent-space Retinex decomposition and diffusion-based unsupervised restoration, achieving stronger unsupervised performance and comparability with supervised methods.
- Low-light images suffer poor visibility and noise, harming downstream vision tasks.
- LLIE is ill-posed because suitable handcrafted priors are difficult to select across illumination conditions.
- Learning-based methods can overfit and generalize poorly, producing overexposure, color distortion, blurred details, or amplified noise.
- Zero-shot diffusion methods depend on known degradation modes and can under-enhance real-world scenes with diverse, unknown distortions.
- LightenDiffusion decomposes unpaired low-light and normal-light features into content-rich reflectance and content-free illumination maps in latent space.The low-light reflectance and normal-light illumination then guide diffusion-based restoration, while self-constrained consistency reduces normal-light content interference.
- LightenDiffusion outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods with better generalization abilities.
2 Related Work
Prior low-light enhancement methods span handcrafted, learning-based, and diffusion-based approaches. Retinex methods decompose illumination and reflectance, while newer diffusion methods commonly rely on paired supervision or zero-shot pretrained priors.
- Traditional methods use handcrafted priors such as histogram equalization and Retinex theory to improve low-light images.
- Retinex-based methods decompose images into reflectance and illumination maps, improving visual quality by changing illumination dynamic range.
- Learning-based methods are categorized as supervised, semi-supervised, or unsupervised and generally show greater robustness than traditional methods.
- Diffusion restoration methods often train conditionally from scratch with paired data, whereas zero-shot methods use pretrained diffusion priors.
3 Methodology
LightenDiffusion performs Retinex decomposition in latent space, then uses low-light reflectance, normal-light illumination, and low-light feature guidance in a diffusion restoration process. A self-constrained consistency loss further preserves the low-light image’s intrinsic information during reconstruction.
- Overview: The encoder maps unpaired low-light and normal-light images into latent features, which CTDN decomposes into reflectance and illumination maps.CTDN produces content-rich reflectance maps and content-free illumination maps from Flow and Fhigh.
- Content-Transfer Decomposition: Retinex theory models each image as the Hadamard product of reflectance, containing inherent content, and illumination, containing contrast and brightness information.Existing image-space decomposition can leave content information in illumination maps, motivating latent-space decomposition.
- Content-Transfer Decomposition: CTDN performs decomposition in latent space to generate reflectance maps with abundant content details and illumination maps unaffected by content influences.The network refines initial maps through convolutional, cross-attention, and self-attention branches.
- Latent-Retinex Diffusion Models: The diffusion model uses Rlow and Lhigh as its input, progressively adds Gaussian noise, then reversely denoises sampled noise under guidance from the encoded low-light feature.The restored feature is decoded into the final enhanced image.
- Latent-Retinex Diffusion Models: The self-constrained consistency loss encourages the restored feature to retain the low-light image’s intrinsic information during diffusion reconstruction.It uses a pseudo-label based on Rlow and an illumination-corrected map to constrain feature similarity.
4 Experiments
Experiments evaluate LightenDiffusion on paired and unpaired benchmarks, qualitative restoration, face detection, and ablations. The method achieves strong generalization and visual quality, while latent decomposition and self-constrained consistency improve restoration trade-offs.
- Experimental Settings: Experiments cover paired LOL and LSRW datasets, unpaired DICM, NPE, and VV benchmarks, and low-light face detection on DARK FACE.Evaluation uses distortion, perceptual, and downstream detection measures across paired and real-world unpaired settings.
- Quantitative Comparison: LightenDiffusion outperforms all unsupervised competitors on LOL and LSRW, surpasses supervised methods on LSRW in PSNR and SSIM, and achieves the best results on all three unpaired datasets.On LSRW, LPIPS is slightly inferior despite the highest PSNR and SSIM.
- Qualitative Comparison: Qualitative comparisons show improved global and local contrast, sharper details, noise suppression, correct exposure, and vivid color across paired and unpaired scenes.Competing methods exhibit underexposure, color distortion, noise amplification, artifacts, or overexposure in the reported examples.
- Ablation Study: Latent-space decomposition produces cleaner illumination maps than image-space decomposition, while increasing latent scale generally improves performance and inference speed before degradation at k = 4.The default k = 3 balances performance and efficiency because k = 4 substantially reduces feature richness.
- Ablation Study: The content-transfer decomposition network yields content-rich reflectance and content-free illumination maps, avoiding the blurry details and artifacts produced by previous decomposition networks.The comparison replaces CTDN with RetinexNet, URetinexNet, and PairLIE decomposition networks.
- Ablation Study: Removing the self-constrained consistency loss decreases overall performance, whereas increasing vanilla diffusion sampling to S = 100 reaches comparable performance at almost 4 times slower inference.The result supports the loss as an efficiency-oriented alternative to substantially increasing sampling steps.
5 Conclusion
The paper concludes that LightenDiffusion combines latent-space Retinex decomposition with diffusion-based unsupervised restoration. Its experiments report quantitative and visual advantages over state-of-the-art competitors.
- Method: LightenDiffusion uses a content-transfer decomposition network to obtain content-rich reflectance and content-free illumination maps in latent space.The low-light reflectance and normal-light illumination guide subsequent unsupervised restoration.
- Method: A self-constrained consistency loss constrains restored results to preserve the low-light input’s inherent content information.The loss is introduced as an additional constraint on the restored result.
- Results: Experimental results show that LightenDiffusion outperforms state-of-the-art competitors quantitatively and visually.The conclusion summarizes the reported evaluation outcome without specifying a single benchmark or metric.