Source-linked AI summary
Low-Light Image Enhancement with Normalizing Flow
Yufei Wang, Renjie Wan, Wenhan Yang, Haoliang Li, Lap-Pui Chau, Alex C. Kot
TL;DR
Low-light enhancement must model a one-to-many relationship that pixel-wise deterministic methods do not capture well, causing issues such as improper brightness, residual noise, and artifacts. The paper proposes LLFlow, a conditional normalizing flow with an illumination-invariant color-map prior, and reports better enhancement quality on benchmark datasets, including well-exposed illumination, suppressed noise and artifacts, and richer colors.
Problem
Pixel-wise reconstruction losses impose deterministic mappings that inadequately capture diverse normally exposed-image distributions and can produce improper brightness, noise, and artifacts.
Method
LLFlow uses a conditional normalizing flow and an illumination-invariant color map as the prior mean to model normally exposed-image distributions conditioned on low-light inputs.
Results
Benchmark experiments report better enhancement quality, with well-exposed illumination, suppressed noise and artifacts, and richer colors.
Takeaways & Limitations
Modeling the conditional distribution and image-manifold structure provides an enhancement process that better characterizes structural context and visual distance.
Takeaways & Limitations
Classical normalizing-flow components may fail to model global image properties such as color saturation because they favor local pixel correlations.
Abstract
from arXiv · showhide
To enhance low-light images to normally-exposed ones is highly ill-posed, namely that the mapping relationship between them is one-to-many. Previous works based on the pixel-wise reconstruction losses and deterministic processes fail to capture the complex conditional distribution of normally exposed images, which results in improper brightness, residual noise, and artifacts. In this paper, we investigate to model this one-to-many relationship via a proposed normalizing flow model. An invertible network that takes the low-light images/features as the condition and learns to map the distribution of normally exposed images into a Gaussian distribution. In this way, the conditional distribution of the normally exposed images can be well modeled, and the enhancement process, i.e., the other inference direction of the invertible network, is equivalent to being constrained by a loss function that better describes the manifold structure of natural images during the training. The experimental results on the existing benchmark datasets show our method achieves better quantitative and qualitative results, obtaining better-exposed illumination, less noise and artifact, and richer colors.
1 Introduction
Low-light enhancement is ill-posed because one input can correspond to multiple normally exposed images, while pixel-wise deterministic methods inadequately model this variability. LLFlow addresses these issues with a conditional normalizing flow and illumination-invariant color-map prior.
- Pixel-wise losses impose a deterministic mapping even though one low-light image may correspond to reference images with different brightness.
- The proposed framework is reported to improve illumination adjustment, noise and artifact suppression, color saturation, and visual-distance measurement, with experiments and ablations supporting its effectiveness.
- GAN-based approaches partly alleviate this issue but require careful training and may overfit visual features or training-data properties.
- Normalizing flows model a more complicated conditional distribution by mapping the multi-modal image manifold into a latent distribution.
- Classical normalizing flows may fail to model global properties such as color saturation because they favor local pixel correlations.
- LLFlow combines an invertible network for the one-to-many image distribution with an encoder that uses an illumination-invariant color map as the prior mean.
2 Related works
Related work covers Retinex, deep-learning, multi-scale, progressive, and loss-based enhancement methods, alongside normalizing flows and their design trade-offs. Conditional flows extend distribution modeling, while flow invertibility and inductive biases constrain model capacity and captured image properties.
- Early enhancement methods use Retinex theory to correct illumination and suppress artifacts, while later methods use autoencoders and multi-scale features.
- Progressive Retinex frameworks jointly train illumination and reflection maps, and prior work uses MSE, l1, SSIM, smoothness, and color losses.
- A normalizing flow transforms a simple probability distribution into a complex one through invertible differentiable mappings and permits exact sample-density computation.
- Flow architectures require carefully designed layers for tractable inversion and Jacobian determinants, which limits generative-model capacity.
- LLFlow uses a conditional encoder and invertible network to learn normally exposed-image distributions conditioned on low-light inputs, with latent sampling producing different outputs.
- Normalizing flows may encode simple graphical structures and thereby suppress noise, but their inductive biases can limit representation of broader image properties.
3 Methodology
LLFlow replaces deterministic pixel-wise enhancement with a conditional normalizing flow that models normally exposed images given low-light inputs. Its encoder extracts illumination-invariant color information, while the invertible network learns the one-to-many mapping and supports likelihood-based training.
- 3.1 Preliminary: Pixel-wise reconstruction losses impose a deterministic mapping and provide insufficient regularization to distinguish realistic normally exposed images from noisy or artifact-filled outputs.The paper motivates modeling the conditional distribution of normally exposed images instead of relying only on predefined pixel-wise distributions.
- 3.2 Framework: The conditional flow maps a normally exposed image x, conditioned on the low-light image or its features, to a same-dimensional latent code z.The model uses z = Θ(x; xl) and relates image and latent densities through the change-of-variable theorem.
- 3.2 Framework: Maximum likelihood estimation trains the model by minimizing negative log-likelihood rather than the l1 reconstruction loss.This objective is intended to better characterize the distribution of high-quality normally exposed images.
- 3.2 Framework: The encoder input concatenates the low-light image, histogram-equalized image, color map, and noise map into 12 channels.The noise map is fed as an attention map, while histogram equalization helps address overly dark or bright regions.
- 3.2 Framework: The framework combines an encoder that extracts an illumination-invariant color map with an invertible network that maps normally exposed images to latent codes.The encoder uses processed low-light inputs and features, including histogram equalization, color information, and a noise map.
- 3.2 Framework: The invertible network learns a one-to-many relationship because the same scene can correspond to diverse illumination conditions.Training conditions the flow on the encoder output or reference color map, which are expected to be similar under the stated assumption.
4 Experiments
Experiments evaluate LLFlow on LOL and VE-LOL, including cross-dataset and intra-dataset settings, quantitative comparisons, visual comparisons, and ablations. The method outperforms competing methods across reported metrics while reducing artifacts and improving color and brightness consistency.
- LOL evaluation: On LOL, LLFlow significantly outperforms all competitors on PSNR, SSIM, and LPIPS.The authors associate these results with artifact suppression, color recovery, structural detail preservation, and improved alignment with human perception.
- VE-LOL evaluation: On VE-LOL cross-dataset evaluation, LLFlow significantly outperforms other methods on all metrics after training on LOL.Qualitatively, its real-captured outputs contain less noise and better color saturation.
- VE-LOL evaluation: In VE-LOL intra-dataset evaluation, LLFlow achieves the best performance and outperforms other methods by a large margin.Retraining on the more diverse VE-LOL data improves all of LLFlow’s metrics compared with the model trained on LOL.
- Ablation study: Varying latent variable z produces monotonically changing image brightness, indicating that LLFlow encodes dataset brightness variance.The experiment changes the extracted g(x_l) by constants from −0.4 to 0.4 in steps of 0.2.
- Ablation study: Gradient activation maps assign higher activation to artifact regions, showing that LLFlow can distinguish unrealistic areas without a reference image.The map uses histogram equalization for visualization.
- Ablation study: Training with negative log-likelihood substantially improves all metrics over training the same framework with l1 loss, while l1 training produces more obvious artifacts.The comparison supports modeling normally bright image distributions with the flow-based objective rather than a simplified pixel-wise loss.
5 Conclusion
The paper proposes a conditional normalizing-flow framework for low-light enhancement that models normally exposed images and outperforms state-of-the-art techniques quantitatively and qualitatively.
- The framework uses a normalizing flow conditioned on low-light images or features to model the complex conditional distribution of normally exposed images.It is trained with negative log-likelihood rather than relying on deterministic pixel-wise reconstruction.
- The learned distribution better characterizes structural context and measures visual distance in the image manifold.These properties support enhancement with well-exposed illumination, suppressed noise and artifacts, and richer colors.
- Experiments on existing benchmark datasets show better quantitative and qualitative results than state-of-the-art techniques.