Source-linked AI summary
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Chongyi Li, Chunle Guo, Linghao Han, Jun Jiang, Ming-Ming Cheng, Jinwei Gu, Chen Change Loy
TL;DR
Low-light enhancement must improve images and videos captured under poor illumination, yet diverse real-world conditions make evaluation and generalization difficult. The paper surveys deep learning-based LLIE, contributes a cross-phone dataset and online platform, and evaluates methods across image, video, and dark face-detection tasks. Results show task- and data-format-dependent strengths rather than a universally best method, while real-world noise removal and generalization remain unresolved.
Problem
Low-light images and videos impair perception and high-level tasks, while comprehensive synthesis, diverse cross-phone evaluation data, and accessible method comparison have been lacking.
Method
The paper surveys deep learning-based LLIE methods, introduces a dataset captured by different mobile-phone cameras under diverse illumination, and provides an online comparison platform.
Results
No method always wins: strengths differ across image-quality metrics, real-world images, videos, computational efficiency, face detection, and raw-data enhancement.
Takeaways & Limitations
The survey, dataset, and platform provide reference resources for studying LLIE generalization and comparing methods across practical settings.
Takeaways & Limitations
Existing methods often generalize poorly to real-world images and cannot reliably remove unknown noise, sometimes amplifying it.
Abstract
from arXiv · showhide
Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination. Recent advances in this area are dominated by deep learning-based solutions, where many learning strategies, network structures, loss functions, training data, etc. have been employed. In this paper, we provide a comprehensive survey to cover various aspects ranging from algorithm taxonomy to open issues. To examine the generalization of existing methods, we propose a low-light image and video dataset, in which the images and videos are taken by different mobile phones' cameras under diverse illumination conditions. Besides, for the first time, we provide a unified online platform that covers many popular LLIE methods, of which the results can be produced through a user-friendly web interface. In addition to qualitative and quantitative evaluation of existing methods on publicly available and our proposed datasets, we also validate their performance in face detection in the dark.This survey together with the proposed dataset and online platform could serve as a reference source for future study and promote the development of this research field. The proposed platform and dataset as well as the collected methods, datasets, and evaluation metrics are publicly available and will be regularly updated.
1 INTRODUCTION
Low-light conditions degrade visual quality and high-level task performance, motivating deep learning-based enhancement. This survey addresses the field’s fragmented coverage by organizing methods, introducing evaluation resources, and providing an online platform.
- 1 INTRODUCTION: Poor illumination buries scene content, reduces contrast, boosts noise, and distorts color, compromising image quality and information transmission.These degradations affect high-level tasks such as object tracking, recognition, and detection.
- 1 INTRODUCTION: Smartphone low-light photography is especially challenging because cameras have limited aperture size, real-time processing requirements, and memory constraints.Low-light enhancement supports applications including visual surveillance, autonomous driving, and computational photography.
- 1 INTRODUCTION: Deep learning-based LLIE methods have expanded rapidly since 2017 across supervised, reinforcement, unsupervised, zero-shot, and semi-supervised learning strategies.The survey notes more than 100 deep learning-based papers from 2017 to 2021, while Figure 2 shows representative milestones.
- 1 INTRODUCTION: A comprehensive survey of deep learning-based LLIE was lacking despite the field’s dominance by deep learning methods.Earlier reviews focused mainly on conventional methods or existing algorithms and machine-vision evaluation.
- 1 INTRODUCTION: The paper contributes a cross-phone low-light image and video dataset and a unified online platform for evaluating popular enhancement methods without requiring users to have GPUs.The dataset spans diverse illumination conditions, while the platform produces results through a user-friendly web interface.
2 DEEP LEARNING-BASED LLIE
Deep learning-based LLIE maps a low-light input to an enhanced result by optimizing trainable network parameters under a loss function. Existing methods span multiple learning strategies and architectures, but supervised learning remains dominant while generalization and stable training remain difficult.
- 2.1 Problem Definition: Deep learning-based LLIE models a low-light image I as input to a network F with trainable parameters θ, producing an enhanced result.The formulation represents the image as having width W, height H, and three channels.
- 2.1 Problem Definition: Training seeks network parameters that minimize the loss between the predicted enhanced result and ground truth, using supervised or unsupervised losses.The loss function drives network optimization.
- 2.2 Learning Strategies: Existing LLIE methods are categorized into supervised, reinforcement, unsupervised, zero-shot, and semi-supervised learning, with supervised methods further divided by network design.Supervised approaches include end-to-end, deep Retinex-based, and realistic data-driven methods.
- 2.3 Representative Methods: End-to-end, Retinex-based, realistic-data, and synthesized-data approaches use varied network structures and training pipelines to brighten, denoise, decompose, or enhance low-light imagery and video.Examples include LLNet, MBLLEN, Retinex-Net, LightenNet, SIDGAN, and RetinexDIP.
- 2.2 Learning Strategies: 73% of surveyed deep learning-based LLIE methods use supervised learning, but paired-data scarcity, synthetic-real mismatch, and limited real-world generalization remain challenges.Alternative learning strategies seek to bypass supervised-learning difficulties but face issues including unstable training, color deviations, cross-domain relations, and reward design.
3 TECHNICAL REVIEW AND DISCUSSION
The survey reviews deep learning-based LLIE methods across architectures, Retinex integration, data, losses, and evaluation, highlighting generalization and measurement challenges.
- Technical review: The review organizes representative LLIE methods by technical characteristics including network structure, Retinex use, data format, loss functions, and training or testing data.Table 1 summarizes essential method characteristics, including Retinex usage, simulated or self-selected testing data, and trainable parameter counts.
- Network structures: U-Net-like networks dominate LLIE because they integrate multi-scale, low-level, and high-level features.The survey also identifies vanishing gradients in extremely low-light inputs and possible noise or redundant features from skip connections.
- Combination of Deep Model and Retinex Theory: Nearly one-third of methods combine deep networks with Retinex theory by estimating Retinex components or using illumination maps to guide learning.The combination can bridge model-based and deep learning approaches, but may retain reflectance assumptions and deep-network overfitting risks.
- Loss Function: LLIE models commonly use reconstruction, perceptual, smoothness, color, exposure, and adversarial losses for reference-based or non-reference training.Exposure loss measures enhancement exposure levels without paired or unpaired reference images.
- Training Datasets: Paired training data include real-world and synthetic datasets, while synthetic generation can apply Gamma correction followed by Gaussian, Poisson, or realistic noise.Gamma correction brightens inputs when γ <1 and darkens them when γ >1; synthetic-data limitations include artifacts, color deviations, and context loss.
- Testing Datasets and Evaluation Metrics: LLIE testing lacks an acknowledged benchmark, and common test sets may be small, repetitive, based on unknown settings, or not originally collected for LLIE.Evaluation metrics such as PSNR, MSE, MAE, and SSIM also inadequately capture human perception, while low-light video metrics remain unavailable.
4 BENCHMARKING AND EMPIRICAL ANALYSIS
The survey benchmarks low-light enhancement methods across public and newly proposed real-world data, image and video quality, efficiency, and face detection. Results show strong task- and dataset-dependent variation: no method consistently wins, and real low-light generalization remains difficult.
- A New Low-Light Image and Video Dataset: The LLIV-Phone dataset contains 120 videos and 45,148 images captured by 18 mobile-phone cameras under diverse illumination and indoor/outdoor scenes.Its real-world diversity is intended to evaluate model generalization and supports unsupervised training and realistic low-light data synthesis.
- Benchmarking Results: On LLIV-Phone images, all methods struggle to improve brightness while removing noise, and several produce artifacts, color deviations, or exposure errors.For light-source regions, none brightens the image without amplifying surrounding noise, indicating limited generalization to difficult real scenes.
- Benchmarking Results: Supervised methods generally outperform unsupervised, semi-supervised, and zero-shot methods on LOL-test and MIT-Adobe FiveK-test, but no method leads across all metrics and datasets.LLNet leads MSE and PSNR on LOL-test, TBEFN leads SSIM, KinD leads LPIPS, and MBLLEN leads all four metrics on MIT-Adobe FiveK-test without winning both datasets.
- Benchmarking Results: EEMEFN consistently outperforms SID across raw-data patterns and metrics, although both methods remain far from ground truth on some inputs and recover color imperfectly.Raw-format methods improve noisy inputs, but performance depends on sensor pattern and color recovery remains limited.
- Benchmarking Results: TBEFN achieves the best temporal coherence, while ExCNet performs worst with an ALV value of 1375.29 because its consecutive-frame enhancement is unstable.LLNet and EnlightenGAN rank second and third in temporal coherence, respectively.
5 OPEN ISSUES
The survey identifies limited generalization as a central open issue in low-light enhancement, alongside unresolved handling of unknown noise and artifacts.
- Generalization Capability: Existing LLIE methods can produce visually pleasing results but have limited generalization capability across datasets and real-world conditions.A method trained on MIT-Adobe FiveK cannot effectively enhance LOL images, and combining real and synthetic data does not resolve the issue.
- Removing Unknown Noises: Unknown noise remains difficult to remove, and enhancement methods may amplify it when camera noise types differ from training noise.Gaussian and Poisson noise augmentation does not adequately represent real noise in diverse phone-camera images.
- Removing Unknown Artifacts: Unknown artifacts from prior image degradations, such as JPEG compression or editing, remain an unresolved enhancement challenge.Downloaded images may contain artifact patterns not represented by standard enhancement assumptions.
6 FUTURE RESEARCH DIRECTIONS
The survey proposes improving learning strategies, evaluation, generalization, and video enhancement to address the limitations revealed by current LLIE research.
- Effective Learning Strategies: Current LLIE models mainly use supervised learning, which requires massive paired data and may overfit specific datasets.Unsupervised and zero-shot learning have been explored, but their relationships and effectiveness for LLIE remain insufficiently understood.
- Standardized Testing: A standard low-light image and video test dataset should contain many samples, ground truths, diverse scenes, and challenging illumination conditions.The survey identifies such a dataset as needed for consistent evaluation.
- Task-Specific Evaluation Metrics: More accurate task-specific evaluation metrics are needed because current IQA metrics emphasize either human perceptual quality or machine-task effects.The survey highlights this need especially for non-reference measurements.
- Robust Generalization Capability: Most methods fail on real-world test data because of limited generalization associated with synthetic or small-scale training data, ineffective structures, and unrealistic assumptions.The survey calls for methods that improve robust generalization.
- Extension to Low-Light Video Enhancement: Low-light video enhancement remains an important extension area beyond the rapid progress achieved for video enhancement in other low-level vision tasks.The supplied passage introduces video enhancement as a future research direction but does not specify a particular solution.
(Supplementary Material)
The supplementary material provides additional visual comparisons across datasets and evaluates temporal coherence using luminance curves and average luminance variance.
- Resources: The supplement makes the project page, video results, online testing platform, method repository, and proposed dataset publicly accessible.These resources support inspection and testing of the surveyed methods.
- Video Evaluation: Smoother luminance curves indicate better temporal coherence, while lower average luminance variance indicates better video-enhancement performance.The supplementary material reports both the curve-based interpretation and the average luminance variance criterion.
- Visual Comparisons: The supplement adds visual comparisons of enhancement methods across multiple low-light image and raw-image testing datasets.The comparisons cover LOL-test, MIT-Adobe FiveK-test, LLIV-Phone-imgT, SID-test, and DARK FACE, including face-detection results.
- Video Evaluation: Ten low-light videos captured by different mobile-phone cameras are evaluated through luminance curves from the proposed LLIV-Phone-vidT dataset.The supplement uses these curves to compare temporal coherence in enhanced videos.