Source-linked AI summary
Toward Fast, Flexible, and Robust Low-Light Image Enhancement
Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, Zhongxuan Luo
TL;DR
Low-light enhancement methods struggle to balance visual quality, computational efficiency, and adaptation to unknown complex scenes. SCI uses cascaded illumination learning with weight sharing, a self-calibrated training module, and unsupervised loss, while testing with a single basic block. Experiments report strong enhancement quality and efficiency, with benefits extending to dark face detection and nighttime semantic segmentation.
Problem
Existing low-light image enhancement techniques struggle to jointly address visual quality, computational efficiency, and unknown complex scenes.
Method
SCI combines weight-shared cascaded illumination learning, a self-calibrated module for training, and unsupervised fidelity and smoothing losses.
Results
SCI achieves strong low-light enhancement performance and efficiency, with reported effectiveness also demonstrated in dark face detection and nighttime semantic segmentation.
Takeaways & Limitations
The self-calibrated module enables single-block testing while supporting operation-insensitive adaptability and model-irrelevant generality.
Abstract
from arXiv · showhide
Existing low-light image enhancement techniques are mostly not only difficult to deal with both visual quality and computational efficiency but also commonly invalid in unknown complex scenarios. In this paper, we develop a new Self-Calibrated Illumination (SCI) learning framework for fast, flexible, and robust brightening images in real-world low-light scenarios. To be specific, we establish a cascaded illumination learning process with weight sharing to handle this task. Considering the computational burden of the cascaded pattern, we construct the self-calibrated module which realizes the convergence between results of each stage, producing the gains that only use the single basic block for inference (yet has not been exploited in previous works), which drastically diminishes computation cost. We then define the unsupervised training loss to elevate the model capability that can adapt to general scenes. Further, we make comprehensive explorations to excavate SCI's inherent properties (lacking in existing works) including operation-insensitive adaptability (acquiring stable performance under the settings of different simple operations) and model-irrelevant generality (can be applied to illumination-based existing works to improve performance). Finally, plenty of experiments and ablation studies fully indicate our superiority in both quality and efficiency. Applications on low-light face detection and nighttime semantic segmentation fully reveal the latent practical values for SCI. The source code is available at https://github.com/vis-opt-group/SCI.
1. Introduction
Low-light enhancement methods face trade-offs among visual quality, computational efficiency, and robustness in unknown complex scenes. SCI addresses these issues with self-calibrated illumination learning, unsupervised adaptation, and properties supporting broad operation and model generality.
- Related methods: Existing model-based methods rely on defined illumination regularizations, which can produce unsatisfying results and require manual intervention.Reported limitations include overexposure and the need to manually define regularizations.
- Related methods: Network-based methods have improved low-light enhancement through datasets, architectures, recursive processing, and specialized training strategies, but can still produce unnatural results.RetinexNet is specifically described as producing unnatural enhanced results, while later methods modify losses, architectures, or training strategies.
- SCI framework: SCI develops a self-calibrated illumination framework that redevelops intermediate outputs to strengthen a single basic block and accelerate inference.The module is designed to promote convergence between stage results while reducing the computational burden of cascaded processing.
- SCI framework: SCI uses an unsupervised training loss to constrain stage outputs and improve adaptation to diverse scenes.The training objective is intended to address limitations associated with paired data and generalize across scenes.
- SCI properties and applications: SCI exhibits operation-insensitive adaptability and model-irrelevant generality, properties not found in existing works.The paper also evaluates SCI on dark face detection and nighttime semantic segmentation as downstream applications.
2. The Proposed Method
SCI trains a cascaded, weight-shared illumination process assisted by a self-calibrated module, then uses only one illumination-estimation block at testing. Its unsupervised losses and self-calibration promote stage convergence, stable exposure, and lower inference cost.
- 2.1. Illumination Learning with Weight Sharing: SCI progressively estimates illumination with the same mapping Hθ and shared weights at every stage.The illumination and low-light observation are modeled through a residual update, reducing the difficulty of direct illumination prediction.
- 2.2. Self-Calibrated Module: The self-calibrated module adds a learned map to the original low-light input before subsequent illumination-estimation stages.This converted input represents the difference between each stage’s input and the first-stage input.
- 2.2. Self-Calibrated Module: Stage outputs converge to the same value with self-calibration, enabling single-block testing after multi-stage shared-weight training.The t-SNE comparison reports convergence with the module but not consistently without it.
- 2.3. Unsupervised Training Loss: SCI uses an unsupervised total loss combining fidelity and smoothing terms to constrain illumination without hand-crafted ground truth.The fidelity term constrains each output using the redefined input y + s_{t−1}, while the smoothing term uses a spatially variant ℓ1 norm.
3. Exploring Algorithmic Properties
SCI exhibits stable enhancement across simple operator settings and can improve an existing illumination-based method, while its evaluations include visual comparisons on MIT, LSRW, and challenging in-the-wild examples.
- 3.1. Operation-Insensitive Adaptability: SCI maintains stable performance across different simple Hθ settings and produces similar brightened results.The tested settings vary the number of 3×3 convolution+ReLU blocks.
- 3.1. Operation-Insensitive Adaptability: Residual learning and element-wise division together support SCI’s adaptability to different simple operations.
- 3. Exploring Algorithmic Properties: The study reports visual comparisons on MIT, LSRW, and challenging in-the-wild examples, alongside quantitative enhancement metrics and efficiency measurements.Table 3 covers PSNR, SSIM, DE, EME, LOE, and NIQE; Table 4 covers model size, FLOPs, and GPU inference time.
- 3.2. Model-Irrelevant Generality: SCI training significantly improves RUAS performance even when evaluation uses only a single unrolling block.
- 3.2. Model-Irrelevant Generality: Applying SCI to RUAS also remarkably suppresses the overexposure present in the original method.
4. Experimental Results
SCI is evaluated for enhancement, computational efficiency, real-world robustness, and downstream dark face detection and nighttime semantic segmentation. The experiments also analyze its illumination components and algorithmic behavior.
- Experimental Evaluation on Benchmarks: Experiments compare SCI with model-based, supervised, and unsupervised low-light enhancement methods on benchmark and in-the-wild settings.The evaluation includes computational measurements and downstream applications.
- Experimental Evaluation on Benchmarks: SCI achieves competitive enhancement performance, with vivid colors and prominent textures, while advanced deep networks may introduce veils, inconspicuous details, and unnatural colors.The reported visual comparisons include benchmark results and additional examples from challenging DARK FACE and ExDark scenes.
- Computational Efficiency: SCI is the most lightweight among the compared CNN-based networks in model size, FLOPs, and GPU-seconds running time.The comparison is reported in Table 4 against recently proposed CNN-based methods.
- Dark Face Detection: SCI and SCI+ achieve the best dark face detection scores, with SCI+ outperforming the fine-tuned SCI version.The reinforced SCI+ version jointly trains enhancement and detection losses, and visual results show improved detection of smaller objects.
- Nighttime Semantic Segmentation: SCI achieves significantly superior nighttime semantic segmentation performance, while competing enhancement methods produce artifacts that damage segmentation maps.The segmentation model is retrained on enhanced results from all compared methods.
- Algorithmic Analyses: The estimated illumination from SCI has excellent smoothness, supporting more visually friendly reflectance and enhanced results that suppress overexposure while enriching image structures.The analysis compares SCI with RetinexNet, KinD, and SSIENet and examines different illumination-learning modes.
5. Concluding Remarks
The paper concludes that SCI is a lightweight and effective framework for real-world low-light enhancement. Its reported effectiveness extends across image quality, dark face detection, and nighttime semantic segmentation, while its auxiliary training process suggests broader applicability to low-level vision.
- Concluding Remarks: SCI is presented as a lightweight yet effective framework for low-light image enhancement in different real-world scenarios.The conclusion emphasizes both the framework's efficiency and its effectiveness.
- Concluding Remarks: Extensive experiments report SCI's effectiveness and superiority in low-light enhancement, dark face detection, and nighttime semantic segmentation.These results are stated across the enhancement task and two downstream vision tasks.
- Concluding Remarks: SCI combines superior image quality with inference speed and introduces an auxiliary-process perspective for improving the basic unit during training in other low-level vision problems.The broader-impacts discussion frames this perspective as relevant to improving practicability in real-world scenarios.