Source-linked AI summary
MSR-net:Low-light Image Enhancement Using Deep Convolutional Network
Liang Shen, Zihan Yue, Fan Feng, Quan Chen, Shihao Liu, Jie Ma
TL;DR
Low-light images suffer from reduced contrast and brightness, making later computer vision tasks more difficult. The paper connects multi-scale Retinex with CNNs and proposes MSR-net to learn an end-to-end dark-to-bright mapping. Experiments on synthetic and real-world data report qualitative and quantitative advantages over other state-of-the-art methods.
Problem
Low-light images often have very low contrast and brightness, obscuring details and increasing the difficulty of subsequent computer vision tasks.
Method
MSR-net treats low-light enhancement as supervised learning and directly learns an end-to-end mapping between dark and bright images using CNNs and Retinex theory.
Results
Experiments on synthetic and real-world images reveal qualitative and quantitative advantages over other state-of-the-art methods.
Takeaways & Limitations
Multi-scale Retinex can be represented as a feedforward CNN, enabling its surround functions and parameters to be optimized through back-propagation.
Takeaways & Limitations
Because of its limited receptive field, MSR-net can sometimes produce halo effects in very smooth regions such as clear sky.
Abstract
from arXiv · showhide
Images captured in low-light conditions usually suffer from very low contrast, which increases the difficulty of subsequent computer vision tasks in a great extent. In this paper, a low-light image enhancement model based on convolutional neural network and Retinex theory is proposed. Firstly, we show that multi-scale Retinex is equivalent to a feedforward convolutional neural network with different Gaussian convolution kernels. Motivated by this fact, we consider a Convolutional Neural Network(MSR-net) that directly learns an end-to-end mapping between dark and bright images. Different fundamentally from existing approaches, low-light image enhancement in this paper is regarded as a machine learning problem. In this model, most of the parameters are optimized by back-propagation, while the parameters of traditional models depend on the artificial setting. Experiments on a number of challenging images reveal the advantages of our method in comparison with other state-of-the-art methods from the qualitative and quantitative perspective.
1. Introduction
Low-light images often have low contrast and brightness, obscuring details and hindering subsequent vision tasks. The paper proposes MSR-net, a supervised CNN–Retinex approach that learns enhancement mappings from data.
- Low-light capture can bury image details in dark backgrounds and increase the difficulty of subsequent computer vision tasks.
- The paper combines convolutional neural networks with Retinex theory for low-light image enhancement.
- Multi-scale Retinex is related to a feedforward CNN with different Gaussian convolution kernels.
- The model formulates Retinex surround functions as convolutional layers whose parameters are optimized by back-propagation.
- MSR-net treats enhancement as supervised learning, mapping dark images to bright images through an end-to-end model.
- Experiments on challenging images report brighter, more natural results with clearer texture and richer details than other methods.
2. Related Work
Prior low-light enhancement work is organized around histogram-based and Retinex-based methods, while deep learning provides a related framework for learning image transformations.
- Low-light enhancement methods are broadly categorized as histogram-based or Retinex-based.
- Histogram equalization balances an image histogram, while Gamma Correction expands dark regions and compresses bright ones.
- Histogram-based methods process pixels individually without neighborhood dependence, which limits their treatment of surrounding image structure.
- Retinex methods decompose an image into reflection and illumination, forming the basis for several enhancement approaches.
- Deep neural networks have improved multiple high-level vision tasks and have also been applied to low-level tasks such as super-resolution and rain removal.
3. CNN Network for Low-light Image Enhancement
The paper reframes multi-scale Retinex as a feedforward CNN and builds MSR-net to learn low-light-to-bright image enhancement through three Retinex-inspired components.
- 3.1. Multi-scale Retinex is a CNN Network: Multi-scale Retinex is equivalent to a feedforward CNN using different Gaussian convolution kernels and a residual structure.Cascaded convolution layers represent Gaussian kernels, while concatenation and 1 × 1 convolution implement weighted averaging.
- 3.2. Proposed Method: MSR-net contains multi-scale logarithmic transformation, difference-of-convolution, and color restoration function components.These components correspond respectively to logarithmic enhancement, the difference-of-Gaussian-like operation, and color restoration.
- 3.2. Proposed Method: Multi-scale logarithmic transformation combines several logarithmic representations before convolution and ReLU layers to obtain a better image and accelerate convergence.The transformations use scale-dependent bases and are concatenated into a larger tensor for learned weighted processing.
- 3.2. Proposed Method: The color restoration function is implemented by a 1 × 1 convolutional layer with three output channels, producing the final enhanced image.The model uses this component to imitate MSR color restoration and address unnatural color appearance.
- 3.3. Objective function: MSR-net trains its output to approximate bright-image labels under the Frobenius norm, optimizing weights and biases by back-propagation while selecting hyper-parameters by grid search.The regularization parameter, logarithmic scales, transformation count, and convolutional depth are treated as hyper-parameters.
4. Experiments
Experiments evaluate MSR-net on synthetic and real-world low-light images, comparing visual quality, quantitative metrics, color correction, runtime, denoising, and hyper-parameter choices. Across these evaluations, MSR-net generally achieves stronger enhancement results, with deeper networks and multi-scale logarithmic transformation improving performance.
- Experimental setup: The experiments use synthetic test data, public real-world datasets, comparisons with four recent methods, runtime analysis, and hyper-parameter studies.The real-world evaluation uses MEF, NPE, and VV datasets, while the synthetic evaluation uses 2,000 test images.
- Results on synthetic and real-world data: MSR-net performs better in dark regions than competing methods on synthesized images and produces brighter or more natural results on real-world images.The real-world examples also show clearer texture and richer details in the Garden image.
- Results on synthetic test data: On synthesized images, MSR-net achieves higher SSIM and lower NIQE averages than the other evaluated methods across 2,000 test images.SSIM measures closeness to ground truth, while lower NIQE indicates higher image quality.
- Results on real-world data: Across the MEF, NPE, and VV datasets, MSR-net obtains lower NIQE and higher discrete entropy than the compared methods.Higher discrete entropy is associated with richer color and clearer outlines.
- Denoising: BM3D denoising after the deep network further improves visual quality on real-world low-light images.The paper treats denoising as post-processing for noise that can arise when handling real-world low-light images.
- Running time: The trained feedforward MSR-net runs significantly faster on GPU than methods,, and, except method.The comparison averages processing time over 100 test images at three image sizes.
- Study of MSR-net Parameters: Adding hidden layers improves SSIM, while multi-scale logarithmic transformation performs better than single-scale transformation; the final setting uses n = 4 and K = 10.The chosen configuration balances performance with running-time and hardware limits.
5. Conclusion
MSR-net is a feedforward convolutional network inspired by the equivalence between multi-scale Retinex and Gaussian-kernel convolution. Experiments report advantages over state-of-the-art methods, while limited receptive fields can produce halo effects in very smooth regions.
- MSR-net directly learns an end-to-end mapping from dark images to bright images with little extra pre/post-processing after optimization.
- The method is motivated by showing that multi-scale Retinex is equivalent to a feedforward convolutional neural network with different Gaussian convolution kernels.
- Experiments on synthetic and real-world data reveal advantages over other state-of-the-art methods from qualitative and quantitative perspectives.
- Limited receptive fields can cause halo effects in very smooth regions such as clear sky.The paper suggests enlarging the receptive field or adding hidden layers as possible remedies.