Source-linked AI summary
Attention Guided Low-light Image Enhancement with a Large Scale Low-light Simulation Dataset
Feifan Lv, Yu Li, Feng Lu
TL;DR
Low-light enhancement requires recovering brightness while handling color distortion and noise, but large-scale paired training data is limited. The paper builds a simulated paired dataset and an attention-guided multi-branch network for simultaneous enhancement and denoising. Extensive experiments report high-fidelity results and large-margin improvements over state-of-the-art methods.
Problem
Low-light enhancement must address brightness, color, contrast, and noise together, while effective training is constrained by limited large-scale paired datasets.
Method
The paper constructs a realistic large-scale simulated paired dataset and uses under-exposure and noise attention maps to guide a multi-branch network performing enhancement and denoising simultaneously.
Results
Extensive experiments report that the proposed solution outperforms state-of-the-art methods by a large margin.
Takeaways & Limitations
The proposed dataset and attention-guided solution provide a benchmark and method for region-adaptive low-light enhancement with simultaneous denoising.
Takeaways & Limitations
The method fails to recover face details when parts of the face are totally dark and can produce blocking artifacts under heavy compression.
Abstract
from arXiv · showhide
Low-light image enhancement is challenging in that it needs to consider not only brightness recovery but also complex issues like color distortion and noise, which usually hide in the dark. Simply adjusting the brightness of a low-light image will inevitably amplify those artifacts. To address this difficult problem, this paper proposes a novel end-to-end attention-guided method based on multi-branch convolutional neural network. To this end, we first construct a synthetic dataset with carefully designed low-light simulation strategies. The dataset is much larger and more diverse than existing ones. With the new dataset for training, our method learns two attention maps to guide the brightness enhancement and denoising tasks respectively. The first attention map distinguishes underexposed regions from well lit regions, and the second attention map distinguishes noises from real textures. With their guidance, the proposed multi-branch decomposition-and-fusion enhancement network works in an input adaptive way. Moreover, a reinforcement-net further enhances color and contrast of the output image. Extensive experiments on multiple datasets demonstrate that our method can produce high fidelity enhancement results for low-light images and outperforms the current state-of-the-art methods by a large margin both quantitatively and visually.
1 Introduction
Low-light enhancement must jointly address visibility, contrast, color, brightness, and noise, while paired training data remains limited. The paper responds with a large-scale simulated dataset and an attention-guided network for simultaneous enhancement and denoising.
- Low-light images suffer from poor visibility, low contrast, color and brightness issues, and noise that complicate enhancement.
- Sequential denoising and enhancement is problematic because denoising first causes blurring, whereas enhancement first amplifies noise.
- The method uses under-exposed and noise attention maps to guide region-adaptive brightness enhancement and denoising simultaneously.
- The proposed simulation pipeline creates realistic low-light images with well-exposed references, addressing the shortage of large-scale paired training data.
- Experiments report that the method outperforms state-of-the-art methods by a large margin.
2 Related Work
Prior low-light enhancement research spans traditional histogram- and Retinex-based methods, learning-based enhancement, and specialized denoising techniques. Existing datasets use captured or synthetically degraded paired images, with limitations in scene coverage and exposure severity.
- Traditional enhancement methods: Traditional enhancement methods primarily use histogram equalization or Retinex-based illumination and reflection modeling.HE-based methods incorporate different priors and constraints, while Retinex methods estimate or manipulate illumination maps.
- Learning-based enhancement methods: Learning-based methods apply end-to-end networks, GANs, auto-encoders, and raw-sensor pipelines to low-light enhancement and related image-translation tasks.Representative approaches address enhancement, denoising, perceptual translation, computational cost, unpaired learning, or extremely low-light raw images.
- Image denoising methods: Image denoising research includes filter-based, deep-learning-based, dictionary-learning, sparse-representation, and variance-stabilizing approaches.BM3D and DnCNN represent Gaussian-denoising methods, while other approaches target Poisson noise using PCA-related or VST-based techniques.
- Low-light Image Enhancement Datasets: Low-light datasets have been built through paired captures with varied camera configurations or through retouching and linear degradation of images.LOL and SID use capture-based solutions, whereas DeepUPE provides 3,000 underexposed images with expert-retouched references.
- Low-light Image Enhancement Datasets: The proposed dataset selects normal-exposed public images, simulates low-light conditions, adds noise, and enhances the originals to create high-quality references.The construction pipeline is designed to synthesize realistic low-light images alongside reference images through exposure correction and contrast/details amplification.
3 Large Scale Low-Light Simulation Dataset
The paper constructs a large-scale paired low-light dataset by selecting high-quality source images and simulating diverse low-light conditions, noise, and enhanced references. The dataset addresses scale, quality, exposure-range, and annotation-compatibility limitations of existing datasets.
- Dataset construction: The dataset begins by selecting normally exposed, sharp, colorful images from large-scale public datasets as enhancement references.Selection considers darkness, blur, and color through sequential estimation stages.
- Dataset construction: 97,030 images are selected from 344,272 public images; the test set contains 965 images and the data-balanced training subset contains 22,656 images.The selected images are used to build the synthetic dataset and its training and test splits.
- Target image synthesis: The simulation synthesizes paired low-light and normal-light images by combining linear and gamma transformations to approximate different exposure levels.The transformation parameters are sampled as α ∼U(0.9, 1), β ∼ U(0.5, 1), and γ ∼U(1.5, 5).
- Target image synthesis: A Gaussian-Poisson mixed noise model incorporates the in-camera processing pipeline to simulate realistic low-light noise.The pipeline models camera response, RGB-to-Bayer conversion, Poisson noise, and demosaicing, while omitting compression.
- Target image synthesis: Exposure fusion generates higher-contrast and better-color reference images for a second enhancement step, addressing the risk of low-contrast regression targets.The limitation arises because selected images may be slightly over-exposed and noise removal can introduce smoothness.
- Dataset comparison: Compared with existing datasets, the synthetic dataset provides varied underexposure levels, high-quality paired pixel-aligned images, noise and exposure maps, and compatibility with annotated public datasets.Annotations such as bounding boxes and semantic segmentation masks can be retained for low-light model adaptation.
4 Attention-guided Low-light Enhancement
The method uses exposure and noise attention to guide simultaneous enhancement and denoising through a multi-branch network, followed by contrast re-enhancement. Its loss combines brightness, structural, perceptual, and regional objectives to improve output quality.
- Network Architecture: The fully convolutional architecture contains Attention-Net, Noise-Net, Enhancement-Net, and Reinforce-Net for guided enhancement, denoising, and contrast re-enhancement.Attention-Net estimates illumination, Noise-Net guides denoising, Enhancement-Net performs both tasks, and Reinforce-Net addresses regression-induced low contrast.
- Attention-Net: The ue-attention map estimates regional underexposure in [0, 1], assigning lower values to more illuminated regions.It guides enhancement toward underexposed areas while avoiding over-enhancement of normally exposed regions.
- Noise-Net: The noise map estimates noise distribution in [0, 1], while exposure guidance and dilated convolutions support adaptive noise estimation.The paper relates noise distribution to exposure and uses a larger receptive field for Noise-Net.
- Enhancement-Net: Enhancement-Net decomposes enhancement into aspect-specific subproblems and uses multi-branch processing to produce the final output.Its branches are implemented with feature extraction, enhancement, and fusion modules, using structures inspired by U-Net, Res-Net, and dilated convolutions.
- Loss Function: The loss function combines brightness, structural, perceptual, and regional components to improve visual quality and balance enhancement across regions.The structural, perceptual, and regional terms target structure preservation, higher-level visual information, and underexposed-region-aware enhancement.
5 Experimental Evaluation
Experiments across synthetic and real low-light datasets show that the method delivers strong enhancement quality, efficiency, perceptual preference, robustness, and downstream benefits, while ablations clarify the contribution of its components.
- Synthetic datasets: The method significantly outperforms competing approaches across all reported quality metrics on synthetic low-light images.Comparisons include synthetic images without and with additional noise, using established enhancement and denoising baselines.
- Synthetic datasets: The method achieves better visual brightness, contrast, and artifact suppression on synthetic low-light images.Representative comparisons are provided in Figure 8.
- Efficiency: The method performs well in both quality and efficiency, whereas JED and Robust require substantial computation and LLNet processes large images patch by patch.Runtime is evaluated on ten 1920 × 1080 images using a specified CPU, memory, and GPU setup.
- Real datasets: On the LOL dataset, the method performs better than RetinexNet and a lightweight variant in both quality and efficiency comparisons.The model is fine-tuned on LOL, and knowledge transfer is additionally used for the lightweight version.
- Real datasets: On the SID dataset, the model is more lightweight and efficient while achieving comparable enhancement quality and better visual effects.SID addresses extremely low-light raw-image processing, which is related to but not identical with conventional RGB enhancement.
- Real images: On natural low-light images, the method restores more natural color, contrast, and detail than the compared methods, and receives more best ratings from human subjects.The user study rating distribution is shown in Figure 12.
- Robustness and downstream tasks: The method remains effective on unseen low-light images and improves Mask-RCNN performance when used as preprocessing without fine-tuning.The paper also reports effectiveness on denoising, dehazing, and deblurring tasks.
- Ablation study: Ablations show that adding loss components and network components improves enhancement quality, while increasing model size does not always improve performance; ten branches are used by default.These findings are reported in Table 5.
6 Conclusion
The paper concludes that its attention-guided, multi-branch approach jointly enhances brightness and removes noise using region-adaptive guidance, supported by a large-scale synthetic benchmark. Extensive experiments show large-margin advantages over state-of-the-art methods, with low-light video enhancement identified as future work.
- Conclusion: The paper combines a low-light simulation pipeline and large-scale benchmark dataset with an attention-guided multi-branch network.The network uses underexposure and noise maps to guide brightness enhancement and denoising simultaneously.
- Conclusion: Extensive experiments demonstrate that the proposed solution outperforms state-of-the-art methods by a large margin.The conclusion summarizes the reported experimental evidence without specifying a single metric.
- Conclusion: Extending the proposed method to low-light video enhancement is identified as future work.The conclusion explicitly names video enhancement as the future direction.