PaperlayerView exact revision on arXiv

Source-linked AI summary

R2RNet: Low-light Image Enhancement via Real-low to Real-normal Network

Jiang Hai, Zhu Xuan, Songchen Han, Ren Yang, Yutong Hao, Fengzhu Zou, Fang Lin

arXiv:2106.14501v2cs.CVeess.IV

TL;DR

Low-light images suffer from degradations that affect visual quality and high-level vision, while real-world paired training data are limited. R2RNet combines Retinex decomposition, denoising, spatial relighting, and frequency-based detail preservation, trained with the LSRW dataset. Experiments report stronger visual and quantitative enhancement and improved face detection under low-light conditions.

  • Problem

    Low-light images have degraded quality, and most learning-based methods rely on synthetic data because real-world paired training images are scarce.

  • Method

    R2RNet uses Retinex-based Decom-Net, Denoise-Net, and Relight-Net subnets, combining spatial information for contrast with frequency information for detail preservation.

  • Results

    R2RNet improves contrast, suppresses noise, preserves details, achieves the highest reported PSNR and SSIM scores, and improves face detection under low-light conditions.

  • Takeaways & Limitations

    The method and LSRW dataset support real-world low-light enhancement while improving the reported performance of face detection in insufficient illumination.

Abstract

from arXiv · show

Images captured in weak illumination conditions could seriously degrade the image quality. Solving a series of degradation of low-light images can effectively improve the visual quality of images and the performance of high-level visual tasks. In this study, a novel Retinex-based Real-low to Real-normal Network (R2RNet) is proposed for low-light image enhancement, which includes three subnets: a Decom-Net, a Denoise-Net, and a Relight-Net. These three subnets are used for decomposing, denoising, contrast enhancement and detail preservation, respectively. Our R2RNet not only uses the spatial information of the image to improve the contrast but also uses the frequency information to preserve the details. Therefore, our model acheived more robust results for all degraded images. Unlike most previous methods that were trained on synthetic images, we collected the first Large-Scale Real-World paired low/normal-light images dataset (LSRW dataset) to satisfy the training requirements and make our model have better generalization performance in real-world scenes. Extensive experiments on publicly available datasets demonstrated that our method outperforms the existing state-of-the-art methods both quantitatively and visually. In addition, our results showed that the performance of the high-level visual task (i.e. face detection) can be effectively improved by using the enhanced results obtained by our method in low-light conditions. Our codes and the LSRW dataset are available at: https://github.com/abcdef2000/R2RNet.

I. INTRODUCTION

Low-light enhancement must address contrast, visibility, noise, and detail degradation while supporting real-world training and downstream vision tasks. R2RNet combines Retinex-based decomposition, denoising, spatial contrast enhancement, and frequency-based detail preservation, supported by a real-world paired dataset.

  • Insufficient illumination degrades contrast, visibility, and image quality, motivating enhancement for recognition, detection, segmentation, and practical intelligent systems.
  • Existing approaches often improve contrast, but separate denoising before or after enhancement can cause blurring or noise amplification.
  • R2RNet uses Retinex-based Decom-Net, Denoise-Net, and Relight-Net subnets for decomposition, noise suppression, contrast improvement, and detail preservation.
  • The network uses spatial information for contrast enhancement and Fourier-based frequency information in Relight-Net to preserve image details without suppressing them during decomposition or denoising.
  • Most learning-based methods rely on synthetic training images because sufficient real-world paired data are difficult to collect, limiting real-scene generalization.

A. Low-light Image Enhancement methods

Low-light enhancement methods span histogram, Retinex, denoising, and learning-based approaches, but real paired data remain limited and separate denoising can damage enhancement quality. R2RNet addresses these issues by combining spatial and frequency information and introducing the LSRW dataset.

  • Traditional enhancement methods include histogram equalization and Retinex-based illumination-reflectance modeling, while later methods add denoising or learned decomposition.
  • Learning-based methods can improve decomposition and contrast, but most use only spatial information; R2RNet combines spatial contrast enhancement with frequency-based detail restoration.
  • Directly applying denoising before or after enhancement can produce blurring or noise amplification, motivating simultaneous contrast enhancement and denoising.
  • Most training datasets use synthetic low-light images because real paired images are scarce, creating a challenge for real-world enhancement models.
  • The LSRW dataset contains 5650 low/normal-light image pairs captured in real scenes using a Nikon D7500 camera and a HUAWEI P40 Pro mobile phone.

III. LSRW DATASET

The paper introduces the LSRW dataset and the R2RNet architecture for enhancing real low-light images. R2RNet decomposes, denoises, relights, and reconstructs details using paired data and spatial-frequency processing.

  • LSRW Dataset: The LSRW dataset contains 5,650 paired real-world low/normal-light images captured with Nikon D7500 and HUAWEI P40 Pro cameras.The collection includes 3,170 Nikon pairs and 2,480 Huawei pairs.
  • LSRW Dataset: LSRW images were collected across indoor and outdoor scenes using different exposure settings to create low- and normal-light pairs.Low-light images generally use reduced ISO and shorter exposure, while normal-light images use larger ISO and longer exposure, with adjustments to avoid extreme darkness or overexposure.
  • Network Architecture: Decom-Net separates each low-light input into illumination and reflectance maps, while Denoise-Net suppresses noise in the reflectance map using illumination as a constraint.During training, paired low- and normal-light images provide reflectance consistency and illumination smoothness guidance without requiring ground-truth decomposition maps.
  • Network Architecture: Relight-Net improves illumination contrast and reconstructs details through spatial Contrast Enhancement and frequency-based Detail Reconstruction modules.The CEM uses spatial information, whereas the DRM uses Fourier-derived frequency information to preserve image details.
  • Network Architecture: The Detail Reconstruction Module alternates spatial and frequency processing with SFSC blocks and uses an FIP block to enhance high-frequency edge contours.SFSC blocks apply spatial processing, fast Fourier transformation, complex frequency processing, and inverse transformation; FIP combines feature- and image-level frequency signals.
  • Network Architecture: The final enhanced image is produced by element-by-element multiplication of the denoised reflectance map and enhanced illumination map.Relight-Net receives the illumination map from Decom-Net and the denoised reflectance map from Denoise-Net, outputting the enhanced illumination map.

B. Loss Function

R2RNet trains its three subnets separately with dedicated decomposition, denoising, and relighting losses. These losses combine content and perceptual objectives, while relighting additionally uses detail preservation through frequency-domain supervision.

  • Loss design: The three subnets are trained separately using decomposition, denoise, and relight losses.Each loss contains content and perceptual components.
  • Decomposition loss: Decomposition loss combines L1 content loss with VGG-16 perceptual features extracted before activation.The decomposition loss is formulated from these content and perceptual terms.
  • Denoise loss: Denoise loss combines L1 content loss with perceptual loss computed from pre-activation VGG-16 features.The cited passages describe the denoising loss components and feature extraction strategy.
  • Relight loss: Relight loss contains content, perceptual, and detail-preservation losses.The content and perceptual terms follow the same strategy used for decomposition and denoising.
  • Subnet outputs: Figure 5 identifies illumination and reflectance outputs from Decom-Net, enhanced illumination from Relight-Net, and denoised reflectance from Denoise-Net.The figure caption maps each decomposition result to its producing subnet.
  • Frequency-domain supervision: Frequency loss converts enhanced and sharp images with fast Fourier transform and minimizes real- and imaginary-part differences using Wasserstein distance.This supervision is introduced because Relight-Net uses frequency information to recover more details.

A. Implementation Details

The implementation uses PyTorch and trains R2RNet on the LSRW dataset for 20 epochs, with Adam optimization, specified patch and batch sizes, and learning-rate decay.

  • Training setup: The model is implemented in PyTorch and converges after 20 training epochs on a 1080Ti GPU using the LSRW dataset.The implementation uses Adam with learning rate 10^-3, β1 = 0.9, and β2 = 0.999.
  • Training setup: Training uses batch size 4 and patch size 96, with learning rate reduced to 10^-4 after 10 epochs.The cited implementation details also specify the optimizer and loss-weight settings.

B. Comparison with State-of-the-Arts on the Real Datasets

R2RNet is compared with state-of-the-art methods across six public datasets. On LOL, it achieves the best reported PSNR and SSIM, while additional evaluations use NIQE and visual comparisons.

  • Datasets and baselines: R2RNet is compared with 12 existing methods across LOL, LIME, DICM, NPE, MEF, and VV datasets.Released implementations are used without modification; supervised methods are trained on LSRW, while unpaired methods use published pretrained models.
  • Quantitative comparison: 20.207dB PSNR and 0.816 SSIM are R2RNet’s best LOL results, exceeding MBLLEN by 1.347dB in PSNR.The cited passage reports these as the best performance on LOL for both metrics.
  • Visual comparison: The LOL visual comparison includes state-of-the-art enhancement methods, with EG denoting EnlightenGAN.The caption directs readers to zoom in for detailed review.
  • Visual comparison: The visual results are described as enhancing contrast while suppressing noise simultaneously.Readers are asked to zoom in to compare details.
  • Non-reference evaluation: For LIME, DICM, NPE, MEF, and VV, which contain only low-light images, performance is evaluated with non-reference NIQE rather than PSNR or SSIM.The passage also points to Table III and Figure 7 for quantitative and visual comparisons.

C. User Study

A user study evaluates enhanced real-world low-light images across five methods using ratings from 10 participants. R2RNet receives the best overall result in the reported score distribution.

  • Study design: The study uses 20 additional real-world low-light images and compares NPE, LIME, EnlightenGAN, MBLLEN, and R2RNet.Ten participants evaluate contrast, artifacts, noise, details, and color.

D. Ablation Study

Ablation experiments show that the proposed modules, architecture, and loss design each contribute to R2RNet's performance on the LOL dataset.

  • Effectiveness of CEM and DRM: Removing either CEM or DRM significantly reduces Relight-Net performance.Combining spatial and frequency information performs better than using either alone.
  • Effectiveness of deep-narrow architecture: 0.971dB higher PSNR and 0.011 higher SSIM are achieved by DN-ResUnet than ResUnet.The comparison is based on 20.207 versus 19.236 PSNR and 0.816 versus 0.805 SSIM.
  • Loss function setting: 0.676dB higher PSNR and 0.012 higher SSIM are achieved with L1 content loss than MSE loss.The corresponding scores are 20.207 versus 19.531 PSNR and 0.816 versus 0.804 SSIM.
  • Loss function setting: Removing perceptual loss decreases PSNR by 0.868dB and SSIM by 0.043.These changes compare with the full setting's 20.207 PSNR and 0.816 SSIM.
  • Loss function setting: Removing frequency loss decreases PSNR by 0.451dB and SSIM by 0.012.The ablation compares 19.756 versus 20.207 PSNR and 0.804 versus 0.816 SSIM.

E. Pre-Processing for Improving Face Detection

R2RNet enhancement is evaluated as preprocessing for face detection on low-light DARK FACE images, improving detection performance for two pretrained detectors.

  • Face detection results: 33.98% AP with DSFD+R2RNet and 25.97% AP with RetinaFace+R2RNet exceed their low-light baselines.The baselines are 17.12% for DSFD and 15.28% for RetinaFace.
  • Evaluation setup: R2RNet is applied before DSFD and RetinaFace on 100 randomly selected DARK FACE training images.The DARK FACE dataset contains 6,100 real-world nighttime low-light images.

VI. CONCLUSION

The paper concludes that R2RNet enhances low-light images using Retinex-based decomposition, denoising, spatial contrast improvement, and frequency-based detail reconstruction. It also introduces the 5,650-pair LSRW dataset and reports strong enhancement and face-detection results, while identifying broader enhancement tasks for future work.

  • VI. CONCLUSION: R2RNet contains Decom-Net, Denoise-Net, and Relight-Net for decomposition, denoising, contrast enhancement, and detail preservation.Decom-Net separates illumination and reflectance; Denoise-Net suppresses reflectance noise; Relight-Net improves contrast and reconstructs details.
  • VI. CONCLUSION: Experiments report large-margin improvements over state-of-the-art methods in contrast, brightness, detail preservation, and noise suppression.The paper also reports the highest PSNR and SSIM scores on publicly available datasets.
  • VI. CONCLUSION: R2RNet preprocessing effectively improves face detection under insufficient illumination.The conclusion links the enhancement method to improved high-level visual task performance.
  • VI. CONCLUSION: The LSRW dataset contains 5650 paired real-world low/normal-light images for network training.The dataset is presented as the first large-scale real-world paired dataset of this type.
  • VI. CONCLUSION: Future work will explore more effective models and apply them to low-light video and extremely low-light image enhancement.These applications are identified as future enhancement tasks.
Loading 2106.14501v2…