PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Retinex Decomposition for Low-Light Enhancement

Chen Wei, Wenjing Wang, Wenhan Yang, Jiaying Liu

arXiv:1808.04560v1cs.CV

TL;DR

Low-light enhancement is difficult because existing Retinex methods depend on hand-crafted constraints and parameters for an ill-posed decomposition. The paper introduces a data-driven, end-to-end Retinex-Net trained with paired low/normal-light images, and reports visually pleasing enhancement plus good image-decomposition representation.

  • Problem

    Existing Retinex-based decomposition and illumination adjustments rely on hand-crafted constraints and parameter tuning that may have limited capacity across various scenes.

  • Method

    Retinex-Net learns reflectance and illumination decomposition from paired low/normal-light images without decomposed ground truth, then enhances illumination and denoises reflectance end-to-end.

  • Results

    The method produces visually pleasing low-light enhancement results and a good representation of image decomposition.

  • Takeaways & Limitations

    The learned decomposition is by nature suitable for lightness adjustment while supporting subsequent illumination enhancement and reflectance denoising.

  • Takeaways & Limitations

    Direct total-variation smoothing is structure-blind: it can blur illumination and leave strong black edges on reflectance near structures or drastic lightness changes.

Abstract

from arXiv · show

Retinex model is an effective tool for low-light image enhancement. It assumes that observed images can be decomposed into the reflectance and illumination. Most existing Retinex-based methods have carefully designed hand-crafted constraints and parameters for this highly ill-posed decomposition, which may be limited by model capacity when applied in various scenes. In this paper, we collect a LOw-Light dataset (LOL) containing low/normal-light image pairs and propose a deep Retinex-Net learned on this dataset, including a Decom-Net for decomposition and an Enhance-Net for illumination adjustment. In the training process for Decom-Net, there is no ground truth of decomposed reflectance and illumination. The network is learned with only key constraints including the consistent reflectance shared by paired low/normal-light images, and the smoothness of illumination. Based on the decomposition, subsequent lightness enhancement is conducted on illumination by an enhancement network called Enhance-Net, and for joint denoising there is a denoising operation on reflectance. The Retinex-Net is end-to-end trainable, so that the learned decomposition is by nature good for lightness adjustment. Extensive experiments demonstrate that our method not only achieves visually pleasing quality for low-light enhancement but also provides a good representation of image decomposition.

1 Introduction

Low-light enhancement is needed because poor lighting reduces visibility, detail, contrast, and the usability of vision systems. Retinex-Net addresses limitations of hand-crafted decomposition and adjustment by learning a data-driven decomposition and enhancement framework.

  • Motivation: Insufficient lighting reduces image visibility, detail, and contrast, while impairing computer vision systems designed for normal-light images.Low-light conditions can result from the environment, camera limitations, or inappropriate equipment settings.
  • Limitations of Existing Methods: Existing Retinex-based methods rely on hand-crafted decomposition constraints and illumination manipulations whose capacity and parameter tuning may not generalize across scenes.Retinex methods decompose images into reflectance and illumination, but designing adaptive constraints for this ill-posed problem is difficult.
  • Proposed Framework: Retinex-Net learns image decomposition with a deep network and integrates decomposition with successive enhancement operations.The framework is designed as a data-driven alternative to manually specified decomposition and illumination processing.
  • Dataset: The authors build a large-scale dataset of paired low/normal-light images captured in real scenes for low-light enhancement research.They describe it as the first such attempt in the field to their knowledge.
  • Proposed Framework: Decom-Net separates input images into reflectance and illumination, while Enhance-Net brightens illumination and removes noise from reflectance before reconstruction.The enhancement framework performs decomposition, adjustment, and reconstruction, with multi-scale concatenation for illumination adjustment.
  • Proposed Framework: End-to-end training makes the learned decomposition suitable for light-condition adjustment and supports structure-aware smoothing of illumination while retaining main structures.The structure-aware total variation constraint reduces smoothing at strong gradients.

2 Retinex-Net for Low-Light Enhancement

Retinex-Net learns decomposition and enhancement jointly from paired low/normal-light images, using shared reflectance and structure-aware illumination smoothness constraints. It then adjusts illumination with an encoder-decoder Enhance-Net while reconstructing normal-light output.

  • Data-Driven Image Decomposition: Retinex theory decomposes an observed image into reflectance, representing object properties, and illumination, representing lighting variation.The model is expressed as S = R◦I, with reflectance assumed consistent across lighting conditions.
  • Retinex-Net: Retinex-Net jointly performs decomposition, illumination adjustment, and reconstruction using a Decom-Net followed by an Enhance-Net.During training, Decom-Net receives paired low/normal-light images; at testing, it can process a low-light image individually.
  • Data-Driven Image Decomposition: Decom-Net learns without decomposed-image ground truth by enforcing reconstruction, reflectance consistency, and illumination smoothness losses.The reflectance consistency term constrains Rlow and Rnormal, while the loss weights balance consistency and smoothness.
  • Structure-Aware Smoothness Loss: Structure-aware smoothness weights total variation by reflectance gradients, relaxing illumination smoothing near strong structures while preserving boundaries.Uniform TV can blur illumination and leave strong black edges in reflectance; the reflectance-weighted formulation addresses this structure-blindness.
  • Multi-Scale Illumination Adjustment: Enhance-Net uses an encoder-decoder with multiscale concatenation to brighten illumination while maintaining global consistency and adapting local distributions.Downsampling captures large-region illumination context, while upsampling reconstructs local illumination distributions with skip connections.
  • Multi-Scale Illumination Adjustment: Enhance-Net is trained with reconstruction and illumination-smoothness losses to produce a normal-light reconstruction from the adjusted illumination.Its smoothness loss uses the enhanced illumination weighted by the low-light reflectance gradient map.

3 Dataset

The LOL dataset addresses the shortage of paired real-scene low/normal-light images by combining captured pairs with synthetic training data. Its construction spans diverse scenes, alignment correction, and histogram-matched raw-image synthesis.

  • 3.1 Dataset Captured in Real Scenes: LOL contains 500 low/normal-light image pairs captured in real scenes and is described as the first such dataset for low-light enhancement.
  • 3.1 Dataset Captured in Real Scenes: Images cover varied scenes including houses, campuses, clubs, and streets, with exposure time and ISO changed while other camera configurations remain fixed.
  • 3.1 Dataset Captured in Real Scenes: A three-step procedure removes misalignments caused by camera shaking, object movement, and changing lightness before resizing and PNG conversion.
  • 3.2 Synthetic Image Pairs from Raw Images: Synthetic low-light images are generated from raw RAISE images after analyzing low-light Y-channel histograms in YCbCr.
  • 3.2 Synthetic Image Pairs from Raw Images: Figure 4 plots Y-channel fitting curves with pixel values on the horizontal axis from 16 to 240 and logarithmically scaled histogram values vertically.

4 Experiments

Experiments evaluate Retinex-Net on real-scene datasets and compare its decomposition, enhancement, and joint denoising against established methods. The reported visual results show consistent reflectance, effective brightening without overexposure, and better edge preservation.

  • 4 Experiments: The LOL dataset uses 485 pairs for training and 15 for evaluation, while training also includes 1000 synthetic pairs.
  • 4.3 Evaluation: Retinex-Net brightens dark objects without overexposure, avoids dark edges, and outperforms compared methods visually on three natural images.
  • 4.2 Decomposition Results: Decom-Net extracts reflectance consistent across paired lighting conditions, while illumination maps represent image lightness and shadows.
  • 4.2 Decomposition Results: Compared with LIME, Retinex-Net leaves less illumination information on reflectance in the bookshelf example.
  • 4 Experiments: Retinex-Net is evaluated on LIME, MEF, and DICM images against DeHz, NPE, SRIE, and LIME.
  • 4.4 Joint Low-Light Enhancement and Denoising: In joint denoising, Retinex-Net preserves details better than LIME with denoising post-processing and JED, which blur edges.

5 Conclusion

The method learns Retinex decomposition without decomposed ground truth, then enhances illumination and denoises reflectance end-to-end. It produces visually pleasing low-light enhancement and a useful image-decomposition representation.

  • The method learns to decompose images into reflectance and illumination without ground-truth decompositions.
  • Illumination enhancement and reflectance denoising are integrated into the end-to-end trainable pipeline.
  • The method produces visually pleasing enhancement results and a good representation of image decomposition.
Loading 1808.04560v1…