Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation

Chongyi Li; Chunle Guo; Chen Change Loy

Source-linked AI summary

Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation

Chongyi Li, Chunle Guo, Chen Change Loy

arXiv:2103.00860v1Mar 1, 2021cs.CV

TL;DR

Low-light enhancement often depends on paired or unpaired reference data, while real images vary widely in illumination. Zero-DCE learns differentiable image-specific curves with non-reference losses, and Zero-DCE++ makes the approach lightweight and fast while retaining enhancement performance. The method is reported to perform strongly across diverse lighting conditions and benchmarks, with scope limited to RGB low-light enhancement.

Problem
Existing low-light enhancement methods rely on paired or carefully selected unpaired data, which can be costly and may generalize poorly to varied real-world lighting.
Method
Zero-DCE uses a lightweight DCE-Net to estimate pixel-wise, high-order differentiable curves and trains them with non-reference losses without paired or unpaired data.
Results
Zero-DCE achieves state-of-the-art qualitative and quantitative performance, while Zero-DCE++ retains enhancement performance with substantially lower computational cost.
Takeaways & Limitations
Zero-reference curve estimation provides a practical route to low-light enhancement across diverse lighting conditions and resource-constrained applications.
Takeaways & Limitations
The study focuses on low-light enhancement for RGB images and excludes methods requiring raw inputs or designed for general photo enhancement.

Abstract

from arXiv · show

This paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network. Our method trains a lightweight deep network, DCE-Net, to estimate pixel-wise and high-order curves for dynamic range adjustment of a given image. The curve estimation is specially designed, considering pixel value range, monotonicity, and differentiability. Zero-DCE is appealing in its relaxed assumption on reference images, i.e., it does not require any paired or even unpaired data during training. This is achieved through a set of carefully formulated non-reference loss functions, which implicitly measure the enhancement quality and drive the learning of the network. Despite its simplicity, we show that it generalizes well to diverse lighting conditions. Our method is efficient as image enhancement can be achieved by an intuitive and simple nonlinear curve mapping. We further present an accelerated and light version of Zero-DCE, called Zero-DCE++, that takes advantage of a tiny network with just 10K parameters. Zero-DCE++ has a fast inference speed (1000/11 FPS on a single GPU/CPU for an image of size 1200*900*3) while keeping the enhancement performance of Zero-DCE. Extensive experiments on various benchmarks demonstrate the advantages of our method over state-of-the-art methods qualitatively and quantitatively. Furthermore, the potential benefits of our method to face detection in the dark are discussed. The source code will be made publicly available at https://li-chongyi.github.io/Proj_Zero-DCE++.html.

1 INTRODUCTION

The paper introduces Zero-DCE, a lightweight zero-reference approach that enhances low-light images through differentiable, image-specific curve estimation. It also presents Zero-DCE++, which reduces computational cost while retaining enhancement performance.

Method: Zero-DCE formulates low-light enhancement as image-specific curve estimation rather than image-to-image mapping.A network predicts high-order curves that adjust the input image’s dynamic range.
Method: Differentiable curves preserve pixel-value range and neighboring-pixel contrast while enabling parameter learning through a convolutional network.The curve can be iteratively applied to approximate higher-order adjustments.
Training: Zero-DCE requires neither paired nor unpaired reference data, using non-reference losses to evaluate enhancement quality indirectly.The losses include spatial consistency, exposure control, color constancy, and illumination smoothness objectives.
Results: Zero-DCE and Zero-DCE++ brighten nonuniformly illuminated images while preserving color and details, whereas compared methods can under- or over-enhance regions.The paper reports state-of-the-art qualitative and quantitative performance and potential benefits for face detection in darkness.
Efficiency: Zero-DCE++ reduces Zero-DCE’s 79K parameters and 84.99G FLOPs to 10K and 0.115G, respectively, for 1200×900×3 inputs.On a single NVIDIA 2080Ti GPU, runtime increases from 500 FPS to 1000 FPS without compromising enhancement performance.

2 RELATED WORK

Prior low-light enhancement methods rely on histogram or physical models, paired data, or carefully selected unpaired data, creating generalization and artifact concerns. Zero-DCE instead combines zero-reference learning with image-specific curve mapping and emphasizes efficient deployment.

Conventional Methods: Conventional methods enhance illumination through histogram expansion or Retinex-based decomposition into reflectance and illumination.These approaches use global or local histogram adjustment, or formulate enhancement as illumination estimation.
Data-driven Methods: Paired-data methods face costly data collection and potentially unrealistic training data, with reported artifacts and color casts on varied real-world lighting.The related-work discussion links these constraints to poor generalization of CNN-based methods.
Data-driven Methods: Unpaired GAN-based enhancement removes paired-data requirements but usually depends on careful selection of unpaired training data.EnlightenGAN is cited as an example of this approach.
Zero-DCE: The paper combines zero-reference learning, non-reference losses, and efficient network design as alternatives to existing data-driven strategies.The method evaluates output quality implicitly through non-reference losses and emphasizes computational efficiency.

3 METHODOLOGY

Zero-DCE enhances low-light images by estimating image-specific, pixel-wise higher-order curves with a lightweight network and applying them iteratively to RGB channels. Non-reference losses guide training while preserving spatial structure, exposure, color, and illumination smoothness.

Light-Enhancement Curve: DCE-Net estimates pixel-wise curve parameter maps that iteratively map each input RGB channel to an enhanced image.The network produces parameter maps for higher-order curves, enabling direct pixel-wise curve mapping.
Light-Enhancement Curve: The quadratic LE-curve constrains enhanced pixels to [0,1], preserves monotonicity, and remains differentiable for gradient-based learning.Its parameter α lies in [−1,1], controls adjustment magnitude and exposure, and is applied separately to RGB channels.
Light-Enhancement Curve: Iterating the curve increases curvature, while pixel-wise parameter maps adapt dynamic-range adjustment to local regions instead of applying one global adjustment.The paper sets the iteration count n to 8 and assumes locally similar pixels share adjustment behavior.
DCE-Net: DCE-Net uses seven convolutional layers with skip concatenation and outputs 24 maps for eight iterations, totaling 79K trainable parameters.The architecture avoids down-sampling and batch normalization to preserve neighboring-pixel relations.
Non-Reference Loss Functions: Four differentiable non-reference losses train the network by enforcing spatial consistency, exposure control, color constancy, and illumination smoothness.The total loss balances the component losses, while illumination smoothness regularizes each curve parameter map.

4 ZERO-DCE++

Zero-DCE++ accelerates Zero-DCE by reducing network complexity, reusing curve maps across iterations, and estimating them from downsampled inputs. These changes target real-time enhancement while retaining high-order curve adjustment and good enhancement performance.

Motivation: Zero-DCE++ is proposed because practical enhancement of large mobile-device images still requires lower computational cost and faster inference.The redesign balances enhancement performance against computational resources.
Network redesign: Zero-DCE++ replaces standard convolutions with depthwise separable convolutions to reduce network parameters.Each layer uses 3×3 depthwise and 1×1 pointwise convolutions.
Curve estimation: Zero-DCE++ reuses three curve parameter maps across eight iteration stages instead of estimating 24 maps.The iteration process preserves the high-order property despite parameter-map reuse.
Input resolution: A 12× downsampled input estimates curve parameters, which are resized to the original resolution for enhancement.The mapping itself is conducted at the original image resolution under a local-region similarity assumption.
Efficiency: 10K trainable parameters and 0.115G FLOPs yield 1000/11 FPS on a single GPU/CPU for 1200×900×3 images.The same modifications also enable fast training reported as 20 minutes.

5 EXPERIMENTS

Experiments examine color-channel adjustment, curve iteration, training data, input size, architecture, perceptual quality, image quality, runtime, and face detection. Across these evaluations, Zero-DCE generally achieves strong enhancement quality, while Zero-DCE++ substantially improves efficiency with some challenging-case trade-offs.

Color adjustment: Three-channel RGB adjustment preserves inherent color and reduces over-saturation more effectively than single-channel CIE Lab or YCbCr adjustment.All variants improve brightness and contrast, but CIE Lab and YCbCr show color deviations and over-saturation.
Curve and model configuration: Higher-order curves and multiple iterations improve adjustment capability, while one iteration produces the worst quantitative performance.Increasing iterations from 8 to 16 only marginally improves average PSNR, motivating the chosen eight-iteration model.
Training data: Removing over-exposed training data causes over-enhancement of well-lit regions, while more multi-exposure data improves recovery of dark regions.The selected training data provide a balance between over-enhancement and under-enhancement.
Input size: 12× input downsampling has an unnoticeable effect on enhancement performance while significantly reducing computational cost.The 12× setting achieves the highest average PSNR among the tested input sizes and becomes Zero-DCE++’s default.
Architecture comparison: Zero-DCE++ combines fewer parameters and FLOPs with a negligible PSNR decrease, but Zero-DCE handles some challenging lighting cases better.The authors present the two models as options chosen according to quality and efficiency requirements.
Visual quality: Zero-DCE and Zero-DCE++ produce clear details, proper contrast, vivid color, and less noise across visual comparisons.Compared with baselines, the methods better recover challenging back-lit faces and avoid blocking artifacts in tested low-light images.
Perceptual evaluation: Zero-DCE achieves the highest average user-study score and superior average perceptual quality, while Zero-DCE++ remains competitive.The perceptual-quality comparison reports Zero-DCE as best on the LIME, MEF, and DICM sets.
Full-reference evaluation: Zero-DCE achieves the best PSNR, SSIM, and MAE values under all reported Part2 testing cases.These metrics evaluate similarity to ground truth in structural and pixel-level image content.

6 CONCLUSION

The paper presents Zero-DCE as a zero-reference, curve-estimation approach for low-light enhancement and extends it with a lightweight, fast variant. Experiments support strong enhancement performance and efficiency relative to existing methods.

Conclusion: Zero-DCE performs low-light enhancement through image-specific curve estimation trained end-to-end with differentiable non-reference losses.The method does not require reference images during training.
Conclusion: Redesigned network structure, curve estimation, and input-size control make Zero-DCE lightweight and fast for practical applications.The conclusion reports superiority in both enhancement performance and efficiency against existing light-enhancement methods.