Source-linked AI summary
Zero-Reference Low-Light Enhancement via Physical Quadruple Priors
Wenjing Wang, Huan Yang, Jianlong Fu, Jiaying Liu
TL;DR
Low-light enhancement methods can be sensitive to supervision, training-data distributions, and illumination-specific tuning, limiting adaptability to unseen scenarios. This paper introduces a zero-reference framework trained with normal-light images, using a physical quadruple prior and diffusion-based prior-to-image mapping; experiments report strong performance across diverse scenarios, with robustness and efficiency claims.
Problem
Existing low-light enhancement methods depend on supervisory information or are sensitive to training-data distributions and manually tuned illumination parameters, limiting adaptation to unseen scenarios.
Method
The framework derives a physical quadruple illumination-invariant prior and uses it to condition a pretrained diffusion model for prior-to-image restoration trained without low-light data.
Results
The model demonstrates superior performance across diverse scenarios and robustness across LOL and MIT while reducing sensitivity to specific datasets.
Takeaways & Limitations
The framework provides a zero-reference enhancement approach combining interpretability, robustness, and efficiency without relying on specific low-light data.
Takeaways & Limitations
The prior derivation assumes matte surfaces and uniform illumination.
Abstract
from arXiv · showhide
Understanding illumination and reducing the need for supervision pose a significant challenge in low-light enhancement. Current approaches are highly sensitive to data usage during training and illumination-specific hyper-parameters, limiting their ability to handle unseen scenarios. In this paper, we propose a new zero-reference low-light enhancement framework trainable solely with normal light images. To accomplish this, we devise an illumination-invariant prior inspired by the theory of physical light transfer. This prior serves as the bridge between normal and low-light images. Then, we develop a prior-to-image framework trained without low-light data. During testing, this framework is able to restore our illumination-invariant prior back to images, automatically achieving low-light enhancement. Within this framework, we leverage a pretrained generative diffusion model for model ability, introduce a bypass decoder to handle detail distortion, as well as offer a lightweight version for practicality. Extensive experiments demonstrate our framework's superiority in various scenarios as well as good interpretability, robustness, and efficiency. Code is available on our project homepage: http://daooshee.github.io/QuadPrior-Website/
1. Introduction
Low-light enhancement remains difficult because existing methods depend on supervision, training-data distributions, or manually tuned illumination parameters. The paper proposes an illumination-invariant physical quadruple prior and a prior-to-image framework trained only on normal-light images.
- Existing low-light methods are limited by dependence on supervisory information and adaptability to unseen domains.
- Zero-Reference Methods: Zero-reference methods avoid paired or unpaired training data but remain sensitive to training distributions and parameter tuning.SCI can produce over-exposed or under-exposed results when training data varies.
- Our Contributions: The proposed physical quadruple prior provides an illumination-invariant intermediary between low-light and normal-light images.It is derived from the Kubelka–Munk theory of light transfer.
- Our Contributions: The prior-to-image framework learns from typical normal-light images and restores images from the prior without using low-light data.The framework is designed to learn bright-lighting knowledge from readily available normal-light imagery.
- Our Contributions: A pretrained generative model is used to map the prior back to images, while a bypass decoder addresses detail distortion.The mapping integrates the prior as a condition for Stable Diffusion; the framework also includes a lightweight version.
- Our Contributions: The framework is presented as combining interpretability, robustness, and efficiency across diverse under-lit scenarios.The paper reports favorable subjective and objective performance without relying on specific low-light data.
2. Physical Prior-based Image Restoration
The framework uses a physical quadruple prior to represent illumination-invariant information and reconstruct images through a prior-to-image diffusion model. It is trained on normal-light images and applies the learned mapping to low-light inputs, while acknowledging that the prior loses information and reconstruction is non-trivial.
- Motivation: Retinex-style decomposition separates illumination and reflectance, but existing approaches rely on handcrafted policies or paired brightness data and lack robustness in unknown scenarios.The framework instead seeks illumination-invariant features without explicitly decomposing illumination-related information.
- Physical Quadruple Prior: Kubelka-Munk light-transfer theory derives illumination invariants by eliminating illumination and specular terms while retaining material reflectivity, which is independent of illumination.The derivation includes assumptions such as equal-energy, matte, or uniform illumination conditions.
- Physical Quadruple Prior: The physical quadruple prior combines H, C, W, and O, with H associated with hue, C with chroma, and W with intensity-normalized spatial derivatives.The RGB channel-order feature O adds color information because the first three invariants lose part of the color content.
- Prior-to-Image Framework: Because the quadruple prior omits some information, reconstructing images from it is non-trivial and motivates the framework's restoration design.The paper explicitly notes that retaining illumination-invariant information while discarding lighting-related information is difficult.
- Prior-to-Image Framework: A frozen Stable Diffusion U-Net conditions reverse diffusion on the physical quadruple prior to predict z0, which is then decoded back into image space.During training, the model encodes normal-light images into z0, samples noisy latents, and predicts noise using prior features integrated into the frozen U-Net.
3. Experiments
Experiments evaluate the framework across benchmark datasets, prior and framework ablations, decoder choices, and a distilled lightweight model. Results show robustness across datasets, the importance of the complete physical quadruple prior, effective detail preservation, and substantial efficiency gains.
- Benchmarking Results: The model surpasses most unsupervised methods and narrows the gap with supervised methods across low-light benchmarks.It suppresses noise and avoids overexposure or excessive darkness, while remaining robust across LOL and MIT.
- Benchmarking Results: Supervised methods can overfit their training sets, whereas the proposed model adapts better to previously unseen scenarios.Retinexformer and DiffLL trained on LOL perform worse on MIT, and vice versa, while the proposed model remains competitive across both.
- Prior Design: Removing any one of H, C, W, or O reduces performance, showing that the four prior elements jointly preserve the information needed for reconstruction.The ablation evaluates the prior on LOL under the same setting as the benchmark.
- Prior Design: Omitting H or C causes color bias or washed-out appearance, while omitting W removes light-and-shadow changes.Without O, blue is mistakenly enhanced to orange; the full version provides more refined details and contrast.
- Auto-Encoder: The bypass decoder uses illumination-irrelevant input details to produce clear, undistorted text, unlike the original and Consistency decoders.The comparison is performed within the Stable Diffusion-based restoration framework.
- Framework Distillation: The lightweight model runs 500x faster, processes a 1024×1024 image in 0.03 seconds, and reduces parameters from 1.3G to 327.36k.Its LOL/MIT PSNR is 20.45/18.15, with comparable performance to the full model.
4. Conclusion
The paper concludes that its zero-reference framework uses a physical quadruple prior and diffusion-based prior-to-image mapping to enhance low-light images without low-light training data. Experiments report superior performance across diverse scenarios.
- 4. Conclusion: The framework combines a physical quadruple prior, an efficient prior-to-image framework, and a lightweight version for zero-reference enhancement without low-light data.The paper reports interpretability, robustness, and efficiency alongside favorable subjective and objective performance.