PaperlayerView exact revision on arXiv

Source-linked AI summary

DARD: Zero-Shot Degradation-Aware Retinex-Guided Diffusion for Low-Light Image Enhancement

Wenjie Cai, Yuezhe Yang, Jianyang Xia, Xingbo Dong, Zhe Jin

arXiv:2608.29243v1cs.CV

TL;DR

Zero-shot LLIE needs strong generative restoration without paired supervision or reliable scene constraints. DARD calibrates image-specific Retinex priors and regulates them throughout reverse diffusion, achieving strong distortion and perceptual performance across benchmarks while improving downstream semantic segmentation.

  • Problem

    Zero-shot LLIE seeks restoration without costly paired low-light and normal-light supervision, while diffusion methods can lack reliable scene constraints for structurally consistent enhancement.

  • Method

    DARD uses test-time degradation-aware Retinex decomposition, timestep-adaptive frequency fusion, and guided reverse refinement with physical and CLIP semantic guidance.

  • Results

    DARD achieves the best PSNR and SSIM among unsupervised and zero-shot methods across four paired test splits, including 22.18 dB PSNR, 0.877 SSIM, and 0.135 LPIPS on LOLv2-real.

  • Takeaways & Limitations

    DARD provides a zero-shot framework with strong restoration quality and a favorable computational balance, completing enhancement in 4.46 s per image using 3.27 GB peak GPU memory.

  • Takeaways & Limitations

    In extremely dark scenes, severe information loss can make Retinex decomposition and CLIP guidance less reliable, causing residual color bias or over-smoothed structures.

Abstract

from arXiv · show

Existing diffusion-based enhancement methods provide strong generative capability for low-light image enhancement (LLIE), yet they either rely on paired supervision or lack reliable scene constraints in zero-shot settings, often leading to structural inconsistency and color drift. Motivated by conventional Retinex models, which offer physically interpretable priors that can serve as reliable scene constraints yet struggle with mixed degradations in real-world scenarios, we propose DARD, a zero-shot Degradation-Aware Retinex-guided Diffusion framework for LLIE. DARD first extracts image-specific physical priors from the degraded input through a test-time degradation-aware Retinex decomposition, thereby providing reliable structural guidance for zero-shot restoration. It then injects these priors into reverse diffusion through a timestep-adaptive frequency fusion strategy to balance structural anchoring and detail generation. Finally, a guided reverse refinement process with physical consistency and Contrastive Language-Image Pre-training (CLIP)-based semantic guidance is introduced to suppress structural artifacts and semantic drift during sampling. Extensive experiments show that DARD achieves strong distortion and perceptual performance and consistently outperforms existing zero-shot baselines across multiple real-world low-light benchmarks. To further validate the practical utility of our method for downstream applications, we evaluated its impact on semantic segmentation. Experiments demonstrate that images enhanced by DARD achieve a 28.10% relative improvement in mIoU over AGLLDiff.

I. Introduction

DARD addresses zero-shot LLIE by calibrating physically interpretable Retinex priors to image-specific degradation and regulating their influence during diffusion sampling. Its design combines reliability-aware physical guidance, adaptive frequency fusion, and semantic refinement for real-world low-light restoration.

  • Real-world LLIE involves spatially varying noise, color distortion, and non-uniform illumination, while paired supervision is costly and impractical to collect.
  • Diffusion models recover realistic details in severely degraded dark regions but can produce structurally inconsistent results without reliable scene constraints.
  • DARD calibrates pretrained Retinex priors using test-time residual modeling and spatial confidence estimation to identify where physical structure is trustworthy.
  • Timestep-adaptive frequency fusion adjusts physical guidance across frequency components, spatial regions, and sampling stages according to prior reliability and diffusion noise.
  • Guided reverse diffusion refinement combines physical consistency with CLIP-based semantic guidance to suppress structural artifacts and semantic drift during sampling.

A. Regression-based Supervised Methods for LLIE

Prior LLIE research spans supervised regression, physics-based zero-reference methods, and diffusion restoration, but DARD instead calibrates pretrained physical information and dynamically regulates it throughout reverse diffusion. This positions DARD as a reliability-aware zero-shot framework rather than a simple combination of Retinex, frequency, or semantic guidance.

  • A. Regression-based Supervised Methods for LLIE: Supervised LLIE methods learn deterministic enhancement from paired low-light and normal-light images using architectures such as multi-branch, Retinex, Transformer, and state-space models.
  • A. Regression-based Supervised Methods for LLIE: Physics-based zero-reference methods reduce paired-data dependence through illumination estimation, curve modeling, noise suppression, and image-specific local correction.
  • A. Regression-based Supervised Methods for LLIE: Diffusion models provide generative priors for recovering realistic details from severely degraded observations, including supervised and zero-shot restoration variants.
  • A. Regression-based Supervised Methods for LLIE: DARD differs from FourierDiff and AGLLDiff by modeling residual degradation and calibrating pretrained Retinex-prior reliability instead of directly imposing frequency constraints or bypassing degradation estimation.
  • A. Regression-based Supervised Methods for LLIE: DARD regulates the calibrated prior through timestep-adaptive fusion and guided refinement, intervening in the reverse sampling trajectory of a pretrained diffusion model.

B. Test-Time Degradation-Aware Retinex Decomposer

DARD treats the pretrained Retinex decomposition as an initial physical prior whose reliability must be calibrated for each degraded test image. Test-time degradation estimation and confidence modeling produce a spatially calibrated prior that regulates later diffusion guidance.

  • DARD retains the pretrained Retinex decomposition as an initial physical prior rather than redesigning it during inference.
  • The residual signal captures observation components unexplained by Retinex, including sensor noise, color corruption, severe underexposure, and mixed degradations.
  • A test-time estimator Φdeg uses the residual signal and illumination to predict an image-specific degradation map while keeping the pretrained Retinex network fixed.
  • The method estimates spatial confidence for the physical prior, then attenuates confidence according to degradation magnitude so unreliable regions exert weaker constraints.
  • The resulting prior set contains reflectance, illumination, residual degradation, and calibrated confidence, regulating when, where, and how strongly physical information constrains diffusion.

C. Timestep-Adaptive Frequency Fusion

Timestep-Adaptive Frequency Fusion combines frequency-wise prior reliability with the reverse process's evolving uncertainty. It suppresses guidance in degraded components and regions while strengthening reliable structure at noisier stages and relaxing it as sampling stabilizes.

  • TAFF transforms Retinex reflectance, diffusion predictions, and degradation information into the Fourier domain to separate structural components by degradation-dependent reliability.
  • A Wiener-style reliability ratio estimates bounded frequency-wise physical reliability from physical-structure power relative to degradation power.
  • The fusion weight also uses reverse-process noise as a proxy for diffusion uncertainty, rather than relying on a manually assigned timestep schedule.
  • When the reverse state is highly noisy, reliable physical structures receive stronger influence; as noise approaches zero, that influence decreases to permit finer detail recovery.
  • A spatial confidence gate and frequency-wise reliability jointly suppress physical guidance in degraded regions and components, while adapting its strength across sampling stages.

D. Guided Reverse Diffusion Refinement

Guided Reverse Diffusion Refinement corrects photometric, structural, content, and semantic drift during sampling by combining physical consistency, a Retinex-derived pseudo target, and CLIP guidance.

  • Guided Reverse Diffusion Refinement: GRDR complements timestep-adaptive physical-prior fusion by correcting photometric, structural, content, and semantic deviations in the reverse trajectory.It is introduced because fused predictions may still drift despite degradation-aware prior injection.
  • Guided Reverse Diffusion Refinement: The refinement objective combines physical, pseudo-target, structural, and content losses with separate weighting coefficients.These terms constrain photometric plausibility, illumination, geometric structure, and content consistency in reliable regions.
  • Guided Reverse Diffusion Refinement: DARD constructs the pseudo target y_proc = R⊙L^γ using Retinex reflectance and an adaptive illumination exponent estimated from the illumination map.The resulting target supplies an observation-dependent illumination anchor for scene consistency.
  • Guided Reverse Diffusion Refinement: The pseudo, structural, and content weights increase as reverse noise decreases, making scene-consistency constraints stronger during later denoising.The schedule follows λ_pseudo, λ_struct, λ_content ∝ (1 − σ_t).
  • Guided Reverse Diffusion Refinement: Adaptive gradient normalization controls correction magnitude and applies stronger corrections at early noisy steps through a σ_t^2 factor.The base guidance scale is η_0 and ε is a small constant.
  • Guided Reverse Diffusion Refinement: Intermediate sampling additionally applies CLIP semantic guidance with positive and negative prompts before iteratively updating the reverse trajectory to obtain x_0.The semantic guidance scale is η_clip, and correction-update steps repeat from t = T to t = 0.

A. Experimental Settings

The evaluation uses paired real-world low-light benchmarks with reference images, covering LOLv1, LOLv2-real, and LSRW datasets and their stated training and testing splits.

  • Experimental Settings: Experiments use LOLv1, LOLv2-real, and LSRW as paired real-world low-light benchmarks for quantitative evaluation with ground-truth references.LOLv1 has 485 training and 15 testing pairs; LOLv2-real has 689 training and 100 testing pairs.
  • Experimental Settings: LSRW contains Huawei and Nikon subsets, each organized into training and testing pairs for benchmark evaluation.The supplied passage introduces the two subsets but truncates the Nikon split details.

1) Datasets and Evaluation Metrics:

The study evaluates zero-shot inference on paired and unpaired real-world benchmarks using fidelity, structural, perceptual, and illumination-naturalness metrics.

  • Datasets and Evaluation Metrics: DARD is evaluated on no-reference datasets DICM, MEF, LIME, VV, and NPE to assess perceptual robustness under unconstrained real-world conditions.These datasets contain diverse low-light scenes with severe underexposure.
  • Datasets and Evaluation Metrics: Paired-benchmark evaluation uses PSNR, SSIM, and LPIPS to measure pixel fidelity, structural consistency, and perceptual similarity.The paired datasets are LOLv1, LOLv2-real, and LSRW.
  • Datasets and Evaluation Metrics: No-reference evaluation uses Q-Align, MUSIQ, and LOE to assess perceptual quality and illumination naturalness.Q-Align and MUSIQ measure overall perceptual quality, while LOE measures illumination naturalness.
  • Implementation Details: DARD performs zero-shot inference without task-specific training or fine-tuning on paired benchmark images.The implementation uses frozen pretrained components, with image-specific optimization limited to a lightweight degradation estimator.

B. Comparison and Evaluation

DARD is compared with supervised, unsupervised, and diffusion-based baselines, showing stronger visual fidelity and benchmark performance while combining degradation-aware physical guidance with refinement.

  • Comparison and Evaluation: DARD is compared against supervised and unsupervised state-of-the-art methods using official codes and recommended parameters where available.The comparison includes methods such as KinD, URetinexNet, RUAS, SCI, PairLiE, AGLLDiff, FourierDiff, and RRDNet.
  • Qualitative Evaluation: DARD preserves balanced exposure, fine structures, and coherent colors while suppressing noise amplification and blotchy artifacts in qualitative comparisons.The reported visual advantages appear in challenging bright-dark mixtures, boundaries, thin lines, and repetitive textures.
  • Qualitative Evaluation: No-reference visual comparisons expose characteristic artifacts from competing methods, including orange halos, purple contamination, and grayish veils.These examples are reported on MEF, LIME, and VV, respectively.
  • Quantitative Evaluation: DARD achieves the best PSNR and SSIM among unsupervised and zero-shot methods across all four paired test splits.The result is reported for LOLv1, LOLv2-real, and the two LSRW subsets.

2) Quantitative Evaluation:

DARD shows strong restoration quality across benchmark, ablation, and efficiency evaluations, with timestep-adaptive frequency fusion producing the largest component-level improvement.

  • Benchmark results: On LOLv2-real, DARD reaches 22.18 dB PSNR, 0.877 SSIM, and 0.135 LPIPS, outperforming recent zero-shot diffusion baselines.It also attains the lowest LPIPS among all compared methods on this dataset.
  • No-reference evaluation: DARD ranks first among unsupervised methods in Q-Align and MUSIQ across five no-reference datasets and achieves the lowest LOE on four.These results support consistent perceptual quality and illumination naturalness under diverse real-world conditions.
  • Computational efficiency: DARD completes enhancement in 4.46 s per image with 3.27 GB peak GPU memory, substantially faster than LLIEDiff’s 65.05 s runtime.Its memory consumption remains comparable to AGLLDiff and FourierDiff.
  • Ablation studies: Removing TAFF reduces PSNR from 21.14 dB to 19.13 dB and increases LPIPS from 0.143 to 0.219 on LOLv1.This is the largest reported ablation drop and indicates TAFF is important for balancing physical priors with generated details.
  • Ablation studies: Removing TTO or GRDR lowers restoration quality, with TTO reaching 20.31 dB PSNR and GRDR reaching 20.29 dB PSNR on LOLv1.Both modules therefore contribute to the full model’s restoration quality.

2) Impact of guidance weights:

Guidance performance depends on balanced constraint weights, complementary physical and CLIP guidance, and a suitable diffusion-step and prompt configuration; DARD remains limited in severely degraded settings and efficiency.

  • Impact of guidance weights: Balanced GRDR weights outperform uniformly weakened or uniformly strengthened guidance, showing that relative constraint contributions matter more than simply increasing guidance strength.The relevant constraints are physical, pseudo-target, structural, and content guidance.
  • Guidance complementarity: Physics guidance improves PSNR by 0.17 dB, SSIM by 0.012, and LPIPS by 0.020, while CLIP guidance improves PSNR by 0.22 dB, SSIM by 0.003, and LPIPS by 0.012.Physics guidance yields larger SSIM and LPIPS gains, whereas CLIP guidance yields a slightly larger PSNR gain.
  • Diffusion-step selection: Using 100 diffusion steps provides the best overall trade-off between restoration quality and inference efficiency.Increasing steps from 10 to 100 steadily improves all three reported metrics, while 200 steps slightly improves LPIPS but decreases PSNR and SSIM.
  • CLIP prompt selection: CLIP prompt Setting C achieves 21.14 dB PSNR, 0.835 SSIM, and 0.143 LPIPS, making it the default configuration.Setting C describes brightness, clarity, natural illumination, and realistic colors.
  • Limitations: DARD’s initial Retinex decomposition becomes less reliable under extreme darkness, while complex illumination, saturation, and unstable semantic cues can weaken later guidance.Per-image optimization and iterative reverse diffusion also make inference more costly than feed-forward enhancement.
Loading 2608.29243v1…