PaperlayerView exact revision on arXiv

Source-linked AI summary

ControlLight: Towards Controllable, Consistent, and Generalizable Low-Light Enhancement

Yufeng Yang, Jianzhuang Liu, Jisheng Chu, Yuqi Peng, Xianfang Zeng, Jiancheng Huang, Shifeng Chen

arXiv:2605.25569v2cs.CV

TL;DR

Low-light enhancement methods often lack continuous supervision and controllability, while generative editing can introduce structural inconsistencies. ControlLight combines Light100K, a continuous real-world degradation dataset, with misalignment-aware weighted flow matching and LoRA conditioning; it reports state-of-the-art performance, controllable enhancement, and real-world generalization.

  • Problem

    Existing low-light enhancement methods commonly use limited datasets with single enhancement targets, restricting controllability and generalization in real-world applications.

  • Method

    ControlLight constructs Light100K with continuous illumination-strength supervision and trains a LoRA-conditioned FLUX.2-klein-9B model using misalignment-aware weighted flow matching.

  • Results

    ControlLight achieves state-of-the-art performance against continuous and non-continuous enhancement methods while providing smoothly controllable results and strong real-world generalization.

  • Takeaways & Limitations

    The framework enables flexible enhancement-strength control while preserving visual consistency, scene structure, and realistic output quality.

Abstract

from arXiv · show

Existing deep learning-based low-light enhancement methods are typically trained on limited datasets with single enhancement targets, which restricts their generalization ability and controllability in real-world applications. To overcome these limitations, we propose ControlLight, a controllable, consistent, and generalizable framework for low-light enhancement. We first construct a large-scale dataset of real-world degraded images with continuous illumination-strength supervision. To further ensure consistent outputs under different control strengths, we introduce a misalignment-aware weighted flow matching loss that preserves image structure across continuous enhancement strengths. ControlLight allows users to edit real-world degraded low-light images toward satisfactory enhancement results by flexibly controlling the strength while preserving visual consistency and realism. Extensive experiments show that ControlLight achieves state-of-the-art performance against existing low-light enhancement approaches while demonstrating strong continuous controllability and generalization to real-world scenarios.

1 Introduction

ControlLight addresses limited controllability and structural unreliability in low-light enhancement by combining continuous supervision with misalignment-aware training. Its Light100K dataset and controllable LoRA-conditioned model target smooth enhancement changes while preserving scene structure.

  • Most existing datasets provide one supervision target per low-light image, limiting enhancement to a fixed strength despite users’ need for flexible control.
  • Large image editing models offer plausible details and scene preservation but may produce hallucinated textures or structural distortions under single-strength instruction-based enhancement.
  • Light100K supplies real degraded images with structure-consistent pseudo-enhanced targets spanning different illumination strengths for fine-grained controllable enhancement.
  • Conditioning ControlLight on LoRA strength enables continuous, fine-grained enhancement with smooth illumination changes while preserving scene structure.
  • The Misalignment-Aware Weighted Flow Matching Loss down-weights unreliable target-edge regions and anchors enhanced edges to input structure.
  • ControlLight achieves state-of-the-art performance against continuous and non-continuous low-light enhancement methods while producing smoothly controllable results.

2 Related Work

Prior low-light enhancement methods commonly use fixed supervision, while controllable and continuous editing approaches remain constrained by limited or inconsistent supervision. ControlLight uses Retinex-based continuous pseudo-paired data to address these limitations.

  • Classical-prior and learned low-light enhancement methods generally map low-light inputs to normal-light outputs under paired or unpaired training schemes.
  • Fixed supervision causes conventional methods to produce a single enhancement strength, limiting their suitability when flexible brightness control is required.
  • ReCoRo and CLE Diffusion investigate controllable enhancement, but CLE Diffusion remains limited by model capacity and simple training-data construction.
  • Large image editing models leverage semantic priors but can introduce hallucinations, pixel shifts, and structural deformation undesirable for restoration consistency.
  • Continuous editing methods using embeddings, modulation features, or low-rank adaptors are limited by scarce continuous supervision and can exhibit inconsistency or unstable control.
  • ControlLight constructs continuous pseudo-paired supervision using Retinex theory to support controllable low-light enhancement.

3 Method

The method constructs continuous pseudo-paired supervision through Retinex-based illumination interpolation and trains ControlLight with misalignment-aware flow matching and strength-conditioned LoRA control.

  • 3.1 Light100K: Continuous Pseudo-Paired Data Construction: Approximately 20K high-quality paired samples are retained after filtering real-world low-light images and generated counterparts for structural consistency.The pipeline first collects approximately 30K low-light images, generates enhanced counterparts, and removes severely mismatched pairs.
  • 3.1 Light100K: Continuous Pseudo-Paired Data Construction: At enhancement strengths s ∈ {0.2, 0.4, 0.6, 0.8}, the reconstructed pseudo-ground truths provide a continuous training trajectory.Retinex interpolation preserves local shading, scene depth, and contrast variations better than direct RGB averaging.
  • 3.1 Light100K: Continuous Pseudo-Paired Data Construction: Retinex-inspired interpolation models continuous enhancement as illumination transition rather than direct whole-image RGB interpolation.The method decomposes images into illumination and reflectance, interpolates illumination in the log domain, and conservatively interpolates reflectance.
  • 3.2 Misalignment-Aware Weighted Flow Matching: Misalignment-aware weighted flow matching down-weights unreliable target-edge regions to reduce structural drift from subtle pseudo-pair edge offsets.The method identifies target edges distant from input edges, dilates the mismatch mask, and derives a soft weight map for supervision.
  • 3.2 Misalignment-Aware Weighted Flow Matching: The structural edge-difference map suppresses low-frequency illumination and color discrepancies so it primarily measures local structural misalignment.It operates on high-pass log-luminance features and compares gradient-based edge responses between images.
  • 3.3 ControlLight: ControlLight conditions LoRA scaling and pseudo-ground-truth selection on enhancement strength, enabling inference at any s ∈ [0, 1].Strength values are used during training with per-strength pseudo targets, calibrating the LoRA direction against an illumination trajectory rather than applying control only at inference.

4 Experiments

ControlLight is evaluated across paired, real-world, and continuous-editing benchmarks, with results indicating strong enhancement quality, generalization, and controllability. Ablations further examine its loss and interpolation design choices.

  • Evaluation Setup: ControlLight is evaluated on paired LOL-v1 and LWSR benchmarks, plus real-world DICM, LIME, and RealIR-Bench datasets.The comparisons include specialized low-light enhancement methods and universal continuous image-editing methods.
  • Evaluation Setup: Four enhancement strengths, s ∈{0.25, 0.50, 0.75, 1.00}, are evaluated and averaged for ControlLight.CLE Diffusion is tested at the same four strengths on datasets without ground-truth references.
  • Low-light Enhancement Evaluation: Figure 8 shows natural textures and colors, preserved structures, and smooth, approximately linear enhancement changes across strengths.The visual outputs may differ from ground-truth color appearance, which can affect reference-based metrics.
  • Low-light Enhancement Evaluation: ControlLight achieves the best results on most metrics among domain-specific methods on paired benchmarks and consistently outperforms all baselines on real-world benchmarks.These results are reported in Tables 1 and 2 and are used to demonstrate real-world generalization.
  • Linear Control Evaluation: ControlLight achieves the highest CLIP-Dir score among continuous editing methods, indicating stronger semantic alignment with increasing enhancement strength.The comparison uses aligned four-point control strengths and also reports trajectory smoothness with δsmooth.
  • Ablation Study: The misalignment-aware weighted flow matching loss reduces structural inconsistency while improving perceptual quality on the low-light subset of RealIR-Bench.The ablation compares the proposed loss with a standard flow matching objective using LI-LPIPS for consistency evaluation.
  • Ablation Study: Retinex-based interpolation provides richer degradation cues and a more pronounced, physically reasonable quality gradient than alpha blending interpolation.The resulting gradient is described as important for learning the enhancement mapping.

5 Conclusions

The conclusion presents ControlLight as a framework combining continuous pseudo-paired data, misalignment-aware flow matching, and LoRA fine-tuning. It reports state-of-the-art performance with consistency, controllability, and real-world generalization.

  • 5 Conclusions: ControlLight combines Light100K, a Retinex-inspired interpolation strategy, and Misalignment-Aware Weighted Flow Matching Loss with Offline Edge-Mask Generation.The loss is intended to suppress the effects of edge shifts during training.
  • 5 Conclusions: Fine-tuning FLUX.2-klein-9B with LoRA and the proposed LwFM establishes state-of-the-art performance against existing enhancement and continuous editing methods.The conclusion reports superior consistency, controllability, and generalization in real-world scenarios.

A Continuous Pseudo-Paired Data Construction Details

Light100K is constructed from filtered real-world low-light images, high-consistency restored references, and Retinex-inspired intermediate targets at multiple enhancement strengths.

  • Image Collection and Filtering: Low-light images are collected from Pexels and Pinterest and filtered using CLIP similarity to darkness-related prompts and Qwen3-VL-8B-Instruct degradation assessment.The pipeline selects images with relevant low-light semantics and degradation characteristics.
  • Reference Construction: FLUX.2-klein-9B generates normal-light references, and Sobel edge consistency filtering produces 17,809 high-consistency low-/normal-light image pairs.Pairs with obvious edge shifts or structural misalignment are removed.
  • Continuous Target Construction: Retinex-inspired interpolation generates intermediate pseudo targets at enhancement strengths s ∈{0.2, 0.4, 0.6, 0.8}, forming a continuous pseudo-paired dataset.Light100K is intended to provide real-world supervision for controllable low-light enhancement.

B Misalignment Analysis and Offline Edge-Mask Generation

Residual edge misalignment in pseudo targets can produce inconsistent outputs under standard flow matching. ControlLight uses structure-aware edge differences to down-weight unreliable regions during training.

  • Misalignment Analysis: Even visually negligible misalignment can become problematic during training because the generative base model may amplify it.The issue remains possible after edge-consistency filtering when residual offsets fall below its threshold.
  • Misalignment Analysis: Standard flow matching fits all target regions equally, which can introduce randomness and inconsistent outputs when pseudo targets are slightly misaligned.The inconsistency arises from learning against unreliable target regions.
  • Offline Edge-Mask Generation: The method compares illumination-normalized log-luminance structures and extracts structural edges to preserve input-image geometry during enhancement.Log-luminance avoids direct RGB comparison so differently bright images can share similar structural responses.
  • Offline Edge-Mask Generation: A structural edge-difference map generates spatial masks and adaptive weights for the flow-matching objective, with weight maps resized to latent resolution and cached offline.The implementation uses d = 3 pixels, α = 0.8, and wmin = 0.2.

C Implementation Details

ControlLight is fine-tuned with LoRA-based parameter updates while the VAE and text encoders remain frozen. Training uses fixed 1024 × 1024 bucket resolution, global batch size 16, and four NVIDIA A6000 GPUs.

  • Training configuration: LoRA fine-tuning updates only the DiT blocks while freezing the VAE and text encoders.LoRA layers are applied to both single-stream and double-stream DiT blocks.
  • Training configuration: The LoRA layers use rank 64 across the single-stream and double-stream DiT blocks.
  • Training configuration: Training uses 1024 × 1024 bucket resolution and a global batch size of 16.The experiments use four NVIDIA A6000 GPUs.

D.1 More Qualitative Results

Additional qualitative comparisons evaluate ControlLight against both low-light enhancement methods and general continuous image-editing methods.

  • Qualitative comparisons: Figure 9 presents additional qualitative comparisons with low-light enhancement methods.
  • Qualitative comparisons: Figure 10 presents additional qualitative comparisons with general continuous image-editing methods.
  • Qualitative comparisons: The comparisons cover both specialized low-light methods and general continuous image-editing methods.

D.2 More Ablation Study Details

The ablation study evaluates interpolation strategies through image-quality trajectories and qualitative comparisons across low-light and continuous image-editing settings. Retinex-based interpolation produces more natural quality trends and preserves low-light degradation cues.

  • Interpolation ablation: NIQE and MUSIQ measure image-quality trajectories across four intermediate enhancement strengths on 200 Light100K images.The evaluation targets smooth and monotonic transitions from low-light to normal-light quality.
  • Interpolation ablation: Retinex-based interpolation yields more natural image-quality trends than alternative interpolation strategies.It also provides richer degradation cues for continuous enhancement learning.
  • Interpolation ablation: Retinex-based interpolation preserves intrinsic degradation at low enhancement levels, whereas Alpha Blending produces artificially high scores that deviate from the low-light distribution.
  • Qualitative evaluation: Additional qualitative comparisons include paired low-light benchmarks LOL-v1 and LSWR, RealIR-Bench, and four continuous enhancement strengths.The reported figures compare ControlLight with both low-light enhancement and universal continuous image-editing methods.
Loading 2605.25569v2…