PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Interpolation for Training-Time Prediction Reshaping in Low-Light Image Enhancement

Peirui Zhu, Shijie Hao, Jingxi Liao, Richang Hong, Meng Wang

arXiv:2610.18018v1cs.CV

TL;DR

Low-light image enhancement must contend with coupled illumination, noise, and structural degradation, while existing architecture and objective designs can bring deployment burdens or tuning demands. DITM uses brightness-aware interpolation to reshape predictions during supervised training. The authors report improvements across multiple datasets and backbone families, with no added inference-time computation; gains were marginal and inconsistent on the two investigated diffusion models.

  • Problem

    Low-light image enhancement must balance insufficient illumination, noise amplification, and structural degradation, while existing architecture- and loss-based methods can increase deployment burden or require empirically tuned weights.

    Source: §I · p. 1

  • Method

    DITM adaptively interpolates a model prediction with a mean brightness-aligned image during supervised feed-forward LLIE training, without adding learnable parameters.

    Source: §I · pp. 1–2

  • Results

    The authors report stable quantitative and qualitative improvements across multiple benchmark datasets and CNN-, Transformer-, and Mamba-based LLIE backbones, with no additional inference cost.

    Source: §I · p. 2, §V · p. 16

  • Takeaways & Limitations

    DITM lets supervised feed-forward LLIE backbones retain their deployed inference architecture while using a training-time prediction reshaping strategy.

    Source: §I · p. 2, §V · p. 16

    On the two investigated diffusion-based LLIE models, DITM provides only marginal and inconsistent gains, with reshaped reconstruction supervision not effectively connected to diffusion denoising parameters.

    Source: E. Applicability Scope and Limitations… · p. 16

Abstract

from arXiv · show

Low-light image enhancement (LLIE) is an important image/video processing problem for reliable visual systems under poor illumination, yet it remains challenging due to the strong coupling among insufficient illumination, noise amplification, and structural degradation. Existing methods typically rely on sophisticated network architectures or engineered objective functions. While effective, the former increases model complexity and deployment burden, whereas the latter relies on empirically tuned loss weights. We revisit supervised LLIE from a training-dynamics perspective and observe that brightness-related errors tend to converge early, while residual noise and structural artifacts often remain insufficiently optimized in later stages. To address this issue, we propose a Dynamic Interpolation Training-only Module (DITM), which introduces no additional learnable parameters and adaptively reshapes the model prediction for supervised feed-forward LLIE training. Specifically, DITM employs brightness-aware interpolation, in which the model prediction is combined with a mean brightness-aligned image constructed by rescaling the model prediction according to the mean brightness of the ground-truth image. By reducing the relative dominance of global brightness discrepancy, DITM makes residual local errors more influential during backpropagation after global luminance has largely stabilized, thereby facilitating artifact suppression and fine-detail restoration in later training stages. DITM is used only during training and is fully removed at inference, introducing no additional inference cost. Extensive experiments on multiple benchmark datasets and CNN-, Transformer-, and Mamba-based LLIE backbones show that DITM consistently improves fidelity and perceptual quality while preserving the inference-time architecture and introducing no additional inference-time computation.

I. INTRODUCTION

Low-light image enhancement must balance illumination restoration with noise suppression and structural fidelity, while existing architecture- and loss-based approaches can add deployment burden or require empirical tuning. DITM addresses an observed training imbalance by reshaping predictions so residual local errors matter more after global brightness has largely stabilized, without changing inference-time architecture or cost.

I. INTRODUCTION

  • Low-light image enhancement couples poor illumination with noise amplification and structural degradation, making visibility, denoising, and detail preservation difficult to balance.Architecture-based solutions may increase computational overhead or latency, while specialized loss terms can require extensive empirical weight tuning.Source: §I · p. 1
  • 72.3%–96.2% of total BMAE reduction occurs within the first 20% of training, while local noise, blurred structures, and artifacts may persist into late stages.The authors use this asynchronous convergence pattern to motivate further optimization of local restoration errors.Source: §I · pp. 1–2
  • DITM interpolates the model prediction with a mean brightness-aligned image constructed by rescaling the prediction according to the ground-truth mean brightness.An adaptive interpolation factor adjusts the training signal by reducing global luminance discrepancy’s relative dominance, making residual local errors more influential during later backpropagation.Source: §I · pp. 1–2
  • The authors report cleaner local structures and textures with fewer residual artifacts in DITM-based model examples.The method is presented as a lightweight, architecture-agnostic training-time change for supervised feed-forward LLIE.Source: §I · p. 2
  • DITM adds no learnable parameters, changes neither the backbone architecture nor the retained reconstruction loss, and is fully removed at inference.The deployed model retains the baseline’s inference parameters, operations, and latency.Source: §I · p. 2

A. Learning-based Low-Light Image Enhancement

Prior LLIE work improves restoration through architecture and objective design, while training-side strategies seek optimization gains without necessarily changing deployment. DITM instead reshapes the supervised prediction through brightness-aware interpolation and is removed from the inference pipeline.

A. Learning-based Low-Light Image Enhancement

  • LLIE methods span CNN, Transformer, state-space, and diffusion architectures, as well as degradation- and frequency-aware designs; architecture changes may add inference cost, and loss weights often need empirical tuning.Low-light video enhancement also has to address temporal consistency across consecutive frames.Source: A. Learning-based Low-Light Image… · p. 3

B. Training-Side Strategies for Efficient Vision Models

  • Training-side approaches include temporary branches, auxiliary objectives, curricula, sample selection, and adaptive reweighting; DITM instead reshapes the supervised prediction while keeping the original loss form and inference model unchanged.Unlike many structural plug-ins, DITM does not become part of the deployed model.Source: B. Training-Side Strategies for… · pp. 3–4

A. Overview

  • DITM is a plug-and-play training-only module that operates on the model prediction rather than modifying the backbone’s internal architecture.It is designed for integration across supervised LLIE backbones while leaving the inference architecture unchanged.Source: A. Overview · p. 4
  • DITM rescales the model prediction using the ground-truth image’s global mean brightness, interpolates the resulting prediction, and computes the training loss against the ground truth.At inference, the baseline network directly produces the enhanced image; DITM introduces no inference parameters, operations, or latency.Source: A. Overview · p. 4

B. Dynamic Construction of Interpolated Prediction

DITM constructs a mean-brightness-aligned version of the model prediction and interpolates it with the raw prediction. The resulting prediction is used for training loss, with interpolation strength adjusted according to brightness discrepancy.

B. Dynamic Construction of Interpolated Prediction

  • B is formed by uniformly rescaling prediction A to match ground-truth Y’s global mean brightness while retaining A’s spatial layout.This provides global mean-brightness alignment, not explicit spatial illumination modeling.Source: B. Dynamic Construction of… · p. 4
  • DITM interpolates A and B using sample-wise factor s to produce C, the prediction used for loss computation.s is based on brightness discrepancy and is treated as constant during backpropagation.Source: B. Dynamic Construction of… · pp. 4–5, Figure 2 · p. 5
  • Supervising C weakens global brightness mismatch’s relative dominance so residual local errors can exert greater influence after luminance is largely corrected.The resulting gradient emphasis can help suppress artifacts and restore details.Source: B. Dynamic Construction of… · p. 5

C. Adaptive Interpolation Factor Design

DITM’s interpolation factor balances the raw prediction A and brightness-aligned image B using weights tied to brightness discrepancy. Its design shifts their relative contributions smoothly as the discrepancy changes during training.

C. Adaptive Interpolation Factor Design

  • The factor s balances A and B through non-negative weights wA and wB, with s defined as wA/(wA+wB).The weighted-centroid interpretation approaches A when wA dominates and B when wB dominates.Source: C. Adaptive Interpolation Factor Design · pp. 5–6
  • wA quantifies global brightness discrepancy between A and Y using their mean intensities, with larger deviations emphasized by a quadratic form.A and Y are normalized to [0, 1] when computing the brightness means and wA.Source: C. Adaptive Interpolation Factor Design · p. 6
  • A complementary exponential function makes wB increase smoothly as wA decreases, with λ controlling transition sharpness and δ its transition point.λ and δ are fixed empirically across datasets and backbones in this paper.Source: C. Adaptive Interpolation Factor Design · p. 6
  • Early in training, larger brightness discrepancy preserves a larger s and stronger contribution from A; as discrepancy falls, s decreases and B contributes more.In the latter stage, residual noise and structural artifacts exert relatively stronger influence on gradient updates.Source: C. Adaptive Interpolation Factor Design · p. 6

D. Analysis of Training Response Induced by DITM

The response analysis treats s as fixed within each optimization step and examines how DITM changes the training objective’s input-space response. As brightness discrepancy declines, the response becomes less dominated by global luminance correction and more aligned with spatially non-uniform residual errors.

D. Analysis of Training Response Induced by DITM

  • Because s is recomputed per sample but its gradient is stopped, the response derivation treats its forward-pass value as constant within each optimization step.The analysis uses loss response with respect to input X as a local indicator of objective behavior across spatial regions.Source: D. Analysis of Training Response… · p. 6
  • The response combines spatially varying backbone gradients with a global luminance-coupling term arising from β’s dependence on prediction mean brightness.The global term does not impose an explicit local structural constraint.Source: D. Analysis of Training Response… · pp. 6–7
  • As brightness discrepancy decreases, s decays and B gains relative influence, making residual local discrepancies more influential in the gradient response.The response tends toward structural boundaries, texture regions, and noise-contaminated areas; experiments in Section IV-D provide empirical support for this pattern.Source: D. Analysis of Training Response… · p. 7

E. Further Discussion on Training–Inference Consistency

During training, the backbone receives only low-light input X, while ground truth Y is used on the loss side to construct C. At inference, DITM is removed and the trained backbone directly produces A*.

E. Further Discussion on Training–Inference Consistency

  • The backbone receives X alone during training; Y is used only to construct the loss-side prediction C, not as an input, condition, or feature.The training objective is L(C, Y).Source: E. Further Discussion on… · p. 7
  • DITM adds neither learnable parameters nor a prediction branch, and gradients update backbone parameters through C’s dependency on raw prediction A.The adaptive factor s is detached and treated as constant for each sample within the optimization step.Source: E. Further Discussion on… · p. 7
  • B = βA means C uses only Y’s scalar mean-brightness statistic and inherits all spatial information from A, not Y.Consequently, local detail restoration and artifact suppression are learned by the backbone and reflected in A.Source: E. Further Discussion on… · p. 7
  • After training, DITM is removed and the deployed model maps X directly to A* without requiring Y or its brightness statistics.Validation and test results use raw output A*, not the ground-truth-dependent C.Source: E. Further Discussion on… · p. 7

F. Computational Complexity of DITM

DITM adds only linear image-sized training computations and storage, with no trainable parameters or inference-time changes. Experiments evaluate it across paired and unpaired datasets, diverse backbones, and visual comparisons.

F. Computational Complexity of DITM

  • DITM adds O(H × W) computation and storage for images of size H × W, while adding no trainable parameters and leaving inference architecture and complexity unchanged.The extra memory stores B and C, each of size H × W; the paper describes the overhead as negligible relative to the backbone.Source: F. Computational Complexity of DITM · p. 7

A. Experimental Setup

  • The experiments span eight LLIE datasets and seven CNN-, Transformer-, and Mamba-based models, with paired settings retrained under matched conditions except for enabling DITM.Each baseline and DITM counterpart was independently trained three times; results are reported as mean and standard deviation.Source: A. Experimental Setup · p. 8

B. Quantitative Evaluation

  • DITM improves mean performance in 124 of 126 paired comparisons across three LOL datasets, seven backbones, and six metrics.Gains cover PSNR, SSIM, LPIPS, and DeepSSIM; the exceptions are minor IAA degradations for MIRNet on LOLv1 and MambaLLIE on LOLv2-real.Source: B. Quantitative Evaluation · p. 8
  • Across five unpaired datasets, all seven DITM-equipped methods improve weighted-average scores on three metrics except MUSIQ for RetinexNet and CID-Net.The reported pattern suggests generally improved or maintained cross-dataset generalization, especially on Q-Align IQA and IAA.Source: B. Quantitative Evaluation · p. 8

C. Visual Quality Evaluation

  • The paired visual comparisons show reduced noise and artifacts and improved structural consistency, with color restoration, sharper edges, or texture gains noted for different backbone families.Fig. 3 provides zoomed-in regions comparing input, baseline, DITM, and ground truth.Source: C. Visual Quality Evaluation · p. 8, Figure 3 · p. 9
  • On unpaired real-world images, DITM alleviates examples of halo, local-artifact, exposure-instability, stripe, and color-blotch problems across supervised models.The examples are described as showing fewer artifacts and better brightness balance.Source: C. Visual Quality Evaluation · p. 8

D. Further Analysis of DITM

Comparisons with fixed, heuristic, and replacement interpolation strategies test whether DITM’s adaptive prediction interpolation matters. DITM consistently outperforms the alternatives across the evaluated paired datasets and metrics.

1) Comparison with Alternative Interpolation Strategies:

  • The comparison includes the original backbone without interpolation, B-only replacement, fixed interpolation, heuristic dynamic interpolation, and DITM.The study was designed to distinguish adaptive interpolation gains from generic prediction interpolation.Source: 1) Comparison with Alternative… · p. 10
  • DITM consistently outperforms the alternative interpolation strategies across LOLv1, LOLv2-real, and LOLv2-synthetic under different evaluation metrics.The comparison uses RetinexFormer as the baseline.Source: 1) Comparison with Alternative… · p. 10, Table III · p. 11
  • Unlike fixed-weight ISW and manually scheduled HDI, DITM estimates the interpolation factor per sample, adapting to varying image conditions.The tested ISW uses weights 0.8 for A and 0.2 for B; HDI changes its weight over three fixed training stages.Source: 1) Comparison with Alternative… · pp. 10–11
  • Replacing prediction A entirely with B removes A’s direct brightness-discrepancy supervision signal and substantially degrades training stability and final performance.The reported strategy comparison also says this weakens effective optimization.Source: 1) Comparison with Alternative… · pp. 10–11

2) Hyperparameter Study:

Validation performance is relatively stable across tested λ and δ settings, and the study uses (10, 0.3) uniformly. Training analyses show DITM’s interpolation factor shifts toward smaller values as training progresses, alongside more focused later-stage responses and improved detail restoration.

2) Hyperparameter Study:

  • Validation performance remains relatively stable across tested λ and δ settings, and (λ, δ) = (10, 0.3) is selected for all other experiments without model- or dataset-specific retuning.λ controls the variation rate of s, while δ determines the transition region where s changes more rapidly.Source: 2) Hyperparameter Study · p. 11

3) Analysis of Training Process with DITM:

  • At 10K iterations, both models show broadly distributed gradients, large image-wide reconstruction errors, and visibly underenhanced results with less-suppressed noise.This early-stage comparison precedes the reported middle- and late-stage differences.Source: 3) Analysis of Training Process with… · p. 11
  • At 80K iterations, DITM shows more concentrated structural-region gradients and more localized, reduced reconstruction errors than the baseline, alongside clearer structures and less noise.The authors relate this pattern to the growing contribution of the brightness-aligned branch as s evolves.Source: 3) Analysis of Training Process with… · p. 12, Figure 7 · p. 13
  • At 150K iterations, DITM has cleaner, more uniform error maps and fewer non-structural gradient responses, while its outputs show less noise and more consistent structural details than the baseline.The comparison uses input-gradient and reconstruction-error maps at different training stages.Source: 3) Analysis of Training Process with… · p. 12, Figure 8 · p. 13
  • DITM’s factor s shifts toward smaller values on LOLv1, LOLv2-real, and LOLv2-synthetic from 10K to 150K iterations, while the proportion below 0.1 increases on each dataset.For example, on LOLv1, mean s falls from 0.381 to 0.082 and R_s<0.1 rises from 49.1% to 85.4%.Source: 3) Analysis of Training Process with… · p. 14, Figure 9 · p. 14
  • At later stages, thin upper tails in the s distributions indicate that some samples retain larger values, preserving more raw-prediction contribution when brightness discrepancies remain.This per-sample behavior differs from a fixed iteration-dependent schedule.Source: 3) Analysis of Training Process with… · p. 14

4) Robustness under Extremely Low-Light Conditions:

On the darkest 10% of LOLv1 training samples, tested RetinexFormer and RetinexMamba settings remain numerically stable across training stages, with no NaN or Inf values reported.

4) Robustness under Extremely Low-Light Conditions:

  • For the darkest 10% of LOLv1 training samples, tested RetinexFormer and RetinexMamba settings show no NaN or Inf values, and the reported luminance statistics and scaling ratio remain numerically stable.Across training stages, mean s decreases from 0.5739 to 0.1163 for RetinexFormer and from 0.6324 to 0.1354 for RetinexMamba.Source: 4) Robustness under Extremely… · p. 14, Table IV · p. 14

5) Performance under Spatially Non-Uniform Illumination:

On the top 20% spatially non-uniform LOLv2-synthetic samples, DITM improves both evaluated models across several metrics. The section also reports training-time overhead and describes the method’s applicability boundaries.

5) Performance under Spatially Non-Uniform Illumination:

  • DITM consistently improves RetinexFormer and RetinexMamba on the spatially non-uniform subset across five metrics, with positive average PSNR gains.The subset is the top 20% of LOLv2-synthetic test images ranked by illumination nonuniformity; Table V reports three independent runs.Source: Table V · p. 15, 5) Performance under Spatially… · p. 15

E. Applicability Scope and Limitations of DITM

  • DITM is scoped to paired supervised LLIE; diffusion models without image-space reconstruction cannot directly adopt it, and reconstruction-branch benefits depend on optimization connections.The current formulation requires a paired normal-light reference during training.Source: E. Applicability Scope and Limitations… · pp. 15–16

6) Training-Time Overhead:

  • Table VII reports DITM’s training-time overhead on the adopted LLIE backbones.Source: Table VII · p. 15

V. CONCLUSION

DITM reshapes predictions during supervised feed-forward LLIE training without modifying the backbone or adding inference-time cost. Experiments report improvements across benchmark datasets and CNN-, Transformer-, and Mamba-based backbones; the current design relies on paired supervision.

V. CONCLUSION

  • DITM generally improves quantitative and qualitative results across multiple benchmark datasets and CNN-, Transformer-, and Mamba-based LLIE backbones.The strategy preserves the backbone architecture and adds no inference-time cost.Source: §V · p. 16
  • During training, DITM interpolates raw predictions with brightness-aligned predictions so residual artifact-, structure-, and detail-related errors become more involved in optimization.The brightness-aligned prediction is based on the ground-truth mean brightness.Source: §V · p. 16
  • The current DITM design is limited to paired supervised LLIE because it relies on ground-truth mean brightness during training.Future work will explore pseudo brightness targets or learned references for unpaired, self-supervised, and zero-reference LLIE.Source: §V · p. 16
  • Future work will explore spatially adaptive illumination guidance for non-uniform lighting and prediction reshaping in diffusion-based restoration frameworks.For diffusion frameworks, the paper highlights determining appropriate placement along the denoising trajectory.Source: §V · p. 16

Related papers

Loading 2610.18018v1…