Physics Guided Diffusion Model Based Exposure Interpolation for High Dynamic Range Imaging

I. INTRODUCTION

• Sequentially captured exposures can produce ghosting from moving objects, while direct fusion of simultaneous LER images can produce BOR artifacts.The paper motivates interpolating an image between the LER exposures before fusion.Source: §I · p. 1
• The proposed exposure interpolation algorithm outperforms existing exposure interpolation algorithms [18]–.It combines a physics-driven component with a new diffusion-model component.Source: §I · p. 2
• The new diffusion model uses a predicted target designed to reduce noisy-data correlation variation and fitting-error amplification.The paper reports that this model outperforms the model in.Source: §I · p. 2

A. Exposure interpolation

• Neural augmentation learns the residual after physics-driven interpolation, rather than learning the entire target image, when the initial interpolation is accurate.The cited approach's convergence rate is much faster under that condition.Source: A. Exposure interpolation · p. 2
• CNN-based components have difficulty learning high-frequency information, motivating their replacement with a diffusion model for neural-augmented interpolation.The proposed algorithm reports preserving more high-frequency information.Source: A. Exposure interpolation · p. 2, §I · p. 2

B. Diffusion models

• The forward process maps data toward noise using coefficients a11(t) and a12(t), which specify the forward path.The coefficients' choices affect training and sampling efficiency.Source: B. Diffusion models · p. 2
• A neural network maps noisy data Xt and time t to a predicted target, which supports estimating ground-truth data and Gaussian noise during reversal.Those estimates are used in the reverse process.Source: B. Diffusion models · p. 3
• The reverse process converts noise back to data, and Φ2(t′, t) is the amplification factor on network fitting error.The reverse update uses the noisy data and the network prediction.Source: B. Diffusion models · p. 3
• The overall reverse-process prediction error also depends on fitting errors and the Pearson correlation between noisy data Xt and target ωt.The number of reverse steps should be kept small to reduce reverse-process time.Source: B. Diffusion models · p. 3
• Because the network is shared across times, the paper identifies reducing correlation variation and the magnitudes of Φ1 and Φ2 as diffusion-model design guidelines.The figure caption links these reductions to lower reverse-process fitting error and improved interpolation quality.Source: Figure 1 · p. 3, B. Diffusion models · p. 4

III. PROBLEM FORMULATION

• The inputs are aligned LER images Z1 and Z7, and the target is an interpolated image with an intermediate exposure time.The images span the scene’s dynamic range; their exposure times are ordered, with ∆t1 < ∆t4 < ∆t7.Source: §III · p. 4
• The interpolation combines the physics-driven representation fm(Z1, Z7) with a data-driven representation fd,t(Z1, Z7).The initial interpolated image is set to the captured middle-exposure image at t = 0.Source: §III · p. 4
• The residual e0 is the difference between the physics-driven estimate and the captured image Z4, and it mainly contains missing high-frequency details.The learned residual is added to the physics-driven representation to form the interpolated image.Source: §III · p. 4

IV. GENERATIVE AI-BASED EXPOSURE INTERPOLATION

• The conditional distribution q(e0|fm(Z1, Z7)) is learned with a diffusion model to recover high-frequency information missing from the physics-driven interpolation.The paper states that diffusion has more chance to learn these details than CNN-based methods.Source: §IV · p. 4

A. The proposed target to be predicted

• A neural network gθ(et, fm, t) maps the noisy residual et to a predicted target ωt during the forward process.The training procedure uses aligned LER inputs and a ground-truth middle-exposure image; it computes the physics-driven prior and high-frequency target residual before adding sampled Gaussian noise.Source: §IV · p. 4, A. The proposed target to be predicted · p. 5
• The target choice is designed to reduce variation in the Pearson correlation between noisy data and the predicted target, as well as the fitting-error amplification factor.The target is compared with the choice in [30]; the paper sets τ = T/5 and α = 0.02.Source: A. The proposed target to be predicted · pp. 4–5
• The reverse process progressively reduces noise to recover the residual e0 from a noise sample, using the network output and noisy data to estimate the ground truth and noise.The final predicted residual is combined with the physics-driven image to reconstruct the middle-exposure image.Source: A. The proposed target to be predicted · pp. 4–5
• The diffusion model can be transferred to latent space, compressing spatial dimensions by a factor of four to reduce computational cost.The transfer substitutes latent codes for Z4 and fm(Z1, Z7).Source: A. The proposed target to be predicted · p. 5

B. Comparison with the target to be predicted in [30]

• The reverse process for the alternative target uses Φ(2)2(t′, t) and the network prediction g(2)θ(et, fm, t).The process is given as a reverse update from noisy residual data and the predicted target.Source: B. Comparison with the target to be… · p. 5
• For T = 5, |Φ(1)2(t′, t)| and the variation of Ψ for the proposed target are smaller than for the alternative target choice.Figure 2 plots |Φ1|, |Φ2|, and Ψ for the two target choices in equations (23) and (32).Source: Figure 2 · p. 6
• Figure 3 compares the input pair, outputs from physics-driven methods in [18] and [19], the adopted component, and the ground truth.Source: Figure 3 · p. 6

A. Experimental Settings

• The datasets from and are split into 420 training, 40 validation, and 60 testing sequences.The dataset from is used only for testing and is not involved in training.Source: A. Experimental Settings · p. 6
• Camera motion and object movement are controlled, and sequences with noticeable misalignment are excluded.Images are randomly cropped into 512 × 512 patches to augment training data and improve generalization.Source: A. Experimental Settings · p. 6
• The experiments are implemented in PyTorch on an NVIDIA A6000 GPU using Swin-UNet as the backbone.Training uses batch size 4 and an initial learning rate of 5 × 10−5 with cosine annealing.Source: A. Experimental Settings · p. 6

B. Comparison of Different Physics-driven Exposure Interpolation Components

• The adopted physics-driven component plays an important role in the proposed neural-augmented framework.Source: B. Comparison of Different… · p. 6
• The adopted component consistently outperforms the methods in and across full-reference and no-reference image-quality metrics.The comparison includes SSIM, PSNR, FSIM, LPIPS, CLIPIQA, Entropy, MUSIQ, and HyperIQA.Source: B. Comparison of Different… · p. 7, Table I · p. 7
• Compared with and, its interpolated images retain natural color tones and fine textures and appear closest to ground truth.The cited examples contrast pavement and building color casts, washed-out skies, cloud structures, and tile patterns.Source: B. Comparison of Different… · p. 7
• For T=5, both compared models improve rapidly early in training and then stabilize, indicating stable training.Fig. 4 compares training curves for two choices of ωt; the text identifies SSIM and PSNR as the measures.Source: C. Comparison Between Different ωt’s · p. 7, Figure 4 · p. 7

D. Ablation study on different α

• α = 0.02 achieves the best results on six of eight metrics, including PSNR, FSIM, LPIPS, Entropy, MUSIQ, and HyperIQA, and is selected overall.The comparison tests α ∈ {0.02, 0.1, 1.0}, with τ fixed at T/5 and T set to 5.Source: D. Ablation study on different α · p. 7, Table III · p. 8

E. Comparison of Different Neural Augmentation-based Exposure Interpolation Components

• Compared with CNN-based augmentation,, diffusion-based interpolation produces sharper results with clearer fine details, especially in the highlighted foliage region.The figure compares the two input exposures with results from,, and the proposed method; red circles mark enlarged regions.Source: E. Comparison of Different Neural… · p. 7, Figure 5 · p. 8

VI. CONCLUSION REMARKS

• The proposed interpolation achieves the best results on all eight metrics across datasets,, and, and its integrated MEF algorithm outperforms six state-of-the-art exposure fusion algorithms.The conclusion also identifies speed as an issue and points to reducing reverse-process sampling cost as future work.Source: E. Comparison of Different Neural… · p. 8, §VI · p. 9, Table IV · p. 9

F. Application of Exposure Interpolation to Exposure Fusion

• Adding the interpolated image to exposure fusion outperforms six alternatives on MEF-SSIM, TMQI, SSIM, and PSNR.Subjectively, BOR is alleviated and detail or color issues noted for alternatives are addressed, though the method can still suffer from halo artifacts.Source: F. Application of Exposure… · p. 8, Table V · p. 9

• The initial image is interpolated from LER images Z1 and Z7 at exposure time ∆t4 using corresponding intensity mapping functions estimated from camera response functions.Source: APPENDIX: PHYSICS-DRIVEN EXPOSURE… · p. 9
• The intermediate image has base and detail layers, with weights specified for constructing those layers.Source: APPENDIX: PHYSICS-DRIVEN EXPOSURE… · pp. 9–10
• The mapped image uses local channel deviations from means computed in a 3 × 3 window centered at each pixel.The prose defines the deviation relative to the local mean; the mapped images are generated from the two LER inputs.Source: APPENDIX: PHYSICS-DRIVEN EXPOSURE… · p. 9
• Because the physics-driven image has limited representation capacity, residual information from Z4 is explicitly interpolated using an asymmetric diffusion model.Source: APPENDIX: PHYSICS-DRIVEN EXPOSURE… · p. 10