Source-linked AI summary
Pyramid Diffusion Models For Low-light Image Enhancement
Dewei Zhou, Zongxin Yang, Yi Yang
TL;DR
Low-light enhancement must recover noise-covered details, but diffusion models introduce constant-resolution sampling and global degradation. PyDiff uses pyramid diffusion and a global corrector to address these issues, achieving superior effectiveness, efficiency, and generalization on benchmarks.
Problem
Low-light enhancement methods can leave noise-covered details insufficiently restored, while diffusion models introduce constant-resolution sampling and occasional global degradation.
Method
PyDiff combines progressively increasing-resolution sampling in one reverse process with a global corrector for reverse-process degradation.
Results
PyDiff achieves superior performance and generalization on popular benchmarks, with nearly 2× faster speed than LLFLOW.
Takeaways & Limitations
Pyramid diffusion preserves performance while improving diffusion sampling efficiency, and the global corrector improves performance with little additional computational consumption.
Abstract
from arXiv · showhide
Recovering noise-covered details from low-light images is challenging, and the results given by previous methods leave room for improvement. Recent diffusion models show realistic and detailed image generation through a sequence of denoising refinements and motivate us to introduce them to low-light image enhancement for recovering realistic details. However, we found two problems when doing this, i.e., 1) diffusion models keep constant resolution in one reverse process, which limits the speed; 2) diffusion models sometimes result in global degradation (e.g., RGB shift). To address the above problems, this paper proposes a Pyramid Diffusion model (PyDiff) for low-light image enhancement. PyDiff uses a novel pyramid diffusion method to perform sampling in a pyramid resolution style (i.e., progressively increasing resolution in one reverse process). Pyramid diffusion makes PyDiff much faster than vanilla diffusion models and introduces no performance degradation. Furthermore, PyDiff uses a global corrector to alleviate the global degradation that may occur in the reverse process, significantly improving the performance and making the training of diffusion models easier with little additional computational consumption. Extensive experiments on popular benchmarks show that PyDiff achieves superior performance and efficiency. Moreover, PyDiff can generalize well to unseen noise and illumination distributions.
1 Introduction
PyDiff introduces diffusion models to low-light enhancement to recover more realistic details, while addressing constant-resolution sampling and global degradation. Its pyramid diffusion and global corrector improve speed, quality, and generalization.
- Previous enhancement methods can leave blurred details and distorted colors, motivating diffusion models for more realistic restoration.
- Diffusion-based enhancement faces constant resolution during reverse sampling and occasional global degradation resembling RGB shifts.
- PyDiff progressively increases resolution during one reverse process, making sampling faster while preserving performance.Lower-resolution noisy sampling is faster and provides a larger receptive field for global information recovery.
- A global corrector alleviates degradation that denoising networks may not notice, improving performance with little additional computation.
- Experiments report new state-of-the-art performance, nearly 2× faster inference than LLFLOW, and generalization to unseen noise and illumination distributions.On unseen noise distributions, PyDiff is reported to achieve 10 points higher SSIM than the second-place method.
2 Related Work
Prior low-light enhancement methods use paired supervision, Retinex decomposition, unsupervised learning, specialized networks, or normalizing flows. Diffusion research improves synthesis quality but generally retains slow, constant-resolution reverse processes, which PyDiff targets.
- Supervised methods learn enhancement from low/normal-light pairs, while Retinex-based methods decompose illumination and reflectance with designed losses.
- Unsupervised approaches avoid paired data, and other methods improve results through noise-color disentanglement or efficient network designs.
- LLFLOW models low-light enhancement as an ill-posed problem using a normalizing flow model.
- Diffusion models generate high-quality images through denoising, but typically require many iterations and therefore run slowly.
- Existing cascaded, latent-space, and related diffusion approaches still sample at constant resolution within one reverse process, limiting speed.
3 Background: Denoising Diffusion Probabilistic Models
DDPMs define a T-step Markov chain that progressively adds Gaussian noise, then use a learned reverse process to recover target data. For image translation, the reverse process can be conditioned on an input signal.
- A DDPM forward process starts from data and repeatedly adds Gaussian noise through a T-step Markov chain.
- The cumulative forward process transforms the data distribution toward an isotropic Gaussian distribution when the terminal signal contribution is near zero.
- The reverse process converts isotropic Gaussian noise back to the target distribution, using a tractable posterior involving the original data.
- Because the original clean sample is unavailable during testing, the denoising network predicts noise and uses it to approximate x0.
- For image-to-image translation, the reverse process is conditioned on a source signal such as a low-light image.
4 Methods
PyDiff combines pyramid-resolution diffusion with a global corrector to accelerate low-light enhancement while addressing global degradation such as RGB shifts. Its reverse process samples noisier states at lower resolutions and applies correction when error amplification is high.
- Pyramid Diffusion: PyDiff uses pyramid diffusion to progressively increase resolution during one reverse process, improving inference speed without performance degradation.Noisier sampling is performed at lower resolution, where the network also receives a larger receptive field for global information recovery.
- Pyramid Diffusion: A downsampling schedule controls the resolution at each sampling step, with lower resolutions assigned to noisier stages.The schedule satisfies s_t ≤ s_{t+1}, producing progressively higher resolutions as sampling proceeds.
- Pyramid Diffusion: Pyramid diffusion uses upsampled denoiser estimates when resolution increases, accepting limited state inconsistency because noisy sampling primarily recovers global information.The method treats upsampled yθ(x_t) as the lower-resolution clean estimate before deriving the next noisy state.
- Global Corrector: Global degradation can arise because the denoising error δ_t is multiplied by a coefficient that becomes large at high noise levels, producing RGB shifts.The denoising network may treat globally degraded images normally and fail to remove the resulting shift.
- Training and Sampling: Training optimizes the global corrector against downsampled normal-light targets while leaving the denoising network unaffected.The procedure samples paired low/normal-light images, diffusion times, and Gaussian noise before updating the corrector.
- Global Corrector: PyDiff adds a lightweight global corrector that performs pixel-independent retouching from global conditions while preserving generated edges and textures.The corrector is used only while the error-amplification factor exceeds a threshold, reducing unnecessary computation later in sampling.
5 Experiments
Experiments on LOL and LOLV2 evaluate PyDiff under standard, unseen-noise, and unseen-illumination conditions, alongside ablations of pyramid diffusion and the global corrector. PyDiff achieves strong quantitative and qualitative results, while the global corrector improves robustness with little additional computation.
- Experimental Setup: PyDiff is evaluated on LOL, LOLV2 REAL, and LOLV2 SYNC, with PyDiff trained only on LOL for supervised comparisons.LOLV2 REAL tests unseen noise distributions, while LOLV2 SYNC tests unseen illumination distributions.
- LOL Dataset: PyDiff outperforms competing methods on LOL in PSNR, SSIM, and LPIPS, beating second place by 2.1, 1, and 1 points, respectively.These gains correspond to more accurate colors, better high-frequency detail preservation, and more eye-pleasing results.
- LOLV2 REAL PART: On LOLV2 REAL, PyDiff beats second place by 10.9 points on SSIM and 21 points on LPIPS while removing unseen noise more effectively.The selected test images include ISO settings absent from LOL training, and qualitative results also show better restoration across exposure times.
- LOLV2 SYNC PART: On LOLV2 SYNC, PyDiff achieves first place and exceeds second place by 1.8 points on SSIM under illumination distributions absent from LOL.The result supports generalization to scenarios with different illumination distributions.
- Ablation Study: Ablations assess pyramid-diffusion downsampling schedules using combined performance across LOL, LOLV2 REAL, and LOLV2 SYNC, with FPS measured on LOL.Schedule [1, 1, 1, 1] represents vanilla diffusion, while Table 4 examines alternatives involving resolution changes.
- Ablation Study: The global corrector improves PyDiff under varied settings, alleviates global degradation, adds little computation, and reduces sensitivity to batch size.It improves performance at batch sizes 4, 8, and 16 relative to corresponding models without the global corrector.
6 Conclusion
The paper proposes PyDiff, a diffusion-based low-light enhancement method combining pyramid diffusion with a global corrector. Experiments report superior effectiveness, efficiency, and generalization, positioning PyDiff as a strong baseline for this task.
- Conclusion: PyDiff uses pyramid diffusion to accelerate sampling without performance degradation and a global corrector to alleviate global degradations with little additional computation.The paper proposes the method specifically for low-light image enhancement.
- Conclusion: Experiments on popular benchmarks show that PyDiff achieves superior effectiveness, efficiency, and generalization ability.The authors suggest PyDiff as a strong baseline and pyramid diffusion as a way to facilitate diffusion models in other low-level vision tasks.