Source-linked AI summary
P-PatchDiff: Progressive Patch Diffusion Models for Low-light Image Enhancement
Ruoyu Guo, Haonan Zhong, Maurice Pagnucco, Yang Song
TL;DR
Low-light enhancement is difficult because diverse degradations obscure details and limit available information, while existing approaches typically rely on pixel-level regression losses. P-PatchDiff progressively increases patch sizes to shift from local to global views and aligns features across scales using a global brightness proxy; experiments across ten datasets demonstrate effectiveness and efficiency, while overlapping full-resolution patches remain computationally costly.
Problem
Low-light enhancement is challenging because diverse degradations obscure fine details, reduce visibility, and limit available information, while deep learning methods typically rely on regression models with pixel-level losses.
Method
P-PatchDiff progressively increases patch sizes across time steps and uses multi-patch alignment with an estimated global brightness proxy to align features across scales and capture global context.
Results
Extensive experiments on ten low-light image enhancement datasets demonstrate P-PatchDiff's effectiveness and efficiency while balancing local dark-region enhancement with globally coherent brightness.
Takeaways & Limitations
P-PatchDiff provides a scalable approach that combines local dark-region enhancement with globally coherent brightness for spatially variant low-light images.
Takeaways & Limitations
Progressive patchifying does not substantially reduce overall computational cost because patches are cropped from the full-resolution image, and overlapping patches introduce redundant computation.
Abstract
from arXiv · showhide
Recent advancements in low-light image enhancement have leveraged diffusion models for their strong ability to generate perceptually realistic, detailed images. Patch diffusion models further offer a promising solution to size-agnostic image restoration while improving efficiency. However, existing methods typically rely on small, fixed patches (e.g., 64$\times$64) that cannot capture image-level brightness context, whereas enlarging the receptive field improves brightness and colour estimation but substantially increases computational cost. Moreover, low-light images often exhibit uneven brightness across regions, making it necessary to ensure that locally enhanced patches remain visually coherent when combined into the full image. To address these limitations, we propose P-PatchDiff, a scalable progressive patch diffusion framework for low-light image enhancement that dynamically adjusts patch size throughout the denoising process, enabling a gradual shift from local to global views. A Multi-Patch Alignment strategy is also introduced to normalise features across varying patch scales using an estimated global brightness proxy. Rather than pursuing pixel-level reconstruction accuracy, P-PatchDiff focuses on scalability and coherent brightness across the whole image, allowing the model to perceive multi-scale information and better enhance regions with varying brightness. We empirically demonstrate that P-PatchDiff effectively enhances images ranging from 400 $\times$ 600 to 4K and is 80$\times$ faster than existing patch diffusion models while using less than 9GB of memory. The code is available at https://github.com/RuoyuGuo/P-PatchDiff.
1 Introduction
Low-light enhancement requires detailed restoration despite diverse degradations, while patch diffusion must balance local illumination changes with globally coherent brightness and scalable computation. P-PatchDiff addresses these needs by progressively increasing patch size and stride, aligning features across scales, and targeting efficient enhancement across varied resolutions.
- Regression models trained with pixel-level losses can produce averaged, overly smooth outputs, whereas diffusion models generate greater detail.
- Low-light degradations obscure fine details, reduce visibility, and limit available information, making enhancement challenging.
- Fixed small patches improve efficiency but limit multi-level context and can create boundary artefacts, while spatially variant illumination requires both local and global information.
- P-PatchDiff is evaluated on images from 400 × 600 to 4K and is reported as 80× faster than existing patch diffusion models while requiring 8.8GB of GPU memory.
- P-PatchDiff progressively increases patch size and stride with the diffusion time step to capture multi-level information while reducing inference complexity.
- Multi-patch alignment normalises features across patch scales using an estimated global brightness proxy to maintain consistent brightness among patches.
2 Related work
Related work spans regression, diffusion, representation-based, prior-guided, and patch-based enhancement methods. Existing patch approaches improve scalability or multi-scale context but retain trade-offs involving resolution, boundary consistency, complexity, or fixed patch size.
- Regression-based methods often produce blurry outputs despite advances in architectures, unsupervised learning, and alternative colour representations.
- Diffusion enhancement methods improve perceptual quality through interpolation, pretrained-model adaptation, and transformed representations such as Retinex, wavelet, or Fourier components.
- Prior-guided diffusion incorporates brightness, Retinex, exposure, degradation, or image-quality-map cues into denoising.
- Patch diffusion improves computational efficiency and supports variable image sizes, but full-size sampling remains impractical for variable-resolution enhancement.
- Feature-level patching and hierarchical diffusion enable high-resolution or multi-scale sampling but require added spatial-consistency mechanisms or staged generation.
- WeatherDiff simplifies inference by averaging independently denoised overlapping patches, yet its fixed small patch size lacks multi-scale context.
3 Methods
P-PatchDiff progressively changes patch size and stride during denoising, shifting from local to global views while controlling computation. Multi-patch alignment uses a global brightness proxy to reduce cross-scale colour and brightness inconsistencies.
- Progressive patch diffusion: P-PatchDiff divides denoising steps into subsets with fixed patch settings within each subset and progressively changing settings across subsets.This guides the model from local toward global enhancement.
- Progressive patch diffusion: Overlapping patches are denoised independently and merged into the full image by averaging overlap areas.The model crops the concatenated noisy and low-light images before denoising.
- Multi-patch alignment: P-PatchDiff estimates a global brightness proxy from a downsampled full-size image and encodes it to normalise features across patch scales.A lightweight U-Net and encoder provide the global brightness feature used by the denoising U-Net.
- Revisiting patch diffusion models: Fixed patch diffusion methods either lack multi-level information or incur higher cost by processing multiple patch scales at each denoising step.WeatherDiff uses one fixed patch size, whereas MDMS creates multiple noisy-image copies for different patch sizes.
- Revisiting patch diffusion models: Larger patches tolerate stride variation but substantially increase training and sampling time, while larger strides reduce runtime until boundary artefacts emerge near s = p/2.The reported trade-off motivates progressively increasing both patch size and stride.
- Progressive patch diffusion: P-PatchDiff targets inconsistent colour and illumination across patches by combining progressive multi-scale processing with alignment across patch sizes.The method is designed for spatially varying low-light illumination rather than uniform adjustment.
4 Experiments
Experiments evaluate P-PatchDiff across in-domain, cross-domain, qualitative, computational, and ablation settings. The results indicate competitive enhancement and scalability, with progressive patching and alignment improving efficiency and brightness consistency.
- Evaluation settings: P-PatchDiff is evaluated on nine real-world and one synthetic dataset under in-domain and cross-domain settings.The evaluation covers varying resolutions, devices, and exposure levels, including 4K images.
- Quantitative comparisons: P-PatchDiff achieves competitive results on low-resolution in-domain benchmarks and cross-dataset evaluations on LSRW and UHD-LL.The cross-domain datasets include diverse ISO and 4K-resolution images.
- Quantitative comparisons: 0.1 lower SSIM and 0.16 higher LPIPS than P-PatchDiff are reported for DiffLL on 4K images, while resizing causes fine-detail loss.DiffLL performs well on 400×600 datasets but fails on 4K resolutions under these metrics.
- Qualitative results: P-PatchDiff produces better colour restoration than single-scale methods and comparable results to multi-scale methods while processing faster.The qualitative comparisons also report undersaturation, inconsistent colours, and excessive smoothness in competing methods.
- Ablation study: P-PatchDiff maintains low sampling times as n increases, whereas fixed-patch and multi-scale strategies become substantially slower.Increasing patch size alone can also reduce performance, as shown by the fixed-patch drop from n = 4 to n = 5.
- Ablation study: 28.04/0.887/2.45 (PSNR/SSIM/sampling time) is achieved at n = 5, while larger n values increase cost without consistently improving performance.The reported n = 6 and n = 7 results are 27.85/0.885/2.89 and 27.92/0.884/3.17, respectively.
- Ablation study: Multi-patch alignment improves brightness consistency and shadow-region enhancement while increasing sampling time by only 10%.Feature visualisations also show separation of different objects to facilitate denoising.
5 Discussion
Discussion experiments extend P-PatchDiff to deblurring and deraining, examine failure cases, and identify computational and structural limitations. The method generalises competitively but can omit structures in extremely dark regions and retains substantial full-resolution computation.
- Generalisation to other restoration tasks: P-PatchDiff generalises competitively to deblurring and deraining, with strong SSIM on GoPro indicating reasonable real-world structure preservation.On Rain100L, it does not achieve the best PSNR, partly attributed to the dataset’s limited scale.
- Generalisation to other restoration tasks: On Rain100L, P-PatchDiff does not achieve the best PSNR, possibly because the synthetic dataset has only 200 low-resolution training images.
- Failure case analysis: In extremely dark regions, P-PatchDiff may omit structures entirely because the input signal is too weak to detect them reliably.The reported failure is region-dependent: the same net texture is restored in a less degraded region.
- Failure case analysis: The authors hypothesise that semantic-consistency modelling could help infer structures from weak signals and leave this direction for future work.
- Limitations: Progressive patchifying improves memory efficiency and flexible processing but does not substantially reduce overall computation because patches are cropped from the full-resolution image.
- Limitations: Stride constraints reduce boundary artefacts but create duplicated pixels and redundant computation in overlapping areas.The authors suggest latent-space progressive patching as a possible way to reduce computation and improve boundary robustness.
6 Conclusion
P-PatchDiff combines progressive patch diffusion with multi-patch alignment to balance local enhancement, global brightness coherence, and computational efficiency. Experiments across ten low-light datasets support its effectiveness and efficiency for spatially variant images.
- Method and contribution: P-PatchDiff progressively shifts from local to global views while multi-patch alignment uses a global brightness proxy to resolve varying-patch output inconsistencies.
- Conclusion: P-PatchDiff balances local dark-region enhancement against globally coherent brightness while maintaining competitive enhancement quality and low computational costs.
- Conclusion: Experiments on ten low-light image enhancement datasets demonstrate the effectiveness and efficiency of the approach.
Declarations
The study reports that all datasets are publicly available and that the implementation is available online.
- Availability: All datasets used in the study are publicly available, and the code is available at the project repository.