PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement

Chuanjun Zheng, Daming Shi, Wentian Shi

arXiv:2110.00984v4eess.IVcs.CV

TL;DR

Real-world low-light images combine poor visibility with variable noise, and estimating that noise is complicated in sRGB. UTVNet learns a noise-level map through adaptive total-variation regularization and an unfolded minimization process. The authors report robustness across noise levels and best performance on the sRGB-SID comparisons described.

  • Problem

    Real-world low-light restoration must improve visibility while handling varying noise, which is harder to estimate in sRGB because the image-processing pipeline affects its distribution.

    Source: §1 · p. 1

  • Method

    UTVNet learns a per-pixel noise-level map as the balancing parameter of adaptive total-variation regularization, using an unfolded minimization process to provide fidelity and smoothness constraints.

    Source: §1 · p. 2

  • Results

    UTVNet is reported as robust across various noise levels in real captured scenes and achieves the best performance in four metrics among compared methods on the sRGB-SID dataset.

    Source: §1 · p. 2, §4.1 · p. 6

  • Takeaways & Limitations

    The approach restores real-world low-light noisy images in sRGB by using the estimated noise map to guide luminance correction and noise suppression across smooth and detail layers.

    Source: §1 · p. 2, §5 · p. 8

    UTVNet and other state-of-the-art approaches may fail to restore details whose colors resemble the background and may introduce artificial halos; this problem is harder in sRGB than raw Bayer space.

    Source: §4.3 · p. 8

Abstract

from arXiv · show

Real-world low-light images suffer from two main degradations, namely, inevitable noise and poor visibility. Since the noise exhibits different levels, its estimation has been implemented in recent works when enhancing low-light images from raw Bayer space. When it comes to sRGB color space, the noise estimation becomes more complicated due to the effect of the image processing pipeline. Nevertheless, most existing enhancing algorithms in sRGB space only focus on the low visibility problem or suppress the noise under a hypothetical noise level, leading them impractical due to the lack of robustness. To address this issue,we propose an adaptive unfolding total variation network (UTVNet), which approximates the noise level from the real sRGB low-light image by learning the balancing parameter in the model-based denoising method with total variation regularization. Meanwhile, we learn the noise level map by unrolling the corresponding minimization process for providing the inferences of smoothness and fidelity constraints. Guided by the noise level map, our UTVNet can recover finer details and is more capable to suppress noise in real captured low-light scenes. Extensive experiments on real-world low-light images clearly demonstrate the superior performance of UTVNet over state-of-the-art methods.

1. Introduction

Real-world low-light restoration must address both poor visibility and noise, whose levels vary and are harder to estimate in sRGB after image processing. UTVNet learns a noise-level map through an unfolded total-variation process to guide restoration.

1. Introduction

  • Existing exposure-enhancement methods can amplify noise, while denoisers designed for normal luminance may miss noise in extremely dark image details.The paper illustrates these problems in Fig. 1 (b–d).Source: §1 · p. 1
  • Noise levels vary in real low-light images, and the image-processing pipeline complicates noise estimation in sRGB compared with raw Bayer data.The paper notes that synthetic noise-map training may be less effective because of inaccurate noise models and the gap between synthetic images and real photographs.Source: §1 · p. 2
  • The method generates a noise-free smooth layer for luminance correction and uses the estimated noise map to guide suppression in the detail layer.This separates luminance adjustment from noise suppression in the proposed restoration process.Source: §1 · p. 2
  • UTVNet uses a learnable, per-pixel noise-level map as the balancing parameter in adaptive TV regularization, controlling noise reduction versus detail preservation.Unfolding the TV minimization provides fidelity and smoothness inferences for learning the map without designing a special loss function.Source: §1 · p. 2
  • UTVNet is reported as robust on real captured low-light images with various noise levels.The method approximates a noise-level map from real sRGB images using adaptive TV regularization and an unfolded minimization process.Source: §1 · p. 2

2. Related Work

Prior low-light enhancement and denoising methods address different parts of restoration, while noise estimation in sRGB remains difficult after the imaging pipeline. UTVNet combines adaptive TV regularization with an unfolded optimization process to learn noise-level information from real low-light images.

2. Related Work

  • Low-light enhancement methods often target underexposure without noise suppression, while methods that suppress noise without estimating it may lack robustness in real captured scenes.The paper also notes that sRGB noise is affected by the ISP, and synthetic-data noise-map learning is limited by the gap between synthetic and real photographs.Source: §2 · p. 2
  • Real-image denoising methods estimate noise or separate signal and noise, while UTVNet seeks to learn noise-level approximation directly from real low-light images.Prior approaches described include cross-channel noise modeling, spatially varying noise maps, and synthetic-image training.Source: §2 · p. 2
  • Model-based denoising balances data fidelity and regularization, but its balancing parameter is often empirical; deep unrolling makes iterative optimization learnable.Unlike cited unrolling methods that fit physical priors with CNNs, UTVNet uses TV regularization and unfolds minimization for fidelity and smoothness inferences.Source: §2 · pp. 2–3
  • UTVNet has an unfolding TV module that estimates a noise-level map and smooth layer, followed by restoration that corrects luminance and suppresses detail-layer noise.The estimated map guides noise suppression in the detail layer.Source: Figure 2 · p. 3

3. Methodology

UTVNet estimates spatially varying noise by learning balancing parameters in an unfolded total-variation denoising process. Its restoration module uses the resulting noise map alongside separate smooth and detail layers for luminance correction and noise suppression.

3.1. Noise Level vs. Balancing Parameter

  • TV regularization makes the balancing parameter learnable for noise-level approximation, while providing sharp-edge preservation and closed-form solutions with fidelity and smoothness constraints.The method links the noise level and balancing parameter through their shared control of the trade-off between noise reduction and detail preservation.Source: §3.1 · p. 3

3.2. Adaptive Total Variation Regularization

  • The adaptive TV objective stretches the balancing parameter into a spatial map, with Mx = My = M serving as the per-pixel noise-level map.Introducing u = Dx enables the minimization to be solved by iteratively updating x, u, and z through ADMM.Source: §3.2 · pp. 3–4

3.3. Unfolding Total Variation Module

  • The unfolding TV module predicts a noise map from the input, then iteratively applies the TV minimization to learn M and generate a noise-free smooth layer.The architecture uses iteration-specific M and ρ parameters, with K set to 8.Source: §3.3 · p. 4
  • The unfolding functions enforce fidelity between the input and smooth layer and use M to smooth details and noise, learning a per-iteration trade-off between suppression and detail preservation.Function N handles the fidelity subproblem, while S applies the M-guided smoothness constraint.Source: §3.3 · p. 4
  • The NLI-block initializes the noise map coarse-to-fine, modifying global noise variation to provide a better initialization and a basic magnitude for learning positive per-pixel values.The activation A(·) sets negative values to the global noise variation ε rather than zero.Source: §3.3 · pp. 4–5

3.4. Low-light Image Restoration Module

  • The restoration module feeds the smooth layer to luminance correction and uses the detail layer yd = y − ys together with M for noise suppression and detail restoration.The luminance sub-network generates per-pixel amplification; the noise-suppression sub-network takes both yd and M as input.Source: §3.4 · p. 5

3.5. End-to-End Training

  • The network is trained end-to-end by minimizing a weighted combination of L2 loss and perceptual loss, with weights 1 and 0.12, respectively.Source: §3.5 · p. 5

4. Experiment

On sRGB-SID, UTVNet achieves the best performance across four reported metrics and preserves finer details; on ELD, it performs competitively under varied capture conditions. The analysis examines computational cost, noise-map behavior, ablations, and a limitation shared with other methods.

4.1. Results on sRGB-SID dataset

  • UTVNet achieves the best performance across PSNR, SSIM, mean L2 error in L*a*b* space, and LPIPS, while recovering finer details and colors than other methods.The visual comparison also reports that LIME may fail to recover noisy details, SID and Pixel2Pixel introduce artifacts, and D&E and MIRNet can be over-smooth or remain noisy.Source: Table 1 · p. 6, §4.1 · p. 6, Figure 4 · p. 6

4. Experiment

  • On 1,196 sRGB-SID test image pairs, UTVNet is compared with 11 recent methods; training uses 4,198 image pairs, and the implementation uses 8 iterations.Images are resized to a 512-pixel long edge during training.Source: §4 · pp. 5–6, §4.1 · p. 6

4.1. Results on sRGB-SID dataset

  • Compared with D&E and MIRNet, UTVNet achieves the best performance with balanced parameter size and lower MAC operations.The computational analysis uses 1024 × 768-pixel sRGB-SID images.Source: §4.1 · p. 6, Table 3 · p. 8

4.2. Performance in Real Captured Scenes

  • On ELD, UTVNet obtains competitive PSNR, SSIM, and LPIPS scores when tested on images captured across different ISO levels and exposure times.The evaluation uses sRGB images from 10 indoor scenes captured with four camera devices.Source: §4.2 · p. 7, Table 2 · p. 7
  • At ISO 3200 and 1/800s, UTVNet produces clearer results than MIRNet and D&E; across exposure times, it introduces fewer artifacts than other state-of-the-art methods.The comparisons visualize Mavg, the average of UTVNet’s noise-level maps across iterations.Source: §4.2 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7

4.3. Analysis and Discussions

  • Ablations show performance drops when global noise variance ε is removed or A(·) is replaced with ReLU, supporting their roles in estimating the noise-level map.The authors state that modifying global noise variance better estimates pixel-level noise differences in extremely low-light images.Source: §4.3 · pp. 7–8
  • In most iterations, estimated noise levels are lower at sharp edges and object boundaries, so fidelity dominates there; the authors adopt 8 iterations for a speed–accuracy trade-off.The paper links these maps to detail preservation after noise suppression.Source: §4.3 · p. 8, Figure 7 · p. 8, Table 4 · p. 8
  • UTVNet and other state-of-the-art methods may fail to restore details similar in color to the background and may introduce artificial halos, a harder problem in sRGB than raw Bayer space.The failure case is shown in Figure 8.Source: §4.3 · p. 8, Figure 8 · p. 8

5. Conclusion

UTVNet enhances real-world noisy low-light images in sRGB by learning a per-pixel noise level map within an adaptive total variation framework. The authors report extensive experimental validation and identify image quality in real captured scenes as an area for future improvement.

5. Conclusion

  • UTVNet uses a learnable noise level map as per-pixel balancing parameters to trade off noise reduction and detail preservation.The map is motivated by the relationship between noise level and the balancing parameter in model-based denoising.Source: §5 · p. 8
  • The network unfolds TV minimization with fidelity and smoothness constraints to learn the map end to end and generate a noise-free smooth layer.A noise level initialization block first predicts maps in a coarse-to-fine way.Source: §5 · p. 8
  • The authors report extensive experimental validation and expect future work to improve image quality, especially in real captured low-light scenes.Source: §5 · p. 8
Loading 2110.00984v4…