PaperlayerView exact revision on arXiv

Source-linked AI summary

DarkIR: Robust Low-Light Image Restoration

Daniel Feijoo, Juan C. Benito, Alvaro Garcia, Marcos V. Conde

arXiv:2412.13443v3cs.CVeess.IV

TL;DR

Night and dark photography combines elevated noise, low illumination, and long-exposure blur, while most restoration methods address these tasks separately. DarkIR uses a CNN with spatial and frequency attention for multi-task low-light restoration, achieving state-of-the-art results across named benchmarks and generalizing to real-world night images.

  • Problem

    Low-light images can combine elevated noise, dimness, and long-exposure blur, but most restoration methods solve enhancement and deblurring separately.

    Source: §1 · pp. 1–2

  • Method

    DarkIR is a CNN that combines spatial large-receptive-field attention for noise and blur with frequency-domain processing for low-light enhancement.

    Source: §1 · p. 2

  • Results

    DarkIR achieves state-of-the-art results on LOLBlur, LOLv2, and Real-LOLBlur, generalizes to real-world night blurry images, and improves PSNR by +1db over LEDNet on LOLBlur.

    Source: §4.2 · p. 5, §6 · p. 8

  • Takeaways & Limitations

    DarkIR is presented as an efficient new baseline for multi-task night/dark image enhancement and restoration.

    Source: §1 · p. 2, §6 · p. 8

    Depth-wise convolutions are not necessarily optimal on some GPU architectures, so inference-time reductions do not scale proportionally with the lower operation count.

    Source: §5 · p. 8

Abstract

from arXiv · show

Photography during night or in dark conditions typically suffers from noise, low light and blurring issues due to the dim environment and the common use of long exposure. Although Deblurring and Low-light Image Enhancement (LLIE) are related under these conditions, most approaches in image restoration solve these tasks separately. In this paper, we present an efficient and robust neural network for multi-task low-light image restoration. Instead of following the current tendency of Transformer-based models, we propose new attention mechanisms to enhance the receptive field of efficient CNNs. Our method reduces the computational costs in terms of parameters and MAC operations compared to previous methods. Our model, DarkIR, achieves new state-of-the-art results on the popular LOLBlur, LOLv2 and Real-LOLBlur datasets, being able to generalize on real-world night and dark images. Code and models at https://github.com/cidautai/DarkIR

1. Introduction

Night photography combines elevated noise with long-exposure blur, while restoration tasks are usually handled separately. DarkIR addresses this gap with spatial- and frequency-domain CNN attention and reports state-of-the-art results on LOLBlur and Real-LOLBlur.

1. Introduction

  • The image-formation model represents the observed dim image as a scene blurred by the lens PSF, adjusted for dynamic range and saturation, and corrupted by additive sensor noise.In the notation, y is the observed dim image, x the unperturbed scene, k the PSF kernel, and n the sensor noise.Source: §1 · p. 1
  • At low light, shot and read noise rise, while long exposures can cause ghosting and blur, especially with smartphones' fixed apertures and limited optics.The paper frames joint enhancement and deblurring as important for mobile computational photography [12].Source: §1 · pp. 1–2
  • Most methods solve low-light enhancement and deblurring individually, and few works address denoising, deblurring, and LLIE jointly end to end.The authors identify the need to fine-tune, store, and apply multiple task-specific models in sequence as a limitation for real-world use.Source: §1 · p. 2
  • DarkIR combines spatial large-receptive-field attention for noise and non-uniform blur with Fourier-domain processing for low-light enhancement.The proposed CNN operates in both spatial and frequency domains.Source: §1 · p. 2
  • DarkIR achieves state-of-the-art results on LOLBlur and Real-LOLBlur, improving PSNR by +1dB over LEDNet with less computational cost.The paper presents DarkIR as a new baseline for multi-task night/dark image enhancement.Source: §1 · p. 2

2. Related Work

Related work covers deblurring and low-light enhancement largely as separate tasks, with relatively few approaches addressing both together. The combined-task literature includes NBDN and LEDNet, which also introduced the LOLBlur and Real-LOLBlur datasets.

Image Deblurring

  • Deblurring methods divide into non-blind approaches that use the blur kernel and blind approaches without prior kernel knowledge; recent work includes CNNs, GANs, frequency transformers, iterative methods, and diffusion models.Source: §2 · p. 2

Low-Light Image Enhancement (LLIE)

  • Low-light enhancement has moved from image-statistics and Retinex priors toward CNNs, transformers, and Fourier-frequency methods.Examples include RetinexNet, ZeroDCE, SCI, RetinexFormer, and FourLLIE.Source: §2 · p. 2

Low-Light Blur Enhancement

  • Joint low-light blur enhancement remains uncommon; NBDN addresses night-saturated images, while LEDNet targets blurry low-light images and introduced LOLBlur and Real-LOLBlur.Source: §2 · pp. 2–3

3. Method

DarkIR assigns low-light enhancement to a Fourier-informed encoder and deblurring to a spatial-attention decoder. Task-specific blocks and a combined reconstruction loss support this division while reducing computational cost.

3. Method

  • DarkIR adapts Metaformer blocks for separate low-light enhancement and deblurring roles, a design intended to reduce parameters and computational cost.The encoder works in the Fourier domain, while the decoder uses large-receptive-field spatial processing.Source: §3 · p. 3

3.1. Low-light Enhancement Encoder

  • The encoder enhances low-light features using FFT amplitude processing and estimates a clean image at one-eighth resolution for the decoder.Its FreMLP operates on Fourier amplitude, while the encoder's spatial attention uses gating; downsampling also permits more blocks at deeper levels without notably increasing operations.Source: §3.1 · p. 3, §3 · p. 3

3.2. Deblurring Decoder

  • The decoder upsamples the encoder's low-resolution estimate and targets blur and sharpness with Di-SpAM's multiscale dilated spatial attention.Di-SpAM combines three depth-wise convolution branches with dilation factors 1, 4, and 9, followed by simplified channel attention and a gated MLP.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3. Loss Function

  • Training combines pixel L1 fidelity, VGG19-based LPIPS perceptual similarity, edge consistency, and a low-resolution encoder-guiding loss.The guiding loss compares the encoder estimate with an 8× downsampled reference.Source: §3.3 · p. 4

4. Experimental Results

DarkIR is evaluated on synthetic and real low-light restoration datasets, with LOLBlur results showing stronger distortion and perceptual performance than prior methods. The experiments also examine all-in-one training, real-image quality, architectural ablations, and computational efficiency.

4.1. Datasets

  • LOLBlur contains 10,200 synthetic training pairs and 1,800 test pairs; Real-LOLBlur adds 482 real night-blurry images without ground truth.The experiments also use paired LOLv2 real and synthetic data and LSRW images from Nikon and Huawei cameras.Source: §4.1 · p. 5

4.2. Results

  • DarkIR-mt outperforms previous LLIE methods while handling blur, with 26.62 dB on LOLBlur, a slight 0.4 dB loss.The multi-task model combines LOLBlur, LOLv2, and LSRW training data.Source: §4.2 · p. 5, Table 3 · p. 7
  • +1 dB PSNR over LEDNet and a halving of LPIPS are reported on LOLBlur, alongside better results than most general-purpose methods at lower cost.Table 1 reports the quantitative LOLBlur evaluation; its caption identifies distortion and perceptual metrics and parameter savings.Source: §4.2 · p. 5, Table 1 · p. 5

4.3. Evaluation on Real Data

  • On real night-blurry images, DarkIR is reported robust to handheld motion blur, sensor noise, saturated pixels, and low illumination; its three perceptual scores are competitive.Real-LOLBlur test images lack ground truth, so evaluation uses blind quality metrics MUSIQ, NRQM, and NIQE.Source: §4.3 · p. 6

4.4. Ablation Study

  • EBlock/DBlock configurations perform best among the tested block variants, and Di-SpAM outperforms LKA with fewer parameters and operations.The block ablations use 256-pixel crops rather than the final model’s 384-pixel crops.Source: §4.4 · p. 6, Table 6 · p. 7, Table 7 · p. 7
  • The 32-channel DarkIR-m embedding offers the best reported efficiency/performance balance; performance rises as channel embedding increases from 16 to 64.The channel-depth study describes scaling that permits adaptation to memory or runtime requirements.Source: §4.4 · p. 7, Table 8 · p. 8

4.5. Efficiency Discussion

  • At 256px, DarkIR uses 7.25 GMACs, versus 33.74 for LEDNet and 141.24 for Restormer, with 55% and 88% fewer parameters, respectively.The paper describes these reductions as an advance toward deployment on devices with low computational power.Source: §4.5 · p. 8

5. Limitations

Depth-wise convolutions reduce computational requirements, but may not yield proportionally faster inference on some GPU architectures.

5. Limitations

  • Depth-wise convolutions can lack arithmetic intensity on some GPU architectures, so reduced operations do not translate proportionally into faster inference.The authors identify faster inference with low computational requirements as future work.Source: §5 · p. 8

6. Conclusion

DarkIR is a multi-task model for denoising, deblurring, and low-light enhancement in dark and night scenes, with reported state-of-the-art results across three benchmarks.

6. Conclusion

  • DarkIR achieves state-of-the-art results on LOLBlur, LOLv2, and Real-LOLBlur while generalizing to real-world night blurry images.The authors also report greater efficiency than previous methods.Source: §6 · p. 8
  • DarkIR performs denoising, deblurring, and low-light enhancement on dark and night scenes.Source: §6 · p. 8
  • The authors describe DarkIR as an efficient and robust neural network for multi-task low-light enhancement and restoration.Source: §6 · p. 8

1. Additional Implementation Details

DarkIR is trained on LOLBlur with specified augmentation and optimization settings; the multi-task model adds LOLv2 and LSRW and maintains LOLBlur performance.

1. Additional Implementation Details

  • DarkIR is trained on LOLBlur using random 384 × 384 crops, flip and rotation augmentations, minibatches of 32, and an H100 GPU.Source: §1 · p. 9
  • Training uses AdamW with β1 = 0.9, β2 = 0.9, weight decay 1e−3, and cosine annealing from 5e−4 to 1e−6.The same configuration is used to retrain other LOLBlur methods.Source: §1 · p. 9
  • The multi-task model achieves essentially state-of-the-art results on real low-light enhancement benchmarks while maintaining performance on LOLBlur.It uses LOLv2 and LSRW as additional datasets, with the same training setup.Source: §1 · p. 9

2. Additional ablation Studies

The ablations identify simple addition as the best encoder–decoder feature propagation choice and find that combining the three added losses gives the best distortion–perception balance.

2. Additional ablation Studies

  • Adding Ledge, Llol, and Lpercep gives the best combination of distortion and perceptual metrics in the loss ablation.Source: §2 · p. 9
  • The feature-propagation ablation compares LEDNet's CurveNLU, depthwise convolutions, and a pointwise-only block resembling a look-up table.The pointwise-only variation is the one incorporated for evaluation.Source: §2 · p. 9
  • Figure A's two proposed feature-propagation blocks differ by whether 1DLUT-double includes channel expansion.Source: §2 · p. 9
  • Simple addition is the optimal encoder–decoder feature-propagation choice in the ablation.The authors therefore do not add the tested feature blocks to DarkIR.Source: §2 · p. 9

3. More quantitative results in unpaired data

The supplementary quantitative results evaluate DarkIR on five unpaired LLIE datasets using BRISQUE and NIQE. Table C identifies the metrics and says its reference results come from [61].

3. More quantitative results in unpaired data

  • The multi-task model trained on LOLBlur and LOL datasets is evaluated on five unpaired LLIE datasets using BRISQUE and NIQE.The datasets are LIME, DICM, MEF, NPE, and VV.Source: §3 · p. 9
  • Table C presents a quantitative comparison across five unpaired LLIE datasets using BRISQUE and NIQE.The caption says the reference results are from [61]; the supplied passage does not include table values.Source: Table C · p. 10
  • Supplementary LOLv2-Real qualitative results compare DarkIR-m with SNR-Net and RetinexFormer.The supplied passages identify the comparison but do not state a quantitative outcome.Source: §3 · p. 10

4. More qualitative results in LLIE

The supplementary figures show qualitative results for the multi-task model on LLIE and LLIE-deblurring tasks. They include synthetic and real-world low-light datasets, with sharper and brighter results reported for LOLBlur.

4. More qualitative results in LLIE

  • Figures B–D show LLIE results on LOLv2-Real, LOLv2-Synthetic, and the real-world LSRW-Huawei and LSRW-Nikon datasets.The LOLv2 comparisons include SNR-Net and RetinexFormer.Source: §4 · p. 10, Figure C · p. 11, Figure D · p. 11
  • Figures E–G present LLIE-deblurring comparisons on RealBlur-Night, Real-LOLBlur, and LOLBlur.The figure passages identify the datasets; only the LOLBlur caption states a comparative outcome.Source: §4 · pp. 10, 12, 13, 14
  • On LOLBlur, DarkIR produces sharper and brighter results than the other methods.Figure G presents qualitative comparisons on LOLBlur.Source: §4 · p. 14

Related papers

Loading 2412.13443v3…