PaperlayerView exact revision on arXiv

Source-linked AI summary

MDIRNET: Multi-Degradation Image Restoration Network via Deep Unfolding

Talha Nadeem, Arslan Majal, Muhammad Tahir, Khurram Ali

arXiv:2610.01655v1eess.IV

TL;DR

Restoration is difficult when images contain unknown or mixed degradations, while many methods target a single known corruption or rely on task-specific designs. MDIRNET unfolds an OVPCA-based low-rank prior and adds adaptive patch and rank processing with attention refinement. It reports competitive performance across denoising, deraining, and deblurring, with real-world mixed-degradation validation remaining future work.

  • Problem

    Many restoration methods target a single known degradation, while unknown or compound distortions motivate a unified approach.

    Source: §I · p. 1

  • Method

    MDIRNET unfolds OVPCA into a learnable low-rank network and combines it with adaptive patch partitioning, dynamic rank allocation, and supervised attention.

    Source: §I · pp. 1–2, 1) Patch Partitioning Module (PPM) · p. 4, 2) Dynamic Rank Allocation Module (DRAM) · p. 4

  • Results

    MDIRNET achieves competitive performance across denoising, deraining, and deblurring while providing an efficiency–accuracy trade-off within a unified framework.

    Source: §VI · p. 11

  • Takeaways & Limitations

    The paper concludes that MDIRNET offers a unified framework for multi-degradation restoration with a strong efficiency–accuracy trade-off.

    Source: §VI · p. 11

    Validation on real-world mixed-degradation datasets remains future work.

    Source: §VI · p. 12

Abstract

from arXiv · show

Real images often exhibit unknown and mixed degradations, making restoration substantially more challenging than single-task image restoration because multiple distortion types interact within the same observation. Consequently, existing methods often rely on prior knowledge of the degradation type or separate task-specific models, which may oversmooth fine structures or leave residual artifacts, motivating a compact model-driven alternative. We propose the Multi-Degradation Image Restoration Network (MDIRNET), a unified framework that combines a model-driven low-rank prior with end-to-end learning. Here, unified refers to joint training on three degradation types: noise, rain, and blur. A single MDIRNET model restores all three without requiring task-specific models, modules, or branches at inference. The low-rank prior exploits the redundancy and compact structure of natural image patches. To identify this underlying low-dimensional representation, we formalize restoration via Orthogonal Variational PCA (OVPCA) and translate its iterative inference into a deep unfolding network. To handle spatially non-uniform corruption and local content variability, we further introduce a learnable patch-partitioning strategy and a lightweight dynamic rank-allocation module that predicts the appropriate subspace dimension for each region. Spatially adaptive reconstruction refinement is performed using a supervised attention module. Extensive experiments on standard denoising, deblurring, and deraining benchmarks show that MDIRNET achieves competitive or superior performance over strong baselines across most metrics, while controlled mixed-degradation experiments demonstrate consistent performance across the evaluated synthetic degradation combinations. The code is available at https://github.com/ScholarForge/mdirnet.git.

I. INTRODUCTION

MDIRNET targets restoration of unknown or compound degradations, which single-task methods and some existing multi-degradation approaches handle with limited flexibility. It unfolds an OVPCA low-rank prior and adds adaptive patching, rank allocation, and attention-based refinement; experiments report competitive or superior results across denoising, deraining, and deblurring benchmarks.

I. INTRODUCTION

  • Existing restoration methods often target one known degradation, while some multi-degradation methods use separate branches or require known corruption types.The paper motivates a unified model for unknown or compound distortions.Source: §I · p. 1
  • DU-OVPCA unfolds OVPCA optimization into learnable layers, providing a compact model-driven low-rank backbone for MDIRNET.The architecture jointly designs adaptive components around this core.Source: §I · p. 1
  • PPM adapts patch sampling to local image characteristics, while DRAM predicts the retained rank for each patch group.Together they support content-adaptive patch processing and subspace allocation.Source: §I · pp. 1–2
  • SAM compares restored-image features with the input and uses an attention map to blend them for targeted reconstruction refinement.The refinement emphasizes regions estimated to need further correction.Source: §I · p. 2
  • MDIRNET achieves competitive or superior quality across denoising, deraining, and deblurring benchmarks with 0.68 M parameters.The paper also reports ablation studies of its components.Source: §I · p. 2

A. Image Denoising

  • Classical denoising uses collaborative filtering, low-rank or multi-scale priors, while deep methods include residual learning, noise-level conditioning, and attention.The review reports deep methods outperforming classical ones, especially for complex or real-world non-Gaussian noise.Source: A. Image Denoising · p. 2

B. Image Deblurring

  • Deblurring approaches include explicit blur priors, known-kernel filtering, multi-scale CNNs, GANs, patch processing, recurrence, and transformers.The reviewed classical blind methods struggle with spatially varying or large-kernel blur, while some non-blind methods assume a known kernel.Source: B. Image Deblurring · p. 2

C. Image Deraining

The section reviews deraining and unified restoration before developing MDIRNET’s OVPCA-based low-rank backbone. OVPCA’s variational updates motivate a deep-unfolded network, while the low-rank rationale is conditioned on bounded perturbations rather than presented as a recovery guarantee.

C. Image Deraining

  • Deraining methods progress from sparsity, dictionary-learning, and low-rank formulations to deep CNN and semi-supervised approaches.Non-local patch grouping can struggle with spatially varied heavy rain; UMRL and SIRR are described as improving generalization to real rain conditions.Source: C. Image Deraining · p. 2

D. Unified Image Restoration Frameworks

  • Unified restoration remains challenging because of computational complexity, spatially varying degradations, and the need for extensive mixed-degradation training.MDIRNET combines OVPCA-based unfolding with adaptive patch partitioning, dynamic rank allocation, and supervised attention.Source: D. Unified Image Restoration Frameworks · p. 2

A. Preliminaries: OVPCA and DU-OVPCA

  • The framework assumes clean image patches lie near a low-rank subspace, treating this as a patch-level statistical prior rather than a universal physical law.The paper contrasts PCA’s rigid linear subspace assumption with OVPCA’s probabilistic treatment of the clean image as a latent variable.Source: A. Preliminaries: OVPCA and DU-OVPCA · p. 3
  • Variational Bayes approximates the intractable posterior with a factored distribution and coordinate-ascent updates that maximize the ELBO.The updates alternately update A, L, X, and ω.Source: A. Preliminaries: OVPCA and DU-OVPCA · p. 3
  • Under ϵ < σr(X), the rth singular value remains nonzero and singular values beyond rank r are bounded by ϵ, motivating low-rank regularization.The observation does not guarantee clean-image recovery; outside the bounded-perturbation regime, the prior is an empirical inductive bias.Source: A. Preliminaries: OVPCA and DU-OVPCA · p. 3
  • DU-OVPCA maps OVPCA iterations to jointly trained network layers, with empirical performance saturating at K = 6 and negligible additional improvement at K = 8.ELBO monotonicity motivates the update sequence, but its theoretical guarantee does not directly extend to the end-to-end trained unfolded network.Source: A. Preliminaries: OVPCA and DU-OVPCA · p. 4

1) Patch Partitioning Module (PPM):

PPM learns image-conditioned patch sampling offsets and uses them to extract shifted patches, providing spatially adaptive patch processing.

1) Patch Partitioning Module (PPM):

  • PPM takes the degraded image as input and predicts a dense flow field interpreted as offsets to a regular patch-sampling grid.The offsets deform the sampling coordinates before patches are extracted.Source: 1) Patch Partitioning Module (PPM) · p. 4
  • Differentiable sampling extracts patches at shifted coordinates, and end-to-end training learns where to redistribute patch centers.Converging offsets can increase effective sampling density in a region, but this behavior is not guaranteed by the architecture.Source: 1) Patch Partitioning Module (PPM) · p. 4
  • PPM uses overlapping 8 × 8 patches and groups each reference patch with its spatially nearest neighbors, avoiding the quadratic complexity of explicit non-local matching.Adaptive multi-scale patch partitioning is left for future work.Source: 1) Patch Partitioning Module (PPM) · p. 4

2) Dynamic Rank Allocation Module (DRAM):

DRAM allocates rank by patch-group content because a single fixed rank is suboptimal for smooth and textured regions. It derives a group-complexity feature from the mean patch and predicts a rank used by DU-OVPCA.

2) Dynamic Rank Allocation Module (DRAM):

  • DRAM predicts an adaptive rank for each patch group, assigning capacity to local image content rather than using one fixed rank.A fixed rank is suboptimal because smooth regions need low rank while textured regions need high rank.Source: 2) Dynamic Rank Allocation Module (DRAM) · p. 4
  • The mean patch summarizes patch-group complexity and feeds a lightweight CNN feature extractor before rank prediction.The feature vector goes to a regression head that outputs a scalar subsequently scaled and quantized to an integer rank.Source: 2) Dynamic Rank Allocation Module (DRAM) · pp. 4–5
  • DRAM learns ranks from the restoration objective without explicit rank, FLOP, or latency penalties, so it is not directly optimizing computational efficiency.A Straight-Through Estimator approximates rounding as the identity during backpropagation, enabling end-to-end training.Source: 2) Dynamic Rank Allocation Module (DRAM) · p. 5

3) Low-Rank Restoration via DU-OVPCA:

DU-OVPCA restores patch groups through a low-rank decomposition with a rank predicted for each group, then averages overlapping restored patches into an initial image. SAM refines that image by spatially gating information from the original observation.

3) Low-Rank Restoration via DU-OVPCA:

  • DU-OVPCA decomposes each patch-group data matrix using the group-specific predicted rank, then estimates the factors through K unfolding layers.The resulting low-rank reconstruction suppresses noise and outlier features that cannot be represented in the rank-r_i subspace.Source: 3) Low-Rank Restoration via DU-OVPCA · p. 5
  • Sharing basis A_i across grouped patches promotes consistency and non-local restoration by sharing information among similar patches.Restored overlapping patches are merged by averaging their values at each pixel to form the initial restored image.Source: 3) Low-Rank Restoration via DU-OVPCA · p. 5

4) Supervised Attention Module (SAM):

B. Loss Function and Training

Training jointly optimizes MDIRNET with a hybrid reconstruction loss and a patch-consistency regularizer. The regularizer penalizes variation among restored patch features within spatial groups.

B. Loss Function and Training

  • The hybrid L1+L2 reconstruction loss uses L1 to preserve edges and L2 to stabilize training.The L2 term is kept small relative to L1 to reduce oversmoothing.Source: B. Loss Function and Training · p. 5
  • A patch-consistency regularizer penalizes intra-group variance of restored patch features to encourage consistency across overlapping patches.The regularizer controls intra-group variation, while λ_unc limits excessive suppression of legitimate differences.Source: B. Loss Function and Training · p. 5
  • The total training loss is a weighted sum of reconstruction and consistency terms.λ_rec and λ_unc balance the two terms.Source: B. Loss Function and Training · p. 5

IV. EXPERIMENTS AND RESULTS

Experiments cover standard denoising, deraining, and deblurring benchmarks and compare MDIRNET with classical, CNN-based, and unified-restoration baselines. Across the three single-degradation tasks, the reported results are competitive or superior while inference time remains competitive.

A. Implementation Details

  • The implementation uses K=6 unfolding layers and remains below 1 M parameters; inference time is measured as the median of 100 forward passes after 10 warm-ups at batch size 1.Source: A. Implementation Details · p. 6

B. Comparison for Single Degradation Scenarios

  • 31.82 dB / 0.898 SSIM is MDIRNET’s average denoising result, comparable to Perceive-IR at 31.70 dB, with 0.19 s inference time versus 0.23–0.33 s.The cited denoising results cover BSD68, Urban100, and DIV2K at σ=25 and 50.Source: B. Comparison for Single Degradation… · p. 6
  • 36.53/0.974 is MDIRNET’s Rain100L result, compared with DeAirRTUR at 38.04/0.983; inference times are 0.23 s and 0.33–0.44 s, respectively.DeAirRTUR uses AirNet retrained with Task Uncertainty Regularization, and the 1.63 dB difference is not isolated to specific causes.Source: B. Comparison for Single Degradation… · p. 6
  • 32.68 dB / 0.959 SSIM on GoPro matches MPRNet, with 0.23 s inference time versus 0.24 s.The experiments use standard benchmarks including BSD68, BSD400, WED, DIV2K, GoPro, and Rain100L.Source: §IV · p. 5, B. Comparison for Single Degradation… · p. 6
  • MDIRNET achieves competitive or superior performance across three representative restoration tasks while maintaining competitive inference time and a favorable efficiency–accuracy trade-off.The reported single-degradation comparisons cover denoising, deraining, and deblurring.Source: B. Comparison for Single Degradation… · p. 6

C. Evaluation on Real-World Sensor Noise (SIDD)

MDIRNET is evaluated on real sensor noise and denoising benchmarks, then on single and mixed degradation scenarios. The reported results pair competitive restoration quality with compact size and evaluate performance under specified synthetic degradation combinations.

C. Evaluation on Real-World Sensor Noise (SIDD)

  • On real SIDD sensor noise, MDIRNET achieves 39.91 dB / 0.960 SSIM, with qualitative results describing preserved color fidelity and fine structure.Table III reports denoising results across BSD68, Urban100, DIV2K, and SIDD; Figure 4 includes real sensor noise examples.Source: C. Evaluation on Real-World Sensor… · p. 6, Table III · p. 7
  • For image denoising, Table III reports results on BSD68, Urban100, DIV2K, and SIDD [71], with PSNR, SSIM, average inference time, and average PSNR/SSIM measures.Source: Table III · p. 7
  • After 25-epoch DIV2K fine-tuning, MDIRNET reports Q10/Q20 results of 29.51/0.816 and 31.72/0.883 on Classic5, and 29.35/0.812 and 31.66/0.881 on LIVE1.The model has 0.68M parameters and is described as competitive with AirNet (8.93M).Source: C. Evaluation on Real-World Sensor… · p. 7

D. Quantitative Comparison for Multi-Degradation Scenarios

  • In mixed-degradation comparisons, MDIRNET has the highest average PSNR among four methods and uses 13–38× fewer parameters than AirNet, MPRNet, and Restormer.The comparison uses the evaluated settings and inference protocols; Figure 3 plots average PSNR against model size, with bubble area denoting GFLOPs.Source: D. Quantitative Comparison for… · p. 8, Figure 3 · p. 8, Table VII · p. 8
  • Across evaluated degradation combinations, MDIRNET records 28–36.5 dB PSNR; with all degradations, it reaches 31.56 dB / 0.913 on BSD68 and 35.21 dB / 0.958 on Rain100L.The synthetic combinations use fixed degradation orders shared between training and evaluation.Source: D. Quantitative Comparison for… · pp. 7–8, Table VII · p. 8
  • In one-by-one comparisons across BSD68, Rain100L, and GoPro, MDIRNET is reported with 0.19–0.23 s runtime, 0.68M parameters, and 14.0 GFLOPs, below the cited AirNet and Restormer sizes and FLOPs.The reported literature comparisons retain the provenance and inference settings summarized in Table II.Source: Table II · p. 7, D. Quantitative Comparison for… · p. 8

E. Qualitative Results on Multiple Degradations

The qualitative comparisons describe MDIRNET’s outputs across single and multiple degradations, while a controlled backbone comparison evaluates the contribution of DU-OVPCA. The reported visual comparisons emphasize detail and structural preservation, and the backbone comparison reports gains over alternatives.

E. Qualitative Results on Multiple Degradations

  • Across noise, rain, and blur examples, MDIRNET is described as producing cleaner restorations while preserving structural details under evaluated benchmark conditions.Figure 7 presents the multi-degradation comparison.Source: E. Qualitative Results on Multiple… · p. 8, Figure 7 · p. 9
  • For noise, rain, and blur individually, the comparisons describe sharper denoising, rain removal with preserved boundaries and textures, and edge restoration with minimal distortion.The noise comparison covers synthetic Gaussian noise and real SIDD sensor noise; the blur comparison contrasts ringing or oversharpening in DeblurGAN and MPRNet.Source: Figure 4 · p. 9, Figure 5 · p. 9, Figure 6 · p. 9

F. Comparison with Alternative Deep-Unfolded Backbones

  • Replacing DU-OVPCA with a parameter-matched CNN reduces PSNR by 1.44 dB on BSD68, 2.34 dB on Rain100L, and 1.44 dB on GoPro; DRAM adds further gains.The comparison fixes PPM and SAM and disables DRAM for the backbone test; DRAM gains are +0.55, +1.78, and +1.47 dB on BSD68, Rain100L, and GoPro, respectively.Source: F. Comparison with Alternative… · pp. 8–9

V. ABLATION STUDY

The ablations test module contributions, sampling and aggregation choices, rank and unfolding depth, and consistency-regularizer weight. Reported results support learned patch partitioning and identify settings balancing restoration performance and computational cost.

V. ABLATION STUDY

  • Mean pooling achieves the highest accuracy and lowest measured inference time among DRAM aggregation strategies, so MDIRNET adopts it.The module ablation also reports that adaptive rank selection improves restoration quality and reduces measured runtime without explicit runtime optimization during training.Source: §V · pp. 9, 11
  • Learned PPM reaches 33.68 dB average PSNR versus 32.68 dB for gradient-based sampling and varies by less than 0.13 dB across five seeds.The reported comparison is evidence for learned spatially adaptive sampling under the evaluated settings.Source: §V · p. 9, Figure 9 · p. 10
  • PPM ablation reports PSNR drops of 2.19 dB on Rain100L and 1.49 dB on GoPro; SAM removal drops PSNR by 0.45, 1.63, and 1.54 dB on BSD68, Rain100L, and GoPro, respectively.SAM removal leaves visible residual artifacts; SAM adds modest GFLOPs while improving both PSNR and SSIM.Source: §V · p. 11
  • Setting rmax = 32 gives the reported accuracy–efficiency balance: rmax = 16 reduces PSNR by about 4.0 dB, while higher values add runtime for minimal gains.Increasing unfolding depth from K = 3 to K = 6 improves quality, while K = 8 adds negligible gains at higher cost.Source: §V · p. 11, Table X · p. 11, Table XII · p. 11
  • The consistency weight λunc = 0.1 improves BSD68 PSNR by 0.53 dB and LPIPS from 0.052 to 0.049; larger weights degrade both metrics.The table caption characterizes the λunc = 0.1 result as improved perceptual quality without oversmoothing.Source: §V · p. 11, Table XIII · p. 11

VI. CONCLUSION

MDIRNET combines an OVPCA-based deep-unfolding core with adaptive patch sampling, rank allocation, and attention refinement. It reports competitive performance across denoising, deraining, and deblurring, with limitations in deployment validation, severe large-kernel blur, and real-world mixed-degradation evaluation.

VI. CONCLUSION

  • MDIRNET achieves competitive performance across denoising, deraining, and deblurring with a unified framework offering an efficiency–accuracy trade-off.The model combines an OVPCA-based core with PPM, DRAM, and SAM; it has 0.68 M parameters.Source: §VI · p. 11
  • The controlled backbone comparison finds that DU-OVPCA provides a substantial performance benefit under the evaluated settings.Source: §VI · p. 11
  • Figure 10 reports per-layer PSNR saturation at K=6 and rmax-ablation saturation at 32.Source: Figure 10 · p. 12
  • Runtime was measured only on an NVIDIA RTX 3090; CPU, edge, and embedded efficiency, peak memory, energy use, and real-world mixed-degradation validation remain unvalidated.The low-rank prior is less effective for severe large-kernel blur, while fixed patch size and spatial grouping may limit adaptability across image structures and degradation scales.Source: §VI · p. 12
Loading 2610.01655v1…