Gradient Step Denoiser for convergent Plug-and-Play

Samuel Hurault; Arthur Leclaire; Nicolas Papadakis

Source-linked AI summary

Gradient Step Denoiser for convergent Plug-and-Play

Samuel Hurault, Arthur Leclaire, Nicolas Papadakis

arXiv:2110.03220v2Oct 7, 2021cs.CVeess.IVmath.OC

TL;DR

Image-restoration PnP methods often lack practical convergence guarantees because generic deep denoisers need restrictive assumptions and do not define an explicit objective. The paper constructs a neural-network gradient-step denoiser and embeds it in a proximal-gradient-like PnP scheme. The resulting method converges to stationary points of an explicit functional and achieves state-of-the-art restoration performance across ill-posed tasks.

Problem
Existing PnP convergence guarantees rely on restrictive denoiser assumptions or strongly convex data terms, while deep denoiser approximations may not preserve the identities needed for analysis.
Method
The method trains a denoiser as an exact gradient step on a neural-network-parameterized regularization function and uses it in a reordered, relaxed PnP-HQS scheme.
Results
The algorithm converges to stationary points of an explicit functional, with objective values decreasing, residuals converging to zero, and cluster points stationary under the stated assumptions.
Takeaways & Limitations
Experiments on deblurring, super-resolution, and inpainting confirm convergence and show state-of-the-art image-restoration performance.
Takeaways & Limitations
The convergence theorem assumes convex lower-semicontinuous data fidelity, differentiable gσ with Lipschitz gradient, a bounded iterate sequence, and a lower-bounded objective satisfying the KL property.

Abstract

from arXiv · show

Plug-and-Play methods constitute a class of iterative algorithms for imaging problems where regularization is performed by an off-the-shelf denoiser. Although Plug-and-Play methods can lead to tremendous visual performance for various image problems, the few existing convergence guarantees are based on unrealistic (or suboptimal) hypotheses on the denoiser, or limited to strongly convex data terms. In this work, we propose a new type of Plug-and-Play methods, based on half-quadratic splitting, for which the denoiser is realized as a gradient descent step on a functional parameterized by a deep neural network. Exploiting convergence results for proximal gradient descent algorithms in the non-convex setting, we show that the proposed Plug-and-Play algorithm is a convergent iterative scheme that targets stationary points of an explicit global functional. Besides, experiments show that it is possible to learn such a deep denoiser while not compromising the performance in comparison to other state-of-the-art deep denoisers used in Plug-and-Play schemes. We apply our proximal gradient algorithm to various ill-posed inverse problems, e.g. deblurring, super-resolution and inpainting. For all these applications, numerical results empirically confirm the convergence results. Experiments also show that this new algorithm reaches state-of-the-art performance, both quantitatively and qualitatively.

1 INTRODUCTION

Image restoration is an ill-posed inverse problem requiring effective regularization, while generic denoisers deliver strong results but lack convergence guarantees and an explicit objective. The proposed scheme uses a denoiser satisfying an explicit-gradient structure to obtain convergence without requiring strongly convex data fidelity.

Image restoration combines data fidelity with a regularizer that injects prior knowledge to address ill-posedness.The degradation model uses y = Aˆx + ξ, with the regularizer weighted by λ ≥0.
Plug-and-Play methods replace the proximity operator of the regularizer with a generic denoiser, often a pretrained deep network.
Generic denoisers generally lack proximal-map structure, making convergence difficult to establish and leaving the regularizer implicit.Consequently, standard PnP methods do not minimize an explicit objective functional.
Existing deep denoisers lack Jacobian symmetry, so RED does not minimize an explicit functional or guarantee convergence.
The proposed PnP scheme converges to a stationary point of an explicit functional without requiring strong convexity of the data-fidelity term.This covers ill-posed tasks including deblurring, super-resolution, and inpainting.

2 RELATED WORKS

Prior PnP convergence analyses impose restrictive or difficult-to-verify denoiser conditions, while deep approximations of ideal denoisers do not generally preserve the relevant theoretical identities. These limitations motivate convergence approaches tailored to practical deep denoisers.

State-of-the-art PnP results commonly use deep denoisers, but convergence proofs require assumptions such as boundedness, symmetric Jacobians, demicontractivity, or averagedness.The cited conditions may be too restrictive or hard to verify in practice.
Tweedie’s Identity relates the MMSE denoiser to the score of a Gaussian-smoothed image prior.Specifically, Dσ(x) = x + σ2∇x log pσ(x).
The MMSE denoiser cannot be computed explicitly, and Tweedie’s Identity does not generally hold for deep approximations of the MMSE denoiser.

3 THE GRADIENT STEP PLUG-AND-PLAY

The method constructs a gradient-step denoiser from a neural-network parameterization of an explicit regularizer, then embeds it in a reordered and relaxed PnP-HQS scheme. This structure connects the iterations to proximal gradient descent and explicit minimization.

Gradient Step Denoiser: The proposed PnP method replaces the prior proximity operator in PnP-HQS with a denoiser implemented as a gradient descent step.
Gradient Step Denoiser: The denoiser is defined as Dσ(x) = x − ∇gσ(x), with its neural-network expression involving Nσ(x) and JNσ(x).
Gradient Step Denoiser: The regularizer gσ is parameterized through a neural network so the denoiser represents a conservative vector field.This permits differentiable denoising architectures while preserving the explicit-gradient structure.
A Plug-and-Play Method for Explicit Minimization: GS-PnP switches the standard PnP-HQS order and relaxes the denoising step, yielding Proxτf ◦ (Id − τλ∇gσ).
A Plug-and-Play Method for Explicit Minimization: Under suitable conditions, fixed points of the GS-PnP operator correspond to critical points of a classical objective function.
A Plug-and-Play Method for Explicit Minimization: Using the gradient-step denoiser introduces explicit regularization and produces a tractable global optimization problem solved by the PnP algorithm.

4 CONVERGENCE ANALYSIS

The analysis establishes convergence of the GS-PnP iterations under convex, lower-semicontinuous data fidelity and a differentiable regularizer with Lipschitz gradient, without requiring continuity of the data term. The objective decreases, residuals vanish, and—under boundedness and the KL property—the iterates converge to a critical point; backtracking preserves these guarantees while avoiding unnecessarily small stepsizes.

Convergence results: For τ < 1/(λL), the objective values are non-increasing and converge, the residual ||xk+1 − xk|| tends to 0, and all cluster points are stationary.These guarantees hold when f is proper, lower-semicontinuous, and convex, while gσ is differentiable with L-Lipschitz gradient and F is bounded below.
Assumptions and scope: The convergence proof extends to nonconvex data fidelity with technical adaptations, including applications such as phase retrieval.The extension does not follow directly from the convex-case proof because the proximal operator need not remain 1-Lipschitz.
Convergence results: If the iterates are bounded and F satisfies the KL property, the sequence converges with finite length to a critical point of F.The theorem permits a nonconvex gσ and requires τ < 1/(λL).
Assumptions and scope: The analysis allows non-continuous data-fidelity terms, including the inpainting setting, because it uses exact forward-backward splitting.Continuity of f is not required under the stated framework.
Backtracking: Backtracking searches for a sufficiently decreasing stepsize because controlling the Lipschitz constant only on the iterate-related subset can avoid suboptimal globally chosen stepsizes.Under Theorem 1 assumptions, the procedure terminates finitely and preserves both convergence theorems.
Backtracking: With λτ0 = 1, backtracking hardly ever activates, and the guarantees apply to standard PnP-HQS using the proposed denoiser.The paper identifies this as the practical initialization used in experiments.

5 EXPERIMENTS

Experiments evaluate the GS denoiser and GS-PnP on denoising, deblurring, super-resolution, and related ill-posed restoration tasks. The method combines competitive image quality with guaranteed convergence and empirically strong restoration results.

Denoising results: GS-DRUNet matches or slightly outperforms unconstrained DRUNet light while reducing processing time by a factor of 7.It remains close to original DRUNet in PSNR and maintains an approximately +0.5dB gap over DnCNN and FFDNet.
Convergence and setup: The experiments empirically confirm convergence through the evolution of objective values and residuals, with the algorithm applied to ill-posed restoration problems.The stated objective is F(x) = 1/(2ν^2)||Ax − y||^2 + λ/2||Nσ(x) − x||^2, and convergence of iterates and objective values is guaranteed by the theorems.
Deblurring: The experiments evaluate GS-PnP on deblurring using CBSD10 images, multiple blur kernels, and three noise levels.The compared methods include EPLL, IRCNN, DPIR, MMO, and RED.
Deblurring: GS-PnP closely follows DPIR at low noise and performs equally or better at higher noise levels, while retaining a convergence guarantee.The deblurring results are reported as state-of-the-art across the tested kernels and noise levels.
Super-resolution: For super-resolution, GS-PnP outperforms all other compared PnP algorithms in PSNR across the tested blur kernels, noise levels, and scale factors.The method also recovers structures and color details more accurately in the illustrated leaves example while function values and residuals converge.

6 CONCLUSION

The paper introduces a convergent PnP algorithm whose neural denoiser realizes an exact gradient step on a learned regularization function. Experiments on ill-posed imaging problems support both its convergence and state-of-the-art restoration performance.

Conclusion: The proposed PnP algorithm converges toward a stationary point of an explicit functional while allowing non-strongly-convex and non-smooth data fidelity terms.Its denoiser is trained to realize an exact gradient step on a neural-network-parameterized regularization function.
Conclusion: Experiments on deblurring, super-resolution, and inpainting confirm convergence results and show state-of-the-art image restoration performance.

7 REPRODUCIBILITY STATEMENT

The paper provides source code, pseudocode, hyperparameter details, dataset information, and complete proofs to support reproduction of the algorithm and theoretical results.

Reproducibility: Anonymous source code includes a README with step-by-step instructions for running the algorithm and replicating the results.
Reproducibility: The pseudocode, experiment-specific hyperparameters and datasets, and complete theoretical proofs are provided across the paper and appendixes.

8 ETHICS STATEMENT

The authors state that the work does not raise potential ethical concerns.

Ethics statement: The authors report no potential ethical concerns for this work.

A.1 LIMITATIONS OF PREVIOUS PNP METHODS

Previous Plug-and-Play guarantees are limited by restrictive denoiser assumptions, strong-convexity requirements, or a lack of explicit objective characterization. The proposed formulation instead uses a conservative-field denoiser to obtain convergence to stationary points while retaining broad inverse-problem applicability.

Existing PnP methods often lack convergence proofs for non-strongly-convex data-fidelity terms, including deblurring, super-resolution, and inpainting.
Many guarantees restrict denoisers to nearly nonexpansive or symmetric-Jacobian classes, which can alter deep-denoiser performance.
Some methods show iterate convergence through decreasing step sizes without characterizing the result as a minimizer or critical point of a functional.
The method can use a possibly expansive denoiser while converging to a stationary point of an explicit functional that includes non-strongly-convex data fidelity.
The proposed denoiser is defined as Dσ = Id −∇gσ, making it inherently a conservative vector field and converting the modified PnP-HQS scheme into proximal gradient descent.

C PROOF OF THEOREM 1

The proof establishes descent and convergence properties for the proximal-gradient operator under the stated regularity assumptions. Fixed points correspond to stationary points, and bounded iterates yield convergence of cluster points to such solutions.

The objective F = f + gσ is analyzed through the proximal-gradient operator Tτ = Proxτf ◦ (Id −τ∇gσ).
The objective values are non-increasing and converge to a finite limit because F is lower-bounded.
A fixed point of Tτ is equivalent to a stationary point satisfying −∇gσ(x*) ∈ ∂f(x*).
Although nonconvexity means Tτ need not be nonexpansive, its cluster points can still be shown to be stationary points of the objective.
The convergence proof assumes bounded iterates; coercivity of F or an additional projection can provide a sufficient boundedness condition.

F ON THE ASSUMPTIONS OF THEOREMS 1 AND 2

The convergence theorems rely on standard nonconvex-optimization assumptions: a suitable convex data-fidelity term, a differentiable smooth regularizer, bounded objective values, controlled step sizes, and the KL property for full iterate convergence.

The data-fidelity term f must be proper, lower semicontinuous, and convex, without requiring differentiability.
The regularizer gσ must be proper, lower semicontinuous, differentiable, and have an L-Lipschitz gradient.
The functional F = f + λgσ must be bounded below, which is stated to follow because its terms are positive.
The stepsize must satisfy τ < 1/(λL), with backtracking used to enforce this condition.
Theorem 2 additionally assumes the KL property of F and boundedness of the generated sequence.

G BACKTRACKING AND PROOF OF PROPOSITION 1

Backtracking selects stepsizes that ensure sufficient objective decrease, while experiments illustrate convergence despite an expansive denoiser and report image-restoration comparisons across deblurring and super-resolution settings.

The backtracking line search is an Armijo-type strategy that reduces the stepsize until the sufficient-decrease condition is satisfied.
The DRUNet light architecture is used to parameterize Nσ in the proposed denoiser.
The denoiser is not necessarily nonexpansive, yet super-resolution experiments still show convergence when its Lipschitz constant exceeds 1.
For deblurring on Set3c, GS-PnP outperforms competing methods at noise levels 0.03 and 0.05 and follows DPIR at noise level 0.01.
For super-resolution on Set3c, GS-PnP shows the same pattern: it outperforms competitors at noise levels 0.03 and 0.05 and follows DPIR at 0.01.

J.3 INPAINTING (WITH NON-DIFFERENTIABLE DATA-FIDELITY TERM)

The proposed PnP scheme applies to inpainting with a non-differentiable data-fidelity term and empirically restores masked images accurately while converging. The experiments also examine parameter sensitivity, initialization robustness, and convergence relative to DPIR.

Inpainting setup: Inpainting is modeled with a binary diagonal degradation matrix, no added noise, and an indicator-function data-fidelity term whose proximal map is an orthogonal projection.Despite the non-differentiability of the data-fidelity term, the paper states that the convergence assumptions remain satisfied.
Inpainting results: With random masking probability p = 0.5, the method restores set3C images with high accuracy, including small details.The algorithm uses σ = 10/255, a fixed stepsize, λτ = 1, and an initialization based on the observed image and mask.
Inpainting results: The inpainting residual empirically converges at rate O(1/k).Figure 9 tracks γk = min0≤i≤k ||x_i+1 − x_i||2/||x_0||2 across iterations.
Convergence and objective behavior: Across deblurring and super-resolution experiments, minimizing F coincides with maximizing PSNR, supporting the optimized functional F = f + λgσ.Figure 10 compares PSNR evolution with the convergence of the function value.
Convergence comparison: GS-PnP converges while DPIR fails to converge under both tested noise-scheduling strategies, with GS-PnP reaching residuals near 10^-13 versus about 10^-4 for DPIR.The paper also reports that DPIR’s divergence is accompanied by a loss of restoration performance in PSNR.