Source-linked AI summary
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
Ernest K. Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, Wotao Yin
TL;DR
PnP has strong empirical performance but insufficient convergence theory for modern denoisers. This paper proves convergence for PnP-FBS and PnP-ADMM under a denoiser Lipschitz condition, introduces real spectral normalization to enforce it during training, and validates the theory experimentally.
Problem
PnP has been empirically successful, but theoretical analysis of when it converges and which denoisers can be used has been insufficient.
Method
The paper analyzes PnP-FBS and PnP-ADMM under a Lipschitz denoiser condition and proposes real spectral normalization to enforce that condition during deep-denoiser training.
Results
The analysis establishes convergence of PnP-FBS and PnP-ADMM, and experiments validate the resulting theory.
Takeaways & Limitations
PnP convergence can be analyzed without assuming denoisers are nonexpansive or differentiable, using a condition intended to cover practical denoisers.
Takeaways & Limitations
The application experiments assume insufficient medical images for training and therefore omit some recent deep-learning-based methods.
Abstract
from arXiv · showhide
Plug-and-play (PnP) is a non-convex framework that integrates modern denoising priors, such as BM3D or deep learning-based denoisers, into ADMM or other proximal algorithms. An advantage of PnP is that one can use pre-trained denoisers when there is not sufficient data for end-to-end training. Although PnP has been recently studied extensively with great empirical success, theoretical analysis addressing even the most basic question of convergence has been insufficient. In this paper, we theoretically establish convergence of PnP-FBS and PnP-ADMM, without using diminishing stepsizes, under a certain Lipschitz condition on the denoisers. We then propose real spectral normalization, a technique for training deep learning-based denoisers to satisfy the proposed Lipschitz condition. Finally, we present experimental results validating the theory.
1. Introduction
PnP extends optimization-based imaging methods by replacing proximal denoisers with modern denoisers, addressing a gap between strong empirical performance and limited convergence theory.
- Optimization-based imaging: Classical imaging problems combine data fidelity and image-complexity terms in an optimization objective solved by iterative methods such as ADMM.The data-fidelity term measures agreement with measurements, while the complexity term encodes prior knowledge such as low noise.
- Modern denoisers: Modern denoisers including NLM, BM3D, and CNNs can perform well without originating from explicit optimization problems.Their denoising strength is controlled by a noise parameter σ, with larger σ producing more aggressive denoising.
- Plug-and-play framework: PnP-ADMM replaces the proximal operator associated with the regularizer by a denoiser, enabling such denoisers to address broader imaging problems.The method was proposed as a direct response to using denoisers that cannot be directly represented through an optimization problem.
- Motivation: PnP achieved strong empirical success across many imaging applications, but prior theoretical analysis did not adequately answer when it converges or which denoisers are admissible.Earlier work covered multiple PnP variants and denoiser classes, including nonexpansive-denoiser assumptions and increasing penalty parameters.
- Paper contribution: This paper analyzes PnP-FBS and PnP-ADMM under a Lipschitz condition, proposes real spectral normalization for training compliant deep denoisers, and validates the theory experimentally.The condition is intended to accommodate denoisers that are neither nonexpansive nor differentiable.
2. PNP-FBS/ADMM and their fixed points
PnP-FBS and PnP-ADMM are formulated as fixed-point iterations that repeatedly combine data-consistency operations with denoising, while PnP-DRS provides an analytical representation for PnP-ADMM.
- Relationship: PnP-FBS and PnP-ADMM are distinct methods that share the same set of fixed points.PnP-FBS requires computing ∇f, whereas PnP-ADMM uses the proximal map of the data-fidelity term.
- PNP-FBS: PnP-FBS applies a gradient-based data-consistency map and a denoiser in alternating fashion to seek a fixed point.Its fixed point can be interpreted as a compromise between agreement with measurements and reduced image noise.
- PNP-ADMM: PnP-ADMM is likewise interpreted as a fixed-point iteration over its image and dual variables.At a fixed point, successive image, auxiliary, and dual variables remain unchanged.
- PNP-DRS: PnP-DRS is introduced as an analytical tool because it is equivalent to PnP-ADMM and can be written as an iteration z_k+1 = T(z_k).Convergence results for PnP-DRS can therefore be translated to PnP-ADMM.
3. Convergence via contraction
The paper establishes convergence by showing that PnP iterations are contractive under a Lipschitz denoiser condition and suitable assumptions on the data-fidelity function.
- Denoiser assumption: The proposed approach avoids the overly strong requirement that 2Hσ − I be nonexpansive, which fails for NLM according to a reported counterexample.Instead, the analysis assumes the denoiser is sufficiently close to the identity map, a condition expected for small σ.
- Contraction framework: Under the Lipschitz assumption, PnP-FBS and PnP-DRS become contractive fixed-point iterations and converge geometrically to fixed points.The geometric convergence follows from the Banach contraction principle when the contraction factor is below 1.
- PNP-FBS: PNP-FBS convergence is proved when f is strongly convex, differentiable, and has an L-Lipschitz gradient, with a stepsize interval determined by the contraction bound.A suitable α exists if ε < 2µ/(L −µ).
- PNP-ADMM: PNP-ADMM convergence is proved when f is µ-strongly convex and ε ∈ [0,1), provided ε (1 + ε −2ε2)µ < α.The result follows from the equivalence between PnP-DRS and PnP-ADMM.
- Scope: The strong-convexity assumptions cover applications such as image denoising, deblurring, and single-photon imaging but exclude some compressed-sensing and super-resolution settings.The authors also note that the contraction analysis can be adapted to local convergence when the Lipschitz assumption holds locally.
4. Real spectral normalization: enforcing Assumption (A)
Real spectral normalization (realSN) adapts spectral normalization to convolutional operators, enabling denoisers to satisfy the Lipschitz condition required by the theory. It constrains the residual mapping’s Lipschitz constant while retaining practical training efficiency.
- RealSN trains deep denoisers to satisfy Assumption (A) by more accurately constraining their network Lipschitz constants.The method was proposed specifically to connect practical denoiser training with the paper’s convergence theory.
- For denoisers H(y) = y − R(y), enforcing Assumption (A) is equivalent to constraining the residual mapping R’s Lipschitz constant.The residual is R(y) = y − H(y) = (I − H)(y).
- Standard SN failed to control SimpleCNN’s layer spectral norms below 1, producing values 3.01, 2.96, 2.82, and 1.31.The paper identifies the inexact convolutional-layer implementation as the reason for this underestimation of the true operator norm.
- Unlike standard SN’s matrix reshaping, realSN directly estimates the spectral norm of the full convolutional linear operator.It maintains convolution-shaped left and right singular-vector estimates and applies power iteration during each forward pass.
- RealSN can constrain the network Lipschitz constant to any upper bound C = Q_L l=1 c_l and adds three one-sample convolution operations per layer per training step.With batch size 128, the authors describe the added computational cost as mild.
5. Poisson denoising: validating the theory
The Poisson denoising experiments examine whether RealSN-trained denoisers exhibit the theoretically predicted convergence behavior and competitive reconstruction performance. RealSN denoisers support more stable convergence patterns, while different PnP methods favor different denoisers.
- Experimental setup: The Poisson denoising objective uses the negative Poisson log-likelihood, with gradients defined for positive image values and nonnegative denoiser outputs.The experiments use BM3D, SimpleCNN, DnCNN, and RealSN variants on 13 test images.
- Convergence validation: Figure 1 tests the denoiser Lipschitz condition by estimating ε from differences between iterates and their limiting values.Convergence of PnP-ADMM requires ε < 1 under Corollary 3.
- Convergence validation: Figure 2 varies α and measures the average contraction factor over 500 iterations, with lower values indicating faster convergence.Smoother curves indicate easier tuning and more stable convergence.
- Convergence validation: RealSN denoisers roughly exhibit the predicted contraction behavior: PnP-FBS contracts over an α interval, whereas PnP-ADMM contracts for sufficiently large α.The experiment qualitatively validates Theorems 1 and 2.
- Empirical performance: The theory addresses convergence but not the recovery quality of the converged output, so the experiments separately assess empirical denoising performance.The authors report competitive denoising results for the RealSN-based methods.
- Empirical performance: RealSN denoisers outperform BM3D in one of the two PnP methods, while denoiser preference depends on the PnP framework.RealSN-SimpleCNN surpasses RealSN-DnCNN under PnP-FBS, whereas RealSN-DnCNN performs best with PnP-ADMM.
6. More applications
The paper evaluates PnP methods on single photon imaging and compressed sensing MRI using denoisers trained on natural images. Across these applications, PnP-ADMM generally outperforms PnP-FBS, with RealSN-DnCNN achieving the strongest reported results.
- Single photon imaging: Single photon imaging models binary photon observations from a QIS and aims to recover the true image.The experiment uses α_sg = K = 8 and reports PSNR at iterations 50, 100, and the best value within 100 iterations.
- Single photon imaging: PnP-ADMM consistently yields higher PSNR than PnP-FBS with the same denoiser in single photon imaging.The best overall result uses PnP-ADMM with RealSN-DnCNN and improves nearly 1 dB over BM3D.
- Compressed sensing MRI: Compressed sensing MRI accelerates acquisition through downsampling, allowing a denoiser trained on natural images to be applied when medical training data are limited.The experiments use random, radial, and Cartesian sampling at 30% with measurement noise σ_e = 15/255.
- Compressed sensing MRI: The CS-MRI comparison includes zero-filling, TV, RecRF, BM3D-MRI, and PnP methods using BM3D, SimpleCNN, and DnCNN variants.Table 3 reports PSNR results for the 30% sampling setting.
- Experimental scope: The comparisons exclude several recent deep-learning methods because the experiments assume insufficient medical images for training.This limits the scope of the CS-MRI comparison.
- Compressed sensing MRI: For CS-MRI, RealSN-DnCNN appears to be the clear overall winner, and PnP-ADMM generally outperforms PnP-FBS with the same denoiser.The deep denoisers are trained on BSD500 without medical images.
7. Conclusion
The paper analyzes convergence of PnP-FBS and PnP-ADMM under a denoiser Lipschitz assumption, proposes real spectral normalization to enforce it, and validates the theory experimentally.
- 7. Conclusion: The paper analyzes convergence of PnP-FBS and PnP-ADMM under a Lipschitz assumption on the denoiser.It then presents real spectral normalization to enforce the proposed condition during deep denoiser training.
8. Preliminaries
The preliminaries define convexity, proximal operators, Lipschitz and averaged mappings, and lemmas used to analyze PnP iterations. These tools connect contraction and averagedness properties to the convergence proofs.
- Convex analysis: A convex function is closed when lower semicontinuous and proper when finite somewhere; strong convexity is defined by subtracting a quadratic term.These assumptions support well-defined proximal operators and later convergence lemmas.
- Proximal operators: For a convex, closed, proper function, the proximal operator is well-defined because its argmin exists uniquely.The proximal operator maps an input to the unique minimizer of the corresponding regularized objective.
- Mapping properties: A mapping is nonexpansive when its Lipschitz constant is at most 1 and contractive when the constant is below 1.These properties quantify how the mapping changes distances between inputs.
- Averaged mappings: A θ-averaged mapping is nonexpansive and can be represented using another nonexpansive mapping.The representation is used to characterize operator compositions.
- Operator lemmas: The composition of two averaged mappings is averaged, with its parameter determined by the two component parameters.The lemmas also relate averagedness of a mapping to averagedness after composition with negation.
- Convergence assumptions: The supporting lemmas impose strong convexity and, for PnP-FBS, Lipschitz continuity of the gradient.These conditions are used in the subsequent convergence analysis.
9. Proofs of main results
The proofs establish equivalence between PNP-DRS and PNP-ADMM and derive convergence conditions by analyzing averagedness and Lipschitz properties of the denoiser and gradient-related operators.
- PNP-DRS and PNP-ADMM are shown equivalent by substitution and reordering iterations to obtain the correct dependencies.
- Assumption (A) is equivalent to G being θ-averaged after reparameterizing Hσ through θ and G.The proof defines θ = ε/(1+ε) and G = Hσ/(1+ε), then matches the resulting inequality to averagedness.
- Under Assumption (A), Hσ is nonexpansive and 2ε/(1+2ε)-averaged.
- Theorem 1’s proof composes the Lipschitz constants of I − α∇f and Hσ, yielding the stated convergence restrictions on α and ε.I − α∇f has Lipschitz constant max{|1 − αµ|, |1 − αL|}, while Hσ is (1+ε)-Lipschitz.
- Theorem 2’s proof represents the relevant operator using a nonexpansive R, applies Young’s inequality, and optimizes the resulting bound.The proof concludes that the optimized bound gives the first part of the theorem; restrictions on α and ε follow algebraically.