PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Maximally Monotone Operators for Image Recovery

Jean-Christophe Pesquet, Audrey Repetti, Matthieu Terris, Yves Wiaux

arXiv:2012.13247v2math.OCeess.IV

TL;DR

Functional regularizers for inverse problems can be difficult to choose, and PnP methods leave open how to characterize their asymptotic solutions. The paper learns an MMO resolvent with a neural network and reports that its PnP-FB method outperforms standard variational and state-of-the-art PnP methods on image deblurring.

  • Problem

    In inverse problems, functional regularizers must be tractable to optimize, and their parameter choices can strongly affect reconstruction quality.

    Source: §1 · p. 2

  • Method

    The paper learns a maximally monotone operator in a supervised manner and models its resolvent with a firmly nonexpansive neural network.

    Source: §1 · pp. 3–4

  • Results

    The proposed PnP-FB algorithm outperforms standard variational methods and state-of-the-art PnP algorithms on image deblurring.

    Source: §5 · p. 25

  • Takeaways & Limitations

    The approach characterizes PnP limits as solutions to variational inclusion problems, extending beyond convex regularization.

    Source: §1 · p. 3, §5 · p. 25

    The demonstrated applications address restoration problems; super-resolution and reconstruction are described as wider-scope applications for future work.

    Source: §5 · p. 25

Abstract

from arXiv · show

We introduce a new paradigm for solving regularized variational problems. These are typically formulated to address ill-posed inverse problems encountered in signal and image processing. The objective function is traditionally defined by adding a regularization function to a data fit term, which is subsequently minimized by using iterative optimization algorithms. Recently, several works have proposed to replace the operator related to the regularization by a more sophisticated denoiser. These approaches, known as plug-and-play (PnP) methods, have shown excellent performance. Although it has been noticed that, under some Lipschitz properties on the denoisers, the convergence of the resulting algorithm is guaranteed, little is known about characterizing the asymptotically delivered solution. In the current article, we propose to address this limitation. More specifically, instead of employing a functional regularization, we perform an operator regularization, where a maximally monotone operator (MMO) is learned in a supervised manner. This formulation is flexible as it allows the solution to be characterized through a broad range of variational inequalities, and it includes convex regularizations as special cases. From an algorithmic standpoint, the proposed approach consists in replacing the resolvent of the MMO by a neural network (NN). We present a universal approximation theorem proving that nonexpansive NNs are suitable models for the resolvent of a wide class of MMOs. The proposed approach thus provides a sound theoretical framework for analyzing the asymptotic behavior of first-order PnP algorithms. In addition, we propose a numerical strategy to train NNs corresponding to resolvents of MMOs. We apply our approach to image restoration problems and demonstrate its validity in terms of both convergence and quality.

1. Introduction.

The introduction motivates replacing hand-chosen functional regularizers with learned maximally monotone operators (MMOs). It presents a firmly nonexpansive neural-network approach to learning resolvents for plug-and-play algorithms, addressing both theoretical modeling and practical constraints.

1. Introduction.

  • Functional regularizers can be hard to choose tractably, and their parameters can strongly affect reconstruction quality.They are commonly used to avoid overfitting or ill-posedness, but choices are limited by tractability and parameter sensitivity.Source: §1 · p. 2
  • Unlike convex optimization, monotone inclusions also cover saddle-point problems and game-theory equilibria.Convex minimization is a special case because a convex function's subdifferential is maximally monotone; the broader formulation permits operator regularization learned from application data.Source: §1 · pp. 2–3
  • Prior PnP approaches face difficulty enforcing firm nonexpansiveness, while some convergence-oriented architectures impose restrictions or rely on restrictive linear-operator conditions.The cited examples include omitting residual skip connections and restricting architectures through proximal-calculus rules.Source: §1 · p. 3
  • A universal approximation theorem shows that firmly nonexpansive neural networks can approximate resolvents of stationary MMOs as closely as desired.This provides the theoretical basis for modeling MMO resolvents with neural networks.Source: §1 · p. 4
  • The proposed training framework regularizes the loss using a Jacobian spectral norm and evaluates the resulting PnP scheme with forward-backward splitting on image restoration.The paper reports positive quality comparisons with state-of-the-art PnP methods and regularized optimization approaches.Source: §1 · p. 4

2. Neural network models for maximally monotone operators.

The section connects maximally monotone operators to nonexpansive neural-network models and proves that resolvents of stationary operators can be approximated on compact sets. It also identifies the structural constraint behind the approximation and its broader scope.

2.1. A property of maximally monotone operators.

  • A maximally monotone operator is characterized through a nonexpansive mapping, providing the operator representation used to build neural-network models.The section also extends the approximation result to convex combinations of stationary MMO resolvents.Source: §2.1 · pp. 4–5, §2.4 · p. 10

2.2. Nonexpansive neural networks.

  • The proposed feedforward-network approach models nonexpansive operators and enforces nonexpansiveness through training regularization.The network is composed of layer operators with activations, weights, and biases; the activation dictionary includes identity and block-size-2 sorting.Source: §2.2 · p. 5

2.3. Stationary maximally monotone operators.

  • Stationary MMOs form a subclass of MMOs, with examples including product operators, unitary transforms, convex subdifferentials, and certain affine operators.The definition uses a product space and per-component bounded linear and self-adjoint nonnegative operators.Source: §2.3 · pp. 6–7

2.4. Universal approximation theorem.

  • On compact subsets of R^K, a firmly nonexpansive neural network can approximate the resolvent of a stationary MMO within any prescribed ε.The theorem also bounds the distance between corresponding solutions by ε when x + y lies in the compact set.Source: §2.4 · p. 8
  • The approximation theorem requires firm nonexpansiveness, making its guarantees less accurate than standard universal approximation results.The paper states that this requirement enables convergent PnP algorithms.Source: §2.4 · p. 11

3. Proposed algorithm.

The proposed method replaces a regularization function with a learned maximally monotone operator and uses its neural-network resolvent in forward-backward iterations. The section establishes convergence conditions and describes a training strategy that constrains the network’s nonexpansiveness.

3.1. Forward-backward algorithm.

  • PnP-FB applies the learned resolvent after a gradient step, and finite iterations unfold into a network with repeated stages and data-dependent gradient biases.For least-squares fidelity, the gradient step uses the observation operator and measured data.Source: §3.1 · pp. 11–13
  • When the solution set Sγ is nonempty and the stated smoothness and averagedness conditions hold, PnP-FB converges weakly to a point satisfying the variational inclusion.The stepsize must satisfy γ ∈ ]0, 2/µ[.Source: §3.1 · pp. 11–12
  • The learned operator need not yield a minimization problem: that interpretation requires maximal cyclic monotonicity, not merely maximal monotonicity.The stepsize γ also affects the solution set, not only the convergence profile.Source: §3.1 · p. 12
  • RED-PRO can be written in the same PnP-FB form, but it trains a relaxed denoiser Q whereas this approach trains the denoiser eJ directly [17].The cited RED-PRO characterization also differs in its limit-point description.Source: §3.1 · p. 12

3.2. Training.

  • Training minimizes denoising error while constraining Qθ = 2eJθ − Id to be nonexpansive through Jacobian-norm penalties evaluated along segments between training images and network outputs.The paper notes that global enforcement is impractical and uses an exterior penalty for the sampled constraints.Source: §3.2 · pp. 13–14
  • The training procedure uses Adam with randomly sampled image batches and estimates the full-network Jacobian spectral norm by power iteration; sufficiently large penalty weights satisfy the sampled constraints.The stated penalty guarantee applies at the selected interpolation points.Source: §3.2 · p. 14

4. Simulations and results.

The simulations test the proposed method on grayscale and color deblurring, examining parameter choices, convergence, and reconstruction quality. The proposed method achieves the best PSNR across eight grayscale blur kernels and improves reconstruction quality over the compared PnP methods in color experiments.

4.1. Experimental setting.

  • Experiments model deblurring as circular convolution with additive white Gaussian noise and evaluate grayscale BSD68/BSD10 and color BSD500/Flickr30 images using a modified 20-layer DnCNN.The blur kernels are drawn from and.Source: §4.1 · pp. 15–16, Figure 3 · p. 16, Figure 2 · p. 16

4.2. Choice of the parameters.

  • For λ ≥ 10^-5, the tested Jacobian norms are below 1 and the PnP-FB convergence profiles are monotonically decreasing; the average PSNR peaks at λ = 10^-5.For λ = 10^-6, none of the tested profiles is monotone, although some cases converge with more iterations.Source: §4.2 · pp. 17–18, Table 1 · p. 18
  • Average PSNR and SSIM peak near σ/(γνeff) = 1; the best reported PSNR uses γ = 1.99 and ratio 1, while the best SSIM uses γ = 1.99 and ratio 0.9.The authors suggest γ = 1.99/μ and σ = 4ν∥h∥/μ based on their results.Source: §4.2 · pp. 19–20, Figure 5 · p. 20

4.3. Comparison with other PnP methods.

  • With the proposed firmly nonexpansive DnCNN, c_n decreases monotonically; with BM3D or RealSN, it does not tend to zero, indicating nonconvergence in the tested setting.The authors report that BM3D and RealSN are not firmly nonexpansive.Source: Figure 7 · p. 21, §4.3 · p. 22
  • On BSD10, the proposed firmly nonexpansive DnCNN delivers the best PSNR for all eight kernels; on Flickr30 and BSD500, it significantly improves reconstruction quality over the compared PnP methods.The color comparisons use the experimental settings G. A, M. A, M. B, and S.Source: Table 2 · p. 21, §4.3 · pp. 22, 23, 25, Figure 9 · p. 23
  • Color reconstructions contain finer details and avoid the oversmoothing of PnP-FB with DnCNN and slight blurring of PnP-FB with BM3D; gains are more noticeable for M. A and M. B.The proposed method shows homogeneous performance across images, unlike PnP-FB with DnCNN, which may diverge.Source: Figure 10 · p. 24, §4.3 · p. 25

5. Conclusion.

The paper learns MMO resolvents for plug-and-play algorithms, grounding the approach in a universal approximation theorem and a training strategy that regularizes the network Jacobian. The authors report convergence and restoration-quality benefits for PnP-FB, while noting applications beyond the restoration settings studied.

Conclusion

  • The universal approximation theorem shows that resolvents of a wide range of MMOs, including stationary MMOs, can be approximated by nonexpansive neural networks.Learning these operators lets the authors characterize algorithm limits as solutions to variational inclusion problems.Source: §5 · p. 25
  • The proposed Jacobian-informed training loss learns MMO resolvents for high-dimensional data under mild architectural conditions and can be optimized with existing training strategies.The authors report good convergence properties for PnP algorithms based on forward-backward splitting when trained with the proposed Jacobian regularization.Source: §5 · p. 25
  • The authors report that PnP-FB outperforms standard variational methods and state-of-the-art PnP algorithms on image deblurring.Their results suggest that optimal stepsize and training noise can be determined from the data-fidelity gradient's Lipschitz constant and the blurring-kernel norm.Source: §5 · p. 25
  • The resolvent approximation applies to iterative algorithms beyond forward-backward splitting, while super-resolution and reconstruction are proposed as applications beyond the restoration problems studied.Automatic selection of λ is also identified as future work.Source: §5 · p. 25

Appendix A. Nonexpansive networks: additional results.

The appendix gives sufficient conditions under which feedforward neural networks are nonexpansive. These conditions constrain weight operators and activation operators, and the illustrated architecture includes a skip connection.

Appendix A. Nonexpansive networks: additional results

  • Proposition A.1 gives sufficient conditions for a feedforward network to be nonexpansive when its activation operators are averaged.One condition is that the product of the weight-operator norms is at most 1.Source: Appendix A · p. 28
  • The proposition also considers separable activation operators, defined coordinatewise through one-variable functions, with diagonal matrices whose entries are 1 − 2α_m or 1.The excerpt does not state the full associated condition.Source: Appendix A · p. 28
  • The authors say Condition (i) in Proposition A.1 is the simplest but usually quite restrictive; if equation (A.2) is weaker, Condition (iii) requires nonnegative network weights.Most activation operators used in neural networks—including ReLU, leaky ReLU, sigmoid, and softmax—satisfy the α-averagedness assumption with α_m = 1/2.Source: Appendix A · p. 29
  • The illustrated feedforward network architecture for applying Proposition A.1 has a global skip connection induced by equation (2.1).The figure caption says its weight operators must satisfy Proposition A.1's conditions.Source: Appendix A · p. 29, Figure 11 · p. 29

Appendix B. Stationnary MMOs: additional proofs.

The appendix supplies proof fragments for examples of stationary maximally monotone operators. These proofs use surjectivity, monotonicity, and reductions to previously established examples.

Appendix B. Stationnary MMOs: additional proofs

  • In Example 2.7, surjectivity of U and maximal monotonicity of B establish that U*BU is maximally monotone and that ran(Id + A) = H.Example 2.8 is identified as a special case of Example 2.7 when each H_k equals R.Source: Appendix B · pp. 29–30
  • For the separable-operator proof, monotonicity of each B_k yields nonnegative inner products, which Proposition 2.6(ii) uses to conclude that A is a stationary MMO.The proof expresses the inner products through the decimation operators D_k and maps U.Source: Appendix B · p. 30
  • When B + B* is nonnegative, the appendix states that B and A are maximally monotone and that the reflected resolvent of B is nonexpansive.It identifies J_A as a shifted resolvent of B.Source: Appendix B · p. 30

Appendix C. Power iterative method.

The appendix uses power iteration to estimate the spectral norm of the network Jacobian without forming or storing the full Jacobian. The experiments use five power-method iterations, which the authors report are sufficient in practice for PnP-FB convergence.

Appendix C. Power iterative method

  • Five power-method iterations were used in experiments and were sufficient in practice to ensure convergence of PnP-FB.The method applies the Jacobian and its transpose to vectors using automatic differentiation, avoiding computation or storage of the full Jacobian.Source: Appendix C · p. 31
Loading 2012.13247v2…