Source-linked AI summary
Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium
Gregery T. Buzzard, Stanley H. Chan, Suhas Sreehari, Charles A. Bouman
TL;DR
Regularized inversion is limited when useful forward or prior models cannot be expressed through cost functions. The paper introduces Consensus Equilibrium, an equilibrium-based framework for fusing such operators, and reports improved consensus denoising with multiple CNNs, while noting nonunique solutions and unresolved algorithmic questions.
Problem
Optimization-based regularized inversion excludes some effective denoisers and neural networks, motivating a framework that can include non-optimization operators.
Method
Consensus Equilibrium fuses multiple operators through equilibrium equations and provides ADMM-based and Newton-based methods for solving them.
Results
Consensus Equilibrium integrates CNN denoisers and can achieve better results than any individual denoiser, while recovering MAP solutions in the proximal-map setting.
Takeaways & Limitations
CE provides an optimization-free reconstruction framework for problems lacking a corresponding regularized optimization formulation.
Takeaways & Limitations
CE may have multiple solutions, and further investigation of efficient algorithms remains open.
Abstract
from arXiv · showhide
Regularized inversion methods for image reconstruction are used widely due to their tractability and ability to combine complex physical sensor models with useful regularity criteria. Such methods motivated the recently developed Plug-and-Play prior method, which provides a framework to use advanced denoising algorithms as regularizers in inversion. However, the need to formulate regularized inversion as the solution to an optimization problem limits the possible regularity conditions and physical sensor models. In this paper, we introduce Consensus Equilibrium (CE), which generalizes regularized inversion to include a much wider variety of both forward components and prior components without the need for either to be expressed with a cost function. CE is based on the solution of a set of equilibrium equations that balance data fit and regularity. In this framework, the problem of MAP estimation in regularized inversion is replaced by the problem of solving these equilibrium equations, which can be approached in multiple ways. The key contribution of CE is to provide a novel framework for fusing multiple heterogeneous models of physical sensors or models learned from data. We describe the derivation of the CE equations and prove that the solution of the CE equations generalizes the standard MAP estimate under appropriate circumstances. We also discuss algorithms for solving the CE equations, including ADMM with a novel form of preconditioning and Newton's method. We give examples to illustrate consensus equilibrium and the convergence properties of these algorithms and demonstrate this method on some toy problems and on a denoising example in which we use an array of convolutional neural network denoisers, none of which is tuned to match the noise level in a noisy image but which in consensus can achieve a better result than any of them individually.
1. Introduction.
Regularized inversion balances data fit and regularity through optimization, but this requirement excludes some effective denoisers and neural-network models. Consensus Equilibrium extends the framework to non-optimization operators and demonstrates consensus denoising with multiple CNNs.
- Regularized inversion uses MAP estimation or related optimization to balance computational complexity, image quality, data fitting, and prior-based regularization.
- Many strong denoising algorithms and denoising neural networks cannot generally be represented as simple optimization problems.
- The paper asks how consensus optimization can encompass models and operators without optimization formulations while retaining efficient solution methods.
- Consensus Equilibrium fuses heterogeneous data-fidelity and prior operators, including denoisers, without requiring either component to arise from a cost function.
- The authors introduce CE equations, anisotropically preconditioned ADMM, and standard and Jacobian-free Newton methods for solving them.
- An array of CNN denoisers, none tuned to the noisy image's noise level, achieves a better consensus result than any individually applied CNN denoiser.
2. Consensus Equilibrium: Optimization and Beyond.
Consensus Equilibrium replaces cost minimization with equilibrium among multiple maps, while recovering consensus optimization when the maps are proximal operators. Beyond that setting, CE supports operators without an underlying optimization problem, though solutions may be nonunique.
- Consensus optimization foundation: The optimization formulation motivates CE by separating auxiliary variables and constraining them to share a common value, with positive weights summing to one.
- Consensus Equilibrium formulation: CE starts with multiple vector-valued maps and defines a solution through consensus of their outputs and balance of associated auxiliary vectors.
- Consensus Equilibrium for proximal maps: When each map is the proximal map of a proper closed convex function under the stated assumptions, CE solutions exactly match solutions of the corresponding consensus optimization problem.
- Beyond optimization: CE also applies when some maps are not proximal mappings and no underlying optimization problem exists, encompassing Plug-and-Play while allowing broader operators and algorithms.
- Beyond optimization: A given set of maps may produce multiple CE solutions; minimizing the weighted-average auxiliary vector's norm is proposed as an analogous selection principle.
3. Solving the Equilibrium Equations.
The paper reformulates Consensus Equilibrium as fixed-point and root-finding problems, then develops preconditioned Mann iteration and Newton-based solvers. Under stated conditions, these methods converge globally or locally, while anisotropic preconditioning can accelerate early convergence in some tomography problems.
- Fixed-point formulation: Consensus Equilibrium is reformulated as a fixed-point problem involving the composition T = (2Gµ − I)(2F − I).The reformulation is equivalent to solving the CE equations.
- Mann iteration and ADMM: Mann iteration solves the fixed-point problem, with ρ = 0.5 corresponding to ADMM up to a change of variables when N = 2.Other values of ρ yield over-relaxed or under-relaxed ADMM variants.
- Anisotropic preconditioning: Anisotropic preconditioning replaces scalar relaxation with a symmetric matrix H whose eigenvalues satisfy the theorem’s positivity and upper-bound conditions.The preconditioner is designed to weight different directions differently.
- Tomography application: In some tomography problems, choosing H appropriately can dampen visible oscillations and accelerate convergence during the initial iterations.The asymptotic rate is not significantly different from isotropic scalar scaling, but early iterations can improve.
- Convergence: Under nonexpansiveness and appropriate conditions on ρ or H, the iterative algorithms have guaranteed global convergence to a fixed point.A separate local-convergence theorem applies when the Jacobian eigenvalues at a fixed point have real parts strictly below one.
- Newton and alternative solvers: The paper also formulates CE as a root-finding problem and discusses Newton, Jacobian-free Newton–Krylov, and other candidate solvers.Further investigation of efficient forward-backward and related algorithms is left for future research.
4. Experimental Results.
The experiments test CE on toy equilibrium problems and image denoising, including cases where component maps are not proximal mappings. CE is evaluated through Newton-type and ADMM-related algorithms and through combinations of CNN denoisers trained for different noise levels.
- The examples include component maps that are not proximal mappings, making traditional optimization formulations inapplicable.
- 4.1. Toy model.: In the 2-dimensional toy model, Newton’s method trajectories converge to points satisfying the consensus equilibrium condition.At equilibrium, the corresponding map vectors share an endpoint and are equal in magnitude and opposite in direction.
- 4.2. Stochastic matrix.: For the stochastic-matrix examples, JFNK converges in both parameter settings, whereas ADMM converges only in the first.The first case has L(T) > 1 but eigenvalues with real parts below 1; the second has an eigenvalue with real part approximately 1.0039, causing ADMM divergence.
- 4.3. Image Denoising with Multiple Neural Networks.: The denoising experiment combines multiple CNN denoisers, none tuned to the noisy image’s actual noise level.The DnCNNs were trained at noise levels 10/255, 15/255, 25/255, 35/255, and 50/255, while experiments use other levels.
- 4.3. Image Denoising with Multiple Neural Networks.: In almost all cases, the full-group CE result has the highest PSNR among the individual DnCNN applications.CE is uniformly better than the linear-combination baseline and compares favorably with a noise-level-matched DnCNN in many cases, though the matched DnCNN is uniformly better for large sigma.
5. Conclusion.
Consensus Equilibrium defines reconstruction through balanced operators rather than cost minimization, while retaining agreement with MAP estimation when conventional optimization applies. Experiments show CE can solve non-optimization problems and sometimes outperform individual operators, including CNN denoisers.
- Framework: Consensus Equilibrium defines image reconstruction through balance among multiple operators rather than minimizing a cost function.The consensus vector arises simultaneously from operator balance and may incorporate varied image-processing operations.
- Framework: CE agrees with the usual MAP estimate for conventional regularized inversion but also applies when no corresponding optimization formulation exists.
- Algorithms: The paper introduces anisotropic preconditioned Mann iteration and Jacobian-free Newton Krylov methods for solving CE equations.It also presents a novel preconditioning method for accelerating Mann iterations.
- Experiments: Experiments across problems of varying complexity show that CE can solve problems lacking a corresponding regularized optimization formulation.
- Experiments: Consensus integration of multiple CNN denoisers achieved a better result than any individual denoiser.
A. Appendix: Proofs.
The appendix establishes optimization and convergence properties underlying Consensus Equilibrium. It connects CE to convex consensus optimization and analyzes convergence of preconditioned and relaxed iterations under stated operator assumptions.
- Consensus optimization: For convex lower-semicontinuous auxiliary functions, first-order KKT conditions are necessary and sufficient for optimality.
- Consensus optimization: When component operators are proximal maps, the CE relations can be converted into proximal-map conditions associated with the convex optimization formulation.The derivation uses subdifferentials, Lagrange multipliers, and resolvent equivalence.
- Preconditioning: A symmetric positive-definite preconditioner admits an orthogonal eigendecomposition and an associated weighted norm used in the convergence analysis.
- Convergence proofs: Under the fixed-point and nonexpansiveness assumptions, residuals tend to zero and the iteration converges to a fixed point.The proof establishes boundedness, existence of a limit point, and convergence in the weighted norm.
- Convergence proofs: The convergence proof shows that the iterates converge to a fixed point, with the weighted distance decreasing monotonically to zero.
- Local convergence: For sufficiently small relaxation parameters, the relaxed map has eigenvalues entering the open unit disk, enabling local geometric convergence.The Jacobian analysis uses Schur triangulation, diagonal scaling, and a neighborhood bound.
- Local convergence: The iterates of the relaxed map converge geometrically to the fixed point for initial values in a suitable neighborhood.