Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems

• Replacing proximal operators in convex optimization algorithms with a denoising network lets the same network serve different reconstruction tasks by changing the data fidelity term, without retraining.Source: Figure 1 · p. 1, §1 · p. 2
• A fixed denoising network used as a proximal operator in PDHG yields state-of-the-art results close to methods trained for specific problems.Source: §1 · p. 2
• The paper analyzes different optimization algorithms for incorporating neural networks and shows that their resulting fixed points coincide.Source: §1 · p. 2
• The paper provides insights into how the algorithm’s step size and the denoising network’s strength influence the final result.Source: §1 · p. 2

2. Related work

• Total variation is a classical regularizer that penalizes the image-gradient norm and preserves image discontinuities.Classical variational methods use regularizers designed to suppress noise while preserving important image features.Source: §2 · p. 2
• Prior work replaced regularization proximal operators with denoisers such as NLM or BM3D, and analyzed ADMM convergence conditions for custom proximal operators [32].This approach was motivated by interpreting the proximal operator as denoising of b.Source: §2 · p. 2
• Deep convolutional denoising networks are proposed as proximal operators, extending prior patch-based approaches and Gaussian-mixture patch learning [40].The paper also analyzes the networks’ behavior numerically and theoretically.Source: §2 · p. 2
• Customized proximal operators have been applied to Poisson denoising [32], bright field electron tomography [37], super-resolution, and hyperspectral image sharpening.These works focused on patch-based denoising methods as proximal operators.Source: §2 · p. 2

3.1. Motivation via MAP estimates

• MAP reasoning makes data fidelity depend on the forward operator and noise model, while the difficulty of modeling natural-image likelihoods makes regularization a candidate for learning.For Gaussian noise, the data term is a squared ℓ2 norm; hand-crafted gradient penalties such as TV cannot fully capture complex natural images.Source: §3.1 · p. 3

3.2. Algorithms for learned proximal operators

• Many convex optimization methods require only the regularizer’s proximal operator, which can be interpreted as Gaussian MAP denoising and replaced by a neural network in PG, ADMM, or PDHG schemes.The paper considers PG, ADMM, and two PDHG variants for incorporating the learned operator.Source: §3.2 · p. 3
• Convergence guarantees require sufficiently friendly convexity or specific nonconvex assumptions, yet modified schemes are reported stable in many cases and share equivalent fixed-point equations for continuous G.The convergence analysis beyond those settings is outside the paper’s scope.Source: §3.2 · pp. 3–4

3.3. Parameters for learned proximal operators

• Any γ > 0 in PDHG2 is update-equivalent to γ = 1 with a reweighted data-fidelity term; across the four schemes, step sizes simply rescale that term.For iterates to coincide, initialization must satisfy y0 = 0; the authors propose fixing denoising strength and tuning data fidelity to control smoothness.Source: §3.3 · pp. 4–5
• Figure 2 shows optimal data-fidelity α varies approximately quadratically with network training noise σ, while PSNR stays stable across sufficiently large denoising strengths.Networks trained on very small noise performed poorly, whereas sufficiently large σ gave good results across a broad range of training noise levels.Source: §3.3 · p. 4, Figure 2 · p. 5

4.1. Algorithmic framework and prior stacking

• PDHG supports prior stacking by combining an implicit neural-network prior with explicit, application-tailored regularizers, following.The scheme introduces variables z and y for the network G and an additional regularizer J, which may itself contain multiple priors.Source: §4.1 · p. 5
• For the stacked-prior scheme, rescaling β and α eliminates arbitrary γ = cτ, leaving data-fidelity and regularization parameters to optimize.Here c usually denotes the operator norm ∥[I, −DT]∥₂; the authors do not treat the product of step sizes as a free parameter.Source: §4.1 · p. 5

4.2. Deep convolutional denoising network

• The denoiser is a 17-layer CNN similar to DnCNN-S [48], trained with Adam [21] to remove Gaussian noise of fixed standard deviation σ.Each 3×3 convolution layer is followed by a ReLU; the input is grayscale or color depending on the application.Source: §4.2 · p. 5
• The trained denoisers achieve state-of-the-art performance, with results described as having similar margins to BM3D across noise levels.Each noise level σ requires an individually trained DnCNN-S; the comparison reports average PSNR over 11 test images.Source: Table 1 · p. 6, §4.2 · p. 6

5. Evaluation

• The fixed network was trained only for Gaussian denoising at σf = 0.02; evaluation covered noise-free Bayer demosaicking and five deconvolution experiments on 11 standard images.Demosaicking used cross-channel and total-variation priors; the deconvolution experiments varied blur kernels and noise.Source: §5 · p. 6, §5.1 · p. 6, §5.2 · pp. 6–7

5.1. Demosaicking

• On McMaster demosaicking, the method achieved very high average PSNR, surpassed only by the demosaicking-trained CNN, and gained about 1 dB over FlexISP∗.The reported denoising advantage over BM3D carried over to the demosaicking inverse problem.Source: §5.1 · p. 6, Figure 4 · p. 7
• Across differently trained denoisers, demosaicking PSNR varied by about 1.1 dB but remained above 36 dB; omitting explicit priors reduced average performance by about 0.4 dB.The grid-search parameters suggested using explicit priors less when the denoiser was trained on larger noise levels.Source: §5.1 · p. 6, Table 2 · p. 7

5.2. Deconvolution

• The denoiser’s advantage over BM3D on plain denoising did not fully transfer to deconvolution, and the conditions for full transfer remain an open question.The authors identify understanding which problems permit full transfer as future research.Source: §5.2 · pp. 7–8
• With the same σ = 0.02 denoiser across five experiments, the method performed on par with other deconvolution methods, matched specialized MLPs in experiments a–d, and outperformed them on motion blur e.The comparison used individual parameter sets for each experiment; FlexISP∗ also used a separately optimized step size γ.Source: §5.2 · p. 7, Table 3 · p. 8
• Deconvolution PSNR remained stable across a large range of denoiser training noise levels, indicating robustness to the specific network used.The paper reports this robustness in the second plot of Table 3.Source: Table 3 · p. 8, §5.2 · p. 8
• Deconvolution runtime averaged ≈2.5 s for the proposed method versus ≈4 s for FlexISP∗, a relative improvement of 37.5%.In both cases, the denoising operator ran on the GPU.Source: §5.2 · p. 8

6. Conclusion

• Four algorithms using neural networks as proximal operators have the same potential fixed-points, while PDHG step size merely rescales data-fidelity and other regularization parameters.The network's training noise level acts much like a MAP-derived regularization parameter, with a quadratic relation between noise standard deviation σ and the data-fidelity parameter.Source: §6 · p. 8
• PDHG with DnCNN-S [48] as a proximal operator and prior stacking achieved state-of-the-art results and robustness on exemplary demosaicking and deblurring problems using a fixed network.The authors report that these results indicate reduced need for problem-specific training.Source: §6 · p. 8
• The concept may ease problem-specific retraining and enable learned natural-image priors for problems without available training data.This is presented as an expected benefit of the approach.Source: §6 · p. 8

• It reports grid-search parameter values, reconstruction PSNRs, and images for both exemplary inverse-reconstruction problems.The supplementary material also provides the proof of Remark 3.1.Source: Abstract · p. 11

Proof of Remark 3.1

• With an arbitrary continuous function G replacing the regularization proximal operator, PG, ADMM, PDHG1, and PDHG2 have equivalent fixed-point equations.The shared equation uses t = τ for PG and PDHG2, and t = 1/γ for ADMM and PDHG1.Source: Proof of Remark 3.1 · p. 11
• For ADMM, PDHG1, and PDHG2, fixed points correspond to solutions of the shared equation, with suitable initializations recovering algorithm fixed points.The proof derives the correspondence from each scheme’s update or optimality conditions and specifies initial variable choices for the converse direction.Source: Proof of Remark 3.1 · pp. 11–12
• For θ = 1, the algorithm is equivalent to the above ADMM algorithm in the convex case with proximity operators; this result also remains valid when the proximity operator is replaced by a neural network.The convex-case relationship is cited to, Section 5.3.Source: Proof of Remark 3.1 · p. 12

Demosaicking

• On 18 noise-free Bayer-filtered McMaster images, the fixed σ = 0.02 network yields channel-wise PSNR results, with green-channel reconstruction attributed to RGGB pattern dominance.The paper refers readers to supplementary material of and for full comparisons with the state-of-the-art methods mentioned in the main paper.Source: Demosaicking · p. 13, Table 1 · p. 13

Deconvolution

• Across five deconvolution experiments on 11 standard test images, the paper reports PSNR results for its approach and FlexISP* alongside networks trained at different noise levels.The experiments use different blur kernels and Gaussian noise levels; the fixed-network examples use σ = 0.02. The experiments were proposed in.Source: Deconvolution · p. 13, Table 3 · p. 16
• For deconvolution, an extensive grid search reports optimal data-fidelity α and regularization β_TV values across networks trained at different σ levels.Following Proposition 3.2, the PDHG dual step size is γ = 1.0, and the primal step size τ is chosen to satisfy τγ < c, where c is the squared norm of the linear operator.Source: Deconvolution · p. 13, Table 2 · p. 14