Source-linked AI summary
A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising
Jun Xu, Lei Zhang, David Zhang
TL;DR
Real-world camera noise varies across signals, channels, patches, and camera settings, challenging methods designed for AWGN. TWSC extends sparse coding with three weight matrices, solves the resulting constrained problem using ADMM, and reports better performance than state-of-the-art denoising methods on realistic noise.
Problem
Realistic CCD or CMOS camera noise is complex, signal-dependent, and variable across cameras, settings, channels, and local patches, unlike AWGN.
Method
TWSC extends sparse coding with two data-fidelity weight matrices for noise statistics and one regularization weight matrix for natural-image sparsity priors.
Results
TWSC achieves much better performance than state-of-the-art denoising methods on real-world noisy-image datasets.
Takeaways & Limitations
Modeling channel-wise and locally signal-dependent noise within sparse coding provides a robust approach to real-world image denoising.
Takeaways & Limitations
Traditional sparse-coding denoisers remain limited for realistic noise because Frobenius-norm data fidelity cannot characterize its non-Gaussian, locally and channel-varying structure.
Abstract
from arXiv · showhide
Most of existing image denoising methods assume the corrupted noise to be additive white Gaussian noise (AWGN). However, the realistic noise in real-world noisy images is much more complex than AWGN, and is hard to be modelled by simple analytical distributions. As a result, many state-of-the-art denoising methods in literature become much less effective when applied to real-world noisy images captured by CCD or CMOS cameras. In this paper, we develop a trilateral weighted sparse coding (TWSC) scheme for robust real-world image denoising. Specifically, we introduce three weight matrices into the data and regularisation terms of the sparse coding framework to characterise the statistics of realistic noise and image priors. TWSC can be reformulated as a linear equality-constrained problem and can be solved by the alternating direction method of multipliers. The existence and uniqueness of the solution and convergence of the proposed algorithm are analysed. Extensive experiments demonstrate that the proposed TWSC scheme outperforms state-of-the-art denoising methods on removing realistic noise.
1 Introduction
Real-world camera noise is more complex and heterogeneous than AWGN, limiting conventional denoising methods. The paper proposes TWSC to model channel-wise and locally varying noise within sparse coding.
- Motivation: Real-world denoising supports photography and computer-vision tasks including segmentation, HDR imaging, and recognition.
- Motivation: Real-world CCD or CMOS noise can depend on signal, camera, settings, channels, and local patches.Examples include different noise levels across patches from the same white object and differing channel statistics.
- Motivation: Synthetic AWGN produces homogeneous noise patterns, unlike the heterogeneous statistics observed in real-world noisy images.
- Limitations of Existing Methods: Existing sparse-coding denoisers rely on ℓ2 or Frobenius data fidelity, which assumes white Gaussian noise.This limits their ability to characterize signal-dependent realistic noise.
- Proposed Approach: TWSC introduces two data-fidelity weight matrices and one regularization weight matrix to model realistic noise and natural-image sparsity priors.The scheme is reformulated as a linear equality-constrained problem and solved using ADMM, with theoretical analysis of solution existence, uniqueness, and convergence.
2 The Proposed Real-World Image Denoising Algorithm
The proposed TWSC denoiser extends sparse coding with three physically motivated weight matrices for channel-, patch-, and coefficient-dependent properties of real-world noise and image priors. It is optimized through variable splitting and ADMM, with convergence supported theoretically and empirically.
- 2.1 The Trilateral Weighted Sparse Coding Model: Real-world noise varies across channels and local patches, so conventional sparse coding's Frobenius-norm data fidelity cannot characterize it well.The method targets non-Gaussian, locally varying noise in images captured by CCD or CMOS cameras.
- 2.1 The Trilateral Weighted Sparse Coding Model: TWSC introduces W1 and W2 into the data-fidelity term to model channel- and patch-dependent noise statistics, and W3 into regularization to encode sparsity priors.W1 regularizes row discrepancies, W2 regularizes column discrepancies, and W3 is determined from the sparse-coefficient priors.
- 2.2 The Setting of Weight Matrices: The three diagonal weight matrices have explicit meanings: W1 represents channel properties, W2 represents patch noise variance, and W3 represents coefficient sparsity priors.Stronger noise in a channel or patch reduces its contribution to the denoised output.
- 2.2 The Setting of Weight Matrices: The noise model assumes channel- and patch-level Gaussian statistics, estimates σ_cm through a log-linear relation, and uses Laplacian coefficient priors scaled by singular values.The model sets σ_cm = σ_c^1/2 σ_m^1/2 and uses the inverse of each singular value as the coefficient-distribution scale factor.
- 2.3 Model Optimization: Variable splitting converts TWSC into a linear equality-constrained problem, which is solved by alternating ADMM updates for C, Z, the multiplier, and ρ.The Z update uses soft thresholding, while the C update is a two-sided weighted least-squares problem represented by a Sylvester equation.
- 2.4 The Denoising Algorithm: The objective is convex with a global optimum, and the convergence diagnostics approach zero simultaneously within 50 iterations on the reported test image.The stopping condition requires the three Frobenius-norm criteria to be at most Tol, or the iteration count to reach K1.
3 Existence and Faster Solution of Sylvester Equation
The paper establishes uniqueness for the Sylvester equation used in its method and develops a faster exact solver for high-dimensional cases. The proposed approach reduces computational complexity substantially while preserving the exact solution.
- Existence and uniqueness: A unique solution exists for the Sylvester equation when its coefficient matrices are symmetric positive definite.This follows from a theorem requiring both matrices to be symmetric positive semidefinite, with at least one positive definite.
- Computational bottleneck: The conventional Bartels-Stewart approach is computationally expensive because it factorizes matrices with dimensions 3p^2M × 3p^2M.Its QR-factorization cost is O(p6M^3).
- Faster exact solution: The paper exploits the problem’s structure to compute an exact Sylvester-equation solution without the expensive general-purpose factorization.The method eigen-decomposes A and computes the inverse on diagonal elements after transforming the equation.
- Faster exact solution: The proposed solver reduces complexity from O(p6M^3) to O(max(p6, p2M)).The paper describes this as a substantial computational saving for solving the Sylvester equation.
4 Experiments
Experiments evaluate TWSC on synthetic AWGN and three real-world noisy-image datasets, using quantitative metrics where ground truth is available and visual comparisons otherwise. TWSC generally matches strong AWGN denoisers, substantially outperforms WSC and competing real-noise methods, but is considerably slower than optimized alternatives.
- Experimental Settings: TWSC is evaluated on synthetic AWGN images and three real-world noisy-image datasets captured by CCD or CMOS cameras.The real-world datasets include uncontrolled images, controlled 500-shot camera scenes, and DND image pairs.
- Experimental Settings: Noise standard deviations are estimated per color channel for TWSC, while fixed parameters are used across the experiments.The implementation fixes parameters according to noise-standard-deviation ranges and uses exploratory channel-wise noise estimation.
- Results on AWGN Noise Removal: On 20 grayscale images corrupted by AWGN, TWSC achieves comparable PSNR and SSIM performance to WNNM, TNRD, and DnCNN, while outperforming WSC.Unlike TNRD and DnCNN, TWSC uses only the tested noisy image; the comparison also attributes WSC’s weaker performance to its treatment of local noise statistics.
- Results on Realistic Noise Removal: On Dataset 1, TWSC removes noise more clearly and preserves image details better than competing methods in visual comparisons without ground-truth clean images.Several methods produce color artifacts or oversmoothing, whereas TWSC is reported to maintain details more effectively.
- Results on Realistic Noise Removal: On Dataset 2, TWSC achieves much better average PSNR and SSIM than competing methods, including WSC, CC, and MCWNNM, with improved visual quality.Dataset 2 uses means of 500 repeated shots as approximate ground truth for evaluation.
- Results on Realistic Noise Removal: On the 1000 cropped images in Dataset 3, TWSC again achieves much better PSNR and SSIM and better visual quality than competing denoising methods.The dataset’s ground-truth images are not publicly released, but benchmark results can be obtained through the project website.
- Comparison on Speed: TWSC requires about 195 seconds to process a 512 × 512 DND image, compared with about 1.1 seconds for Neat Image.The comparison uses Matlab2014b; several competing implementations are compiled or parallelized, whereas TWSC is implemented purely in Matlab.
- Visualization of The Weight Matrices: Weight-matrix visualizations show that W1 reflects image noise levels, W2 varies with patch-dependent noise, and W3 is determined mainly by patch structure.Patch-wise variation is much stronger for real-world noise than for synthetic AWGN with similar noise levels across patches.
5 Conclusion
TWSC models realistic camera noise and natural-image sparsity through three weighted sparse-coding matrices, solved with ADMM. Experiments show superior denoising performance over existing state-of-the-art methods, including real-world-noise methods.
- TWSC exploits noise properties across different channels and local patches using three weight matrices.Two matrices adaptively characterize channel- and patch-level noise statistics, while the third improves the use of sparsity priors.
- The method solves the weighted sparse-coding problem under the ADMM framework.The solution to the resulting Sylvester equation is guaranteed.
- TWSC achieves superior performance over existing state-of-the-art denoising methods, including methods designed for realistic real-world noise.