Source-linked AI summary
Training Deep Learning Based Denoisers without Ground Truth Data
Shakarim Soltanayev, Se Young Chun
TL;DR
Noiseless ground truth can be difficult or infeasible to obtain for some denoising applications, despite its usual role in training DNN denoisers. The paper uses SURE to train and refine denoisers from noisy images, reporting performance comparable to ground-truth-trained networks and gains from refining with a noisy test image.
Problem
DNN denoisers are typically trained against noiseless ground truth, which is expensive or sometimes infeasible to obtain in applications such as hyperspectral remote sensing and medical imaging.
Source: §1 · p. 1Method
The paper proposes SURE-based unsupervised training and noisy-test-image refining for DNN denoisers, and investigates training for Poisson noise.
Source: §1 · p. 2, §6 · p. 11Results
SURE-trained denoisers achieved performance comparable to the same denoisers trained with noiseless ground truth and outperformed BM3D; noisy-test-image refining further improved performance, outperforming BM3D, deep image prior, and often ground-truth-trained networks.
Source: §6 · p. 11Takeaways & Limitations
The methods may be useful when many noisy images but few noiseless images are available, and SURE-FT can combine pretrained denoisers with learning image-specific structure.
Source: §5 · pp. 10–11Although the SURE risk estimator is unbiased for fixed θ, the estimator tuned by SURE is not guaranteed to be unbiased; regularization can address a possible lack of a minimizer.
Source: §5 · p. 11
Abstract
from arXiv · showhide
Recently developed deep-learning-based denoisers often outperform state-of-the-art conventional denoisers such as the BM3D. They are typically trained to minimize the mean squared error (MSE) between the output image of a deep neural network (DNN) and a ground truth image. Thus, it is important for deep-learning-based denoisers to use high quality noiseless ground truth data for high performance. However, it is often challenging or even infeasible to obtain noiseless images in some applications. Here, we propose a method based on Stein's unbiased risk estimator (SURE) for training DNN denoisers based only on the use of noisy images in the training data with Gaussian noise. We demonstrate that our SURE-based method, without the use of ground truth data, is able to train DNN denoisers to yield performances close to those networks trained with ground truth for both grayscale and color images. We also propose a SURE-based refining method with a noisy test image for further performance improvement. Our quick refining method outperformed conventional BM3D, deep image prior, and often the networks trained with ground truth. Potential extension of our SURE-based methods to Poisson noise model was also investigated.
1 INTRODUCTION
Deep neural network denoisers can match or outperform conventional methods, but their usual supervised training depends on noiseless ground truth that can be costly or infeasible to acquire. The paper extends SURE-based unsupervised denoising to color images, test-image refinement, and Poisson noise.
1 INTRODUCTION
- Image denoising reduces noise and can improve the execution of image classification and image restoration tasks.Source: §1 · p. 1
- DNN denoisers typically minimize MSE against noiseless images, making high-quality ground truth important for high performance.In daily-environment photography, high-quality sensors and abundant light make almost noiseless images relatively easy to obtain.Source: §1 · p. 1
- Noiseless ground truth is expensive or infeasible to obtain in applications such as hyperspectral remote sensing and medical imaging.Hyperspectral sensors can have increased noise, while longer acquisitions are challenging; higher CT radiation doses can harm subjects or saturate detectors.Source: §1 · p. 1
- Prior alternatives have trade-offs: Noise2noise needs two noise realizations per image, while DIP uses a noisy image but has lower performance than BM3D and requires architectural design choices.The authors’ earlier SURE-based Gaussian training method requires only one noise realization per image.Source: §1 · p. 1
- SURE has also been used to tune conventional denoiser hyperparameters, with analytical, Monte Carlo, and weak-gradient approaches studied in low-dimensional parameter spaces.These approaches target optimal or near-optimal denoisers with hyperparameters.Source: §1 · pp. 1–2
- The proposed work extends SURE-based unsupervised denoiser training to color images, test-image refinement, and an investigation of Poisson noise.The paper reports that its simulations show state-of-the-art performance over conventional methods and often over ground-truth-trained DNN methods.Source: §1 · p. 2
2 BACKGROUND
The background reviews SURE as an MSE estimator for Gaussian-noise denoising, a Monte Carlo approximation for its divergence term, and the empirical-risk basis of supervised DNN training.
2.1 Stein’s unbiased risk estimator (SURE)
- For fixed x, SURE provides an unbiased estimate of denoising MSE under Gaussian noise, with the denoiser written as h(y).The noise is described as i.i.d. Gaussian with covariance σ^2I; SURE uses the denoiser output, but its divergence term is challenging to calculate analytically for general denoisers.Source: §2.1 · p. 2
2.2 Monte-Carlo Stein’s unbiased risk estimator
- Monte Carlo SURE approximates the divergence with one independent Gaussian perturbation and a small positive ε, subject to regularity conditions on h(y).The approximation is reported to yield accurate unbiased MSE estimates for many conventional denoisers.Source: §2.2 · p. 2
2.3 Supervised training of DNN based denoisers
- Supervised DNN denoisers use an empirical risk based on paired noiseless and noisy training data because the expected risk is infeasible to calculate exactly.The empirical risk is an unbiased estimator of the expected risk; gradient-based optimization trains the network parameters.Source: §2.3 · p. 2
- Mini-batch empirical risk supports efficient training, and image patches can be used instead of whole images.Randomly permuting the training data each epoch preserves the mini-batch estimate’s unbiasedness for the expected risk.Source: §2.3 · pp. 2–3
3 METHODS
The methods replace ground-truth training risk with a noisy-data SURE-based objective for Gaussian denoisers. They extend this approach to blind denoising, test-image refinement, and Poisson noise.
3.1 Unsupervised training of DNN based denoisers
- MC-SURE replaces ground-truth risk with a noisy-data risk for Gaussian denoiser training, using up to 2MP rather than KMP derivative evaluations per minibatch.The authors say available training and test data can be used, and expect applicability to most DNN denoisers satisfying Theorem 2.Source: §3.1 · p. 3, §3 · p. 3
3.2 Unsupervised training of blind image denoisers
- Blind denoising extends the empirical risk to patches or images spanning different noise levels, with convergence requiring a finite noise-level range.The stated sufficient condition is that the relevant sequence converges; the authors say any blind denoiser with a finite range of noise levels should meet it.Source: §3.2 · p. 3
3.3 Unsupervised refining of DNN based denoisers
- A trained denoiser can be refined for a noisy test image by minimizing a SURE-based objective while fixing selected parameters.The mask B selects parameters to fix, with batch-normalization layers given as an example.Source: §3.3 · p. 4
3.4 Unsupervised training for Poisson noise
- For Poisson denoising, the authors derive an unbiased risk estimator for pure Poisson noise using prior PURE and PG-URE estimators.The extension is introduced through a mixed Poisson-Gaussian model in which z is Poisson-distributed and ζ is the acquisition gain.Source: §3.4 · p. 4
4 SIMULATION RESULTS
The simulations compare SURE- and MSE-trained SDA and DnCNN denoisers on MNIST and natural-image datasets, and test color denoising, noisy-image fine-tuning, and Poisson-noise extension.
4.1 Results: MNIST dataset
- On MNIST, SDA-SURE was within 0.01 dB of SDA-MSE-GT and outperformed BM3D at σ = 25 and 50.At σ = 50, visual inspection also found SDA-SURE indistinguishable from SDA-MSE-GT and less blurry than BM3D.Source: Table 2 · p. 4, §4.1 · p. 4, Figure 1 · p. 5
4.2 Regularization effect of DNN denoisers
- SDA-REG scored 25.07 dB at σ = 25 and 19.85 dB at σ = 50, about 2 dB below BM3D, showing that network structure alone did not explain SDA’s strong performance.Visible noise patterns remained despite regularization.Source: §4.2 · p. 5
4.3 Accuracy of MC-SURE approximation
- For MNIST SDA, ϵ values from 10^-2 to 10^-7 worked, but suitable ϵ required tuning for DnCNN; SURE loss became too noisy and diverged from MSE trends after σ = 350.Above σ = 300, both SDA denoisers and BM3D could not produce recognizable digits.Source: §4.3 · p. 5, Figure 2 · p. 5, Figure 3 · p. 5
4.4 Results: high resolution natural images dataset
- On Set12, SURE denoisers matched or beat BM3D at σ = 25 and 50 but trailed it by 0.19 dB at σ = 75; on BSD68, DnCNN-SURE beat BM3D by 0.11–0.41 dB at every noise level.BM3D’s patch matching favors repeated patterns and flat areas, while DnCNN-SURE avoids rare-patch effects; SURE training took about 11 hours versus 7 for MSE-GT.Source: Table 3 · p. 6, Table 4 · p. 6, §4.4 · p. 6
4.5 Extension to blind color image denoising
- On CBSD68, both deep-learning methods substantially outperformed CBM3D, and CDnCNN-SURE showed superior performance to CBM3D.For σ = 50, CBM3D produced false-color artifacts, while CDnCNN methods preserved colors and fine details.Source: Table 5 · p. 7, §4.5 · pp. 7–8
4.6 Unsupervised refining (fine-tuning) with SURE
- SURE-FT improved CDnCNN-SURE on almost all test images, often by more than 1 dB, and outperformed all methods on average at both noise levels.It also beat BM3D and DIP, and outperformed CDnCNN-MSE-GT in many images and on average.Source: Table 6 · p. 8, §4.6 · p. 8
4.7 Extension to Poisson noise denoising
- For Poisson noise, SDA-PURE matched BM3D+VST at ζ = 0.01 and beat it at ζ = 0.1 and 0.2, while SDA-MSE-GT performed best; PURE approximation became inaccurate above ζ = 0.1 and training failed above 0.2.The comparison used MNIST and the conventional BM3D+VST method.Source: Table 7 · p. 10, §4.7 · p. 10
5 DISCUSSION
SURE-FT adapts pretrained denoisers to noisy test images, combining information from large training datasets with learning image-specific structure. The discussion notes limits in noise assumptions and SURE optimization, and identifies further work for real and synthetic noise.
5 DISCUSSION
- The methods may be useful when many noisy images but few noiseless images are available; high-performance denoising for synthetic and real noise remains future work.The authors frame integration with newer DNN architectures as a possibility, not a demonstrated result.Source: §5 · p. 10
- SURE-FT outperformed DIP for Gaussian denoising and can refine pretrained denoisers, while DIP needs a special architecture for high performance.Training from scratch on a single noisy test image did not yield good denoising performance.Source: §5 · pp. 10–11
- SURE-FT combines information from large training datasets with learning the unique structures of an input image.The authors suggest applying it to pretrained networks such as CDnCNN-SURE.Source: §5 · p. 11
- The simulations assumed known Gaussian noise level σ or Poisson parameter ζ; applying the method directly in the image domain with correlated noise requires further investigation.The discussion notes generalized SURE as a potential route to other noise types.Source: §5 · p. 11
- SURE optimization may lack a finite minimizer, and unbiasedness of the fixed-θ estimator does not guarantee an unbiased denoiser after tuning.The paper notes weight decay, network regularity, suitable ϵ, or proper training data as ways to address the minimizer issue in practice.Source: §5 · p. 11
6 CONCLUSION
The paper proposes SURE-based unsupervised training and noisy-test-image refining for deep denoisers. The authors report performance comparable to ground-truth-trained networks or better in many cases, and demonstrate potential extension to Poisson noise.
6 CONCLUSION
- SURE-trained denoisers performed comparably to the same denoisers trained with noiseless ground truth and outperformed BM3D; SURE-based refining further improved performance, outperforming BM3D and DIP and often ground-truth-trained networks.These results were reported for grayscale and color images; potential extension to Poisson noise was also demonstrated.Source: §6 · p. 11