Source-linked AI summary
Probabilistic Noise2Void: Unsupervised Content-Aware Denoising
Alexander Krull, Tomas Vicar, Florian Jug
TL;DR
Image denoising often lacks the clean or repeated noisy image pairs required by supervised and Noise2Noise training, while Noise2Void’s single-image approach can underperform supervised models. PN2V trains CNNs to predict per-pixel intensity distributions, combines them with a general noise model, and uses the resulting probabilistic model for inference. Across microscopy datasets and multiple noise regimes, PN2V consistently exceeds self-supervised methods and can compete with supervised state-of-the-art methods.
Problem
Clean ground-truth images and repeated noisy realizations are often difficult to acquire, while self-supervised Noise2Void methods can fail to compete with supervised training.
Method
PN2V combines a general histogram-represented noise model with predicted samples representing possible true pixel intensities to form a per-pixel probabilistic model.
Results
PN2V consistently outperforms other self-supervised methods and can often compete with state-of-the-art supervised CARE methods across microscopy datasets and noise regimes.
Takeaways & Limitations
PN2V extends self-supervised CARE training to practical applications where the noise model can be determined from available images, including challenging low-light conditions.
Takeaways & Limitations
Noise2Void’s center masking reduces prediction accuracy because excluding the central pixel leaves less information available.
Abstract
from arXiv · showhide
Today, Convolutional Neural Networks (CNNs) are the leading method for image denoising. They are traditionally trained on pairs of images, which are often hard to obtain for practical applications. This motivates self-supervised training methods such as Noise2Void~(N2V) that operate on single noisy images. Self-supervised methods are, unfortunately, not competitive with models trained on image pairs. Here, we present 'Probabilistic Noise2Void' (PN2V), a method to train CNNs to predict per-pixel intensity distributions. Combining these with a suitable description of the noise, we obtain a complete probabilistic model for the noisy observations and true signal in every pixel. We evaluate PN2V on publicly available microscopy datasets, under a broad range of noise regimes, and achieve competitive results with respect to supervised state-of-the-art methods.
1 Introduction
Practical denoising often lacks clean or repeated noisy image pairs, motivating single-image self-supervision. PN2V extends this setting with general probabilistic noise and achieves results competitive with supervised methods.
- Supervised CARE models require pairs of clean and noisy images, but such paired data can be difficult to acquire.Clean ground-truth images may be impractical or impossible to obtain, and even two noisy realizations of the same content are often difficult to collect.
- Noise2Void trains exclusively on single noisy images by masking the receptive-field center, but assumes pixel-wise independent noise and predictable local signal.These assumptions allow self-supervised learning without clean targets or repeated observations.
- Self-supervised models can often not compete with supervised training, motivating probabilistic alternatives.A concurrent approach addressed this gap with Gaussian noise and per-pixel Gaussian intensity distributions.
- PN2V combines a histogram-represented noise model with predicted samples representing possible true intensities to compute each pixel’s posterior distribution.Unlike the concurrent Gaussian approach, PN2V is not restricted to Gaussian noise models or Gaussian intensity predictions.
- MMSE-PN2V consistently outperforms other self-supervised methods and can often achieve results competitive with supervised state-of-the-art CARE networks.The complete probabilistic model permits choosing a statistical estimator; this work uses MMSE estimates for final predictions.
2 Background
Image denoising reconstructs a clean signal from corrupted observations, while conventional and Noise2Noise training require paired data. Noise2Void removes the center pixel to enable single-image training, trading available information for reduced prediction accuracy.
- Image Formation and the Denoising Task: Image denoising seeks to recover a clean signal s from its corrupted observation x using a reconstruction function f(x) = ŝ ≈ s.
- Image Formation and the Denoising Task: Each observed pixel xi is independently drawn from p(xi|si), which is treated as an arbitrary noise-model observation likelihood.
- Traditional Training and Noise2Noise: Fully convolutional denoising networks predict each pixel from a limited surrounding receptive field rather than the entire image.The prediction is parameterized by network weights θ through a function of the receptive-field patch.
- Traditional Training and Noise2Noise: Traditional training uses noisy inputs with corresponding clean targets, whereas Noise2Noise uses two independently corrupted noisy images of the same signal.Noise2Noise can use repeated imaging of a static sample to obtain the paired noisy observations.
- Noise2Void Training: Noise2Void enables training from single noisy images by masking the receptive-field center, preventing the network from learning the identity transformation.The method assumes pixel-wise independent noise and local predictability of the true intensity.
- Noise2Void Training: Masking the center pixel reduces prediction accuracy because the network has less information available.
3 Method
PN2V trains a CNN to represent each pixel’s signal as a prior distribution, combines it with an arbitrary noise-model likelihood, and uses the resulting posterior for denoising estimates.
- Maximum Likelihood Training: PN2V masks the center pixel and trains a CNN to describe a probability distribution rather than directly predict a pixel estimate.The modified receptive field excludes the observed pixel at the target location.
- Probabilistic Model: The pixel posterior combines a prior over possible true intensities with an observation likelihood supplied by an arbitrary noise model.The prior is represented by predicted samples, while the likelihood models the noisy observation.
- Maximum Likelihood Training: PN2V trains the model by maximizing the probability of observed pixels given their masked surrounding receptive fields.The network parameters are tuned using the resulting unsupervised likelihood objective.
- Sample Based Prior: For each pixel, the CNN predicts K = 800 output values interpreted as independent samples from the prior distribution.This sample-based representation enables approximation of the likelihood objective and posterior inference.
- MMSE Inference: The final prediction uses the MMSE estimator, approximated by averaging predicted signal samples weighted by their observation likelihoods.The posterior is determined from the complete surrounding patch and the observed pixel value.
4 Experiments
PN2V is evaluated on single-channel microscopy datasets across five noise regimes using quantitative and qualitative comparisons with denoising baselines, with a documented data inconsistency in one dataset.
- Datasets: Experiments use single-channel datasets with 20 fields of view each, reserving one field for testing and using 19 for training and validation.The evaluation datasets come from Zhang et al. and contain different samples and imaging conditions.
- Noise Regimes: Each field of view contains 50 raw images with different noise realizations, plus four reduced-noise regimes formed by averaging 2, 4, 8, or 16 images.The raw regime is NR1, while the averaged regimes are NR2 through NR5.
- Evaluation Caveat: In the Two-Photon Mice dataset, heavy average-intensity fluctuations distort PSNR values and produce comparatively high standard errors across methods.A scale- and shift-invariant PSNR variant is additionally reported for this dataset.
- Qualitative Results: PN2V is compared with traditional U-Net, Noise2Void U-Net, and ground-truth data in qualitative results across three dataset images.Figure 2 presents rows for images and columns for raw inputs, zooms, model predictions, and ground truth.
- Quantitative Results: Table 1 reports PSNR (dB) ± 2 SEM averaged over all 50 images in each of five noise regimes for PN2V and baseline methods.The table groups supervised and non-supervised methods and marks the highest values in bold.
5 Discussion
PN2V extends self-supervised CARE with an arbitrary noise model and supports practical applications where supervised training data are difficult to obtain. Its performance consistently exceeds self-supervised methods and can often compete with state-of-the-art supervised methods.
- PN2V extends self-supervised CARE training with a fully probabilistic approach.
- PN2V uses an arbitrary noise model that can be determined from available images exposed to the same noise.
- This flexibility enables PN2V to address practical applications where supervised methods require difficult-to-acquire paired images.
- PN2V’s performance consistently exceeds self-supervised training and can often compete with state-of-the-art supervised methods.
- The authors identify challenging low-light conditions as an example application where noise limits downstream analysis.