Deep Mean-Shift Priors for Image Restoration

1 Introduction

• The proposed prior represents a Gaussian-smoothed estimate of the natural-image distribution, rather than relying on image statistics such as sparse gradients.The paper notes that the true distribution cannot be used directly because only a finite set of samples is available.Source: §1 · p. 1
• A denoising autoencoder learns the mean-shift vector field, proportional to the gradient of the log prior, enabling gradient-descent Bayes risk minimization.The framework formulates restoration as a Bayes estimator that includes the smoothed natural-image distribution.Source: §1 · pp. 1–2
• The prior captures image structures with varied curvature, while other visualized priors favor simpler lines or sharp corners.Figure 1 compares the priors using the visualization method of Shaham et al. [30].Source: §1 · p. 1, Figure 1 · p. 2
• The method achieves competitive results for noise-blind deblurring, super-resolution, and image demosaicing.The paper highlights a Bayes-estimator formulation that also supports noise-blind restoration.Source: §1 · p. 2

Image Priors

• CNN-based restoration methods often learn proximal operators, but their relation to the natural-image probability distribution remains unclear.The cited work also reports that no single proximal method systematically outperforms the others.Source: §2 · p. 2
• The proposed approach explicitly uses the Gaussian-smoothed natural-image distribution as a prior and learns its log-gradient with a denoising autoencoder.Romano et al. [25] also use a denoiser whose residual matches the approach's gradient, but do not establish the same probability-distribution connection; Bigdeli and Zwicker [4] do not address noise-blind restoration.Source: §2 · p. 2

Noise- and Kernel-Blind Deconvolution

• Earlier noise-blind deblurring approaches estimate the noise level first or adapt regularization without a noise-variance estimate.The excerpt describes Jin et al. as using adaptive regularization and Zhang et al. as exploring spatially adaptive sparse priors and scale space.Source: §2 · pp. 2–3
• The cited deconvolution methods are task-specific and handle noise-blind or kernel-blind cases, but not the fully blind setting supported by this paper.The paper distinguishes fully blind restoration as covering both noise and kernel blindness.Source: §2 · pp. 2–3

3 Bayesian Formulation

• The Bayes-estimation formulation uses the Gaussian-smoothed natural-image distribution as a prior and allows noise-blind restoration.The prior is the logarithm of the smoothed distribution, and the formulation is designed to address noise-blind problems.Source: §3 · p. 3, §3.1 · p. 4

3.1 Defining the Objective via a Bayes Estimator

• The objective maximizes posterior expected utility, generalizing MAP through a utility function that encourages the estimate and latent image to be similar.The logarithm of the expected utility is bounded to separate a data term and an image likelihood.Source: §3.1 · pp. 3–4
• Because only image samples are available, the method smooths the natural-image distribution with Gaussian noise and learns the gradient of its logarithm.The smoothing variance is σ², and the resulting density is similar to a kernel density estimate.Source: §3.1 · pp. 3–4

3.2 Gradient of the Prior via Denoising Autoencoders (DAE)

• The DAE output is related to the smoothed data distribution by a continuous mean-shift operation, linking its output-minus-input residual to the prior gradient.The DAE is trained with noise variance σ²; the approach uses its error to compute the prior gradient.Source: §3.2 · p. 4

3.3 Stochastic Gradient Descent

• The DAE overfits noisy images and does not produce meaningful results for inputs lacking noise close to its training σ.The authors address this issue by reformulating the prior as a lower bound and adding noise before evaluating the DAE.Source: §3.3 · p. 5
• The stochastic prior gradient is evaluated using one sampled noise value because computing the noise integral at runtime is not possible.The sampled-noise evaluation addresses the overfitting issue by adding noise before each DAE evaluation.Source: §3.3 · p. 5
• The optimization uses momentum gradient descent, combining the data and prior gradients in each update with a running step and step size.The running step uses momentum μ, and α is the step size.Source: §3.3 · p. 5

• The section summarizes gradient-descent steps for different restoration tasks in Table 1, omitting downsampling for super-resolution and masking.Source: §4 · p. 5
• Table 2 reports average PSNR (dB) for non-blind deconvolution on two datasets.Source: Table 2 · p. 6

Non-Blind Deblurring (NB)

• Known-kernel, known-noise non-blind deblurring uses gradient-descent steps summarized in Table 1; K denotes the blur kernel’s Toeplitz matrix.Source: §4 · p. 6

Noise-Adaptive Deblurring (NA)

• When noise variance is unknown, the method optimizes it independently of the prior and substitutes it into the data term, which is adaptively scaled relative to the prior.Source: §4 · p. 6

Noise- and Kernel-Blind Deblurring (NA+KE)

• Noise- and kernel-blind deblurring jointly optimizes the unknown image and kernel, projecting the kernel to be nonnegative and L1-normalized after each step.Source: §4 · p. 6

5 Experiments and Results

• The implementation uses 300 iterations by default and runs in time linear in pixel count, at about 0.2 seconds per iteration per megapixel on an Nvidia Titan X (Pascal).Unless otherwise specified, the step length is 0.1 and momentum is 0.9.Source: §5 · p. 6

5.1 Image Deblurring: Non-Blind and Noise-Blind

• Noise-adaptive deblurring averaged better than the state of the art across experiments, with sharp textures and preserved natural image structure.The comparison covers 32 Levin images and 50 Berkeley images; Figure 2 provides a visual comparison.Source: §5.1 · pp. 6–7, Figure 2 · p. 7

5.2 Image Deblurring: Noise- and Kernel-Blind

• All images in Levin’s fully blind deconvolution set were reconstructed with error ratios below 3.5, using end-to-end image and kernel estimation without an assumed input noise level.The optimization avoids the common two-stage sequence of kernel estimation followed by non-blind deconvolution.Source: §5.2 · p. 7, Figure 3 · p. 7

5.3 Super-resolution

• The method performed on par with state-of-the-art methods across upsampling scales on Set5 and Set14.These tests use no degradation noise; the prior weight is gradually reduced to zero.Source: §5.3 · p. 7, Table 3 · p. 8

5.4 Demosaicing

• On noisy Panasonic Bayer RGGB images, the method performed on par with the state of the art and produced smoother results; its prior is not tied to a specific color-filter subpixel order.The test set contains 100 images, and the qualitative comparison is shown in Figure 4.Source: Table 4 · p. 8, §5.4 · p. 8, Figure 4 · p. 8

6 Conclusions

• The framework uses a generic prior representing the Gaussian-smoothed natural-image distribution for image restoration.A trained denoising autoencoder computes the prior gradient efficiently.Source: §6 · p. 8
• A single learned prior is used for noise-blind deblurring, super-resolution, and image demosaicing.Source: §6 · p. 8
• The reported results are competitive with the state of the art for these restoration applications.The authors identify other smoothing kernels, multiscale optimization, and inverse problems beyond image restoration as future directions.Source: §6 · p. 8