Revisiting Bayesian Blind Deconvolution

1. Introduction

• The paper assumes a convolutional blur model with additive noise and adopts priors to constrain the ill-posed solution space.Its framework uses a zero-mean Gaussian noise model with covariance λI, while noting that this assumption can be relaxed.Source: §1 · p. 2
• Blind deconvolution estimates both the sharp image and blur kernel from a blurry observation, but many image/kernel pairs can explain the data equally well.Even if the kernel were known, recovering the image is ill-posed because blurring loses typically high-frequency information.Source: §1 · p. 2
• The paper reviews MAP and variational Bayes, then analyzes their limitations and reformulates them to clarify VB and suggest improvements, including criteria for image-prior selection.It also studies noise-variance learning and provides experiments to corroborate some theoretical claims.Source: §1 · p. 2

2. MAP versus VB

• MAP methods can suffer from poor minima and convergence issues, requiring delicate balancing of noise levels, trade-off parameters, and heuristic regularizers.The section notes that natural-image-statistic priors have been used in MAP deblurring.Source: §2 · p. 3
• VB approximates the intractable marginalization over the image to bound the negative log marginal likelihood and optimize the kernel.Marginalization is motivated by the image’s higher dimensionality relative to the kernel, allowing estimation to focus on the lower-dimensional kernel parameters.Source: §2 · pp. 3–4
• VB’s free-energy optimization is difficult to interpret, and exact optimization over its distribution-valued arguments and the kernel is generally unavailable.The section introduces structural factorization assumptions to make updates computable, but these yield a looser bound and no guarantee of minimizing the original marginal likelihood.Source: §2 · pp. 4–5
• Practical VB further uses a diagonal covariance approximation because computing the full image covariance requires O(m^3) operations.The diagonal approximation adds another factorization to the VB process.Source: §2 · p. 5
• The factorized approximation can be poor when the posterior is highly coupled, so its estimated kernel may not maximize the marginal likelihood.The section expresses this discrepancy through the KL-divergence gap between the bound and the marginal likelihood.Source: §2 · p. 5
• The paper investigates VB’s mechanism because its empirical advantage over MAP is difficult to explain from the opaque free-energy objective.In direct comparisons using the same basic probabilistic model, VB can perform substantially better, including state-of-the-art performance without additional tuning (Babacan et al., 2012; Levin et al., 2011b,a).Source: §2 · pp. 5–6
• In ideal noiseless settings, the underlying VB and MAP costs are equivalent under the factorial assumptions, while VB’s intrinsic mechanism can largely avoid bad local minima.The paper argues this advantage is not directly due to marginalizing a high-dimensional image; it also recasts VB for easier incorporation of generic non-Gaussian noise models and other extensions.Source: §2 · p. 7
• The paper’s conclusions formally concern the uniform blur model, though it reports similar principles in more complex non-uniform and multi-image models.The cited preliminary results are Zhang and Wipf (2013a,b).Source: §2 · p. 7

3.1 Notation and Definitions

• The analysis defines image derivatives as the signal representation and relative concavity as a way to order sparsity-promoting penalties; smaller ℓp exponents favor greater sparsity.For a sharp image of size M × N and kernel P × Q, the vector sizes are m = MN, n = (M − P + 1)(N − Q + 1), and l = PQ.Source: §3.1 · pp. 8–9

3.2 Connecting VB with MAP

• VB is equivalent to coordinate descent on an objective with a non-separable image penalty that depends on both noise level λ and blur kernel k; at λ = 0, its penalty matches MAP.The equivalence holds for the same prior selection; in the noiseless setting, the underlying VB cost function is effectively no different from regular MAP.Source: §3.2 · pp. 9–10

3.3 Illustrative Example using 1D Signals

• Although MAP and VB share local minima at λ = 0, VB's coupled penalty improves the positive-λ convergence path; a synthetic 1D example shows better image and kernel recovery.The comparison uses the same image prior and λ-reduction schedule; VB also recovers a sparser signal, while the authors qualify that MAP may work with careful tuning and initialization.Source: §3.3 · pp. 10–11, Figure 1 · p. 12

3.4 Evaluating the VB Image Penalty gVB

• VB's image penalty is concave and non-decreasing for every ρ if and only if f is; MAP requires only log(z) + f(z) to be concave and non-decreasing.At any local minimum of the VB objective, at least m−n elements of x are zero.Source: §3.4 · pp. 12–13
• As ρ increases, the VB penalty transitions from an ℓ0-like form toward a scaled ℓ1 form; increasing noise makes it less concave, while increasing kernel norm makes it more concave.The ratio ρ couples the penalty shape to noise level λ and kernel k, unlike conventional MAP penalties, which retain a fixed shape. Constant f satisfies the stated conditions and induces a Jeffreys noninformative prior on coefficient magnitudes.Source: §3.4 · pp. 13–14

3.5 Noise Dependency Analysis

• VB deconvolution uses a coarse-to-fine schedule: large λ initially emphasizes dominant, low-frequency structures, then λ is reduced to recover finer details.The passage describes repeated kernel re-estimation at successively higher resolutions.Source: §3.5 · pp. 14–15
• VB begins with a nearly convex image penalty when noise and kernel error are high, then increases concavity as estimates improve, reducing instability and local-minimum risk.Unlike MAP approaches with fixed penalty shape, VB adapts the penalty across noise levels and resolutions.Source: §3.5 · p. 15

3.6 Blur Dependency Analysis

• Kernel norm modulates penalty concavity: sparse kernels permit a more concave penalty, while diffuse kernels receive a more convex, conservative one.The delta kernel gives the most concave case; a broad diffuse blur is more challenging and has many local minima.Source: §3.6 · p. 16
• As λ decreases, VB gradually increases concavity and can refine both the kernel and image without the extra penalty factors or complexity of MAP structure-selection strategies.This calibrated progression limits local-minimum risk while allowing a more refined kernel when the image estimate can also become sparser.Source: §3.6 · pp. 16–17

3.7 Other Choices for f

• Theorem 5 establishes that the relative-concavity ordering across ρ values holds if and only if f is affine; increasing its slope weakens shape modulation and gradually degrades performance.The paper therefore favors the constant choice f(γ) = b over affine choices with positive slope.Source: §3.7 · pp. 17–18
• With constant f, rescaling the optimal image by α^-1 and kernel by α preserves optimality under the correspondingly modified kernel constraint, unlike MAP or VB with other f choices.The other methods may require careful tuning of trade-off parameters when the kernel constraint is omitted.Source: §3.7 · p. 18

3.8 Analysis Summary

• The analysis attributes VB’s concavity control to the ratio of noise variance to squared kernel norm and identifies constant f as an optimal, scale-invariant choice, not a claim about natural-image statistics.The constant choice induces a Jeffreys prior, but the authors explicitly do not claim real imaging data follows it.Source: §3.8 · pp. 18–19
• The reformulation makes extensions straightforward: alternative data-fidelity terms can replace the quadratic likelihood, and blur priors or observation models can incorporate domain knowledge.This transparency recovers much of the extensibility associated with MAP.Source: §3.8 · p. 19

4. The Trouble with Natural Image Statistics

• A sufficiently sparse approximation to the true sharp image can make the relaxed objective favor a non-delta blur kernel when p is small and λ is set appropriately.The exponent p need not match the true image statistics for the blur kernel to be estimated accurately.Source: §4 · p. 21
• A prior should favor a good sharp estimate over its blurred version, even when it does not reflect true natural-image statistics.The authors argue this discrimination matters more than matching the true image distribution exactly.Source: §4 · p. 21
• Smaller p reduces sensitivity to image variance, becoming invariant as p approaches zero, and can better distinguish blurred from unblurred images.The authors therefore argue that the optimal prior need not match commonly reported generalized-Gaussian image statistics.Source: §4 · p. 21
• For strong edges p≤1 can avoid the delta solution, while finer structures require p≪1; on real-image slices, the relaxed criterion favors the true blur kernel over a wide p range.The VB penalty’s patch preferences resemble those of the ℓp penalty for sufficiently small p.Source: §4 · pp. 21–22
• VB’s penalty modulation lowers the risk of getting trapped in bad local minima while allowing use of highly discriminative, non-convex image penalties.Directly minimizing the small-p relaxed objective is difficult because it has numerous local minima.Source: §4 · pp. 21–22
• The argument does not contradict Type II MAP results: natural-image statistics can support kernel estimation after marginalizing the image in large-image settings, unlike joint Type I MAP.The paper states that VB is fundamentally different from minimizing the kernel posterior after marginalizing the image.Source: §4 · p. 23

5. Learning λ

• Jointly optimizing x, k, and λ admits infinitely many exact-fit solutions y=k∗x, creating a degenerate global minimum as λ approaches zero.The image and kernel together have more dimensions than the observation, and the VB image penalty can further decrease near zero-valued image elements.Source: §5 · pp. 23–24
• Adding d/λ to the cost prevents λ from reaching zero and guarantees any minimizing λ satisfies λ≥d/n.The authors describe this as a lower bound on the normalized residual variance.Source: §5 · p. 24
• The fixed choice d=n×10^-4 was effective across the reported tests, and incorporating it requires only a slight modification to VB updates.The authors report that this single fixed value is less burdensome than specifying a λ-reduction schedule.Source: §5 · p. 24
• Learning λ from a large initialization can provide a natural coarse-to-fine process while making deblurring more noise-aware and mostly parameter-free.The reported experiments support this conclusion.Source: §5 · p. 24

6. Experimental Results

• VB-Jeffreys uses a Jeffreys image prior and automatically learned λ; the recovered kernel feeds a standardized non-blind deconvolution step.All compared algorithms use the same final non-blind step to estimate the latent image.Source: §6 · p. 25
• The benchmark contains 32 blurred images from four base images and eight blur effects, evaluated using the SSD ratio against ground-truth-kernel deconvolution.The ratio normalizes for the differing difficulty of the non-blind deconvolution problems.Source: §6 · p. 25
• Close to 90% of test images have error ratio below 2 with VB-Jeffreys, which achieves the highest reported result among existing algorithms on this benchmark.Figure 5 reports cumulative error-ratio histograms across 32 examples; taller bars indicate better performance.Source: §6 · pp. 26–27, Figure 5 · p. 27
• VB-Jeffreys significantly exceeds the other tested VB methods in the share of images below error ratio 2, though existing VB algorithms remain reasonable overall.The paper suggests their drop-off at lower error ratios may reflect limited fine-detail resolution with priors that do not produce exactly sparse estimates.Source: §6 · p. 26
• The tested MAP methods use additional regularizers and structure-selection schemes, yet the simpler VB strategy still outperforms the specially tuned MAP estimates.The authors report careful parameter adjustment for MAP-Shan and MAP-Xu and use published results for MAP-Cho.Source: §6 · pp. 26–27

• A rigorous evaluation of VB priors must account for the implementation heuristics and assumptions needed by practical scalable systems.Source: §7 · p. 27
• In ideal noiseless settings, VB and MAP have identical underlying cost functions after accounting for the requisite approximations.Source: §7 · p. 27
• A preferred sparse image prior can distinguish blurry from sharp images, promote scale invariance, and couple the image penalty, blur kernel, and noise level.Source: §7 · p. 28
• MAP can fail when its prior either poorly distinguishes blurry from sharp images or makes convergence to bad local solutions likely.Source: §7 · p. 28
• VB deconvolution algorithms are fundamentally ill-equipped to learn the noise level, and the paper proposes a potential workaround.Source: §7 · p. 28
• Recasting VB as sparse regression offers a route to incorporate additional penalty functions and noise models commonly used with MAP.Source: §7 · p. 28

• For differentiable gx, the γ subproblem has a convenient closed-form solution; otherwise it can be solved numerically as a one-dimensional optimization problem.Source: Proof of Theorem 1 · p. 30
• The kernel update becomes a quadratic optimization problem after applying concave-conjugate upper bounds, and the resulting program is convex.Source: Proof of Theorem 1 · pp. 30–31
• Coordinate updates over x, k, γ, z, and v are guaranteed to reduce or preserve L(x, k, γ) and correspond one-to-one with Algorithm 1.Source: Proof of Theorem 1 · p. 31
• The upper bounds are chosen to connect the reformulation to Algorithm 1, while other bounds could potentially improve convergence or other properties.Source: Proof of Theorem 1 · p. 31

• At ρ = 0, gVB(x, 0) equals gx(x), the image penalty expressed as −2 log p(x) up to constants.Source: Proof of Corollary 1 · p. 32

Proof of Theorem 2

• gVB(x, ρ) is concave and non-decreasing in |x| when it can be expressed as a minimum over upper-bounding hyperplanes with nonnegative slopes.Source: Proof of Theorem 2 · p. 32
• For gVB(x, ρ) to be concave and non-decreasing for every ρ, f must be concave and non-decreasing.Source: Proof of Theorem 2 · p. 33

Proof of Corollary 2

• When f is concave and non-decreasing, the resulting sparse estimation problem has m − n zero elements at any local minimizer.Source: Proof of Theorem 2 · p. 33

Proof of Theorem 3

• As z → ∞, the difference between the costs for ρ1 and ρ2 converges to zero.The optimizing γ grows arbitrarily large, so the difference between log(γ+ρ1) and log(γ+ρ2) vanishes.Source: Proof of Theorem 3 · p. 33
• The proof compares the VB costs using the fact that log(γ + ρ2) > log(γ + ρ1) for all γ.The supplied passage gives this logarithmic inequality as the reason for the stated VB-cost comparison.Source: Proof of Theorem 3 · p. 33
• For ρ1 < ρ2, the proof identifies an additional monotonically increasing penalty on γ, which makes the minimizing γ smaller.The penalty term increases from log(ρ1/ρ2) to zero.Source: Proof of Theorem 3 · pp. 33–34

Proof of Theorem 4

• By unimodality of ϕ, γopt is the unique minimizer; substituting (52) into (50) and omitting irrelevant constant factors yields the resulting expression.Source: Proof of Theorem 4 · p. 34

Proof of Corollary 3

• An increasing ∂x condition in ρ is equivalent to the relative-concavity condition in Definition 1, assuming the requisite derivatives exist.The proof invokes concave duality in establishing the derivative condition.Source: Proof of Corollary 3 · pp. 34–35

Proof of Theorem 5

• For the converse, any strictly negative f′′ would violate the required monotonicity as ρ becomes arbitrarily large.The proof uses concavity, which implies f′′(γ) ≤ 0.Source: Proof of Theorem 5 · p. 36
• ψρ1 ≺ ψρ2 holds if and only if f(γ) = aγ + b with a ≥ 0.The proof assumes twice differentiability when applying the derivative characterization.Source: Proof of Theorem 5 · pp. 35–36

Proof of Theorem 6

• The rescaled pair {α−1x∗, αk∗} remains optimal for every α > 0 if and only if f is constant.In the forward direction, the rescaling leaves the data-fidelity term and total cost unchanged.Source: Proof of Theorem 6 · pp. 36–37

Appendix B: Noise Level Estimation

• The λ update minimizes the VB cost with an additional λ-dependent penalty, but the minimizing solution has no closed form.The appendix uses a convenient upper bound for optimization and says these updates can be merged into Algorithm 1 without disrupting convergence properties.Source: Appendix B · p. 37
• The λ optimum has lower bound d/n, and d is set to n × 10^-4 in the reported simulations.The appendix says this choice leads to good performance and that d can reflect an expectation about minimal noise or modeling error.Source: Appendix B · p. 37