PaperlayerView exact revision on arXiv

Source-linked AI summary

Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation

Suhas Sreehari, S. V. Venkatakrishnan, Brendt Wohlberg, Lawrence F. Drummy, Jeffrey P. Simmons, Charles A. Bouman

arXiv:1512.07331v1cs.CVeess.IV

TL;DR

Sparse electron microscopy reconstruction must recover images from limited measurements while exploiting non-local repetition. The paper adapts plug-and-play priors to tomography and interpolation, proves sufficient convergence conditions, and designs a doubly-stochastic-gradient NLM variant. Experiments on simulated and real microscope data report improved reconstruction quality and convergence, including sparse interpolation from as little as 5% sampled pixels.

  • Problem

    Sparse sampling can reduce acquisition time and electron-beam damage, but existing MBIR requires difficult explicit prior selection and does not directly incorporate denoisers such as NLM or BM3D.

  • Method

    The paper uses ADMM-based plug-and-play priors to alternate forward-model inversion with denoising, derives convergence conditions, and redesigns NLM to have a doubly stochastic gradient.

  • Results

    The method produces higher-quality reconstructions on simulated and real microscope data, while DSG-NLM fully converges and achieves the least RMSE in the reported sparse interpolation experiments.

  • Takeaways & Limitations

    Plug-and-play provides an MBIR framework compatible with a wide variety of denoising algorithms as prior models for tomography and sparse interpolation.

Abstract

from arXiv · show

Many material and biological samples in scientific imaging are characterized by non-local repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a 2D image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the non-local redundancy in images. We adapt a framework, termed plug-and-play (P&P) priors, to solve these imaging problems in a regularized inversion setting. The power of the P&P approach is that it allows a wide array of modern denoising algorithms to be used as a "prior model" for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the P&P approach, and we use these insights to design a new non-local means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.

I. INTRODUCTION

The paper targets sparse electron tomography and image interpolation by exploiting non-local redundancy with plug-and-play priors. Its framework separates forward-model inversion from denoising, enabling diverse denoisers as prior models while addressing convergence.

  • Motivation: Sparse sampling can accelerate acquisition and reduce electron-beam damage, but reconstruction must exploit repeating structures in material and biological samples.The motivation covers sparse pixels in image acquisition and sparse projection sampling in tomography.
  • Motivation: Existing MBIR can model image redundancy, but selecting an appropriate explicit log prior is challenging, while NLM and BM3D lack explicit cost functions.These limitations complicate incorporating successful denoisers into MBIR.
  • Plug-and-Play Framework: The proposed plug-and-play algorithm separates forward-model inversion from prior processing and replaces prior optimization with a denoising operation.The method is based on ADMM and repeatedly applies an inversion step followed by a denoising step.
  • Applications: The framework is applied to bright field electron tomography and sparse interpolation using an existing MBIR tomography model that handles Bragg scatter and anomaly detection.The paper evaluates simulated and real microscope images.
  • Applications: The method produces high-quality reconstructions and interpolation on simulated and real microscope data, with improved convergence properties compared with standard NLM or BM3D regularization.The framework is designed to accommodate a wide variety of denoising algorithms as spatial prior models.
  • Plug-and-Play Framework: Plug-and-play permits denoisers such as NLM and BM3D to serve as prior models even when they are not readily represented as optimization cost functions.The denoising operator is selected independently of the forward model.

III. CONVERGENCE OF THE PLUG-AND-PLAY ALGORITHM

The paper establishes sufficient conditions for plug-and-play convergence without explicitly specifying the prior function. The conditions connect denoiser structure, likelihood properties, and existence of a MAP solution.

  • Problem: The convergence question arises because plug-and-play uses general denoisers without an explicitly available prior function.The paper therefore seeks conditions on both the denoiser and negative log likelihood.
  • Sufficient Conditions: The theorem requires a continuously differentiable denoiser whose gradient is doubly stochastic, alongside feasibility and proper, closed, convex likelihood conditions.The listed conditions include a likelihood lower bound that prevents minimization from escaping toward infinity.
  • Theoretical Results: Under these conditions, the denoiser is a proximal mapping, a MAP estimate exists, and the plug-and-play iterations converge.The resulting MAP cost is implicitly defined through the denoiser rather than explicitly specified.
  • Theoretical Results: The first denoiser conditions ensure the Moreau-theorem requirements by making the operator the gradient of a convex function and non-expansive.This provides the link between denoising and an ADMM prior update.
  • Sufficient Conditions: Additional likelihood conditions ensure that the feasible set is nonempty and that the MAP cost attains a global minimum rather than minimizing at infinity.These conditions establish existence of the target estimate independently of explicit prior knowledge.

IV. NON-LOCAL MEANS DENOISING WITH DOUBLY STOCHASTIC GRADIENT

The paper modifies non-local means so its denoising operator satisfies the doubly-stochastic gradient condition required for guaranteed plug-and-play convergence.

  • Convergence motivation: Standard NLM does not satisfy the doubly-stochastic gradient condition required for plug-and-play convergence.The paper introduces doubly stochastic gradient NLM (DSG-NLM) to address this mismatch.
  • DSG-NLM: DSG-NLM modifies NLM to produce a denoising operator with a doubly-stochastic gradient.The construction relies on a weight matrix that is symmetric, normalized, and practically non-negative.
  • Non-local means: NLM estimates each pixel as a weighted mean of pixels within a search window using patch-based similarities.Larger search windows can improve results but increase computational cost.
  • DSG-NLM: The modified weight construction makes W symmetric with rows and columns summing to 1, thereby satisfying the convergence condition.The normalization procedure adjusts the diagonal coefficient to guarantee normalization.

V. 3D BRIGHT FIELD EM FORWARD MODEL

The paper instantiates the plug-and-play inversion operator for 3D bright-field electron tomography using a forward model that handles measurement variability and physical constraints.

  • Forward model: The 3D bright-field EM inversion operator adopts the forward model and optimization algorithms of an existing MBIR framework.That framework models Bragg scatter and anomaly detection.
  • Likelihood model: The negative log likelihood uses tilt-specific measurements, projection matrices, variance weights, offsets, and a generalized Huber penalty.The generalized Huber function is used to reject measurements with large errors.
  • Optimization: The inversion operator minimizes a cost function with respect to the image, offset vector, and noise-related parameter.Alternating minimization and a surrogate function handle the optimization of the generalized Huber term.
  • Constraints: Positivity is enforced by assigning infinite cost to image values below zero.This constraint is incorporated into the likelihood rather than the denoiser.
  • Theoretical scope: The tomography application violates the convexity and growth assumptions used by the convergence theory.Therefore, convergence to a global minimum is not guaranteed theoretically in this setting.
  • Empirical behavior: Despite these deviations and approximate three-step alternating minimization, the authors report never observing drift to infinity with real data and useful denoisers.They also report consistent empirical convergence under these approximations.

VI. SPARSE INTERPOLATION FORWARD MODEL

For sparse interpolation, the paper formulates a plug-and-play inversion operator for recovering a full image from noisy measurements of only a small subset of pixels.

  • Problem formulation: Sparse interpolation recovers x ∈ R^N from a noisy subsampled image y ∈ R^M where M << N.The sampling matrix records which pixels were measured.
  • Forward model: The sampling matrix A contains one nonzero entry per measurement row, while image columns may be unsampled or sampled once.This structure enables a simple sparse forward model.
  • Likelihood and constraints: The sparse-sampling negative log likelihood is modified to assign infinite cost to negative image values.Positivity is placed in the likelihood so the denoising operator remains continuously differentiable.
  • Inversion operator: The interpolation inversion operator has an explicit pixel-wise form because of the simple structure of A.The resulting interpolation is forced to match measured values at sampled locations.

VII. RESULTS

The experiments evaluate plug-and-play reconstruction on simulated and real bright-field tomography data, comparing multiple reconstruction methods and their convergence behavior.

  • Convergence evaluation: The experiments track normalized primal and dual residuals across plug-and-play iterations.These residuals are used to compare convergence under different priors.
  • Experimental design: The experiments compare filtered backprojection, qGGMRF MBIR, and plug-and-play reconstructions using 3D NLM and 3D DSG-NLM priors.The evaluation includes simulated aluminum spheres and real aluminum-sphere and silicon-dioxide datasets.
  • Convergence results: DSG-NLM makes the plug-and-play algorithm converge fully, while NLM converges to within a fraction of a percent.This observation is reported across the experiments summarized in Tables III and IV.

1) Aluminum spheres (simulated) dataset:

Tomographic reconstructions were evaluated on simulated aluminum spheres and real aluminum-spheres and silicon-dioxide datasets. Plug-and-play reconstructions using NLM and DSG-NLM were reported as clearer, less artifact-prone, and, for DSG-NLM, fully convergent.

  • 1) Aluminum spheres (simulated) dataset:: The simulated aluminum-spheres phantom used 47 equally spaced y-axis tilts spanning [−70°, +70°], with Gaussian noise and Bragg-scatter-like effects.Its dimensions were 256 nm × 512 nm × 512 nm along z, x, and y.
  • 1) Aluminum spheres (simulated) dataset:: NLM and DSG-NLM reconstructions had no shadow artifacts, low RMSE, and sharper edges than the other simulated reconstructions.The comparison used a slice along the x-z plane against ground truth.
  • 1) Aluminum spheres (simulated) dataset:: NLM and DSG-NLM produced clearer, relatively artifact-free simulated reconstructions, while DSG-NLM achieved complete plug-and-play convergence.The convergence evaluation tracked primal and dual residuals.
  • 2) Aluminum spheres (real) dataset:: The real aluminum-spheres dataset contained 67 equally spaced y-axis tilts spanning [−65°, +65°].NLM had less smear than qGGMRF and more clarity than filtered backprojection; NLM and DSG-NLM suppressed missing-wedge artifacts.
  • 2) Aluminum spheres (real) dataset:: DSG-NLM achieved complete convergence for the real aluminum-spheres and silicon-dioxide tomographic reconstructions.The figures report plug-and-play primal and dual residual convergence for both real datasets.
  • 3) Silicon dioxide (real) dataset:: NLM and DSG-NLM reconstructions of real silicon dioxide had less smear than qGGMRF and far more clarity than filtered backprojection.The silicon-dioxide dataset used 31 y-axis tilts spanning [−65°, +65°].

B. Sparse Interpolation

Sparse interpolation experiments applied plug-and-play with several denoisers to simulated and real microscope images. DSG-NLM generally produced the lowest interpolation error and fully converged, including with sparse sampling.

  • B. Sparse Interpolation: NLM, DSG-NLM, and BM3D were used as plug-and-play prior models for sparse interpolation of simulated and real microscope images.DSG-NLM weights were stopped after 12 plug-and-play iterations in all sparse-interpolation experiments.
  • B. Sparse Interpolation: The experiments used simulated super ellipses and a real zinc-oxide nanorod microscope image, with images scaled to [0, 255].The super ellipses mimic material-grain shapes such as Ni-Cr-Al alloy.
  • B. Sparse Interpolation: Plug-and-play interpolation results were clearer than Shepard interpolation, and DSG-NLM typically produced the least RMS interpolation error.Interpolation error was evaluated using normalized RMSE against ground truth.
  • B. Sparse Interpolation: DSG-NLM made plug-and-play converge fully in the sparse-interpolation experiments.The convergence assessment used normalized primal and dual residual errors after 150 iterations.
  • B. Sparse Interpolation: The plug-and-play framework supports multiple denoisers as prior models and separates forward-model inversion from denoising.This modularity permits different image-prior choices within the same inverse-problem framework.
  • B. Sparse Interpolation: Using as little as 5% of pixels, DSG-NLM achieved the least RMSE and complete plug-and-play convergence across the interpolation experiments.The comparison used Shepard’s interpolation as the baseline and included NLM, DSG-NLM, and BM3D.

APPENDIX A PROOF OF PLUG AND PLAY CONVERGENCE THEOREM

The appendix proves convergence of the plug-and-play algorithm by establishing that its denoiser is a proximal mapping and applying standard ADMM convergence results. It also proves existence of a global MAP minimizer under the stated convexity and feasibility conditions.

  • Proximal-mapping characterization: A denoiser H is a proximal mapping if and only if it is non-expansive and the sub-gradient of a convex function.This criterion is supplied by Moreau’s theorem and underpins the appendix’s convergence proof.
  • Proximal-mapping characterization: Symmetric Jacobians make H conservative, while doubly stochastic Jacobians give positive eigenvalues and non-expansiveness.These properties imply that H corresponds to a proper, closed, and convex implicit prior function s(x).
  • Existence of a MAP estimate: The combined objective h(x) = l(x) + s(x) is proper, closed, and convex when its effective domain is nonempty.The proof establishes properness using a point y = H(x) with finite likelihood and prior values, then applies the convex-sum lemma.
  • ADMM convergence: The plug-and-play iterations converge because the problem satisfies the standard ADMM assumptions, including a saddle point of the un-augmented Lagrangian.Strict feasibility invokes Slater’s theorem and strong duality; the appendix then adapts the ADMM convergence theorem to obtain the stated result.
Loading 1512.07331v1…