PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Bayesian-based Multi-View Deconvolution

Stephan Preibisch, Fernando Amat, Evangelia Stamataki, Mihail Sarov, Robert H. Singer, Eugene Myers, Pavel Tomancak

arXiv:1308.0730v3q-bio.QM

TL;DR

Large light-sheet microscopy datasets can be difficult to segment and track because optical limitations and acquisition trade-offs reduce image quality. This paper derives a Bayesian multi-view deconvolution method using conditional probabilities and OSEM, achieving substantially faster convergence while improving visual image quality in tested datasets.

  • Problem

    Large light-sheet microscopy datasets have limited image quality, making accurate segmentation and tracking of nuclei and cells challenging.

  • Method

    The method derives a Bayesian multi-view deconvolution formulation incorporating conditional probabilities between views and applies an OSEM-based sequential update scheme.

  • Results

    For typical seven-view, high-signal-to-noise scenarios, the method requires 7-fold fewer iterations than OSEM, SGP, and MAPG, while improving visual image quality compared with MAPG.

  • Takeaways & Limitations

    The method reduces multi-view deconvolution convergence time and can improve visual image quality on real and simulated datasets.

  • Takeaways & Limitations

    Performance depends on point-spread-function estimates that may differ from the microscope's true system PSF because of factors including misalignment and refraction.

Abstract

from arXiv · show

Light sheet fluorescence microscopy is able to image large specimen with high resolution by imaging the sam- ples from multiple angles. Multi-view deconvolution can significantly improve the resolution and contrast of the images, but its application has been limited due to the large size of the datasets. Here we present a Bayesian- based derivation of multi-view deconvolution that drastically improves the convergence time and provide a fast implementation utilizing graphics hardware.

MAIN DOCUMENT

The paper derives a Bayesian multi-view deconvolution algorithm that models conditional probabilities between views and combines it with OSEM to accelerate convergence. GPU implementation enables processing of large light-sheet datasets while improving image contrast and resolution in simulated and biological data.

  • Problem: Light-sheet microscopy captures large specimens from multiple directions, making deconvolution more tractable but leaving image quality limited by optics and acquisition trade-offs.Deconvolution uses optical-system knowledge to increase spatial resolution and contrast after acquisition.
  • Method: The method extends Richardson-Lucy deconvolution to multiple views without assuming that the views are independent observations.Its multiplicative multi-view update combines the observed views through their conditional probabilities and converges to the maximum-likelihood solution.
  • Method: Virtual views derived from known point-spread functions provide intermediate updates at approximately 2-fold lower computational effort, while the sequential OSEM-equivalent scheme reduces convergence time 13-fold.Further simplifications for conditional-probability estimation achieve up to 40-fold improvement over view-independent deconvolution methods.
  • Performance: For typical seven-view, high-SNR SPIM data, the method requires 7 fold fewer iterations than OSEM, SGP, and MAPG, while running 50 fold faster than SGP, 7 fold faster than OSEM, and 3 fold faster than MAPG on CPU.CUDA provides a further 3 fold speedup and reduces memory consumption.
  • Biological applications: Applications to Drosophila and C. elegans data produced higher contrast and resolution than content-based fusion, with computation typically taking a few minutes per multi-view acquisition.The deconvolved C. elegans L1 dataset allowed manual segmentation of all 558 nuclei, compared with 357 of 558 previously identified unambiguously in confocal images.
  • Implementation and scale: The open-source Fiji plugin provides GPU and CPU implementations integrated into a multi-view processing pipeline for datasets of arbitrary size.An entire 236-time-point Drosophila embryonic-development time course was processed in 24 hours using two Nvidia Quadro 4000 graphics cards.

Supplementary Figure 13 Multi-View Deconvolution of Spinning-Disc Confocal Data

The supplementary figures evaluate multi-view deconvolution across datasets, noise levels, PSF uncertainty, resolution, and competing imaging methods. They also illustrate the Bayesian formulation, sequential updates, and performance comparisons.

  • Method: Conditional-probability diagrams motivate modeling dependencies between views and using virtual views with sequential updates.The supplementary figures contrast the classical combined-update scheme with the new sequential approach.
  • Performance: Performance analyses compare convergence time, iteration counts, updates, and optimized implementations across several multi-view deconvolution algorithms.The comparisons include Bayesian-based methods, OSEM, SGP, MAPG, and combinations with optimization schemes.
  • Noise and regularization: Tikhonov regularization is evaluated across signal-to-noise ratios, with intermediate reconstructions shown for multiple regularization parameters.The figure includes no-noise and Poisson-noise cases and compares λ=0, λ=0.0006, λ=0.006, and λ=0.06.
  • PSF effects: PSF quality is examined through angular misestimation, spatial variation across the light sheet, and bead-based characterization in SPIM acquisitions.These analyses test robustness to rotated PSFs and document the region used for deconvolution.
  • Biological datasets: The figures compare reconstruction quality and resolution enhancement for Drosophila, C. elegans, structured-illumination light sheet, two-photon, and spinning-disc confocal data.Multi-view deconvolution is shown alongside content-based fusion, single-view deconvolution, and other microscopy outputs.

1. REMARKS

The supplementary derivations use the notation introduced by Lucy and Richardson and present the mathematics in one dimension while retaining validity for n-dimensional cases.

  • Notation: The document follows Lucy’s notation whenever possible.This establishes the notation used throughout the supplementary derivations.
  • Derivation scope: The derivations are presented in the one-dimensional case for simplicity.The simplified presentation is a notational choice rather than a restriction on equation validity.
  • Derivation scope: The equations remain valid for any n-dimensional case.The stated dimensional generality extends the derivations beyond the one-dimensional presentation.

2. BAYESIAN-BASED SINGLE-VIEW DECONVOLUTION

The single-view section derives Bayesian Richardson–Lucy deconvolution by modeling blurred observations, PSFs, and the underlying image as probability distributions. It connects the Bayesian formulation to an iterative maximum-likelihood expectation-maximization update.

  • Problem formulation: The single-view goal is to estimate an underlying signal distribution from finite measurements and its observed blurred distribution.The observed image is represented through the underlying signal and the measurement probability defined by the PSF.
  • Probability model: Bayesian deconvolution treats the image and PSF as probability distributions with nonnegative values and normalized conditional probabilities.The constraints include P(x|ξ)dx = 1, ψ(ξ) > 0, φ(x) ≥0, and P(x|ξ) ≥0.
  • Bayesian derivation: Bayes’ theorem reformulates the inverse conditional probability into the Richardson–Lucy deconvolution scheme.The desired distribution appears on both sides of the resulting equation, motivating iteration toward a solution.
  • Iterative update: The EM update convolves the current estimate with the PSF, compares the prediction with the input image, and back-projects the ratio through the PSF.The disparity ratio becomes small as the expected and observed distributions approach one another.
  • Assumptions: Convergence toward the correct solution requires a correct PSF estimate and an observation free from noise and other degradations.The stated result assumes an initial constant image estimate.
  • Convolution details: For nonsymmetric kernels, the back-convolution requires inverted coordinates, whereas symmetric kernels allow the inversion to be ignored.Multi-view datasets commonly have nonsymmetric kernels because of transformations from image alignment.

3. BAYESIAN-BASED MULTI-VIEW DECONVOLUTION

The multi-view derivation extends Bayesian Richardson–Lucy deconvolution to multiple observed distributions and PSFs. By modeling relationships between views, it supports a multiplicative update formulation and motivates faster computation.

  • Model: The multi-view model combines N input views, N point spread functions, and one underlying deconvolved signal distribution.Each observed distribution is associated with a view-specific coordinate system and PSF.
  • Probability structure: Conditional independence of measurements given the underlying event simplifies the joint probability formulation across views.The derivation first expresses the joint distribution through conditional probabilities and then applies the stated independence assumptions.
  • View dependencies: The derivation identifies relationships between views that can be incorporated without assuming independence of the observed distributions.The paper links this formulation to faster convergence in later optimization steps.
  • Update rule: The final multi-view update is the per-pixel multiplication of the single-view Richardson–Lucy updates.This multiplicative combination contrasts with the additive combination produced by the maximum-likelihood EM derivation.

4. PROOF OF CONVERGENCE

The derivation is shown to converge to the maximum-likelihood solution by proving that the likelihood does not decrease across iterations. For noise-free data, this follows from inequalities relating the deconvolved distribution to the observed views.

  • 4.1 PROOF FOR NOISE-FREE DATA: Convergence is established by showing that the likelihood of the solution increases or remains unchanged at every iteration.The proof follows the bounded-likelihood argument used for expectation-maximization methods.
  • 4.1 PROOF FOR NOISE-FREE DATA: The Bayesian update is normalized by the number of observed distributions through a geometric mean.This normalization corresponds to division by |V| in maximum-likelihood EM derivations using an arithmetic mean.
  • 4.1 PROOF FOR NOISE-FREE DATA: The proof reduces convergence to an inequality for the arithmetic average, using Jensen’s inequality and the arithmetic-geometric mean relation.These transformations simplify the condition required to show that the likelihood increment is nonnegative.
  • 4.1 PROOF FOR NOISE-FREE DATA: Because all input views have identical integrals, the final inequality always holds and the derivation converges to the maximum-likelihood solution.The argument establishes convergence for the Bayesian-based multi-view deconvolution under noise-free data.

5. DERIVATION OF BAYESIAN-BASED MULTI-VIEW DECONVOLUTION WITHOUT ASSUMING INDEPENDENCE OF THE VIEWS

The derivation removes the usual independence assumption between observed views while retaining the Bayesian-based multi-view formulation. Algebraic application of Bayes’ theorem shows that the final equation remains valid for dependent views.

  • 5. DERIVATION OF BAYESIAN-BASED MULTI-VIEW DECONVOLUTION WITHOUT ASSUMING INDEPENDENCE OF THE VIEWS: Previous multi-view Richardson-Lucy derivations assumed independent views, but this formulation does not require that assumption.The derivation explicitly models the joint relationships among observed distributions.
  • 5. DERIVATION OF BAYESIAN-BASED MULTI-VIEW DECONVOLUTION WITHOUT ASSUMING INDEPENDENCE OF THE VIEWS: Bayes’ theorem rewrites the joint conditional probability using sequential dependencies among the views.The formulation replaces the posterior conditioned on all observations with a product involving conditional probabilities between observations.
  • 5. DERIVATION OF BAYESIAN-BASED MULTI-VIEW DECONVOLUTION WITHOUT ASSUMING INDEPENDENCE OF THE VIEWS: Cancellation and further application of Bayes’ theorem recover the same final formulation used for Bayesian-based multi-view deconvolution.This establishes that independence is unnecessary for deriving equation 38.

6. EXPRESSION IN CONVOLUTION ALGEBRA

The Bayesian multi-view formulation is rewritten in convolution algebra so its integrals can be computed efficiently in Fourier space. This conversion relies on spatially invariant PSFs and aligned coordinate systems.

  • 6. EXPRESSION IN CONVOLUTION ALGEBRA: The integrals in the Bayesian formulation are expressed as convolutions for efficient computation using the Convolution Theorem in Fourier space.This reformulation connects the probabilistic derivation to fast image-processing operations.
  • 6. EXPRESSION IN CONVOLUTION ALGEBRA: The convolution form assumes that each point spread function is constant across spatial locations.This is the spatial-invariance assumption required by the algebraic reformulation.
  • 6. EXPRESSION IN CONVOLUTION ALGEBRA: The coordinate systems of the underlying image and all views are assumed identical, or related by an identity transformation.This assumption allows the distributions to be combined directly in convolution algebra.
  • 6. EXPRESSION IN CONVOLUTION ALGEBRA: In the convolution notation, the mirrored kernel represents the reflected version of the point spread function.The notation distinguishes convolution operations from scalar multiplication and division.

7. EFFICIENT BAYESIAN-BASED MULTI-VIEW DECONVOLUTION

The method models conditional relationships between views to generate virtual views and incorporate them into deconvolution updates. This preserves the computational cost of standard multi-view deconvolution while substantially reducing convergence time.

  • 7.1 MODELING CONDITIONAL PROBABILITIES: The initial inverse-PSF estimate uses the PSF itself as a worst-case upper bound because no prior knowledge of the inverse relationship is available.This makes the virtual distribution less precise while retaining all possible solutions.
  • 7.1 MODELING CONDITIONAL PROBABILITIES: Virtual views estimate one observed distribution from another using both views’ point spread functions and conditional probabilities.The estimate is an upper boundary over possible appearances of the target view given the source view and PSFs.
  • 7.2 INCORPORATING VIRTUAL VIEWS INTO THE DECONVOLUTION SCHEME: The method performs updates using both observed and virtual distributions to incorporate dependencies among views.Virtual distributions are generated from the conditional relationships modeled by equation 79.
  • 7.2 INCORPORATING VIRTUAL VIEWS INTO THE DECONVOLUTION SCHEME: The final formulation combines observed and virtual updates into one update per observed distribution through convolution-algebra simplifications.The derivation applies assumptions about convolution products, including a condition illustrated for Gaussian-like distributions after normalization.
  • 7.2 INCORPORATING VIRTUAL VIEWS INTO THE DECONVOLUTION SCHEME: The efficient formulation has the same number of computations as normal multi-view deconvolution, apart from initial compound-kernel computation.The compound kernels are computed once, and their precomputation is usually negligible because PSFs are small relative to images.

8. AD-HOC OPTIMIZATIONS OF THE EFFICIENT BAYESIAN-BASED MULTI-VIEW DECONVOLUTION

The authors optimize virtual-view computation to reduce convergence time while retaining reasonable deconvolution results. Skipping one convolution and assuming a constant PSF term provide progressively faster approximations, with an explicit low-noise qualification.

  • The optimized formulation uses virtual distributions whose sharpening is intended to accelerate convergence without omitting possible solutions.
  • Skipping the convolution with P∗w significantly reduces convergence time while producing almost identical results, including with noise.The optimization changes the computation of virtual distributions rather than the core deconvolution objective.
  • Assuming Pw is constant further reduces convergence time while still producing reasonable results.The authors describe this assumption as ad hoc.
  • The efficient formulation resembles a single-view Richardson–Lucy optimization that applies an exponent to a correction factor.

9. ALTERNATIVE ITERATION FOR FASTER CONVERGENCE

The paper replaces combined multi-view updates with sequential application of individual view contributions. This connects the scheme to OS-EM and reduces convergence time by updating the underlying distribution view by view.

  • Sequentially multiplying each view’s contribution updates the underlying distribution directly and saves computation time.
  • Leaving out the compound-kernel multiplicative part makes the scheme equivalent to sequential standard Richardson–Lucy updates.
  • OS-EM applies Richardson–Lucy updates sequentially to balanced subsets of observed data.In multi-view deconvolution, each observed distribution acts as a balanced subset because all underlying-distribution elements are updated for each view.
  • The sequential formulation is directly suggested by the multiplicative derivation rather than the additive EM derivation.

10. BENCHMARKS

Benchmarks show that the proposed method and its optimizations reduce computation time and iterations while reaching comparable deconvolution quality. Performance depends on view count, PSF geometry, noise, and PSF estimation accuracy.

  • 10.1 Convergence time, number of iterations & updates compared to classical multi-view deconvolution: Optimization II is 3–8× faster than the Bayesian-based derivation, while sequential updates add an approximately n-fold speed-up.
  • 10.1 Convergence time, number of iterations & updates compared to classical multi-view deconvolution: For combined updates, convergence time increases almost linearly with view count, whereas sequential updates decrease in iterations and then plateau in time.
  • 10.1 Convergence time, number of iterations & updates compared to classical multi-view deconvolution: Adding a second view decreases computation time by an average of 45-fold, while a third view adds another average 1.5-fold reduction.
  • 10.1 Convergence time, number of iterations & updates compared to classical multi-view deconvolution: At 7 views, Optimization II converges in 7 iterations versus around 50 for OSEM, SGP, and MAPG.
  • 10.2 Convergence time, number of iterations & updates compared to other optimized multi-view deconvolution schemes: At 7 views, Optimization II converges in 1.6 seconds, compared with around 80 seconds for SGP and OSEM in IDL.
  • 10.3.1 Comparison to MAPG: Optimization II is faster than MAPG while producing better overall image quality and fewer artifacts in the evaluated 7-view Drosophila acquisition.
  • 10.5 Noise and regularization: All methods converge to visually identical results in the typical light-sheet microscopy SNR range of 15–63.
  • 10.6 PSF-estimation: Imprecise PSF estimation is a potential error source, although randomly rotating PSFs by an average of 10 degrees produced practically identical deconvolved images.

11. MULTI-VIEW DECONVOLUTION, RESOLUTION AND OTHER OPTIAL SECTIONING MICROSCOPY

The paper evaluates multi-view deconvolution across optical-sectioning modalities and partially overlapping datasets. It emphasizes improved image quality and practical handling of overlap, while noting that update weighting must remain stable and uniform.

  • Multi-view deconvolution can be performed with optical-sectioning microscopes capable of sample rotation, including spinning-disc confocal microscopy.
  • Compared with raw data and content-based fusion, multi-view deconvolution with structured illumination increases image contrast.The cited comparison does not establish the relative resolution increase between structured illumination and multi-view deconvolution.
  • Multi-view deconvolution improves fluorescent-bead resolution relative to input views and multi-view fusion.
  • Sequential updates support partially overlapping datasets by updating only regions covered by subsets of input views.
  • Keeping per-pixel weights at or below 1 prevents updates larger than a Richardson–Lucy step and avoids unstable descent.
  • In partially overlapping datasets, the speed-up matches that of fully overlapping datasets, but reduced overlap increases convergence time.
  • The Fiji plugin reports the number of contributing views per pixel to help users choose overlap and adjust imaging strategy.

13. SIMULATION OF SPIM DATA

The paper simulates realistic three-dimensional multi-view SPIM data by modeling specimen structure, optical degradations, sampling, and noise, then evaluates reconstruction quality and implementation options.

  • Simulation setup: Each virtual view rotates the ground truth, applies light attenuation and view-specific PSF convolution, samples reduced axial resolution, and rotates the acquired volume back.Linear interpolation is used for the alignment transformations.
  • Simulation setup: The simulated specimen is a three-dimensional object composed of many randomly located, sized, and intensified small spheres, rendered at twice the final resolution before downsampling.This reduces artificially sharp edges in the ground-truth image.
  • Sampling and noise: Reduced axial resolution is modeled by sampling every third z slice while sampling every lateral xy pixel, with pixel intensities generated by a Poisson process.This corresponds to a typical multi-view acquisition.
  • Remarks on results: MAPG and Optimization II outperform content-based fusion in spatial resolution, but MAPG produces strong ring-like artifacts around simulated spheres.Earlier manual stopping can reduce noise in the deconvolved image.
  • Implementation: The implementation uses Fourier convolutions on the CPU or an alternative CUDA GPU convolution, but the entire pipeline cannot run on the GPU because graphics-card memory is limited.The workflow integrates registration, bounding-box fusion, and final CPU or GPU deconvolution in Fiji.
Loading 1308.0730v3…