PaperlayerView exact revision on arXiv

Source-linked AI summary

Structured illumination microscopy image reconstruction algorithm

Amit Lal, Chunyan Shan, Peng Xi

arXiv:1602.06904v1cs.CV

TL;DR

SIM reconstruction has lacked a unified treatment of several important algorithmic steps, with prior work often focusing mainly on illumination-phase estimation. This paper consolidates the theoretical workflow, provides OpenSIM code for interactive parameter study, and reports approximately two-fold spatial-resolution improvement for SIM.

  • Problem

    SIM reconstruction theory and implementation details are scattered, while modulation estimation, object-power-spectrum estimation, filtering, frequency relocation, and component merging are often underemphasized.

  • Method

    The paper presents a step-by-step SIM reconstruction algorithm covering parameter estimation, power-spectrum modeling, Wiener filtering, frequency shifting, phase matching, and component merging, together with OpenSIM Matlab code.

  • Results

    SIM improves spatial resolution by approximately two-fold, while the reconstructed image shows enhancement relative to a deconvolved wide-field image.

  • Takeaways & Limitations

    OpenSIM enables users to vary reconstruction parameters interactively to study their effects and reduce reconstruction errors or artifacts.

Abstract

from arXiv · show

Structured illumination microscopy (SIM) is a very important super-resolution microscopy technique, which provides high speed super-resolution with about two-fold spatial resolution enhancement. Several attempts aimed at improving the performance of SIM reconstruction algorithm have been reported. However, most of these highlight only one specific aspect of the SIM reconstruction -- such as the determination of the illumination pattern phase shift accurately -- whereas other key elements -- such as determination of modulation factor, estimation of object power spectrum, Wiener filtering frequency components with inclusion of object power spectrum information, translocating and the merging of the overlapping frequency components -- are usually glossed over superficially. In addition, most of the work reported lie scattered throughout the literature and a comprehensive review of the theoretical background is found lacking. The purpose of the present work is two-fold: 1) to collect the essential theoretical details of SIM algorithm at one place, thereby making them readily accessible to readers for the first time; and 2) to provide an open source SIM reconstruction code (named OpenSIM), which enables users to interactively vary the code parameters and study it's effect on reconstructed SIM image.

I. INTRODUCTION

SIM uses structured illumination and Moiré effects to access specimen frequencies beyond the optical transfer function limit, but its reconstruction details have been fragmented and unevenly treated. This manuscript consolidates the theory, algorithmic steps, and OpenSIM implementation needed for artifact-reduced super-resolved reconstruction.

  • SIM motivation and concept: SIM supports real-time live-cell imaging, addressing the temporal-resolution limitations of techniques that obtain very high spatial resolution at the expense of speed.The passage specifically contrasts SIM and SOFI with techniques that are difficult to apply below 50 nm in vivo.
  • Motivation and contribution: Prior SIM reconstruction work often emphasizes illumination-phase estimation while treating modulation, frequency relocation, and overlap merging superficially.The manuscript identifies these as additional key reconstruction elements.
  • Motivation and contribution: The manuscript provides theoretical details, step-by-step SIM reconstruction procedures, and open-source Matlab code for interactively varying parameters and studying reconstruction artifacts.OpenSIM is intended to help users understand and modify reconstruction parameters.
  • SIM motivation and concept: SIM uses sinusoidal structured illumination to produce Moiré patterns whose measured frequency content can reveal specimen frequencies beyond the optical system OTF.The method can theoretically access frequencies up to twice the conventional OTF limit.
  • SIM reconstruction workflow: The reconstruction separates frequency components, Wiener-filters their noisy estimates, shifts them to correct reciprocal-space locations, and merges them into a super-resolved image.The separated components arise from the three circular frequency regions associated with structured illumination.
  • SIM motivation and concept: Sequential illumination at three orientations provides frequency information within a region extending to twice the OTF-limited frequency radius.The conventional reconstruction uses nine images from three orientations and three phase shifts.

IV. SIM RECONSTRUCTION ALGORITHM (SIM-RA)

The SIM reconstruction algorithm estimates illumination frequency and phase before using those estimates to recover specimen frequency components. The parameter estimates are obtained through iterative correlation-based optimization.

  • Parameter estimation: Illumination spatial frequency is estimated a posteriori by optimizing the autocorrelation between a frequency-domain image term and its shifted variant.The desired frequency corresponds to the value producing the maximum autocorrelation magnitude.
  • Parameter estimation: The illumination frequency is selected from the peak of the autocorrelation objective.The passage identifies the value of pθ corresponding to the maximum of |C1| as the desired illumination spatial frequency.
  • Parameter estimation: After estimating illumination frequency, the algorithm constructs a sinusoidal pattern from an initial phase guess and iteratively optimizes correlation to estimate the true phase shift.The correlation reaches its maximum when the initial phase guess equals the illumination phase.

C. Estimation of object power spectrum

The algorithm estimates the object’s average power spectrum before determining modulation and filtering frequency components. It then shifts, phase-matches, merges, and inverse-transforms the recovered components.

  • Frequency-component processing: The reconstruction obtains noisy central and shifted frequency estimates by solving the SIM equations with m initially set to 1, then later adjusts for the estimated modulation factor.The three central estimates are averaged to estimate the object power-spectrum parameters.
  • Frequency-component processing: Wiener filtering produces noise-filtered estimates before shifted components are relocated, phase-matched, and merged with generalized Wiener filtering.The final inverse Fourier transform yields the reconstructed SIM image.
  • Object power-spectrum estimation: The object’s average power spectrum is modeled as |H̃(k)|^2A^2|k|^-2α plus average noise power, with A and α estimated by nonlinear regression.Noise power is estimated from frequencies outside the OTF support, where signal power is zero.
  • Modulation-factor estimation: The algorithm uses the estimated power-spectrum parameters to determine the modulation factor m from the noisy shifted-frequency estimate.The cited relation leaves m as the only unknown after the power-spectrum and noise terms are specified.

E. Wiener Filtering

The reconstruction estimates three noise-corrupted frequency components, applies Wiener filtering using noise-power information, and shifts the separated components to their true frequency positions before merging.

  • Noise filtering: White-noise assumptions allow linear combinations of the raw SIM measurements to estimate three specimen frequency components corrupted by noise.The estimates correspond to the central, negative-shifted, and positive-shifted components.
  • Noise filtering: Wiener filtering is applied to the shifted components, with separate average noise powers specified for the central and shifted frequency estimates.The filter expressions include noise-power terms Ψo,θ, Ψp,θ, and Ψq,θ.
  • Noise filtering: An extra factor 1/m compensates for computing shifted-component estimates with modulation factor m set to 1 during an earlier algorithm step.This correction is applied to the estimates of the negative- and positive-shifted components.
  • Frequency shifting: Fourier shift theorem relocates the separated components from their displaced centers to their true positions in the frequency domain.The shifted variants are denoted ˜Ss(k−pθ) and ˜Ss(k+pθ).
  • Frequency shifting: Fourier transform and inverse Fourier transform define the forward and reconstruction operations used in the frequency-domain pipeline.The notation F and F−1 denotes Fourier transform and inverse Fourier transform, respectively.

G. Phase matching

Phase matching corrects inconsistencies between overlapping central and shifted frequency components caused by inaccurate illumination-phase estimates. The corrected components are then combined with an empirically adjusted Wiener-filter parameter, while discrete-frequency effects require a rounded-frequency approximation for shifted OTF power spectra.

  • G. Phase matching: Phase mismatch over overlapping frequencies should sum to zero, but inaccurate illumination-phase estimates violate this consistency condition.The violation motivates a corrective phase for the shifted frequency components.
  • G. Phase matching: The corrective phase is computed over frequencies where the unshifted central component and shifted component overlap, then applied to the shifted components.The correction is applied to both shifted frequency components obtained from the preceding shift steps.
  • H. Merging all frequency components using generalized Wiener filter: Triplets of central and phase-corrected shifted components from three orientations are combined with an approximate generalized Wiener filter to form the SIM Fourier image.The reconstructed image is obtained by inverse Fourier transforming the combined spectrum.
  • H. Merging all frequency components using generalized Wiener filter: When shifted frequency coordinates are not integral multiples of 1/N, directly shifting the system OTF with the Fourier shift theorem produces an erroneous result.The error arises from frequency discretization in digital images and discrete Fourier transforms.
  • H. Merging all frequency components using generalized Wiener filter: The method instead rounds the illumination frequency coordinates to the nearest integral multiple of 1/N and uses the resulting shifted OTF to approximate the required power spectrum.Only the shifted OTF power spectrum is required for the relevant Wiener-filter expression.
  • Implementation: The SIM reconstruction algorithm and its TIRF-SIM variant are implemented as freely available Matlab scripts and functions in OpenSIM.The same files were used to obtain the reported results.

VI. RESULTS

The study evaluates SIM-RA using simulated noisy images and compares its reconstruction with Wiener-filtered widefield imaging. SIM reconstruction visibly improves resolution, although the theoretical two-fold limit is not reached under the simulated illumination and noise conditions.

  • Simulation setup: The evaluation used a Lenna-based test object with periodic patterns and simulated optical imaging under a circumsymmetric OTF.The simulated images included additive Gaussian noise at 10% (20 dB).
  • Simulation setup: The simulations used three illumination orientations at 0°, 60°, and 120°, with deliberately fractional illumination frequency and phase errors uniformly distributed from −15° to 15°.The reconstruction estimated the illumination phases a posteriori.
  • Reconstruction performance: Nine simulated raw SIM images were reconstructed with SIM-RA, producing a super-resolution image whose improvement over the widefield image was evident in spatial and Fourier domains.The relevant comparison is with the Wiener-filtered widefield image because SIM-RA itself includes Wiener filtering.
  • Reconstruction performance: Visual comparison showed that the SIM reconstruction had higher resolution than the deconvolved widefield image.The comparison used effective point spread functions computed by linear algebra from the test, deconvolved widefield, and SIM-reconstructed images.
  • Resolution analysis: SIM theoretically improves spatial resolution two-fold, while deconvolution alone improves it by approximately 1.4-fold.These conclusions follow from the reported relationships between the effective PSF widths.
  • Resolution analysis: The theoretical SIM limit was not achieved because the illumination frequency was about 75% of the OTF cutoff and the simulated raw images contained 10% noise.The maximum theoretical condition is FWHM(PSFSIM) = 1/2×FWHM(PSFdeconvWF).

1) Acquisition of raw SIM images:

Experimental SIM reconstruction requires preprocessing, frequency-parameter estimation, artifact suppression, and comparison against a Wiener-filtered wide-field image. The approach achieves artifact-free reconstruction under the stated experimental conditions, while its spatial-domain phase method has a defined OTF-support limitation.

  • More than 100 fluorescent microspheres were averaged to estimate the system PSF, whose Fourier transform provided the system OTF.
  • Raw experimental SIM images contained severe background fluorescence blur and were normalized across nine images before heuristic background removal.Normalization matched global mean and standard deviation across the nine images; background removal used Matlab's imopen morphological operation.
  • Residual frequency peaks from imperfect background removal produced hexagonal artifacts, so Wiener-filtered sideband estimates were multiplied by a heuristic notch filter.The notch-filter parameters were set by trial and error to ao = 0.05 and β = 1.2 in the reported case.
  • The reconstructed SIM image showed readily observable resolution enhancement relative to the Wiener-filtered wide-field comparison image.The comparison used the averaged central frequency component and displayed the wide-field-equivalent image alongside the SIM reconstruction.
  • The authors rejected three Fourier-domain phase-estimation methods because background fluorescence errors near zero frequency can affect two methods, while additive Gaussian noise affects the local peak method.They instead determined phase in the spatial domain, obtaining estimates accurate enough for artifact-free experimental reconstruction.
  • The spatial-domain phase method does not work when illumination spatial frequency lies beyond system OTF support, as in TIRF-SIM.For that setting, the manuscript recommends the iterative method based on separated-frequency-component cross-correlation.

C. Approximation for Generalized Wiener Filter

Generalized Wiener filtering provides the conceptual basis for merging separated SIM frequency components, but direct implementation requires approximations when frequency shifts are nonintegral. The manuscript follows and makes explicit a practical approximation strategy, with apodization used to reduce reconstruction artifacts.

  • Direct generalized Wiener-filter merging is invalid when shifted frequency coordinates are not integral multiples of 1/N because shifted products do not equal products of separately shifted terms.
  • Practical generalized Wiener-filter implementation therefore requires approximations and includes an element of design judgment.The manuscript follows the approach of Gustafsson et al. and presents it more explicitly.
  • An apodization filter helps avoid hard edges and ringing in the reconstructed SIM image.The experimental reconstruction used apodization, while the simulated reconstruction did not; OpenSIM includes an apodization subroutine.
  • Preprocessing is necessary because normalization compensates for intensity variation, background removal reduces blur, and heuristic removal leaves residual peaks requiring a notch filter.
  • A TIRF-SIM variant is provided for cases where illumination spatial frequency typically lies beyond system OTF support.

A. Formulation

The formulation reconstructs SIM images by estimating illumination and object parameters, Wiener-filtering separated components, shifting and phase-matching them, and merging all components before inverse transformation.

  • Algorithm 2 takes the system OTF and nine raw SIM images as input and outputs a reconstructed SIM image.
  • The algorithm estimates illumination spatial frequency, averages central components to estimate object-power-spectrum parameters, and determines the modulation factor.
  • Wiener filtering converts noisy component estimates into noise-filtered, ungraded estimates of the central and shifted specimen spectra.
  • The shifted sideband components are moved to their true frequency locations and phase-matched to the unshifted component.The frequency estimate is obtained by maximizing a normalized cross-power spectrum, whose maximum occurs at the illumination frequency.
  • All nine frequency components are merged with a generalized Wiener filter, and an inverse Fourier transform produces the reconstructed image.
  • Relative-phase estimates are used to form phase-adjusted shifted components, with iterative cross-correlation minimization estimating the unknown phases.The weighting function reduces noise effects; for Gaussian noise, the stated optimum is w(k) = H̃(k)H̃*(k).
  • When approximate phases are used, the resulting cross-talk coefficients vanish under the matching condition, separating the corresponding shifted components.

D. Remarks on Algorithm 2

Algorithm 2 reverses the order of spatial-frequency and phase estimation and uses relative rather than absolute phases. Its phase-matching step compensates for this choice, and its reconstructions were visually similar to those from Algorithm 1.

  • Algorithm 2 estimates illumination phase before spatial frequency, whereas Algorithm 1 uses the reverse order.
  • Algorithm 2 estimates relative phases φ′2 and φ′3 rather than the true absolute phases φ1, φ2, and φ3.
  • Its phase-matching step compensates for using relative-phase estimates throughout the preceding computations.
  • For simulated and experimental cases with illumination spatial frequency within OTF support, Algorithm 2 produced reconstructions visually similar to those shown for Algorithm 1.

IX. CONCLUSION

SIM provides frequency-domain super-resolution, and OpenSIM makes its reconstruction algorithm available for interactive parameter study. The manuscript restricts its discussion to 2D SIM, while 3D SIM remains ongoing work.

  • SIM uses interference between illumination and sample spatial frequencies to recover information within the objective OTF support.Changing the illumination angle enables computation of frequency content beyond direct observation.
  • OpenSIM is a freely available Matlab-based implementation that lets users interactively manipulate parameters to minimize reconstruction errors and artifacts.The code is intended to support readers from instrumentation and biological backgrounds.
  • The manuscript restricts its discussion to 2D SIM, while 3D SIM is described as a similar extension and an ongoing project.

APPENDIX A WIENER FILTERING MULTIPLE BUT NON IDENTICAL

The appendix describes Wiener-filter estimation for multiple, nonidentical images and a matrix-based procedure for estimating an unknown point-spread function. It also explains an empirically tuned filtering parameter and concrete PSF reconstruction settings.

  • Wiener filtering: Multiple images are modeled as blurred views of the same object plus additive noise, with an equivalent Fourier-domain representation.Each image uses its own system PSF and noise term.
  • Wiener filtering: Generalized and conventional Wiener filters estimate object frequency components using the image, optical transfer function, noise power, and object power spectrum.The appendix defines the object power spectrum as the mean power spectrum of the ungraded image.
  • Wiener filtering: Replacing the denominator’s additive constant with w yields a modified filter whose parameter is empirically searched over 0 < w ≤ 1.The approximation used for the noise-filtered object becomes exact when noise power is zero.
  • PSF estimation: Object–PSF convolution can be written as O × P = I by rearranging local object and PSF elements into row and column vectors.This matrix formulation converts image formation into a linear equation.
  • PSF estimation: Unknown PSFs are estimated by constructing O and I from randomly selected interior image pixels, using more than p^2 rows for an assumed p × p PSF.Selected pixels must lie more than half a PSF width from the image edge.
  • PSF estimation: For the reported PSF determinations, a 40 × 40 PSF and r = 7 × 40 × 40 rows were used, with 100 solved PSF vectors averaged for reconstruction.The procedure was applied to PSF_deconvWF and PSF_SIM.
Loading 1602.06904v1…