Source-linked AI summary
Deep-STORM: super-resolution single-molecule microscopy by deep learning
Elias Nehme, Lucien E. Weiss, Tomer Michaeli, Yoav Shechtman
TL;DR
Dense overlapping emitters make localization microscopy slow and computationally demanding, while existing methods often require iterative procedures and parameter tuning. Deep-STORM uses a convolutional neural network trained on simulated or experimental data to reconstruct super-resolved images directly. It achieves faster processing and improved reconstruction outcomes across simulated and experimental evaluations, without requiring prior structure knowledge.
Problem
Dense overlapping emitters create computational and acquisition-time challenges, while existing methods can require time-consuming iteration and parameter tuning.
Method
Deep-STORM trains a fully convolutional neural network on simulated or experimentally obtained images to output super-resolved frames directly from raw blinking-emitter data.
Results
Deep-STORM outperforms FALCON with 37% compared to 61% NMSE on simulated data and experimentally trained models detect 96% compared to 88% of emitters with lower false-positive rates.
Takeaways & Limitations
The method combines state-of-the-art resolution enhancement, high speed, and parameter-free operation for super-resolution localization-microscopy data.
Abstract
from arXiv · showhide
We present an ultra-fast, precise, parameter-free method, which we term Deep-STORM, for obtaining super-resolution images from stochastically-blinking emitters, such as fluorescent molecules used for localization microscopy. Deep-STORM uses a deep convolutional neural network that can be trained on simulated data or experimental measurements, both of which are demonstrated. The method achieves state-of-the-art resolution under challenging signal-to-noise conditions and high emitter densities, and is significantly faster than existing approaches. Additionally, no prior information on the shape of the underlying structure is required, making the method applicable to any blinking data-set. We validate our approach by super-resolution image reconstruction of simulated and experimentally obtained data.
1 Introduction
Localization microscopy enables nanoscale imaging but struggles with densely overlapping emitters, slow acquisition, iterative computation, and parameter tuning. Deep-STORM addresses these constraints with direct deep-learning reconstruction that is fast, parameter-free, and structure-agnostic.
- High emitter densities create overlapping point-spread functions that challenge localization-microscopy algorithms.The resulting emitter-sparsity constraint can require acquisition times from seconds to minutes, limiting imaging of fast cellular dynamics.
- Existing dense-emitter methods use sequential fitting, blinking statistics, sparsity, maximum likelihood, or dictionary learning.These approaches can localize densely spaced emitters but remain subject to broader computational or parameter-selection drawbacks.
- Iterative sparse-recovery methods remain time-consuming and scale poorly with recovered grid size.Their tradeoff parameters require careful trial-and-error tuning and user expertise.
- Deep-STORM uses a fully convolutional neural network to reconstruct super-resolved images directly from raw dense-emitter data.It does not explicitly produce a localization list or require prior knowledge of the sample structure.
- Deep-STORM is presented as precise, fast, parameter-free, and applicable to single-molecule datasets without user expertise.GPU computation can further enhance its speed, while training can use simulated or experimental data.
2 Methods
Deep-STORM processes sets of blinking-emitter frames with a fully convolutional encoder-decoder, producing one super-resolved prediction per frame before summation. Training uses simulated or experimental data and a regression loss combining smoothed reconstruction error with sparsity regularization.
- 2.1.1 Architecture: Deep-STORM maps a set of low-resolution emitter frames to per-frame super-resolved images, which are summed into one final reconstruction.The network receives possibly dense point emitters and outputs images rather than a localization list.
- 2.1.1 Architecture: The architecture is a fully convolutional encoder-decoder that compresses image features and restores the original spatial dimensions.It uses convolutional and pooling layers during encoding, followed by upsampling and convolutional layers during decoding.
- 2.1.2 Training: Training data consist of 10K pairs of upsampled low-resolution regions and high-resolution images containing spikes at ground-truth emitter positions.Twenty simulated 64 × 64 images generate 500 random 26 × 26 regions each, upsampled by a factor of 8 to 208 × 208 pixels.
- 2.1.3 Loss Function: The loss combines squared ℓ2 distance between predicted and Gaussian-smoothed ground-truth images with an ℓ1 sparsity penalty.The ground truth contains delta functions at emitter positions, while the prediction is the network’s super-resolution frame.
- 2.1.3 Loss Function: The network is trained for 100 epochs with batches of 16 samples using Adam and default optimizer parameters.The Gaussian kernel has σ = 1 pixel and the initial learning rate is 0.001.
3 Results
Deep-STORM reconstructs simulated and experimental super-resolution data across dense emitters, microtubules, and quantum dots. It resolves fine structures, matches or improves competing methods, and provides substantial runtime gains.
- Simulated resolution: 19 nm at 1 [ emitter µm2 ] and 31 nm at 9 [ emitter µm2 ] were the minimal resolvable stripe separations under simulated conditions.Resolution decreased as emitter density increased; the analysis used 1000 signal photons and 10 background photons per pixel.
- Simulated microtubules: Deep-STORM achieved 37% NMSE on simulated microtubules, compared with 72% for CEL0 raw histograms and 69 for Gaussian-convolved CEL0.The CEL0 Gaussian used σ = 1 pixel, selected to minimize its NMSE.
- Simulated microtubules: Deep-STORM better recovered nearby microtubule edges and underlying curvature than CEL0 in simulated reconstructions.The comparison highlights improved recovery of twisted structure and nearby edges.
- Experimental microtubules: 37% NMSE versus 61% for FALCON was reported on simulated data, while Deep-STORM produced better-resolved structures on experimental microtubules.The experimental comparison also found more continuous shapes than CEL0.
- Quantum dots: 96% of emitters were detected with a 1.6% false-positive rate, compared with 88% and 8.7% for a simulation-trained net on the same experimental images.The experimentally trained network outperformed the simulation-trained network, and a small number of experimental images was sufficient for high-quality training.
- Runtime: ∼1 −3 orders of magnitude faster was the reported runtime advantage over leading algorithms, with GPU-accelerated processing at ∼20000 emitters per second versus ∼1500 for FALCON.Table 1 compares runtimes on simulated and experimental microtubule datasets.
4 Discussion
Deep-STORM addresses the computational cost and parameter-tuning demands of high-density emitter fitting with a fast, precise, parameter-free convolutional-neural-network method. Experiments indicate strong performance across emitter densities and signal-to-noise conditions, including video-rate analysis.
- Deep-STORM provides fast, precise, parameter-free super-resolution imaging for localization-microscopy data.
- The method uses a convolutional neural network trained on easily simulated or experimentally obtained data rather than explicitly compiling molecular positions.
- Deep-STORM performs well up to an emitter density of ∼6 [ emitter µm2 ], comparable to leading multi-emitter fitting methods after parameter tuning.The maximal allowable density also depends on SNR.
- Training data can be generated straightforwardly in large numbers because realistic noisy single-molecule images are easy to simulate.
- Pre-training multiple networks for different SNR and density values is proposed for time-varying experimental conditions.This generalization is possible because training is performed offline.
- Deep-STORM combines resolution enhancement, speed, and parameter-free operation to support video-rate analysis without requiring end-user expertise.
Funding Information
The paper acknowledges institutional, foundation, fellowship, chairship, grant, and hardware support.
- E.N. received support from a Google research award.
- L.E.W. and Y.S. received support from the Zuckerman Foundation, with Y.S. also receiving additional chairship support.
- T.M. received support from multiple foundations, an Alon Fellowship, and Israel Science Foundation grant No. 852/17.
Supplemental Documents
Supporting information is provided in the supplementary material.
- The supplementary material contains supporting information for the paper.