Source-linked AI summary
Neural Blind Deconvolution for Image Scanning Microscopy
Eliot Guez, Nathan Buskulic, Lisa Cuneo, Luca Calatroni, Giuseppe Vicidomini
TL;DR
Theoretical PSFs used in ISM may fail to capture experimental aberrations, limiting the models used to fuse detector-element images. This paper proposes blind DIP that jointly estimates the specimen and PSFs through shared excitation and emission PSFs tied to detector geometry. On experimental tubulin data, it produces visually finer filament structures than MID and fixed-theoretical-PSF DIP, while estimating PSFs that differ from theory.
Problem
Theoretical PSFs may not capture experimental aberrations, even though scanned-image differences encode information about them.
Source: §1 · p. 1Method
Results
Blind DIP produces visually more pronounced fine tubulin filaments than MID and DIP with fixed theoretical PSFs, while its recovered effective PSFs differ from theoretical ones.
Source: §4.1 · p. 3Takeaways & Limitations
The qualitative tubulin results support further investigation, including PSF stability and physical interpretation and quantitative validation on suitable reference measurements.
Source: §5 · p. 3The non-convex objective has no claimed convergence guarantee, and available analyses do not directly cover the blind setting.
Source: §3 · p. 2
Abstract
from arXiv · showhide
Image scanning microscopy (ISM) acquires one scanned image per detector element, which are then fused into a single image by multi-image deconvolution to obtain a super-resolved image. Its accuracy, however, depends on point spread function (PSF) models that sample- and system-induced aberrations often invalidate. Since differences between the scanned images encode aberration information, we propose a training-free blind Deep Image Prior (DIP) framework that jointly estimates the specimen and the detector-dependent PSFs. The PSFs are parametrised through the detector geometry and shared excitation and emission PSFs, reducing the unknowns while preserving the physical ISM model, and a Gaussian pre-initialisation stabilises the blind optimisation. On experimental tubulin data, blind DIP recovers finer filamentous structures than multi-image deconvolution and DIP with fixed theoretical PSFs, while adapting the PSFs to the measurements.
1. INTRODUCTION
ISM combines complementary scanned images from detector-array elements to produce super-resolved images, but theoretical PSFs may miss experimental aberrations. The paper proposes blind DIP reconstruction that uses image differences and detector geometry to estimate the specimen and PSFs jointly.
1. INTRODUCTION
- Differences among detector-element images encode aberration information, but theoretical PSFs may not capture experimental aberrations.ISM elements collect complementary spatial information while preserving a high photon budget.Source: §1 · p. 1
- The blind inverse problem jointly recovers the specimen and 25 detector-dependent PSFs from Poisson-modeled observations.The SPAD array detects individual photons without readout noise.Source: §1 · p. 1
- Blind deconvolution has nonunique solutions, including scale and opposite-translation ambiguities, and real paired ISM training data are difficult to obtain.The no-blur solution is another classical ambiguity in single-image blind deconvolution.Source: §1 · p. 1
- The proposed blind DIP method derives detector-dependent kernels from shared excitation and emission PSFs and detector positions, reducing unknowns while preserving ISM structure.A pre-initialisation strategy for kernel-generating networks stabilises optimisation and improves robustness to translation ambiguities.Source: §1 · p. 1
- Experimental ISM results show promising qualitative reconstructions with the blind approach.The introduction reports qualitative results without specifying a quantitative comparison.Source: §1 · p. 2
2. IMAGE SCANNING MICROSCOPY
The ISM forward model relates each detector element's effective PSF to shared excitation and emission PSFs and its spatial detection response. With known kernels, multi-image deconvolution estimates the specimen from the scanned images.
2. IMAGE SCANNING MICROSCOPY
- Each detector element has a shifted copy of a common binary detection response, under an assumption of uniform detection efficiency.The shift corresponds to the element's position in the image plane.Source: §2 · p. 2
- Each effective PSF is narrower than the diffraction-limited widefield emission PSF, providing the basis for ISM super-resolution.The effective PSF combines excitation with detection derived from emission and the element response.Source: §2 · p. 2
- With fixed detector geometry, all channel-dependent effective PSFs are determined by the shared excitation and emission PSFs.The discretized model combines the excitation PSF with the emission PSF convolved with each shifted aperture.Source: §2 · p. 2
- When the effective PSFs are known, specimen recovery is posed as a multi-image Kullback–Leibler divergence problem, typically solved with an extended Richardson–Lucy scheme.The conventional model calculates excitation and emission PSFs using vectorial diffraction theory.Source: §2 · p. 2
3. DEEP PARAMETRISATION FOR BLIND ISM RECONSTRUCTION
The method adapts blind SelfDeblur to ISM by jointly estimating the specimen and shared excitation and emission PSFs, with detector geometry coupling the effective kernels. A Gaussian pre-initialisation addresses translation sensitivity, while the non-convex optimisation has no convergence guarantee.
3. DEEP PARAMETRISATION FOR BLIND ISM RECONSTRUCTION
- The method adapts SelfDeblur to jointly estimate the specimen and shared excitation and emission PSFs, coupling detector-dependent kernels through geometry.This retains physical relationships that would be lost by estimating each kernel independently.Source: §3 · p. 2
- Kernel normalisation removes scale ambiguity while preserving relative channel intensities, and a Tikhonov term stabilises kernel estimation.Non-negativity and normalisation are enforced by the kernel-network output construction.Source: §3 · p. 2
- The objective is differentiated by backpropagation and minimised with Adam, but its non-convexity means no convergence guarantee is claimed.The cited analyses do not directly cover the blind setting.Source: §3 · p. 2
3.1. On the choice of the initialisation
- A centered isotropic Gaussian kernel pre-initialises the kernel network to mitigate sensitivity to translated solutions.Centered solutions were reported in practice across noise levels, initialisations, and choices of σ.Source: §3.1 · pp. 2–3
- The output-level stabilisation requires no training set; pretraining also reshapes the kernel network’s Jacobian to help keep early optimisation near its pretrained state.The authors report that a few singular values become much more important than others.Source: §3.1 · p. 3
4. NUMERICAL RESULTS
On experimental tubulin data, blind DIP produces visually more pronounced fine filament structures than MID and DIP with fixed theoretical PSFs. Its recovered PSFs differ from theoretical ones, although peripheral intensity in the latent PSFs is considered likely artifact.
4. NUMERICAL RESULTS
- The experiment used a custom ISM system with a 5 × 5 SPAD array and compared blind DIP with MID and fixed-theoretical-PSF DIP.MID used a multi-image Richardson–Lucy scheme and theoretical detector-dependent PSFs.Source: §4 · p. 3
4.1. Qualitative comparison
- Blind DIP produces visually more pronounced fine tubulin filament structures than MID and DIP with fixed theoretical PSFs.The authors suggest this may reflect adaptation to optical aberrations affecting acquisition.Source: §4.1 · p. 3
- The recovered effective PSFs differ from theoretical PSFs in apparent extent and spatial distribution, while peripheral artifacts are largely suppressed in the effective kernels.The excitation PSF’s diagonally elongated central lobe is described as consistent with astigmatism; peripheral intensity is considered optically implausible and probably an estimation artifact.Source: §4.1 · p. 3
5. CONCLUSION
The paper presents a blind DIP approach that jointly estimates the specimen and excitation and emission PSFs for ISM. On experimental tubulin data, it reveals fine filament details and estimates PSFs that differ from the theoretical model; further validation remains an open direction.
5. CONCLUSION
- The approach jointly estimates the specimen image and excitation and emission PSFs for ISM.Detector-dependent effective PSFs are constructed from the shared PSFs and detector geometry, incorporating their physical relationships.Source: §5 · p. 3
- On experimental tubulin data, the reconstruction reveals fine filament details and estimates PSFs that differ from the theoretical model.Source: §5 · p. 3
- The qualitative findings motivate further study of recovered-PSF stability and physical interpretation, plus quantitative validation using suitable reference measurements.Source: §5 · p. 3