Source-linked AI summary
SHARP: a distributed, GPU-based ptychographic solver
Stefano Marchesini, Hari Krishnan, Benedikt J. Daurer, David A. Shapiro, Talita Perciano, James A. Sethian, Filipe R. N. C. Maia
TL;DR
Ptychography can produce large, high-resolution images, but recovering images from diffraction measurements requires computationally demanding phase retrieval. SHARP combines GPU-based high-throughput reconstruction with live feedback; the authors report successful ALS reconstructions and near-real-time feedback for microscopists.
Problem
Ptychography offers large, high-resolution imaging, but reconstructing images from diffraction intensities requires recovering phase and handling noise, experimental uncertainty, and geometry perturbations.
Source: §I · p. 1Method
SHARP is a CUDA-based, distributed GPU software environment that uses computationally efficient phase-retrieval algorithms for high-throughput streaming reconstruction.
Source: §I · p. 2, C. Computational Methodology · p. 4Results
SHARP has successfully reconstructed ALS experimental data and provides near-real-time feedback to microscopists.
Source: §III · p. 5, §IV · p. 6Takeaways & Limitations
Its modular software can be adapted to different needs and is described as suitable for demanding higher-dimensional analyses such as spectro-ptychography or tomoptychography.
Source: §IV · p. 6The current release’s Fourier-transform relationship focuses on far-field and paraxial approximations.
Source: A. Forward model · p. 3
Abstract
from arXiv · showhide
Ever brighter light sources, fast parallel detectors, and advances in phase retrieval methods, have made ptychography a practical and popular imaging technique. Compared to previous techniques, ptychography provides superior robustness and resolution at the expense of more advanced and time consuming data analysis. By taking advantage of massively parallel architectures, high-throughput processing can expedite this analysis and provide microscopists with immediate feedback. These advances allow real-time imaging at wavelength limited resolution, coupled with a large field of view. Here, we introduce a set of algorithmic and computational methodologies used at the Advanced Light Source, and DOE light sources packaged as a CUDA based software environment named SHARP (http://camera.lbl.gov/sharp), aimed at providing state-of-the-art high-throughput ptychography reconstructions for the coming era of diffraction limited light sources.
I. INTRODUCTION
Ptychography uses far-field diffraction measurements to recover sample structure, combining large-field imaging with resolution beyond the illuminating beam’s size, but phase recovery makes analysis challenging. SHARP pairs high-throughput phase retrieval with a GPU-based software environment and visual feedback during acquisition and reconstruction.
I. INTRODUCTION
- Recovering an image from measured diffraction intensities requires finding their phases, a nonlinear problem complicated by noise, experimental uncertainties, and geometry perturbations.Overlapping measurements provide redundancy that can help address missing parameters and measurement uncertainties.Source: §I · p. 1
- Earlier approaches include coherent diffractive imaging for single diffraction patterns and numerical methods such as alternating directions, gradient, Newton, and spectral methods.Existing software implementations include ptypy and PtychoLib.Source: §I · pp. 1–2
- Far-field diffraction patterns contain information about features smaller than the illuminating beam, enabling large, high-resolution images from scanned samples.Overlapping regions provide redundancy that can help overcome missing experimental parameters and measurement uncertainties.Source: §I · p. 1
- SHARP combines computationally efficient phase-retrieval algorithms with a C/CUDA backend for NVIDIA GPUs to support high-throughput streaming analysis.The backend can be accessed or adapted through a Python interface or source-code changes.Source: §I · p. 2
- At the Advanced Light Source, SHARP’s graphical interface provides visual feedback on diffraction data and reconstructed images during acquisition and reconstruction.Source: §I · p. 2
A. Forward model
The forward model describes diffraction measurements produced as a sample is scanned through an illumination beam. It represents each illuminated sample frame and its illumination in a Fourier-based formulation, with the current release focused on far-field, paraxial geometry.
A. Forward model
- Each measurement records scattered photons on an area detector as the sample is rastered across an x-ray, electron, or visible-light beam.The illumination may be formed by an x-ray optic such as a zone plate.Source: A. Forward model · p. 2
- In the discrete model, a beam of dimensions m_x × m_y illuminates a subregion, or frame, of an object of dimensions n_x × n_y, with m < n.There are K frames or views; the exposition assumes square matrices for simplicity, with non-square generalization described as straightforward but more notationally complex.Source: A. Forward model · p. 2
- The figure depicts a sample transmission rastered through an illuminating beam, producing a sequence of diffraction measurements at a detector a distance z_D from the sample.Source: Figure 1 · p. 2
- The model uses diffraction amplitudes obtained from intensity patterns and a two-dimensional discrete Fourier transform; the illuminated frames are extracted from the object and multiplied pointwise by the illumination.The stacked frames are denoted z, with F the associated 2D DFT matrix.Source: A. Forward model · pp. 2–3
- The relationship between z, the sample ψ, and illumination w can be used to find either ψ or w when the other variable is known.The illumination operator replicates w across K frames before its pointwise product with the extracted object frames.Source: A. Forward model · p. 3
- The current SHARP release’s Fourier-transform relationship is valid under far-field and paraxial approximations; other geometries can use an appropriate propagator instead.The passage names near-field, Fresnel, Fourier ptychography, through-focus, and partially coherent multiplexed geometries as examples.Source: A. Forward model · p. 3
B. Phase retrieval
SHARP uses projection operators to enforce agreement with measured data and consistency among overlapping frames, while iteratively updating the image and illumination. A typical reconstruction initializes missing inputs, applies optional constraints, and stops when a progress metric or iteration limit is reached.
B. Phase retrieval
- The data projector Pa makes frames z match the experiment, while projection onto the range of Q ensures overlapping frames are consistent with each other.Pa is relatively robust to Poisson noise, and weighting factors for noisy pixels can be added.Source: B. Phase retrieval · p. 3
- Updating both image and illumination requires correcting real-space drift by centering the illumination and suppressing Fourier-space drift or grid pathologies with a Fourier constraint.The constraint can fix either the Fourier magnitude or the support of the illumination.Source: B. Phase retrieval · p. 3
- A typical run takes measured data and translations, initializes missing illumination from the average frame and a missing image with uniform random values, then builds operators and initial frames.The image may instead be loaded or set to a constant; input data, translations, and starting images can be loaded or set through Python.Source: B. Phase retrieval · pp. 3–4
- Each iteration updates frames with the projection operators, then may retrieve illumination, apply Fourier-mask or intensity constraints, recenter the illumination, and retrieve background.The user-set β lies in (0.5, 1], defaults to 0.75, and is reported to work in most cases.Source: B. Phase retrieval · p. 4
- The iteration returns image ψ and illumination w when a progress metric falls below a user-defined threshold or the maximum iteration count is reached.Progress and stagnation are monitored using normalized mean-square-root errors from projections; simulations with a known solution can also use ε0, accounting for an arbitrary global phase.Source: B. Phase retrieval · p. 4
C. Computational Methodology
SHARP distributes ptychographic reconstruction across GPUs on multiple nodes, using framewise parallelism and image reductions to perform the required operators. Its modular controls and per-iteration synchronization options let users adapt the reconstruction and balance performance against quality.
C. Computational Methodology
- SHARP distributes the repeated Q, Q∗, F, F∗, S, and S∗ operations across GPUs on multiple nodes.The split Qψ and FFT F operations parallelize framewise; assembling Q∗z requires reductions across image pixels and MPI nodes.Source: C. Computational Methodology · p. 4
- The system assigns translations to nodes and loads their corresponding frames onto GPUs after determining the final image size from translations, frame size, and resolution.The split and overlap kernels are bandwidth-demanding and materially affect performance.Source: C. Computational Methodology · p. 4
- Kernel fusion and shared or constant memory reduce global-memory access, while delaying cross-node image summation can trade slower convergence for less frequent synchronization.Cross-node summation is performed every iteration by default.Source: C. Computational Methodology · p. 4
- The modular framework exposes illumination constraints, β, retrieval operations, and GPU synchronization frequency so users can balance reconstruction performance and quality.The schematic describes four-GPU parallelization with expected even workload division for scans of constant point density, plus default overlap and illumination agreement constraints each iteration.Source: C. Computational Methodology · p. 4, Figure 2 · p. 5
III. APPLICATIONS AND PERFORMANCE
SHARP is evaluated with simulations and experimental data, including streaming reconstruction for rapid feedback. The reported simulation processes 10,000 frames in under two seconds, and ALS measurements across several material applications were successfully reconstructed.
III. APPLICATIONS AND PERFORMANCE
- The evaluation uses simulations to compare convergence with a known solution and explore effects of light source, contrast, scale, noise, detectors, and samples.The authors identify the simulations as a way to study conditions for which experimental data do not yet exist.Source: §III · p. 5
- Experimental ALS data for battery materials, green cement, and magnetic materials at different wavelengths and orientations were successfully reconstructed with SHARP.The paper also describes streaming data from an experiment-proximal frontend to a remote GPU/CPU-cluster backend.Source: §III · p. 5
- ε0 < 5e −4 reconstruction of 10,000 128×128 frames from a 1000×1000 image takes under 2 seconds on 16 GTX Titan GPUs across four compute nodes.The simulation supplied the exact illumination, used periodic boundary conditions, and reported rapidly decreasing residuals.Source: Figure 3 · p. 5
A. Simulations and performance
The simulation demonstration starts from a nanoparticle sample imaged by scanning electron microscopy and constructs a complex transmission function and zone-plate illumination for reconstruction.
A. Simulations and performance
- The sample contains 50 nm and 10 nm colloidal gold nanoparticles deposited on a transparent silicon nitride membrane.The starting image comes from scanning electron microscopy, which provides surface information at high resolution and contrast.Source: A. Simulations and performance · p. 5
- The simulation setup uses an SEM-derived sample image and modeled transmission and zone-plate illumination as its inputs.These are setup details; the supplied passages do not report a reconstruction outcome for this demonstration.Source: A. Simulations and performance · p. 5
- The simulated complex transmission function scales image amplitude to 0–50 nm thickness and uses gold's complex refractive index at 750 eV.The illumination is a 220-micron-diameter zone plate with 60 nm outer zone width, discretized into 128×128 far-field pixels.Source: A. Simulations and performance · p. 5
B. Experimental example
The experimental example reconstructs a gold-ball test sample from ALS soft X-ray data using SHARP. It describes the reconstruction settings and compares images produced with and without two algorithm options.
B. Experimental example
- The test sample contains gold balls with 50 nm and 10 nm diameters, and the reconstructed phase-image pixel size is 10 nm.The data were generated using 750 eV X-rays at ALS beamline 5.3.2.1.Source: B. Experimental example · p. 5, Figure 4 · p. 6
- Figure 4 compares a reconstruction using an illumination Fourier mask and background retrieval with variants omitting each option separately.Panel A uses both options; B omits the mask constraint, and C omits background retrieval.Source: Figure 4 · p. 6
- Reconstructions used 300 RAAR iterations, with illumination and background retrieval performed every other iteration.The initial illumination was derived from averaged measurements, thresholded, and inverse Fourier transformed.Source: B. Experimental example · p. 6
C. Interface and Streaming
The streaming pipeline integrates experimental data acquisition and ptychographic analysis so users can monitor and respond to changes. Its frontend operates near the experiment, while backend workers handle processing tasks.
C. Interface and Streaming
- The pipeline is designed to let users monitor and quickly act on changes across the experimental and computational workflow.It addresses processing pipelines involving multiple I/O operations and components.Source: C. Interface and Streaming · p. 6
- The frontend includes a GUI, a detector-frame worker, and an interface monitoring network activity and experimental parameters while showing reconstruction progress.The frontend operates close to the experiment, and the GUI receives a live reconstruction view as diffraction data is recorded.Source: C. Interface and Streaming · p. 6, Figure 5 · p. 6
- Backend workers handle dark calibration, detector correction, data reduction, ptychographic reconstruction, and output writing.The backend runs remotely on a GPU/CPU cluster.Source: C. Interface and Streaming · p. 6, Figure 5 · p. 6
- The integrated architecture provides responsive monitoring and control intended to deliver the feedback users expect from an STXM instrument.Source: C. Interface and Streaming · p. 6
IV. CONCLUSIONS
SHARP is a high-performance ptychography environment used in an ALS quick-feedback system. Its modular interface and fast throughput support adaptation and near-real-time feedback, with potential use in higher-dimensional analyses.
IV. CONCLUSIONS
- SHARP is a high-performance ptychography environment applied in a quick-feedback system at the ALS.Source: §IV · p. 6
- Its modular interface can be adapted to different needs, and its throughput provides near-real-time feedback to microscopists.Source: §IV · p. 6
- The authors identify spectro-ptychography and tomoptychography as demanding higher-dimensional analyses for which SHARP may serve as a cornerstone.Source: §IV · p. 6
- Faster detectors and next-generation light sources make rapid analysis of large datasets for high-dimensional imaging an enabling tool for science.Source: §IV · p. 6