Source-linked AI summary
On the use of deep learning for phase recovery
Kaiqiang Wang, Li Song, Chutian Wang, Zhenbo Ren, Guangyuan Zhao, Jiazhen Dou, Jianglei Di, George Barbastathis, Renjie Zhou, Jianlin Zhao, Edmund Y. Lam
TL;DR
Phase-recovery methods face limitations in capturing sample information and unavoidable coherent noise. This review organizes neural approaches by where they are used and reports better results from physics-constrained schemes in ill-posed cases, while noting that optimal network and loss-function choices remain inconclusive.
Problem
Conventional methods cannot capture some sample information, while coherent noise remains an unavoidable issue in phase recovery.
Method
The review categorizes neural approaches according to where they are used in the phase-recovery pipeline and discusses architectures including mixed-context convolutions.
Results
Physics-constrained schemes can infer better results than network-only approaches in ill-posed cases, including settings with unknown scattering media and target objects.
Takeaways & Limitations
The review provides a comprehensive account of neural methods for phase recovery and a live-updating resource for further learning.
Takeaways & Limitations
The best network and loss function for phase recovery remains inconclusive, and some approaches apply only to particular acquisition parameters or sample classes.
Abstract
from arXiv · showhide
Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and outlook on how to better use DL to improve the reliability and efficiency in PR. Furthermore, we present a live-updating resource (https://github.com/kqwang/phase-recovery) for readers to learn more about PR.
1. Introduction
Phase recovery infers light-field phase from intensity measurements, enabling quantitative imaging and wavefront correction despite an ill-posed inverse problem. This review introduces conventional approaches and organizes deep-learning support across preprocessing, in-processing, post-processing, and phase-image processing.
- Motivation: Phase recovery is needed because optical detectors record intensity or amplitude but not the phase required for structural and wavefront measurements.Phase can encode sample thickness, refractive-index distribution, or surface height, and is used in quantitative imaging and adaptive optics.
- Motivation: Directly recovering phase from propagated intensity is ill-posed because sensors lose phase information and sample the field with finite-area pixels.Consequently, the complex field at the sample plane generally cannot be calculated straightforwardly from a sensor intensity measurement.
- Conventional methods: Conventional phase recovery adds information through holography, interferometry, transport of intensity, or Shack–Hartmann sensing, or solves the problem iteratively with phase-retrieval algorithms.Examples include Gerchberg–Saxton–Fienup, ptychographic iterative engine, and Fourier ptychography.
- Conventional methods: In-line holography uses four phase-shifted holograms and numerical propagation to obtain a wrapped phase map before phase unwrapping, whereas off-axis holography can recover phase from one measurement through spatial spectral filtering.Off-axis designs must respect the camera Nyquist limit and typically use only a small part of the spatial spectrum.
- Review scope: The review classifies deep-learning support into preprocessing, in-processing, post-processing, and phase-image processing.These stages cover intensity enhancement or correction, direct or physics-assisted recovery, phase refinement, and downstream tasks such as segmentation and classification.
- Review scope: The review concludes by summarizing effective uses of deep learning, discussing development directions, and providing a live-updating phase-recovery resource.The resource is available at https://github.com/kqwang/phase-recovery.
2. DL-pre-processing for phase recovery
The review organizes DL-based pre-processing for phase recovery around pixel super-resolution, noise reduction, hologram generation, and autofocusing. Across these tasks, neural networks are used to enhance inputs before conventional or subsequent phase-recovery procedures.
- DL pre-processing for phase recovery covers pixel super-resolution, noise reduction, hologram generation, and autofocusing.
- For hologram pre-processing, the reviewed studies compare networks including SRCNN, ResNet, U-Net, DnCNN, FFDNet, and MPRNet across simulated and experimental datasets.The summary table records input-output task types, network choices, dataset sizes, and loss functions.
- 2.1 Pixel super-resolution: U-Net pixel super-resolution matches iterative methods in resolution improvement while reducing inference time and requiring fewer low-resolution holograms.
- 2.1 Pixel super-resolution: Pixel super-resolution networks infer high-resolution holograms from low-resolution measurements before conventional phase recovery.The approach applies to single or multiple low-resolution holograms, including sub-pixel-shifted inputs.
- 2.1 Pixel super-resolution: FSRCNN-based white-light hologram super-resolution improves 3D measurement identification and accuracy while running nearly ten times faster than SRCNN.
- 2.2 Noise reduction: DL denoising methods improve fringe-pattern processing, with reported advantages in boundary precision, inference time, and performance across different noise levels.DnCNN-based variants include sub-image and frequency-domain processing, while comparisons report advantages over WFT or earlier networks.
3. DL-in-processing for phase recovery
DL-in-processing uses neural networks either directly for intensity-to-phase inference or jointly with physical models and physics-based algorithms. Dataset-driven methods learn this mapping from paired intensity–phase data, while physics-driven methods incorporate measurement consistency during recovery.
- 3.1 Network-only strategy: Network-only strategies infer phase directly from measured intensity images using a neural network.The network input is the measured intensity image and the output is the phase.
- 3.2 Network-with-physics strategy: Network-with-physics strategies combine neural-network inference with a physical model or physics-based algorithm.The physical model can represent numerical propagation, Fourier transforms, or a Fourier ptychography measurement model.
- Dataset-driven approach: Dataset-driven recovery presupposes paired intensity images and conventional-method phase labels, then trains an end-to-end intensity-to-phase mapping.The learned mapping is internalized as network parameters updated iteratively through the loss gradient.
- Dataset-driven approach: The Sinha et al. scheme generated paired diffraction images and phase labels with a phase-only spatial light modulator, enabling tests on unseen datasets and defocus distances.The approach could not be used in practical applications because it required a phase-type spatial light modulator.
- 3.2 Network-with-physics strategy: Reported physics-integrated methods improve recovery under challenging conditions, including weak-light illumination, sparse samples, and noisy measurements.The supplied review reports stronger performance for physics-integrated recovery in dense samples under weak light, while network-only recovery is more robust for sparse samples.
- Network architectures: Several architectures improve efficiency or quality, including a network that saves 75% of parameters and a NAS-based network whose PSNR rises from 34.7 dB to 36.1 dB.The NAS-based network also increases inference speed by 27-fold.
4. DL-post-processing for phase recovery
DL post-processing is applied after phase recovery to reduce noise, enhance resolution, correct aberrations, and unwrap phase. Reported methods target both improved reconstruction quality and faster inference than conventional procedures.
- Noise reduction: 0.92 s is required to reconstruct a 2048×2048 hologram with an untrained U-Net, versus tens of seconds for conventional methods.The conventional methods require multiple holograms.
- Noise reduction: Neural networks reduce phase noise by processing wrapped phase, sine and cosine images, holograms, amplitudes, or unpaired datasets.Processing sine and cosine images avoids wrong wrapped-phase jumps that can produce larger unwrapped-phase errors.
- Noise reduction: Unpaired-data methods can reduce speckle noise without paired clean/noisy phase measurements, and cycle-GAN results were demonstrated on tomograms of different cell samples.The review contrasts this with non-data-driven methods that either blur entire images or do not remove coherent noise.
- Resolution enhancement: DL resolution enhancement combines low-resolution acquisition with inferred high-resolution phase or directly predicts high-resolution versions from raw RI tomograms.One trained U-Net retains the high acquisition speed of DPM and the high transverse resolution of SLIM.
- Resolution enhancement: The review reports resolution-enhanced methods outperforming conventional iterative methods by more than an order of magnitude in regularization.The cited method used both pixel super-resolution and diffraction-limited super-resolution systems.
- Aberration correction: CNN+ZPF segments background regions before aberration compensation, producing flatter compensated backgrounds and supporting real-time phase compensation with a phase-only SLM.The approach is presented as a segmentation-based compensation strategy.
- Phase unwrapping: Deep learning phase unwrapping addresses noise, breakpoints, and aliasing that violate the Itoh condition assumed by many conventional methods.Reported networks can unwrap noisy wrapped phase without that phase-continuity assumption and reduce the computational cost of branch-cut algorithms.
5. Deep learning for phase processing
DL phase processing extends phase images and RI tomograms into segmentation, classification, and imaging-modal transformations. These applications support biomedical analysis by extracting structures, identifying samples, and inferring fluorescence or bright-field contrasts.
- 5.1 Segmentation: Phase-based segmentation uses neural networks to divide cells, tissues, or organelles into regions of interest.Applications include sperm ultrastructure, SARS-CoV-2 detection, live–dead assays, cell-cycle detection, cancer screening, and cytometric analysis.
- 5.1 Segmentation: Phase distributions can provide a more efficient segmentation input for unlabeled cells or tissues when bright-field contrast is low.The review describes phase as the neural-network input for segmentation.
- 5.3 Imaging modal transformation: Artificial confocal microscopy combines non-destructive phase imaging with confocal fluorescence microscopy’s depth sectioning and chemical specificity.The review also describes RI-based bridges from optical diffraction tomography to fluorescence imaging.
6. Conclusion and outlook
Deep learning supports phase recovery across pre-processing, in-processing, and post-processing, but its reliability depends on dataset coverage, physical modeling, architecture, loss design, interpretability, and uncertainty estimation.
- DL support for phase recovery: Deep learning supports phase recovery before, during, and after recovery, including intensity pre-processing, direct inference, and initial-phase post-processing.The review also considers phase image processing and multiple network–physics arrangements.
- Datasets and generalization: Paired datasets encode the input-to-label mapping, so their richness and coverage constrain generalization across device parameters and sample classes.Fixed settings can restrict networks to particular defocus distances, off-axis angles, wavelengths, or sample types.
- Datasets and generalization: Dataset spatial-frequency content limits fine-feature resolution, while preprocessing power spectral density can improve this ability to some extent.Shannon entropy is proposed to quantify dataset information richness, which affects generalization ability.
- Networks and loss functions: Network architecture and loss-function choices remain unsettled because down-sampling can lose high-frequency information and common norms emphasize low frequencies.Dilated or Fourier-domain convolutions and alternatives such as NPCC, GAN, and perceptual losses are discussed.
- Physics integration: Physics-connect networks can infer better results than network-only strategies in severely ill-posed cases by letting physical propagation handle known light transport.The network can then focus on learning regularization rather than both propagation and artifact removal.
- Reliability and outlook: Interpretability and reliability remain practical concerns, although Bayesian uncertainty maps can closely match real error maps without ground truth.Interpretability failures make it difficult to diagnose unsuccessful inference, while uncertainty estimation can assess inferred-result reliability.