On the use of deep learning for phase recovery

• Phase carries primary information for transparent samples, aberrated wavefronts, and surface profiles, where phase delay relates to refractive-index distribution, atmospheric aberration, or surface height.These applications support quantitative bioimaging, adaptive aberration correction, and material inspection.Source: §1 · p. 2
• Because optical detectors record intensity but not phase, recovering the sample-plane complex field from sensor measurements is generally ill-posed and complicated by finite pixel sampling.Propagation couples phase changes to amplitude changes, but intensity-only detection loses the phase needed for straightforward numerical back-propagation.Source: §1 · p. 3
• Classical phase recovery makes the problem deterministic by adding information through holography, interferometry, transport-of-intensity measurements, or Shack–Hartmann sensing, or solves it iteratively with phase-retrieval algorithms.The introduction names Gerchberg-Saxton-Fienup, ptychographic iterative engine, and Fourier ptychography as iterative examples.Source: §1 · pp. 3–4
• Holography converts phase differences into measurable interference intensity: in-line methods use multiple phase-shifted holograms, whereas off-axis methods recover phase from one measurement through carrier-frequency filtering.In-line recovery includes numerical propagation, arctangent-based wrapped phase extraction, and phase unwrapping; off-axis recovery selects a spatial-frequency order before inverse Fourier transformation.Source: §1 · p. 4
• Off-axis holography must respect the camera Nyquist limit and typically wastes spatial bandwidth because only a small spectral region is used, while TIE relies on scalar, paraxial, and weak-defocusing approximations.Digital holographic microscopy can omit numerical propagation by conjugating the sensor and sample planes with additional imaging components.Source: §1 · p. 5

Conventional phase recovery

• TIE recovers continuous phase without phase unwrapping, but it is limited to paraxial approximation and weak defocus.Axially shifted intensity measurements approximate the axial intensity derivative used for TIE.Source: §1 · p. 6
• Shack-Hartmann sensing integrates phase gradients estimated from focal-point displacements across a microlens array, with resolution determined by microlens density.Quantitative differential interference contrast microscopy, quantitative differential phase contrast microscopy, and quadriwave lateral shearing interferometry may achieve higher resolution than Shack-Hartmann sensing.Source: §1 · p. 7
• Phase recovery is ill-posed from a single propagated intensity measurement, so conventional methods add constraints or prior information through regularization.Gerchberg–Saxton, error reduction, hybrid input-output, multi-intensity alternating projection, ptychographic iterative engine, Fourier ptychography, Wirtinger flow, truncated amplitude flow, and PhaseLift are described as representative approaches.Source: §1 · pp. 7–8

Deep learning for phase recovery

• Interest in deep learning for phase recovery is rapidly growing after deep learning demonstrated strong potential for computational imaging.Most implementations use supervised learning with paired input-label datasets, and CNNs are the most widely used neural networks for phase recovery.Source: §1 · p. 10

Review scope

• The review organizes methods by neural-network placement, distinguishing DL pre-processing of intensity measurements from DL in-processing that directly performs or participates in phase recovery.Pre-processing includes pixel super-resolution, noise reduction, hologram generation, and autofocusing; in-processing combines neural networks with physical models or physics-based algorithms.Source: §1 · pp. 11–12

• DL post-processing improves recovered phases through noise reduction, resolution enhancement, aberration correction, and phase unwrapping.Source: §1 · p. 12
• DL for phase processing uses recovered phase images for segmentation, classification, and imaging modal transformation.Source: §1 · p. 12
• The review summarizes effective DL use in PR, discusses potential development directions, and provides a live-updating resource for further learning.The resource is available at https://github.com/kqwang/phase-recovery.Source: §1 · p. 12

2.1 Pixel super-resolution

• DL-based pixel super-resolution improves hologram resolution before conventional phase recovery, with U-Net inference preserving resolution gains using fewer low-resolution inputs. [50]U-Net offers faster inference than iterative methods, while maintaining the same resolution improvement and high performance with fewer inputs.Source: §2.1 · p. 14
• FSRCNN improves white-light hologram super-resolution and 3D measurement accuracy while running nearly ten times faster than SRCNN. [52]HG-SRCNN also infers higher-quality high-resolution holograms from defocus in-line holograms, while ResNet/SubPixelNet reconstruction outperforms conventional interpolation methods and performs best with l2-norm training. [51, 53]Source: §2.1 · p. 14

2.2 Noise reduction

• Noise from coherent illumination, environmental disturbances, and image-sensor recording should be reduced before phase recovery, but filter-based methods trade filtering quality against time cost. [89]Deep CNN denoising, including DnCNN-based approaches, was introduced for fringe-pattern denoising (Fig. 14). [90]Source: §2.2 · p. 15
• DL fringe-pattern denoising improves boundary precision and inference speed, while frequency-domain DnCNN outperforms earlier methods across noise levels and enhanced U-Net variants preserve details with lower complexity.DnCNN-based methods reduce noise through spatial or frequency-domain processing; V-Net retains more details through asymmetric channel allocation, while dense and residual learning further reduce model complexity.Source: §2.2 · pp. 15–16

2.3 Hologram generation

• DL hologram generation enables phase recovery from fewer measurements, with hologram-domain inference generalizing better than direct phase recovery when training and testing phase structures differ substantially.Networks generate phase-shifted, defocused, or multi-wavelength holograms from known holograms, using methods including Y-Net, GANs, MPRNet, and alternating projection; source-target similarity improves generalization, but multi-wavelength generation can exhibit information crosstalk.Source: §2.3 · pp. 16–17

2.4 Autofocusing

• Continuous regression provides more precise defocus estimation than discrete classification, while neural networks outperform other machine-learning methods and ViTs outperform typical CNNs for autofocus.Autofocus networks estimate defocus distance from holograms through classification or regression; regression superiority was verified with amplitude and phase samples under varying exposure times and incident angles, and alternatives include ROI, untrained, and spectrum-based inputs.Source: §2.4 · p. 18

3.1 Network-only strategy

• The network-only strategy maps measured intensity images directly to phase, whereas dataset-driven methods train end-to-end networks on paired intensity–phase data.Conventional methods generate ground-truth phase labels, and iterative loss updates internalize the intensity-to-phase mapping for inference.Source: §3.1 · pp. 19–20
• Sinha et al.95 demonstrated end-to-end phase recovery from diffraction images, with adaptability to unseen dataset types and defocus distances, although the SLM-based setup was impractical.Their SLM generated known phase labels and corresponding diffraction inputs in large quantities.Source: §3.1 · p. 21
• Li et al.96 doubled spatial resolution using NPCC and power-spectral-density flattening, while Deng et al.97 linked higher training-dataset Shannon entropy to stronger generalization.These studies show that loss design and dataset statistics materially affect network-only phase-recovery performance.Source: §3.1 · pp. 21–22
• End-to-end deep learning was applied to Fourier ptychography and extended to CDI, while reported applications described superiority over conventional algorithms, improved robustness, direct phase inference, and faster 3D ODT reconstruction.Source: §3.1 · p. 22
• Network-only designs use specialized architectures to recover phase, amplitude, or complex fields, including Y-Net, RedCap, MCN, FIN, NAS-based networks, and resolution-enhancing models.Y-Net simultaneously infers amplitude and phase; RedCap saves 75% of network parameters while maintaining higher holographic reconstruction accuracy; MCN is more accurate and compact than U-Net in one-sample learning.Source: §3.1 · pp. 22–23
• FIN generalizes better to new sample types and runs 9-fold faster than a previous recurrent network and 27-fold faster than conventional iterative algorithms.NAS further raises PSNR from 34.7 dB to 36.1 dB over conventional U-Net while increasing inference speed 27-fold.Source: §3.1 · p. 23
• Physics-driven approaches replace ground-truth phase with measured intensity and a physical model, enabling untrained iterative inference or trained inference on unseen intensities.The untrained scheme optimizes a network for each measurement, whereas the trained scheme pre-trains with many intensity images before inferring unseen samples.Source: §3.1 · pp. 23–25
• Untrained physics-driven recovery is unsuitable for many dynamic applications because it needs many iterations, and large-scale inference can be prohibitively expensive.Inferring 3,600 diffraction images with the uPD scheme was reported to require as much as 600 hours; explicit denoising can mitigate noise overfitting from single-measurement inputs.Source: §3.1 · pp. 25–26
• Pre-training supports diffraction-image inference by training a neural network on a small part of measured diffraction images to infer remaining ones, while iterative refinement is 100 and 10 times faster than conventional iterative methods.Source: §3.1 · p. 26
• Network-only learning with generated data and a physics-consistency loss showed superior external generalization to holograms of real tissues with arbitrary conditions.Source: §3.1 · p. 26

3.2 Network-with-physics strategy

• Network-with-physics strategies connect physical models and neural networks serially, integrate networks into physics algorithms, or embed physics into networks.These configurations are termed physics-connect-network (PcN), network-in-physics (NiP), and physics-in-network (PiN).Source: §3.2 · p. 26
• Physics-connect-network reduces required holograms by 2-3 times and improves computation time by more than three times over axial multi-intensity alternating projection.Source: §3.2 · p. 28
• Network-only recovery is more robust to sparse samples, whereas PcN performs better inference on dense samples and is stronger under weak-light illumination, especially at 1 photon.Source: §3.2 · p. 28
• Physics-informed training and architectures extend phase recovery to finer details, improved resolution and noise robustness, recurrent processing, denoising regularization, and online inference.Examples include VGG perceptual loss, separate low- and high-frequency networks, PrDeep with a DnCNN denoiser and FASTA solver, and an online PrDeep extension.Source: §3.2 · pp. 28–29
• Hybrid solver designs embed neural denoisers or total-variation priors into HIO or ADMM, removing artifacts, regularizing recovery, and improving noise robustness.Related approaches expand PrDeep to complex-field recovery and combine multiple unrolled or domain-specific operators for phase recovery [152–154,173–176].Source: §3.2 · pp. 29, 32
• Deep image prior (DIP) uses an untrained neural network with trainable parameters as a structural prior for regularization, taking measured intensity images and a fixed random vector as latent code [180,181].Source: §3.2 · pp. 29–30
• DIP-based phase recovery achieves better reconstructions at low subsampling and handles extreme cases including low photon counts and beamstop-obscured frequencies, outperforming gradient-descent and projected-gradient methods.Source: §3.2 · p. 30
• Generative neural networks convert multidimensional phase recovery into searching a low-dimensional tensor while limiting the solution space to the trained network’s output range.The generative-prior algorithm is robust to low subsampling ratios and strong noise.Source: §3.2 · p. 31
• Physics-in-network methods unroll iterative phase-recovery algorithms into interpretable neural networks with trainable parameters and end-to-end training [182].The unrolled deGEC-SR network is more robust with fewer iterations, while integrating forward and inverse physics models enables efficient training with few datasets and transfer learning [170,175].Source: §3.2 · pp. 32–33

4.1 Noise reduction

• Noise-reduction methods operate after phase recovery, using U-Net, cycle-GAN, or sine-and-cosine-image processing to suppress noise while preserving sample features.Jeon et al.183 processed reconstructed holograms end-to-end, whereas Choi et al.184 trained with unpaired datasets; Yan et al.186,187 denoised sine and cosine images before arctangent phase calculation.Source: §4.1 · pp. 34–35
• 0.92 s is required for a 2048×2048 reconstructed hologram, compared with tens of seconds for conventional methods requiring multiple holograms.The reported speed comes from Jeon et al.183’s deep-learning method for end-to-end speckle noise reduction.Source: §4.1 · p. 34
• Direct wrapped-phase denoising can create wrong jumps and larger unwrapped-phase errors, motivating denoising of sine and cosine images instead.Montresor et al.188 used simulated paired data for speckle reduction, obtaining comparable standard deviation to WFT, better peak-to-valley, and lower time cost.Source: §4.1 · p. 35

4.2 Resolution enhancement

• Resolution-enhancement networks infer high-resolution phase from low-resolution measurements, retaining either faster acquisition or wider-field imaging advantages.Jiao et al.196 used a U-Net to infer noise-free high-resolution phase while retaining DPM’s acquisition speed and SLIM’s transverse resolution; other methods target higher spatial bandwidth product or fewer measurements.Source: §4.2 · p. 36
• Resolution enhancement is also extended to partially coherent holography, structured-illumination holography, and qDPC imaging using paired or cascaded neural-network designs.These methods use high-resolution phase as ground truth, may combine low-resolution phase with inferred amplitude, and can address large inputs through sub-patching.Source: §4.2 · p. 36
• 3D U-Nets infer high-resolution RI tomograms from raw low-axial-resolution reconstructions, outperforming conventional iterative methods by more than an order of magnitude in regularization.The approach was demonstrated on bacterial cells and a human leukemic cell line, addressing the missing cone problem caused by limited projection angles.Source: §4.2 · p. 37

4.3 Aberration correction

• CNN+ZPF segments sample and background regions before Zernike fitting, producing flatter compensated backgrounds than PCA and enabling real-time phase compensation with a phase-only SLM [203, 205].The segmentation step avoids estimating aberration from object regions, while hologram-based input was proposed to mitigate failures from weak boundary phases and large tilted aberrations.Source: §4.3 · pp. 37–38

4.4 Phase unwrapping

• Deep-learning phase unwrapping includes regression (dRG), which estimates absolute phase directly from wrapped phase; wrap-count (dWC), which estimates wrap count before calculating absolute phase; and assisted (dAS), which estimates wrap-count gradients or discontinuities [209–222, 185,223–233].Source: §4.4 · p. 39
• dRG methods demonstrated superiority over conventional methods in noisy and aliasing cases, while congruence improved dRG accuracy and robustness, especially with limited wrap count [211, 214].The literature also explored U-Net, residual, recurrent, LSTM, GAN, and larger-capacity architectures, alongside preprocessing, skip connections, composite losses, and specialized noise handling.Source: §4.4 · pp. 39–40
• Residual wrap-count errors can propagate through phase maps, motivating refinement or median-filter post-processing to correct wrong pixels.Huang et al.234’s HRNet method still requires median filtering, although it performs better than Phase-Net and DeepLab-V3+.Source: §4.4 · pp. 40–41
• Adding real-data noise to simulated training data and combining FRRNet with U-Net yields higher accuracy than Phase-Net and DeepLab-V3+ on real data.This approach was developed for Doppler optical coherence tomography.Source: §4.4 · p. 41
• Other post-processing designs include Y-Net-based wrap reconstruction, quality-map-guided estimation of the wrap-count gradient, and single-fusion-gradient network estimation to improve efficiency.Source: §4.4 · p. 41
• U-Net enhancements using ASPP, PSA, and EEB improve accuracy and robustness, while a Laplacian- and Prewitt-based EEB further improves classification accuracy and avoids post-processing.Source: §4.4 · p. 41
• Neural estimation of wrap-count gradients recovers correct results from noisy wrapped phases without relying on the phase-continuity assumption.The result is obtained by minimizing the difference between unwrapped-phase gradients and the network output.Source: §4.4 · p. 41
• Embedding a neural network into a branch-cut algorithm to predict branch-cut maps from residual images reduces the algorithm’s computational cost.Source: §4.4 · p. 42

5. Deep learning for phase processing

• Phase processing is organized around segmentation, classification, and imaging modal transformation.This part covers the segmentation and classification subsections.Source: §5 · p. 42

5.1 Segmentation

• When bright-field intensity is low and inefficient for image segmentation, phase distributions of cells or tissues offer an alternative, with phase used directly as neural-network input.Source: §5.1 · p. 43
• Phase-based segmentation has been applied to sperm ultrastructure, SARS-CoV-2 detection, cell viability and cycle-stage analysis, cancer screening, and cytometric analysis.Reviewed methods include mask R-CNN for cancer screening and DeepLab-V3+ for cytometric analysis.Source: §5.1 · p. 44

5.2 Classification

• Phase is well suited to classifying transparent samples because it encodes information related to refractive index and three-dimensional topography, including cells, tissues, and microplastics267,305.Unlike conventional approaches that manually extract features before classification, deep learning takes phase or derived representations directly as input and automatically performs feature extraction.Source: §5.2 · p. 44
• Phase-based deep-learning classification generally achieves accuracy higher than 95% for binary classification but cannot achieve comparable accuracy in multi-type classification.Combining neural-network and manually extracted features can improve classification accuracy because manual features add human-expert priors.Source: §5.2 · p. 45
• Neural-network inputs incorporating 3D refractive-index tomograms from phase data yielded 99.6% accuracy for binary lymphoid–myeloid classification and 96.7% accuracy in another classification setting.Source: §5.2 · p. 45
• Direct classification from raw holograms is feasible because holograms encode amplitude and phase, with demonstrated applications in molecular diagnostics, microplastic pollution assessment, and neuroblastoma cell classification.325–329Time-lapse detection of growing bacteria shortened detection time by >12 h compared with the environmental- baseline.Source: §5.2 · pp. 46–47

5.3 Imaging modal transformation

• Deep networks transform phase-derived images into bright-field modalities, including speckle- and artifact-free images and whole 3D volumetric reconstructions of color samples from single-snapshot holograms.289Cycle-GAN implementations extend bright-field holography to unpaired datasets.290Source: §5.3 · p. 48
• Virtual staining transfers phase images of label-free tissue into bright-field stain-like images, with PhaseStain demonstrated on human skin, kidney, and liver slices and reported to save staining time and costs.291The method was compared with bright-field images of tissue slices stained using H&E, Jones' stain, and Masson's trichrome stain; adaptations include Fourier ptychographic microscopy and unpaired cycle-GAN datasets.292Source: §5.3 · p. 48
• Virtual staining using phase and phase gradients increased sperm-classification recall and F1 scores twofold to fourfold over other inputs, enabling rapid stain-free analysis.The approach supports real-time applications such as stain-free imaging flow cytometry.Source: §5.3 · p. 48
• Label-free phase-based fluorescence inference avoids fluorescence microscopy disadvantages such as phototoxicity and photobleaching while providing molecular-specific information.Related methods also inferred 3D mitochondrial fluorescence from phase measurements at different depths.Source: §5.3 · p. 49
• Phase-based neural networks inferred fluorescence information for brain-tissue architecture, human-brain-slice myelination, and long-term live-cell growth without loss of viability.Inputs included phase with polarization or retardance and orientation, or a single phase image.Source: §5.3 · p. 49
• Artificial confocal microscopy paired same-field phase and fluorescence data to combine nondestructive phase imaging with confocal depth sectioning and chemical specificity.This addresses the limited high-resolution 3D fluorescence ground truth available from wide-field fluorescence microscopy.Source: §5.3 · pp. 49–50
• Using 3D refractive-index tomograms as input, deep learning bridged optical diffraction tomography to fluorescence imaging for profiling endogenous subcellular structures and dynamics in living cells.The refractive-index input was described as absolute and unbiased, supporting application to new species.Source: §5.3 · p. 50

Datasets

• Dataset diversity governs generalization: fixed settings restrict networks to specific device parameters or sample classes, whereas richer settings and samples improve robustness.Shannon entropy can quantify dataset information richness, while training-sample spatial-frequency content limits fine-feature resolution; spectral-density preprocessing can help.Source: §6 · p. 51

Networks and loss functions

• No neural-network architecture or loss function is yet established as best for phase recovery, because encoder-decoder down-sampling can lose high-frequency information.Dilated or Fourier-domain convolutions, multi-network frequency decomposition, neural architecture search, and alternatives such as NPCC, GAN, and VGG perceptual losses address these limitations.Source: §6 · p. 52

Network-only or physics-connect-network (PcN)

• Physics-connect networks outperform network-only strategies in severely ill-posed cases, including weak-light illumination and dense samples, by numerically enforcing free-space propagation.Using numerical propagation before the network reduces its burden to learning regularization; related hologram-generation and speckle-correlation schemes improve generalization to unknown media and objects.Source: §6 · p. 52

Interpretability

• Deep networks’ limited interpretability makes failures difficult to diagnose or correct, while algorithm unrolling offers physics-based interpretability and efficient training with small datasets.Unrolled layers represent fixed iterations of physics-based algorithms, with physical parameters and regularization coefficients transferred as trainable parameters.Source: §6 · p. 53

Uncertainty

• Because test-sample ground truth is usually unavailable, Bayesian CNNs provide uncertainty maps that describe the confidence of each inferred phase pixel.This directly addresses the difficulty of determining the reliability of inferred results in practical use.Source: §6 · p. 53

Reliability and uncertainty

• Uncertainty maps closely match real error maps, enabling reliability assessment of inferred results without ground truth109,335.The review identifies epistemic and aleatoric uncertainty as sources of inference uncertainty.Source: §6 · p. 53

Optical neural networks

• Optical neural networks offer a potential route to high-speed, parallel, low-power computing, with diffractive deep neural networks particularly suited to image processing339.Initial examples demonstrate their potential for phase processing.Source: §6 · p. 54

Physics-informed deep learning

• Because deep neural networks offer efficiency while physics-based methods are more reliable, the review advocates incorporating physical models instead of assigning all tasks to black-box networks.Training can incorporate datasets, architectures, and loss functions; deployment can use one-time inference followed by physical-model iterative fine-tuning for more accurate results.Source: §6 · p. 54