Source-linked AI summary
Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
Yichen Wu, Yair Rivenson, Yibo Zhang, Zhensong Wei, Harun Gunaydin, Xing Lin, Aydogan Ozcan
TL;DR
Holographic reconstruction requires phase recovery and autofocusing, which can be cumbersome and time-consuming. The paper introduces HIDEF, a CNN trained on randomly defocused holograms and in-focus phase-recovered targets to perform both tasks from a single hologram. HIDEF extends the usable depth of field and reduces the stated reconstruction complexity from O(nm) to O(1), while its reliable operation is bounded by the defocus range represented during training.
Problem
Holographic reconstruction must recover phase and focus objects at their depths from intensity-only recordings, making digital reconstruction cumbersome and time-consuming.
Method
HIDEF trains a CNN on randomly defocused back-propagated holograms paired with in-focus phase-recovered images, then processes a single hologram for joint reconstruction.
Results
HIDEF extends reconstructed depth of field and changes the stated holographic reconstruction time-complexity from O(nm) to O(1).
Takeaways & Limitations
The approach provides a data-enabled route to computationally extending depth of field in holographic and potentially other imaging modalities.
Takeaways & Limitations
Reliable reconstruction is limited to defocus conditions represented in training; beyond the trained range, HIDEF can hallucinate unrelated features.
Abstract
from arXiv · showhide
Holography encodes the three dimensional (3D) information of a sample in the form of an intensity-only recording. However, to decode the original sample image from its hologram(s), auto-focusing and phase-recovery are needed, which are in general cumbersome and time-consuming to digitally perform. Here we demonstrate a convolutional neural network (CNN) based approach that simultaneously performs auto-focusing and phase-recovery to significantly extend the depth-of-field (DOF) in holographic image reconstruction. For this, a CNN is trained by using pairs of randomly de-focused back-propagated holograms and their corresponding in-focus phase-recovered images. After this training phase, the CNN takes a single back-propagated hologram of a 3D sample as input to rapidly achieve phase-recovery and reconstruct an in focus image of the sample over a significantly extended DOF. This deep learning based DOF extension method is non-iterative, and significantly improves the algorithm time-complexity of holographic image reconstruction from O(nm) to O(1), where n refers to the number of individual object points or particles within the sample volume, and m represents the focusing search space within which each object point or particle needs to be individually focused. These results highlight some of the unique opportunities created by data-enabled statistical image reconstruction methods powered by machine learning, and we believe that the presented approach can be broadly applicable to computationally extend the DOF of other imaging modalities.
1. Introduction
Holographic reconstruction requires both phase recovery and autofocusing, but conventional procedures can be computationally burdensome and inaccurate across 3D samples. HIDEF addresses these constraints by jointly learning phase recovery and autofocusing from a single hologram while extending reconstruction depth of field.
- Reconstruction challenges: Phase recovery is required because digital holograms record intensity information, while autofocusing estimates the sample-to-sensor distances of different 3D object parts.Both steps affect reconstruction quality, and autofocusing is ideally performed after phase recovery.
- Reconstruction challenges: Autofocusing can require back-propagation and criterion evaluation at typically >10-20 axial distances for every object at different depths.This repeated search is time-consuming even for a small field of view.
- Prior approaches: Earlier CNN autofocusing methods provided relatively coarse distance estimates, with classification bins spanning ~3 mm or regression accuracy exceeding 5 mm.These approaches were less suited to precise axial localization of individual objects.
- Prior approaches: Prior CNN phase-recovery methods degraded rapidly outside the system depth of field because they were trained with precisely focused images.A ~4 µm focus deviation in high-resolution pathology imaging caused loss of resolution and distorted sub-cellular details.
- Proposed approach: HIDEF jointly performs autofocusing and phase recovery from a single hologram, extending reconstructed depth of field and changing reconstruction complexity from O(nm) to O(1).The network is trained using both in-focus and randomly defocused image patches.
2. Methods
The method uses a U-Net-inspired CNN with down-sampling, up-sampling, convolutional blocks, ReLU operators, and residual shortcuts to reconstruct holographic images. Training minimizes an l1 loss against reference images using Adam optimization.
- Network architecture: The CNN uses symmetric down-sampling and up-sampling paths with 3×3 convolution blocks and ReLU operators.The down-sampling path captures and separates true-image and twin-image features at multiple scales.
- Network architecture: Residual shortcut paths pass information forward and are used to increase network training speed.The architecture is inspired by U-Net.
- Training: Training minimizes the l1-norm distance between network outputs and target images while updating weights and biases with Adam at a learning rate of 10^-4.The training-to-cross-validation ratio was 14:3.
- Training: Training and blind testing used a six-core 3.60 GHz CPU, 16GB of RAM, and an Nvidia GeForce GTX 1080Ti GPU.A representative training run took ~40 h for 200,000 iterations, or about 100 epochs.
3. Results and Discussion
Experiments show that HIDEF reconstructs holographic images rapidly across an extended defocus range and improves similarity relative to conventional and in-focus-trained approaches. Its reliability remains bounded by the defocus range represented during training.
- Aerosol experiments: HIDEF was evaluated on aerosols distributed across multiple depths in an in-line, lens-free on-chip holographic microscope.The particles’ depth variation arose from differences in particle mass, flow speed, and flow direction during air sampling.
- Aerosol experiments: >20-fold slower was the iterative MH-PR baseline, requiring ~6.4 s for phase recovery and autofocusing compared with HIDEF for the same field of view and object volume.The comparison used eight axial planes for the individual objects.
- Defocus range: HIDEF remains stable when the initial back-propagation distance varies because it was trained with defocus values spanning ±0.1 mm.Beyond this training range, including |dz| > 120 µm in Fig. 2(c), reliable reconstruction is not expected.
- Tissue experiments: Across 180 blindly tested breast-tissue fields of view, HIDEF produced higher SSIM than back-propagated hologram intensities over a defocus range of -100 µm to 100 µm.The SSIM comparison used the auto-focused MH-PR result from eight in-line holograms as reference.
- Tissue experiments: HIDEF had much higher SSIM than a CNN trained only on in-focus holograms across a DOF of ~0.2 mm, while the in-focus-trained network exceeded it only at dz = 0 µm.The result supports improved performance for defocused holograms rather than at the exact focus point.
Figures and Figure Captions
The figures show HIDEF simultaneously performing phase recovery and autofocus, extending holographic reconstruction across defocus ranges in aerosol and breast-tissue samples. Performance remains reliable within the trained range, while conventional MH-PR has a narrower depth of field.
- Fig. 1: HIDEF uses a down-sampling decomposition path, a symmetric up-sampling expansion path, and skip connections forming residual blocks.The channel count varies across network levels, and angular spectrum propagation provides the propagation step.
- Fig. 2: HIDEF focuses aerosol particles at different depths simultaneously, whereas MH-PR requires different depths and can render particles invisible or distorted.HIDEF preserves a particle’s correct image and FWHM diameter across a DOF greater than 0.2 mm after training on ±0.1 mm defocus.
- Fig. 3: HIDEF performs phase recovery and autofocus for breast-tissue images over the trained axial defocus range of |dz| ≤ 100 µm.Outside this range, the network output is not reliable; the comparison includes single-hologram inputs and eight-hologram MH-PR reconstruction.
- Fig. 4: SSIM curves averaged over 180 breast-tissue FOVs confirm that HIDEF maintains extended depth of field up to its trained defocus range.Each FOV contains 512-by-512 pixels.