PaperlayerView exact revision on arXiv

Source-linked AI summary

Phase recovery and holographic image reconstruction using deep learning in neural networks

Yair Rivenson, Yibo Zhang, Harun Gunaydin, Da Teng, Aydogan Ozcan

arXiv:1705.04286v1cs.CVcs.IRcs.LGphysics.app-phphysics.optics

TL;DR

Intensity-only imaging cannot directly record phase, making phase recovery important for reconstructing specimen images. This paper trains a neural network for phase recovery and holographic reconstruction, reporting fewer holograms and faster computation than multi-height phase retrieval.

  • Problem

    Intensity-only sensors cannot directly record phase, creating a need for methods to recover phase and reconstruct specimen phase and amplitude images.

    Source: Introduction · p. 3

  • Method

    The authors train a deep neural network to perform phase retrieval and holographic image reconstruction for lensfree in-line holograms of transmissive samples.

    Source: Introduction · p. 3, Results · p. 4

  • Results

    The neural network reduced hologram acquisition by 2–3 times and computation time by more than 3- and 4-fold versus multi-height phase retrieval with Nholo = 2 and Nholo = 3, respectively.

    Source: Results · p. 5

  • Takeaways & Limitations

    The framework was demonstrated for lensfree digital in-line holography of transmissive human tissue, blood, and Pap-smear samples.

    Source: Results · p. 4

    The average refractive index of fixed and stained red blood cells is difficult to determine or estimate, so the authors define an effective refractive volume.

    Source: Calculation of red blood cell (RBC)… · p. 24

Abstract

from arXiv · show

Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography. Here we demonstrate that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training. This deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference related spatial artifacts. Compared to existing approaches, this neural network based method is significantly faster to compute, and reconstructs improved phase and amplitude images of the objects using only one hologram, i.e., requires less number of measurements in addition to being computationally faster. We validated this method by reconstructing phase and amplitude images of various samples, including blood and Pap smears, and tissue sections. These results are broadly applicable to any phase recovery problem, and highlight that through machine learning challenging problems in imaging science can be overcome, providing new avenues to design powerful computational imaging systems.

Introduction

The paper presents a convolutional neural network that recovers phase and reconstructs holographic images from a single intensity-only hologram. It targets phase loss and twin-image and self-interference artifacts, which otherwise motivate acquiring additional measurements.

Introduction

  • Because sensor arrays record intensity rather than phase, phase recovery is needed to reconstruct specimen amplitude and phase and separate real images from holographic artifacts.The proposed convolutional neural network performs this recovery from a single intensity-only hologram.Source: Introduction · p. 3

Results

  • Single-hologram neural-network reconstruction achieved SSIM comparable to multi-height phase retrieval using 2–3 times fewer holograms, with computation more than 3-fold and 4-fold faster than the Nholo = 2 and Nholo = 3 comparisons, respectively.The reported comparisons include Pap smear and breast tissue at Nholo = 2, and blood smear at Nholo = 3.Source: Results · pp. 4–5
  • For 127 red blood cells, phase-integral values from the network output were substantially improved over the artifact-corrupted input image, supporting cell-volume estimation.The phase integral is proportional to cell volume; the comparison also included multi-height recovery with Nholo = 8.Source: Results · p. 5
  • A network trained on in-focus images reconstructed defocused breast-tissue holograms across z = [-20 µm, +20 µm], with negligible image-quality drop across the approximately 4 µm depth of field.Output amplitude SSIM was evaluated against multi-height recovery with Nholo = 8.Source: Results · p. 5

Discussion

  • The trained network suppresses out-of-focus interference artifacts as well as twin-image artifacts, treating spatially defocused interference as a similar artifact class to reject.The paper notes that unwanted particles or optical-surface imperfections can superimpose holographic fringes on the object hologram.Source: Discussion · p. 6
  • The same architecture can be trained separately for known sample types or expanded into a universal network that performs similarly, with runtime increasing from 6.45 sec to 7.85 sec per 1 mm2 field of view.The universal network uses 32 rather than 16 feature maps per convolutional layer and takes longer to train.Source: Discussion · p. 6

Multi-height phase recovery

  • The multi-height phase-recovery procedure formed training ground truth from holograms at eight sensor heights, iterating 50 times before back-propagating the reconstructed field to the sample plane.The eight positions ranged from approximately 0 µm to 180 µm; the procedure used a TIE initial guess and iterative updates across heights.Source: Multi-height phase recovery · p. 7

Generation of training data

  • Training pairs were created by dividing reconstructed and single-hologram images into overlapping 5×5 sub-tiles, yielding 100 training, 25 validation, and 25 blind-test pairs per sample type.Training took approximately 14.5 hours per sample-specific network and 22 hours for the universal network.Source: Generation of training data · p. 7

Network architecture

The network maps the real and imaginary channels of a back-propagated hologram field to real and imaginary object-image channels, using multi-scale paths for object-type-specific processing and a widened universal variant.

Network architecture

  • The object-type-specific network processes complex-field inputs through four convolutional layers and four scale-specific paths to produce the real and imaginary object-image channels.The input channels are the real and imaginary parts of the back-propagated hologram intensity; each initial convolutional layer outputs 16 feature maps. The paths use ×1, ×2, ×4, and ×8 scales, residual blocks, and upsampling before concatenation.Source: Network architecture · p. 21
  • The universal network retains the architecture but doubles convolutional-layer output channels from 16 to 32 in the residual blocks.Source: Network architecture · p. 21
  • Training minimizes the average mean-squared error between predicted and ground-truth real and imaginary object images, using eight-hologram multi-height phase retrieval for the ground truth.The network parameters are optimized with ADAM at a learning rate of 10^-4; sample-specific training uses batch size 2, while universal training uses 256×256-pixel patches and batch size 30.Source: Network architecture · pp. 21–22

Pixel super resolution (PSR)

Pixel super resolution (PSR) combines multiple subpixel-shifted holograms into a higher-resolution hologram to mitigate spatial undersampling from the sensor’s relatively large pixel pitch.

Pixel super resolution (PSR)

  • A 6-by-6 grid shifted the sensor in 0.37 μm increments, approximately one-third of a pixel, so a Bayer color channel could cover its full period.With a monochrome sensor, a 3-by-3 grid would achieve the same PSR factor.Source: Pixel super resolution (PSR) · p. 23
  • An efficient non-iterative fusion algorithm combined the shifted images into one higher-resolution hologram while preserving maximum-likelihood optimality [5].Source: Pixel super resolution (PSR) · p. 23
  • The Bayer channel was selected according to illumination wavelength: green channels were used at ~530 nm and red at ~630 nm.Source: Pixel super resolution (PSR) · p. 23

Calculation of red blood cell (RBC) phase integral and effective refractive volume

The section defines an RBC phase integral by integrating relative optical phase over the cell area, then relates it to cell volume and defines effective refractive volume for fixed, stained RBCs whose average refractive-index difference is difficult to estimate.

Calculation of red blood cell (RBC) phase integral and effective refractive volume

  • Integrating the RBC’s relative phase over its area yields a phase integral in rad·µm2, related to cell volume through the average refractive-index difference.The relative phase depends on sample thickness and refractive-index difference from the background; the phase integral is evaluated over the cell area Si.Source: Calculation of red blood cell (RBC)… · p. 23
  • For fixed, stained RBCs, the average refractive-index difference is difficult to determine, so the section defines an effective refractive volume with volume units.The effective refractive volume is based on the phase integral and is expressed in units such as femtoliters.Source: Calculation of red blood cell (RBC)… · pp. 23–24

Structural similarity (SSIM) index calculation

The section defines the structural similarity index (SSIM) for comparing two images using their image statistics, with stabilization constants to avoid division by a small denominator.

Structural similarity (SSIM) index calculation

Evaluation of scattering strength of the samples

The section evaluates the weak-scattering condition for breast tissue, Pap smear, and blood smear samples using complex images reconstructed from multiple hologram heights.

Evaluation of scattering strength of the samples

Calculation of the sample-to-sensor distance

The method estimates relative hologram-plane spacing for TIE and multi-height phase recovery, then refines the absolute sample-to-sensor distance for final back-propagation.

Calculation of the sample-to-sensor distance

  • Relative hologram-plane spacing is estimated with axial-magnitude-differential autofocusing to support TIE and multi-height phase recovery.The absolute z2 distance is needed to back-propagate the recovered complex wave onto the sample plane.Source: Calculation of the sample-to-sensor… · p. 25
  • A coarse 100–800 μm scan in 10 μm steps is followed by golden-section search around the minimum, reaching 0.01 μm precision.The scan minimum is located using the axial magnitude differential7; the refinement uses a golden section search algorithm8.Source: Calculation of the sample-to-sensor… · p. 25
  • After multi-height phase recovery converges, the absolute z2 distance is refined by refocusing the phase-recovered hologram near the previously found focus.Source: Calculation of the sample-to-sensor… · p. 25

Related papers

Loading 1705.04286v1…