PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantitative Phase Imaging and Artificial Intelligence: A Review

YoungJu Jo, Hyungjoo Cho, Sang Yun Lee, Gunho Choi, Geon Kim, Hyun-seok Min, YongKeun Park

arXiv:1806.03982v2cs.CVphysics.data-anphysics.optics

TL;DR

The paper addresses how to exploit QPI’s fast, label-free, large-scale data while handling its limited chemical specificity. It reviews machine-learning and deep-learning approaches that analyze QPI data and enhance QPI itself. The review reports applications spanning biomedical classification, segmentation, chemical-information augmentation, and data-driven imaging-system design, while emphasizing that data-driven methods cannot discover patterns absent from the data.

  • Problem

    QPI provides rich quantitative imaging but has limited chemical specificity, motivating data-driven methods for extracting information from refractive-index distributions.

  • Method

    The paper reviews the synergy between QPI and machine learning, with emphasis on deep learning, biomedical applications, QPI enhancement, and practical guidance.

  • Results

    Machine learning is reviewed across QPI applications including classification, segmentation, chemical-information augmentation, and enhancement of QPI measurement and imaging-system design.

  • Takeaways & Limitations

    QPI’s fast, label-free, uniform-quality data provide a basis for systematic machine-learning analysis and for data-driven improvements to QPI.

  • Takeaways & Limitations

    Data-driven methods cannot find patterns that do not exist in the data, so applications require careful assessment of whether machine learning is appropriate.

Abstract

from arXiv · show

Recent advances in quantitative phase imaging (QPI) and artificial intelligence (AI) have opened up the possibility of an exciting frontier. The fast and label-free nature of QPI enables the rapid generation of large-scale and uniform-quality imaging data in two, three, and four dimensions. Subsequently, the AI-assisted interrogation of QPI data using data-driven machine learning techniques results in a variety of biomedical applications. Also, machine learning enhances QPI itself. Herein, we review the synergy between QPI and machine learning with a particular focus on deep learning. Further, we provide practical guidelines and perspectives for further development.

I. INTRODUCTION

QPI and AI have recently converged as practical research fields, creating opportunities for biomedical imaging. QPI supplies fast, label-free, uniform-quality data, while machine learning provides data-driven analysis and modeling approaches.

  • QPI and artificial intelligence became practically realizable after advances in digital image sensors and graphics processing units.
  • The phase problem concerns the loss of phase information in optical measurements, which limits direct observation of microscopic life.
  • Computational phase retrieval enabled two-dimensional quantitative phase images, followed by three-dimensional and four-dimensional refractive-index mapping.
  • Machine learning fits adjustable computational-model parameters from data rather than relying on explicitly programmed rules.
  • QPI generates uniform-quality, large-scale data because it uses endogenous refractive-index contrast and operates rapidly without labeling.
  • The review surveys the QPI–machine-learning interface and offers practical guidelines and perspectives for further development.

II. QUANTITATIVE PHASE IMAGING

QPI retrieves quantitative optical-field information, including amplitude and phase, using specialized measurement and computational methods. Two-dimensional QPI records holograms for field retrieval and is especially useful for label-free imaging of transparent live cells.

  • QPI visualizes quantitative optical-field maps containing both amplitude and phase information.
  • In 2D QPI, interference with a known reference wavefront converts phase information into a detectable holographic fringe pattern.
  • Computational field-retrieval algorithms recover the measured light’s amplitude and phase from the hologram.
  • QPI is advantageous for live-cell imaging because transparent cells provide inadequate contrast in conventional bright-field microscopy.
  • QPI has limited chemical specificity because different molecular species can share similar optical properties.

B. Three-dimensional Quantitative Phase Imaging

Three-dimensional QPI reconstructs refractive-index distributions from optical fields measured under varying illumination angles. Its biomedical value is paired with a chemical-specificity limitation that motivates machine-learning analysis.

  • B. Three-dimensional Quantitative Phase Imaging: ODT reconstructs a sample’s three-dimensional refractive-index distribution from optical fields measured at varying incident angles.
  • B. Three-dimensional Quantitative Phase Imaging: ODT formulates reconstruction as an inverse problem governed by the Helmholtz equation and maps measured fields into three-dimensional Fourier space.
  • B. Three-dimensional Quantitative Phase Imaging: Unlike X-ray CT’s straight-ray assumption, visible-light RI tomography must account for light diffraction.
  • B. Three-dimensional Quantitative Phase Imaging: Coherent ODT varies illumination angles, rotates samples, or scans wavelengths, while partially coherent and incoherent methods reduce coherence or speckle noise at the cost of additional scanning.
  • QPI provides quantitative morphological and dynamical measurements, including volume, surface area, membrane fluctuation, and motility.
  • QPI’s limited chemical specificity reflects the difficulty of mapping refractive-index distributions to chemical composition.

A. Classification: Conventional Approaches

Conventional QPI classification methods train classifiers to map images to biological sample classes and then rapidly identify new images. Studies have varied input dimensionality, output classes, and applications, including three-dimensional cell sorting and time-lapse analysis.

  • A. Classification: Conventional Approaches: QPI classification builds and trains a classifier that maps an input image to a corresponding biological sample class.
  • A. Classification: Conventional Approaches: QPI classification applications include rapid screening and diagnostic purposes, with potential for ultra-high-throughput interrogation.
  • A. Classification: Conventional Approaches: Training integrates characteristics encoded in refractive-index distributions to extract class-dependent fingerprints across biological variation.
  • A. Classification: Conventional Approaches: After training, QPI enables rapid, label-free identification of newly measured test images.
  • A. Classification: Conventional Approaches: Earlier work used numerically reconstructed two-dimensional optical fields at multiple depths with Gabor wavelet features to recognize algae species.
  • A. Classification: Conventional Approaches: Three-dimensional refractive-index tomograms enabled label-free sorting of lymphocytes, while time-lapse QPI supported additional applications.

B. Classification: Deep Learning Approaches

Deep learning addresses QPI classification and segmentation by learning flexible representations from raw data, reducing reliance on hand-designed, problem-specific features. CNN-based methods support image-wise recognition and pixel-wise quantitative analysis, including histopathology and cellular applications.

  • Classification: Conventional QPI classification often requires problem-specific feature extraction and extensive domain knowledge, limiting applicability across complex biological systems.Hand-designed features can become cumbersome when systems lack simple biophysical models.
  • Classification: Deep neural networks can approximate flexible functions and learn directly from raw data without manual feature extraction.CNNs learn hierarchical representations through convolutional filtering.
  • Classification: HoloConvNet was the first CNN-based QPI classification network, designed to distinguish Bacillus anthracis spores from genetically similar Bacillus species.Its training combined batch normalization, dropout, ReLU, momentum, augmentation, backpropagation, and hyperparameter search.
  • Segmentation: Segmentation performs pixel-wise recognition and is required for quantitative QPI analysis, unlike classification, which recognizes an image as a whole.CNN-based segmentation architectures use transpose convolution layers, unlike the classification architecture shown for HoloConvNet.
  • Segmentation: Machine learning enabled QPI-based prostate cancer tissue segmentation and more accurate RBC segmentation using a fully convolutional network.The review also identifies multiclass segmentation as a potential extension for heterogeneous cell populations.
  • Segmentation: Related work demonstrated label-free visualization and segmentation of intracellular organelles by mapping conventional microscopy images to fluorescence images.Fluorescent markers supplied ground-truth images for the trans-modal approach.

D. Unsupervised Learning

Unsupervised learning analyzes QPI data without ground-truth labels, supporting dimensionality reduction, visualization, compression, and clustering. Its use with neural networks remains comparatively unexplored and may expand as the field matures.

  • Unsupervised learning: Unsupervised learning differs from supervised learning by omitting ground-truth data and learning without supplied input-output labels.This changes the learning problem from fitting a labeled correspondence to analyzing structure in the data.
  • Applications: Dimensionality reduction supports QPI data visualization and compression through methods such as PCA, autoencoders, and t-SNE.PCA is equivalent to a linear-decoder autoencoder when optimized with mean squared error.
  • Applications: Unsupervised methods also support exploratory analysis by clustering data points with similar properties.Clustering is presented as a separate application from dimensionality reduction.
  • Outlook: Neural-network-based unsupervised learning remains largely unexplored in QPI and may provide new opportunities upon further development.The review presents this as an open area rather than an established application domain.

E. New Data and Methods

Emerging QPI data and machine-learning methods extend analysis beyond established 2D workflows to 3D, 4D, interpretable, and physics-informed approaches. These methods can improve QPI processing, but inverse-problem ill-posedness creates risks when data distributions change.

  • New data: Three-dimensional QPI classification remains an open application for deep learning, despite prior conventional machine-learning classifiers for label-free lymphocyte sorting.The review identifies 3D CNNs as a straightforward extension that had not yet been applied to 3D QPI.
  • New data: Time-lapse QPI enables recurrent models such as LSTMs and GRUs to model inter-frame relations, while CNN-RNN combinations can address 4D data.QPI supports time scales ranging from seconds to days because it avoids photobleaching and phototoxicity.
  • Interpretability: Visualization and attention techniques can improve interpretation of high-performing deep networks and help identify patterns or testable hypotheses in QPI data.These approaches address the common criticism that deep networks are black boxes.
  • Enhancing QPI: Machine learning can enhance QPI efficiency and performance by learning aspects of underlying physics instead of relying entirely on hand-crafted models and assumptions.The review contrasts data-driven modeling with domain-specific formulations such as ODT based on the Helmholtz equation and weak-scattering assumptions.
  • Enhancing QPI: Image-to-image regression predicts pixel-wise values and is presented as a primary technique for emerging QPI applications.The review links recent deep-learning advances to accelerating this direction.
  • Enhancing QPI: Data-driven phase retrieval and CNN-based tomographic reconstruction offer alternatives to conventional processing pipelines, including faster end-to-end 3D reconstruction.The conventional phase-retrieval, ODT, and iterative-reconstruction workflow is described as time-consuming even with GPU acceleration.
  • Limitations: Phase-retrieval and tomographic models may perform poorly on new sample types because ill-posed inverse problems allow training-data-specific solutions.Examples include single-plane phase recovery and the missing-cone problem in ODT.

B. Image Enhancement

Machine learning is being applied to improve QPI images, optical-system design, and measurement control. The reviewed directions include faster focusing, tracking, denoising, aberration correction, resolution improvement, and closed-loop acquisition.

  • Image enhancement: Machine-learning image enhancement extends QPI beyond postmeasurement analysis and opens a new direction for computational image improvement.The review frames image enhancement as a long-standing goal enabled by recent learning-based methods.
  • Image enhancement: Learning-based methods provide fast autofocusing and support holographic tracking and characterization of particles.These applications address computational refocusing and particle analysis.
  • Image enhancement: Data-driven methods may improve coherent-noise and speckle suppression more efficiently than approaches requiring image diversity or hand-crafted denoising.Conventional approaches may use angular, spectral, or polarization diversity.
  • Image enhancement: Related image-enhancement directions include aberration correction, depth-of-field extension, CNN-based resolution improvement, and GAN-based style transfer.The review cautions that resolution improvement and style transfer should be addressed carefully.
  • System design: Machine learning can jointly optimize a Fourier-ptychography light-source configuration and postprocessing classifier through a learnable image-formation layer.This extends learning from postmeasurement analysis to imaging-system design.
  • Measurement control: Reinforcement learning can control QPI measurements by selecting actions that maximize reward in large measurement-parameter spaces.Deep reinforcement learning is proposed for efficient and ultrafast online control, including illumination-angle selection in ODT.

V. PRACTICAL GUIDELINES FOR DEEP LEARNING IN QPI

Practical deep-learning guidance for QPI begins by defining the biomedical imaging task, with classification and segmentation as common choices, then selecting architectures suited to each task.

  • Problem definition: Classification and segmentation are the most common deep-learning tasks in biomedical image analysis.
  • Classification: ResNet and Inception-v3 are widely used classification models that introduce efficient deep-structure designs.DenseNet and ResNeXt extend these ideas with dense connectivity, pre-activation, group convolution, or related combinations.
  • Segmentation: U-net is a strong baseline model for biomedical image segmentation.Segmentation quality can be improved with deeper shortcut or dense networks, dilated convolutions, or combinations of these approaches.
  • Segmentation: Segmentation models are evaluated with task-appropriate losses and similarity metrics, including cross entropy, Dice similarity coefficient, Jaccard index, and mIOU.Binary segmentation commonly uses Dice or Jaccard measures, whereas multiclass segmentation typically uses mIOU.

B. Data Preparation

Data preparation and model design must account for biomedical-image variability, dimensionality, computational constraints, and task-specific choices of architecture, activation, loss, optimization, and regularization.

  • Data preparation: Registration, normalization, and image enhancement address variation in sample preparation, instruments, and experimenters that can degrade machine-learning performance.Registration aligns coordinates, normalization aligns ranges, and enhancement reduces noise.
  • Data preparation: Patch-based processing reduces memory demands for large image-to-image tasks but may lose global context, whereas whole-image processing preserves local and global features for classification.Patch-size diversification and cascade structures can combine local and global processing strategies.
  • Model design: Network architecture should match data complexity to avoid overfitting or underfitting, while learning rules and hyperparameters require careful design.
  • Model design: Activation functions introduce non-linearity, and their performance depends on the problem setting, so comparative experiments are needed despite ReLU’s generally stable performance.
  • Model design: Kernel size, bottlenecks, shortcut connections, loss functions, regularization, optimizers, and batch normalization encode computational, task-specific, and training choices.Examples include 3-by-3 kernels, parameter-reducing pointwise convolutions, task-dependent losses, Adam or SGD, and normalization that also regularizes.
  • Model design: Automated architecture and hyperparameter search can reduce manual design but may require substantial computation, and universal cross-domain automation remains unrealized.Parameter sharing offers a more efficient automation strategy, while domain insight remains useful.

D. Training and Evaluation

Training and evaluation depend on disjoint dataset splits, sufficient high-quality labeled data, and metrics selected for the task and target application.

  • Dataset splitting: Supervised learning should use disjoint training, validation, and test subsets before model training and evaluation.Validation data supports architecture and hyperparameter optimization, while the test set provides final evaluation.
  • Data availability: Limited labeled biomedical data can be addressed with transfer learning and data augmentation.
  • Evaluation: Evaluation metrics should match the task, including ROC, precision-recall, F1 score, and AUC for binary classification.The supplied passage also describes distinct metric choices for multiclass classification.

VI. OUTLOOK

The review presents QPI–AI as a growing biomedical toolbox whose progress depends on collaboration, standardized datasets, and careful assessment of when data-driven methods are appropriate.

  • Field development: The QPI–AI interface is expected to become an important toolbox, with commercial QPI systems making the technology more accessible to biomedical experts.
  • Field development: Publicly available standard datasets would enable algorithm comparison and help guide new investigations.Current algorithms are mostly based on experimental data produced by individual research groups.
  • Limitations: Machine learning cannot discover patterns absent from the data, so target applications should be assessed carefully for their suitability to a data-driven approach.
  • Extensions: AI-aided QPI can extend to wavefront shaping because both fields involve measuring and modulating optical-field information.Convolutional neural networks have been used for imaging through turbidity, with reported advantages over conventional approaches based on light-transport measurements.
  • Future applications: Future biomedical applications may combine QPI–AI with image-based cell profiling, rapid imaging cytometry, and correlative imaging.
Loading 1806.03982v2…