Source-linked AI summary
PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
Yair Rivenson, Tairan Liu, Zhensong Wei, Yibo Zhang, Aydogan Ozcan
TL;DR
Histochemical staining is standard for revealing tissue morphology, but label-free QPI does not directly produce equivalent brightfield-stained images. PhaseStain addresses this by using a GAN-trained deep neural network, matching virtual stains to H&E-, Jones’-, and Masson’s-trichrome-stained skin, kidney, and liver sections. The framework also offers rapid inference and may reduce staining-related time, labor, and costs while preserving tissue for later analyses.
Problem
Brightfield imaging of histochemically labeled samples remains the clinical and research gold standard, limiting direct use of quantitative contrast maps from label-free QPI.
Method
PhaseStain trains a generative adversarial network on paired QPI and corresponding stained brightfield images to digitally stain label-free tissue sections.
Results
0.617 sec average inference time was achieved for a ~0.45 mm^2 field of view containing ~3.22 Megapixels on a standard dual-GPU desktop computer.
Takeaways & Limitations
PhaseStain may eliminate chemical staining, reduce sample-preparation time, labor, and costs, and preserve unlabeled tissue for subsequent molecular or genetic analysis.
Takeaways & Limitations
The study used separate models for specific tissue–stain combinations, although more general stain-specific models are possible with increased network size and training and inference times.
Abstract
from arXiv · showhide
Using a deep neural network, we demonstrate a digital staining technique, which we term PhaseStain, to transform quantitative phase images (QPI) of labelfree tissue sections into images that are equivalent to brightfield microscopy images of the same samples that are histochemically stained. Through pairs of image data (QPI and the corresponding brightfield images, acquired after staining) we train a generative adversarial network (GAN) and demonstrate the effectiveness of this virtual staining approach using sections of human skin, kidney and liver tissue, matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin, Jones' stain, and Masson's trichrome stain, respectively. This digital staining framework might further strengthen various uses of labelfree QPI techniques in pathology applications and biomedical research in general, by eliminating the need for chemical staining, reducing sample preparation related costs and saving time. Our results provide a powerful example of some of the unique opportunities created by data driven image transformations enabled by deep learning.
INTRODUCTION
PhaseStain uses a GAN-trained deep neural network to transform label-free QPI of tissue sections into digitally stained brightfield-equivalent images. The approach was demonstrated across human skin, kidney, and liver tissues with corresponding histochemical stains.
- Motivation: QPI provides label-free, quantitative optical-path-delay images with low-intensity illumination and rapid acquisition.These properties reduce phototoxicity compared with commonly used fluorescence imaging modalities.
- Motivation: Histochemical brightfield imaging remains the clinical and research gold standard because staining reveals cellular and sub-cellular morphology.
- Approach: PhaseStain uses a GAN-trained deep neural network to transform QPI of label-free tissue sections into digital equivalents of chemically stained brightfield images.
- Demonstration: The method was experimentally demonstrated on human skin, kidney, and liver sections using H&E, Jones’ stain, and Masson’s trichrome stain, respectively.
- Results: Three tissue–stain-specific networks were blindly tested on held-out holographically reconstructed QPI images, producing virtual stains that reproduced reported tissue structures.The examples included tumor morphology in skin and kidney and liver architecture without significant fibrosis or inflammation.
- Results: 0.8113, 0.8141, and 0.8905 SSIM were obtained for virtual staining of skin, kidney, and liver images, respectively.The analysis used approximately 10-megapixel images with a field of view of approximately 1.47 mm^2 per sample.
- Robustness: At L∆~3 µm, SSIM remained approximately 0.8 across a wide range of perturbation coefficients, β, indicating reduced sensitivity to preparation imperfections.The network remained sensitive to local phase variations and improved as filter size increased while SNR stayed fixed.
DISCUSSION
PhaseStain digitally converts label-free QPI into brightfield-equivalent stained images, with fast inference and applicability across imaging configurations and specimen types. The framework could reduce staining burdens while preserving tissue for later analyses, although coherent-imaging artifacts and model specialization remain practical considerations.
- DISCUSSION: 0.617 sec on average produces a digitally stained ~3.22-megapixel field of view, supporting integration with automated QPI whole-slide scanning.The network can output virtually stained image patches while tissue is being scanned.
- DISCUSSION: PhaseStain could save staining time, labor, and costs while preserving unlabeled tissue for molecular, immunofluorescence, and therapeutic analyses.Avoiding chemical staining leaves tissue constituents unaltered for later investigations.
- DISCUSSION: The study trained separate models for three tissue–stain combinations, while more general stain-specific models would increase network size and training and inference times.Because tissue type and required stain are known in clinical diagnostics, selecting the appropriate model is straightforward.
- DISCUSSION: The framework can be applied to QPI images from different imaging configurations, hardware, and phase-recovery methods.The demonstrated lensfree holographic microscope is not the only compatible QPI platform.
- DISCUSSION: Coherence-related artifacts, including speckle and interference fringes from out-of-focus particles, can distort PhaseStain outputs.A separate deep learning hologram-reconstruction method is proposed to digitally remove such artifacts.
- DISCUSSION: Although demonstrated on fixed paraffin-embedded thin tissue sections, PhaseStain should also apply to frozen sections, other fixation methods, cells, and smear samples.These extensions would require a similar training process.
Sample preparation and imaging
The study prepared label-free tissue sections for QPI and paired them with accurately registered brightfield images for GAN-based virtual staining. The PhaseStain model uses generator–discriminator training with pixel, total-variation, and adversarial loss components.
- Sample preparation and imaging: Samples were deidentified existing specimens obtained through UCLA pathology facilities, so the study did not alter routine care or collection.
- Sample preparation and imaging: For training, FFPE tissue was sectioned into approximately 2–4 µm sections, deparaffinized, mounted, and subsequently processed for histochemical staining.
- Sample preparation and imaging: QPI acquisition used an on-chip lens-free holographic setup with pixel super-resolution and multi-height phase recovery.
- Sample preparation and imaging: Cross-modality registration combined coarse field-of-view matching, affine rotation correction, and a learned distortion-correction step.
- GAN architecture and training: The GAN was trained on overlapping 256×256-pixel aligned patches, with a U-net generator producing three-channel YCbCr output and a discriminator classifying stained-image realism.
- GAN architecture and training: Generator loss combined pixelwise L1 error, total-variation regularization, and discriminator-based penalty, with λ=0.02 and α=2000.
Implementation details
After training each tissue-specific network, the authors tested it on four 1792×1792-pixel image patches not used for training.
- Implementation details: Each tissue-specific network was evaluated after training on four 1792×1792-pixel image patches.
5. Greenbaum, A. et al. Imaging without lenses: achievements and remaining challenges of wide-field
This section consists of cited related-work references covering lens-free imaging, Fourier ptychographic microscopy, quantitative phase contrast, tissue refractive index, scattering phase, pathology, and deep learning.
- The cited literature includes achievements and challenges in wide-field lens-free imaging and Fourier ptychographic microscopy.
- Additional references address quantitative differential phase contrast imaging and tissue refractive-index or scattering-phase measurements.
- The references also include deep learning in medical image analysis, pathology-image metastasis detection, and nonlinear microscopy for breast pathology assessment.
13. Giacomelli, M. G. et al. Virtual Hematoxylin and Eosin Transillumination Microscopy Using Epi-
This section lists related work on virtual staining, stimulated Raman scattering microscopy, deep learning microscopy, and deep-learning-enhanced mobile-phone microscopy.
- The cited studies include virtual Hematoxylin and Eosin transillumination microscopy and laser-based stimulated Raman scattering microscopy.
- Other references describe deep learning microscopy and deep learning enhanced mobile-phone microscopy.
17. Wang, H. et al. Deep learning achieves super-resolution in fluorescence microscopy. bioRxiv 309641
The cited work concerns deep learning for lensless computational imaging, phase recovery, and holographic image reconstruction.
- Deep learning has been applied to lensless computational imaging.
- Related work addresses phase recovery and holographic image reconstruction using neural networks.
- The references include optical phase recovery as an application of deep learning.
21. Jo, Y. et al. Quantitative Phase Imaging and Artificial Intelligence: A Review. ArXiv E-Prints 1806,
The cited review-related works cover deep learning applications across optical tomography, ptychography, localization microscopy, super-resolution, and ultrashort-pulse reconstruction.
- Deep learning has been used for optical tomography and Fourier ptychography video reconstruction.
- Neural networks have been applied to three-dimensional and super-resolution localization microscopy.
- The cited studies include deep learning for accelerated super-resolution single-molecule microscopy.
- One cited work concerns resolution localization microscopy.
- Deep learning reconstruction is also reported for ultrashort pulses.
28. Rivenson, Y. et al. Deep learning-based virtual histology staining using auto-fluorescence of label-
The cited work concerns deep learning-based virtual histology staining using tissue autofluorescence.
- Deep learning has been used for virtual histology staining based on tissue autofluorescence.
- The referenced study is presented as an arXiv e-print from 2018.
- The cited work concerns label-free tissue imaging.
29. Goodfellow, I. et al. Generative Adversarial Nets. in Advances in Neural Information Processing
The references span generative adversarial networks, tissue imaging and pathology guidance, quantitative phase microscopy, holography, super-resolution, and biomedical image segmentation.
- Additional references concern tissue polarimetry, molecular pathology guidance, and formalin-fixed paraffin-embedded tissue preparation.
- The references include quantitative phase microscopy for biological sample dynamics and cancer cell identification.
- Several cited studies address lens-free or holographic computational imaging for pathology and wide-field microscopy.
- The bibliography includes multiframe super-resolution, holographic autofocusing, and multi-height lens-free microscopy.
- Biomedical image segmentation is represented by the U-Net reference.
FIGURES AND TABLES
The figures and tables present PhaseStain’s virtual-staining workflow, tissue-specific staining demonstrations, robustness analyses, convergence behavior, and GAN architecture and inference details.
- Workflow: PhaseStain virtually stains quantitative phase images of label-free specimens using a deep neural network, bypassing histochemical staining.
- Tissue demonstrations: Virtual H&E staining is demonstrated for label-free skin tissue using the PhaseStain framework.
- Tissue demonstrations: Kidney tissue is virtually stained with Jones’ stain, while liver tissue is virtually stained with Masson’s Trichrome.
- Robustness: Noise robustness is evaluated using noisy phase inputs, with SSIM-based inference quality analyzed as a function of Gaussian filter length.
- Robustness: Holographic fringes from out-of-focus particles are examined for their impact on digital-staining performance.
- Training behavior: PhaseStain convergence is shown for skin-tissue virtual H&E staining using validation-set L1-loss and generator-loss plots over iterations.
- Architecture and training details: The GAN architecture table identifies the generator and discriminator components and abbreviations including LReLU, Conv, FC, GO, and GT.
- Architecture and training details: Blind inference takes ~0.617 s for a FOV of ~0.45 mm2, corresponding to ~3.22 Megapixels.