Source-linked AI summary
Deep learning enhanced mobile-phone microscopy
Yair Rivenson, Hatice Ceylan Koydemir, Hongda Wang, Zhensong Wei, Zhengshuang Ren, Harun Gunaydin, Yibo Zhang, Zoltan Gorocs, Kyle Liang, Derek Tseng, Aydogan Ozcan
TL;DR
Low-cost microscopes can have low numerical apertures, aberrations, and infrastructure requirements that limit accessibility. The paper uses deep learning to restore distorted smartphone-microscope images, enhancing fine features and correcting colour, noise, and warping toward higher-NA benchtop performance.
Problem
Low-cost microscopes often use low numerical apertures and exhibit aberrations, while requiring supporting infrastructure that can limit accessibility.
Method
A deep-learning approach enhances and restores distorted images acquired with a cost-effective smartphone-based microscope.
Results
The method enhances fine spatial features and corrects colour aberrations, noise artefacts, and warping, with results compared against a higher-NA benchtop microscope.
Takeaways & Limitations
These results demonstrate the potential of smartphone-based microscopy as an alternative to more costly and bulky microscope designs.
Takeaways & Limitations
The deep networks were trained with sample-specific datasets in this study.
Abstract
from arXiv · showhide
Mobile-phones have facilitated the creation of field-portable, cost-effective imaging and sensing technologies that approach laboratory-grade instrument performance. However, the optical imaging interfaces of mobile-phones are not designed for microscopy and produce spatial and spectral distortions in imaging microscopic specimens. Here, we report on the use of deep learning to correct such distortions introduced by mobile-phone-based microscopes, facilitating the production of high-resolution, denoised and colour-corrected images, matching the performance of benchtop microscopes with high-end objective lenses, also extending their limited depth-of-field. After training a convolutional neural network, we successfully imaged various samples, including blood smears, histopathology tissue sections, and parasites, where the recorded images were highly compressed to ease storage and transmission for telemedicine applications. This method is applicable to other low-cost, aberrated imaging systems, and could offer alternatives for costly and bulky microscopes, while also providing a framework for standardization of optical images for clinical and biomedical applications.
Introduction
The study uses deep learning to correct spatial, spectral, noise, and warping distortions in low-cost smartphone microscopy. The resulting images recover fine features, improve colour fidelity and depth of field, and support compressed-image workflows for telemedicine and resource-limited settings.
- Motivation: Mobile microscopes introduce colour distortions and micro-scale image errors through inexpensive illumination, low numerical apertures, and aberrated or misaligned optics.These limitations can reduce accessibility to advanced imaging technologies, especially in resource-limited settings.
- Approach: A deep convolutional network learns mappings from smartphone microscope images to matched gold-standard benchtop images, without relying on a physical degradation model.The learned transformation addresses spatial and colour aberrations introduced by the mobile microscope.
- Broader scope: The framework is proposed for other low-cost aberrated microscopes, optical-image standardization, and biomedical or clinical applications.The authors also report generalized colour-corrected responses across smartphone colour settings and battery-powered illumination conditions.
- Image enhancement: The network restores spatial details, corrects severe dye-colour distortions, denoises images while retaining fine features, and improves structural similarity against gold-standard images.Colour correction is particularly relevant to stained tissue imaging and telepathology.
- Quantitative evaluation: 4–11-fold improvement in average CIE-94 colour distance was achieved across samples, although the improvement was sample dependent.The colour improvement is reported as especially significant for pathology, where dyes distinguish tissue structures.
- Limitations: The study used sample-specific training datasets, and image quality can be affected by non-uniform sensor colour response and illumination instabilities.These conditions define important scope boundaries for generalization and deployment.
Figures
The figures illustrate a deep network transforming smartphone microscope images into outputs with improved spatial, spectral, and noise characteristics. Comparisons with benchtop images and compressed inputs show the method across tissue, smear, and parasite samples.
- Lung tissue: The lung-tissue comparison shows deep-network output resembling high-end benchtop microscopy while extending depth of field.The smartphone input, network output, and 20×/0.75NA benchtop image are compared, with arrows indicating extended depth of field.
- Pathology comparisons: Deep-network processing recovers fine structural details and reveals spatial and spectral information in pathology images.The comparisons identify recovered fine details and significant improvements in spatial and spectral detail.
- Image quality: Cross-section profiles demonstrate noise removal while retaining high-resolution spatial features.The profiles correspond to displayed sample regions and are used to show the denoising effect.
- Compressed inputs: JPEG-compressed smartphone images are compared with their deep-network outputs and zoomed regions.The figure contrasts JPEG-compressed input with the corresponding network output and localized regions of interest.
- Blood smear: For trypanosome-infected blood, the network accurately colours and resolves parasite nuclei that have very low visibility in the raw smartphone image.The comparison includes smartphone input, network output, and a 20×/0.75NA benchtop image.
Tables
The tables organize training details and image-quality comparisons for pathology samples. They include SSIM comparisons, colour-distance evaluation, and reference benchtop images.
- Training details: Table 1 lists deep neural network training details for different samples.The accompanying comparisons involve smartphone microscope images and corresponding deep-network outputs.
- Structural similarity: Table 2 reports average SSIM for pathology samples, comparing bicubic ×2.5 upsampling with the deep-network output.The table caption identifies SSIM as the comparison measure and bicubic ×2.5 upsampling as the baseline.
- Colour accuracy: Table 3 reports average and standard-deviation CIE-94 colour distances relative to gold-standard images.The evaluation is part of the colour comparison against standard pathology images.