Source-linked AI summary
Deep learning-based super-resolution in coherent imaging systems
Tairan Liu, Kevin de Haan, Yair Rivenson, Zhensong Wei, Xin Zeng, Yibo Zhang, Aydogan Ozcan
TL;DR
Coherent imaging resolution can be limited by sensor pixel size or optical diffraction, while existing approaches may require additional measurements or iterative reconstruction. This paper uses a GAN to super-resolve coherent images and demonstrates improved resolution in both pixel-limited and diffraction-limited systems.
Problem
Coherent imaging resolution can be limited by pixel size or diffraction, and many existing enhancement methods require sequential measurements, additional hardware, reduced field-of-view, or iterative optimization.
Source: §1 · p. 2Method
A GAN is trained to transform low-resolution coherent images into super-resolved images in a feed-forward, non-iterative step.
Source: §1 · p. 2, §2.5 · p. 6Results
The framework demonstrated resolution enhancement in both pixel size-limited and diffraction-limited coherent systems, with evaluation using SSIM and spatial frequency analysis.
Source: §1 · p. 2, §4 · p. 15Takeaways & Limitations
The framework is broadly applicable to coherent imaging and provides rapid, non-iterative resolution enhancement without additional parameter optimization.
Source: §4 · p. 15In the lens-based system, coherence-related background noise and image artifacts partly dominate SSIM, which increased only from 0.876 for the input to 0.879 for the network output.
Source: §3.2 · p. 14
Abstract
from arXiv · showhide
We present a deep learning framework based on a generative adversarial network (GAN) to perform super-resolution in coherent imaging systems. We demonstrate that this framework can enhance the resolution of both pixel size-limited and diffraction-limited coherent imaging systems. We experimentally validated the capabilities of this deep learning-based coherent imaging approach by super-resolving complex images acquired using a lensfree on-chip holographic microscope, the resolution of which was pixel size-limited. Using the same GAN-based approach, we also improved the resolution of a lens-based holographic imaging system that was limited in resolution by the numerical aperture of its objective lens. This deep learning-based super-resolution framework can be broadly applied to enhance the space-bandwidth product of coherent imaging systems using image data and convolutional neural networks, and provides a rapid, non-iterative method for solving inverse image reconstruction or enhancement problems in optics.
1. Introduction
Coherent imaging captures complex-field information, but recovering phase and increasing resolution remain central challenges. The paper applies a GAN to super-resolve coherent images limited by pixel size or diffraction.
1. Introduction
- Coherent imaging characterizes absorption and scattering and supports numerical refocusing, but recovering the missing phase is necessary to infer an object's complex field.Off-axis holography addresses phase retrieval but generally reduces the imaging system's space-bandwidth product.Source: §1 · p. 2
- Many existing resolution-enhancement methods require sequential measurements, additional hardware, or reduced field-of-view, while sparsity-based approaches require iterative optimization.Sparse recovery generally searches a parameter space and takes longer to reconstruct images.Source: §1 · p. 2
- The paper demonstrates a GAN that super-resolves pixel-limited and diffraction-limited coherent images, including lung tissue and Pap smear samples.Performance is quantified using SSIM and spatial frequency content against higher-resolution ground truth images.Source: §1 · p. 2
2. Methods
The methods pair pixel size-limited lensfree imaging with NA-limited lens-based holography, generating registered high- and low-resolution image pairs for separate GAN super-resolution networks.
2.1 Generation of ground truth super-resolved image labels
- Pixel-super-resolution labels came from laterally shifted holograms, while lens-based low- and high-resolution pairs used 4×/0.13NA and 10×/0.30NA objectives.The lensfree system used 2×2, 3×3, or 6×6 shifted holograms to produce effective pixel sizes of 1.12, 0.7467, or 0.3733 μm.Source: §2.1 · pp. 4–5
2.2 Autofocusing and singular value decomposition-based background subtraction
- Eight hologram heights were used for phase recovery; lens-based imaging additionally required SVD-based background subtraction before ToG autofocusing.The lensfree setup used the ToG criterion directly to estimate heights; the lens-based setup used it after subtracting particle- and dust-related background patterns.Source: §2.2 · p. 5
2.3 Multi-height phase recovery
- Multi-height phase recovery initialized the first measurement with zero phase, propagated fields through all eight heights, then backpropagated while updating amplitudes.At each height, phase was retained and amplitude was updated by averaging the current amplitude with the square root of measured intensity.Source: §2.3 · p. 5
2.4 Registration between lower resolution and higher resolution (ground truth) images
- Pixel-limited image pairs were registered by phase-pattern correlation and an affine transform; diffraction-limited pairs first required rough field-of-view matching.For lens-based images, higher-resolution phase images were stitched, then upsampled lower-resolution edges were correlated against patches to select a matching crop.Source: §2.4 · p. 6
2.5 GAN architecture and training process
- The GAN generator maps lower-resolution inputs toward ground-truth features, while a discriminator distinguishes generated images from labels.The lensfree generator loss combines L1 and total-variation terms with an adversarial term; the lens-based loss also includes SSIM.Source: §2.5 · pp. 6–8
- Registered image pairs were cropped into 128×128 patches for network training, and both systems used generator and discriminator networks.The generator used an adapted U-net with down- and up-sampling paths and linked features; the discriminator classified generated versus label images.Source: §2.5 · pp. 6, 8
2.6 Software implementation details
- Training used Adam, image flips and rotations, and four generator updates per discriminator update; software and training and inference timings were documented.The networks were implemented in Python with TensorFlow; testing times were reported for producing a 1940×1940-pixel image.Source: §2.5 · p. 9, §2.6 · p. 9, Table 2 · p. 9
3. Results and Discussion
The GAN super-resolved pixel size-limited and diffraction-limited coherent images, with finer input pixels recovering lung-tissue details missed at the coarsest setting. Structural-similarity and spatial-frequency analyses supported pixel-system improvement, while coherence-related noise and artifacts constrained SSIM’s increase in the lens-based system.
3.1 Super-resolution of a pixel size-limited coherent imaging system
- Pixel-system tests used Pap-smear and lung-tissue samples, with networks blindly tested on tissue sections from patients not used for training.Two patients supplied Pap-smear training samples and three supplied lung-tissue sections.Source: §3.1 · p. 10
- For pixel size-limited imaging, finer input pixels recovered lung-tissue details missed at 2.24 µm, and Pap-smear outputs included amplitude and phase channels.Fig. 5 also marks the network’s processing of artifacts from out-of-focus particles.Source: §3.1 · p. 10, Figure 5 · p. 11, Figure 6 · p. 12
- Network outputs improved structural similarity and added higher spatial-frequency components approaching the ground truth in the lensfree on-chip system.The section reports SSIM comparisons and frequency spectra for the network input, output, and target images.Source: §3.1 · pp. 10, 13, Figure 7 · p. 13
3.2 Super-resolution of a diffraction-limited coherent imaging system
- For diffraction-limited imaging, outputs from a network trained on two patients were tested blindly on a third and showed higher spatial frequencies approaching the ground truth.Inputs used a 4×/0.13 NA objective; reference images used a 10×/0.30 NA objective.Source: §3.2 · p. 14, Figure 8 · p. 14, Figure 9 · p. 15
- In the lens-based system, SSIM rose only from 0.876 for the input image to 0.879 for the network output, with coherence-related artifacts and noise partly dominating its calculation.The setup’s optical components and surfaces make it susceptible to coherence-induced background noise and related image artifacts.Source: §3.2 · p. 14
4. Conclusion
The conclusion presents a GAN framework for super-resolution in pixel size-limited and diffraction-limited coherent imaging. It describes the approach as broadly applicable across coherent imaging techniques and identifies off-axis holography as a possible application.
4. Conclusion
- The framework addresses both pixel size-limited and diffraction-limited coherent imaging, demonstrated on lung and Pap-smear tissue sections.The reported assessments used structural similarity and spatial-frequency spectra.Source: §4 · p. 15
- The authors characterize the reconstruction engine as rapid and non-iterative, requiring no additional parameter optimization.Source: §4 · p. 15
- The approach is described as broadly applicable across coherent imaging techniques; off-axis holography is proposed as a possible use for bridging the space-bandwidth-product gap while retaining single-shot, high-sensitivity acquisition.Source: §4 · p. 15