Source-linked AI summary
High-throughput molecular imaging via deep learning enabled Raman spectroscopy
Conor C. Horgan, Magnus Jensen, Anika Nagelkerke, Jean-Phillipe St-Pierre, Tom Vercauteren, Molly M. Stevens, Mads S. Bergholt
TL;DR
Raman spectroscopy provides label-free molecular imaging, but slow acquisition limits high-throughput imaging. DeepeR uses deep learning for Raman spectral denoising, hyperspectral image super-resolution, and transfer learning. Combining denoising and super-resolution achieved up to 160× speed-ups while preserving spectral and spatial fidelity.
Problem
Weak Raman signals require long acquisition times, limiting high-resolution cell and tissue imaging for high-throughput applications.
Source: Introduction · p. 1Method
DeepeR uses deep learning for Raman spectral denoising and hyperspectral image super-resolution, with transfer learning to extend pretrained models from cells to tissues.
Source: Introduction · p. 2Results
Up to 160× speed-ups reduced Raman cell imaging from over 68 minutes to less than 1 minute while maintaining high spectral and spatial fidelity.
Source: Discussion · p. 12Takeaways & Limitations
DeepeR can be applied to existing Raman systems without hardware modifications, including through transfer learning for other biomedical applications.
Source: Discussion · p. 12Hyperspectral Raman images with substantially different spatial and spectral features still require a sufficiently large dataset for effective performance.
Source: Discussion · p. 12
Abstract
from arXiv · showhide
Raman spectroscopy enables non-destructive, label-free imaging with unprecedented molecular contrast but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep learning enabled Raman spectroscopy, termed DeepeR, trained on a large dataset of hyperspectral Raman images, with over 1.5 million spectra (400 hours of acquisition) in total. We firstly perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 9x improvement in mean squared error over state-of-the-art Raman filtering methods. Next, we develop a neural network for robust 2-4x super-resolution of hyperspectral Raman images that preserves molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 160x, enabling high resolution, high signal-to-noise ratio cellular imaging in under one minute. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.
Introduction
Raman spectroscopy offers label-free molecular imaging but its weak signal makes high-resolution imaging slow. DeepeR applies deep learning to Raman spectra and images to improve acquisition speed and reconstruction quality.
Introduction
- Raman spectroscopy provides non-destructive, label-free molecular imaging with subcellular resolution, but weak signals require roughly one second of acquisition per sampled point.High-resolution cell or tissue imaging can consequently take hours, limiting high-throughput applications.Source: Introduction · p. 1
- Pulsed-laser CARS and SRS enable rapid broadband Raman imaging but are technically demanding and costly..Source: Introduction · p. 1
- Hyperspectral Raman images contain complex spatial and spectral correlations that existing chemometric and multivariate methods do not adequately exploit.DeepeR uses deep learning to reconstruct low-SNR spectra and super-resolve images; its denoising outperformed state-of-the-art spectral smoothing algorithms by 9×.Source: Introduction · p. 2
- 40–160× imaging speed-ups were achieved by combining denoising and super-resolution while preserving molecular cellular information.The framework also achieved robust 2–4× super-resolution and extended pretrained models from cells to tissues using transfer learning.Source: Introduction · p. 2
Results
DeepeR combines spectral denoising and hyperspectral image super-resolution to accelerate Raman imaging while reconstructing molecular information. The approach also transfers a cell-trained model to tissue-engineered cartilage using a small dataset.
Deep Learning Enabled Raman Denoising
- 9× lower spectral MSE than the best-performing Savitzky-Golay filter was achieved by 1D ResUNet denoising.The denoised images also preserved key molecular species better than Savitzky-Golay output.Source: Results · pp. 4, 6, Figure 2 · p. 5
Deep Learning Enabled Hyperspectral Image Super-Resolution
- At 3× and 4×, HyRISR blur increased, but it qualitatively delineated cell boundaries and subcellular features better, with fewer artefacts than bicubic upsampling.The authors associate the increased blur with training the super-resolution network using an L1 loss function.Source: Results · p. 7
- HyRISR significantly reduced imaging time from 68:15 to 17:03 at 2× and 07:20 at 3×, with limited high-frequency-detail loss and biochemical spectral information well maintained.At 4×, imaging time fell further to 04:15, but both bicubic upsampling and HyRISR showed much greater fine-detail loss.Source: Results · p. 8
Hybrid Denoising and Super-Resolution Raman Spectroscopy for High-Throughput Molecular Imaging
- 40×, 90×, and 160× speed-ups produced high-SNR, high-resolution images; quality remained accurate at 40× and 90×, while higher-scale reconstruction could degrade or introduce artefacts.The combined pipeline reduced imaging time from over 68 minutes to under 2 minutes at 40× and under 1 minute at 90×.Source: Results · pp. 8, 10, Figure 4 · p. 9
Generalised Hyperspectral Imaging from Cells to Tissues Using Transfer Learning
- Transfer learning outperformed nearest-neighbour, bicubic, and from-scratch models on PSNR and SSIM for 4× cartilage-image super-resolution using a small training dataset.The fine-tuned cell-trained model reconstructed the unrelated cartilage data with few artefacts, though with some over-smoothing.Source: Results · p. 10, Figure 5 · p. 11
Discussion
DeepeR combines Raman spectral denoising, hyperspectral image super-resolution, and transfer learning to accelerate imaging while preserving molecular information. The discussion highlights its potential across biomedical imaging while noting constraints on fine-detail reconstruction, data needs, and validation.
Discussion
- Up to 10× imaging-time reduction accompanied denoising that outperformed state-of-the-art Raman filtering and preserved biochemical information.The framework also reconstructed high-SNR spectra from low-SNR inputs.Source: Discussion · p. 11
- A neural network for hyperspectral Raman-image super-resolution reconstructed spatial and spectral information, outperformed nearest-neighbour and bicubic upsampling on PSNR, SSIM, and MSE, and achieved 4×, 9×, or 16× speed-ups depending on scale.Source: Discussion · p. 11
- Up to 160× speed-ups produced high-resolution, high-SNR cellular images in under one minute while maintaining high spectral and spatial fidelity to ground truth.VCA endmember regression supported the fidelity assessment.Source: Discussion · p. 12
- Transfer learning extended pretrained DeepeR models to an unrelated biomedical application, and the framework can be applied online or offline without hardware modifications.GPU-equipped systems can perform online inference in seconds.Source: Discussion · p. 12
- DeepeR learns spatial and spectral correlations in large hyperspectral datasets, supporting denoising and super-resolution beyond conventional chemometric approaches.The discussion points to possible relevance for clinical diagnostics, chemometric analysis, and sample identification.Source: Discussion · p. 12
- Very fine details, such as single-pixel features, may not be reconstructed accurately, limiting suitability for high-resolution imaging of small, complex specimens.The authors contrast this with larger specimens, where fine details are sparser and better suited to the framework.Source: Discussion · p. 12
- Transfer to images with substantially different spatial and spectral features still requires sufficiently large datasets, and each application needs large-scale prospective validation.The validation should assess whether diagnostic or scientific decisions match those made from corresponding high-resolution, high-SNR data.Source: Discussion · p. 12
- The authors propose extending equivalent neural-network architectures to FT-IR, hyperspectral, and mass-spectrometry imaging, alongside future label-free analysis applications.The complete dataset, pretrained models, and Python code are open source.Source: Discussion · pp. 12–13
Materials and Methods
Methods describe cell and cartilage sample preparation, Raman acquisition and processing, and the training and evaluation setup for spectral denoising and hyperspectral super-resolution.
Cell Culture; Tissue Culture and Sample Preparation
- MDA-MB-231 breast cancer cells were cultured on MgF2 windows, while cartilage constructs were prepared from cow metacarpophalangeal joints and harvested after 14, 28, or 42 days.The cartilage samples were sectioned at a 45° angle and prepared at thickness greater than 1 mm.Source: Materials and Methods · p. 13
Raman Spectroscopic Imaging
- Raman images were acquired at 0.5 μm spatial resolution for cells and 1 μm for tissue-engineered cartilage, with 1-second integration across 0–3700 cm-1 at 11 cm-1 spectral resolution.Imaging used a 532 nm laser and a confocal Raman microscope.Source: Materials and Methods · p. 13
Hyperspectral Raman Image Processing
- Spectra were cropped to 600–1800 cm-1, background autofluorescence was subtracted, and hyperspectral images were exported to MATLAB.The background filter used α = 500.Source: Materials and Methods · p. 13
Neural Network Architecture and Implementation: Raman Spectral Denoising
- The denoising network was a 1D ResUNet, trained for 500 epochs and evaluated across 11 folds, each holding out one cell image for testing.Training and validation used spectra from the other 10 cell images.Source: Materials and Methods · pp. 13–14
Neural Network Architecture and Implementation: Hyperspectral Super-Resolution
- Hyperspectral super-resolution used a residual channel attention network, trained for 600 epochs and evaluated on the test set using PSNR and SSIM.The stated training used Adam, an L1-norm loss, and a constant learning rate.Source: Materials and Methods · p. 14
Data Augmentation; Implementation
- Image augmentation included subsampling, flipping, rotation, and mixup, with spectral shifting and flipping also applied.Training and implementation used a custom PyTorch DataGenerator; the reported systems included an NVIDIA DGX-1 and a desktop Titan V GPU.Source: Materials and Methods · p. 14
Conflict of Interest
The authors disclose that TV is a founding director and shareholder of Hypervision Surgical Ltd and holds shares from Mauna Kea Technologies.
Conflict of Interest
- TV is a founding director and shareholder of Hypervision Surgical Ltd and holds shares from Mauna Kea Technologies; the authors declare no other competing financial interest.Source: Conflict of Interest · p. 14
Supplementary Information
The supplementary material documents the super-resolution network, data augmentation, and VCA visualizations of Raman images and their molecular components.
Supplementary Figure 1
- The super-resolution architecture uses 18 residual groups and 16 residual channel attention blocks, with spectral downsampling to reduce computational load and upsampling to restore input spectral resolution and length.The architecture also contains channel attention blocks.Source: Figure 1 · p. 17
Supplementary Figure 3
- Training augmentation includes image subsampling, horizontal and vertical flips, rotations, and mixup, plus spectral shifting and flipping.Mixup adds two images following a beta distribution.Source: Figure 3 · p. 19
Supplementary Figures 4 and 5
- Super-resolution examples compare nearest-neighbour, bicubic, and neural-network outputs at 2×, 3×, and 4×, showing reduced imaging times while recovering spatial and spectral information.One example visualization labels four components; another labels five, with colors assigned to the components by VCA on the target high-resolution image.Source: Figure 4 · p. 20, Figure 5 · p. 21
Supplementary Figures 2, 6, and 7
- VCA endmember figures identify cell-image components including background, nucleic acids, proteins, and lipids, with component colors specified separately for each figure.The combined denoising and super-resolution image shows nucleic acids, background, proteins, and lipids; the denoising image also distinguishes proteins/lipids and lipid droplets.Source: Figure 2 · p. 18, Figure 6 · p. 22, Figure 7 · p. 23
Supplementary Figure 8
- The tissue-engineered cartilage VCA figure labels substrate, background, dense and sparse extracellular matrix regions, and cells.Source: Figure 8 · p. 24
Supplementary Tables 1 and 2
- Supplementary Tables 1 and 2 provide hyperparameter details for training the 1D residual UNet denoiser and the residual channel attention super-resolution network.Source: Table 1 · p. 25, Table 2 · p. 25