Source-linked AI summary
Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
Yunzhe Li, Yujia Xue, Lei Tian
TL;DR
Fixed-medium transmission-matrix imaging is difficult to scale because it is measurement-demanding and vulnerable to speckle decorrelation. The paper trains a statistical one-to-all CNN across same-class diffusers, achieving generalization to unseen media and improved scalability. The demonstrated framework supports shift-variant scattering but leaves volumetric multiple scattering for future work.
Problem
Transmission-matrix imaging relies on fixed-medium one-to-one mappings that are large and highly susceptible to speckle decorrelations.
Method
A CNN learns a one-to-all statistical mapping from speckles collected across multiple diffusers sharing the same macroscopic parameter.
Results
The trained model makes high-quality object predictions through entirely different unseen diffusers and improves robustness to speckle decorrelations and space-bandwidth product.
Takeaways & Limitations
One model can fit scattering media within the same class, enabling highly scalable imaging with larger information throughput.
Takeaways & Limitations
The demonstrated technique uses a thin diffuser and does not yet address volumetric multiple scattering.
Abstract
from arXiv · showhide
Imaging through scattering is an important, yet challenging problem. Tremendous progress has been made by exploiting the deterministic input-output "transmission matrix" for a fixed medium. However, this "one-to-one" mapping is highly susceptible to speckle decorrelations - small perturbations to the scattering medium lead to model errors and severe degradation of the imaging performance. Our goal here is to develop a new framework that is highly scalable to both medium perturbations and measurement requirement. To do so, we propose a statistical "one-to-all" deep learning technique that encapsulates a wide range of statistical variations for the model to be resilient to speckle decorrelations. Specifically, we develop a convolutional neural network (CNN) that is able to learn the statistical information contained in the speckle intensity patterns captured on a set of diffusers having the same macroscopic parameter. We then show for the first time, to the best of our knowledge, that the trained CNN is able to generalize and make high-quality object predictions through an entirely different set of diffusers of the same class. Our work paves the way to a highly scalable deep learning approach for imaging through scattering media.
1 Introduction
Imaging through scattering is difficult because speckle patterns depend on complex microscopic media and deterministic models are large, measurement-demanding, and sensitive to decorrelation. The paper proposes a statistical one-to-all CNN trained across same-class diffusers to generalize across unseen media.
- Motivation: Complex scattering produces seemingly random speckles governed by both scatterer microstructure and incident wavefront, making deterministic inversion difficult.Comprehensive characterization often requires large-scale measurements.
- Limitations of existing methods: Transmission-matrix methods characterize one-to-one mappings for fixed media, but their size grows quadratically with transferred pixel number.This makes high-space-bandwidth-product applications measurement- and data-demanding.
- Limitations of existing methods: Speckle decorrelation caused by slight medium changes breaks the previous input-output relation and rapidly degrades transferred images.A new transmission matrix is needed once the Pearson correlation coefficient falls below 1/e.
- Proposed framework: The proposed one-to-all model encompasses statistical variations across same-class media and distills invariant information from correlated or decorrelated speckles.This is intended to support generalization across objects and media sharing statistical characteristics.
- Proposed framework: A CNN trained on multiple same-parameter diffusers predicts high-quality objects through entirely different unseen diffusers.The model learns a statistical mapping rather than the transmission matrix of one medium.
- Evidence: Experiments achieve ∼256×256-pixel space-bandwidth product using up to 2400 training pairs and demonstrate robustness over speckle decorrelation.Training on 1, 2, or 4 diffusers is evaluated, including generalization to new object types through unseen diffusers.
2 Method
The method uses a shift-variant scattering setup and trains a CNN on speckle measurements from multiple same-class diffusers. The network supports binary detection and grayscale reconstruction through a two-channel encoder-decoder design.
- Experimental setup: A defocused diffuser creates shift-variant scattering with an isoplanatic range of approximately one speckle size.The setup uses an SLM illuminated by a coherent laser; the speckle size is approximately 16 µm.
- Experimental setup: Measurements use several 220-grit diffusers sharing the same macroscopic parameter and similar speckle statistical properties.The diffusers are manufactured by the same process and have an average 63 µm surface feature size.
- Data acquisition: The dataset uses 9 diffusers, with up to 4 for training and 5 held out for testing, while Quickdraw objects are reserved for testing.Training data contain 600 objects and up to 2400 speckle images.
- Data preprocessing: Inputs and outputs are downsampled from 512×512 to 256×256 pixels using 2×2 binning to reduce network parameters and training-data requirements.The downsampling can mix intensities from several speckle grains within one pixel.
- Prediction tasks: Two-channel softmax outputs represent complementary object and background predictions for both binary and grayscale tasks.Binary targets are thresholded objects; grayscale targets are normalized binned SLM images with complementary backgrounds.
- Model objective: The CNN learns a statistical relationship between speckle patterns and unscattered objects, targeting predictions through previously unseen diffusers.Training therefore differs from learning a transmission matrix for a single medium.
- CNN architecture: The network follows a dense-block-modified Unet with encoder downsampling, decoder upsampling, and skip connections preserving high-frequency information.The encoder compresses to 16×16 feature maps with 1088 activation maps before decoding.
3 Results
The experiments evaluate generalization across unseen diffusers, object novelty, training-data scale, and comparison with single-diffuser training. The CNN maintains high-quality predictions across these tests, with performance improving as diffuser and dataset diversity increase.
- Generalization across tasks: The CNN predicts seen objects through previously unseen diffusers while producing consistently high-quality results despite visibly different speckle patterns.The task uses pixel-wise prediction, requiring adaptation to diffuser-specific statistical variations.
- Generalization across tasks: The CNN also predicts unseen objects from the same class through unseen diffusers, demonstrating generalization beyond the training objects.The evaluated classes are handwritten digits and letters.
- Generalization across tasks: The CNN makes high-quality predictions for unseen objects from a different object class through unseen diffusers, although performance is lower and varies across object types.The experiment tests six object types and reports mean and standard deviations of the JI; broader training coverage may improve results.
- Training-data scaling: Performance improves with both more training diffusers and larger training datasets, based on JI evaluations across six CNN configurations.The comparisons use 1, 2, or 4 diffusers and training sets of 800, 1600, or 2400 pairs.
- Training-data scaling: The 4-diffuser, 800-dataset condition slightly outperforms the 2-diffuser, 1600-dataset condition under PCC evaluation, highlighting the value of diffuser diversity.Table 1 reports PCC for the seen-object-through-unseen-diffuser task.
4 analysis
The analysis investigates whether the CNN learns invariant information across diffuser-dependent speckles. Activation-map and correlation analyses indicate that learned features become more similar and remain informative across diffuser changes.
- Network analysis: The analysis seeks invariant features among speckles from different diffusers to assess whether a statistical mapping is plausible.The underlying principle is that deep learning identifies statistical invariance across large datasets.
- Network analysis: Activation maps from visually distinct speckles gradually become similar through the CNN’s encoder-decoder layers.Pairwise PCCs between corresponding activation maps quantify the learned invariance.
- Speckle analysis: Cross-correlation between speckle intensities from the same object through different diffusers resembles the corresponding autocorrelation reference pattern.These patterns do not follow the simple relation used in prior speckle-correlography approaches.
- Speckle analysis: The observed cross-diffuser invariance suggests that physically meaningful, learnable features support generalization to unseen objects and diffusers.Results on unseen objects through unseen diffusers indicate that the learned invariance extends across a broader range of measurements.
5 conclusion and discussion
The paper presents a one-to-all deep learning framework intended to improve scalability in imaging through scattering. It reports resilience to speckle decorrelation and improved space-bandwidth-product, while the demonstrated setting remains thin, shift-variant scattering.
- Conclusion: The one-to-all model fits scattering media within the same class instead of using one model for one fixed medium.The framework is presented as an alternative to the traditional one-to-one transmission-matrix strategy.
- Conclusion: The approach is reported to improve resilience to speckle decorrelations and space-bandwidth-product for imaging through complex scattering media.The authors describe the resulting imaging capability as highly scalable and large-throughput.
- Discussion: The technique is envisioned for biological samples characterized by macroscopic scattering parameters such as absorption and scattering coefficients.The authors propose adapting the approach to train, classify, and image through such samples.
- Discussion: The demonstrated experiments use shift-variant scattering induced by a thin diffuser, while volumetric multiple scattering remains a future challenge.The authors suggest adapting the approach to more challenging volumetric scenarios.
Funding
The work acknowledges support from the National Science Foundation.
- Funding: The research was supported by the National Science Foundation under grant 1711156.
Test on seen objects through unseen diffusers
The CNN is evaluated on objects observed through previously unseen diffusers, including seen objects, unseen objects from familiar classes, and unseen object types. Comparisons also test how training on multiple diffusers affects robustness to diffuser changes.
- Seen objects through unseen diffusers: A CNN trained with four training diffusers makes consistently reliable predictions for seen objects through five previously unseen testing diffusers.The same objects appear during training, while the testing diffusers are entirely new and produce visibly different speckle patterns.
- Unseen objects of the same type through unseen diffusers: The CNN makes high-quality binary predictions for previously unused handwritten digits and letters through unseen diffusers.These objects belong to the same classes as the training data but were not themselves used during training.
- Unseen objects of new types through unseen diffusers: The CNN still makes high-quality predictions for Quickdraw objects from a new class through unseen diffusers.The task evaluates objects and an object class never used during training; prediction quality is quantified across object types using the Jaccard index.
- Single- versus multi-diffuser training: A CNN trained on one diffuser succeeds on unseen objects through that same diffuser but fails when the unseen objects are measured through a different diffuser.This comparison demonstrates the importance of training with multiple diffusers for generalization across diffuser changes.