Source-linked AI summary
Learning to see through multimode fibers
Navid Borhani, Eirini Kakkava, Christophe Moser, Demetri Psaltis
TL;DR
Images transmitted through multimode fibers become speckle patterns, challenging direct recognition from intensity measurements. The paper trains DNNs to reconstruct and classify handwritten digits from distal speckle intensity, recovering inputs through fibers up to 1km. Classification improves when reconstruction precedes recognition, although performance decreases with fiber length.
Problem
Images projected into multimode fibers emerge as speckle patterns, motivating methods that interpret image information from distal intensity measurements.
Method
The authors train DNNs on handwritten-digit speckle data, using a U-net to reconstruct inputs and a VGG network to classify them.
Results
Classification accuracy decreases with fiber length, while reconstruction followed by classification improves performance, including 69.9% classification accuracy for the 1km fiber.
Takeaways & Limitations
Intensity-only DNN processing can reconstruct and recognize MMF inputs, including images transmitted through fibers up to 1km.
Abstract
from arXiv · showhide
We use Deep Neural Networks (DNNs) to classify and reconstruct a large database of handwritten digits from the intensity of the speckle patterns that result after the images propagated through multimode fibers (MMF). Images transmitted through fibers with up to 1km length were recovered. The ability of the network to recognize the input degraded with fiber length but the performance could be enhanced if the neural networks were trained to first reconstruct the image and then classify it rather than classify it directly from the speckle intensity.
1. INTRODUCTION
Multimode fibers can transmit image information, but modal dispersion and mode coupling convert projected images into distal speckle patterns. The paper trains DNNs on intensity-only speckle measurements to reconstruct and classify handwritten digits, with reconstruction followed by classification improving performance.
- Motivation: Modal dispersion and mode coupling cause projected MMF images to become distal speckle patterns.Different propagation constants and local defects decorrelate phase between image features after only a few millimeters.
- Approach: The study trains DNNs on 20,000 handwritten digits to interpret MMF inputs from distal intensity speckle patterns.The networks use modern architectures with up to 14 hidden layers.
- Approach: The DNNs reconstruct or recognize proximal images using only distal light intensity measurements.This avoids requiring coherent field measurements for the demonstrated tasks.
- Main finding: Classification performance improves when one DNN reconstructs the input before a separate DNN classifies the reconstructed image.The same dataset supports training the reconstruction and classification networks, and performance is evaluated for fibers up to 1km.
2. MATERIALS AND METHODS
The experiment launches amplitude- or phase-modulated MNIST digits into a GRIN multimode fiber and records both the modulated input and distal output with CCD cameras. A 4f optical system images the SLM plane onto the fiber input and the distal facet onto the output camera.
- Optical apparatus: A 560nm laser illuminates a 62.5μm-core GRIN multimode fiber with numerical aperture 0.275.The fiber supports approximately 4500 spatial modes at the operating wavelength.
- Input generation: The SLM displays MNIST handwritten digits, whose plane is imaged onto the proximal fiber facet by a 4f system.The setup tests both phase and amplitude patterns as fiber inputs.
- Preprocessing: Recorded CCD images are cropped to 1024x1024 pixels, downsampled to 32x32 pixels, and supplied to the DNNs.The cropping is centered on the digit for CCD1 images and on the speckle for CCD2 images.
- Detection: A second 4f system images the distal fiber facet onto CCD2, while CCD1 records the modulated light after the SLM.These measurements provide the input and output image records used in the experiment.
B. Data processing
The data-processing pipeline uses separate CNNs for direct speckle classification and input-image reconstruction. A VGG-type classifier and a U-net-type reconstructor are trained on downsampled speckle data and reconstructed images.
- Classification: The VGG-type CNN classifies both distal speckle images and reconstructed SLM input images.Its convolutional front end encodes features through downsampling before fully connected classification layers.
- Reconstruction: The U-net-type CNN reconstructs SLM input images from distal speckle intensity patterns.Its encoder captures context, decoder upsampling supports localization, and skip connections transfer same-size feature layers.
- Training: 20k speckle images are split into 16k training, 2k validation, and 2k testing examples.The networks use batch shuffling, Adam optimization with a 1x10^-4 learning rate, and up to 50 epochs.
- Data representation: Figure 2 compares digit inputs, amplitude- and phase-modulated outputs, distal speckles, and their difference image.The difference between speckles for digits 0 and 4 reveals distinctions that are difficult to see in the individual patterns.
- Network architectures: Figure 3 depicts the VGG classifier and U-net reconstruction CNN architectures used in the processing pipeline.The two networks correspond to classification and reconstruction roles, respectively.
A. Input reconstruction
The U-net reconstructs input digits from distal speckle intensity patterns for fibers ranging from 0.1m to 1km. Reconstruction remains possible at 1km, although fidelity decreases with fiber length and temporal speckle drift adds noise.
- Evaluation: The reconstruction experiment evaluates amplitude-modulated inputs transmitted through 0.1m, 10m, and 1km fibers.Examples are shown for reconstructed SLM input digits at each length.
- Reconstruction fidelity: 97.6% fidelity at 0.1m decreases to 90.0% at 1km for reconstructed images.Fidelity is measured as the percent mean square error of the reconstruction compared with the input.
- Reconstruction fidelity: Distal speckle patterns contain sufficient information for U-net recovery despite appearing random.The network uses intensity-only distal speckle images to recover the input pattern.
- Limitation: At 1km, temperature nonuniformities and thermal convection make the distal speckle pattern unstable over time.These effects create additional noise; thermal isolation and an isothermal environment might improve performance.
B. Input classification
Classification accuracy decreases as fiber length increases when VGG-CNNs classify distal speckle intensities, while classifying U-net reconstructions substantially improves performance.
- Input classification: Accuracy falls from 90% for a 2cm fiber to 30% for a 1km fiber when classifying distal speckle intensity patterns.The original SLM input digit images reached 98.4% classification accuracy.
- Input classification: Phase-modulated inputs provide slightly better classification accuracies than amplitude-modulated inputs.The paper attributes this difference probably to more uniform light distribution across the fiber modes.
- Input classification: Classification used four fiber lengths, amplitude or phase input patterns, and either distal speckle intensities or reconstructed SLM inputs.Table 1 organizes accuracy comparisons across these input and classification conditions.
- Input classification: Classifying reconstructed SLM input images significantly increases accuracy compared with classifying distal speckle intensity patterns.The comparison is shown in Table 1 and the normalized confusion matrices in Figure 6.
4. DISCUSSION
The paper demonstrates intensity-only DNN reconstruction and recognition of MMF inputs, with performance declining for longer fibers but improving when reconstruction precedes classification. Compared with coherent alternatives, the approach uses simpler intensity detection and can account for input-side distortions.
- 4. DISCUSSION: DNNs reconstruct and recognize MMF inputs from intensity-only measurements of corresponding output patterns.The reported classification accuracy was 96.8% for a 10m fiber and 69.9% for a 1km fiber.
- 4. DISCUSSION: For longer fibers, the input-to-output mapping becomes random, reducing classifier generalization as similar inputs disperse in distal intensity measurements.The discussion links this behavior to decreasing recognition performance with fiber length.
- 4. DISCUSSION: U-net reconstruction partially inverts the random mapping, allowing a subsequent VGG classifier to recognize unseen objects from the same class.Figure 7 compares training and validation accuracies for raw speckles and reconstructed inputs at 10m and 1km.
- 4. DISCUSSION: Intensity-only detection is simpler in practice than coherent field measurement, while transmission-matrix methods learn the fiber rather than the input classes.Transmission-matrix performance should be independent of fiber length in principle, although accumulated background noise can increase with length.
- 4. DISCUSSION: DNNs can account for nonlinearities, aberrations, speckle, pixelation, phase wrapping, and other distortions introduced before light enters the MMF.The discussion contrasts this with coherent-method reconstructions, which are directly degraded by temperature and mechanical instabilities.
5. CONCLUSION
The paper shows that DNNs can recognize and reconstruct distorted images at the output of a multimode fiber. It also identifies future questions involving nonlinear propagation and neural-network control.
- DNNs recognize and reconstruct distorted images at the output of a multimode fiber.
- The networks can recover the phase of the input image despite a doubly nonlinear mapping.The mapping combines the output square law with the exponential dependence of the field in phase.
- Future work could examine stronger illumination and neural-network control of nonlinear light propagation in multimode fibers.
Supplementary material: Learning to see through multimode fibers
The supplementary material provides additional measurements for the multimode-fiber study, covering four fiber lengths and identifying the contributing laboratories and authors.
- The work was authored by Navid Borhani, Eirini Kakkava, Christophe Moser, and Demetri Psaltis.
- The study involved EPFL’s Optics Laboratory and Laboratory of Applied Photonic Devices.
- Borhani and Kakkava contributed equally to the work.
- Supplementary figures report measurements for fibers of 2cm, 10cm, 10m, and 1km length.
1. SLM INPUT RECONSTRUCTION USING THE U-NET CNN
The supplementary reconstruction results show examples and accuracies for images reconstructed from recorded distal speckle intensity patterns, including phase-modulated inputs.
- The U-net CNN reconstructs SLM input images from recorded distal speckle intensity patterns.
- Reconstruction results include both 2cm-fiber examples and phase-modulated inputs.
- Classification accuracy does not deteriorate for fibers up to 10m in length.
2. DATA CLASSIFICATION USING THE VGG CNN
The supplementary classification figures present normalized confusion matrices for amplitude- and phase-modulated inputs, comparing direct fiber speckle patterns with reconstructed SLM inputs across fiber lengths.
- Normalized confusion matrices compare classification from fiber speckle patterns and reconstructed SLM inputs.
- The comparisons cover fiber lengths of 2cm, 10cm, 10m, and 1km.
- The figures separately report amplitude-modulated and phase-modulated proximal inputs.