Source-linked AI summary
Transmission of natural scene images through a multimode fibre
Piergiorgio Caramazza, Oisín Moran, Roderick Murray-Smith, Daniele Faccio
TL;DR
Imaging natural scenes through a multimode fibre remains challenging because existing approaches require full complex-field characterization or may be limited by training-image classes. The paper statistically constructs a complex inverse transmission model from image–speckle data, reconstructing detailed grayscale and full-colour scenes at high frame rates through fibres up to 10 m long. Fibre-geometry changes remain a scope limitation because they alter the speckle pattern and invalidate the learned inversion matrix without configuration knowledge.
Problem
General-purpose transmission of high-resolution, high-frame-rate, full-colour natural scenes through a single multimode fibre remains an open challenge.
Method
The method statistically learns an approximate inverse of a complex-valued intensity transmission matrix from image–speckle pairs using a single fully connected complex matrix.
Results
The learned model reconstructs unseen natural-scene videos, including full-colour Jupiter and Earth sequences, through 1 m fibre and images through 10 m fibre.
Takeaways & Limitations
The approach supports general-purpose multimode-fibre imaging and suggests routes toward efficient video transmission, remote imaging, fibre sensing, and secure data encoding.
Takeaways & Limitations
Without precise knowledge of fibre-configuration changes such as bending, the same inversion matrix cannot reconstruct images from the altered speckle patterns.
Abstract
from arXiv · showhide
The optical transport of images through a multimode fibre remains an outstanding challenge with applications ranging from optical communications to neuro-imaging. State of the art approaches either involve measurement and control of the full complex field transmitted through the fibre or, more recently, training of artificial neural networks that however, are typically limited to image classes belong to the same class as the training data set. Here we implement a method that statistically reconstructs the inverse transformation matrix for the fibre. We demonstrate imaging at high frame rates, high resolutions and in full colour of natural scenes, thus demonstrating general-purpose imaging capability. Real-time imaging over long fibre lengths opens alternative routes to exploitation for example for secure communication systems, novel remote imaging devices, quantum state control processing and endoscopy.
Introduction
Transmitting general-purpose natural-scene images through a single fibre remains difficult because conventional approaches require extensive optical characterization, while neural-network methods may generalize poorly beyond their training classes. The paper addresses this challenge with a statistically constructed inverse model of the fibre.
- Recovering images through a fibre can require reconstructing its transmission matrix, which is difficult in practice.
- Full transmission-matrix methods measure the complex amplitude and phase of many fibre modes and have worked over 0.3-1 m fibre lengths.
- The paper develops an approximate inverse of a complex-valued intensity transmission matrix using statistical image data rather than a deep multilayer ANN.
- The approach reconstructs detailed natural-scene images and supports full-colour video at 20 fps, with potential scaling to kfps.
Results
The method learns a complex inverse transformation from image–speckle pairs and applies it to unseen natural scenes. Experiments demonstrate grayscale and colour video reconstruction through 1 m and 10 m multimode fibres, while fibre configuration remains an important boundary.
- Experimental setup: The experiment couples 92 × 92-pixel intensity images into 1 m or 10 m multimode fibres containing approximately 9000 propagating modes, then records output speckle at 350×350-pixel resolution.
- Grayscale results: The learned matrix reconstructs unseen Muybridge videos, including running horse, jumping cat, flying parrot, and punching boxer sequences transmitted through a 1 m fibre at 4 frames/second.
- Evaluation: The reconstruction quality is quantified with SSIM and PCC, whose maximum value of 1 indicates image identity.
- Colour results: Full-colour videos of rotating Jupiter and Earth are reconstructed by transmitting and recombining the R, G, and B channels independently.
- Image reconstruction: A single fully connected complex-valued matrix statistically approximates the inverse model from many image–speckle pairs.
- Limitation: Changing fibre geometry, such as bending, produces different speckle patterns and prevents use of the same inversion matrix without knowledge of the changed configuration.
Discussion
The paper presents a physically informed approximation to multimode-fibre transmission that enables efficient imaging and data transmission. It identifies communication, sensing, secure encoding, and remote-imaging applications while noting that fibre changes remain an obstacle.
- The approach combines physical modelling with statistical inversion, occupying an intermediate position between full transmission-matrix reconstruction and assumption-free deep learning.
- The method enables efficient video and multimode data transmission through fibres and may support fibre sensing based on image sensitivity to changes along the fibre.
- The multimode fibre’s random coupling and output speckle patterns could be exploited to securely encode and authenticate data.
- Code and example data for 1 m transmission are available through the cited DOI resources.
Material
The supplementary material describes experiments on image focusing, image-quality metrics, and natural-scene validation through multimode fibres. It also provides implementation resources and examples spanning fibre lengths and elapsed times.
- Input focusing: Input-image demagnification produced effective numerical apertures of 0.22, 0.16, and 0.05 using image sides of 10.5, 7.5, and 2.6 mm.The experiments used the same retrieval data and test images while varying image size before fibre coupling.
- Input focusing: Higher effective NA generated smaller output speckles and excited higher spatial-frequency modes, while the smallest NA produced the poorest retrieval.The largest effective NA gave the best image quality, with the intermediate condition slightly worse.
- Image comparison: SSIM and PCC quantify reconstruction quality, with a perfect image match corresponding to the maximum value 1.SSIM uses image means, covariance, variances, stabilizing constants, and pixel dynamic range; PCC uses pixel values relative to their image means.
- Resources: The supplied supplementary package includes code and training and experimental data/images, while the implementation used Python 3.6.5 with Keras and TensorFlow.The code and data are described as downloadable additional material.
A. Model specification
The model specification implements inversion with a custom complex-valued dense layer. Its parameters include complex weights, optional bias handling, random initialization, and L2 regularization.
- Data split: Recorded images are divided into a training set of N = 45,000 and a validation set of 5,000.The model specification describes these as the datasets used for training and validation.
- Architecture: The configured model takes speckle_dim = 120 and produces out_dim = 92, with input shape (speckle_dim^2, 2) and output dimension out_dim^2.The listed architecture omits bias and applies an amplitude operation after the complex dense layer.
- Layer design: The model uses a custom ComplexDense layer whose main task is complex-valued multiplication.The layer represents complex weights with Complex64 64-bit data types.
- Layer design: The layer converts inputs and weights to complex form before multiplying them as true complex quantities.The call implementation combines channels into complex representations and computes the complex product.
B. Parameter optimisation
The model is fitted with stochastic gradient descent using mean squared error, with validation data and callbacks controlling training over 850 epochs.
- Stochastic gradient descent and mean squared error are used for model optimisation.
- Training uses speckle-pattern amplitudes as inputs and original-image amplitudes as targets, with a separate validation set.
- Learning-rate reduction and early stopping monitor the loss, while checkpointing manages model states during optimisation.
- 850 epochs are run with shuffled batches of 32 examples and validation data during fitting.
- After convergence, predictions can be generated; the reported 850-iteration run takes approximately two days on an Nvidia TitanXp GPU.