Source-linked AI summary
Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light
Yi Luo, Yifan Zhao, Jingxi Li, Ege Cetintas, Yair Rivenson, Mona Jarrahi, Aydogan Ozcan
TL;DR
Accurately reconstructing images through random diffusers remains difficult without computer-based recovery. This paper designs deep-learning-trained diffractive layers that reconstruct objects through unknown diffusers, including unseen resolutions and diffusers, while retaining faithful images despite reduced contrast.
Problem
Accurately recovering images through diffusers lacks a simple solution without computer-based reconstruction.
Method
Deep learning designs diffractive layers within an optical network for all-optical image reconstruction through diffusers.
Results
Test targets with unknown resolutions and new random diffusers were reconstructed, including diffusers excluded from training, with faithful images despite reduced contrast.
Takeaways & Limitations
The trained diffractive networks serve as general-purpose imagers for reconstructing objects through unknown diffusers.
Takeaways & Limitations
The results ignore multiple scattering within a volumetric diffuser.
Abstract
from arXiv · showhide
Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers. We present a computer-free, all-optical image reconstruction method to see through random diffusers at the speed of light. Using deep learning, a set of diffractive surfaces are designed/trained to all-optically reconstruct images of objects that are covered by random phase diffusers. We experimentally demonstrated this concept using coherent THz illumination and all-optically reconstructed objects distorted by unknown, random diffusers, never used during training. Unlike digital methods, all-optical diffractive reconstructions do not require power except for the illumination light. This diffractive solution to see through diffusers can be extended to other wavelengths, and might fuel various applications in biomedical imaging, astronomy, atmospheric sciences, oceanography, security, robotics, among others.
4 Department of Surgery, David Geffen School of Medicine, University of California, Los
The paper introduces a computer-free optical approach for reconstructing images distorted by unknown random diffusers. Deep-learning-designed diffractive layers perform reconstruction during light propagation, requiring power only for coherent illumination.
- Main Text: Diffractive networks reconstruct object images through unknown random phase diffusers without computers or digital computation.The physical network is placed between the diffuser and output plane, where successive trained layers reconstruct the image optically.
- Main Text: The networks are trained with image pairs and many randomly selected diffusers so they generalize beyond diffusers seen during training.Training adjusts diffractive-feature phase values by back-propagation using a loss comparing the output pattern with the ground-truth image.
- Main Text: The passive reconstruction operates at light-propagation speed and requires no external power source beyond coherent illumination.This distinguishes the optical implementation from computer-based reconstruction methods described in the paper.
- Main Text: Experimentally, coherent THz illumination and fabricated diffractive networks reconstructed objects distorted by randomly generated diffusers excluded from training.The validation used fabricated networks and unknown phase diffusers under coherent THz illumination.
- Main Text: The framework is presented as applicable across electromagnetic wavelengths and potential imaging settings including biomedical imaging, astronomy, robotics, and security.The paper specifically mentions extension to visible and far/mid-infrared wavelengths.
Results
Experiments show that fabricated diffractive networks reconstruct unknown objects through new random diffusers, including objects and resolutions not represented during training. Reconstruction generalizes across novel diffusers, improves with network depth, and benefits from training with multiple diffusers per epoch.
- Results: Unknown objects were reconstructed through new diffusers never seen during training, demonstrating all-optical generalization.Figures 2–4 compare distorted inputs and optical outputs for new diffusers and unseen test objects.
- Results: 10.851±0.121 mm and 11.233±0.531 mm were measured for the 10.8 mm target through known and new diffusers, respectively.For the 12 mm target, the corresponding measurements were 12.269±0.431 mm and 12.225±0.245 mm.
- Results: Training with n=10, n=15, or n=20 diffusers per epoch performed similarly and significantly better than n=1 for new-diffuser generalization.Using only one diffuser produced relatively inferior generalization and poorer all-optical reconstruction.
- Results: Performance for the last known diffuser was higher because of overfitting, which reduced generalization to new diffusers.Training strategies using n=10, n=15, and n=20 diffusers per epoch generalized better than the n=1 strategy.
- Results: Additional trainable diffractive layers increased average PCC for objects distorted by both known and new random diffusers.The result demonstrates a depth advantage for all-optical image reconstruction.
Discussion
The trained diffractive networks reconstruct objects through unknown diffusers all-optically, including test diffusers with finer phase distortions than those used in training. Their results support generalization beyond random phase diffusers while remaining bounded by the optically thin-diffuser assumption.
- Generalization: Trained diffractive networks reconstruct unknown objects through new diffusers, supporting their use as general-purpose imagers rather than dataset-specific reconstructors.The networks still produced correct images, with improved fidelity when the diffuser was removed.
- Generalization: Unknown test objects were faithfully reconstructed through an unseen diffuser with a smaller correlation length, despite reduced image contrast.The test diffuser had an approximately 5λ correlation length versus approximately 10λ for the training diffusers.
- Limitations: All reported results assume optically thin phase diffusers and therefore ignore multiple scattering within volumetric diffusers.Future work is proposed to train networks that generalize over volumetric diffusers affecting both phase and amplitude.
- Generalization: Experiments with 3D-printed diffusers indicate robustness to phase-and-amplitude distortions absent from training.Training used random phase diffusers, whereas fabricated diffusers introduced amplitude distortions through THz absorption across feature thicknesses.
- Implications: The approach performs reconstruction at the speed of light without digital computation and requires power only for illumination.The diffractive layers are trained once, fabricated, and positioned between the unknown diffuser and output plane.
- Implications: The framework may extend to visible and infrared wavelengths and applications including biomedical imaging, astronomy, atmospheric sciences, security, and robotics.Future directions include handling volumetric diffusers and more complicated dynamic scatters.
Methods
The method models random phase diffusers and trains multilayer diffractive networks to reconstruct distorted images optically, with THz experiments measuring the resulting output fields. Training uses randomized diffuser perturbations, a customized loss, and raw-data evaluation without contrast enhancement.
- Experimental measurement: The experiment generates coherent 0.4 THz radiation, scans the output diffraction pattern with a single-pixel detector, and reads the signal through amplification, filtering, and lock-in detection.Quantitative PCC and resolution measurements use raw data, although displayed experimental images receive digital contrast enhancement.
- Network training: The customized loss combines output-to-ground-truth PCC with an object-specific energy-efficiency penalty based on the input object's transmittance mask.The mask identifies transmitting pixels, and the optimized hyperparameters are α=1 and β=0.5.
- Network training: The networks are optimized by back-propagating the loss and updating pixel phase values with Adam using a decaying learning rate.A typical model is trained for 100 epochs with n=20 diffusers per epoch, requiring approximately 24 hours on the reported GPU system.
Figures
The figures present a four-layer diffractive system for all-optical imaging through unknown random diffusers, with simulated, experimental, generalization, memory, and depth-comparison results.
- System design: A four-layer diffractive network is trained to reconstruct images distorted by unknown or new randomly generated phase diffusers.The system is designed for all-optical reconstruction through random diffusers.
- Simulation: Simulation results evaluate reconstruction for test objects seen through both known and new diffusers.The comparison uses the trained diffractive network introduced in Figure 1.
- Generalization: Networks trained on MNIST generalize to unseen resolution targets and resolve their periods through known and new diffusers.The tested line-based targets were not seen during training.
- Experimental results: Experiments measure all-optical reconstructions of objects distorted by known and new diffusers, including labeled resolution-test periods.The apparatus uses coherent THz illumination and a four-layer network.
- Memory: Memory tests compare PCC across networks trained with n=1, 10, 15, and 20 diffuser conditions for known, recent-training, and new diffusers.The comparisons include averages and standard deviations over different diffusers.
- Network depth: Additional trainable diffractive surfaces improve all-optical reconstruction of objects viewed through unknown random diffusers.Figure 7 reports averages with error bars representing variation across diffusers.