Source-linked AI summary
Misalignment Resilient Diffractive Optical Networks
Deniz Mengu, Yifan Zhao, Nezih T. Yardimci, Yair Rivenson, Mona Jarrahi, Aydogan Ozcan
TL;DR
Multiple diffractive layers improve network performance but make fabrication and alignment challenging. This paper trains diffractive networks with modeled random layer misalignments, extending the approach to differential and hybrid designs. The resulting systems maintain performance across misalignments, including in physical experiments.
Problem
Multiple diffractive layers offer performance benefits but create practical challenges for fabrication and opto-mechanical assembly.
Source: §1 · p. 2Method
Vaccinated D2NNs train with continuous random layer displacements in the optical forward model, and the strategy is extended to differential and jointly trained hybrid networks.
Source: §1 · p. 2, §2.1 · p. 3Results
Under lateral misalignments of (-8.48λ,8.48λ) in both directions, the error-free design achieves 12.8% test accuracy, vaccination yields 67.53% blind inference performance, and the hybrid system reaches 79.6% accuracy.
Source: §3 · p. 6Takeaways & Limitations
Vaccination improves resilience to physical misalignments in diffractive, differential, and hybrid network designs, and may also address modeled detection noise or fabrication defects.
Source: §1 · p. 2, §3 · p. 7Layer displacements are modeled independently, with the numerical investigation treating lateral and axial misalignments as separate cases.
Source: §2.1 · p. 3
Abstract
from arXiv · showhide
As an optical machine learning framework, Diffractive Deep Neural Networks (D2NN) take advantage of data-driven training methods used in deep learning to devise light-matter interaction in 3D for performing a desired statistical inference task. Multi-layer optical object recognition platforms designed with this diffractive framework have been shown to generalize to unseen image data achieving e.g., >98% blind inference accuracy for hand-written digit classification. The multi-layer structure of diffractive networks offers significant advantages in terms of their diffraction efficiency, inference capability and optical signal contrast. However, the use of multiple diffractive layers also brings practical challenges for the fabrication and alignment of these diffractive systems for accurate optical inference. Here, we introduce and experimentally demonstrate a new training scheme that significantly increases the robustness of diffractive networks against 3D misalignments and fabrication tolerances in the physical implementation of a trained diffractive network. By modeling the undesired layer-to-layer misalignments in 3D as continuous random variables in the optical forward model, diffractive networks are trained to maintain their inference accuracy over a large range of misalignments; we term this diffractive network design as vaccinated D2NN (v-D2NN). We further extend this vaccination strategy to the training of diffractive networks that use differential detectors at the output plane as well as to jointly-trained hybrid (optical-electronic) networks to reveal that all of these diffractive designs improve their resilience to misalignments by taking into account possible 3D fabrication variations and displacements during their training phase.
1 Introduction
Vaccinated D2NNs train with modeled layer misalignments to improve robustness, and experiments show they preserve recognition performance across physical shifts that often defeat standard designs. The strategy also extends to differential and hybrid networks and may address other modeled error sources.
1 Introduction
- D2NNs combine deep-learning training with physical light propagation through diffractive layers to perform all-optical inference.Prior studies reported generalization to unseen image data, including >98% blind testing accuracy for handwritten digits and >90% for fashion products using a five-layer design.Source: §1 · p. 1
- Adding diffractive layers can improve inference accuracy, diffraction efficiency, and signal contrast, but complicates fabrication and opto-mechanical assembly.A depth advantage was also demonstrated for spatially controlled wavelength demultiplexing.Source: §1 · p. 2
- Vaccination models physical variations and layer-to-layer misalignments as continuous random variables during training, yielding designs robust across the displacement range used.The five-layer handwritten-digit models used independently and uniformly distributed x, y, and z displacement vectors for each layer.Source: §1 · p. 2
- Experiments show vaccinated networks preserve inference accuracy across a wide range of physical misalignments, while standard designs frequently fail to recognize the correct class.The comparison used two five-layer, 3D-printed networks for handwritten-digit classification and deliberately shifted the third layer.Source: §1 · p. 2
- The vaccination strategy also applies to differential detectors and jointly trained hybrid networks, whose electronic back end uses a single fully connected layer with 110 learnable parameters.Differential detectors use positive and negative signals per class to address optical intensity's non-negativity constraint; hybrid systems showed stronger adaptation than standard or differential diffractive networks.Source: §1 · p. 2
- The framework may also mitigate detection noise and fabrication imperfections when their approximate analytical models and probability distributions are included in training.The authors anticipate applications in machine vision and sensing.Source: §1 · p. 2
2 Results
Training with modeled 3D layer displacements improves misalignment tolerance in standard, differential, and hybrid diffractive networks. Experiments with deliberately shifted layers found fewer classification errors for the vaccinated design.
2.1 Training and testing of v-D2NNs
- v-D2NN training samples independent layer displacements from uniform distributions, treating lateral and axial misalignments separately; blind testing resamples layer positions for each MNIST image.Each test image propagates through a different misaligned configuration.Source: §2.1 · p. 3
2.2 Misalignment analysis of all-optical and hybrid diffractive systems
- At Δtest=2.12λ lateral misalignment, vaccination raised blind accuracy from 38.40% to 94.44%, while ideal-alignment accuracy was 96.1% versus 97.77% for the error-free design.For axial misalignment at Δ(z,test)=2.4λ, the vaccinated and error-free designs reached 97.39% and 94.88%, respectively.Source: §2.2 · p. 4
- Vaccinated differential designs reached 97.3% versus 96.91% at 0.53λ and 96.18% versus 95.79% at 1.06λ, though detector cross-talk can erase their advantage.Differential designs have twice as many detectors in each output region of interest, making their signals more prone to cross-talk under uncontrolled displacement.Source: §2.2 · pp. 4–5
- Vaccinated hybrid systems achieved 97.92% versus 96.91% at 0.53λ and 97.15% versus 95.79% at 1.06λ, outperforming standard all-optical designs under those tested misalignments.Under ideal assembly, the jointly optimized hybrid model achieved 98.3% classification accuracy.Source: §2.2 · p. 5
2.3 Experimental results
- The experimentally tested vaccinated design used training ranges of 4.24λ laterally and 4.8λ axially, reaching 89.5% accuracy under ideal conditions after prioritizing power efficiency and signal contrast.Six test digits were selected to be equally favorable to both designs, and the third layer was shifted to 12 locations.Source: §2.3 · p. 6
- Across 72 shifted-layer measurements, the error-free design misclassified 23 cases, while the vaccinated network misclassified 2; both classified all six objects correctly at ideal alignment.The third layer was moved to 12 locations for each of six test objects.Source: §2.3 · p. 6
3 Discussion
Under severe lateral misalignment, vaccinated systems recover some recognition performance, and the hybrid system performs best in the reported case. The discussion presents vaccination as a way to address physical displacements and potentially other error sources.
3 Discussion
- The tested 16.96λ per-direction displacement range was described as comparable to the 106.66λ diffractive-layer size, probing the limits of vaccinated systems’ adaptability.The discussion notes that lens-based imaging would also fail under such severe opto-mechanical misalignments because of acute aberrations.Source: §3 · p. 7
- Hybrid systems with a diffractive front-end and electronic back-end achieved higher inference accuracy under mechanical perturbations than all-optical systems.The authors identify the use of a few discrete output detectors as an opportunity for low-latency, power-efficient, memory-friendly machine vision.Source: §3 · p. 7
- The conclusion proposes probabilistic layer-to-layer misalignments during training as a way to improve robustness, and notes possible extension to detection noise and fabrication defects.The authors anticipate uses in diffractive-network-based machine vision and sensing systems.Source: §3 · p. 7
Figures
The figures introduce three D2NN classification architectures, compare their performance across misalignment types and training ranges, and summarize numerical and experimental tests of vaccination.
Figures
- Standard D2NNs assign one output detector to each class, differential D2NNs use detector pairs and normalized differences, and hybrid systems feed detector signals to an electronic classifier.The figure contrasts two all-optical readouts with a jointly trained optical-electronic system.Source: Figure 1 · p. 10
- Figure 2 compares vaccinated and non-vaccinated standard, differential, and hybrid systems under axial and lateral testing misalignments, showing wider tolerance with a modest accuracy compromise.Panels A–C test axial misalignments; panels D–F test lateral misalignments.Source: Figure 2 · p. 11
- Figure 3 compares standard, differential, and hybrid systems across matched training ranges, with axial tests in panels A–F and lateral tests in panels G–L.The training ranges vary across panels, including error-free designs.Source: Figure 3 · p. 12, Figures · p. 13
- Figure 4 summarizes differential and hybrid system accuracies across testing misalignment ranges and compares the three architectures when training and testing ranges match.Panels A and B compare non-vaccinated and vaccinated systems; panel C compares the three architectures at Δtr = Δtest.Source: Figure 4 · p. 13
- Figure 5 depicts a vaccinated network trained for lateral and axial misalignments of 4.24λ and 4.8λ, respectively, and the 12 off-center positions used to perturb its third layer.The figure also shows the error-free and vaccinated 3D-printed designs and the experimental setup.Source: Figure 5 · p. 14
- Figure 6 presents measured class scores for handwritten digits 0 and 5 at different physical misalignments, using red and green rectangles to mark incorrect and correct inferences.The red dot in each coordinate system indicates the physical misalignment.Source: Figure 6 · p. 15