PaperlayerView exact revision on arXiv

Source-linked AI summary

Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks

Deniz Mengu, Yi Luo, Yair Rivenson, Aydogan Ozcan

arXiv:1810.01916v4cs.NEcs.LGphysics.optics

TL;DR

D2NN-based optical learning needs improved training and practical integration with electronic networks. The paper changes the loss and modulation parameterization, achieving strong five-layer optical classification and hybrid performance while compressing electronic inputs to 10×10 pixels.

  • Problem

    D2NN optical classifiers require improved training, while hybrid optical-electronic systems seek lower-dimensional electronic inputs and reduced electronic complexity.

  • Method

    The paper analyzes D2NN design, replaces training components to reduce vanishing-gradient effects, and jointly optimizes a diffractive front-end with an electronic neural network.

  • Results

    Five-layer phase-only D2NNs reached 97.18% and 89.13% accuracy, while a jointly optimized hybrid reached 98.71% and 90.04% with input compressed more than 7.8 times to 10×10 pixels.

  • Takeaways & Limitations

    D2NN-based hybrids can combine optical processing with simple electronic networks using substantially fewer electronic input pixels.

Abstract

from arXiv · show

Optical machine learning offers advantages in terms of power efficiency, scalability and computation speed. Recently, an optical machine learning method based on Diffractive Deep Neural Networks (D2NNs) has been introduced to execute a function as the input light diffracts through passive layers, designed by deep learning using a computer. Here we introduce improvements to D2NNs by changing the training loss function and reducing the impact of vanishing gradients in the error back-propagation step. Using five phase-only diffractive layers, we numerically achieved a classification accuracy of 97.18% and 89.13% for optical recognition of handwritten digits and fashion products, respectively; using both phase and amplitude modulation (complex-valued) at each layer, our inference performance improved to 97.81% and 89.32%, respectively. Furthermore, we report the integration of D2NNs with electronic neural networks to create hybrid-classifiers that significantly reduce the number of input pixels into an electronic network using an ultra-compact front-end D2NN with a layer-to-layer distance of a few wavelengths, also reducing the complexity of the successive electronic network. Using a 5-layer phase-only D2NN jointly-optimized with a single fully-connected electronic layer, we achieved a classification accuracy of 98.71% and 90.04% for the recognition of handwritten digits and fashion products, respectively. Moreover, the input to the electronic network was compressed by >7.8 times down to 10x10 pixels. Beyond creating low-power and high-frame rate machine learning platforms, D2NN-based hybrid neural networks will find applications in smart optical imager and sensor design.

I. INTRODUCTION

The paper analyzes and improves Diffractive Deep Neural Networks (D2NNs), then integrates them with electronic neural networks to reduce electronic input dimensionality while maintaining strong classification performance.

  • D2NN framework: D2NNs use computer-trained diffractive layers whose optimized transmission or reflection coefficients process light into a desired task without operational power beyond illumination and detection.The layers can be physically fabricated and stacked as passive all-optical networks.
  • Analysis and improvements: The study examines design parameters, multiple diffractive layers, improved loss functions, and reduced vanishing-gradient effects in backpropagation.It evaluates five-layer classifiers on MNIST handwritten digits and Fashion-MNIST products.
  • All-optical classification: 97.18% and 89.13% blind-testing accuracy were achieved by five-layer phase-only D2NNs for handwritten digits and fashion products, respectively.Complex-valued modulation further improved accuracy to 97.81% and 89.32%, respectively.
  • Hybrid neural networks: Hybrid systems place a D2NN before an electronic neural network to optically compress information, reducing digitized detector pixels and electronic-network complexity.The front-end can use an ultra-compact layer spacing of a few wavelengths.
  • Hybrid results: 98.71% and 90.04% blind classification accuracy were obtained for digits and fashion products using a jointly optimized five-layer phase-only D2NN and one fully connected electronic layer.The electronic input was compressed more than 7.8 times to 10×10 pixels.
  • Applications: D2NN-based hybrids are positioned for low-power, high-frame-rate learning platforms and compact optical imaging and sensing systems.The proposed design merges all-optical computation with low-power electronic networks and opto-electronic sensor arrays.

II. RESULTS AND DISCUSSION

The paper replaces sigmoid-based modulation parameterization with a ReLU-based formulation to mitigate vanishing gradients and expands training improvements to complex-valued optical modulation.

  • Neuron parameterization: D2NN neurons use complex transmission coefficients whose amplitude and phase are represented through trainable latent variables.The latent variables are mapped to the physical modulation parameters during training.
  • Sigmoid limitations: Sigmoid parameterization bounds amplitude and phase but can suffer vanishing gradients and restrict use of the neurons’ available dynamic range.The sigmoid acts on auxiliary variables rather than information flowing through the network.
  • ReLU formulation: The replacement formulation uses ReLU for amplitude and phase parameterization, with explicit normalization keeping amplitude within (0,1).ReLU is identified as the Rectified Linear Unit, and M denotes the number of neurons per layer.
  • Parameterization results: 85.40% and 86.68% Fashion-MNIST accuracy followed the parameterization changes, up from 81.13% and 86.33% for phase-only and complex-valued five-layer D2NNs, respectively.These results used 40×λ axial spacing between layers.
  • Alternative formulation: The reported results use the equation-based formulation, although real-imaginary parameterization with Wirtinger derivatives is also possible.This identifies an alternative training formulation rather than a result evaluated in the paper.

B. Effect of the learning loss function on the performance of all-optical diffractive neural networks

The section compares loss functions for training all-optical D2NNs and reports higher classification accuracy after adopting softmax-cross-entropy alongside revised neuron modulation.

  • Cross-entropy loss is generally more suitable for classification because it minimizes negative log-likelihood, whereas MSE was used in earlier D2NN work.Because detector signals are not necessarily probabilities, training introduces a softmax layer; inference still assigns classes using the maximum output-plane optical signal.
  • 97.18% MNIST accuracy was achieved by a 5-layer phase-only D2NN, up from 91.75% with the earlier configuration.The design used a 40×λ axial distance between layers and combined revised neuron-modulation parametrization with cross-entropy loss.
  • 97.81% MNIST accuracy resulted when the same approach used complex-valued modulation with learnable phase and amplitude coefficients.
  • 89.13% Fashion-MNIST accuracy was obtained with a 5-layer phase-only D2NN, compared with 81.13% previously.
  • 89.32% Fashion-MNIST accuracy was obtained with complex-valued modulation, compared with 86.33% for the earlier complex-valued design.
  • The results indicate that linear-material D2NNs can already achieve decent classification performance, while motivating research on optical nonlinearities.

C. Performance trade-offs in D2NN design

D2NN design involves a trade-off between classification accuracy and optical-output quality: SCE improves inference performance, while MSE favors efficiency and signal contrast.

  • Power efficiency is defined as the target-label detector intensity divided by total output-plane optical signal for correctly classified test samples.The analysis compares 1-, 3-, and 5-layer phase-only D2NNs on MNIST and Fashion-MNIST.
  • Signal contrast is the normalized difference between the correct-class detector signal and the strongest competing detector signal.
  • SCE is less restrictive about output-light distribution, giving D2NNs more freedom to improve classification accuracy.This flexibility is contrasted with MSE’s pixel-by-pixel comparison against a user-designed output distribution.
  • MSE promotes higher signal contrast and power efficiency, whereas SCE improves inference accuracy at the cost of both output measures.
  • Reduced power efficiency and signal contrast under SCE increase vulnerability to opto-electronic detection noise.
  • Mechanical misalignment in axial or lateral directions can cause discrepancies between the trained network model and its physical implementation.The cited mitigation uses neuron sizes greater than 3–4 times the available fabrication resolution.

D. Advantages of multiple diffractive layers in D2NN framework

Multiple diffractive layers provide additional optical degrees of freedom that improve classification, diffraction efficiency, and output signal contrast. D2NN-based hybrid systems can reduce electronic input resolution while maintaining or improving classification performance, although larger pixel counts and more advanced electronic networks can reverse this advantage.

  • D. Advantages of multiple diffractive layers in D2NN framework: Multiple diffractive layers improve classification accuracy, diffraction efficiency, and signal contrast compared with a single layer.These properties reduce required illumination power and signal-to-noise-related error rates in experimental systems.
  • D. Advantages of multiple diffractive layers in D2NN framework: Less than 50×λ layer spacing enables multiple learnable diffractive layers to perform operations that a single layer cannot generally realize between the same planes.The additional layers provide the depth advantage without requiring large physical separation.
  • D. Advantages of multiple diffractive layers in D2NN framework: The architecture study varies axial spacing, wavelength, neuron size, and diffractive-layer width to control layer-to-layer optical connectivity.The simulations used neurons approximately 0.53×λ in size.
  • E. Connectivity in diffractive neural networks: A jointly optimized single fully-connected electronic layer can compensate for limitations from partial optical connectivity or restricted neuron modulation.The hybrid interface uses 10 detectors and is optimized together with the optical network.
  • E. Connectivity in diffractive neural networks: Jointly trained D2NN hybrids preserve classification performance with approximately 7.8× fewer input pixels, using 10×10 detectors instead of 28×28 raw data.For Fashion-MNIST, one D2NN-plus-LeNet configuration achieved 90.24% versus 87.44% for LeNet after perfect imaging optics.
  • E. Connectivity in diffractive neural networks: For higher pixel counts and more advanced electronic networks, D2NN hybrids can perform worse than perfect-imager computer-vision systems.The reported discrepancy with ResNet at 50×50 pixels was approximately 0.5% for MNIST and 4% for Fashion-MNIST, favoring the perfect-imager front end.

A. Diffractive neural network architecture

The D2NN propagates encoded optical fields through trainable diffractive layers using free-space wave propagation. Each layer modulates incident light through trainable phase and/or amplitude parameters.

  • A. Diffractive neural network architecture: Input objects are encoded in amplitude for MNIST or phase for Fashion-MNIST and illuminated by a uniform plane wave.The input plane may represent object data projected by another imaging system or lens.
  • A. Diffractive neural network architecture: Optical fields are sampled on a grid with approximately 0.53λ spacing, while propagation between layers uses the angular spectrum method.Each diffractive neuron has approximately 0.53λ×0.53λ dimensions.
  • A. Diffractive neural network architecture: The physical model follows the Rayleigh-Sommerfeld equation without requiring small diffraction angles, the Fresnel approximation, or far-field analysis.A neuron is modeled as a secondary source of wave propagation.
  • A. Diffractive neural network architecture: The distance r between a source neuron and an observation point is defined by their three-dimensional coordinate separation, with j denoting the square root of −1.These quantities parameterize the propagation kernel between diffractive layers.
  • A. Diffractive neural network architecture: Each layer's output field is formed from propagated contributions of preceding neurons and a complex-valued modulation function.The modulation combines relative amplitude and an additional phase delay at each neuron.

C. Training loss function

The diffractive network was trained with either MSE over the full output diffraction pattern or softmax-cross-entropy over detector intensities, with the latter providing additional design freedom.

  • Detector-based classification: The output plane uses D discrete detectors, one per target class, and classification selects the maximum detected intensity.Each detector measures 6.4 λ×6.4 λ, with geometry, location, and size fixed before training.
  • MSE loss: MSE compares the output-plane intensity with the target intensity distribution across all K sampling points.K represents the total number of sampling points in the entire diffraction pattern.
  • Softmax-cross-entropy loss: Softmax-cross-entropy adds a training-only softmax layer applied to normalized intensities from the D output detectors.The detected intensities are normalized to the interval (0,10) because softmax is not scale invariant.
  • Loss-function comparison: Softmax-cross-entropy acts only on detector signals, whereas MSE acts on the entire diffraction signal at the output plane.Ignoring field values outside the detectors gives the diffractive network additional degrees of freedom.
  • Loss-function comparison: Softmax-cross-entropy was also used for the imaging-optics and hybrid D2NN-plus-electronic systems reported in Tables I and II.

D. Diffractive network training

The optical, hybrid, and electronic networks were simulated and trained on MNIST and Fashion-MNIST using fixed training, validation, and test splits.

  • Implementation: All optical, hybrid, and electronic networks were simulated in Python with TensorFlow and trained for 50 epochs.Training used a GeForce GTX 1080 Ti GPU, an Intel Core i9-7900X CPU, 64 GB RAM, and Windows 10.
  • Datasets and evaluation: MNIST and Fashion-MNIST each contributed 55,000 training images, 5,000 validation images, and 10,000 test images.Each dataset contains 70,000 objects/images in total.
  • Datasets and evaluation: The best model was selected using validation performance measured after every training epoch.Testing on the reserved 10,000-image set followed completion of training.

E. D2NN-based hybrid network design and training

Hybrid systems use a D2NN optical front-end to transform detector intensities for an electronic back-end, with jointly trained configurations evaluated across detector resolutions and electronic architectures.

  • Hybrid architecture: Detected output-plane intensities from the D2NN serve as inputs to a digital neural network in the hybrid system.
  • Hybrid architecture: A single fully connected layer maps signals from D detectors to a vector containing one output for each dataset class.MNIST and Fashion-MNIST each have 10 classes.
  • Training strategy: Directly cascaded and jointly trained D2NN-CNN systems stagnated at a local minimum, motivating a two-stage training procedure.The first stage trains the optical front-end with a virtual optical classifier; the second replaces that classifier with an electronic network.
  • Training strategy: The second stage jointly trains an already trained 5-layer D2NN front-end with a digital network initialized from scratch.This procedure resembles transfer learning because additional layers and data augment a trained model.
  • Detector and electronic configurations: Detector arrays of 10×10, 25×25, and 50×50 pixels cover approximately 53.3λ×53.3λ while representing different levels of spatial undersampling.
  • Detector and electronic configurations: The electronic back-end comparison spans networks differing in trainable parameters, forward-pass FLOPs, and energy consumption.The simplest fully connected network uses as few as 1,000 trainable parameters.
  • Detector and electronic configurations: The evaluated electronic architectures include a compact 2C2F-1 CNN and LeNet, with detector values resized for LeNet's 32×32-pixel input.
  • System assumptions: Hybrid and imager-based systems assume amplitude-modulating objects and diffraction-limited imaging, making detector pitch determine captured-intensity resolution.

F. Details of D2NN-based hybrid system training procedure

The hybrid training procedure separates optical initialization from joint optical-electronic optimization to address imbalance between D2NN front-ends and powerful electronic networks.

  • Motivation: Direct joint training can be suboptimal because deep electronic CNNs adapt and converge faster than D2NN optical front-ends.The imbalance is especially relevant when the electronic component is a powerful CNN such as ResNet.
  • Stage one: The first training stage inserts a detector array and virtual diffractive layer, with the virtual layer acting as an all-optical classifier.The virtual layer is removed from the final hybrid system and provides a better initialization for the preceding D2NN.
  • Stage one: The detector array is modeled as an intensity sensor that discards phase information before the virtual classifier evaluates the signal.The virtual classifier therefore bases its decision solely on measured intensity or underlying amplitude.
  • Stage transition: After 50 epochs, the trained diffractive layers before the detector array initialize the optical component of the second stage.
  • Stage two: The second stage jointly trains the initialized D2NN optical section and an electronic back-end connected through the detector array.

APPENDIX A

Appendix A documents the experimental configurations, training procedure, and evaluation tables for all-optical and hybrid D2NN classifiers across MNIST and Fashion-MNIST.

  • MNIST evaluation: Table I reports MNIST blind-testing accuracies for all-optical D2NNs and D2NN-based hybrid systems using five diffractive layers.The hybrid systems pair the D2NN with electronic networks ranging from one fully connected layer to ResNet-50.
  • Fashion-MNIST evaluation: Table II reports Fashion-MNIST blind-testing accuracies for all-optical and hybrid systems, with amplitude encoding used for the all-optical results.Phase encoding instead yields 89.13% for phase-only and 89.32% for complex D2NNs at ΔZ=40×λ.
  • Electronic-network comparison: Table III compares the electronic components by trainable parameters, FLOPs, and energy consumption per image.The energy comparison assumes 1.5 pJ per multiply-accumulate operation.
  • Electronic-network comparison: Table IV specifies the parameters of the custom 2C2F-1 electronic architecture and directs comparison with the other electronic networks in Table III.The architecture is evaluated as part of the paper’s hybrid-network comparisons.
  • Joint training: Appendix Table A1 reports MNIST accuracies for jointly training the D2NN and electronic network from scratch instead of using the two-stage procedure.The table distinguishes ΔZ=40×λ and ΔZ=4×λ configurations.
  • Joint training: Appendix Table A2 applies the same from-scratch joint-training comparison to Fashion-MNIST with phase encoding over 0–2π.It reports phase-only and complex-modulation results for both layer spacings.
  • Optical configurations: Figure 1 depicts the MNIST and Fashion-MNIST all-optical D2NN setups, including five-layer phase-only designs and input/output field distributions.MNIST inputs use amplitude encoding, whereas Fashion-MNIST inputs use phase encoding in the illustrated designs.
  • Convergence and classification analysis: Figures 2 and 3 show convergence plots and confusion matrices for phase-only and complex-modulation D2NNs under different layer connectivities and datasets.For MNIST, the compared spacings are ΔZ=40λ and ΔZ=4λ; complex modulation trains both amplitude and phase.
Loading 1810.01916v4…