PaperlayerView exact revision on arXiv

Source-linked AI summary

Design of optical neural networks with component imprecisions

Michael Y. -S. Fang, Sasikanth Manipatruni, Casimir Wierzynski, Amir Khosrowshahi, Michael R. DeWeese

arXiv:2001.01681v1cs.NEcs.ETphysics.optics

TL;DR

The paper asks how ONN architecture affects robustness to imprecise photonic components, an important issue for scalable fault-resistant systems. It trains and evaluates GridNet and FFTNet on handwritten-digit classification under modeled component errors. GridNet performs better ideally, but FFTNet becomes more robust under realistic errors, while architecture and configuration choices both shape tolerance.

  • Problem

    ONNs are susceptible to fabrication and other photonic-component imprecisions, but their relationship with architecture requires systematic analysis for scalable design.

  • Method

    The authors train GridNet and FFTNet for handwritten-digit classification, then evaluate accuracy and robustness across component imprecisions, architectures, and configurations.

  • Results

    97.8% versus 94.8% ideal accuracy favors GridNet, but realistic errors drive GridNet below FFTNet, while FFT-like designs remain more robust at matched depth.

  • Takeaways & Limitations

    Fault-resistant ONN design should consider both architectural choices and smaller algorithmic configuration choices such as singular-value ordering.

  • Takeaways & Limitations

    A complete analysis of ONN robustness to perturbations, including software-training effects, remains outside the paper’s scope.

Abstract

from arXiv · show

For the benefit of designing scalable, fault resistant optical neural networks (ONNs), we investigate the effects architectural designs have on the ONNs' robustness to imprecise components. We train two ONNs -- one with a more tunable design (GridNet) and one with better fault tolerance (FFTNet) -- to classify handwritten digits. When simulated without any imperfections, GridNet yields a better accuracy (~98%) than FFTNet (~95%). However, under a small amount of error in their photonic components, the more fault tolerant FFTNet overtakes GridNet. We further provide thorough quantitative and qualitative analyses of ONNs' sensitivity to varying levels and types of imprecisions. Our results offer guidelines for the principled design of fault-tolerant ONNs as well as a foundation for further research.

1 Introduction

This study examines how ONN architecture affects robustness to photonic imprecision, comparing trainable GridNet and fault-tolerant FFTNet designs for handwritten-digit classification. Although GridNet performs better ideally, FFTNet becomes superior under realistic errors, motivating architecture-aware design.

  • The study addresses ONNs’ susceptibility to fabrication and other photonic-component imprecisions while extending prior work from unitary matrices to trained nonlinear classifiers.Its primary figure of merit is classification accuracy because the learned linear operations are not known in advance.
  • Pre-fabrication software training transfers parameters to multiple imprecise fabricated instances without subsequent device characterization or tuning.This approach is presented as more scalable than post-fabrication optimization, whose characterization cost scales with network dimension as N^2.
  • 97.8% ideal accuracy for GridNet versus 94.8% for FFTNet, but GridNet rapidly falls below FFTNet under realistic component errors.The comparison evaluates classification accuracy after software training, rather than fidelity to a target unitary matrix.
  • GridNet uses a more tunable universal architecture, whereas FFTNet uses an FFT-like architecture with greater robustness to error at matched depth.The authors report that FFT-like designs are more robust than Grid-like designs beyond the effect of FFTNet’s shallower depth.
  • The paper studies localized MZI imprecisions and finds sensitivity depends on algorithmic choices as well as physical architecture.These analyses aim to provide a foundation for scalable, error-resistant ONN design.

2 Physical design of optical neural networks

ONNs combine programmable optical linear multipliers with nonlinearities, implementing linear transformations through MZI-based unitary and diagonal components. The paper contrasts universal grid-like multipliers with shallower, compact FFT-like multipliers that trade universality for reduced depth.

  • An ONN alternates programmable optical linear multipliers and element-wise nonlinearities, with linear multipliers implemented by two unitary multipliers and a diagonal layer.The unitary components use configurable MZIs built from two beamsplitters and two phaseshifters.
  • MZIs perform unitary mixing between optical channels using fixed 50:50 beamsplitters and learned phaseshifters parameterized by θ and φ.Complex-valued inputs are encoded as coherent signals on single-mode waveguides.
  • GridUnitary multipliers implement arbitrary unitary matrices directly, while FFTUnitary multipliers provide a compact non-universal alternative inspired by the Cooley-Tukey FFT.FFTUnitary reduces multiplier depth from N to log2(N), lowering overall noise and loss but requiring waveguide crossings.
  • A diagonal multiplier attenuates channels separately, and global optical amplification compensates for the inability of MZIs to amplify signals.Together with two unitary multipliers, it enables a universal linear multiplier through singular-value decomposition.
  • GridNet uses universal unitary multipliers, whereas FFTNet uses FFT-Unitary multipliers in the three-layer MNIST network design.

3 Neural network architecture and software implementation

The authors implement complex-valued GridNet and FFTNet classifiers for MNIST, train their optical parameters, and model fabrication and calibration errors during inference. GridNet achieves higher ideal accuracy, while FFTNet is substantially less sensitive to component and quantization errors.

  • GridNet and FFTNet classify MNIST handwritten digits using fully connected feedforward networks with complex-valued inputs and two hidden layers.The 784 real-valued pixels are paired into 392 complex-valued input dimensions.
  • GridNet is trained through complex-valued network weights followed by SVD decomposition into unitary and diagonal optical multipliers, whereas FFTNet is trained directly.GridNet’s singular values are randomly permuted for a fair comparison because SVD is ambiguous under permutations.
  • Inference imprecision is modeled as independent zero-mean Gaussian noise on phaseshifter phases and beamsplitter transmittance.Representative values are σ_PS ≈ 0.01 rad and σ_BS ≈ 0.01, or 1%.

4 Results

GridNet achieves higher ideal accuracy, but FFTNet becomes more robust under component imprecision. Matched-depth and localized-error experiments show that FFT-like architecture and algorithmic choices both shape ONN robustness.

  • 4.1 Degradation of network accuracy: FFTNet is more resistant to phaseshifter error than to beamsplitter error, revealing qualitatively different sensitivities across component types.
  • 4.1 Degradation of network accuracy: At σP S = 0.01 and σBS = 1%, FFTNet surpasses GridNet despite GridNet's higher ideal accuracy.GridNet starts near 98% accuracy, while FFTNet starts lower; increasing component errors rapidly degrades GridNet but changes FFTNet little.
  • 4.2 Stacked FFTUnitary and truncated GridUnitary: FFTNet's robustness is not explained solely by shallower depth, because matched-depth comparisons still favor FFT-like architectures.The comparison uses StackedFFT and TruncGrid to equalize the numbers of MZI layers between FFT-like and grid-like designs.
  • 4.2 Stacked FFTUnitary and truncated GridUnitary: With equal MZI depth, FFT-like multipliers retain higher fidelity than grid-like multipliers as component error increases.StackedFFT is more robust than GridUnitary, and TruncGrid is more robust than FFTUnitary in the reported matched-depth comparisons.
  • 4.3 Localized imprecisions: Ordering singular values makes GridNet more susceptible to uniform imprecision, while shuffling them improves resistance in the same architecture.The resulting sensitivity pattern reflects both physical locality and algorithmic choices, not architecture alone.

5 Conclusion

The study finds that FFT-like optical neural network architectures are more robust to component imprecision than grid-like designs, even when controlling for MZI layer depth. It also shows that signal flow and singular-value configuration affect sensitivity, while complete robustness analysis remains beyond scope.

  • 5 Conclusion: 97.8% versus 94.8% ideal accuracy favors GridNet, but FFTNet maintains near-constant performance as imprecision increases while GridNet falls below 50%.Under σBS > 1% and σPS > 0.01 rad, FFTNet can be preferable despite GridNet’s greater expressivity.
  • 5 Conclusion: FFT-like designs are inherently more robust than grid-like designs when MZI layer depth is controlled.This conclusion follows from comparisons using modified truncated-grid and stacked-FFT unitary multipliers.
  • 5 Conclusion: Localized sensitivity depends more on optical signal flow than on physical MZI position, and shuffling singular values can alter robustness.Thus, procedural configuration choices can affect imprecision tolerance in addition to architectural design.
  • 5 Conclusion: The results provide architectural guidelines for designing efficient, fault-resistant ONNs.The authors identify algorithmic and training strategies as important directions for further investigation.
  • 5 Conclusion: A complete account of ONN robustness must also consider software training and algorithmic effects, which this architecture-focused study leaves outside scope.The paper points toward a unified framework covering architectural and training-related robustness.

6 Funding

This material specifies the MZI and saturable-absorption models, estimates laser phase noise, and compares the absorption response with its Softplus approximation.

  • A MZI transfer matrix: The MZI uses two beamsplitters and two phase shifters, with bar and cross states determined by the internal phase θ.In the stated convention, T = 0 at θ = π and T = 1 at θ = 0.
  • B Laser phase noise: 50 MHz linewidth is easily achieved by modern lasers, and laser phase noise is roughly two orders of magnitude smaller than MZI phase-noise contributions.The estimate uses a 100 µm distance between phase shifters and targets σφ = 0.01 rad.
  • C Approximating saturable absorption: Saturable absorption is represented with a shifted and biased Softplus because the exact Lambert W form is difficult to implement in common deep-learning libraries.The bias is selected so the approximation and exact response both satisfy σ(0) = f(0) = 0.
  • C Approximating saturable absorption: Figure 10 compares the saturable-absorption response curve with the derived Softplus approximation.The approximation is constructed from the saturable-absorption relation and its Lambert W solution.

D Confusion matrices

The study visualizes how component imprecisions degrade GridNet and FFTNet outputs using confusion matrices under ideal and increasing-error conditions.

  • D Confusion matrices: Confusion matrices compare GridNet and FFTNet in ideal conditions and at σBS = 1%, σPS = 0.01 rad and σBS = 2%, σPS = 0.02 rad.Rows denote target classes and columns denote the ten predicted classes; the matrices use logarithmic scaling.
  • D Confusion matrices: The imprecisions were simulated 10 times, and the mean output was used to generate each confusion matrix.

E Quantization error

The quantization experiment evaluates GridNet and FFTNet after training across 4–10-bit phase-shifter precision, with FFTNet showing greater robustness.

  • E Quantization error: The quantization procedure discretizes the dimensionless voltage u using B-bit precision after defining u = V/V2π.V2π is the voltage required for a 2π phase shift.
  • E Quantization error: 4–10-bit quantization was applied after training to networks with similar ideal accuracies of approximately 95% and 98%.Ten instances of each architecture were trained before quantization, and mean accuracy with 20–80% quantiles was plotted.
  • E Quantization error: FFTNet is more robust than GridNet to post-training quantization, as in the experiments with simulated Gaussian noise.The comparison uses ten randomized instances of each network and evaluates classification accuracy across varying bit depths.

F Empirical distribution of phases

GridNet concentrates phase-shift variance near the center of its GridUnitary multipliers, whereas FFTNet exhibits a mostly spatially uniform phase distribution.

  • F Empirical distribution of phases: GridNet has lower internal phase-shift variance near the center of GridUnitary multipliers, and this trend becomes stronger as dimension N increases.The empirical trained-network distribution follows the general theoretical trend despite not matching the Haar-random prediction exactly.
  • F Empirical distribution of phases: The GridNet phase-distribution prediction assumes Haar-random unitary matrices, an assumption not guaranteed or expected for trained neural-network layers.The empirical result supports the general center-versus-edge trend but differs from the theoretical distribution.
  • F Empirical distribution of phases: FFTNet phase shifts are mostly uniform across MZIs, with nearly identical center and top histograms, consistent with its non-local connectivity.The observation is based on an ensemble of 100 trained FFTNets with random initial weights.
  • F Empirical distribution of phases: The relevant analysis concerns sensitivity to unknown post-fabrication errors without further reconfiguration, rather than optimization after known imperfections are introduced.

G BlockFFTNet

BlockFFTNet combines GridNet-like MZI blocks with FFTNet-like crossing waveguides to encourage a uniform phase distribution, but this design does not improve pre-fabrication error robustness.

  • G BlockFFTNet: BlockFFTNet uses non-local crossing waveguides between GridNet-like blocks, producing an empirically uniform phase distribution after training.The crossing-waveguide arrangement is illustrated in Fig. 15(a), while the trained phase distribution appears in Fig. 15(b).
  • G BlockFFTNet: Uniform phase distributions motivated by post-fabrication optimization do not directly yield better error resistance for networks optimized before fabrication.The paper explicitly distinguishes post-fabrication optimization tolerance from pre-fabrication error resistance.
  • G BlockFFTNet: BlockFFTNet shows no robustness improvement over GridNet and instead has a significant decrease under increasing component imprecision.The accuracy loss due to increasing imprecision is compared in Fig. 16.

H FFT algorithm and convolution

The paper maps the Cooley–Tukey FFT algorithm onto configured FFTUnitary multipliers and notes that convolution can then be implemented through Fourier-domain multiplication and inverse transformation.

  • H FFT algorithm and convolution: The Cooley–Tukey FFT algorithm can be implemented by configuring the phases of an FFTUnitary multiplier.For bit-reversed inputs, layer k uses θ = π/2 and φ = 2πk/N, up to a global phase.
  • H FFT algorithm and convolution: For N = 2^K, FFT recursively transforms the even and odd input elements across K iterations, with in-place computation possible for bit-reversed inputs.The even- and odd-element transforms are denoted E_k and O_k.
  • H FFT algorithm and convolution: Convolution is performed by multiplying the Fourier-transformed signal by the Fourier-transformed kernel, followed by an inverse Fourier transform.This provides the paper's Fourier-domain route from FFT operations to convolution.
Loading 2001.01681v1…