Source-linked AI summary
Reinforcement Learning in a large scale photonic Recurrent Neural Network
Julian Bueno, Sheler Maktoobi, Luc Froehly, Ingo Fischer, Maxime Jacquot, Laurent Larger, Daniel Brunner
TL;DR
Large-scale photonic neural networks require demonstrations of learning with many nonlinear nodes and efficient parallel hardware. The paper implements reinforcement learning in a diffractively coupled photonic RNN using a DMD readout, achieving low error on chaotic time-series prediction despite Boolean weights. The results show that symmetry breaking can compensate for exclusively positive optical intensities in analogue neural networks.
Problem
Learning in large-scale photonic neural networks with many nonlinear nodes and fully parallel, efficient hardware had not yet been demonstrated sufficiently.
Method
The paper uses SLM pixels as nonlinear RNN nodes, a DOE for recurrent coupling, and a DMD for Boolean readout reinforcement learning on Mackey-Glass prediction.
Results
The photonic RNN reaches ε ≈0.013 after training and ε = 0.042 on 4,500 unseen consecutive datapoints, with testing error matching training error.
Takeaways & Limitations
Symmetry breaking through nodes with positive and negative response slopes can partially compensate for the absence of negative weights in the analogue photonic network.
Abstract
from arXiv · showhide
Photonic Neural Network implementations have been gaining considerable attention as a potentially disruptive future technology. Demonstrating learning in large scale neural networks is essential to establish photonic machine learning substrates as viable information processing systems. Realizing photonic Neural Networks with numerous nonlinear nodes in a fully parallel and efficient learning hardware was lacking so far. We demonstrate a network of up to 2500 diffractively coupled photonic nodes, forming a large scale Recurrent Neural Network. Using a Digital Micro Mirror Device, we realize reinforcement learning. Our scheme is fully parallel, and the passive weights maximize energy efficiency and bandwidth. The computational output efficiently converges and we achieve very good performance.
I. INTRODUCTION
Photonic neural networks offer parallel processing for large nonlinear-node systems, while reservoir computing reduces implementation complexity. This work demonstrates scalable diffractively coupled photonic recurrent networks with parallel reinforcement learning and promising scaling properties.
- Photonic neural networks are attractive because neural-network computation requires massively parallel vector-matrix products.
- Reservoir computing reduced the complexity of analog electronic and photonic recurrent neural networks, supporting their implementation on photonic platforms.
- Up to 2025 nonlinear network nodes were demonstrated using SLM pixels, with recurrent connections implemented by a passive DOE.
- Simulations indicated scalability to well over 20,000 nodes, while the realized network size was limited by the imaging field of view rather than the concept.
- A DMD enabled parallel, passive, and energy-efficient readout learning for a 900-node photonic RNN, with bandwidth and power consumption independent of system size.
II. NONLINEAR NODES AND DIFFRACTIVE NETWORK
The photonic RNN uses SLM pixels as nonlinear Ikeda-map nodes and a DOE to create heterogeneous recurrent coupling. The optical state is imaged, modulated, detected, and fed back to update the network.
- A single input node injects information into recurrently connected nonlinear nodes, whose summed state forms the computational output.
- The experimental setup images the SLM state through a PBS and DOE onto a camera, while the DMD forms a spatially modulated state for output detection.
- Each SLM pixel’s modulated optical field is detected as a state, rescaled to the SLM grid, and processed as an Ikeda map after phase and feedback operations.
- The DOE establishes the recurrent coupling matrix and intentionally creates heterogeneous connectivity because local diffractive-order intensities vary across pixels.
- The external input is injected with strength γ through a weighted input matrix while software updates the network state and controls the instruments.
- The realized network currently supports approximately 2,500 nodes, with size limited by the imaging setup’s field of view.
III. NETWORK READOUT WEIGHTS
Learning is restricted to the photonic RNN’s readout layer, implemented with a DMD that applies Boolean spatial weights to the network state. The resulting detector signal provides the RNN output.
- The phase-offset strategy shown in the learning curves produces nodes with both negative- and positive-slope responses.
- A lens images the 900-node SLM state onto a DMD, whose micromirrors direct selected optical contributions to a detector.
- The DMD readout weight vector combines the network state into the RNN output measured by the detector.
- DMD readout weights are Boolean because each node contribution is either on or off, and they are spatial rather than temporal modulations.
IV. PHOTONIC LEARNING
The photonic RNN learns nonlinear time-series prediction by iteratively modifying Boolean readout weights according to reinforcement-learning feedback. Despite positive-only weights, symmetry breaking through nonlinear nodes with positive and negative slopes enables strong convergence and generalization.
- Learning procedure: Reinforcement learning updates one Boolean DMD readout weight at a time, retaining a change when it reduces prediction error and reverting it otherwise.The selected weight is chosen using randomized initialization and a bias against recently updated weights.
- Learning procedure: The task is one-time-step prediction of the chaotic Mackey-Glass sequence, evaluated with normalized mean square error after discarding transient data.The target is yT(n + 1) = u(n + 2), and the first 30 data points are excluded from error evaluation.
- Symmetry breaking: Positive-only Boolean weights restrict the available functional space, but phase offsets create nodes with predominantly positive or negative response slopes.This supplies effective slope diversity without introducing negative optical weights.
- Symmetry breaking: µ = 0.45 gives the best performance among the tested operating-point ratios, near an equal distribution of positive- and negative-slope nodes.The experiments use β = 0.8 and γ = 0.4 while varying µ.
- Performance: ε ≈ 0.013 is reached after 500 training steps, while testing error on 4500 unseen consecutive datapoints matches the training error.The predicted output is visually difficult to distinguish from the normalized target, indicating successful generalization of the target system’s properties.
- Performance: The achieved error is larger by a factor of 2.2 than a semiconductor-laser delay RC and by 6.5 than a Mach-Zehnder-modulator setup.Those comparison systems used digitally applied double-precision readout weights; increasing DMD resolution may reduce the present error.
V. CONCLUSION
The authors demonstrate a photonic recurrent neural network with hundreds of nonlinear nodes and photonic reinforcement learning. Its Boolean DMD readout achieves very low prediction error, while symmetry breaking addresses positive-intensity constraints and supports future fully implemented analogue-network experiments.
- The photonic RNN contains hundreds of nonlinear nodes and implements photonic reinforcement learning.
- A Boolean-valued DMD readout trains the system to predict the chaotic MG sequence.
- The resulting prediction error is very low despite Boolean readout weights.
- Symmetry breaking inside the RNN compensates for exclusively positive intensities in analogue neural-network hardware.
- Physical hardware networks and readout weights enable experiments evaluating the robustness and efficiency of learning strategies in fully implemented analogue neural networks.
- The same 4f architecture could support extremely fast all-optical systems because it allows self-coupling.
FUNDING INFORMATION
The work acknowledges support from the Region Bourgogne Franche-Comté, the Labex ACTION program, and the Volkswagen Foundation NeuroQNet project.
- The work was supported by the Region Bourgogne Franche-Comté.
- Additional support came from the Labex ACTION program and the Volkswagen Foundation NeuroQNet project.