Source-linked AI summary
Massively Parallel Amplitude-Only Fourier Neural Network
Mario Miscuglio, Zibo Hu, Shurui Li, Jonathan George, Roberto Capanna, Philippe M. Bardet, Puneet Gupta, Volker J. Sorger
TL;DR
Electronic machine-intelligence systems face physics and data-handling constraints that motivate heterogeneous processors. This paper introduces an amplitude-only Fourier-optical processor using massively parallel DMD-based filtering, achieving high classification accuracy and faster convolution processing. The authors also identify camera acquisition and electronic interfacing as practical boundaries for the current and projected systems.
Problem
Electronic machine-intelligence systems face limitations from capacitive wire charging and data storage and handling, motivating processor heterogeneity.
Method
The paper develops an amplitude-only electro-optic Fourier filter using passive 4f optics and programmable DMDs for dynamic convolutional filtering.
Results
98% and 54% classification accuracy were obtained on MNIST and CIFAR-10, respectively, with up to 1,000 convolutions per second between two 2MP images.
Takeaways & Limitations
The demonstration supports amplitude-only optical processing as a high-speed, dynamically programmable alternative for hybrid optical-electronic CNNs.
Takeaways & Limitations
The current implementation is constrained by camera acquisition speed, while projected higher-throughput configurations would make the electronic interface the bottleneck.
Abstract
from arXiv · showhide
Machine-intelligence has become a driving factor in modern society. However, its demand outpaces the underlying electronic technology due to limitations given by fundamental physics such as capacitive charging of wires, but also by system architecture of storing and handling data, both driving recent trends towards processor heterogeneity. Here we introduce a novel amplitude-only Fourier-optical processor paradigm capable of processing large-scale ~(1,000 x 1,000) matrices in a single time-step and 100 microsecond-short latency. Conceptually, the information-flow direction is orthogonal to the two-dimensional programmable-network, which leverages 10^6-parallel channels of display technology, and enables a prototype demonstration performing convolutions as pixel-wise multiplications in the Fourier domain reaching peta operations per second throughputs. The required real-to-Fourier domain transformations are performed passively by optical lenses at zero-static power. We exemplary realize a convolutional neural network (CNN) performing classification tasks on 2-Megapixel large matrices at 10 kHz rates, which latency-outperforms current GPU and phase-based display technology by one and two orders of magnitude, respectively. Training this optical convolutional layer on image classification tasks and utilizing it in a hybrid optical-electronic CNN, shows classification accuracy of 98% (MNIST) and 54% (CIFAR-10). Interestingly, the amplitude-only CNN is inherently robust against coherence noise in contrast to phase-based paradigms and features an over 2 orders of magnitude lower delay than liquid crystal-based systems. Beyond contributing to novel accelerator technology, scientifically this amplitude-only massively-parallel optical compute-paradigm can be far-reaching as it de-validates the assumption that phase-information outweighs amplitude in optical processors for machine-intelligence.
Results
The amplitude-only Fourier engine uses DMD-based 4f optics to perform programmable Fourier-domain filtering with massive spatial parallelism. Experiments demonstrate close simulation–hardware agreement, high classification accuracy, and faster convolution processing than GPU- and SLM-based systems.
- Results: Fourier-domain filtering is implemented with a DMD-based 4f system that transforms, multiplies, and inverse-transforms optical signals.The second DMD acts as a spatial mask in the Fourier plane, while lenses perform the domain transformations.
- Results: 4 Peta operations per second is the nominal throughput of the 2MPx optical convolution engine.The throughput is defined for space-domain-equivalent convolution operations and is enabled by the system’s parallel optical processing.
- Results: 10 kHz programmability allows both feature maps and kernels to be updated at the same high rate.Unlike fixed filters based on physical transparencies, the DMD implementation supports dynamic kernel updates.
- Results: 2MPx DMD and camera arrays provide massive parallelism for amplitude-only Fourier filtering.The architecture separates in-plane programmability from the z-direction of information flow and uses passive optical operations.
Conclusions
The paper demonstrates an amplitude-only electro-optical Fourier-filter engine for massively parallel convolution and integrates it into CNN classification. The prototype combines high throughput, fast programmability, and competitive classification accuracy while showing that trained networks can compensate for phase-information loss.
- Implications: Dynamic Fourier filtering is implemented with high-speed programmability and throughput, exploiting optical parallel computing and power efficiency.The authors position the hardware as an alternative machine-learning architecture for inference tasks.
- Experimental demonstration: 98% and 54% classification accuracy were obtained for MNIST and CIFAR-10, respectively.The experimentally obtained convolution outputs were passed to a fully connected layer for final prediction.
- Performance: The amplitude-only approach reaches classification accuracy comparable to the phase-only optical engine while offering 2 orders of magnitude faster programmability.The paper attributes compensation for phase neglect to robustness provided by neural-network training.
- Implications: The platform could support flexible image analysis, classification, and super-resolution, as well as high-bandwidth free-space communication with local edge preprocessing.The paper describes these as potential application directions for the compact optical information-processing scheme.
Supplementary Online Information for
The supplementary information describes the optical convolution system, its physical modeling and parallelization strategies, and simulation results for MNIST and CIFAR-10.
- System Data I/O: The streaming architecture links a PC, FPGA, DLP boards, and camera for synchronized optical convolution and electronic postprocessing.The FPGA controls the DLP boards, generates patterns, and synchronizes DMD refresh; camera output returns to the PC.
- Applications: The system demonstrates high-speed applications including 20kHz filtering of an optical illusion and retina recognition using DMD-based spatial-frequency processing.The optical illusion separates Marilyn Monroe into low spatial frequencies and Albert Einstein into high spatial frequencies.
- Physical Modeling: The physical model accounts for DMD modulation fidelity, micromirror geometry, orientation-induced phase, lens aberrations, apertures, and spatial filtering.The DMD magnitude transfer function includes contrast and effective pixel size, while Seidel coefficients model optical aberrations.
- Physical Modeling: The 4f processor Fourier-transforms the input, applies a pixel-wise spatial-frequency filter with the second DMD, and inverse-transforms the result into a convolution.The filtered image is acquired by a high-speed camera; camera integration maps each expanded 17x17 super-pixel to one pixel.
- Parallelization Strategies: Parallelization tiles multiple 28×28 input images in the object plane so their frequency components are processed simultaneously by one Fourier-plane filter.The supplementary architecture also describes CNN training and evaluation using the amplitude-only Fourier filtering layer.
- Simulation Results: 98% MNIST simulation accuracy matches a full-precision single-layer space-domain CNN, while CIFAR-10 reaches 62% versus 63% for a similar full-precision model.The reported table results use a 32×32 Fourier kernel, different from the final adopted model.
- Simulation Results: 55% to 61% CIFAR-10 simulation accuracy follows from switching kernel weights from 1-bit to 2-bit mode, with a small throughput loss.The reported performance loss in binarized CIFAR-10 versions is attributed mainly to binarized kernel weights and the harder benchmark.