Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems

Logan G. Wright; Tatsuhiro Onodera; Martin M. Stein; Tianyu Wang; Darren T. Schachter; Zoey Hu; Peter L. McMahon

Source-linked AI summary

Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems

Logan G. Wright, Tatsuhiro Onodera, Martin M. Stein, Tianyu Wang, Darren T. Schachter, Zoey Hu, Peter L. McMahon

arXiv:2104.13386v1Apr 27, 2021cs.LGcond-mat.dis-nncs.ETphysics.optics

TL;DR

Growing energy requirements limit the scaling and broader use of modern deep neural networks. The paper introduces physics-aware training to directly train sequences of real physical systems with backpropagation, demonstrating physical neural networks across optical, mechanical, and electrical systems. The results include a hybrid optical system whose trainable physical transformations raise accuracy from roughly 90% to 97%, while the discussion projects substantial speed and energy advantages for suitable tasks and architectures.

Problem
Growing DNN computational requirements increasingly limit scaling and broader use through hardware energy efficiency.
Method
Physics-aware training directly trains sequences of controllable physical input-output transformations in situ with backpropagation to implement deep neural networks.
Results
Trainable SHG transformations increased hybrid-system accuracy from roughly 90% to 97%.
Takeaways & Limitations
Physical neural networks can combine trainable physical transformations with conventional machine-learning hardware and methods.
Takeaways & Limitations
The expected benefits depend on co-designing physics and algorithms, and on suitable task-system combinations that make physical computation the operational bottleneck.

Abstract

from arXiv · show

Deep neural networks have become a pervasive tool in science and engineering. However, modern deep neural networks' growing energy requirements now increasingly limit their scaling and broader use. We propose a radical alternative for implementing deep neural network models: Physical Neural Networks. We introduce a hybrid physical-digital algorithm called Physics-Aware Training to efficiently train sequences of controllable physical systems to act as deep neural networks. This method automatically trains the functionality of any sequence of real physical systems, directly, using backpropagation, the same technique used for modern deep neural networks. To illustrate their generality, we demonstrate physical neural networks with three diverse physical systems-optical, mechanical, and electrical. Physical neural networks may facilitate unconventional machine learning hardware that is orders of magnitude faster and more energy efficient than conventional electronic processors.

I. INTRODUCTION

Growing DNN energy requirements are motivating alternatives to conventional electronic hardware. This paper proposes directly training arbitrary physical systems as deep neural networks, addressing limitations in existing physical-learning approaches.

Motivation: DNN computational requirements have grown rapidly, increasingly limiting models through hardware energy efficiency.The passage links expanding model size to computational demands that outpace Moore’s Law.
Motivation: Existing physical hardware proposals often approximate DNN mathematics, requiring engineering to suppress hardware physics that violates the analogy.This dependence can make performance sensitive to how closely devices match idealized operations.
Physical computation: Controlled physical evolutions naturally provide hierarchical, nonlinear, and matrix-vector-like transformations relevant to deep learning.The paper identifies structural similarities between DNNs and physical processes, including hierarchy and nonlinearity.
Related work: Reservoir computing uses physical transformations with a trainable output layer, but its training remains inherently shallow.Unlike approaches that train physical transformations themselves, reservoir computing does not learn hierarchical processes.
Research gap: Few experimental physical-learning studies existed, and those studies relied on gradient-free algorithms despite the importance of backpropagation for efficient training and generalization.Proposals for physical backpropagation often impose restrictive assumptions or train only in numerical simulations.
Contribution: The paper introduces physics-aware training to directly train arbitrary real physical systems as deep neural networks across mechanical, optical, and electrical examples.The demonstrated systems are a driven metal plate, a nonlinear transistor oscillator, and ultrafast optical second harmonic generation.

II. RESULTS

The paper trains multilayer physical neural networks directly with Physics-Aware Training, combining physical forward passes with digital gradient estimation. Optical, mechanical, and electronic systems perform classification, while PAT succeeds where in silico training fails.

Physics-Aware Training: Physics-Aware Training applies backpropagation and stochastic gradient descent directly to sequences of physical input-output transformations.The physical system computes forward passes, while a differentiable digital model estimates gradients for parameter updates.
Optical PNN: The SHG-PNN cascades five trainable physical layers, using encoded pulse spectra and seven final spectral bins to classify vowels.It uses approximately 250 physical parameters and 10 digital parameters; the largest-sum spectral bin determines the vowel class.
Physics-Aware Training: PAT trains physical systems despite imperfect digital models, noise, and other hardware imperfections by grounding optimization in measured physical outputs.The actual forward pass preserves the true loss and intermediate-layer inputs during training.
Training comparison: In silico training reached only ∼40% vowel-classification accuracy, whereas PAT successfully trained the SHG-PNN even as physical layers were added.The failure of in silico training persisted despite an accurate digital SHG and noise model.
Diverse physical systems: 87% test accuracy was achieved by the mechanical PNN, while the hybrid optical PNN improved digital-only accuracy from roughly 90% to 97%.The mechanical experiment showed that replacing physical transformations with identity operations reduced performance to random guessing, while the optical system offloaded computation to physical transformations.
Diverse physical systems: 93% test accuracy was achieved by the electronic PNN, whose transistor circuit produces a noisy, highly nonlinear transient response.Its final prediction averages seven independent three-layer PNNs.

III. DISCUSSION AND CONCLUSION

The discussion frames PNNs as proof-of-concept demonstrations across mechanics, electronics, and optics, while identifying open questions about practical speed and energy benefits. Simulation analysis suggests substantial self-simulation advantages, but reaching their ceiling requires co-designing physics and algorithms.

Discussion and conclusion: The experiments leave open which physical systems are best suited to PNNs and how much they can improve machine-learning speed or energy consumption.These practical questions remain beyond the proof-of-concept demonstrations.
Discussion and conclusion: The figure compares mechanical, electrical, and optical PNNs on handwritten-digit classification, with final test accuracies of 87%, 93%, and 97%, respectively.The architectures differ across the three physical systems.
Discussion and conclusion: Realistic device implementations can achieve self-simulation advantages exceeding 10^6 for speed and 10^9 for energy, with larger values possible through size-scaling.Simulation analysis also identifies physical systems that approximate controllable matrix-vector multiplication and related machine-learning operations.
Discussion and conclusion: Reaching the performance ceiling implied by self-simulation requires co-design of physical dynamics and learning algorithms.Candidate systems include multimode optical waves and networks of coupled transistors, lasers, or nano-oscillators.

DATA AND CODE AVAILABILITY

The paper provides expandable demonstration code for applying Physics-Aware Training to physical neural networks.

Data and code availability: An expandable demonstration codebase for applying Physics-Aware Training to physical neural networks is available on GitHub.The passage also states that all data generated and code used for the work are available.

Deep physical neural networks enabled by a backpropagation algorithm for arbitrary

The supplied passages identify the paper as a physical-neural-network study and provide its arXiv version and date.

Paper identification: The paper is identified as arXiv:2104.13386v1, posted on 27 Apr 2021.The title passage is truncated in the supplied material.

1. PHYSICS-AWARE TRAINING

This section introduces physics-aware training (PAT) and sets out its high-level rationale for training physical neural networks. It previews a numerical comparison of training algorithms on a simulated feedforward PNN.

1. PHYSICS-AWARE TRAINING: PAT is introduced as a hybrid physical-digital algorithm for efficiently training physical neural networks.The section presents the algorithm before developing its mathematical formulation and applying it to a simulated vowel-classification task.
1. PHYSICS-AWARE TRAINING: A simulated feedforward PNN is trained on vowel classification using the different training algorithms introduced in this section.
1. PHYSICS-AWARE TRAINING: The section first gives intuition for why PAT works and why alternative approaches fail, with rigorous support developed later.

A. Intuition for why physics-aware training works

PAT addresses the difficulty of applying efficient gradient-based training to real physical operations. It mitigates simulation-reality errors that accumulate across network depth and repeated training steps by grounding computation in the physical system.

A. Intuition for why physics-aware training works: Backpropagation efficiently trains parameterized operation sequences, but direct analytic differentiation is unavailable for real physical systems.Finite-difference estimation would require n physical-system uses per backward pass, where n can reach 10^6 or 10^9.
A. Intuition for why physics-aware training works: PAT solves the problem of applying backpropagation to sequences of real physical operations for desired physical functions.
A. Intuition for why physics-aware training works: Simulation-reality errors accumulate through layers because each physical layer receives the previous layer’s output as input.Even a small model mismatch can become increasingly consequential as a hierarchical process grows deeper.
A. Intuition for why physics-aware training works: Simulation-reality errors also accumulate across training steps, making even highly accurate models insufficient for training real physical systems.The authors identify this training-step problem as more significant than error accumulation through layers because training involves many more sequential steps.
A. Intuition for why physics-aware training works: PAT mitigates depth and training-step errors by using true physical transformations and more accurate parameter-update estimates.Its gradients use correct physical inputs, and training remains grounded in the true loss rather than relying solely on simulations.
A. Intuition for why physics-aware training works: PAT also supports training with forward-pass noise, device-device variation, and models that omit parts of the true physics.

B. General formulation of physics-aware training

PAT combines physical forward computation with differentiable digital backward computation to train arbitrary physical transformations using backpropagation. The formulation extends conventional autodiff to feedforward physical neural networks whose layers pass outputs forward and trainable parameters.

B. General formulation of physics-aware training: PAT computes loss gradients with respect to network parameters and uses them to update the parameters through backpropagation.
B. General formulation of physics-aware training: Conventional backpropagation uses reverse-mode autodiff to compute a loss in the forward pass and gradients in the backward pass.
B. General formulation of physics-aware training: In PAT, the physical transformation fp is used in the forward pass, while a differentiable digital model fm estimates the backward pass.This custom autodiff design makes PAT suitable for arbitrary real physical input-output transformations and compatible with conventional machine-learning libraries.
B. General formulation of physics-aware training: Unlike conventional backpropagation, PAT approximates gradient propagation because its backward-pass function differs from the physical forward-pass function.
B. General formulation of physics-aware training: PAT can be implemented as custom PyTorch autograd functions, allowing users to build and train physical neural networks with complex architectures.The implementation separates the physical forward function from the differentiable backward function and saves forward-pass inputs and parameters for backward computation.
B. General formulation of physics-aware training: A feedforward PNN passes each physical system’s output to the next system and supplies each layer with a trainable parameter θ[l].The layer relation is x[l+1] = y[l].
B. General formulation of physics-aware training: The training loop consists of a forward pass, error computation, backward pass, and parameter update, with algorithms differing in whether fp or fm is used.For in silico training, fm is used in both passes; PAT instead uses physical outputs saved during the forward pass when evaluating Jacobians.

C. Motivating physics-aware training with multilayer feedforward architecture

The section develops backpropagation for multilayer physical neural networks and motivates physics-aware training as a practical alternative to ideal and in-silico training. PAT combines physical forward computation with differentiable digital modeling during the backward pass.

Feedforward PNN: A feedforward PNN passes each layer’s physical output to the next layer, with the final physical output serving as the prediction.The architecture consists of N physical systems connected sequentially.
Backpropagation: Backpropagation efficiently computes parameter gradients from the final layer backward, avoiding redundant chain-rule calculations for each layer.The procedure begins with the output error vector and iteratively propagates gradients through preceding layers.
Training limitations: Ideal backpropagation is impractical because physical transformations lack analytic gradients, requiring repeated physical evaluations for finite-difference approximations.Each backward call requires repeated calls to the physical system.
Training limitations: In-silico training uses a differentiable digital model for both forward and backward computation, but model mismatch can produce inaccurate error vectors and gradients, especially in deep PNNs.The mismatch between predicted and physical outputs increases with depth, while gradients are evaluated at increasingly inaccurate layer inputs.
Physics-aware training: Physics-aware training performs the forward pass on the physical system and uses the differentiable digital model only for the backward pass.This hybrid arrangement reduces dependence on exceptionally accurate digital models while avoiding the repeated physical calls required by finite differences.

D. Numerical example

A simulated vowel-classification PNN compares ideal backpropagation, in-silico training, and PAT. PAT follows ideal backpropagation closely and reaches about 96% test accuracy, whereas accumulated simulation–reality mismatch causes in-silico training to diverge.

Numerical example: The numerical example trains a multilayer feedforward PNN for vowel classification using a differentiable model of a hypothetical physical transformation.The transformation is an autocorrelation-based approximation of broadband optical second-harmonic generation.
Task and architecture: The vowel task uses 12-dimensional inputs and 7 output classes, with preprocessing and postprocessing adapted to a physical function having 24-dimensional inputs and outputs.The input vector is repeated to form 24 dimensions, and selected final outputs are summed into 7 class scores.
Training results: By about ∼350 epochs, in-silico training no longer trains the physical system effectively because differences between the physical and digital transformations increase.After approximately 2000 epochs, PAT’s loss and accuracy fluctuate, which the authors associate with minor Jacobian differences near an optimal region.
Training results: PAT initially follows ideal backpropagation closely and consistently lowers the loss until reaching approximately 96% test classification accuracy.The reported accuracy is close to that achieved by ideal backpropagation.
Gradient comparison: PAT estimates gradients at the correct physical layer inputs, unlike in-silico training, whose layerwise gradient locations increasingly mismatch reality with depth.The single-step comparison shows that in-silico gradients differ from ideal gradients in both direction and magnitude.
Training trajectories: After 800 epochs, in-silico-trained parameters classify 2 of 3 vowels incorrectly, while PAT and ideal backpropagation classify all 3 accurately with high confidence.At 300 epochs, all three methods show some learning, but the later simulation–reality mismatch causes in-silico training to diverge.

1. Experimental setup

The experiments implement PNNs with optical second-harmonic generation and nonlinear analog electronics, using controllable input waveforms and high-throughput measurement. Characterization shows strongly nonlinear, coupled physical transformations suitable for computation.

Optical setup: The optical PNN uses a pulse shaper and a BBO crystal to transform amplitude-modulated near-infrared pulses through femtosecond second-harmonic generation.The setup converts approximately 780–800 nm inputs into blue or ultraviolet outputs near 400 nm.
Optical setup: The optical setup operated at a sustained 200 Hz for months, supporting the repeated physical evaluations required for PNN training.The system was designed for high-throughput operation and was operated reliably, including remotely during the COVID-19 pandemic.
Optical characterization: Adding two narrowband optical inputs produces a new central-wavelength output rather than the sum of the separate outputs, demonstrating SHG nonlinearity.The individual outputs are sharply peaked at half the input wavelength, while their combined input generates an additional output component.
Optical characterization: Varying a small constant section of a complex optical waveform changes the output spectrum across a wide region, with strongest nonlinearity near amplitude A ≈1.The experiment compares input waveforms, resulting spectra, and output intensity at a selected wavelength.
Electronic setup: The electronic PNN uses standard bulk components, a breadboard, and a high-speed data-acquisition device for analog input and output.The circuit used for MNIST classification was designed to be simple while retaining nonlinear behavior.
Electronic characterization: The analog circuit exhibits nonlinear responses and coupling between output time bins when a constant section of the input waveform is varied.At the highest amplitudes, additional nonlinearity from DAQ limiting appears and is retained in the operating regime.

1. Experimental setup

The oscillating-plate PNN converts digitally encoded time-series inputs into microphone-recorded outputs and uses the plate’s transient dynamics as a controllable physical transformation. Its response is predominantly linear, with memory that implements a convolution over preceding inputs, while PAT uses differentiable digital models and physical forward passes for training.

Hardware and information flow: The setup amplifies computer-generated control signals to drive a speaker-mounted titanium plate, while a microphone records the resulting sound for return to the computer.The plate is driven through the speaker’s voice coil, and the microphone provides the measured output.
Input and output encoding: A 48-dimensional digital input is transformed by the oscillating plate and digitized into a 24-dimensional output signal.The plotted amplitudes are arbitrary normalized units.
Physical transformation: The plate’s output response is overwhelmingly linear with respect to input voltage, despite the initial aim of producing high-amplitude nonlinear oscillations.The setup could not reach the amplitudes required for strongly nonlinear oscillations.
Physical transformation: At 192 kHz, plate transients persist for about 2 ms, so each output convolves approximately N = 384 previous inputs.At a slower 5 kHz input rate, the corresponding memory would cover approximately N = 10 previous inputs.
Training procedure: PAT relies on differentiable digital models for backward-pass gradient estimation while executing the forward pass on the actual physical system.The models include physical-system noise, but simulations with noise still train the actual system less effectively than PAT.

E. Descriptions of PNN architectures

The PNN architectures combine trainable digital rescaling and pooling with physical transformations, using repeated SHG layers for vowel and MNIST classification. Their designs include explicit physical inputs and parameters, intermediate renormalization, and comparisons against identity-operation baselines.

Common architecture and training: All PNNs use a common training procedure with the Adadelta optimizer, optional learning-rate scheduling, range-constraining Lagrangian terms, and identity-operation baseline models.The baselines replace physical input-output transformations with identity operations to test whether digital operations alone provide most of the functionality.
SHG physical transformation: The SHG input-output transformation maps 100 physical inputs, split between data and trainable parameters, to a 50-dimensional output spectrum.The design allocates distinct pulse-shaper spectral modulations to input data and trainable parameters.
SHG vowel PNN: The SHG vowel model rescales normalized 12-dimensional vowel formant vectors, repeats them to form 48-dimensional inputs, and concatenates trainable parameters before each physical transformation.Each SHG layer produces a physical output that is renormalized and digitally rescaled before being passed onward.
SHG vowel PNN: The SHG vowel architecture repeats physical computation and renormalization through five SHG devices, with each device receiving the preceding output together with trainable parameters.The output of each SHG device is shown as an intermediate quantity in the architecture figures.
Training comparison: For simple PNN architectures with accurate linear digital twins, in silico training can achieve high MNIST accuracy, while PAT still produces gains after a few transfer-learning epochs.The in-silico-to-experiment performance gap is usually only a few percentage points for these linear systems.
SHG MNIST PNN: The hybrid SHG MNIST model improves accuracy by approximately 7% over replacing SHG transformations with identity transformations.The paper notes that this task becomes substantially harder near 96% accuracy than near 90%.

3. SIMULATION ANALYSIS OF CANDIDATE PNN SYSTEMS

The simulation analysis evaluates candidate physical systems by their mathematical operations, self-simulation advantages, and scaling with physical parameters. It finds that complex nonlinear, broadly coupled systems can offer especially large potential advantages, while tensor operations present an underused opportunity for physical learning hardware.

Analysis goals: The analysis identifies candidate-system operations, estimates self-simulation advantages as upper bounds on computational benefits, and studies how those advantages scale with physical parameters.These are the three stated purposes of the simulation analysis.
Candidate systems: Physical systems can implement controllable matrix-vector operations, convolutions, nonlinearities, and tensor contractions despite not necessarily supporting universal computation.Their suitability depends on whether these operations can be controlled for machine-learning models.
Scaling with physical complexity: Higher self-simulation advantages generally occur in systems with higher-order nonlinearities and heterogeneous coupling among many degrees of freedom.The analysis specifically associates greater complexity with nonlinear interactions and broad coupling across degrees of freedom.
Potential computational advantage: Self-simulation advantages exceeding 10^6 are common among candidate systems, although their practical value depends on model design, task relevance, and device implementation.The paper compares this scale with the historical increase in supercomputer speed from roughly 10^11 to 10^17.5 GFLOPs.
General dynamical model: For coupled nonlinear dynamics, simulation complexity is dominated by the highest-order nonlinear term and depends on bandwidth, simulated duration, mode count, and coupling range.The physical system’s power is usually linear in mode count, while its evolution time is the simulated duration.
Physical computations: Adjustable physical coefficients can realize matrix-vector operations and higher-order tensor contractions, with locality and symmetry determining when these operations resemble convolutions.Controllable state variables or drive terms can serve as parameters for these transformations.
Open opportunity: High-dimensional tensor operations may provide the greatest physical advantages because their higher-order nonlinear coupling is not widely used in modern deep neural networks.The paper identifies efficient machine-learning use of such operations as an open research direction.

1. Quantitative example

The paper estimates substantial speed and energy advantages for physical computation, while emphasizing that these advantages depend on realistic device assumptions and simulation caveats.

Device assumptions: Data conversion and modulation contribute explicitly to the physical inference-cost estimate, including a 537 pJ present-day total in the cited calculation.The near-future device’s computation time is limited by 33 ps propagation, corresponding to input/output rates above 30 GHz.
Single-mode nonlinear propagation: 5 × 10^8 simulation operations are required for the single-mode nonlinear optical example.The estimate follows a Fourier split-step calculation using the assumed device length, spatial step, bandwidth, and pulse duration.
Single-mode nonlinear propagation: 10^5 times more energy and 10^5 times longer are required on a Titan RTX than in the present-day experiment for the single-mode example.The simulation costs 5 mJ and takes 16 µs on the Nvidia Titan RTX.
Caveats: The self-simulation advantage for single-mode nonlinear optics is only marginally superlinear in the number of optical degrees of freedom, Nω.The logarithmic factor is weak, and practical usefulness of the resulting exotic operations remains uncertain.
Multimode wave propagation: 6 W supports approximately 8 × 10^17 operations per second in a multimode-fiber example, corresponding to about 10 aJ per operation.The estimate uses 5 × 10^6 modes, a 30 kHz micromirror device, and a 1 Mpixel camera.
Multimode nonlinear propagation: 5 × 10^-22 J/operation is estimated for multimode nonlinear propagation, making the physics 10^10–10^11 more energy efficient than digital simulation.The effective operation rate is 2 × 10^23 operations/s, limited by a 31 kHz DMD update rate.

D. Multimode nonlinear oscillators/oscillator networks

Coupled nonlinear oscillators can implement computations resembling deep residual networks, with modeled self-simulation advantages that grow as the network is scaled, subject to coupling and spatial-interaction constraints.

Neural-network correspondence: Euler integration of the coupled-oscillator dynamics produces the layer-by-layer update of a deep residual neural network.The correspondence assumes local oscillator nonlinearities and controllable linear coupling.
Simulation estimate: 2 × 10^5 operations are required to simulate 100 oscillators with coupling range M = 100 for 1 ns at 20 GHz bandwidth.The estimate assumes a 10 GHz resonance and one operation per oscillator per step for the local nonlinearity.
Scaled network: The larger oscillator network requires 2 × 10^8 simulation operations and has estimated self-simulation advantages of 10^6–10^7 for energy and 70–6000 for speed.This example uses N = 10^5 coupled oscillators, with 100 input and 100 output oscillators.
Scaling: Increasing N while holding N∆T constant increases the speed advantage linearly, but exploitable spatial scaling depends on interaction times and required transformations.The minimum useful ∆T and the need for distant oscillators to interact constrain this scaling.
Caveats: Neglecting power required to maintain controllable coupling can overstate the energy advantage when connections require active power.The effect may become non-negligible because of the enormous number of connections.

E. Nonlinear electrical circuit dynamics/analog transistor networks

Analog transistor networks can perform operations resembling neural-network layers and may offer large self-simulation advantages, although estimates depend on simulation abstractions and low-energy controllable coupling.

Neural-network correspondence: Controllable linear coupling and relatively local nonlinearities make transistor networks resemble typical neural networks.The paper notes that higher-order physical effects may provide additional computational benefit, but their magnitude is difficult to estimate.
Physical and conversion costs: 1 pJ is estimated for transistor energy, versus 10 nJ for DAC and ADC operations in the 100-transistor example.The paper retains the transistor-power term for a conservative order-of-magnitude estimate despite its smaller size.
Simulation estimate: 10^7 operations are estimated to simulate a 100-transistor network over 10 ns with 20 GHz maximum frequency and coupling range M = 100.The estimate uses 10 equivalent-circuit dimensions per transistor and includes linear and nonlinear simulation terms.
Estimated advantage: A small analog transistor network has a self-simulation advantage of approximately 10^4–10^5 for energy and 0.4–30 for speed.The corresponding digital simulation costs 0.1 mJ and takes 310 ns on the GTX GPU, versus 0.4 mJ and 3.6 ns on the DGX SuperPOD.
Scaling: Increasing the transistor count to 10^3 while holding total evolution time proportional can produce speed advantages of 310 over the GTX GPU and 4 over the DGX SuperPOD.The example holds NT∆T constant, keeping total physical energy unchanged while shortening evolution time.
Caveats: Passive or nearly passive connectivity is important because active connection power could reduce predicted energy advantages by up to M^1.The analysis also neglects details of realizing controllable linear coupling between transistors.

4. SUPPLEMENTARY DISCUSSION: WHEN CAN PNNS PROVIDE TECHNOLOGICAL BENEFITS?

PNN benefits depend on making the physical system perform most computational work while minimizing digital read-in, read-out, and overhead. The discussion identifies task structure, physical feed-forward, and physical-domain interfaces as key conditions for practical gains.

Operational bottleneck: PNNs provide computational benefits when most operations occur within the physical system rather than during digital read-in, read-out, or overhead.These external operations add time and energy at digital-computer rates, limiting achievable speed-ups and energy gains.
Operational bottleneck: 1960 total equivalent operations versus 196 read-in, 10 read-out, and 196 digital operations gives MNIST a maximum possible gain of approximately 5×.The example concludes that this simple task is far from operationally bottlenecked and offers little benefit from a physical co-processor.
Operational bottleneck: 1–100 billion equivalent operations for high-accuracy CIFAR10 classification yield a maximum possible PNN advantage of approximately 10^5.The larger computational workload can dwarf read-in and read-out operations, making physical execution more consequential.
Architectural opportunities: A PNN can use a complex, repeatedly applied physical feature map with relatively few trainable parameters, while PAT trains sequences of such maps with backpropagation.This distinguishes PAT from reservoir computing, whose physical feature map is essentially untrained and cannot be backpropagated through.
Architectural opportunities: Physical feed-forward can remove digital intermediaries between PNN layers, while physical-domain inputs can avoid read-in costs and enable trained smart sensors.Physical-domain processing may support higher-speed, higher-resolution, or lower-energy measurements.
Architectural opportunities: Physical-output PNNs produce a desired physical quantity or functionality, with digital measurements used to assess the state during physics-aware training.Examples include output pulses, chemical effects, nanostructure functionality, and small-robot behavior.