PaperlayerView exact revision on arXiv

Source-linked AI summary

Photonic reservoir computing with dimensionally compressed readout

Gerald Kobi, Mohab Abdalla, Miguel C. Soriano, Damien Rontani

arXiv:2609.07418v1cs.NEphysics.optics

TL;DR

Physical readout constraints limit access to the high-dimensional states of photonic reservoir computers. This work studies random projection as a way to compare compressed large reservoirs with natively small ones, finding differences in their capacity distributions and better NARMA-10 performance within a particular compression range.

  • Problem

    Physical photonic readouts face detection constraints and are limited to a few tens of states, while the comparison between compressed large reservoirs and natively small reservoirs remains largely unexplored.

    Source: §I · p. 1

  • Method

    The study applies random projection to reduce the readout of a high-dimensional time-delay reservoir and compares it with a standalone reservoir of the same observable size.

    Source: §I · p. 2, B. Projection method presentation · p. 3

  • Results

    At equal observable counts, projected reservoirs can have different capacity distributions and perform much better than unprojected cases on NARMA-10 within a particular compression-ratio range.

    Source: D. Further discussion · p. 10, §IV · p. 11

  • Takeaways & Limitations

    For a fixed output-layer size, processing capabilities depend on the dimension of the reservoir’s high-dimensional state space, and projection provides a lower-dimensional view of that source space.

    Source: §IV · pp. 10–11

    Total processing capacity remains bounded by output-layer size, so gaining linear processing capacity inevitably trades off against nonlinear processing capacity.

    Source: §IV · p. 11

Abstract

from arXiv · show

This work addresses a hardware constraint in reservoir computing: the limited size of the readout layer imposed by systems with a physical readout. We investigate a strategy to accommodate this constraint based on random projection, which compresses high-dimensional reservoir states into a lower-dimensional subspace while preserving key properties of the source space and information- processing capabilities. To evaluate this approach, we compare a small, standalone time delay reservoir against a larger configuration whose output is projected down to match the same restricted readout dimension. Using task-independent metrics, we demonstrate that the distribution of information-processing capacities may differ between the two configurations, even at identical readout sizes. Furthermore, we perform a comprehensive hyperparameter scan to assess how both systems behave under varying physical regimes. Finally, we benchmark this approach on the standard NARMA10 task, showing that the random projection framework can yield superior performance compared to a standalone constrained reservoir, within specific compression range. These results provide a scalable pathway to bypass physical readout bottlenecks in hardware-based reservoir computing.

I. INTRODUCTION

Physical readouts in photonic reservoir computers are constrained in how many reservoir states they can access, leaving limited output layers a practical challenge. The study examines whether random projection and task-independent metrics can help characterize performance under that constraint.

I. INTRODUCTION

  • Integrated photonic readouts face detection constraints, and current implementations are restricted to a few tens of readout states.This limits access to the reservoir’s high-dimensional state space.Source: §I · p. 1
  • The work compares a large reservoir with a compressed readout against a natively small reservoir, investigating how random projection affects time-delay reservoir performance.The comparison addresses a gap the authors describe as largely unexplored.Source: §I · pp. 1–2

II. SCOPE OF THE STUDY: METHODOLOGY AND METRICS

  • The study evaluates processing capacity with task-independent metrics across physical hyperparameters, then examines performance on a nonlinear regression task.Its methodology centers on optoelectronic Ikeda-model reservoirs, dimensionality reduction, and task-independent metrics.Source: §I · p. 2, §II · p. 2

A. Ikeda system

The study uses an Ikeda-based time-delay reservoir and adds a dimensionality-reduction stage between the reservoir states and the final readout.

A. Ikeda system

  • The Ikeda reservoir’s dynamics are governed by a delay differential equation with a sin^2 nonlinearity driven by an external input.The default settings are Tr = 0.4θ, β = 1.6, γ = 0.01, ϕ0 = 0.2, and τ = Tin × 0.9.Source: A. Ikeda system · p. 2
  • The input is masked across N virtual nodes, with the mask updating every θ = Tin/N.The input term is m(t) = u(t) · mask(t), where u(t) is the raw input signal.Source: A. Ikeda system · p. 2
  • A dimensionality-reduction layer is inserted between the reservoir output and final output, producing a compressed readout for the optimized target.The figure contrasts collection of the full state vector with collection of a lower-dimensional projected state.Source: A. Ikeda system · p. 2, Figure 1 · p. 2

B. Projection method presentation

The projection method maps a high-dimensional reservoir state matrix to fewer observables using a random linear projection, with the study’s notation distinguishing source size from readout size.

B. Projection method presentation

  • Random projection applies a linear map grounded in the Johnson–Lindenstrauss lemma to reduce dimensionality while preserving pairwise distances.The method is presented through a matrix-based application associated with Achlioptas.Source: B. Projection method presentation · p. 2
  • The projected observables are linear combinations of reservoir states, and a highly sparse projection matrix can sample states with minimal information loss.The text links this property to constrained readouts.Source: B. Projection method presentation · p. 3
  • The notation (N, O) denotes a reservoir with N source neurons projected to O observables, where O ≤ N; (N) denotes the unprojected case with O = N.The figure also distinguishes source size from output dimension in projected configurations.Source: B. Projection method presentation · p. 3, Figure 2 · p. 3

C. Metrics used to compare the different cases

The study uses memory capacity and information processing capacity to compare systems independently of a specific task, with the latter separating linear and nonlinear processing by degree.

C. Metrics used to compare the different cases

  • Memory Capacity (MC) measures how well a reservoir reconstructs past inputs from its current state.The study sums capacities over tasks that reconstruct inputs at different delays; its MC configuration uses 20,000 training samples, a maximum delay of 300, and threshold 10^-2.Source: C. Metrics used to compare the… · p. 3
  • Information Processing Capacity (IPC) evaluates reconstruction of linear and nonlinear functions of past inputs, using orthogonal Legendre-polynomial targets.Capacities are decomposed by total degree, and first-degree capacity C1 corresponds to linear MC.Source: C. Metrics used to compare the… · p. 3
  • For IPC, training samples increase with reservoir size: 2×10^6 for Nx = 20, 3×10^6 for 20 < Nx ≤ 50, and 5×10^6 for Nx > 50.The passage gives these as the specific IPC training configurations.Source: C. Metrics used to compare the… · p. 3

Linear and non-linear memory evaluation

The section compares native and randomly projected reservoirs at matched readout sizes, finding that projection can raise linear memory capacity while redistributing linear and nonlinear capacity. Capacity distributions differ most under strong compression and converge as projected readouts approach their source dimension.

Linear and non-linear memory evaluation

  • MC increased sub-linearly with observable count, and the (200) and (1000) sources had MC values of 15.91 and 24.22, respectively.The reported scaling exponents follow α_unprojected < α_N=200 < α_N=1000 < 1.Source: Linear and non-linear memory evaluation · p. 4
  • Within the tested range, projected configurations maintained equal or higher MC than native (O) reservoirs at the same observable count.The projected configurations never exceeded the MC of their source reservoir.Source: Linear and non-linear memory evaluation · p. 4, Figure 3 · p. 5
  • At strong compression, projected IPC distributions differed from both the (200) baseline and other (200, O) configurations, then tended to converge as O increased.Higher-degree capacities emerged sequentially as observable count increased.Source: Linear and non-linear memory evaluation · p. 4, Figure 3 · p. 5
  • Projection did not create capacities absent from the source reservoir or exceed its capacity, so projected degree capacities remained bounded by the source.The source’s available capacity limits what its projected readout can retain.Source: Linear and non-linear memory evaluation · pp. 4–5
  • For full-capacity (200, O) projections, linear and nonlinear capacity differences had equal absolute values, indicating a trade-off that favored linear memory at smaller O.The normalized difference was largest for small observable counts.Source: Linear and non-linear memory evaluation · pp. 4–5
  • As O approached 200, projected and native configurations converged to the same IPC distribution and their MC and NLC differences vanished.The section attributes this convergence to reduced JL constraints and closer preservation of source geometry.Source: Linear and non-linear memory evaluation · p. 5

B. Physical hyperparameter tuning

The section examines how physical time scales and input scaling factor γ relate to the processing capacities of native and projected configurations. Figure 4 compares memory capacity under time-scale mismatch and across response times.

B. Physical hyperparameter tuning

  • The analysis covers time-scale hyperparameters and varies input scaling factor γ to examine linear and nonlinear processing capacities.The time-scale analysis precedes the γ investigation.Source: B. Physical hyperparameter tuning · p. 5
  • Figure 4(a) reports memory capacity (MC) for different mismatches between τ and Tin, comparing the three cases listed as (200), (20), and (200, 20).Source: Figure 4 · p. 6
  • Figure 4(b) compares ΔMC between (20) and (200, 20) as response time Tr varies, with absolute capacities shown on the right axis.Source: Figure 4 · p. 6

1. Time & Synchronization

Projection can increase memory capacity at a fixed readout size, but the gain depends on synchronization and response-time settings. Poorly conditioned source dynamics are not repaired by projection.

1. Time & Synchronization

  • At Tin = 0.9τ, (200, 20) gains up to 3 MC units over (20), while synchronous operation gives the lowest MC in both configurations.Across the explored hyperparameter space, asynchronous operation augments MC.Source: §1 · p. 6
  • The reported MC comparison across Tin/τ includes (20), (200), and (200, 20), with the projected configuration following the memory trend of (200).The system response time Tr governs internal dynamics, while θ/Tr sets coupling between successive virtual neurons.Source: §1 · p. 6
  • At Tr ≈ 5θ, (200, 20) exceeds (20) by 2.5 MC units; the gain vanishes for Tr ≪ θ and Tr ≫ θ.In the short-response limit both configurations have their highest absolute MC; in the long-response limit performance decays toward zero.Source: §1 · pp. 6–7
  • Projection cannot compensate when the source reservoir is already performance-limited by an improper choice of Tr or Tin.In that case, projection propagates a poorly conditioned state space.Source: §1 · p. 7

2. Impact of the input scaling factor

The input scaling factor changes both the size of the memory advantage from projection and the allocation of information-processing capacity across degrees. The projected system’s lower-degree capacity gains are most evident at low and intermediate input scalings.

2. Impact of the input scaling factor

  • At low γ, (200, O) has a substantial MC surplus over (O), but the difference declines monotonically and converges toward the low-dimensional profile as γ approaches unity.The authors relate the gain to available MC in (200), which is depleted at high input scaling.Source: §2 · p. 7
  • Across configurations, gains in MC are offset by losses in NLC because both projected and low-dimensional systems reach full IPC; the difference is indistinguishable at γ > 0.5.The paper describes this as a conservation-of-total-capacity trade-off.Source: §2 · pp. 7–8
  • At low input gains, (200, 20) enhances first- and second-degree IPC, while (20) has a higher proportion of third-degree components.As input scaling increases, the trade-off diminishes and higher-degree components emerge more rapidly in (200, 20).Source: §2 · p. 8
  • At small and intermediate input scalings, projection increases lower-degree IPC seemingly at the expense of higher-order capacities, making the source reservoir’s IPC profile a design target.The paper argues that tuning this profile can help maximize task-relevant information for a fixed number of observables.Source: §2 · p. 8

C. Effect on a real task: NARMA performance

On NARMA10, projected reservoirs outperform standalone reservoirs with the same number of observables, especially under high compression, although the gap narrows at larger readouts. The advantage depends on capacity allocation: larger sources can improve faster initially, while the 200-state source retains nonlinear degrees that can support stronger performance at some readout sizes.

C. Effect on a real task: NARMA performance

  • (200, O) consistently achieves lower NARMA10 NRMSE than (O), especially under high compression, and the gap narrows at sufficiently large observable counts.The comparison averages 60 independent trials, with shaded regions showing standard deviation.Source: C. Effect on a real task: NARMA… · p. 8, Figure 6 · p. 9
  • NARMA10 requires at least 10 MC and depends on second-degree IPC; projected cases reach these first two capacity thresholds with fewer observables than (O).The paper suggests this efficiency may explain the advantage in the low-observable regime; increasing linear capacity beyond a threshold does not alone drive further improvement.Source: C. Effect on a real task: NARMA… · p. 9
  • At 40 observables, (200, O) achieves superior performance to the larger-source projections, despite crossing NRMSE = 0.4 at a higher observable count.The (1000, O) cases reach 0.4 more rapidly, while the (1000, O) and (400, O) curves plateau and (200, O) continues improving.Source: C. Effect on a real task: NARMA… · p. 9, Figure 6 · p. 9
  • For 30 observables, (200, 30) tracks the target’s rapid fluctuations and peak amplitudes more closely than (30).The standalone (30) trace has attenuated amplitude and misses prominent variations.Source: Figure 6 · p. 9, C. Effect on a real task: NARMA… · p. 10

D. Further discussion

The discussion connects compressed-readout capacities to the source reservoir’s dimension and parameter regime, and describes projection as redistributing capacities across degrees. It also notes that the geometric argument does not explain the preference for lower-degree capacities.

D. Further discussion

  • Compression can preserve or improve processing capabilities with fewer observables, but gains and capacity redistribution depend on source dimension and parameters.The discussion compares reservoirs with equal readout sizes and considers how the larger source reservoir may provide more favorable state diversity.Source: D. Further discussion · p. 10
  • Projected degree-d capacities are fractions of source-reservoir capacities, scaled by a multiplicative factor ζ_d between 0 and 1.Source: D. Further discussion · p. 10
  • At high compression ratios, lower-degree capacities survive projection more than higher-degree capacities because ζ_d diminishes faster for higher degrees.Source: D. Further discussion · p. 10
  • High-dimensional quasi-orthogonality is presented as supporting linear separability, but it does not explain projection’s preference for lower-degree capacities.Source: D. Further discussion · p. 10

IV. CONCLUSION

With a fixed output-layer size, performance depends on the dimension and information of the source state space; random projection offers a lower-dimensional view without creating computational resources. On NARMA-10, projected cases perform much better than unprojected cases at the same observable count within a particular compression-ratio range, subject to a linear/nonlinear capacity trade-off.

IV. CONCLUSION

  • With a fixed output-layer size, performance can be leveraged by operating in a larger state space and applying dimension reduction, whose quality depends on source-reservoir information.Random projection provides an alternative lower-dimensional view; it does not create additional computational resources.Source: §IV · pp. 10–11
  • Total processing capacity remains bounded by output-layer size, so gaining linear capacity entails losing nonlinear capacity.Source: §IV · p. 11
  • On NARMA-10, projected cases perform much better than unprojected cases at the same observable count within a particular compression-ratio range.Source: §IV · p. 11
Loading 2609.07418v1…