PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal Connectivity of Random Boolean Networks for Reservoir Computing

Christopher Michael Skeide Vibe, Aliaksandr Hubin, Stefano Nichele

arXiv:2610.33498v1cs.NE

TL;DR

The study asks how reservoir connectivity should vary with design and task, while separating input fan-out from recurrent connectivity. Across majority and parity recall, optimal connectivity falls as fan-out grows and rises with self-loops, with substantial variation among reservoirs of the same design.

  • Problem

    The study asks how optimal connectivity depends on reservoir design and task, including input fan-out, which can otherwise be tied to recurrent connectivity.

    Source: §1 · p. 2

  • Method

    The authors jointly vary six reservoir design factors and evaluate majority and parity recall using random Boolean networks.

    Source: §1 · p. 2, §5 · p. 11

  • Results

    Optimal connectivity falls as input fan-out grows and rises with self-loops, while hardly depending on task or other design factors.

    Source: §5 · p. 10, §6 · p. 11

  • Takeaways & Limitations

    Choose connectivity together with input fan-out and self-loops, and judge a design by the median performance of many reservoirs.

    Source: §6 · p. 11

    The study covers one reservoir size, one input distribution, and two tasks at short delays; its results apply to the explored grids.

    Source: §5 · p. 11

Abstract

from arXiv · show

Reservoir computing shifts the focus from training a system's internal dynamics to designing dynamics that expose useful computational capabilities via linear projection. We study random Boolean networks as reservoirs: cellular automata with random recurrent wiring and a random rule at each node. Their main parameter is the connectivity, the mean number of edges into a node. Without input, these networks pass from ordered to chaotic dynamics at a connectivity of 2, where their dynamics are expected to facilitate computation. How the input is connected is itself a design factor: if the input shares the network's edges, the connectivity sets both the dynamics and how many nodes receive the input, the input fan-out. We set the input fan-out separately and compare 7,560 reservoir designs, all combinations of six design factors, each built as many random reservoirs, on recalling the majority or the parity of a few input bits seen several steps earlier. The optimal connectivity falls from 2.5 to 1.75-2 as more nodes receive the input, but hardly depends on the task or on the other design factors, except self-loops, which connect a node to itself and raise the optimal connectivity. A measure computed from the reservoir states without a trained readout peaks at a connectivity that also falls with the fan-out and rises with self-loops, but does not replace training a readout. Reservoirs of one design vary widely, so a design should be judged by its typical reservoir.

1 Introduction

Random Boolean reservoirs are expected to compute well near the undriven order-to-chaos transition at connectivity 2, but input-driven reservoirs may have a different optimum. This study varies input fan-out alongside other design factors and tests whether state-based measures can estimate that optimum.

1 Introduction

  • Without input, unbiased random Boolean networks are ordered below connectivity 2 and chaotic above it, motivating the expectation that reservoirs work best near this transition.Connectivity is the mean in-degree of the network’s nodes.Source: §1 · p. 1
  • The optimal connectivity of a driven reservoir may differ from the undriven critical value because input wiring affects its dynamics and fan-out.Earlier studies report different optima, but vary only a few design factors while fixing the rest.Source: §1 · p. 2
  • The study jointly varies six design factors, including separately set input fan-out, and tests recall of past-window majority and parity.Majority tolerates some lost bits, whereas parity falls to chance if one bit is lost.Source: §1 · p. 2
  • Kernel quality and generalization rank measure state separability and similarity without a trained readout; the study tests whether their difference’s peak estimates optimal connectivity.Prior reported peaks were 2.25 in 25-node reservoirs and near 2 when summed over fan-out.Source: §1 · p. 2

2 Input Fan-Out in Boolean Reservoirs

In shared-pool wiring, input and recurrent edges come from the same source pool, so connectivity also determines input fan-out. The paper contrasts this with separate input wiring, which can alter kernel quality by fixing fan-out independently.

2 Input Fan-Out in Boolean Reservoirs

  • In the shared pool, each in-edge source is chosen from the input and reservoir nodes together, making input fan-out grow with connectivity.Each pool edge has probability 1/(|R|+1) of originating at the single input node.Source: §2 · p. 3
  • At connectivity 1, more than a third of shared-pool reservoirs receive no input, so connectivity sets both recurrent-edge count and input fan-out.Source: §2 · p. 3
  • Separating input edges from recurrent edges changes kernel quality little at low connectivity, while one-to-one input mapping at fan-out 5 raises it at every connectivity.At connectivity 1, kernel quality rises from 1.5 to 4.2 of 25.Source: §2 · p. 3

3 Method

The method separates input fan-out from recurrent connectivity, trains only linear readouts on delayed majority or parity targets, and compares a full factorial grid of random reservoirs. It also measures kernel quality and generalization rank without training a readout.

3.1 Model

  • Each input bit is copied to r distinct reservoir nodes, while recurrent connectivity is the mean in-degree; self-loops replace an in-edge and wiring may be homogeneous or heterogeneous.The main design grid uses heterogeneous wiring, with fractional mean connectivity allowed.Source: §3.1 · p. 4

3.2 Tasks and Readout

  • Samples contain 10 washout bits and 15 pattern bits; targets are majority or parity of a 3- or 5-bit window delayed 5 or 10 steps, evaluated with final-state or summed linear readouts.Each sample starts from the same reservoir initial state, and only the linear readout is trained.Source: §3.2 · pp. 4–5

3.3 Kernel Quality and Generalization Rank

  • Kernel quality is the rank of the state matrix, generalization rank the rank for streams sharing selected bits, and their signed difference is measured for each readout.The probe set contains 512 streams; the altered streams share their last five bits with a reference stream.Source: §3.3 · pp. 5–6

3.4 Experiments

  • The full factorial grid contains 7,560 designs, each built from 20 random reservoirs; design accuracy is their median, and factor comparisons follow prespecified operating-point and grid-wide analyses.The study defines task floors and compares self-loops at each combination’s optimal connectivity.Source: §3.4 · pp. 6–7

4 Results

The optimal connectivity falls as more nodes receive input, while task and most other design factors have little effect; self-loops raise it. Accuracy varies with fan-out and injection, and reservoirs with identical designs can differ substantially.

4.1 Accuracy, Input Fan-Out and Injection

  • At w=5, majority is most accurate at fan-out 64 (80%) for final-state d=5 and 16 (67%) for d=10; parity peaks at 64 (63%) for d=5, while d=10 stays at 50–57%.For summed readout, majority peaks at fan-out 16 (85–86%) and parity at 64 (61%) for d=5; parity at d=10 stays at 58–60%.Source: Figure 3 · p. 7, §4.1 · p. 8
  • Across fan-outs, override beats xor on majority by 8–17% (final-state) and 10–14% (summed); xor beats override on final-state parity at d=5 by 6% at w=5 and 1% at w=3.For summed parity, override and xor differ by at most 2%.Source: §4.1 · p. 8

4.2 The Optimal Connectivity

  • The no-readout Δ peak also falls with fan-out: its connectivity is at most 0.4% less accurate than k* on final-state readout, but is 3.1% less accurate on summed majority at fan-out 4.From fan-out 16 up, both the Δ peak and k=2 are at most 0.8% less accurate than k*.Source: §4.2 · p. 8
  • Self-loops raise k* by 0.25–0.75 in all 20 comparisons, while task and delay move it beyond its interval only when best accuracy is at most 58%.With self-loops, the Δ peak lies within 0.5 of k* in 95% of combinations.Source: §4.2 · pp. 8–9
  • In Fig. 4, majority’s optimal connectivity falls from 2.5 at fan-out 4 to 2 (final-state) and 1.75 (summed) at 512; parity matches it from fan-out 16 up.At fan-out 4, parity’s curves stay within 2% of chance from k=2 to 8, so its optimum is not informative there.Source: §4.2 · p. 8, Figure 4 · p. 9

4.3 Other Design Factors and Variation Between Reservoirs

  • At the operating point, a second tick lowers accuracy by 3–11%, self-loops lower it, and initialization changes design accuracy by at most 0.5%.With self-loops on a tenth of nodes, no median accuracy decrease exceeds 2.2%.Source: §4.3 · p. 10
  • Random wiring and rules account for 17–45% of single-reservoir accuracy variance on majority and 16–65% on parity; among the 20 reservoirs, the best exceeds the median by 6–28%.Individual reservoirs reach 100% on every task variant, including parity designs whose median is about 51%.Source: §4.3 · p. 10

5 Discussion

Optimal connectivity mainly follows input fan-out and self-loops, not task or most other design factors. The connectivity suggested by state-based measures can help estimate the optimum, but does not replace training a readout.

What moves the optimal connectivity

  • Optimal connectivity falls as input fan-out grows and rises with self-loops, while task, delay, injection, ticks, and initialization hardly move it.With a self-loop on every node, optimal connectivity rises by only 0.25–0.75, rather than by 1.Source: §5 · p. 10

Connectivity and input

  • The critical connectivity ¯k=2 is close to optimal only at fan-out 64 and above; with few input nodes, reservoirs are most accurate above ¯k=2.At fan-out 4, majority is most accurate at ¯k=2.5, while parity shows no clear optimum.Source: §5 · p. 10

Estimating the optimal connectivity

  • The ¯k at Δ’s peak tracks the optimal connectivity’s shifts with fan-out and self-loops, but neither it nor ¯k=2 replaces training a readout.On the final-state readout, Δ’s peak is the better estimate at fan-out 4 and 16; on the summed readout, both estimates are over 3% less accurate than the optimum at fan-out 4, and each is within 0.8% from fan-out 16 upward.Source: §5 · p. 11

Accuracy

  • Final-state parity at d=10 stays at 50–57% across fan-outs; explanations for accuracy differences, including the effects of xor and a second tick, were not tested.Initialization makes no difference, as expected if ten washout bits remove the initial state’s effect.Source: §5 · p. 11

Limits

  • The study covers one reservoir size and input distribution, two tasks at short delays, and intervals that capture only variation between reservoirs of the same design.Self-loops that add rather than replace an edge, and a readout of every tick, are untested; the results apply to the explored grids.Source: §5 · p. 11

6 Conclusion

In the reservoirs studied, optimal connectivity falls with input fan-out and rises with self-loops, while hardly depending on task or most other design factors. Reservoirs of a design vary widely, so typical performance and optimal connectivity should be judged across many reservoirs.

6 Conclusion

  • Optimal connectivity falls as input fan-out grows and rises with self-loops, but hardly depends on task or the other design factors.The connectivity ¯k=2 is close to optimal only at fan-out 64 and above.Source: §6 · p. 11
  • At fan-out 4, majority is most accurate at ¯k=2.5 and up to 4.5% less accurate at ¯k=2, while parity has no clear optimum.The ¯k at Δ’s peak estimates optimal connectivity on the final-state readout, but not on the summed readout.Source: §6 · p. 11
  • Both tasks are at least 9% above their floors except final-state parity at the longer delay; majority favors intermediate fan-out and override, while parity favors high fan-out.One update per input bit is better than two on every task; self-loops on many nodes lower accuracy, and initialization does not matter.Source: §6 · p. 11
  • Reservoirs of one design vary more than changes in ticks, self-loops, or initialization shift the median, so judge a design and its optimal connectivity by many reservoirs.The median, rather than an unusually strong individual reservoir, represents the design’s typical performance.Source: §6 · p. 11

A.1 Design Factors

Table 2 inventories the design factors in the grid, their values, and what each controls.

A.1 Design Factors

  • Table 2 lists the design factors used in the grid.Each factor is defined in Section 3.1.Source: Table 2 · p. 13
  • The table gives the values associated with each design factor.Source: Table 2 · p. 13
  • The table states what each design factor controls.Source: Table 2 · p. 13

A.2 Snyder Replication

The replication reproduces Snyder et al.’s kernel-quality curve, then shows how changing input wiring affects it. Fixing fan-out through one-to-one input mapping raises kernel quality at r=5 compared with the shared-pool setup.

A.2 Snyder Replication

  • KQ rises from 1.5 of 25 at ¯k=1 to 24.9 at ¯k=8 in the shared pool with heterogeneous in-degrees; with homogeneous in-degrees it rises from 2.2 to 25.0.GR rises with KQ in the replicated shared-pool setup.Source: §A.2 · p. 13
  • With one-to-one input mapping at r=5, KQ is 4.2, 18.6, and 25.0 at ¯k=1, 3, and 8, respectively, above the shared-pool value at each connectivity.Source: §A.2 · p. 13
  • At ¯k=8, one-to-one mapping with r=1 reaches KQ 16.6, while separately drawn input edges change the curve little at low ¯k.The separate-edge comparison uses as many input edges on average as the shared pool.Source: §A.2 · p. 13
  • The replication uses |R|=25, one input node, 25 streams, 25 reservoirs per point, and no washout.KQ and GR streams are drawn independently; the GR streams share their last τ=3 steps.Source: §A.2 · p. 13

A.3 Statistical Checks

The checks quantify how design factors and reservoir-to-reservoir variability affect accuracy and the estimated optimal connectivity. They also test the stability of those estimates and the cost of selecting designs on a separate run.

Precision of k*

  • The 95% interval for optimal connectivity is at most one grid step wide at every fan-out, readout, and task except summed parity at fan-out 4.The within-1%-of-peak connectivity range is 0.25–1 from fan-out 16 upward; it describes peak breadth, not precision.Source: §A.3 · p. 14

Each combination’s k*

  • From fan-out 4 upward, each combination’s median optimal connectivity matches the mean-curve optimum at 19 of 20 points, with no interquartile range wider than 0.5.At fan-out 1, the medians are 4–7.Source: §A.3 · pp. 14–15

Choosing on the selection run

  • Selecting a design on the selection run reduces accuracy on reported reservoirs by a mean 3.4–10.5% at fan-outs 1–64 and 0.4–1.7% at 256–512.The largest drop is 22.6% at fan-out 1; within-design single-reservoir standard deviations range from 1.4% to 19.5%.Source: §A.3 · p. 15

Task and delay

  • Three task pairs and three delay pairs are excluded because final-state parity at delay 10 and fan-outs 256–512 stays at chance, leaving their optimal-connectivity differences imprecise.Their difference intervals span 3–4.25 grid steps; every other interval is at most 0.75 wide.Source: §A.3 · p. 15

Other factors and k*

  • With self-loops, optimal connectivity rises by less than one edge per node; at s=1 it rises by 0.75 in 7 of 20 combinations.At s=0.5, where no edge is moved, optimal connectivity already rises by 0–0.5.Source: §A.3 · p. 15

Homogeneous wiring

  • Homogeneous wiring exceeds heterogeneous wiring by a mean 0.4–4.1 percentage points across fan-outs, with 95% intervals above zero at fan-outs 64 and 256.The mean differences from fan-out 1 to 512 are +1.1, +3.7, +4.1, +2.7, +2.0, and +0.4 percentage points.Source: §A.3 · pp. 15–16

Variance shares

  • Input fan-out and connectivity explain 42–86% of design-accuracy variance, while same-design reservoir variation accounts for more than half of single-reservoir accuracy variance on three final-state parity variants.The other factors each account for at most 7%, except injection, which accounts for at most 21%.Source: §A.3 · p. 16

Other factors

  • At the operating point, second ticks and self-loops on at least 20% of nodes lower accuracy on every readout and task, while the tested initialization effects include zero.The regression estimates mean effects; its conclusions agree with the paired-difference medians.Source: §A.3 · p. 16

A.4 The Probe

The probe compares state-based measures with trained-readout accuracy while accounting for connectivity and self-loops. Its usefulness varies by task, design, and reservoir, and peak locations are generally close to the task-specific optimum.

Probe settings

  • Under last + steal with τ=3 or 5, ∆ correlates with single-reservoir accuracy within self-loop levels only on summed parity; other probe settings show correlations no greater than 0.11.For summed parity, the correlations are 0.37–0.39; why only these settings correlate remains open.Source: §A.4 · p. 16

Factor effects

  • Table 3 reports factor effects as average marginal changes in accuracy, using one tick, random initialization, and no self-loops as the baseline.It distinguishes comparisons at the operating-point connectivity from self-loop comparisons at each combination’s own optimum.Source: Table 3 · p. 17

Locating k*

  • The ∆ peak lies within 0.5 of each task variant’s own optimal connectivity in 85–100% of combinations, and within 0.5 at every self-loop level in 95–100%.The two exceptions where neither the ∆ peak nor connectivity 2 is close are final-state parity at delay 10 and fan-outs 256 and 512, where accuracy stays at chance.Source: §A.4 · p. 17, Table 4 · p. 18

Ranking designs

  • At the operating point, ∆ correlates positively with design accuracy on 9 of 14 task variants, but can rank designs in reverse when a second tick raises ∆ and lowers accuracy.On final-state majority at window 5 and delay 10 without self-loops, the second tick raises ∆ from 21 to 30 while accuracy falls from 67% to 55%.Source: §A.4 · p. 18, Table 5 · p. 19

Ranking reservoirs

  • Across reservoirs within a design, ∆ correlates positively with accuracy on 15 of 16 task variants, yet the highest-∆ reservoir can be less accurate than the median reservoir.For the specified one-tick, no-self-loop designs, this occurs by up to 4.6 percentage points on five task variants.Source: §A.4 · p. 18, Table 5 · p. 19

A.5 Supplementary Scatters

The supplementary figures display individual reservoir outcomes and design-level comparisons across connectivity, fan-out, and self-loop settings.

Fig. 6

  • In Fig. 6, each point is a reservoir’s final-state accuracy against connectivity; the lower optimal connectivity at fan-out 256 than at fan-out 4 appears for both majority and parity.Parity also has many reservoirs near 50%, especially at fan-out 4.Source: Figure 6 · p. 20

Fig. 7

  • In Fig. 7, ∆ and accuracy fall together as self-loop fraction rises at delay 5, but at delay 10 a second tick raises ∆ and lowers accuracy below full self-loop fraction.Points represent designs; filled and open markers distinguish one and two ticks.Source: Figure 7 · p. 20

Fig. 8

  • In Fig. 8, development accuracy is plotted against test accuracy for individual reservoirs; those at 100% on development remain at 100% or just below on test.The plot uses final-state readout at window 5 and delay 5.Source: Figure 8 · p. 21
Loading 2610.33498v1…