PaperlayerView exact revision on arXiv

Source-linked AI summary

Synapse Loss Estimation for the BrainScaleS Wafer-scale Neuromorphic System

Bernhard Vogginger, Christian Mayr

arXiv:2610.07321v1cs.ETcs.NE

TL;DR

Mapping uniform random networks to BrainScaleS can leave model synapses without suitable hardware synapses, due to hardware constraints or mapping-software shortcomings. The paper develops analytical estimates for lossless network size and synapse loss, and compares them with mapping runs. Estimates match actual loss for small and large recurrent networks, but actual loss is higher than estimated for networks between 6,000 and 16,000 neurons.

  • Problem

    Mapping to BrainScaleS can leave model synapses without suitable hardware synapses because of structural limits or mapping-software shortcomings.

    Source: §1 · p. 2

  • Method

    The paper develops analytical estimates of maximum lossless network size and synapse loss for uniform random networks, then compares estimates with mapping-software runs.

    Source: §1 · p. 2, §3.2 · p. 19

  • Results

    For small and large recurrent networks, estimated loss matches actual mapping loss fairly well, but actual loss is much higher than estimated for networks between 6,000 and 16,000 neurons.

    Source: §4.1 · p. 22

  • Takeaways & Limitations

    The estimates can help search for optimal mapping scenarios for a given uniform random network without running the mapping software.

    Source: §4.3 · p. 23

    The analytic approaches omit supported hardware features, including different short-term plasticity parameters per routing source and different synapse-weight ranges requiring distinct gmax settings.

    Source: §4.3 · p. 23

Abstract

from arXiv · show

Neuromorphic hardware combines memory (synapses) and computation (neurons) into the same silicon substrate to avoid the von-Neumann bottleneck. This way, neuromorphic computing aims to make computational neuroscience simulations and AI processing faster and more energy-efficient. The common crossbar architecture integrates a synapse matrix (analog, mixed-signal or digital) with a neuron array into a neurosynaptic core, whereas multiple cores are interconnected via a dedicated spike routing network. One of such neuromorphic architecture is the BrainScaleS wafer-scale system which allows the realization of very flexible network architecture, supporting both dense and sparse connectivity by highly configurable neurosynaptic cores. Yet, when network models from computational neuroscience are mapped to the BrainScaleS wafer, synapse loss can occur which means that for some model synapses no hardware synapse is available due to the restricted connectivity of the hardware. In this work we analyze the synapse loss when mapping uniform random networks to BrainScaleS both theoretically and empirically. We first develop a methodology to estimate maximum-sized networks without synapse loss based on probability distributions and the hardware's synapse routing architecture. Next, we adopt the methodology to predict the synapse loss for denser or larger network models. Then, we compare the predictions with results from actual runs of the mapping software: The results show the capabilities and limitations of the BrainScaleS system itself and spot shortcomings of the current mapping algorithms. The developed methodology can be helpful in multiple ways: to find the optimal settings when mapping a given network model to BrainScaleS, to act as a benchmark for the mapping software, and as tool for design space exploration for new hardware.

1 Introduction

BrainScaleS combines neurosynaptic cores with wafer-wide spike routing to support flexible connectivity, but mapping can leave model synapses without suitable hardware synapses. This work develops analytical estimates for lossless uniform random networks and for synapse loss in larger or denser networks, then compares predictions with mapping-software runs.

1 Introduction

  • BrainScaleS combines 384 neurosynaptic cores on a wafer-wide spike-routing grid, with crossbar cores and configurable routing that support dense and sparse networks.The system supports compound neurons with more than 10,000 input synapses per neuron.Source: §1 · pp. 1–2
  • Mapping can leave model synapses without suitable hardware synapses because of structural limits or shortcomings in mapping software and algorithms.The complex routing architecture requires a multi-step mapping process.Source: §1 · p. 2
  • The analysis estimates maximum lossless uniform-random network sizes, predicts synapse loss for larger or denser networks, and compares predictions with mapping-software runs.The paper also discusses applications, limitations, and design parameters of the routing architecture.Source: §1 · p. 2

2.1 BrainScaleS

The BrainScaleS wafer combines mixed-signal HICANNs, synapse arrays, and a sparse on-wafer routing network. Mapping places neurons, routes spikes, and connects vertical buses to synapse drivers and hardware synapses; losses may occur during routing and synapse assignment.

2.1.1 Hardware description

  • Each HICANN has 512 neurons and 114,688 synapses, while up to 64 neuron circuits can form a compound neuron with up to 14,336 incoming synapses.The paper considers the HICANNv2 layout with 224 synapse rows per block.Source: §2.1.1 · p. 3
  • Sparse switches connect wafer-wide horizontal and vertical buses to route spikes from source neurons to target HICANNs and synapse drivers.The synapse array decodes each 6-bit source address in two stages: the 2 MSBs at the driver and the 4 LSBs in the synapse circuits.Source: §2.1.1 · p. 3, Figure 2 · p. 4

2.1.2 Mapping process

  • Mapping places neurons, assigns addresses and buses, routes sources to target HICANNs, and connects vertical buses to synapse drivers before assigning model synapses to hardware synapses.Synapse loss may occur during routing or synapse assignment; the article focuses on calculating driver requirements for the latter steps.Source: §2.1.2 · pp. 6–7

2.2 Synapse driver requirements

The section derives synapse-driver requirements from the hardware mapping constraints, then develops a probability-based estimate of the resources uniform random networks require. It links per-neuron synapse counts to half-row and driver allocations.

2.2 Synapse driver requirements

  • The section introduces the prerequisites and software calculation before deriving requirements for uniform random connectivity, forming the basis for later lossless-size and synapse-loss estimates.The analysis assumes static synapses without STP in parts of the setup.Source: §2.2 · p. 7, §2.2.1 · p. 8, §2.2.3 · p. 9

2.2.1 Prerequisites

  • A synapse driver is the smallest resource connected to one vertical bus and provides two rows totaling 512 synapses; mapping also requires matching synapse type and decoder addresses.The analysis assumes neuron size of at least 2, and its simple target mapping assigns excitatory inputs left and inhibitory inputs right.Source: §2.2.1 · pp. 7–8

2.2.2 Synapse driver requirements calculation

  • Driver requirements are calculated by counting synapses per target, pattern, and type, taking maximum half-row needs across neurons, then summing rows and rounding up to drivers.The described calculation simplifies the software algorithm, which also handles STP modes, additional synaptic input circuits, and flexible target mappings.Source: §2.2.2 · p. 9

2.2.3 Synapse driver estimation for uniform random networks

  • For uniform random networks, binomial synapse counts are combined across target neurons and four 2-MSB patterns to estimate required half rows and synapse drivers per routing source.The method estimates the maximum synapse count across targets, then combines pattern-level row requirements; Figure 5 illustrates the resource calculation.Source: §2.2.3 · pp. 9–10

2.3 Estimating the maximum number of source neurons per HICANN

The section estimates the maximum input-neuron count for a connection probability by relating expected driver demand to the 224-driver HICANN capacity.

2.3 Estimating the maximum number of source neurons per HICANN

  • The expected number of routing sources is limited by the 224 synapse drivers per HICANN, using expected drivers required per source.The source count is an expectation rather than a discrete allocation.Source: §2.3 · p. 11
  • Multiplying the estimated routing-source count by 64 addresses per source gives the maximum input-neuron estimate for a given connection probability.Source: §2.3 · p. 11
  • The calculation neglects hardware constraints including four driver blocks and sparse driver switches that can hinder some connection setups.Source: §2.3 · p. 12

2.4 Synapse loss estimation

The section estimates synapse loss by allocating hardware resources to network requests in stages, prioritizing the half-row assignments expected to realize the most synapses.

2.4 Synapse loss estimation

  • The reverse allocation assigns synapse drivers to routing sources, then half synapse rows to 2-MSB patterns, estimating the unrealized fraction of synapses.Source: §2.4 · p. 12
  • For each pattern, the expected added synapses from a half row depend on row capacity and target-neuron count; row gains are sorted to maximize realized synapses within assigned drivers.The row capacity depends on neuron size, while the number of targets is a multiplicative factor.Source: §2.4 · p. 12
  • When distributing 224 drivers across L routing sources, the method apportions floor and ceiling driver counts when L ≤ 224, and leaves L−224 sources without drivers when L exceeds 224.Total realized and lost synapses are calculated by summing the corresponding outcomes across source groups.Source: §2.4 · p. 13

2.5 Hardware limitations

The section identifies address availability, driver concatenation, and usable-HICANN counts as hardware constraints incorporated into the synapse-loss estimates.

2.5 Hardware limitations

  • One 4-bit address is reserved to disable synapses, reducing the number of neurons per routing source from 64 to 60.A zero digital weight produces a non-zero positive current, motivating the reserved address.Source: §2.5 · p. 13
  • Address 0 is reserved for repeater DLL locking, leaving 59 usable 6-bit addresses for real neurons.Source: §2.5 · p. 13
  • Driver-chain length is limited because longer chains degrade signal quality; the study assumes a maximum chain length of 5, while the software default is 3.Source: §2.5 · p. 13
  • Of 384 wafer HICANNs, 16 are unavailable for experiments, so the loss estimate uses 368 usable HICANNs.Source: §2.5 · p. 13

2.6 List of symbols and constants

Table 2 is identified as the article’s list of symbols and constants.

2.6 List of symbols and constants

  • Table 2 lists the symbols and constants used in the article.Source: §2.6 · p. 13

3 Results

The results estimate lossless network capacity and synapse loss across mapping scenarios, then compare those estimates with actual mapping. Capacity and loss depend on connection density, neuron size, and routing constraints; for smaller networks, mapping can lose more synapses than the estimate predicts.

3.1 Maximum size of lossless uniform random networks

  • For p > 0.7, synapse utilization scales proportionally with p and reaches 100% at p = 1.0; larger neuron sizes can leave more hardware synapses unused.The paper attributes lower utilization at larger sizes to discretization in the number of synapses per half row.Source: §3.1 · p. 14, Figure 6 · p. 15
  • Decoding all six address bits in each synapse supports more realizable source neurons and higher utilization at low connection probabilities than split 2-bit/4-bit decoding, at added hardware cost.The alternative needs two additional broadcast lines, SRAM cells, and comparators per synapse, and may reduce the number of synapse rows fitting in the chip area.Source: Figure 7 · p. 15, §3.1 · p. 16
  • Using only 59 of 64 routing addresses reduces maximum source capacity most for size-16 neurons at low connection probabilities, while reducing utilization for sizes 8 and 4.The driver-chain limitation affects sizes 8 and 4 when the probability of needing more than five drivers per routing source exceeds 80%.Source: Figure 8 · p. 16, §3.1 · p. 16
  • For p < 0.02, neuron size 8 supports recurrent networks above 12,000 neurons; sizes 12 and 16 yield the largest networks at progressively higher connection probabilities.Size 12 is best for 0.02–0.08; size 16 is best for p ≥ 0.08, with network sizes of 3,360–6,720 depending on driver assignment.Source: Figure 9 · p. 17, §3.1 · p. 17

3.2 Synapse loss estimation for uniform random networks

  • At p = 0.01, feedforward loss stays close to zero through 13,216 source neurons for neuron sizes ≥8; loss rises with connection probability and source count.Beyond 13,216 sources, routing sources exceed the number of synapse drivers, causing unavoidable loss.Source: Figure 10 · p. 18, §3.2 · p. 19
  • Recurrent loss generally falls with larger neuron sizes until driver limits are reached, but the best mapping depends on network size and p; estimates exclude potential on-wafer routing loss.For p ≤ 0.2, near-zero loss can extend beyond estimated maximum lossless sizes; the analysis assumes at least 224 routing sources reach each target HICANN after on-wafer routing.Source: §3.2 · p. 19, Figure 11 · p. 20

3.3 Comparison to actual mapping

  • For small networks, mapped synapse loss exceeds estimates, while estimates fit actual loss well above 20K neurons; on-wafer routing loss is much smaller than final loss.The comparison uses recurrent networks with connection probabilities 0.01 and 0.05; the authors identify synapse-driver limits, rather than inter-HICANN connectivity, as the dominant loss factor.Source: Figure 12 · p. 21, §3.3 · p. 22
  • At p = 0.01, actual loss exceeds 10% at 10K neurons despite estimates near zero; at p = 0.05, actual loss exceeds 25% at 10,000 neurons before falling to 20% at 14,000.The authors attribute these deviations to sparse synapse-driver switches and the heuristic driver-assignment algorithm; they note alternative algorithms may improve results.Source: §3.3 · pp. 21–22

4 Discussion

The discussion presents analytical estimates of lossless network capacity and synapse loss, then compares estimates with mapping results and examines BrainScaleS routing trade-offs. It also identifies uses and scope limits of the methods.

4.1 Summary

  • The theory estimates lossless maximum sizes for feed-forward and recurrent networks; tolerating up to 1% loss permits much larger networks, especially when p ≤0.2.The maximum-size analysis covers different network sizes and mapping scenarios and considers hardware constraints.Source: §4.1 · p. 22
  • For recurrent networks, estimates match actual mapping loss at small and large sizes, but actual loss is much higher for networks of 6,000–16,000 neurons.The discrepancy is attributed to sparse vertical-bus-to-synapse connections and limitations of the MappingTool routing algorithm; newer mapping software is expected to improve results.Source: §4.1 · p. 22

4.2 Analysis of BrainScaleS routing architecture

  • More than 224 routing sources per HICANN makes synapse loss unavoidable; intermediate-size mapping loss is also attributed to switch matrices with only 1 of 8 possible switches.Structured networks can also be affected by the routing-source limit.Source: §4.2 · p. 22
  • A HICANN decoder that decodes all 6-bit addresses in the synapse allows larger lossless networks and lower estimated loss for p ≤0.2; BrainScaleS-2 adopted this decoding.BrainScaleS-2 has not been scaled to a wafer system; its multi-chip systems use FPGAs for inter-chip routing.Source: §4.2 · p. 22

4.3 Applications, limitations, related and future work

  • Users can estimate maximum sizes, select mapping scenarios without running the mapper, and benchmark its loss; designers can compare address decoders while accounting for area overhead.The benchmark estimates how low loss may be for a network and setup, though a particular network instance can achieve lower loss.Source: §4.3 · p. 23
  • The estimates omit prior routing losses and some supported hardware features; accounting for routing-source gain variation can make predicted loss higher than actual loss.The analysis assumes no connections were lost in earlier routing steps and does not model, for example, per-source short-term plasticity or differing per-row gmax scaling.Source: §4.3 · p. 23
  • The methods are demonstrated for uniform-random networks using a binomial distribution, but other discrete distributions and hierarchical connectivity are possible extensions.Examples of alternative connectors include PyNN's FixedNumberPreConnector and FixedTotalNumberConnector.Source: §4.3 · p. 23
  • The approach addresses a need to map neuroscience models without synapse loss and may also inform energy-efficient AI hardware design.The methods are described as applicable to hierarchical circuit-switched crossbar architectures, including spiking and non-spiking systems.Source: §4.3 · pp. 23–24

A Appendix

Figure 14 compares estimated synapse loss across address-decoding architectures and source-neuron counts per HICANN. It marks the 224-driver limit beyond which the excess routing sources lose their synapses.

A Appendix

  • The figure compares estimated loss for HICANN decoding (2 bits per half synapse row, 4 bits per synapse) and 6-bit in-synapse decoding across connection probabilities and source-neuron counts.The scenario fixes 64 neurons of size 8 and assumes 64 addresses per routing source; dashed lines indicate loss from exceeding 224 synapse drivers per HICANN.Source: Figure 14 · p. 27
Loading 2610.07321v1…