PaperlayerView exact revision on arXiv

Source-linked AI summary

A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)

Saber Moradi, Ning Qiao, Fabio Stefanini, Giacomo Indiveri

arXiv:1708.04198v2cs.ARcs.AI

TL;DR

Large neuromorphic systems must manage asynchronous event traffic without excessive routing memory, while retaining configurable connectivity. The paper proposes a two-stage mixed hierarchical-mesh architecture with heterogeneous distributed memories and validates it in a multi-core prototype. The prototype achieves state-of-the-art-comparable power and latency, supports a tiled three-layer CNN, and produces reliable accurate results on event-based data with very low power and latency.

  • Problem

    Large-scale neuromorphic networks combine high fan-in and fan-out with substantial routing circuit and memory requirements, creating a scalability challenge.

  • Method

    The paper develops a two-stage tag-based routing scheme combining hierarchical and 2D-mesh routing with heterogeneous distributed programmable memories.

  • Results

    The prototype achieves power consumption and latency comparable to analogous state-of-the-art neuromorphic systems and supports a tiled three-layer CNN with reliable accurate event-based classification.

  • Takeaways & Limitations

    The architecture provides a configurable trade-off between routing-memory demand and network flexibility for scalable, reconfigurable spiking neural networks.

Abstract

from arXiv · show

Neuromorphic computing systems comprise networks of neurons that use asynchronous events for both computation and communication. This type of representation offers several advantages in terms of bandwidth and power consumption in neuromorphic electronic systems. However, managing the traffic of asynchronous events in large scale systems is a daunting task, both in terms of circuit complexity and memory requirements. Here we present a novel routing methodology that employs both hierarchical and mesh routing strategies and combines heterogeneous memory structures for minimizing both memory requirements and latency, while maximizing programming flexibility to support a wide range of event-based neural network architectures, through parameter configuration. We validated the proposed scheme in a prototype multi-core neuromorphic processor chip that employs hybrid analog/digital circuits for emulating synapse and neuron dynamics together with asynchronous digital circuits for managing the address-event traffic. We present a theoretical analysis of the proposed connectivity scheme, describe the methods and circuits used to implement such scheme, and characterize the prototype chip. Finally, we demonstrate the use of the neuromorphic processor with a convolutional neural network for the real-time classification of visual symbols being flashed to a dynamic vision sensor (DVS) at high speed.

I. Introduction

The paper addresses routing and memory challenges caused by large fan-in and fan-out in scalable neuromorphic networks. It proposes a programmable two-stage, mixed tag-based scheme that combines hierarchical and mesh routing with heterogeneous distributed memories to reduce memory requirements while preserving flexibility.

  • Proposed approach: The proposed scheme combines point-to-point source-address routing with multicast destination-address routing through shared tags and clustered connectivity.Clusters use independent address spaces, allowing tag identifiers to be reused without loss of generality.
  • Validation: The approach was validated in a four-core prototype containing 1k VLSI neurons and programmable routers for address-event traffic.The prototype integrates the routing scheme with hybrid analog/digital neuron and synapse circuits.
  • Proposed approach: The two-stage design reduces each source neuron's stored entries from F to F/M and routes them through N/C intermediate nodes before cluster-wide broadcast.Each target cluster contains C neurons, with M neurons subscribed to the incoming tag.
  • Design trade-offs: The ratio α = K/C and parameter M expose trade-offs between routing flexibility, memory usage, point-to-point copying, and broadcasting.Larger clusters and fewer tags reduce memory, while increasing the number of clusters can counterbalance reduced tag flexibility.
  • Memory scaling: Less than 1.2k bits per neuron are required for the proposed scheme versus 160k bits per neuron for conventional routing in a 1 million-neuron, near-10000-fan-out example.The comparison assumes a cluster size of 256 neurons and the same network size and fan-out.

III. Mixed-mode hierarchical-mesh routing architecture

The architecture combines hierarchical routing for low latency with mesh routing for lower bandwidth requirements. Three router levels distribute local, intermediate, and distant event traffic across cores and tiles.

  • Router hierarchy: R1 routers handle local core traffic by returning events locally or forwarding non-local events to the next hierarchy level.Events delivered from R1 to a core are broadcast to all neurons and accepted when their CAM entries match the incoming tags.
  • Router hierarchy: R2 routers connect cores through bidirectional channels and can form multiple tree-based hierarchy levels for increasingly distant destinations.The example architecture links groups of four cores at level 1 and uses higher R2 levels across groups.
  • Mesh routing: R3 routers connect farther destinations, including different tiles, using four cardinal directions in a 2D mesh.This mesh stage complements the hierarchical routing structure for large-scale communication.
  • Design rationale: The mixed-mode design combines mesh routing's low bandwidth requirements with hierarchical routing's low latency.The stated trade-off is that mesh routing has higher latency, whereas hierarchical routing has higher bandwidth requirements.

A. Quasi Delay-Insensitive asynchronous circuit design methodology

The router circuits are designed with a quasi-delay-insensitive methodology that progresses from CHP descriptions through handshaking expansion and production rules to CMOS implementations. A controlled-pass process illustrates this synthesis flow.

  • Design flow: R1, R2, and R3 routers are synthesized using QDI methods based on CHP and HSE formalisms.The design flow converts high-level communicating processes into handshaking expansions and then production rules for circuit implementation.
  • QDI methodology: QDI design assumes delays only for signals that fan out to multiple gates, not for individual gate delays.This timing model supports asynchronous circuit design without gate-delay assumptions.
  • Controlled-pass example: The controlled-pass process receives input data and a dual-rail control signal, then either copies the data to the output or skips it.When passing data, the process waits for input and control signals to return neutral before releasing the output handshake.
  • Circuit realization: The CMOS controlled-pass circuit is derived from production rules, with circles representing C-elements.The example spans CHP, HSE, production-rule, and transistor-level circuit descriptions.

B. Asynchronous routing fabric

The architecture uses asynchronous hierarchical routers and distributed SRAM to route events from local cores toward same-chip or inter-chip destinations. Each event’s routing information is read iteratively and then directed according to its destination.

  • R1 router: Each core pairs with an R1 router whose SRAM stores source-node routing information and whose input receives address-events from that core.The event packet is extended with two bits encoding first-level fan-out before entering the router’s memory-address loop.
  • R1 router: The R1 memory-address loop merges, buffers, splits, reads SRAM, and repeats until all routing information for the event has been retrieved.The loop includes controlled-pass and decrement processes for iterating through the relevant memory entries.
  • R1 router: Each SRAM word contains a 10-bit tag, 6-bit routing header, and 4-bit destination-core identifier.The header encodes ΔX and ΔY hop counts plus direction signs, supporting relative routing decisions.
  • R1 router: After SRAM lookup, a controlled split broadcasts events locally, sends them point-to-point within the chip, or forwards them through R2 and R3 for inter-chip mesh routing.The selected output depends on the core and chip destination fields.
  • R2 router: Each R2 router connects bidirectionally to four same-chip R1 routers and to a higher-level router for inter-tile and longer-distance communication.This forms the central level of the hierarchical 2D-tree routing structure.

2) The R2 router:

The upper routing fabric combines hierarchical links with a global 2D mesh. R3 routers use XY routing to forward events across tiles by consuming relative X and Y hop counts.

  • R3 router: The R3 router routes events among multiple tiles using relative-distance addressing across a 2D mesh.It connects the local R2 router to neighboring chips or tiles.
  • R2 router: Each R1 router connects its local core bidirectionally to the R2 router, while each R2 connects four same-chip R1 routers to a higher-level link.These links establish the hierarchical path before global mesh traversal.
  • R3 router: XY routing first moves events along X until ΔX reaches zero, then routes them along Y.The packet carries direction signs, and the corresponding hop count is decremented at each step.
  • R3 router: On downstream paths, R3 checks the remaining ΔY or ΔX value to choose whether an event continues across the mesh or returns to the local R2 router.The control splits determine the next directional output based on the remaining hop count.
  • Input Interface: The input interface receives external address-events and dispatches them to neuron stimuli, bias-generator registers, or SRAM and CAM memory cells.Successive control splits classify the packet and deliver it to its final programmed destination.

4) Input Interface:

The prototype integrates global mesh routing with distributed heterogeneous memories and mixed-signal neuron and synapse circuits. External events can program memories, biases, and neural stimuli through the input interface.

  • R3 router: One R3 router per chip or tile manages global traffic in the 2D mesh.It provides the global-routing level above the R2 routers.
  • Area breakdown: The core area is 38.5 mm2, with approximately 30% used for memory circuits and 20% for neuron and synapse circuits.The area breakdown is reported for the chip core excluding the pad frame.
  • Neuron and synapse circuits: Neuron dynamics use Adaptive-Exponential Integrate-and-Fire circuits, while synapse dynamics use DPI log-domain filters supporting EPSCs and IPSCs across microsecond-to-hundreds-of-millisecond time constants.Programmable temperature-compensated bias generators set analog circuit parameters and support different neural population types.
  • Input Interface: The input interface programs distributed SRAM and CAM memories, configures network parameters, and sends external stimuli to neurons.It also supports configuration of circuit biases through external address-events.
  • Prototype architecture: The prototype contains four cores with 256 neurons each, and neurons across cores and chips communicate through R1, R2, and R3 routers.Neuron and synapse dynamics are programmable through on-chip bias generators.
  • Mixed-signal implementation: The chip uses mixed-mode analog/digital circuits with distributed heterogeneous memory elements including capacitors, CAM, and SRAM cells.This creates an in-memory, non-von Neumann computing architecture.

A. The core memory/computing module

Each computing node combines programmable synapse memory, asynchronous event processing, analog synapse dynamics, and an integrate-and-fire neuron. The prototype demonstrates low-latency, scalable routing with reduced-voltage operation and power figures comparable to more advanced technologies.

  • Node architecture: Each node integrates 64 10-bit CAM words, 64 2-bit SRAM cells, four synapse circuits, and one leaky integrate-and-fire neuron.The broader node includes pulse generators, pulse extenders, pulse decoders, digital-to-analog converters, DPI filters, and handshaking circuitry.
  • Memory and synapse programming: CAM stores source-neuron address tags, while SRAM selects four synapse behaviors: fast or slow excitatory, subtractive inhibitory, or shunting inhibitory.A CAM match triggers a pulse generator and tunable pulse extender, whose square wave drives the selected DPI synapse circuit.
  • Event processing: A matched event produces a local synaptic response, and accumulated synaptic currents drive the neuron to emit an address-event through the handshaking block and associated router.The synaptic response is an EPSC or IPSC with programmable temporal dynamics and weight controlled by shared biases.
  • Prototype characterization: 1.3 V operation was achieved without loss of functionality, although overall system performance was limited by I/O speed.The fabricated process expected a 1.8 V core supply; the chip’s broadcast time was set to 27 ns under worst-case timing assumptions.
  • Power and latency: The chip achieves power consumption and latency figures comparable to analogous state-of-the-art architectures fabricated with more advanced technology nodes.Power measurements covered different supply voltages and firing rates in a worst-case all-neuron firing condition, excluding synaptic-current effects.
  • Throughput and scalability: ≈27 ns broadcast time provides ≈38 Mevents/sec local bandwidth, supporting 7200 fan-in at 20 Hz and 1400 fan-in at 100 Hz.The prototype has 256 neurons per core; across-chip latency is approximately 15.4 ns, while the R3 router latency is 2.5 ns at 400 Mevent/sec.

Example application: Convolutional Neural Networks

The prototype was used to implement a multilayer CNN for Poker card suit recognition with dynamic vision inputs. A 2560-neuron architecture achieved 100% performance on the test data, while programmable routing supports broader network configurations.

  • Example application: Convolutional Neural Networks: The prototype implements CNNs that extract low-level visual features and combine them into higher-level detectors for classification.A nine-chip PCB supports multilayer networks, with programmable on-chip routers managing inter-chip communication and an FPGA configuring memories and analog parameters.
  • Poker card suit recognition: Four output populations of 64 neurons classify each Poker card suit using the most active population during the response.All-to-all pooling-to-output connections are tuned with an off-line Hebbian-like rule, and majority voting stabilizes performance.
  • Recognition result: 100% performance was achieved on the test data set using 2560 neurons.The nine-chip board can support networks of up to 9k neurons.
  • Routing flexibility: The memory-optimized programmable routing scheme efficiently allocates memory, synapse, and neural resources across a wide range of architectures, including CNNs.The paper evaluates this flexibility by comparing resource requirements for CNN models implemented on TrueNorth.

VI. Discussion

Neuromorphic systems co-localize memory and computation for scalable, low-power processing, but routing overhead creates trade-offs among bandwidth, flexibility, and memory. The proposed architecture addresses these trade-offs through heterogeneous distributed memory and mixed routing, validated in a prototype and CNN demonstration.

  • Neuromorphic processors co-localize memory and computation, enabling scalable low-power processing but introducing non-trivial routing overhead.Routing optimization must address communication bandwidth alongside the von Neumann bottleneck.
  • Existing approaches trade communication efficiency against programmability, on-chip memory area, or reliance on external memory.The discussion contrasts optimized routing with configurable systems that require more silicon memory or external banks.
  • The architecture combines distributed heterogeneous memory with flexible routing to optimize area and power while supporting sensory-processing networks beyond classical CNNs and DNNs.It targets real-time streaming data without saving inputs in external memory structures.
  • The two-stage routing scheme minimizes memory for scalable, reconfigurable spiking networks with bounded connectivity by balancing point-to-point specificity against broadcasting efficiency.The scheme analytically addresses the trade-off between connection specificity and memory budget.
  • The prototype integrates asynchronous QDI routing circuits with mixed-signal neuron and synapse circuits and demonstrates a three-layer CNN on event-based data.Multiple processors were tiled and configured for the experiment.
  • The fabricated 0.18 um processor achieved power comparable to state-of-the-art digital solutions, while the configured system produced accurate results with low power and latency.The paper also reports scaling studies projecting advantages in a 28 nm FDSOI implementation.

Appendix

The routing analysis explains how clustered connectivity and sparse long-range links reduce address collisions and guide the optimal two-stage design. It derives constraints on cluster size and illustrates a concrete parameter choice.

  • Clustered connectivity lets neurons reuse tag identifiers within independent address spaces, reducing routing memory for large networks.The approach uses mixed tag-based shared addressing rather than plain source or destination addressing.
  • Sparse long-range connections keep the number of address collisions and required tags relatively low, with tag reassignment providing additional control.The exact number of tags depends on the specific connectivity pattern.
  • The optimal design point must satisfy cluster-size constraints for the two-stage routing scheme to remain valid.The appendix states that the requirements constrain the relationship among cluster size, fan-out, and the optimal intermediate fan-out.
  • C ≥ N^(1/F) is a safe validity condition for the optimal design point, imposing few constraints when fan-out is typically 10^3–10^4.For fan-out 10, the example requires C ≥ 10, while much larger N would be needed to make the bound restrictive.
  • The second cluster condition is more restrictive: with F = 5000 and N = 10^10, clusters must satisfy C ≥ 152.This requirement ensures the cluster can support the anticipated fan-out.
  • With C = 256 and α = 1, the optimal M is 144, yielding fan-outs of 35 and 144 and total fan-out 5040.The example reports storage per neuron as 424.26 p.

B. Communicating Hardware Process (CHP)

The CHP notation defines communication, synchronization, assignment, selection, and repetition constructs used to specify asynchronous circuit behavior. These constructs provide the basic language for describing the paper’s communicating hardware blocks.

  • Send and receive commands transfer channel values and store received data in variables.Send uses X!e, while receive uses Y?v.
  • Probe tests whether a channel communication can complete without suspending.The probe expression is Boolean and indicates communication readiness.
  • Sequential and parallel composition specify whether program parts execute in order or concurrently.The notation uses S;T for sequential composition and S’T or S,T for parallel composition.
  • Assignment updates a variable, with upward and downward arrows denoting assignment of true and false.The notation is a := b, with a↑ and a↓ as Boolean shorthands.
  • Selection waits for a true guard and executes its associated statement, while repetition repeatedly chooses enabled guarded statements until none remain.The notation distinguishes mutually exclusive selection from non-exclusive guarded choice.

C. CHP, HSE, and PR examples or asynchronous circuits used in the routing scheme

The appendix presents CHP and related asynchronous-circuit examples that implement routing communication, arbitration, controlled splitting, and event forwarding. These examples connect high-level communicating processes with routing hardware behavior.

  • The examples specify repeated input-to-output forwarding and controlled routing decisions using CHP communication commands.One listing repeatedly receives from IN and sends to OUT.
  • Separate CHP processes raise arbitration requests for each input when valid data and enable conditions are present.The two input paths use distinct request and enable signals.
  • An arbitration process grants one of the competing input requests and withdraws the grant after the corresponding request ends.The example encodes mutually exclusive selection between the two input channels.
  • The controlled-split example copies buffered data to output fields while coordinating arbitration and enable signals for two input channels.The circuit handles input arbitration and output acknowledgment behavior.

D. Asynchronous Content-Addressable Memory (CAM) timing assumptions

The asynchronous CAM timing design uses worst-case propagation assumptions and a dummy CAM cell to complete handshaking reliably. Although local communication is not speed-optimized, the scheme reduces pulse-width mismatch across matched CAM words while remaining sufficient for each core.

  • Timing assumptions: Worst-case timing assumptions ensure correct handshaking between the local router and asynchronous CAM array.The design assumes the validity-check block and a duplicate CAM cell are the last elements to receive successive control signals.
  • Timing assumptions: A dummy CAM cell positioned at the array’s opposite corner determines when match-line reset and handshaking completion can occur.When the dummy cell’s match line is lowered, Check is de-asserted and the handshake finishes.
  • Timing assumptions: The local communication scheme is not optimized for speed, but its latency is sufficient to cover all neurons within one core.The small number of neurons per core permits this trade-off while supporting distributed CAM matches.
  • Timing assumptions: The validity-check block detects neutral or valid input states and controls precharge and comparison enable propagation across the CAM array.Neutral inputs set PreB low and pre-charge match lines high; valid events eventually enable comparisons.
  • Pulse-width control: Similar propagation delays for PreB and Check make the generated pulse weight approximately independent of a CAM word’s physical position.The approximation follows when td1x and td2x are similar, yielding a pulse weight near tpre − tck.
Loading 1708.04198v2…