PaperlayerView exact revision on arXiv

Source-linked AI summary

Snitch: A tiny Pseudo Dual-Issue Processor for Area and Energy Efficient Execution of Floating-Point Intensive Workloads

Florian Zaruba, Fabian Schuiki, Torsten Hoefler, Luca Benini

arXiv:2002.10143v2cs.AR

TL;DR

Data-parallel workloads increase demand for floating-point performance, while energy efficiency is a central design concern and specialized accelerators can be difficult to adapt. Snitch pairs a small integer core with a double-precision FPU and adds SSR and FREP extensions. It reports speed-ups up to 6.45× on data-oblivious kernels and twice the energy efficiency of existing state-of-the-art systems.

  • Problem

    Data-parallel workloads increase demand for floating-point performance, while energy efficiency is central and specialized accelerators can be difficult to adapt as algorithms change.

    Source: §1 · p. 1

  • Method

    Snitch pairs a tiny integer control core with a double-precision FPU and uses SSR and FREP extensions to streamline memory operations and decouple floating-point instruction sequencing.

    Source: §6 · pp. 14–15

  • Results

    Snitch achieves up to 6.45× speed-up on data-oblivious kernels and 2× greater energy efficiency than existing state-of-the-art systems.

    Source: §6 · p. 15

  • Takeaways & Limitations

    The small integer control unit preserves programmability and supports adaptation to changing algorithmic requirements without specializing to one problem domain.

    Source: §6 · p. 15

    The authors could not make accurate direct measurements against the A64FX system because they lacked access to it.

    Source: §5.3 · p. 14

Abstract

from arXiv · show

Data-parallel applications, such as data analytics, machine learning, and scientific computing, are placing an ever-growing demand on floating-point operations per second on emerging systems. With increasing integration density, the quest for energy efficiency becomes the number one design concern. While dedicated accelerators provide high energy efficiency, they are over-specialized and hard to adjust to algorithmic changes. We propose an architectural concept that tackles the issues of achieving extreme energy efficiency while still maintaining high flexibility as a general-purpose compute engine. The key idea is to pair a tiny 10kGE control core, called Snitch, with a double-precision FPU to adjust the compute to control ratio. While traditionally minimizing non-FPU area and achieving high floating-point utilization has been a trade-off, with Snitch, we achieve them both, by enhancing the ISA with two minimally intrusive extensions: stream semantic registers (SSR) and a floating-point repetition instruction (FREP). SSRs allow the core to implicitly encode load/store instructions as register reads/writes, eliding many explicit memory instructions. The FREP extension decouples the floating-point and integer pipeline by sequencing instructions from a micro-loop buffer. These ISA extensions significantly reduce the pressure on the core and free it up for other tasks, making Snitch and FPU effectively dual-issue at a minimal incremental cost of 3.2%. The two low overhead ISA extensions make Snitch more flexible than a contemporary vector processor lane, achieving a $2\times$ energy-efficiency improvement. We have evaluated the proposed core and ISA extensions on an octa-core cluster in 22nm technology. We achieve more than $5\times$ multi-core speed-up and a $3.5\times$ gain in energy efficiency on several parallel microkernels.

1 INTRODUCTION

Rising floating-point demand makes energy efficiency central, but accelerators can be hard to adapt while programmable processors incur efficiency costs. Snitch combines a tiny control core with a double-precision FPU and two ISA extensions to improve utilization while retaining flexibility.

1 INTRODUCTION

  • Data-parallel workloads are increasing demand for floating-point performance, while integration density makes energy efficiency a central design concern.Accelerators can be difficult to adapt as algorithms evolve; programmable CPUs and GPUs offer flexibility but incur overhead and face memory-throughput constraints.Source: §1 · p. 1

1.1 Design Goal: Area and Energy efficiency

  • Four of five dot-product instructions handle bookkeeping; one loop operation consumes 317 pJ, of which only 28 pJ goes to computation.The cited example attributes 289 pJ to auxiliary tasks and reports 17% FPU utilization.Source: §1.1 · p. 2, Figure 1 · p. 2

1.2 Existing Mitigation Techniques and Architectures

  • Existing approaches trade efficiency for flexibility: SIMD can reduce loop overhead but may require shuffles for individually accessed elements, while GPU thread management adds hardware cost.Vector architectures process larger data chunks, whereas the cited GPU approach uses hardware scheduling and coalescing to manage parallel threads and memory traffic.Source: §1.2 · p. 2

1.3 Contributions

  • Snitch pairs a tiny integer core with a double-precision FPU, using SSRs and FREP to reduce core workload and enable pseudo-dual-issue execution.SSRs map affine memory streams to register reads and writes; FREP sequences floating-point instructions from a buffer while freeing the integer core for control tasks.Source: §1.3 · pp. 2–3
  • The extensions add less than 7% area to the core complex and 3.2% at cluster level, while the design reports 3.5× higher energy efficiency and 4.5× better FPU utilization on small matrices than the state of the art.The implementation also reports FFT speed-ups of 4.7× on one core and close to 3× on an octa-core cluster.Source: §1.3 · p. 3

2 ARCHITECTURE

The system is built from configurable Snitch core complexes, Hives, and clusters, with shared resources at each level.

2 ARCHITECTURE

  • A Snitch core complex contains an integer core and FPU subsystem; configurable numbers of cores form Hives, and Hives form clusters sharing TCDM.The hierarchy can be scaled down to a single-core Hive and a single-Hive cluster with private scratchpad memory.Source: §2 · p. 3

2.1 Snitch Core Complex

A Snitch core complex combines a compact integer core with a floating-point subsystem and configurable interfaces for offloading and memory access.

2.1 Snitch Core Complex

  • The core complex contains an RV32IMAFD RISC-V core and can be configured with or without the proposed ISA extensions.Its offloading and TCDM load/store interfaces can be decoupled to raise clock frequency at the cost of one cycle of latency.Source: §2.1 · p. 3

2.1.1 Integer Core

  • Snitch is a 9–20 kGE, 32-bit integer RISC-V core designed for a small area footprint, with a single-stage, single-issue, in-order pipeline.Its accelerator interface offloads integer multiply/divide and floating-point instructions; the integer core can execute ready instructions in one cycle.Source: §2.1.1 · pp. 3–4
  • The integer core offloads floating-point instructions to a separate subsystem, allowing floating-point logic to be decoupled and sequenced independently.The accelerator interface also supports sharing multiply/divide hardware across cores in a Hive.Source: §2.1.1 · p. 5

2.1.2 FPU Subsystem

  • The floating-point subsystem combines an IEEE-754-compliant FPU, a 32×64-bit register file, a dedicated scoreboard, and two stream semantic registers.Floating-point operations are pipelined; the subsystem’s separate load/store unit uses address calculations performed by the integer core.Source: §2.1.2 · p. 5

2.2 Snitch Hive

The Snitch Hive groups core complexes around shared instruction and integer-arithmetic resources and supports modular physical design.

2.2 Snitch Hive

  • Cores in a Hive share an instruction cache and a hardware multiply/divide unit.Each core also has a private L0 instruction cache that can refill from the shared L1 cache.Source: §2.2 · p. 5
  • The Hive provides a physical-design boundary with decoupled signals and optional pipeline registers, enabling separately synthesized and placed-and-routed Hives to form larger clusters.Source: §2.2 · p. 5

2.3 Snitch Cluster

The Snitch cluster connects Hives to shared, banked TCDM memory and cluster-level interconnects. Its peripherals expose hardware information and performance counters, while SSR hardware supports configured streaming.

2.3 Snitch Cluster

  • Hives connect to a banked shared memory through the TCDM crossbar and to cluster peripherals and other clusters through the AXI cluster crossbar.The cluster crossbar has both slave and master ports.Source: §2.3 · p. 5

2.3.1 Tightly Coupled Data Memory (TCDM)

  • The TCDM uses a fully connected, combinational crossbar with two memory banks per initiator port to reduce banking conflicts.Requests to a configurable memory range go to the TCDM; other requests use the cluster crossbar, and alternative interconnects can trade off scalability and conflicts.Source: §2.3.1 · p. 5

2.3.2 Cluster Peripherals

  • Read-only peripheral registers expose TCDM bounds, core count, and performance counters; writable registers include scratch registers and a wake-up register that triggers an IPI.Counters include effective FPU utilization, cycle count, and TCDM bank conflicts.Source: §2.3.2 · pp. 5–6
  • SSR hardware maps three input operands and one output operand to two configurable read-or-write lanes, with affine addresses using up to N loop counters.Once configured, it sends memory requests; a credit-based queue hides memory latency.Source: Figure 3 · p. 6

2.4 Stream Semantic Register (SSR)

SSR configures up to two affine memory streams and maps selected floating-point register accesses to those streams. Shadow registers allow the next stream configuration to be prepared while current streaming continues.

2.4 Stream Semantic Register (SSR)

  • SSR configures up to two memory streams with affine address patterns of up to N dimensions, where N depends on the available loops and can be parameterized.The integer core configures each streamer through memory-mapped I/O; other cores cannot write its core-private configuration registers.Source: §2.4 · p. 6
  • When activated through a CSR write, SSR redirects floating-point register accesses to ft0 or ft1 into the corresponding internal queue.Each lane uses a two-phase handshake: the core signals a valid read or write, the lane signals readiness when data is available, and the core signals when it consumes the element.Source: §2.4 · p. 6
  • Shadow registers let the core configure the next memory stream while the current stream is still running, overlapping loop-bound calculation with computation when FREP is used.This extends the SSR configuration scheme.Source: §2.4 · p. 6

2.5 FPU Sequence Buffer

FREP configures a buffer that sequences floating-point instructions independently of the integer core. Its controls specify the instruction group, repetition mode and count, and optional register staggering.

2.5 FPU Sequence Buffer

  • At the off-loading interface, floating-point instructions may use a bypass lane, enter the sequence buffer, or place a new loop configuration in a configuration queue.The optional stagger stage shifts operand names to avoid false dependency stalls and provides software-defined operand renaming.Source: Figure 4 · p. 6
  • The instruction-count field is a 4-bit immediate for up to 16 instructions, while a register identifier supplies the repetition count.The supplied count can represent up to 2^32 iterations.Source: §2.5 · p. 6
  • A four-bit stagger mask selects source or destination operands for register-name increments, and a three-bit stagger count sets the iterations before wrapping.The instruction anatomy and examples show outer and inner repetition patterns with staggered registers.Source: §2.5 · p. 6, Figure 5 · p. 6
  • FREP specifies how many subsequent instructions enter the sequence buffer, how often they repeat, and whether repetition applies to the whole group or each instruction.An outer configuration repeats the group; an inner configuration repeats each instruction before advancing.Source: §2.5 · pp. 6–7
  • Staggering can mitigate stalls from dependencies in short loops by changing register operands as instructions are sequenced through the pipelined FPU.The sequencer can stagger selected operands and wrap their register names after the configured count.Source: §2.5 · p. 7

3 PROGRAMMING

The programming approach combines a RISC-V toolchain and a header-only library with SSR and FREP extensions. In the dot-product example, SSR reduces loop instructions and FREP further sequences floating-point work; the sequencer can also stagger operands to address pipeline dependencies.

3 PROGRAMMING

  • A header-only C library and hand-tuned routines provide an intrinsic-like way to use SSR and FREP, and an LLVM prototype shows automatic SSR setup generation is feasible.The proposed system is a programmable RISC-V multicore with a software-managed TCDM and atomic memory operations.Source: §3 · p. 7

3.2 FPU Sequencer

  • The SSR dot-product kernel runs 2x faster than baseline, while FREP runs 6x faster than baseline and 3x faster than plain SSR.SSR elides loads and tracks one loop counter, saving three instructions; FREP removes innermost-loop instructions, leaving one.Source: Figure 6 · p. 7, §3.2 · p. 8, §3.1 · p. 8

3.1 Stream Semantic Registers

  • For the dot product, SSR elides all loads and tracks one loop counter, saving three instructions and providing a 2x speed-up over vanilla RISC-V.The integer core can prepare the next loop configuration in shadow registers while the current iteration runs.Source: §3.1 · p. 8

3.2 FPU Sequencer

  • FREP lets the FPU sequencer issue repeated floating-point instructions while the integer core continues executing, producing pseudo-dual issue until a blocking event occurs.The subsystems can execute in parallel until data movement or a dependent instruction blocks them.Source: §3.2 · p. 8

3.2.1 Operand Staggering

  • Operand staggering changes selected register names to mitigate dependency stalls in pipelined floating-point operations.The expected multiply-add pipeline depth is two to six stages; the mask selects operands and the count determines when register names wrap.Source: §3.2.1 · p. 8

4 RESULTS

The evaluation uses an eight-core cluster and data-oblivious parallel microkernels spanning arithmetic, machine-learning, memory-bound, and less-linear workloads. The kernels are compared as baseline C, SSR-enabled, and SSR-plus-FREP implementations, with data sizes chosen to fit in TCDM.

4 RESULTS

  • The eight-core cluster contains 128 KiB of TCDM and 8 KiB of instruction cache, and the evaluation uses a cycle-accurate RTL simulation for speed-ups and post-layout simulation for power.The placed-and-routed configuration has two hives of four cores each and is implemented in 22 nm technology.Source: §4 · p. 8, Figure 8 · p. 9, §4.1 · p. 9

4.1 Microkernels

  • The microkernel suite includes dot product, ReLU, matrix multiplication, FFT, AXPY, kNN, 2D convolution, and Monte Carlo estimation of π.The workloads include affine high-reuse convolution, memory-bound AXPY, and less-linear FFT; kNN measurements focus on Euclidean-distance calculation rather than dominant sorting.Source: §4.1 · p. 9
  • AXPY cannot benefit from FREP with only two streamers because three are needed, and its three memory accesses per two floating-point operations exceed each core’s two TCDM memory operations per cycle.The kernel is memory-bound.Source: §4.1 · p. 9
  • For kNN, SSR and FREP accelerate Euclidean-distance calculation, while sorting remains a dominant runtime component not readily accelerated by the extensions.Measurements focus on distance calculation, and the sampling step is distributed across cores.Source: §4.1 · p. 9
  • For Monte Carlo estimation, SSR and FREP can overlap integer random-number generation with floating-point evaluation, but pure SSR has a slight speed-up drop after reformulating the workload for streamers.The reformulation creates dependent floating-point instructions in the FP-SS.Source: §4.1 · p. 9

4.2 Single-Core

SSR and FREP improve single-core execution by reducing explicit memory and control instructions and by allowing floating-point instructions to be sequenced independently of the integer core. The integer core can be configured from 9 to 21 kGE, while the 16-entry FREP implementation adds 3.2% of SoC area.

4.2.1 Performance

  • Matrix multiplication, 2D convolution, kNN distance calculation, and Monte Carlo achieve cumulative IPC above one by overlapping a block’s computation with the next block’s SSR setup and integer instructions.The single-core setting has no shared-TCDM contention.Source: §4.2.1 · pp. 9–10
  • SSR implicitly maps loads and stores to register reads, avoiding integer-core issue slots while the core remains busy issuing arithmetic floating-point instructions.The baseline FPU utilization ranges from 0.14 to 0.36 across benchmarks.Source: §4.2.1 · p. 10
  • FREP raises single-core FPU utilization to 0.57–0.93 and improves speed-up and FPU utilization across microkernels, while FFT sees IPC reduction from setup and load-use stalls.FREP sequences floating-point operations repeatedly from its buffer, reducing integer-core utilization to 0.03–0.24.Source: §4.2.1 · p. 10, Figure 9 · p. 10

4.2.2 Area

  • The integer core ranges from 9 kGE to 21 kGE depending on register-file and PMC configuration, while the 16-entry FREP extension occupies 3.2% of SoC area.SSR hardware is 8.5% of a core complex; the cluster’s integer cores occupy 5% of total cluster area and its FPUs over 23%.Source: §4.2.2 · p. 11, Figure 10 · p. 11, Figure 11 · p. 11

4.2.1 Performance

  • FFT speed-up reaches 2.8× despite reduced FPU utilization from frequent SSR setup and synchronization between stages.The single-core results also note reduced IPC from SSR setup and load-use dependencies.Source: §4.2.1 · p. 10, Figure 13 · p. 11

4.3 Multi-Core

On an octa-core cluster, parallelization and the SSR and FREP extensions improve performance, though shared-memory contention, synchronization, and workload characteristics limit some gains. The measured 32×32 matrix-multiplication power breakdown shows low integer-core and extension power, while energy efficiency rises across microbenchmarks.

4.3.1 Performance

  • Parallelization on eight cores yields speed-ups from 3× to 8×, with ideal eightfold scaling for pure-SSR 2D convolution and the kNN baseline.FFT, dot product, and AXPY scale less well, mostly because small problem sizes magnify reduction and synchronization overhead.Source: §4.3.1 · p. 12
  • SSR and FREP yield multi-core speed-ups from 1.29× to 6.45×, over 94% FPU utilization for 128×128 matrices, and 4.5× performance over vector processors on small matrix multiplication.For larger matrix multiplication problems, performance is equal or better; shared TCDM bank conflicts and Amdahl’s law can reduce gains.Source: §4.3.1 · p. 12, Table 3 · p. 12
  • FFT reaches 2.8× total speed-up despite decreased FPU utilization, attributed to less-linear accesses and frequent synchronization that increases memory-bank contention.Cores can restart fetching from the same bank after synchronization.Source: §4.3.1 · p. 12

4.3.2 Area

  • SSR adds cluster-level interconnect area: the 16-request-port, 32-bank TCDM interconnect occupies 5% of total area, and its crossbar complexity scales with master and slave ports.Estimated crossbars grow to about 630 kGE for 32 request ports and 64 banks, and about 2.5 MGE for 64 request ports and 128 banks.Source: §4.3.2 · p. 12

4.3.3 Energy Efficiency and Power

  • For 32×32 matrix multiplication, the cluster consumes 171 mW; the integer core uses 1% of overall power and SSR and FREP hardware use less than 4% and 1%, respectively.The FPU accounts for 42% of energy in this benchmark.Source: §4.3.3 · p. 12, Figure 14 · p. 12
  • Energy efficiency improves by 1.5 to 4.1 times over baseline across microbenchmarks, reaching nearly 80 DPGflop/s/W and 104 SPGflop/s/W for double-precision matrix multiplication and up to 95 DPGflop/s/W for 2D convolution.The estimated 22 nm peak is 120 DPGflop/s/W, and Snitch reaches 79% of that theoretical peak.Source: §4.3.3 · p. 13, Figure 15 · p. 13

5 RELATED WORK

The related-work comparisons position Snitch against vector, GPU, and superscalar approaches to floating-point efficiency. They report higher efficiency or performance in several comparisons while noting limits to direct cross-system measurement.

5 RELATED WORK

  • The related work contrasts approaches that pursue high floating-point utilization with the paper’s emphasis on energy efficiency.Source: §5 · p. 13

5.1 Vector Architectures

  • An eight-lane Ara configuration matches the cluster’s 16 DPflop/cycle peak, but vector-register access needs shuffle instructions that consume instruction bandwidth and issue slots.Snitch’s byte-wise TCDM access provides implicit shuffle semantics; SSR and FREP support up to four access dimensions without new instructions or changes to the scalar core’s front end and decoding.Source: §5.1 · p. 14
  • On smaller matrix multiplication problems, Snitch outperforms Ara by 4.5× and retains an advantage as problem sizes increase.The vector architecture catches up in performance on larger problems.Source: §5.1 · p. 14
  • On 32 × 32 matrix multiplication, Snitch sustains more than 1.5× Ara’s floating-point operations at twice its energy efficiency: almost 80 versus 40 Gflop/s W.The reported energy-efficiency improvements over Ara are 2.45× for axpy and 2.37× for 2D convolution.Source: Table 4 · p. 13, §5.1 · p. 14
  • Compared with Ara, Snitch’s energy-efficiency gains are attributed mainly to higher area efficiency and a higher compute/control ratio.Source: §5.1 · p. 14

5.2 GPUs

  • On single-precision workloads, the Snitch system exceeds a Tegra Xavier Volta SM’s energy efficiency by over 1.98×, without accounting for technology scaling.The comparison treats one SM quadrant as directly comparable to one Snitch cluster.Source: §5.2 · p. 14

5.3 Super-scalar CPUs

  • Against Tegra Xavier’s eight-core Carmel cluster, Snitch shows more than 10× energy-efficiency improvement for FP32 and 15× for FP64, without accounting for technology scaling.The comparison notes Carmel’s greater hardware cost and lower area efficiency.Source: §5.3 · p. 14
  • The authors could not make accurate direct comparisons with the Green500-winning A64FX system because they lacked access for detailed measurements.The cited system-level figure includes cooling, board, and power supplies.Source: §5.3 · p. 14

6 CONCLUSION

The conclusion presents Snitch as a flexible, energy-efficient multicore system built around a small integer core and floating-point unit. It highlights speedups on data-oblivious kernels and identifies sparse linear algebra and compiler improvements as future directions.

6 CONCLUSION

  • The system pairs a tiny integer core with a double-precision FPU and targets general-purpose computing with high energy efficiency.It uses a TCDM scratchpad for deterministic data placement and energy-efficient data sharing without cache coherence.Source: §6 · pp. 14–15
  • SSRs encode stream accesses through register reads and writes, while FREP sequences floating-point instructions from a loop buffer and frees the integer core for auxiliary tasks.The loop buffer also relieves instruction-cache pressure, saving energy.Source: §6 · p. 15
  • Up to 6.45× speedup is reported on data-oblivious kernels while the system remains fully programmable and avoids specialization to one problem domain.The authors report that eight cores per cluster offer a trade-off between speedup and interconnect complexity.Source: §6 · p. 15
  • The authors identify improved sparse-linear-algebra efficiency, extended benchmarking, and compiler-infrastructure improvements as future research directions.Source: §6 · p. 15
Loading 2002.10143v2…