Source-linked AI summary
Bio-inspired Learning and Decision-Making with Probabilistic In-Memory Computing Hardware: Part 2
Thomas Dalgaty, Eiji Kawasaki, Miguel de Prado, Tommaso Salvatori, Germain Haugou, Eric Flamand
TL;DR
Low-arithmetic-intensity probabilistic energy-based models face execution-latency challenges on architectures that rely on external-memory interfaces. This report maps the Part 1 model to a probabilistic AIMC processor in a transaction-level model and reports over 1000× faster execution than data-center GPUs and TPUs.
Problem
Low-arithmetic-intensity, massively parallel probabilistic models face execution-latency challenges on processor architectures that rely on external-memory interfaces.
Source: Introduction · p. 1, Results and Outlook · p. 3Method
The report implements the Part 1 energy-based model on a probabilistic AIMC processor using a transaction-level model.
Source: Introduction · pp. 1–2Results
Over 1000× faster execution than data-center GPUs and TPUs is reported for the energy-based model on the AIMC processor model.
Source: Introduction · p. 2Takeaways & Limitations
AIMC combines analogue-memory sampling and vector-matrix multiplication without further data movement, providing a structurally different approach to this computation.
Source: Results and Outlook · p. 3The report does not focus on a specific memory technology and assumes each physical sample can be represented as at least a 16-bit integer.
Source: AIMC Processor Transaction-Level Model · p. 2
Abstract
from arXiv · showhide
This report extends our previous work (Part 1), which introduced an energy-based model for learning and decision-making under uncertainty. The model leverages stochastic Langevin dynamics to continuously evolve approximate probability distributions over neuron states and model weights. However, as noted in Part 1 and confirmed through GPU-based implementations, large-scale probabilistic energy-based models of this nature face significant scalability challenges due to excessive execution latency. This latency stems from a fundamental mismatch: massively parallel models with low arithmetic intensity (such as energy-based models) are being executed on processor architectures like GPUs that rely on high-bandwidth memory (HBM) interfaces. The HBM imposes brutally sequential execution constraints on inherently parallelizable models, creating the false impression that such models are unscalable. In reality, it is the GPU architecture itself, with its dependence on HBM interfaces, that is not a scalable processor architecture for this class of AI model. In this report, we demonstrate using a detailed transaction-level model (TLM) of a probabilistic analogue in-memory computing (AIMC) processor that the same energy-based model can execute well over 1000x faster than data-center-grade hardware by eliminating the HBM interface and performing computation directly within on-chip memory.
Introduction
The report describes the external-memory interface in GPU architectures and motivates bringing model data closer to computation. It maps the Part 1 energy-based model to probabilistic AIMC and reports over 1000× faster execution than data-center GPUs and TPUs.
Introduction
- GPUs use off-chip DRAM and on-chip SRAM, with an external-memory interface identified as a bottleneck in the GPU hierarchy.The passage identifies the modern external-memory interface as high-bandwidth memory (HBM).Source: Introduction · p. 1
- Near-memory, wafer-scale, and in-memory approaches seek to reduce the HBM bottleneck by bringing model data into on-chip memory.The passage says these approaches can provide at least an order of magnitude higher bandwidth in a distributed fashion.Source: Introduction · p. 1
- Probabilistic AIMC performs computation in the same memory devices that store model data, while using intrinsic noise and variability to generate controlled probability distributions.The report presents probabilistic in-memory computing as a way to use analogue error sources in the algorithm.Source: Introduction · p. 1
- Over 1000× faster execution than data-center GPUs and TPUs is reported for the Part 1 energy-based model on the AIMC processor model.The report uses a transaction-level model of the processor.Source: Introduction · p. 2
AIMC Processor Transaction-Level Model
The transaction-level model combines analogue memory arrays, peripheral circuitry, RISC-V cores, and software support. It assumes a minimum sample precision and models the system as a non-cycle-accurate framework with latency estimates.
AIMC Processor Transaction-Level Model
- The processor model uses 512×512 analogue memory arrays and assumes each physically stored sample can be represented as at least a 16-bit integer.The report does not focus on a specific memory technology.Source: AIMC Processor Transaction-Level Model · p. 2
- Four memory planes share DACs, ADCs, and address drivers, with an adjacent Spatz RISC-V core connected through AXI and DMA.The DMA cycles data between analogue memory and the core at a rate matched to the analogue memory interface.Source: AIMC Processor Transaction-Level Model · p. 2
- Analogue reads and writes, DMA transfers, and RISC-V processing run in parallel; the Spatz vector extensions process array data for subsequent operations.Without the vector extensions, the core would become a bottleneck relative to the array bandwidth.Source: AIMC Processor Transaction-Level Model · p. 2
- The register file separates SAMPLE distribution parameters, DATA input vectors, and RESULT outputs for analogue-memory operations.The core triggers operations by writing to memory-mapped registers.Source: AIMC Processor Transaction-Level Model · p. 2
- The TLM uses GVSOC and open-source IP, and provides rapid design exploration with latency estimates despite not being cycle-accurate.The model includes a 16×16 cluster array, and its 10GB analogue memory provides bandwidth in excess of some PB/s.Source: AIMC Processor Transaction-Level Model · p. 2
EBM Model and Mapping Overview
The mapped energy-based model is a large, replicated particle system requiring 10 GB of memory. Its operations use analogue arrays to sample distributions and support vector-matrix computation, with pipelined kernels cycling data between the arrays and core.
EBM Model and Mapping Overview
- The EBM has five layers of 256 neurons, replicated across 64 chains of 96 particles, each retaining a sliding window of 512 samples.The model requires 10 GB of memory.Source: EBM Model and Mapping Overview · p. 2
- The mapping uses analogue arrays to sample 128-element vectors from specified distributions, then processes them for scaling and vector-matrix operations.The EBM operations include scalar products for Langevin gradients, outer products for weight updates, element-wise scaling, and Gaussian noise sampling.Source: EBM Model and Mapping Overview · p. 2
- Pipelined software kernels cycle outputs from the AIMC array through the Spatz core and back to the array for sampling or vector-matrix products.The report describes an active-inference example that continuously updates distributions over an agent’s latent world model and internal generative model.Source: EBM Model and Mapping Overview · p. 2
Results and Outlook
Both EBM kernels run at least three orders of magnitude faster on the AIMC processor than on the GPU and TPU, in the reported comparison. The evaluation times neuron-distribution sampling and weight updates against data-center-grade processors.
Results and Outlook
- The two kernels generate neuron-distribution samples across all particles and update particle weights using the most recent sample in the sliding window.The timing results cover both operations.Source: Results and Outlook · pp. 2–3
- The comparison benchmarks kernels on an Nvidia A100 GPU and Google v6e TPU against the AIMC processor, using 16-bit Bfloat on GPU and TPU.The Bfloat format matches the analogue memory samples’ 16-bit data format.Source: Results and Outlook · p. 3
- At least three orders of magnitude faster execution is reported for both EBM kernels on AIMC than on the GPU and TPU.Table 1 presents the execution-time comparison across architectures.Source: Table 1 · p. 3, Results and Outlook · p. 3