PaperlayerView exact revision on arXiv

Source-linked AI summary

Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode

Davide Rossi, Francesco Conti, Manuel Eggimann, Alfio Di Mauro, Giuseppe Tagliavini, Stefan Mach, Marco Guermandi, Antonio Pullini, Igor Loi, Jie Chen, Eric Flamand, Luca Benini

arXiv:2110.09101v1cs.ARcs.AI

TL;DR

IoT end-nodes need low-power always-on operation as well as efficient, flexible computation for complex near-sensor analytics. Vega combines programmable cognitive wake-up with multi-precision processing and reports up to 32.2 GOPS at 1.3 TOPS/W on convolutional workloads.

  • Problem

    IoT end-nodes must sustain low-power always-on sensing while handling complex, fast-evolving near-sensor analytics that exceed the flexibility of specialized accelerators.

    Source: §I · p. 1, B. Cognitive Wake-Up Unit · p. 3

  • Method

    Vega combines MRAM-supported programmable cognitive wake-up with a multi-core SoC supporting multi-precision computation and programmable machine-learning accelerators.

    Source: §I · p. 1, §VI · p. 11

  • Results

    32.2 GOPS at up to 1.3 TOPS/W on convolutional workloads with hardware acceleration, within a 49.4 mW power envelope.

    Source: §III · p. 7

  • Takeaways & Limitations

    Vega demonstrates configurable cognitive wake-up and programmable processing across near-sensor analytics, including mobile DNN inference.

    Source: §I · p. 1, §V · p. 10

Abstract

from arXiv · show

The Internet-of-Things requires end-nodes with ultra-low-power always-on capability for a long battery lifetime, as well as high performance, energy efficiency, and extreme flexibility to deal with complex and fast-evolving near-sensor analytics algorithms (NSAAs). We present Vega, an IoT end-node SoC capable of scaling from a 1.7 $\mathrm{\mu}$W fully retentive cognitive sleep mode up to 32.2 GOPS (@ 49.4 mW) peak performance on NSAAs, including mobile DNN inference, exploiting 1.6 MB of state-retentive SRAM, and 4 MB of non-volatile MRAM. To meet the performance and flexibility requirements of NSAAs, the SoC features 10 RISC-V cores: one core for SoC and IO management and a 9-cores cluster supporting multi-precision SIMD integer and floating-point computation. Vega achieves SoA-leading efficiency of 615 GOPS/W on 8-bit INT computation (boosted to 1.3TOPS/W for 8-bit DNN inference with hardware acceleration). On floating-point (FP) compuation, it achieves SoA-leading efficiency of 79 and 129 GFLOPS/W on 32- and 16-bit FP, respectively. Two programmable machine-learning (ML) accelerators boost energy efficiency in cognitive sleep and active states, respectively.

I. INTRODUCTION

IoT end-nodes must combine low-power always-on sensing with efficient, flexible computation for increasingly intelligent near-sensor analytics. Vega addresses these demands with MRAM-supported cognitive wake-up and programmable multi-precision processing, evaluated across several analytics domains.

I. INTRODUCTION

  • Near-sensor analytics need low-power sleep alongside high-performance, energy-efficient active computation, motivating a new class of IoT processors.State-retentive memory supports autonomous wake-up after always-on sensors detect a condition.Source: §I · p. 1
  • Specialized DNN accelerators can be highly efficient, but their inflexibility does not match the variety and rapid evolution of near-sensor analytics algorithms.Prior work also explores low-precision and tunable-precision arithmetic to reduce energy use.Source: §I · p. 1
  • Vega combines a 1.7 µW cognitive wake-up unit backed by 4 MB of MRAM with dynamically configurable integer and floating-point processing.The authors demonstrate it on bio-signal, audio/vibration, imaging, and DNN inference workloads.Source: §I · p. 1

II. VEGA SOC ARCHITECTURE

Vega partitions its SoC into four switchable power domains, separating the MCU and SRAM, programmable cluster, MRAM, and cognitive wake-up unit.

II. VEGA SOC ARCHITECTURE

  • Four switchable power domains contain the MCU and SRAM, programmable cluster, 4 MB MRAM, and Cognitive Wake-Up unit.The SoC domain also includes peripherals.Source: §II · p. 2

A. Non-Volatile SoC Subsystem

Vega’s memory subsystem combines embedded SRAM and independently powered MRAM, with DMA-mediated transfers and selectable retention to balance sleep power against wake-up restoration.

A. Non-Volatile SoC Subsystem

  • The Fabric Controller uses 1.7 MB of embedded SRAM and connects to external DRAM and Flash through a 1.6 Gbit/s HyperBus/OCTA SPI/SDIO DDR interface.Source: A. Non-Volatile SoC Subsystem · p. 2
  • Each peripheral has a dedicated DMA channel for autonomous transfers to and from L2 memory.Source: A. Non-Volatile SoC Subsystem · p. 2
  • 4 MB of MRAM stores machine-learning parameters and program code, and master resources access its contents after transfer to L2 memory.An auxiliary I/O DMA channel connects MRAM to L2; a dedicated controller handles MRAM read/write protocol conversion.Source: A. Non-Volatile SoC Subsystem · p. 2
  • The L2 memory has four word-level interleaved banks totaling 1.5 MB, plus 64 kB of private core memory.Source: A. Non-Volatile SoC Subsystem · p. 2
  • The Cognitive Wake-Up Unit architecture comprises an autonomous SPI master, configurable preprocessor, and HDC accelerator.Source: Figure 2 · p. 3
  • SRAM retention ranges from 1.2 to 112 µW for 16 kB to 1.6 MB, while restoring from MRAM avoids sleep retention power but requires copying the program into L2 after wake-up.The choice depends on the application’s duty cycle and wake-up latency requirement.Source: A. Non-Volatile SoC Subsystem · p. 3

B. Cognitive Wake-Up Unit

Vega’s Cognitive Wake-Up Unit (CWU) autonomously classifies external sensor data using programmable HDC and can trigger the PMU to power up the cluster for further analytics. Its measured power and comparison with other smart wake-up units characterize its low-power, configurable operation.

B. Cognitive Wake-Up Unit

  • The CWU runs end-to-end machine learning on external sensor data and triggers the PMU to power up the cluster for more advanced analytics.After initial configuration, it continuously processes and classifies sensor data without further core interaction.Source: B. Cognitive Wake-Up Unit · p. 3
  • The CWU combines a programmable SPI master, a low-power preprocessor, and a configurable HDC accelerator.Preprocessing can convert data formats, remove offsets, filter, subsample, and apply local-binary-pattern filtering.Source: B. Cognitive Wake-Up Unit · p. 3
  • HDC encodes sensor time series as high-dimensional binary search vectors, which are compared with class prototype vectors in associative memory.Hypnos supports 512-, 1024-, 1536-, or 2048-bit vectors using a 512-bit datapath; its encoder rematerializes item memory using hardwired random permutations.Source: B. Cognitive Wake-Up Unit · p. 4
  • At 32 kHz, the CWU consumes 2.97 µW during real-time HDC inference; at 200 kHz, it consumes 14.9 µW.The measurements use data from three SPI peripherals at 16-bit, 150 samples/s per channel at 32 kHz and 1 ksample/s per channel at 200 kHz.Source: B. Cognitive Wake-Up Unit · p. 4, Table I · p. 4
  • Among general-purpose smart wake-up units, Hypnos offers the highest reported configurability while using similar power to a less-flexible threshold-sequence design.The comparison concerns general-purpose solutions; the other listed designs primarily implement application-specific algorithms [12],,.Source: B. Cognitive Wake-Up Unit · p. 4, Table II · p. 5

C. Parallel Compute Cluster

Vega’s nine-core parallel cluster combines multi-precision RISC-V processing with shared FPUs and a convolution accelerator. Its HWCE supports several precisions and convolution configurations, including up to 19 MAC/cycle on a 3×3 convolutional layer.

C. Parallel Compute Cluster

  • The cluster has nine RISC-V cores sharing 128 kB of L1 memory, with a ninth core commonly managing DMA while the other eight handle computation.The banked L1 can serve 16 parallel memory requests and deliver up to 28.8 GB/s at 450 MHz.Source: C. Parallel Compute Cluster · p. 5
  • The cores support SIMD integer operations and FP32, FP16, and bfloat formats through four shared FPUs, with multi-format operations such as 16-bit multiplication returning a 32-bit result.FPU access uses a static mapping that limits cross-core sharing flexibility but reduces interconnect complexity and preserves single-cycle FP instruction latency.Source: C. Parallel Compute Cluster · p. 5
  • The HWCE streams pixels from L1 into a line-buffered sliding window and uses shared sum-of-products datapaths that can also be configured for 5×5 convolutions.Fine-grain data and clock gating can disable unused reduction-tree branches; partial-sum buffers support accumulation across input channels.Source: C. Parallel Compute Cluster · p. 6, Figure 4 · p. 6
  • 19 MAC/cycle is achieved on a 3×3 convolutional layer, a configuration suited to VGG-style networks dominated by 3×3 convolutions.The HWCE accelerates multi-precision 4b/8b/16b 3×3 convolution using 27 MACs in total.Source: C. Parallel Compute Cluster · pp. 5–6

III. CHIP IMPLEMENTATION AND MEASUREMENTS

Vega spans a 1.7 µW cognitive sleep mode and active computation across its FC and cluster. In the reported 8-bit workloads, the HWCE-assisted cluster reaches 32.2 GOPS within a 49.4 mW power envelope.

III. CHIP IMPLEMENTATION AND MEASUREMENTS

  • 32.2 GOPS at 1.3 TOPS/W is achieved on convolutional workloads with the HWCE; general-purpose processors reach 15.6 GOPS at 614 GOPS/W, within a 49.4 mW power envelope.The measurements use an 8-bit 3×3 convolution on the HWCE and an 8-bit matrix-multiplication kernel from PULP-NN on software-programmable cores.Source: Figure 6 · p. 7, §III · p. 7
  • The SoC uses four main power domains, and its two programmable accelerators occupy less than 15% of total area.The always-on power domain includes the PMU that switches the other domains on and off.Source: §III · pp. 6–7
  • In cognitive sleep, the CWU consumes 1.7 µW at 32 kHz, rising to 20.9 µW when 128 kB of L2 memory is retained.The SoC’s active-state power ranges from 0.7 to 15 mW, delivering up to 1.9 GOPS at up to 200 GOPS/W on 8-bit integer computation.Source: §III · p. 7, Figure 7 · p. 7

IV. BENCHMARKING

The benchmarking section covers general-purpose near-sensor analytics using 32-bit and 16-bit floating-point arithmetic, as well as DNN workloads.

IV. BENCHMARKING

  • The Vega SoC benchmarking covers general-purpose near-sensor analytics using 32-bit and 16-bit FP arithmetic, alongside DNN workloads.Source: §IV · p. 8

A. Floating-Point NSAA Workloads

Vega evaluates floating-point workloads spanning DSP, machine learning, and linear algebra, using shared multi-precision FPUs. Vectorial FP operations improve performance over scalar operations, with fused multiply-and-accumulate benefiting several kernels.

A. Floating-Point NSAA Workloads

  • 1.46×: vectorial FP operations improve performance over scalar operations, below theoretical speedup because ALU and control operations are not vectorized.FP and memory operations benefit from vectorization; ALU and control operations do not.Source: A. Floating-Point NSAA Workloads · p. 8
  • MATMUL, FFT, and FIR gain above-average performance and efficiency from fused multiply and accumulate, which performs 2 FP operations per cycle.The register-file design maintains this gain in vectorial FP16 versions.Source: A. Floating-Point NSAA Workloads · p. 8
  • The benchmark set covers DSP, machine-learning, and basic linear-algebra kernels used in audio, image, and ExG processing.Performance and efficiency are presented for FP NSAA workloads.Source: A. Floating-Point NSAA Workloads · p. 8, Figure 8 · p. 8

B. DNN NSAA Workloads and Data Flow

Vega runs tiled, 8-bit DNN inference with a double-buffered software pipeline and can place weights in on-chip MRAM or external HyperRAM. In MobileNetV2, MRAM substantially reduces inference energy, while HWCE acceleration benefits depend on network structure.

B. DNN NSAA Workloads and Data Flow

  • DNN inference uses four overlapped, double-buffered stages: copy weights, copy input and weight tiles, compute, and copy out activations.Eight cluster cores compute; a ninth orchestrates tiling and data movement, with DORY generating tiling solutions and orchestrator code.Source: B. DNN NSAA Workloads and Data Flow · pp. 8–9
  • Over 40×: on-chip MRAM provides better energy efficiency per byte than HyperRAM, while both offer similar bandwidth.SRAM accesses provide much higher bandwidth at low energy per byte, but at a steep area cost and without state retention.Source: B. DNN NSAA Workloads and Data Flow · p. 9
  • MobileNetV2 runs at 224×224 input with depth multiplier 1.0; all layers except the final one are compute-bound.Front layers have more activation transfers, while back-end layers have more MRAM weight transfers.Source: B. DNN NSAA Workloads and Data Flow · p. 9, Figure 10 · p. 9
  • 3.5×: MobileNetV2 total energy per inference falls from 4.16 mJ to 1.19 mJ with weights on MRAM rather than HyperRAM.Inference time is essentially unchanged; the energy reduction comes from a 40× lower memory-access cost, as HyperRAM accounts for almost 25% of total energy in the legacy flow.Source: B. DNN NSAA Workloads and Data Flow · p. 9, Figure 11 · p. 9
  • HWCE acceleration is poorly suited to 1×1-convolution-dominated MobileNetV2: speeding up depthwise 3×3 convolutions by about 3× yields only about 5% whole-network speedup.The paper identifies VGG-style networks dominated by 3×3 convolutions as a better fit for HWCE filter reuse.Source: B. DNN NSAA Workloads and Data Flow · pp. 9–10
  • RepVGG-A0 networks exceed on-chip MRAM capacity, so Vega keeps early-layer weights in MRAM and places later-layer weights in HyperRAM.The networks use stages of 3×3 convolutions followed by a fully connected layer.Source: B. DNN NSAA Workloads and Data Flow · p. 10, Table VII · p. 10
  • 3×: HWCE-based execution speeds up the MobileNetV2 case study over software and boosts system-level energy efficiency by 60–90%, depending on HyperRAM traffic.Almost all layers are compute-dominated except the final fully connected layer.Source: B. DNN NSAA Workloads and Data Flow · pp. 8, 10

V. COMPARISON WITH S.O.A.

Vega is compared with programmable embedded platforms and low-power IoT systems across NSAA and DNN workloads. The reported comparisons emphasize performance and efficiency gains over related multicore and hardware-accelerated IoT end-nodes, alongside flexible FP support.

V. COMPARISON WITH S.O.A.

  • On a matrix-multiplication benchmark, the vector processor in has only 1.62× and 1.16× better peak energy efficiency than Vega for FP16 and FP32, respectively.The passage notes that has much higher absolute performance, with larger area and higher operating frequency.Source: §V · p. 10
  • Vega supports low-precision FP formats including 16-bit and bfloat, and the authors report leading-edge FP efficiency across 32-bit and 16-bit FP NSAA workloads.The comparison passage cautions that vector-processor efficiency can degrade on small datasets and irregular NSAA patterns.Source: §V · p. 10
  • Compared with traditional low-end IoT end-nodes such as, Vega provides orders-of-magnitude better active-mode performance and efficiency, enabling complex NSAAs not feasible on those systems.The passage says low-power-state optimization is the main focus of most IoT MCU research, rather than a key contribution of Vega.Source: §V · p. 10
  • Against multicore IoT end-nodes,, Vega delivers more than 1.3× peak performance and more than 3.2× peak efficiency on non-DNN NSAA workloads.The paper attributes the gains, in part, to integration between the core prefetch buffer and hierarchical instruction cache.Source: §V · p. 10, Table VIII · p. 11
  • Against Mr.Wolf, Vega delivers 2× peak performance and 4.3× peak efficiency on 32-bit single-precision FP workloads.The passage attributes these gains to the shared FP unit’s support for single-cycle NSAA operations such as fused multiply and accumulate.Source: §V · p. 10
  • Against hardware-accelerated IoT end-nodes, Vega achieves similar DNN-inference energy efficiency at 5.5× better performance.On non-DNN NSAA workloads, Vega achieves 10× higher performance and 2.5× higher energy efficiency.Source: §V · p. 10
  • The authors describe Vega as the only IoT end-node with a configurable 1.7 µW cognitive wake-up unit capable of fully on-chip execution of mobile DNNs from non-volatile memory.MobileNetV2 and RepVGG are named examples.Source: §V · p. 10

VI. CONCLUSION

Vega combines a fully retentive cognitive wake-up unit with a scalable SoC for near-sensor analytics and mobile DNN inference. Its reported capabilities include up to 32.2 GOPS peak performance and fully on-chip execution of MobileNetV2 and RepVGG-A0.

VI. CONCLUSION

  • Vega combines a 1.7 µW fully retentive cognitive wake-up unit with a scalable SoC that reaches up to 32.2 GOPS at 49.4 mW on NSAAs.The system includes 1.6 MB of state-retentive SRAM, 4 MB of MRAM, one management core, and a nine-core cluster supporting multi-precision SIMD integer and FP computation.Source: §VI · p. 11
  • Vega’s two programmable ML accelerators target energy efficiency in sleep and active states, respectively.The SoC reports peak efficiency up to 1.3 TOPS/W and fully on-chip inference at 1.19 mJ for MobileNetV2 and 4.4 mJ for RepVGG-A0.Source: §VI · p. 11

Related papers

Loading 2110.09101v1…