PaperlayerView exact revision on arXiv

Source-linked AI summary

DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Training

Xiaochen Peng, Shanshi Huang, Hongwu Jiang, Anni Lu, Shimeng Yu

arXiv:2003.06471v1cs.ETcs.LG

TL;DR

On-chip training requires CIM benchmarks to capture synaptic-device non-idealities and training-specific hardware operations beyond inference evaluation. DNN+NeuroSim V2.0 adds these models and circuits, then benchmarks SRAM and eNVM-based accelerators to identify device specifications important for training.

  • Problem

    Existing CIM benchmarking addressed inference, but on-chip training requires evaluating additional synaptic weight-update non-idealities and training hardware.

    Source: §I · p. 1

  • Method

    DNN+NeuroSim V2.0 combines a flexible Python-wrapper interface, device non-ideality models, automatic hardware mapping, and NeuroSim simulation of training circuits and metrics.

    Source: §I · p. 2, §II · p. 3, B. Feed-Forward · p. 6

  • Results

    The benchmark identifies cycle-to-cycle variation, on-state resistance, write-pulse width, and update nonlinearity as important device specifications for on-chip training.

    Source: A. Impacts of Non-ideal Synaptic… · p. 10, D. Benchmark Across Technologies · p. 15, §V · p. 15

  • Takeaways & Limitations

    The reported desired specifications are cycle-to-cycle variation below 1%, on-state resistance above 100 kΩ, write-pulse width below 1μs, and nonlinearity below +3/-3.

    Source: §V · p. 15

Abstract

from arXiv · show

DNN+NeuroSim is an integrated framework to benchmark compute-in-memory (CIM) accelerators for deep neural networks, with hierarchical design options from device-level, to circuit-level and up to algorithm-level. A python wrapper is developed to interface NeuroSim with a popular machine learning platform: Pytorch, to support flexible network structures. The framework provides automatic algorithm-to-hardware mapping, and evaluates chip-level area, energy efficiency and throughput for training or inference, as well as training/inference accuracy with hardware constraints. Our prior work (DNN+NeuroSim V1.1) was developed to estimate the impact of reliability in synaptic devices, and analog-to-digital converter (ADC) quantization loss on the accuracy and hardware performance of inference engines. In this work, we further investigated the impact of the analog emerging non-volatile memory non-ideal device properties for on-chip training. By introducing the nonlinearity, asymmetry, device-to-device and cycle-to-cycle variation of weight update into the python wrapper, and peripheral circuits for error/weight gradient computation in NeuroSim core, we benchmarked CIM accelerators based on state-of-the-art SRAM and eNVM devices for VGG-8 on CIFAR-10 dataset, revealing the crucial specs of synaptic devices for on-chip training. The proposed DNN+NeuroSim V2.0 framework is available on GitHub.

I. INTRODUCTION

DNN+NeuroSim V2.0 extends an inference-oriented CIM benchmarking framework to on-chip training by modeling synaptic-device non-idealities and training-specific hardware. It connects flexible neural-network descriptions to hierarchical hardware evaluation and benchmarks SRAM and eNVM options.

  • CIM addresses extensive data-transfer bottlenecks caused by high-dimensional convolutional layers and large CNN parameter sets.Source: §I · p. 1
  • The framework models nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation during synaptic weight updates.Source: §I · p. 1, Figure 1 · p. 1
  • Training support adds peripheral circuits for error and weight-gradient computation during backpropagation.Source: §I · p. 2
  • DNN+NeuroSim V2.0 extends the framework to evaluate training performance across SRAM and versatile eNVM devices using VGG-8 on CIFAR-10.Source: §I · p. 2
  • The Python wrapper introduces training non-idealities, while NeuroSim automatically maps network structures and estimates hierarchical hardware metrics under hardware-constrained accuracy.Source: §II · pp. 2–3

III. CIM ARCHITECTURE FOR TRAINING

The training architecture divides CIM execution into four sequential steps and uses hierarchical hardware components to support feed-forward, error, weight-gradient, and weight-update operations. Its design trades area and memory overhead against gradient-accumulation latency and training support.

  • Training comprises feed-forward, error computation, weight-gradient computation, and weight update, supported by additional peripheral circuits.Source: §III · p. 3
  • The chip hierarchy contains tiles, global buffers, neural-functional units, SRAM-based weight-gradient arrays, local buffers, interconnects, and computational units.Source: Figure 3 · p. 3, A. CIM Architecture · p. 4
  • Weight-gradient computation uses SRAM-based CIM arrays because frequent writes and vector-matrix multiplication make fast writing more suitable despite SRAM area and leakage costs.Source: A. CIM Architecture · p. 4
  • Mini-batch execution stores activations, errors, and weight gradients across batch size B before producing delta weights for update.Source: A. CIM Architecture · p. 4
  • The framework assumes the four training steps are not pipelined, limiting hardware overhead while potentially increasing latency.Source: A. CIM Architecture · pp. 4–5

B. Feed-Forward

The feed-forward design maps convolutional weights by spatial kernel location into multiple sub-matrices, enabling input reuse and supporting efficient CIM computation. The same mapping also facilitates transposable-array error computation during training.

  • Convolutional feed-forward processes sliding K×K×D input regions against N kernels to produce W×W×N output feature maps under same padding and stride one.Source: B. Feed-Forward · p. 5
  • Weights at different kernel spatial locations are mapped into separate sub-matrices, with K×K sub-matrices representing a K×K kernel.Source: B. Feed-Forward · p. 6
  • This spatial mapping maximizes input-data reuse by directing corresponding input data to the appropriate sub-matrices.Source: B. Feed-Forward · p. 6
  • The mapping is reused for CIM training because spatial kernel grouping makes transposable synaptic arrays easier to implement for efficient error calculation.Source: B. Feed-Forward · p. 6

C. Backpropagation for Error

Backpropagation computes layer errors by reversing the feed-forward dataflow, requiring synaptic arrays that support transposed computation. DNN+NeuroSim V2.0 provides such arrays across SRAM and eNVM technologies with sequential or parallel readout.

Error computation

  • During backpropagation, errors from deeper layers are fetched backward and multiplied and accumulated with prior-layer kernels.Each deeper-layer error is distributed across corresponding channels and kernel locations.Source: C. Backpropagation for Error · pp. 6–7
  • Products are accumulated along CIM-array rows, while partial sums across submatrices are summed to produce layer errors.Errors from layer<n+1> are sent to submatrices according to their spatial locations during kernel sliding.Source: C. Backpropagation for Error · pp. 6–7

Transposable arrays

  • Transposable synaptic arrays support both conventional feed-forward and transposed error computations.The framework offers SRAM, two-terminal eNVM, and three-terminal eNVM implementations with sequential or parallel readout.Source: Figure 7 · p. 6, C. Backpropagation for Error · p. 7

SRAM implementation

  • SRAM arrays use transposed word-lines and bit-lines with 8T-SRAM cells to activate columns and accumulate products along rows.Feed-forward instead activates rows and accumulates products along columns.Source: C. Backpropagation for Error · p. 7

eNVM implementation

  • eNVM arrays reuse their switch matrices to rotate inputs and outputs, adding readout circuitry for horizontal partial-sum accumulation.The added circuitry may include sense amplifiers or ADCs, adders, shift-adders, and registers.Source: C. Backpropagation for Error · p. 7

Design assumption

  • Because feed-forward and backpropagation are not simultaneous, the framework avoids complex logical-control circuitry above the synaptic arrays.Source: C. Backpropagation for Error · p. 7

D. Weight Gradient Caculation

Weight gradients are formed by channel-wise products between activations and backpropagated errors, then accumulated across channels and spatial positions. The framework uses SRAM-based CIM arrays for this workload because gradients require frequent rewriting and vector-matrix multiplication.

Gradient computation

  • Weight gradients combine layer<n> activations with layer<n+1> errors through element-wise multiplication and accumulation in a channel-to-channel scheme.Source: D. Weight Gradient Caculation · p. 7
  • Each channel’s element-wise products is accumulated into the corresponding weight-gradient entry for a kernel and spatial location.Source: D. Weight Gradient Caculation · p. 7

Matrix mapping

  • Errors can be unrolled into D long columns, while activations provide N unrolled input vectors for matrix computation of weight gradients.The resulting group corresponds to weights at a specific spatial location.Source: D. Weight Gradient Caculation · p. 7

Array selection

  • SRAM-based CIM arrays are selected for gradient computation because frequent rewrites and vector-matrix multiplications favor fast writing and effectively infinite endurance.The passage contrasts this with eNVMs’ better area efficiency and SRAM’s standby leakage.Source: D. Weight Gradient Caculation · p. 8

E. Weight Update

Weight gradients are transferred through off-chip memory, accumulated across each batch, and used to update portions of the synaptic arrays. DNN+NeuroSim V2.0 exposes buffer sizing as an area–latency design tradeoff.

Gradient accumulation

  • Per-image weight gradients are sent to off-chip memory, then reloaded and accumulated into delta weights before accelerator-side updates.Source: E. Weight Update · p. 8
  • Updates proceed by selecting a synaptic-array portion and accumulating the corresponding gradients across the batch.Source: E. Weight Update · p. 8

Buffered accumulation

  • Successive buffering overlaps loading, accumulation, and storage, requiring global-buffer capacity of 2×synaptic-array-size×highestpossible accumulated weight gradients precision.Source: E. Weight Update · p. 8

Area–latency trade-off

  • The minimal-buffer assumption limits training area overhead but incurs latency of B× (buffer-read + accumulation + buffer-write).Source: E. Weight Update · p. 8
  • Increasing the buffer overhead constraint enlarges buffers and accumulation units, enabling more synaptic arrays to update simultaneously and reducing accumulation latency.This option lets users seek a design satisfying a specific area constraint.Source: E. Weight Update · p. 8

IV. BENCHMARK RESULTS

The benchmark emphasizes non-ideal synaptic-device properties for in-situ training, evaluating SRAM and eNVM technologies with VGG-8 on CIFAR-10. It specifically examines nonlinearity, asymmetry, and update variation, including momentum optimization.

Training-device effects

  • The benchmark evaluates nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation during in-situ weight updates.Source: §IV · p. 8

Benchmark scope

  • SRAM and versatile eNVM device technologies are benchmarked using VGG-8 on the CIFAR-10 dataset.Source: §IV · p. 8

Optimization comparison

  • Figure 10 analyzes nonlinearity and asymmetry for in-situ training with and without momentum optimization.Source: Figure 10 · p. 9

A. Impacts of Non-ideal Synaptic Device Properties

DNN+NeuroSim V2.0 evaluates how synaptic-device nonidealities affect on-chip training accuracy. Momentum can mitigate nonlinear asymmetry, whereas cycle-to-cycle variation causes substantial degradation.

Evaluation setup

  • The framework sweeps nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation to quantify their individual effects on VGG-8 training accuracy.The evaluation uses CIFAR-10 and fixes the precision of the analog-to-digital converter.Source: A. Impacts of Non-ideal Synaptic… · p. 9

Nonlinearity and asymmetry

Device-to-device variation

  • Device-to-device variation with σ=0.5 does not affect accuracy under momentum optimization and can salvage degradation at larger mean nonlinearity.The variation is modeled by randomly assigning nonlinearities to synaptic weights around a mean value.Source: A. Impacts of Non-ideal Synaptic… · p. 10

Cycle-to-cycle variation

  • Cycle-to-cycle variation of 1%, 3%, or 5% causes significant accuracy drops as variation increases.The variation can reverse an update relative to the desired gradient and induce opposite momentum directions.Source: A. Impacts of Non-ideal Synaptic… · p. 10

B. Hardware Performance Per Epoch

DNN+NeuroSim V2.0 generates per-epoch reports covering hardware area, latency, energy, accuracy, efficiency, and throughput. Weight-gradient computation is the principal training bottleneck because of repeated memory transfers and SRAM writes.

Reporting

  • The framework produces detailed reports for every epoch, including component and operation breakdowns plus summarized accuracy, efficiency, throughput, and weight statistics.The default VGG-8/CIFAR-10 training configuration generates 256 detailed reports.Source: B. Hardware Performance Per Epoch · p. 10
  • The 100th-epoch FeFET-based CIM report includes area, latency, and energy breakdowns by components and operations, including peak synaptic-array computation.Peak metrics exclude off-chip memory, buffers, and interconnect operations.Source: Figure 13 · p. 11

Area

  • The 6-bit flash ADC dominates total area, while weight-gradient units, buffers, and control circuits also occupy substantial area.The weight-gradient units use SRAM-based CIM arrays.Source: B. Hardware Performance Per Epoch · p. 11

Latency and energy

  • Buffer latency and DRAM energy are hardware-performance bottlenecks because training involves extensive on-chip data transfer.Feed-forward and error computation have similar operation schemes, data volumes, and hardware resources.Source: B. Hardware Performance Per Epoch · p. 11

Operation bottlenecks

  • Weight-gradient computation dominates total latency and energy because it repeatedly loads activations and errors, writes SRAM arrays, and transfers gradients off-chip.The repeated off-chip memory accesses and SRAM writes make gradient computation the bottleneck.Source: B. Hardware Performance Per Epoch · p. 11
  • Weight update contributes little averaged latency and energy because it occurs once per batch, whereas the other operations run for every input.The benchmark batch size is 200, so weight-update cost is averaged by 200× per epoch.Source: B. Hardware Performance Per Epoch · p. 11

Layer-wise behavior

  • Across VGG-8 layers, feed-forward and error-computation trends follow activation size, while weight-update latency and energy depend only on weight size.Deeper layers can accelerate gradient computation by duplicating smaller errors and fetching multiple activations simultaneously.Source: B. Hardware Performance Per Epoch · p. 12

C. Hardware Performance Across Epoch

The framework tracks layer-wise weights, activities, delta weights, latency, and energy across training epochs, linking hardware behavior to changing conductance and convergence. FeFET and EpiRAM designs show distinct epoch-dependent trade-offs between latency, energy, currents, and weight-update cost.

Trace-Based Evaluation

  • Real traces of activations and updated weights are used per epoch to estimate feed-forward and error-computation hardware performance.Weight-gradient estimation uses a pseudo-traced method, while weight-update estimation accesses old and updated weights.Source: C. Hardware Performance Across Epoch · pp. 12–13

FeFET Training Traces

  • FeFET analysis tracks weight, delta-weight, and input-activity distributions alongside peak latency, energy, and throughput across epochs.The traces are normalized by activation and weight size to account for differing hardware resources and computation counts.Source: C. Hardware Performance Across Epoch · p. 13
  • After 200th epoch, increasing weights and activities raise column currents, decreasing latency while increasing dynamic energy through more activated peripheral transistors.Delta-weight means approach zero after 200th epoch as the network converges, while peak throughput increases significantly.Source: C. Hardware Performance Across Epoch · pp. 13–14
  • Before 200th epoch, decreasing normalized weights and activities reduce column currents, increasing synaptic-array latency and energy.Higher resistance lengthens delay and increases ADC delay rapidly enough to raise overall energy despite lower column-current power.Source: C. Hardware Performance Across Epoch · pp. 13–14

EpiRAM Comparison

  • EpiRAM’s lower on-state resistance produces decreasing normalized weights and increasing input activities before 200th epoch, unlike the FeFET trend.Its higher write voltage and pulse width also increase weight-update energy relative to FeFET.Source: C. Hardware Performance Across Epoch · p. 14

D. Benchmark Across Technologies

The benchmark compares SRAM and eNVM CIM accelerators for VGG-8 training on CIFAR-10 across technology nodes and device precisions. It identifies on-state resistance, write-pulse width, cycle-to-cycle variation, and SRAM scaling as key determinants of hardware performance and training accuracy.

Benchmark Scope

  • The benchmark evaluates sequential and parallel SRAM accelerators at 7nm and 32nm alongside parallel-read eNVM accelerators at 32nm.Device options include reported analog synaptic technologies, with VGG-8 trained on CIFAR-10.Source: D. Benchmark Across Technologies · p. 14, Table I · p. 14

Precision Settings

  • Device precision determines network precision, spanning 5-bit for 32 conductance levels to 7-bit for 128 levels.Digital SRAM designs are evaluated at 5-bit specifications when using 32 levels.Source: D. Benchmark Across Technologies · pp. 14–15

Device and Circuit Trade-offs

  • On-state resistance affects area, latency, and throughput because avoiding voltage drop requires larger transistors, increasing capacitance loading.The benchmark identifies on-state resistance as an important hardware-performance factor.Source: D. Benchmark Across Technologies · p. 15
  • Write pulse widths below 1μs avoid detrimental speed effects because weight-update cost is averaged over the batch size.Cycle-to-cycle variation above the preferred range can drive opposite momentum updates and prevent learning.Source: D. Benchmark Across Technologies · p. 15
  • A preferred cycle-to-cycle variation is lower than 1%, while advanced 7nm parallel-read SRAM shows superior energy efficiency and throughput despite leakage and area overhead.The SRAM overhead comparison is stated for designs at the same technology node.Source: D. Benchmark Across Technologies · p. 15

V. CONCLUSION

DNN+NeuroSim V2.0 extends end-to-end CIM benchmarking to on-chip training with flexible networks and device options. Its benchmark identifies target analog-device specifications for training-aware accelerator design.

Framework Contribution

  • DNN+NeuroSim V2.0 benchmarks CIM architectures for on-chip training while supporting flexible network structures and versatile device options.The framework models device nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation during weight updates.Source: §V · p. 15

Technology Targets

  • Momentum optimization helps investigate analog synaptic-device non-idealities during in-situ training, with desired specifications of variation below 1%, resistance above 100 kΩ, and pulse width below 1μs.The reported desired nonlinearity is below +3/-3.Source: §V · p. 15

Related papers

Loading 2003.06471v1…