PaperlayerView exact revision on arXiv

Source-linked AI summary

DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Training

Xiaochen Peng, Shanshi Huang, Hongwu Jiang, Anni Lu, Shimeng Yu

arXiv:2003.06471v1cs.ETcs.LG

TL;DR

On-chip training requires CIM benchmarks to capture synaptic-device non-idealities and training-specific hardware operations beyond inference evaluation. DNN+NeuroSim V2.0 adds these models and circuits, then benchmarks SRAM and eNVM-based accelerators to identify device specifications important for training.

  • Problem

    Existing CIM benchmarking addressed inference, but on-chip training requires evaluating additional synaptic weight-update non-idealities and training hardware.

  • Method

    DNN+NeuroSim V2.0 combines a flexible Python-wrapper interface, device non-ideality models, automatic hardware mapping, and NeuroSim simulation of training circuits and metrics.

  • Results

    The benchmark identifies cycle-to-cycle variation, on-state resistance, write-pulse width, and update nonlinearity as important device specifications for on-chip training.

  • Takeaways & Limitations

    The reported desired specifications are cycle-to-cycle variation below 1%, on-state resistance above 100 kΩ, write-pulse width below 1μs, and nonlinearity below +3/-3.

Abstract

from arXiv · show

DNN+NeuroSim is an integrated framework to benchmark compute-in-memory (CIM) accelerators for deep neural networks, with hierarchical design options from device-level, to circuit-level and up to algorithm-level. A python wrapper is developed to interface NeuroSim with a popular machine learning platform: Pytorch, to support flexible network structures. The framework provides automatic algorithm-to-hardware mapping, and evaluates chip-level area, energy efficiency and throughput for training or inference, as well as training/inference accuracy with hardware constraints. Our prior work (DNN+NeuroSim V1.1) was developed to estimate the impact of reliability in synaptic devices, and analog-to-digital converter (ADC) quantization loss on the accuracy and hardware performance of inference engines. In this work, we further investigated the impact of the analog emerging non-volatile memory non-ideal device properties for on-chip training. By introducing the nonlinearity, asymmetry, device-to-device and cycle-to-cycle variation of weight update into the python wrapper, and peripheral circuits for error/weight gradient computation in NeuroSim core, we benchmarked CIM accelerators based on state-of-the-art SRAM and eNVM devices for VGG-8 on CIFAR-10 dataset, revealing the crucial specs of synaptic devices for on-chip training. The proposed DNN+NeuroSim V2.0 framework is available on GitHub.

I. INTRODUCTION

DNN+NeuroSim V2.0 extends an inference-oriented CIM benchmarking framework to on-chip training by modeling synaptic-device non-idealities and training-specific hardware. It connects flexible neural-network descriptions to hierarchical hardware evaluation and benchmarks SRAM and eNVM options.

  • CIM addresses extensive data-transfer bottlenecks caused by high-dimensional convolutional layers and large CNN parameter sets.
  • The framework models nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation during synaptic weight updates.
  • Training support adds peripheral circuits for error and weight-gradient computation during backpropagation.
  • DNN+NeuroSim V2.0 extends the framework to evaluate training performance across SRAM and versatile eNVM devices using VGG-8 on CIFAR-10.
  • The Python wrapper introduces training non-idealities, while NeuroSim automatically maps network structures and estimates hierarchical hardware metrics under hardware-constrained accuracy.

III. CIM ARCHITECTURE FOR TRAINING

The training architecture divides CIM execution into four sequential steps and uses hierarchical hardware components to support feed-forward, error, weight-gradient, and weight-update operations. Its design trades area and memory overhead against gradient-accumulation latency and training support.

  • Training comprises feed-forward, error computation, weight-gradient computation, and weight update, supported by additional peripheral circuits.
  • The chip hierarchy contains tiles, global buffers, neural-functional units, SRAM-based weight-gradient arrays, local buffers, interconnects, and computational units.
  • Weight-gradient computation uses SRAM-based CIM arrays because frequent writes and vector-matrix multiplication make fast writing more suitable despite SRAM area and leakage costs.
  • Mini-batch execution stores activations, errors, and weight gradients across batch size B before producing delta weights for update.
  • The framework assumes the four training steps are not pipelined, limiting hardware overhead while potentially increasing latency.

B. Feed-Forward

The feed-forward design maps convolutional weights by spatial kernel location into multiple sub-matrices, enabling input reuse and supporting efficient CIM computation. The same mapping also facilitates transposable-array error computation during training.

  • Convolutional feed-forward processes sliding K×K×D input regions against N kernels to produce W×W×N output feature maps under same padding and stride one.
  • Weights at different kernel spatial locations are mapped into separate sub-matrices, with K×K sub-matrices representing a K×K kernel.
  • This spatial mapping maximizes input-data reuse by directing corresponding input data to the appropriate sub-matrices.
  • The mapping is reused for CIM training because spatial kernel grouping makes transposable synaptic arrays easier to implement for efficient error calculation.

C. Backpropagation for Error

Backpropagation computes layer errors by reversing the feed-forward dataflow, requiring synaptic arrays that support transposed computation. DNN+NeuroSim V2.0 provides such arrays across SRAM and eNVM technologies with sequential or parallel readout.

  • Error computation: During backpropagation, errors from deeper layers are fetched backward and multiplied and accumulated with prior-layer kernels.Each deeper-layer error is distributed across corresponding channels and kernel locations.
  • Error computation: Products are accumulated along CIM-array rows, while partial sums across submatrices are summed to produce layer errors.Errors from layer<n+1> are sent to submatrices according to their spatial locations during kernel sliding.
  • Transposable arrays: Transposable synaptic arrays support both conventional feed-forward and transposed error computations.The framework offers SRAM, two-terminal eNVM, and three-terminal eNVM implementations with sequential or parallel readout.
  • SRAM implementation: SRAM arrays use transposed word-lines and bit-lines with 8T-SRAM cells to activate columns and accumulate products along rows.Feed-forward instead activates rows and accumulates products along columns.
  • eNVM implementation: eNVM arrays reuse their switch matrices to rotate inputs and outputs, adding readout circuitry for horizontal partial-sum accumulation.The added circuitry may include sense amplifiers or ADCs, adders, shift-adders, and registers.
  • Design assumption: Because feed-forward and backpropagation are not simultaneous, the framework avoids complex logical-control circuitry above the synaptic arrays.

D. Weight Gradient Caculation

Weight gradients are formed by channel-wise products between activations and backpropagated errors, then accumulated across channels and spatial positions. The framework uses SRAM-based CIM arrays for this workload because gradients require frequent rewriting and vector-matrix multiplication.

  • Gradient computation: Weight gradients combine layer<n> activations with layer<n+1> errors through element-wise multiplication and accumulation in a channel-to-channel scheme.
  • Gradient computation: Each channel’s element-wise products is accumulated into the corresponding weight-gradient entry for a kernel and spatial location.
  • Matrix mapping: Errors can be unrolled into D long columns, while activations provide N unrolled input vectors for matrix computation of weight gradients.The resulting group corresponds to weights at a specific spatial location.
  • Array selection: SRAM-based CIM arrays are selected for gradient computation because frequent rewrites and vector-matrix multiplications favor fast writing and effectively infinite endurance.The passage contrasts this with eNVMs’ better area efficiency and SRAM’s standby leakage.

E. Weight Update

Weight gradients are transferred through off-chip memory, accumulated across each batch, and used to update portions of the synaptic arrays. DNN+NeuroSim V2.0 exposes buffer sizing as an area–latency design tradeoff.

  • Gradient accumulation: Per-image weight gradients are sent to off-chip memory, then reloaded and accumulated into delta weights before accelerator-side updates.
  • Gradient accumulation: Updates proceed by selecting a synaptic-array portion and accumulating the corresponding gradients across the batch.
  • Buffered accumulation: Successive buffering overlaps loading, accumulation, and storage, requiring global-buffer capacity of 2×synaptic-array-size×highestpossible accumulated weight gradients precision.
  • Area–latency trade-off: The minimal-buffer assumption limits training area overhead but incurs latency of B× (buffer-read + accumulation + buffer-write).
  • Area–latency trade-off: Increasing the buffer overhead constraint enlarges buffers and accumulation units, enabling more synaptic arrays to update simultaneously and reducing accumulation latency.This option lets users seek a design satisfying a specific area constraint.

IV. BENCHMARK RESULTS

The benchmark emphasizes non-ideal synaptic-device properties for in-situ training, evaluating SRAM and eNVM technologies with VGG-8 on CIFAR-10. It specifically examines nonlinearity, asymmetry, and update variation, including momentum optimization.

  • Training-device effects: The benchmark evaluates nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation during in-situ weight updates.
  • Benchmark scope: SRAM and versatile eNVM device technologies are benchmarked using VGG-8 on the CIFAR-10 dataset.
  • Optimization comparison: Figure 10 analyzes nonlinearity and asymmetry for in-situ training with and without momentum optimization.

A. Impacts of Non-ideal Synaptic Device Properties

DNN+NeuroSim V2.0 evaluates how synaptic-device nonidealities affect on-chip training accuracy. Momentum can mitigate nonlinear asymmetry, whereas cycle-to-cycle variation causes substantial degradation.

  • Evaluation setup: The framework sweeps nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation to quantify their individual effects on VGG-8 training accuracy.The evaluation uses CIFAR-10 and fixes the precision of the analog-to-digital converter.
  • Nonlinearity and asymmetry: With momentum β=0.9, in-situ training reaches ~85% for NL=+3/-3 and ~77% for NL=+6/-6.These results are reported without device-to-device or cycle-to-cycle variations.
  • Device-to-device variation: Device-to-device variation with σ=0.5 does not affect accuracy under momentum optimization and can salvage degradation at larger mean nonlinearity.The variation is modeled by randomly assigning nonlinearities to synaptic weights around a mean value.
  • Cycle-to-cycle variation: Cycle-to-cycle variation of 1%, 3%, or 5% causes significant accuracy drops as variation increases.The variation can reverse an update relative to the desired gradient and induce opposite momentum directions.

B. Hardware Performance Per Epoch

DNN+NeuroSim V2.0 generates per-epoch reports covering hardware area, latency, energy, accuracy, efficiency, and throughput. Weight-gradient computation is the principal training bottleneck because of repeated memory transfers and SRAM writes.

  • Reporting: The framework produces detailed reports for every epoch, including component and operation breakdowns plus summarized accuracy, efficiency, throughput, and weight statistics.The default VGG-8/CIFAR-10 training configuration generates 256 detailed reports.
  • Reporting: The 100th-epoch FeFET-based CIM report includes area, latency, and energy breakdowns by components and operations, including peak synaptic-array computation.Peak metrics exclude off-chip memory, buffers, and interconnect operations.
  • Area: The 6-bit flash ADC dominates total area, while weight-gradient units, buffers, and control circuits also occupy substantial area.The weight-gradient units use SRAM-based CIM arrays.
  • Latency and energy: Buffer latency and DRAM energy are hardware-performance bottlenecks because training involves extensive on-chip data transfer.Feed-forward and error computation have similar operation schemes, data volumes, and hardware resources.
  • Operation bottlenecks: Weight-gradient computation dominates total latency and energy because it repeatedly loads activations and errors, writes SRAM arrays, and transfers gradients off-chip.The repeated off-chip memory accesses and SRAM writes make gradient computation the bottleneck.
  • Operation bottlenecks: Weight update contributes little averaged latency and energy because it occurs once per batch, whereas the other operations run for every input.The benchmark batch size is 200, so weight-update cost is averaged by 200× per epoch.
  • Layer-wise behavior: Across VGG-8 layers, feed-forward and error-computation trends follow activation size, while weight-update latency and energy depend only on weight size.Deeper layers can accelerate gradient computation by duplicating smaller errors and fetching multiple activations simultaneously.

C. Hardware Performance Across Epoch

The framework tracks layer-wise weights, activities, delta weights, latency, and energy across training epochs, linking hardware behavior to changing conductance and convergence. FeFET and EpiRAM designs show distinct epoch-dependent trade-offs between latency, energy, currents, and weight-update cost.

  • Trace-Based Evaluation: Real traces of activations and updated weights are used per epoch to estimate feed-forward and error-computation hardware performance.Weight-gradient estimation uses a pseudo-traced method, while weight-update estimation accesses old and updated weights.
  • FeFET Training Traces: FeFET analysis tracks weight, delta-weight, and input-activity distributions alongside peak latency, energy, and throughput across epochs.The traces are normalized by activation and weight size to account for differing hardware resources and computation counts.
  • FeFET Training Traces: After 200th epoch, increasing weights and activities raise column currents, decreasing latency while increasing dynamic energy through more activated peripheral transistors.Delta-weight means approach zero after 200th epoch as the network converges, while peak throughput increases significantly.
  • FeFET Training Traces: Before 200th epoch, decreasing normalized weights and activities reduce column currents, increasing synaptic-array latency and energy.Higher resistance lengthens delay and increases ADC delay rapidly enough to raise overall energy despite lower column-current power.
  • EpiRAM Comparison: EpiRAM’s lower on-state resistance produces decreasing normalized weights and increasing input activities before 200th epoch, unlike the FeFET trend.Its higher write voltage and pulse width also increase weight-update energy relative to FeFET.

D. Benchmark Across Technologies

The benchmark compares SRAM and eNVM CIM accelerators for VGG-8 training on CIFAR-10 across technology nodes and device precisions. It identifies on-state resistance, write-pulse width, cycle-to-cycle variation, and SRAM scaling as key determinants of hardware performance and training accuracy.

  • Benchmark Scope: The benchmark evaluates sequential and parallel SRAM accelerators at 7nm and 32nm alongside parallel-read eNVM accelerators at 32nm.Device options include reported analog synaptic technologies, with VGG-8 trained on CIFAR-10.
  • Precision Settings: Device precision determines network precision, spanning 5-bit for 32 conductance levels to 7-bit for 128 levels.Digital SRAM designs are evaluated at 5-bit specifications when using 32 levels.
  • Device and Circuit Trade-offs: On-state resistance affects area, latency, and throughput because avoiding voltage drop requires larger transistors, increasing capacitance loading.The benchmark identifies on-state resistance as an important hardware-performance factor.
  • Device and Circuit Trade-offs: Write pulse widths below 1μs avoid detrimental speed effects because weight-update cost is averaged over the batch size.Cycle-to-cycle variation above the preferred range can drive opposite momentum updates and prevent learning.
  • Device and Circuit Trade-offs: A preferred cycle-to-cycle variation is lower than 1%, while advanced 7nm parallel-read SRAM shows superior energy efficiency and throughput despite leakage and area overhead.The SRAM overhead comparison is stated for designs at the same technology node.

V. CONCLUSION

DNN+NeuroSim V2.0 extends end-to-end CIM benchmarking to on-chip training with flexible networks and device options. Its benchmark identifies target analog-device specifications for training-aware accelerator design.

  • Framework Contribution: DNN+NeuroSim V2.0 benchmarks CIM architectures for on-chip training while supporting flexible network structures and versatile device options.The framework models device nonlinearity, asymmetry, device-to-device variation, and cycle-to-cycle variation during weight updates.
  • Technology Targets: Momentum optimization helps investigate analog synaptic-device non-idealities during in-situ training, with desired specifications of variation below 1%, resistance above 100 kΩ, and pulse width below 1μs.The reported desired nonlinearity is below +3/-3.
Loading 2003.06471v1…