PaperlayerView exact revision on arXiv

Source-linked AI summary

Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator

Matthew J. Marinella, Sapan Agarwal, Alexander Hsia, Isaac Richter, Robin Jacobs-Gedrim, John Niroula, Steven J. Plimpton, Engin Ipek, Conrad D. James

arXiv:1707.09952v2cs.ARcs.NE

TL;DR

The paper addresses how to obtain further neural-network accelerator gains as CMOS scaling slows and device behavior increasingly affects analog computation accuracy. It develops and co-designs a 14/16 nm analog ReRAM accelerator for three training kernels, finding large energy and latency advantages over digital ReRAM and SRAM but reduced training accuracy and remaining device-level constraints.

  • Problem

    The paper asks whether analog ReRAM crossbars can provide further efficient neural-network acceleration when CMOS scaling slows, despite device effects that can affect algorithm-level accuracy.

  • Method

    The authors co-design an analog ReRAM accelerator for VMM, MVM, and outer-product update and compare it with digital ReRAM and SRAM using circuit, device, and algorithm-level analysis.

  • Results

    The analog accelerator reports 270X lower energy and 540X lower latency than digital ReRAM, 430X lower energy and 34X lower latency than SRAM, and about 11 fJ per MAC.

  • Takeaways & Limitations

    The reported gains support continued optimization of analog resistive memories and provide a foundation for further architecture-level accelerator studies.

  • Takeaways & Limitations

    The analog accelerator's training accuracy is degraded, while area and energy remain constrained by high-voltage transistors and ramp-ADC comparator current.

Abstract

from arXiv · show

Neural networks are an increasingly attractive algorithm for natural language processing and pattern recognition. Deep networks with >50M parameters are made possible by modern GPU clusters operating at <50 pJ per op and more recently, production accelerators capable of <5pJ per operation at the board level. However, with the slowing of CMOS scaling, new paradigms will be required to achieve the next several orders of magnitude in performance per watt gains. Using an analog resistive memory (ReRAM) crossbar to perform key matrix operations in an accelerator is an attractive option. This work presents a detailed design using a state of the art 14/16 nm PDK for of an analog crossbar circuit block designed to process three key kernels required in training and inference of neural networks. A detailed circuit and device-level analysis of energy, latency, area, and accuracy are given and compared to relevant designs using standard digital ReRAM and SRAM operations. It is shown that the analog accelerator has a 270x energy and 540x latency advantage over a similar block utilizing only digital ReRAM and takes only 11 fJ per multiply and accumulate (MAC). Compared to an SRAM based accelerator, the energy is 430X better and latency is 34X better. Although training accuracy is degraded in the analog accelerator, several options to improve this are presented. The possible gains over a similar digital-only version of this accelerator block suggest that continued optimization of analog resistive memories is valuable. This detailed circuit and device analysis of a training accelerator may serve as a foundation for further architecture-level studies.

I. INTRODUCTION

The paper motivates analog ReRAM acceleration as CMOS scaling slows and presents a co-designed accelerator for three neural-network training kernels. It evaluates analog ReRAM against digital ReRAM and SRAM using circuit, device, architecture, and algorithm-level analysis.

  • Motivation: Analog ReRAM acceleration is motivated by slowing CMOS scaling and the need for continued performance-per-watt gains.The paper notes that voltage and frequency scaling have slowed, limiting conventional improvements.
  • Contribution: The accelerator targets vector matrix multiply, matrix-vector multiply, and outer-product update, which commonly bottleneck neural-network training.These operations are implemented in an analog module intended to improve efficiency when floating-point precision is unnecessary.
  • Method: The study uses co-design because ReRAM device properties can directly affect algorithm-level accuracy.Measured device data are used to predict training accuracy through the CrossSim methodology.
  • Evaluation: The paper compares analog-ReRAM, digital-ReRAM, and SRAM implementations across energy, latency, and area for all three kernels.It also analyzes training accuracy using experimental ReRAM properties.
  • Neural-network context: Neural-network training adjusts weights through backpropagation after processing training examples, motivating efficient hardware support for repeated weight updates.The paper uses backpropagation of errors to assess training.

C. Analog Neural Accelerator Related Work

Prior work establishes analog crossbar matrix operations but reports varied gains and practical constraints. This paper addresses that uncertainty with detailed co-design analysis using a commercial 14/16 nm process and experimental ReRAM data.

  • Prior demonstrations: Earlier analog demonstrations show VMM feasibility but can require roughly 10 μA ReRAM currents, making large parallel arrays difficult to use.The cited demonstration used a 24 x 36 crossbar in 500 nm CMOS with monolithically integrated ReRAM.
  • Related-work gap: Reported accelerator gains range widely, while prior comparisons often omit latency and absolute energy-per-operation metrics.The paper attributes this variation partly to algorithm dependence and identifies a need for more precise analysis.
  • This paper's scope: This work uniquely analyzes energy, latency, area, and accuracy for a training and inference accelerator using a commercial 14/16 nm PDK and experimental TaOx data.The broader architecture combines an analog crossbar neural core with digital processing and routing.
  • Accelerator architecture: The neural core supports three kernels: parallel VMM, transpose-parallel MVM, and parallel rank-one outer-product update.These operations underlie algorithms including backpropagation, sparse coding, and restricted Boltzmann machines.
  • Analog crossbar operation: Analog crossbars perform VMM, MVM, and outer-product updates in parallel, reducing crossbar work from O(N^2) to O(N) inputs or outputs.Voltages encode vector elements, conductances encode weights, and Kirchhoff laws provide multiplication and summation.

A. Vector Matrix Multiply

The neural core maps matrix operations onto an analog ReRAM crossbar using temporal and voltage coding, integration, and multiplexed read/write circuitry. Differential reference weights represent signed values but add current and calibration considerations.

  • Vector matrix multiply: VMM drives encoded input pulses across rows and reads column multiply-accumulates through integration followed by analog-to-digital conversion.A constant-voltage, variable-length pulse represents each input, while an offset-correction row supports the analog summation.
  • Matrix-vector multiply: MVM reverses the orientation: columns receive the temporal input signal and rows are read through voltage coding and analog multiplexing.The operation is analogous to VMM but drives columns and reads rows.
  • Outer-product update: Outer-product updates encode one vector temporally and the other by voltage, avoiding the update-time increase associated with using time coding alone.The vectors are supplied through separate registers and update the conductance weights in parallel.
  • Negative weights: Signed weights use paired positive and negative memory elements whose differential currents are integrated to represent the matrix value.Negative weights are initialized to a midpoint reference resistance, which subtracts a fixed offset.
  • Negative weights: Reference-weight subtraction roughly doubles analog current but reduces integrated charge and can maximize integrator dynamic range.Reference-resistance variation shifts the zero point and can be handled through initial calibration or absorbed into random initialization.

2) Temporal Coding Drivers

The temporal coding drivers convert digital inputs into variable-length pulses and provide high-voltage, polarity-controlled drive signals for ReRAM operations. The design also addresses approximately 1 ns inputs using a current-conveyor integrator and calibrated offset correction.

  • 2) Temporal Coding Drivers: Variable-length pulses encode digital input bits, while row drivers select positive or negative voltages according to the input sign.The same drivers can disconnect or enter high impedance when other crossbar operations run.
  • 2) Temporal Coding Drivers: High-voltage level shifters convert 0.8 V logic control signals into approximately 1.8 V array-drive signals for complementary positive and negative arrays.The level shifter uses positive feedback to increase output voltage with minimum-size devices.
  • 2) Temporal Coding Drivers: Four write phases capture all combinations of positive or negative row and column voltages, while voltage rails are selected separately for reads and writes.The voltage coding driver uses stored polarity to determine which rail is driven during writes.
  • 2) Temporal Coding Drivers: A current-conveyor integrator processes the high-speed inputs, and an added calibration row supplies fixed current to correct integrator and op-amp offsets.The design targets inputs with time dependence around 1 ns.

IV. CIRCUIT BLOCK EFFICIENCY

The efficiency analysis models analog ReRAM, digital ReRAM, and SRAM-based accelerator blocks with a 14/16 nm FinFET PDK across area, energy, and latency. It specifies precision, memory, device, and architecture assumptions for comparing the designs.

  • IV. CIRCUIT BLOCK EFFICIENCY: The model compares analog ReRAM with digital ReRAM and digital CMOS-only SRAM architectures using 8-bit weights and a 14/16 nm FinFET PDK.All logic operates at a 1 GHz clock, and the digital comparison uses 256 parallel MACs.
  • IV. CIRCUIT BLOCK EFFICIENCY: The analysis evaluates reduced 4-bit and 2-bit input/output variants while retaining 8-bit weights, increasing 2-bit read and write pulses to 7 ns.The longer pulses provide sufficient integrated charge and write strength.
  • IV. CIRCUIT BLOCK EFFICIENCY: A 100 MΩ ReRAM on-state resistance limits wire current below approximately 10 µA and is paired with an access device for parallel writes.The access device also prevents current flow at low voltages.
  • IV. CIRCUIT BLOCK EFFICIENCY: The analog design supports VMM, MVM, and outer-product updates, with read and transpose operations requiring equal time and energy while MAC operations are free.The digital outer-product update additionally incurs read, write, and MAC costs.

A. Analog Array

The analog-array analysis estimates area, energy, and delay for read and write operations under temporal coding and four-phase updates. It accounts for capacitive and static-current energy, line capacitance, and shared-cycle timing.

  • A. Analog Array: The analog-array latency is dominated by temporal driver delays because the array’s approximately 0.2 ns RC time constant is negligible by comparison.The 90% rise time is modeled as 2.2×τRC.
  • A. Analog Array: Read energy includes dynamic CV2 and static IV terms, doubled for positive and negative weight arrays under randomly distributed inputs.Each bit has a 50% chance of being active and driving static current.
  • A. Analog Array: The line capacitance is modeled as the number of columns multiplied by the combined wire and ReRAM-plus-access-device capacitances.This capacitance contributes to the dynamic read-energy calculation.
  • A. Analog Array: Writes use four phases, with one quarter of devices receiving the full write voltage in each phase while unselected devices carry negligible current.Only one array is written; the reference array remains unchanged.
  • A. Analog Array: Write-energy estimates include setup, transition, and I-V components across the four-phase cycle.The temporal driver is modeled with a 50% probability of being active during relevant transitions.

B. Temporal Drivers

The temporal-driver implementation combines synthesized digital control with high-voltage analog drivers. Its energy estimate includes level-shifter transitions across the full driver array.

  • B. Temporal Drivers: Each 8-bit temporal driver row uses synthesized digital buffers and logic, with 8.6 µm2 of area including input data storage.The control sequence detects the leading counter bit, gates it with registered data, selects polarity, and sends signals to voltage shifters.
  • B. Temporal Drivers: The analog driver uses 20 high-voltage transistors and occupies 7 µm2 per driver, with total driver area scaling by the larger array dimension.The high-voltage devices include level shifters and drive transistors.
  • B. Temporal Drivers: Each level shifter and attached driver is modeled at approximately 200 ps and 15 fJ per transition.The feedback-based level-shifter design minimizes transistor count, and the energy is averaged across 1024 drivers.

C. Voltage Drivers

The voltage driver uses four-bit signed voltage coding, with area dominated by per-column high-voltage drivers and level shifters. The integrator and its capacitor are sized for the required current and output dynamic range.

  • C. Voltage Drivers: Per-column driver area is dominated by eight 1.8 V transistors per rail, with additional rails and level shifters determined by voltage-bit count.The driver includes four transistors per level shifter and two drive transistors per array.
  • C. Voltage Drivers: Control logic selects enabled rails with the correct polarity, applies temporally coded outputs to columns, and stores ADC results in an included register.The driver can use the described voltage rails and receives ADC results for storage.
  • C. Voltage Drivers: Four-bit signed voltage coding uses 3 magnitude bits and 1 sign bit, limiting driver area while supporting the update operation.The design uses four bits because only a few bits are needed for the update, and driver area is dominant.
  • C. Voltage Drivers: The integrator requires 12 longer-channel transistors and four minimum-sized pass-gate transistors, totaling 6.4 µm^2 per column for a maximum 1 µA input.The longer channel length increases the area by 19%.
  • C. Voltage Drivers: A ~10 fF integration capacitor is sufficient because sparse or near-zero-average inputs reduce the required output dynamic range below the maximum accumulated charge.The maximum possible charge would require 330 fF, but the expected output range is only a few percent of that.

E. Analog to Digital Converter (ADC)

The ADC uses shared ramp-generation and control logic with one comparator per column, while the digital ReRAM comparison is optimized around parallel array access and parasitic constraints. The resulting digital ReRAM array requires substantial read and write energy and latency.

  • E. Analog to Digital Converter (ADC): The ADC uses one shared ramp generator and control logic plus one comparator per column, with comparator area dominating across 1024 columns.Each comparator is estimated at 5.7 µm^2.
  • E. Analog to Digital Converter (ADC): An eight-bit ADC ramp runs for 256 ns at one level per nanosecond, while 1024 comparators each consume 20 µA at 1.8 V.SPICE simulations verified the comparator current and switching speed.
  • E. Analog to Digital Converter (ADC): The digital ReRAM design maximizes density with eight 1024x1024 arrays holding 8 MB, while requiring all array values to be read or written in one cycle.The design is optimized for area and throughput.
  • E. Analog to Digital Converter (ADC): Parallel digital ReRAM access is constrained by parasitic voltage drops and electromigration, so the design limits voltage drop to roughly 100 mV to suppress half-select leakage.The array is optimized where half-select leakage can be ignored in read/write energy.
  • E. Analog to Digital Converter (ADC): The digital ReRAM array reads 512 bits in parallel and writes 64 bits in parallel, yielding 10 ns writes, 86 ns reads, 164 µs full-array writes, and 176 µs full-array reads.The corresponding total energies are 676 nJ for writing and 166 nJ for reading.
  • E. Analog to Digital Converter (ADC): Reading 256 rows requires 256 sense amplifiers occupying 9,500 µm^2, about twice the array size, with 5 fJ consumed per measurement.The sense-amplifier design uses 60 low-voltage transistors per amplifier.

H. Digital SRAM Array

The SRAM baseline combines synthesized cache blocks and digital MAC hardware, but transpose access requires eight times more reads than row-major access. At the accelerator-kernel level, the analog design is reported as substantially more efficient than both digital baselines, while full architectural utilization remains unresolved.

  • H. Digital SRAM Array: SRAM transpose processing requires 8X additional reads because row-major storage does not align returned data with the input vectors for multiply-accumulate units.Alternative non-1D-blocked arrangements were considered but introduce other costs.
  • H. Digital SRAM Array: The synthesized digital block operates at 1 GHz and delivers one pipelined 8-bit multiply-add per clock using approximately 1.46 pJ including writeback.Each operation takes 2 ns to complete internally.
  • H. Digital SRAM Array: The analog accelerator is reported as 270X better in energy, 1040X in latency, and 1.8X in area than digital ReRAM, and 430X, 34X, and 11X better than SRAM, respectively.These comparisons are reported at the kernel level rather than for a complete accelerator architecture.
  • H. Digital SRAM Array: A full accelerator architecture is still required to exploit the analog circuit-block advantages beyond the reported kernel-level comparisons.The limitation concerns system-level utilization rather than the reported circuit-block results.

V. ASSESSING THE SUITABILITY OF RERAM FOR USE AS AN ANALOG NEURAL ACCELERATOR TRAINING ELEMENT

Analog ReRAM training accuracy depends on device-specific, stochastic write behavior, requiring measured-device modeling rather than a simple low-precision abstraction. The study characterizes these effects through repeated pulse measurements and uses them to assess algorithm-level accuracy.

  • V. ASSESSING THE SUITABILITY OF RERAM FOR USE AS AN ANALOG NEURAL ACCELERATOR TRAINING ELEMENT: Device behavior cannot be represented accurately as a simple six-bit value because conductance changes are stochastic functions of current state and change direction.The paper therefore models measured device characteristics to predict algorithm-level accuracy.
  • V. ASSESSING THE SUITABILITY OF RERAM FOR USE AS AN ANALOG NEURAL ACCELERATOR TRAINING ELEMENT: Write nonlinearity, asymmetry, and stochasticity—not read noise below about 5% current—are the main determinants of accuracy in open-loop analog training.Nonlinearity and asymmetry depend on conductance state and voltage polarity, while stochasticity produces random conductance changes.
  • V. ASSESSING THE SUITABILITY OF RERAM FOR USE AS AN ANALOG NEURAL ACCELERATOR TRAINING ELEMENT: Repeated fixed-voltage, fixed-width pulse trains measure the combined write effects, and the resulting data populate a lookup table for algorithm-level modeling.The routine applies 1000 positive pulses followed by 1000 negative pulses and repeats this for 1k to 10k cycles.
  • V. ASSESSING THE SUITABILITY OF RERAM FOR USE AS AN ANALOG NEURAL ACCELERATOR TRAINING ELEMENT: The study uses Sandia Ta/TaOx bipolar ReRAM cells, whose behavior is considered representative of similar oxide-based bipolar ReRAM in the same conductance range.The cells do not represent record endurance or maximum resistance, which are assessed against literature instead.

C. Conductance Change Characteristic Dataset

The study characterizes ReRAM conductance changes under voltage pulses and evaluates device endurance and array-level write-current constraints. It finds predictable voltage dependence while identifying endurance validation and electromigration limits as important design considerations.

  • Repeated pulsing captures nonlinearity, asymmetry, and stochasticity in the conductance-versus-pulse behavior used for algorithm-level modeling.Statistical data is extracted from repeated pulsing between GMIN and GMAX and sorted into bins for each pulse amplitude and width.
  • Conductance change follows a predictable exponential dependence on SET and RESET pulse voltage, supporting the proposed voltage-encoding write scheme.The model uses device-dependent parameters and minimum positive and negative voltages required to change the device state.
  • An endurance estimate of approximately 8x10^14 single pulses is required for one year of continuous training at 100 kHz with up to 256 pulses per training cycle.The estimate assumes the maximum pulse count for the 8-bit scheme, although the text notes that this worst case is unlikely for every device.
  • Published TaOx memory endurance corresponds to approximately 2x10^12 analog updates, while smaller nudge energies may permit greater endurance than full-state cycling.The paper states that statistically relevant endurance testing with analog cycling to failure is needed to establish this possibility.
  • For a 1024x1024 array, electromigration limits imply a maximum allowable nudge current of approximately 32 nA and a minimum resistance of about 31 MΩ.The design limits nudge current to 10 nA to keep parasitic voltage drop below 20 mV.

A. Accuracy of MNIST Training

Analog TaOx ReRAM training suffers substantial accuracy loss from device nonidealities, especially state-dependent conductance changes, but periodic carry can recover accuracy to near the numerical baseline. The accelerator nevertheless offers large energy and latency advantages, while device endurance and analog behavior remain unresolved challenges.

  • A. Accuracy of MNIST Training: TaOx ReRAM training reaches about 77% accuracy versus approximately 98% for numerical training, with pulse optimization improving accuracy to roughly 85%.The reported accuracy loss is described as unacceptable for most applications.
  • A. Accuracy of MNIST Training: State-dependent nonlinearity in conductance changes is the greatest identified source of accuracy degradation.The comparison removes stochasticity and state dependence separately to isolate degradation mechanisms.
  • B. Accuracy Improvements: Periodic carry uses multiple synapses with increasing significance to represent each weight and brings analog TaOx ReRAM within 1% of numerical accuracy.The method preserves parallel updates while increasing effective weight representation precision.
  • B. Accuracy Improvements: Accuracy depends on the chosen resistive device: CBRAM performs similarly to TaOx, whereas NVRTs have approached numerical accuracy if speed and endurance can be demonstrated.The paper frames device selection and optimization as continuing requirements for practical analog training.
  • 430X lower energy than SRAM and 270X lower energy than digital ReRAM, while analog MAC energy is approximately 11 fJ.The broader accelerator analysis also reports 34X and 540X latency improvements over SRAM and digital ReRAM, respectively.
  • ReRAM endurance and combined analog requirements remain insufficiently established for sustained training operation.The discussion identifies endurance beyond 10^13 cycles and suitable analog behavior as unresolved challenges.
Loading 1707.09952v2…