PaperlayerView exact revision on arXiv

Source-linked AI summary

RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars

Shubham Jain, Abhronil Sengupta, Kaushik Roy, Anand Raghunathan

arXiv:1809.00072v3cs.ETcs.CVcs.LG

TL;DR

Non-idealities in resistive crossbars can introduce VMM errors that degrade large-scale DNN accuracy, while existing models are too slow or limited for application-level evaluation. RxNN evaluates DNN inference using FCM, which captures crossbar non-idealities much faster than circuit simulation. Experiments report significant accuracy degradation for large-scale DNNs.

  • Problem

    Evaluating how crossbar non-idealities affect large-scale DNN accuracy matters, but circuit models are too slow and simplified architectural models are inadequate for application-level evaluation.

    Source: §I · pp. 1–2

  • Method

    RxNN maps DNN computations to crossbar operations and uses FCM, combining peripheral-circuit models with an equivalent non-ideal conductance matrix to evaluate VMMs.

    Source: §I · p. 2, §VI · p. 8

  • Results

    9.6%-32% accuracy degradation from crossbar non-idealities is reported for large-scale DNNs.

    Source: §I · p. 2

  • Takeaways & Limitations

    Re-training can only partly restore accuracy lost to non-idealities, motivating further error mitigation and compensation schemes.

    Source: §IX · p. 11

Abstract

from arXiv · show

Resistive crossbars designed with non-volatile memory devices have emerged as promising building blocks for Deep Neural Network (DNN) hardware, due to their ability to compactly and efficiently realize vector-matrix multiplication (VMM), the dominant computational kernel in DNNs. However, a key challenge with resistive crossbars is that they suffer from a range of device and circuit level non-idealities such as interconnect parasitics, peripheral circuits, sneak paths, and process variations. These non-idealities can lead to errors in VMMs, eventually degrading the DNN's accuracy. It is therefore critical to study the impact of crossbar non-idealities on the accuracy of large-scale DNNs. However, this is challenging because existing device and circuit models are too slow to use in application-level evaluations. We present RxNN, a fast and accurate simulation framework to evaluate large-scale DNNs on resistive crossbar systems. RxNN splits and maps the computations involved in each DNN layer into crossbar operations, and evaluates them using a Fast Crossbar Model (FCM) that accurately captures the errors arising due to crossbar non-idealities while being four-to-five orders of magnitude faster than circuit simulation. FCM models a crossbar-based VMM operation using three stages - non-linear models for the input and output peripheral circuits (DACs and ADCs), and an equivalent non-ideal conductance matrix for the core crossbar array. We implement RxNN by extending the Caffe machine learning framework and use it to evaluate a suite of six large-scale DNNs developed for the ImageNet Challenge. Our experiments reveal that resistive crossbar non-idealities can lead to significant accuracy degradations (9.6%-32%) for these large-scale DNNs. To the best of our knowledge, this work is the first quantitative evaluation of the accuracy of large-scale DNNs on resistive crossbar based hardware.

I. Introduction

Crossbar non-idealities can degrade DNN accuracy, while existing models are either too slow or too simplified for evaluating large networks. RxNN uses FCM to evaluate large-scale DNNs, and experiments report accuracy degradation of 9.6%-32%.

I. Introduction

  • Existing SPICE models are too slow for large-scale evaluations, while architectural models simplify errors and omit dependencies on inputs, conductances, and columns.This leaves application-level accuracy evaluation of large-scale DNNs without a suitable scalable framework.Source: §I · pp. 1–2
  • FCM models crossbar VMMs with peripheral DAC and ADC models and an equivalent non-ideal conductance matrix, running four-to-five orders of magnitude faster than circuit simulation.The conductance matrix is derived by pre-solving the array voltage-current equations.Source: §I · p. 2
  • RxNN maps DNN computations to crossbar operations and evaluates inference using FCM, extending Caffe for six ImageNet DNNs and three simpler networks.The simpler networks classify CIFAR-10 and MNIST.Source: §I · p. 2
  • Crossbar VMM errors vary significantly with data and hardware instance, a dependence the authors say accurate crossbar models should consider.The paper also notes that non-idealities can degrade DNN accuracy despite some tolerance to computational imprecision.Source: §I · pp. 1–2
  • 9.6%-32% accuracy degradation from crossbar non-idealities is reported for large-scale DNNs.The evaluation covers six state-of-the-art DNNs; the degradation can be partly alleviated by re-training.Source: §I · p. 2

II. Related Work

Prior crossbar hardware and mitigation work often used limited non-ideality models or simple networks, while design tools focused on system metrics. The preliminaries describe crossbar VMM operation and the devices used to implement it.

II. Related Work

  • Prior accelerator studies focused on performance, energy, and area, and often omitted non-idealities or modeled only VMM quantization.The cited work includes accelerators for DNN inference and training.Source: §II · p. 2
  • Mitigation studies span re-training, weight conversion, rank clipping, hard-failure handling, and hardware solutions, but are restricted to simple networks and small datasets.Many also lack detailed crossbar models and consider only subsets of non-idealities.Source: §II · p. 2
  • RxNN complements accelerator and mitigation efforts by targeting accurate, efficient modeling of crossbar non-idealities for large-scale DNNs.The related-work discussion contrasts its scope with efforts limited in modeled non-idealities and simulation speed.Source: §II · pp. 2–3
  • Crossbar design tools include simplified accuracy or error models suited to design-space exploration but inadequate for evaluating application-level accuracy of large DNNs.Their primary focus is performance, energy, and area evaluation.Source: §II · p. 3

III. Preliminaries

  • A crossbar uses DACs to apply digital inputs as row voltages and ADCs to sense column currents; programming sets synaptic-device conductances.Evaluation performs the vector-matrix multiplication, while programming writes selected devices.Source: §III · p. 3
  • Synaptic devices are programmable resistors, and the framework described applies across resistive-device types despite using a spintronic device as its explanatory example.Examples named include PCM, ReRAM, and Spintronics.Source: §III · p. 3

IV. Crossbar Non-Idealities

Crossbar non-idealities arise from device, interconnect, and peripheral effects and make VMM errors depend on array size, conductance state, inputs, columns, and variation. The section characterizes these dependencies and their implications for modeling crossbar errors.

Crossbar non-idealities

  • Non-ideal DAC output depends on both the applied input and effective load, while wire-resistance-induced internal voltage changes can create sneak paths that depend on crossbar state and inputs.In the illustrated 3x2 array, current between nodes a22 and b22 flows opposite the expected direction.Source: §IV · pp. 3–4

Sensitivity to crossbar size

  • VMM errors from all, individual, and combined non-idealities increase with crossbar dimension; DAC effects predominate in smaller arrays, while wire and sensing resistance become equally significant in larger ones.The plotted error is in the last column of an NxN crossbar, with all synaptic conductances set to GMIN or GMAX.Source: B. Errors due to Non-Idealities · p. 4

Sensitivity to crossbar state and inputs

  • Errors vary substantially across crossbar states, while input-dependent variation for a given column is noticeable but smaller than state-dependent variation.A sample state also shows a column-error pattern unlike the maximum, minimum, and average patterns across random states.Source: B. Errors due to Non-Idealities · pp. 4–5

Sensitivity to crossbar columns

  • Any column can incur the maximum error depending on crossbar state and applied inputs, despite a slight trend toward higher error in later columns.The last column is not always the column with maximum error.Source: B. Errors due to Non-Idealities · p. 5

Sensitivity to process variation and imperfect programming

  • Conductance variation from process variation or imperfect programming was evaluated by Monte Carlo simulation across 10,000 crossbar states on a 64x64 array.The simulations report maximum, minimum, and average error across the samples.Source: B. Errors due to Non-Idealities · p. 5

Crossbar non-idealities

  • A crossbar model must account for conductance, input, column, and process-variation dependencies to capture non-ideality effects on application-level accuracy.The modeled sources include wire and sensing resistance, driver resistance, sneak paths, conductance variation, and non-ideal DACs.Source: §IV · p. 3, B. Errors due to Non-Idealities · p. 5

V. Crossbar Modeling

FCM abstracts crossbar non-idealities into a model-generation phase and a per-inference evaluation phase. Its conductance-matrix transformations and peripheral models capture non-ideal VMM behavior while running orders of magnitude faster than SPICE.

V. Crossbar Modeling

  • FCM separates one-time model generation from evaluation of each inference, transforming weights into a non-ideal conductance matrix used with non-linear ADC and DAC models.The generator slices and maps weights to crossbar instances, converts them to conductances using device characteristics and variation profiles, then abstracts interconnect effects.Source: A. FCM Overview · pp. 5–6
  • FCM abstracts device and circuit non-idealities into crossbar models that are functionally within 0.28% of HSPICE while achieving several orders-of-magnitude speed-up over SPICE.Well-optimized BLAS libraries accelerate its algebraic operations.Source: A. FCM Overview · p. 6
  • The model generator abstracts sensing and distributed wire resistances by transforming each crossbar instance’s conductance matrix into a non-ideal conductance matrix.The formulation uses circuit laws and linear algebraic operations.Source: B. Abstraction of interconnect… · p. 6
  • For an MxN crossbar, the formulation builds N column linear systems from vertical slices and merges them into a larger column system.The systems are formulated using Kirchhoff’s Current Law.Source: B. Abstraction of interconnect… · pp. 6–7

1 VcolT

The supplied excerpt includes a VcolT term in the column-system formulation.

1 VcolT

  • The column-system formulation includes a VcolT term.Source: §1 · p. 7

1 VAcolT

The row systems are formulated from horizontal crossbar slices using Kirchhoff’s Current Law, then merged into a larger row linear system.

1 VAcolT

  • Row linear systems are formulated from horizontal crossbar slices using Kirchhoff’s Current Law at nodes ai,j.An MxN crossbar has M such row systems.Source: §1 · p. 7
  • The row systems are merged into a larger Row Linear System using a direct-sum operation on matrix Bi and concatenated vectors.The concatenated vectors are CVrowIN, CVrow, and CVArow.Source: §1 · p. 7

1 VrowT

The supplied excerpt includes a VrowT term.

1 VrowT

1 VArowT

The derivation eliminates internal variables to form a non-ideal crossbar model, then reduces its matrix dimensions by exploiting repeated input-vector elements.

1 VrowINT

  • Step 5 eliminates internal variables from the merged row and column systems, using row operations to align their right-hand sides before deriving NETmat.The derivation successively eliminates CVAcol and CVcol to obtain Equation 11.Source: §1 · p. 7
  • Step 6 combines NETmat columns multiplied by repeated CVrowINA elements, producing a compressed NETmatC and the reduced Vinnon-ideal vector.This yields Gnon-ideal as a function of G, Rsense, rcol, and rrow.Source: §1 · p. 8

VI. RxNN Framework

RxNN extends Caffe to evaluate DNN inference on resistive crossbars using FCM models. It also supports retraining and can generate execution traces for performance and energy estimation.

VI. RxNN Framework

  • RxNN modifies Caffe to map convolutional and fully connected computations onto resistive crossbars and evaluate application-level inference accuracy using FCM.It takes a trained network and weights, crossbar-system description, and crossbar parameters as inputs.Source: §VI · p. 8
  • RxNN transforms programmed conductance matrices into non-ideal matrices once per network and architecture, then evaluates test inputs with those matrices and peripheral models.DAC and ADC models process layer inputs and outputs; Caffe data structures and optimized BLAS libraries support performance and scalability.Source: §VI · p. 8
  • Retraining uses a crossbar-based forward pass and floating-point backward pass, with data structures converted and scaled between them to support convergence.This addresses range and resolution constraints and the scaling of back-propagated errors and gradients.Source: §VI · p. 8

VII. Experimental Methodology

The methodology combines circuit-level modeling with RxNN application-level evaluation of benchmark DNNs on resistive crossbar systems.

VII. Experimental Methodology

  • Circuit simulations use an in-house synaptic-device model based on LLG magnetization dynamics and NEGF electron transport.The described setup also uses HSPICE with 45nm bulk CMOS, ADC/DAC circuits, and layout-derived interconnect parasitics.Source: §VII · pp. 8–9
  • The stochastic-gradient descent solver assumes contiguous, differentiable forward and backward passes, conditions that crossbar abstraction of VMM does not ensure.This is the stated challenge motivating special handling during retraining.Source: §VII · p. 8
  • The circuit setup uses ADC and DAC circuits from,, while device, technology, and variation parameters are detailed in Figure 9.A characterized crossbar array supplies a technology parameter for system-level energy estimates.Source: §VII · p. 9, Figure 9 · p. 9
  • Application-level accuracy and energy are evaluated for popular DNNs whose layer counts, datasets, network sizes, and relative model sizes are listed in Table I.Energy is evaluated using an architecture similar to.Source: §VII · p. 9, Table I · p. 9

VIII. Results

FCM closely matches HSPICE while running much faster, and RxNN evaluates how crossbar non-idealities affect large-scale DNN accuracy and energy. The results show substantial accuracy losses, with trade-offs across array sizes and residual distortions that retraining does not fully recover.

A. FCM: Crossbar-level Evaluation

  • 0.28% maximum error deviation for FCM versus HSPICE is substantially below MNSIM's 3.51% maximum deviation.FCM accounts for input, crossbar-state, and column dependence, allowing it to closely match HSPICE.Source: Figure 10 · p. 9, A. FCM: Crossbar-level Evaluation · p. 9
  • About 5 orders of magnitude speedup over HSPICE is achieved by FCM, with greater speedup for larger crossbar arrays.Model-generation overhead is one-time: 0.038, 1.2, and 61 seconds for 16x16, 32x32, and 64x64 arrays, respectively, and is amortized over many inferences.Source: Figure 11 · p. 9, A. FCM: Crossbar-level Evaluation · p. 9

B. RxNN: Application-Level Evaluation

  • Cross16 has less accuracy degradation than Cross32, which has less than Cross64, while energy generally falls as array size increases.LeNet and ConvNet are exceptions: Cross64 uses more energy than Cross32 because the arrays are under-utilized and perform redundant computations.Source: B. RxNN: Application-Level Evaluation · p. 10
  • 25.6%, 27.8%, and 32% accuracy losses occur for VGG-16, OverFeat, and Resnet-50, respectively, on Cross64 versus an ideal crossbar.LeNet and ConvNet losses are 0.05% and 2.2%, respectively; similar simple-versus-large network trends hold for Cross16 and Cross32.Source: Figure 12 · p. 10, B. RxNN: Application-Level Evaluation · p. 10
  • Vector-matrix multiplication is the major energy component for VGG-16, GoogleNet, and AlexNet on Cross64, dominated by ADCs and DACs.Figure 13 breaks energy into input reads, output writes, and crossbar computation.Source: Figure 13 · p. 10, B. RxNN: Application-Level Evaluation · p. 10

C. Sensitivity of accuracy to non-idealities

  • Accuracy drops from FP32 to Cross6 and again to NI-Cross6-64x64 as limited precision and then device- and circuit-level non-idealities are included.The comparison covers AlexNet, VGG-16, GoogleNet, NiN, Overfeat, and ResNet-50.Source: C. Sensitivity of accuracy to… · p. 10

D. Re-training DNNs using RxNN

  • About 9%, 8%, and 26% accuracy improvement follows 150 retraining iterations for AlexNet, VGG-16, and GoogleNet, respectively, but retraining alone leaves a substantial accuracy drop.The remaining loss motivates additional error mitigation and compensation techniques.Source: Figure 14 · p. 11, Figure 15 · p. 11

E. Visualizing the impact of non-idealities

  • Non-idealities produce more feature artifacts at Conv3 than Conv1 in ConvNet on CIFAR-10, consistent with greater accuracy degradation in deeper DNNs.The results motivate further research into mitigation and compensation techniques for large-scale DNNs.Source: E. Visualizing the impact of… · p. 11

IX. Conclusion

The conclusion identifies crossbar non-idealities as a significant accuracy concern for large-scale DNNs; retraining only partly restores lost accuracy, leaving further error mitigation and compensation schemes needed.

IX. Conclusion

  • FCM is a fast, accurate model for evaluating vector-matrix multiplications realized on resistive crossbars.Source: §IX · p. 11
  • Fig. 16 visually demonstrates errors using a ConvNet.Source: Figure 16 · p. 11
  • Non-idealities significantly degrade accuracy in large-scale DNNs; retraining only partly restores lost accuracy, motivating further error mitigation and compensation schemes.Source: §IX · p. 11

Related papers

Loading 1809.00072v3…