Accurate deep neural network inference using computational phase-change memory

Vinay Joshi; Manuel Le Gallo; Simon Haefeli; Irem Boybat; S. R. Nandakumar; Christophe Piveteau; Martino Dazzi; Bipin Rajendran; Abu Sebastian; Evangelos Eleftheriou

Source-linked AI summary

Accurate deep neural network inference using computational phase-change memory

Vinay Joshi, Manuel Le Gallo, Simon Haefeli, Irem Boybat, S. R. Nandakumar, Christophe Piveteau, Martino Dazzi, Bipin Rajendran, Abu Sebastian, Evangelos Eleftheriou

arXiv:1906.03138v2Jun 7, 2019cs.ET

TL;DR

Analog in-memory computing promises efficient DNN inference but suffers accuracy loss when digitally trained weights are transferred to noisy, variable PCM devices. The paper trains ResNet-type networks with hardware-informed weight noise and adds batch-normalization-based compensation, achieving strong mapped accuracy and sustained CIFAR-10 hardware performance.

Problem
Device variability, noise, and temporal drift make it difficult to preserve DNN accuracy when digitally trained weights are mapped to analog PCM synapses.
Method
The paper injects hardware-informed synaptic weight noise during forward-pass training, clips weights, and uses batch-normalization compensation to improve PCM accuracy retention.
Results
93.7% CIFAR-10 classification accuracy and 71.6% ImageNet top-1 accuracy were achieved after mapping trained weights to PCM synapses, with CIFAR-10 hardware accuracy remaining above 92.6% for one day.
Takeaways & Limitations
The results support software-trained ResNet deployment on PCM hardware without appreciable mapped-accuracy loss and with improved retention over time.
Takeaways & Limitations
Communication between computational-memory cores can become a bottleneck, and optimal interconnectivity for state-of-the-art CNNs remains an open research problem.

Abstract

from arXiv · show

In-memory computing is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. Crossbar arrays of resistive memory devices can be used to encode the network weights and perform efficient analog matrix-vector multiplications without intermediate movements of data. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to in-memory computing hardware based on phase-change memory (PCM). We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on the CIFAR-10 dataset and a top-1 accuracy on the ImageNet benchmark of 71.6% after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one day period, where each of the 361,722 synaptic weights of the network is programmed on just two PCM devices organized in a differential configuration.

I. INTRODUCTION

Analog in-memory computing can reduce data movement by storing neural-network weights in resistive-memory crossbars, but device variability and noise make direct transfer from digital training difficult. The paper addresses this gap with software training strategies designed for resilient deployment on PCM hardware.

Motivation: Off-chip memory and processor separation forces repeated data movement, limiting the energy efficiency of conventional DNN inference platforms.This motivates hardware that keeps weights close to computation for edge applications.
Motivation: Crossbar arrays encode network weights as analog charge or conductance states and perform matrix-vector multiplications during inference.The approach targets reduced transfers between memory and processing units.
Research gap: Chip-specific training and hardware optimization require characterizing or retraining each network for each device, limiting deployment practicality.A generic software-only training method could allow one model to deploy across multiple chips.
Approach: The paper introduces software weight-noise training for ResNet networks and validates PCM deployment on CIFAR-10 and ImageNet.The methodology models combined read and write noise during forward propagation while weights are mapped to differential device conductances.
Research gap: Directly mapping FP32 weights to memristive synapses is expected to degrade accuracy because device conductance errors perturb the trained weights.Relevant errors include programming inaccuracies, conductance fluctuations, temporal drift, device variation, defective devices, and circuit non-idealities.
Research gap: Random hardware errors are not equivalent to deterministic reduced-precision quantization errors, so conventional digital-precision training procedures are not automatically suitable for analog in-memory hardware.Noise injection during training is presented as a more natural way to improve tolerance to analog weight perturbations.

B. Training procedure

The training procedure injects hardware-like Gaussian noise into synaptic weights during forward propagation, while using initialization, clipping, and learning-rate choices to preserve convergence and accuracy under inference perturbations.

Noise model: Hardware write and read errors are combined into a weight-noise model used during training and inference.The inference error arises primarily from inaccurate programming and temporal fluctuations of hardware weights.
Noise injection: Gaussian noise is injected into weights during forward propagation, while backward propagation and updates use unperturbed weights.Noise is applied at each forward pass; adding it during backward propagation did not further improve results.
Noise calibration: The training noise scale is recomputed after every update so that ηtr remains constant throughout training.This recomputation was especially important for achieving good convergence.
Weight control: Weights are clipped after each update to [−α ×σW^l, α ×σW^l], with α = 2.0 for ResNet-32 and α = 2.5 for ResNet-34.Clipping controls outliers and facilitates mapping weights into hardware with limited conductance range.
CIFAR-10 behavior: 93.87% software-baseline accuracy remains attainable on CIFAR-10 with approximately ηtr = 8% when inference is unperturbed.Above 8%, convergence is affected and the baseline cannot be reached within the same number of epochs.
Noise matching: Accuracy within 0.5% of the software baseline is achievable for inference noise up to ηinf = 5% when ηtr = ηinf.Using weight initialization, clipping, and learning-rate scheduling is important; omitting all three causes at least a 1% drop at ηtr = ηinf = 3.8%.
ImageNet behavior: ImageNet accuracy is more noise-sensitive than CIFAR-10, but omitting noise from the first convolutional and last dense layers increases accuracy by more than 1%.The first and last layers are treated as more sensitive to noise during both training and inference.

C. Weight transfer to PCM-based synapses

The PCM transfer experiments characterize conductance variability and compare training schemes after mapping weights to PCM synapses. Additive-noise training provides the strongest transfer performance on both CIFAR-10 and ImageNet.

PCM platform: A prototype multi-level PCM chip with 1 million devices was used to validate the training methodology experimentally.The devices were fabricated in 90 nm CMOS technology and support analog conductance programming over more than two orders of magnitude.
CIFAR-10 transfer: 93.7% CIFAR-10 accuracy was obtained after PCM transfer using Gaussian-noise training with ηtr = 3.8%.This was the best overall transfer performance among the compared training procedures.
Training comparisons: Standard FP32 training performed worst after transfer, while 4-bit and ternary training reduced transfer loss but did not match the additive-noise result.The 4-bit transfer loss exceeded 1%, whereas ternary weights had a drop below 0.5% but did not reach the FP32 baseline before transfer.
ImageNet transfer: Additive-noise training improved ImageNet PCM accuracy by approximately 6% compared with FP32 and 4-bit training.The comparison concerns ResNet-34 top-1 accuracy after transfer to PCM synapses.
ImageNet result: 71.6% top-1 ImageNet accuracy was achieved with additive-noise training after PCM transfer.The higher accuracy than the corresponding 3.8% inference-noise result was attributed to many weights mapping to lower-conductance values with lower variability.

D. Hardware/software inference experiment on CIFAR-10

The experiment maps all ResNet-32 synaptic weights onto PCM devices and tracks hardware inference accuracy over one day. Global drift compensation preserves high accuracy despite conductance drift.

361,722 ResNet-32 synaptic weights were individually programmed onto two PCM devices each in a differential configuration.The experiment used weights trained with ηtr = 3.8%.
The behavioral PCM model reproduced array-level statistics, individual-device behavior, and the experimental accuracy evolution over time.It modeled conductance drift and 1/f noise with dependencies on target conductance and time.
93.75% test accuracy was measured 25 seconds after programming on CIFAR-10.This closely matched the accuracy obtained before the time-evolution experiment.
Without drift compensation, accuracy decreased to 10% within approximately 1000 seconds.The decline was attributed to PCM weight reduction from conductance drift, which prevented activations from propagating properly.
GDC retained test accuracy above 92.6% for one day on the PCM chip.The procedure applied a layer-wise scaling factor derived from periodically measured array current and combined it with batch-normalization parameters.

E. Adapting batch normalization statistics to improve the accuracy retention

The paper introduces AdaBS, which updates batch-normalization running statistics to compensate for activation-distribution changes caused by PCM noise and drift. Calibration is performed during device idle periods using images from the inference distribution.

AdaBS updates each layer’s batch-normalization running mean and variance during periodic calibration.The updated statistics are used for subsequent inference, while learned scale and shift parameters remain unchanged.
AdaBS improves retention beyond GDC by correcting activation distributions affected by device noise and drift.The technique changes running mean and variance rather than the learned γ and β parameters, avoiding PCM retraining.
Calibration uses mini-batches from the same distribution as inference, with training-set images used in this study.Running statistics are computed across the calibration dataset and then applied during inference.
5.2% of the calibration images was sufficient after optimizing momentum for the number of mini-batches.Poorly tuned momentum with fewer calibration updates can substantially reduce network accuracy.
71% accuracy was retained for one year on ImageNet when AdaBS was combined with digital FP32 first and last layers.Using 0.1% of the ImageNet training dataset for calibration increased one-year accuracy by 7% compared with GDC when all layers used PCM synapses.

III. DISCUSSION

The study compares noise-injection strategies for making neural networks resilient to analog hardware imperfections and discusses remaining system-level constraints. Its proposed training and compensation strategies achieve strong accuracy after PCM mapping, while activation communication remains a bottleneck across computational-memory cores.

93.7% CIFAR-10 accuracy and 71.6% ImageNet top-1 accuracy were achieved after mapping trained weights to PCM synapses.The reported accuracies were obtained with residual networks close to their original implementations.
Retraining with additive noise primarily adapts batch-normalization parameters while keeping weights close to full-precision baseline values.
Weight-noise injection estimates training noise from a simple hardware characterization and avoids searching for noise-scaling factors.The injected-noise magnitude need not be very precise because a range of values yields similar post-transfer accuracy.
Weight-noise training achieved better accuracy retention over time than the other analyzed noise-injection methods.The finding suggests that weight noise more closely mimics PCM hardware behavior.
Small accuracy deviations from activation and preactivation quantization could be addressed by including quantization during retraining, especially below 8-bit resolution.
88× speedup is a projected outcome for pipelined ResNet-32 execution on a proposed communication topology, although the comparison is not direct.Communication of activations between computational-memory cores can otherwise become a bottleneck.

A. Experiments on PCM hardware platform

The hardware platform uses a prototype PCM chip with integrated addressing and conversion circuitry, connected to FPGA and analog-front-end boards. Conductances are read over one day and evaluated in software to measure accuracy and apply drift compensation.

3 million PCM devices comprise the prototype chip used for the hardware experiments.The chip includes word-line and bit-line organized arrays plus circuitry for device addressing and write and read operations.
300 mV biases the selected bit line during PCM reads, with sensed current integrated and digitized by an on-chip 8-bit cyclic ADC.The read pulse and ADC conversion together take 1 µs.
Analog conductance values encoding network weights are read individually from PCM devices during hardware/software inference experiments.
One-day conductance measurements are serially read at predefined timestamps and passed to TensorFlow for forward propagation and accuracy computation.The software applies GDC and AdaBS drift compensation at each timestamp.

B. PCM-based deep learning inference simulator

A TensorFlow-based simulator was developed to evaluate DNN inference using PCM devices. It leverages native TensorFlow operations and supports porting regular DNN code into the simulation framework.

TensorFlow provides native implementations of activation functions and batch normalization for the PCM inference simulator.
Regular TensorFlow DNN code can be ported easily to the simulator.Custom TensorFlow operations were implemented as part of the framework.

C. Training implementation of ResNet-32 on CIFAR-10

The CIFAR-10 implementation uses a modified ResNet-32 with residual connections, batch normalization, and global average pooling. Training uses stochastic gradient descent, scheduled learning-rate reductions, image augmentation, and channel-wise normalization.

361,722 synaptic weights are contained in ResNet-32, which uses 31 3×3 convolution layers, two 1×1 convolution layers, and a final fully connected layer.The network is organized into three ResNet blocks with ten 3×3 kernels each.
ResNet-32 uses residual feed-forward connections after every two convolution layers and a 1×1 residual convolution when output channels must be made compatible.Each convolution layer is followed by batch normalization, with ReLU placement adjusted around residual summation.
Global average pooling follows the last convolution layer and precedes a single fully connected layer without batch normalization.
50,000 training images and 10,000 test images are used for CIFAR-10 evaluation with stochastic gradient descent and momentum 0.9.The objective is categorical cross entropy over 10 classes.
Learning rate is reduced by 90% every 50th epoch, while cutout augmentation and channel-wise normalization are applied to the images.Normalization uses zero mean and unit standard deviation for both training and test sets.

D. Training implementation of ResNet-34 on ImageNet

The ImageNet ResNet-34 implementation uses a convolutional architecture derived from Ref. 19 and trains a PyTorch baseline with additive Gaussian noise.

ResNet-34 contains 32 3×3 convolution layers, 3 1×1 convolution layers, an initial 7×7 convolution layer, and a final fully connected layer.The network has 21,797,672 synaptic weights.
Training uses ImageNet51 data with standard preprocessing, including random 224 × 224 crops, random horizontal flips, and channel-wise normalization.The ImageNet dataset contains 1.3M training images and 50k test images.
The model minimizes softmax cross entropy with stochastic gradient descent using momentum 0.9.The baseline architecture and parameters come from the PyTorch model zoo.
Additive noise training injects Gaussian noise into the network for 10 training epochs.The passage contrasts this procedure with ResNet-32 training on CIFAR-10 and notes that no learning-rate scheduling was used here.

E. Global drift compensation (GDC) method

Global drift compensation calibrates conductance shifts by reading reference columns and rescales crossbar outputs during inference. The procedure uses device-derived statistics and can run while the array is idle.

GDC computes summed currents from L reference columns containing devices programmed to known conductance values G_mn(t0).Periodic readings under calibration voltage Vcal provide the information used to compensate for global conductance shifts.
The calibration estimate α̂ is computed directly from device data without assuming a conductance-drift model or requiring extra timing information.A small L is sufficient to obtain statistics for the estimate.
The reference conductance sum is computed once, stored in digital memory, and reused for subsequent calibrations.This avoids recomputing that stored quantity during each calibration.
Reading reference columns can occur while the PCM array is idle, with current summation performed on-chip or in the control unit.After calibration, 1/α̂ is stored locally and output scaling can be combined with downstream processing.

F. Adaptive batch normalization statistics update (AdaBS) technique

AdaBS updates batch-normalization statistics after PCM-related changes by recomputing running means and variances from sampled training data. Its accuracy depends on tuning calibration momentum and batch size.

Batch normalization statistics: During training, batch normalization computes mini-batch mean µ_B and variance σ_B^2, then normalizes inputs before scaling and shifting them with γ and β.The learned γ and β parameters are obtained through backpropagation.
Batch normalization statistics: Running global mean µ and variance σ^2 are formed by exponentially averaging mini-batch statistics across training batches.The averaging uses momentum p.
Inference: During inference, batch normalization uses µ, σ^2, γ, and β to normalize, scale, and shift activations.These global statistics replace the mini-batch estimates used during training.
AdaBS calibration: AdaBS recalibrates µ and σ^2 for every batch-normalized layer by feeding randomly sampled mini-batches from the training dataset.Mini-batch size and momentum must be tuned for good inference accuracy.
AdaBS calibration: AdaBS uses an empirically estimated optimal momentum as a function of the number of injected mini-batches n.The authors describe the estimation algorithm in Supplementary Note 3.
AdaBS calibration: For ResNet-32, AdaBS injects 13 mini-batches of 200 images, approximately 5% of the CIFAR-10 training set.For ResNet-34 on ImageNet, the calibration uses batches of 50 images and 26 mini-batches, representing 0.1% of the ImageNet training set.