PaperlayerView exact revision on arXiv

Source-linked AI summary

An Emerging NVM-Based On-Chip Training Architecture with Non-Ideality Mitigation Through Bipolar Weight Distributions

Peng Dang, Youna Huang, Yintao He, Huawei Li

arXiv:2609.01948v1cs.AR

TL;DR

eNVM on-chip training promises lower data-movement overhead, but device non-idealities can make conductance updates miss intended weight values. The paper presents NOVA with experimentally calibrated FeFET modeling and NAT, which guides weights toward stable conductance regions. Across benchmark tasks, NAT improves accuracy under severe asymmetry, while NOVA reports substantially higher energy efficiency than peak GPU efficiency.

  • Problem

    eNVM non-idealities, including nonlinear conductance regions and variability, limit the accuracy and stability of on-chip neural-network training.

  • Method

    NOVA combines an eNVM on-chip-training architecture and experimentally calibrated FeFET modeling with NAT, which steers weights toward conductance-stable regions.

  • Results

    15.1% average training-accuracy improvement over baseline methods is achieved across multiple benchmark tasks under severe device asymmetry.

  • Takeaways & Limitations

    NAT maintains relatively high accuracy under severe nonlinearity, while NOVA provides a practical high-performance approach to eNVM on-chip training.

Abstract

from arXiv · show

The rapid advancement of deep learning has presented significant energy efficiency challenges to the conventional von Neumann architecture. In-memory computing (IMC) architectures based on emerging non-volatile memory (eNVM) are widely regarded as a promising solution for accelerating neural network training due to their high parallelism and low power consumption. However, the intrinsic non-idealities of eNVM devices can cause conductance updates to deviate from target values, thereby limiting the performance of on-chip training. To address this challenge, this paper presents a Non-ideality Optimized eNVM Accelerator (NOVA) architecture for on-chip training. Specifically, we first fabricate a two-dimensional (2D) ferroelectric field-effect transistor (FeFET) and develop a conductance modulation behavioral model calibrated with experimental data. Building upon this device model, we propose, for the first time, a Non-ideality Avoidance Training (NAT) algorithm tailored for eNVM devices, which mitigates accuracy degradation by guiding weight convergence toward the most stable conductance regions of eNVM devices. Experimental results demonstrate that, even under severe device asymmetry, NAT improves the accuracy by an average of 15.1\% over the baseline methods across multiple benchmark tasks. Meanwhile, the NOVA achieves an average energy efficiency gain of approximately 33.58$\times$ compared with the peak energy efficiency of graphics processing units (GPUs).

I. INTRODUCTION

The paper introduces NOVA, an eNVM-based architecture for end-to-end on-chip neural-network training, together with NAT to mitigate device non-idealities. Using experimentally calibrated FeFET modeling, the approach improves training accuracy under severe asymmetry and delivers substantial energy-efficiency gains.

  • Motivation: IMC reduces data movement by embedding computation in memory arrays, addressing the memory-wall overhead of conventional von Neumann architectures.eNVM arrays exploit parallel matrix-vector multiplication for efficient neural-network computation.
  • Motivation: On-chip training performs forward propagation, backpropagation, and weight updates within the eNVM array, reducing external-memory access overhead.It also enables hardware-aware training under real hardware conditions.
  • Challenges: Nonlinear, asymmetric conductance tuning and C2C/D2D variability cause weight-to-conductance errors that degrade training stability and accuracy.These effects make intended weight updates difficult to realize precisely in large-scale arrays.
  • Proposed approach: NOVA combines an eNVM on-chip-training architecture with a FeFET conductance-modulation model calibrated against experimental data.The architecture supports flexible dataflow switching while reusing a single set of peripheral circuits.
  • Proposed approach: NAT steers weights toward conductance-stable regions, reducing mapping errors and mitigating non-idealities during training.This differs from approaches that largely overlook realistic device behavior or depend on specialized fabrication processes.
  • Results: 15.1% average training-accuracy improvement over baseline methods is achieved across multiple benchmark tasks under severe device asymmetry.The paper also reports an average energy-efficiency improvement of 33.58× over peak GPU energy efficiency.

II. BACKGROUND AND MOTIVATION

This section describes how eNVM arrays implement neural-network training stages through analog matrix operations and explains the device non-idealities that complicate accurate updates. Forward and backward computations use different dataflow directions, while conductance tuning exhibits nonlinear and variable behavior.

  • Training workflow: On-chip training comprises forward propagation, backpropagation, gradient computation, and weight update.These stages are implemented within eNVM-based accelerator dataflows.
  • Forward propagation: During forward propagation, programmed conductances and DAC-converted activations perform MVM operations, with ADCs accumulating and digitizing column currents.Convolutions are commonly transformed into GEMM using Img2Col.
  • Error backpropagation: Backpropagation applies DAC-converted errors to array columns and uses the transpose of the original weight matrix for MTVM computation.This requires hardware support to reverse the dataflow direction.
  • Gradient and update: Gradient computation and parameter updates can use in-situ outer-product pulses or explicit peripheral gradient computation followed by weight writing.The in-situ method directly modulates cross-point conductance through coincident row and column pulses.
  • Device non-idealities: eNVM conductance changes exhibit nonlinearity and cycle-to-cycle variation, with stronger nonlinearity represented by larger parameter A.Positive and negative pulses produce long-term potentiation and depression, respectively.

B. Non-ideal Weight Update

eNVM conductance updates exhibit nonlinear behavior, variability, and asymmetry, causing deviations from intended weight updates and accuracy degradation. These effects motivate attention to the device regions and behaviors governing update stability.

  • Conductance changes rapidly during initial LTP and LTD pulses before gradually saturating, producing pronounced nonlinear update behavior.The nonlinear and asymmetric behavior is modeled as a function of programming pulse count P.
  • The parameter A controls conductance-update nonlinearity, while B normalizes the modulation using Gmax, Gmin, and Pmax.Smaller A produces more linear evolution; larger A produces stronger nonlinearity.
  • C2C and D2D variations introduce unpredictable write-operation fluctuations that compromise weight-update precision, training stability, and final accuracy.C2C variation arises across write/erase cycles, whereas D2D variation reflects fabrication, material, and microstructure differences.
  • Accuracy degradation reaches up to 25.0% when devices exhibit severe asymmetry.

C. Strategies for Mitigating Non-Idealities in eNVM Devices

NAT addresses eNVM non-idealities by steering neural-network weights toward stable HRS and LRS regions. NOVA combines this algorithmic strategy with a configurable FeFET-based architecture and shared peripheral resources for on-chip training.

  • C. Strategies for Mitigating Non-Idealities in eNVM Devices: Conventional near-zero-centered weights map mainly to intermediate conductance states, where NL, C2C, and D2D non-idealities are most pronounced.
  • C. Strategies for Mitigating Non-Idealities in eNVM Devices: HRS and LRS provide better retention, near-zero mapping deviation, lower pulse sensitivity, and fewer write iterations than intermediate conductance states.
  • C. Strategies for Mitigating Non-Idealities in eNVM Devices: NAT introduces strong regularization that gradually steers weights toward HRS or LRS during optimization.
  • C. Strategies for Mitigating Non-Idealities in eNVM Devices: Aligning weight distributions with hardware characteristics supports practical deployment by exploiting stable conductance regions.
  • C. Strategies for Mitigating Non-Idealities in eNVM Devices: NOVA dynamically schedules training data paths and computational tasks while sharing configurable peripheral resources across computation phases.
  • C. Strategies for Mitigating Non-Idealities in eNVM Devices: Each NOVA processing engine integrates an FeFET array, input/readout circuitry, and configurable dataflow control logic.The architecture uses FeFET arrays within processing engines as its core computing elements.

2) On-Chip Training Implementation:

NOVA implements on-chip training through configurable forward and backward dataflows, FeFET device fabrication, and experimentally fitted conductance-modulation modeling. Its switching network reverses array dataflow for backward computation and supports shared hardware resources.

  • 2) On-Chip Training Implementation:: Transmission-gate switch circuits dynamically switch NOVA processing elements between forward and backward tasks with low area and power overhead.The switches transpose the conductance matrix and enable peripheral hardware sharing across computation phases.
  • 2) On-Chip Training Implementation:: Forward propagation uses row-line inputs and column-line current accumulation, whereas backpropagation reverses the row-column connection for MTVM computation.
  • 1) Heterostructure preparation and device fabrication:: The fabricated device uses a CuInP2S6/MoS2 heterostructure assembled through dry transfer and patterned with Ti/Au contacts.
  • 2) Nonlinear Behavior Model:: The conductance-modulation model is fitted to experimental measurements at Vg = 0 V and Vds = 0.1 V.The fitted parameters are ALTP = 3.2 and ALTD = −4.3.
  • 2) Nonlinear Behavior Model:: For conductance decrease, the model uses the transformed pulse count P ′ = Pmax − P.GLTP and GLTD denote conductance during LTP and LTD phases, respectively.

3) D2D Behavior Model:

The device model captures both device-to-device and cycle-to-cycle variability in eNVM conductance updates. D2D variability perturbs nonlinear device parameters, while C2C variability scales with the update amplitude and is followed by quantization-aware clipping.

  • D2D variation model: D2D variability is modeled by adding Gaussian perturbations to nonlinear parameters for the LTP and LTD phases.Each device samples perturbed parameters bALTP and bALTD to represent fabrication-induced inter-device variation.
  • D2D variation model: σD ∈ {0, 0.1, . . . , 1.0} defines ten D2D variation intensity levels used to characterize conductance trajectories across devices.The model uses curve colors from dark to light for the ten variation levels.
  • C2C variation model: C2C variability is represented as multiplicative noise that scales linearly with the weight update amount.This contrasts with fixed-amplitude additive noise and better reflects programming behavior in which larger updates receive larger perturbations.
  • C2C variation model: Quantization-aware clipping is applied after each update to enforce the weight boundary constraint, while σC ∈ {0, 0.1, . . . , 1.0} indexes ten C2C intensity levels.The C2C model captures conductance evolution across programming cycles.

C. Training Algorithm Tailored for Non-ideal eNVM

NAT combines quantization-aware training with a magnitude regularizer that drives weights toward ±1, aligning the weight distribution with stable eNVM conductance regions. Its gradient jointly balances task optimization and hardware-oriented weight shaping.

  • Training algorithm: NAT uses 8-bit quantization-aware training for weights, activations, gradients, and errors, while restricting weights and activations to [−1, +1].The low-precision tensors are intended to operate within the accelerator’s limited conductance states.
  • Training objective: The regularization term Lmag penalizes p-norm deviation of weight magnitudes from unity, with p = 1 by default, and λ(t) uses a piecewise linear warm-up schedule.The schedule avoids imposing overly aggressive constraints during early training.
  • Training objective: The total objective combines task loss and magnitude regularization, so backpropagation supplies the task gradient while a separate regularization gradient shapes the weights.The coefficient λ controls the regularization strength.
  • Weight polarization: For |wi| ≠ 1, the regularization gradient pushes weights toward magnitude 1, producing a bipolar distribution near ±1.The method uses a subgradient because the absolute value function is non-differentiable at zero.
  • Weight polarization: NAT differs from L1 and L2 regularization by targeting hardware-friendly boundary regions rather than driving weights toward zero.The paper characterizes this as physical orientation that reduces mapping deviations and update errors in eNVM accelerators.

D. Simulation Framework for On-Chip Training with eNVM

The software framework simulates each on-chip training iteration through weight mapping, conductance tuning, and analog computation. It converts ideal updates into non-ideal conductances, applies pulse-based updates and variability, then performs forward or backward analog operations.

  • Iteration workflow: Each training iteration comprises weight mapping, conductance tuning, and analog computation.The framework is designed to simulate the complete on-chip training process of eNVM accelerators.
  • Weight mapping: During weight mapping, ideal gradient-based updates are scaled to the conductance window, stochastically rounded, and converted into conductance values using nonlinear and D2D models.The mapped quantities include actual conductance G and conductance update ΔG.
  • Conductance tuning: During conductance tuning, conductance values are converted to pulse representations, updated through pulse accumulation, remapped to conductance states, and perturbed by C2C variation.This models the device programming sequence rather than applying an ideal numerical update.
  • Analog computation: During analog computation, the conductance array performs analog MVM and MTVM operations, with switching circuitry selecting forward or backward dataflow.Crossbar outputs are read and decoded to complete neural-network computation.

IV. EVALUATION

The evaluation combines hardware-aware training simulations, device behavioral models, and hardware performance modeling across multiple network architectures. NAT reshapes VGG11 weights from a zero-centered unimodal distribution toward a symmetric bimodal distribution near ±1.

  • Evaluation setup: The evaluation uses PyTorch-based hardware-aware training with conductance models and NAT across VGG6, VGG11, and ResNet18.The framework incorporates SGD with momentum, learning-rate warm-up, and data augmentation.
  • Evaluation setup: NOVA hardware performance is evaluated with NeuroSim and MNSim, using CIPS/MoS2 FeFET-based 1T1F crossbar arrays for forward and backward computation.The evaluation also models NOVA’s dataflow-switching circuitry analytically.
  • Weight-distribution evaluation: Conventional VGG11 training produces a near-zero-mean unimodal distribution with σ = 0.45 and skewness 0.02, concentrating many weights near zero.This baseline is shown for the sixth layer of VGG11.
  • Weight-distribution evaluation: NAT increases VGG11 layer-6 standard deviation from σ = 0.74 at epoch 50 to σ = 0.82 at epoch 300 and σ = 0.98 at λ = 1e−2.The distribution becomes increasingly bimodal as training proceeds and regularization strengthens.
  • Weight-distribution evaluation: NAT preserves approximately 50% negative weights while mean and skewness remain near zero, yielding a symmetric ±1-oriented distribution without systematic bias.The reported transformation reduces weight density near zero and improves compatibility with eNVM characteristics.

2) Evaluation of the Impact of NAT Algorithm Regularization Strength:

NAT’s regularization strength determines how effectively weights are guided toward stable conductance regions without sacrificing optimization flexibility. Across the evaluated settings, λ = 10−3 provides the best balance between device robustness and model performance.

  • Regularization strength: Excessively large λ concentrates weights in boundary regions, reducing parameter-optimization freedom and impairing convergence.The degradation begins when λ reaches 10−2.
  • Regularization strength: Moderate λ values guide weights toward stable boundary regions near ±1 while retaining intermediate-region weights.This reduces device-related update errors while preserving representational capacity.
  • Evaluation setup: NAT evaluation varies device nonlinearity through ALTP/ALTD and examines D2D variability using σD values from 0.0 to 1.0.The comparisons include TaOx/TiO2 OxRRAM, PCMO ReRAM, and IZO FeFET devices.
  • Nonlinearity and D2D results: 15.1% average absolute accuracy improvement is achieved by NAT over the baseline across three tasks under the strongest nonlinearity condition.For VGG6-SVHN, baseline accuracy falls from 94.8% to 86.71%, whereas NAT falls from 94.68% to 93.61%.
  • Nonlinearity and D2D results: Approximately 0.87%, 1.22%, and 1.95% accuracy drops occur on VGG6-SVHN, VGG11-CIFAR10, and ResNet18-CIFAR100 as σD rises from 0.0 to 1.0.These changes are measured under the strongest nonlinearity condition, (3.58/−6.76).

2) Evaluation of NAT under Device Nonlinearity and C2C Variations:

NAT maintains higher accuracy than baseline training under device nonlinearity and shows only mild degradation as C2C variability increases. The broader hardware analysis identifies weight writes as the dominant energy bottleneck, while analog crossbar computation consumes very little energy.

  • C2C variation: NAT consistently achieves higher classification accuracy than the baseline under severe device nonlinearity when σC = 0.0.The comparison covers all evaluated tasks and uses identical training settings for the baseline.
  • C2C variation: 2.8% is the approximate VGG11-CIFAR10 accuracy drop when σC increases from 0.0 to 0.5 under (3.58/−6.76) nonlinearity.Accuracy decreases from 92.29% to 89.47%.
  • C2C variation: NAT pushes weights toward extreme HRS and LRS regions, where C2C fluctuations have clearer boundaries and smaller relative variation.This partially mitigates C2C noise, although increasing σC still produces a clear accuracy decline.
  • Energy and latency: Weight writes overwhelmingly dominate total energy and latency during on-chip training across the evaluated eNVM devices.The analysis compares TaOx/TiO2, PCMO, IZO, and CIPS/MoS2 devices.
  • Energy and latency: IZO achieves the shortest training latency across the three tasks because its linearity and tunability reduce the required number of programming pulses.TaOx/TiO2 has high energy and latency per update, while PCMO lowers energy through a shorter pulse width but still uses many pulses.
  • Energy and latency: Analog computation in the crossbar array consumes extremely low energy compared with the dominant weight-update cost.The analysis highlights reducing programming voltage and shortening pulse width as write-optimization targets.

2) Area Breakdown:

The NOVA architecture combines algorithmic non-ideality mitigation with a complete on-chip training dataflow and competitive energy efficiency. Its area is dominated by ADCs, while comparisons report strong performance at 8-bit input and weight precision.

  • Area breakdown: 43.75 mm2 is the total NOVA system area, with 6-bit ADCs contributing the largest share at 39.6%.The weight-gradient computation unit accounts for 20.8% and buffers for 18.3%.
  • Architecture and mitigation: NOVA reduces weight density in highly nonlinear intermediate-conductance regions and guides weights toward relatively stable, approximately linear boundary regions.This algorithmic strategy is jointly optimized with the task loss to minimize weight-mapping errors.
  • Accuracy comparison: 15.1% average accuracy improvement is reported across multiple networks and datasets compared with the existing method in.The comparison is presented as an experimental result for NOVA’s non-ideality mitigation strategy.
  • Hardware performance comparison: 51.08 TOPS/W forward-propagation energy efficiency is achieved at (8b, 8b) input and weight precision.NOVA supports a complete training dataflow while maintaining competitive forward-propagation efficiency relative to the cited accelerators.
  • Hardware performance comparison: 51.08 TOPS/W forward and 47.63 TOPS/W backward energy efficiencies are reported at (8b, 8b) precision during on-chip training.Relative to the design in, these correspond to 14.27× and 14.61×, respectively.
  • Conclusion: The paper fabricates CIPS/MoS2 heterojunction FeFETs, fits behavioral models to measured electrical data, and develops NAT for eNVM non-idealities.The conclusion states that NAT achieves accuracy comparable to ideal devices under highly asymmetric device characteristics.
Loading 2609.01948v1…