PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixed-Precision In-Memory Computing

Manuel Le Gallo, Abu Sebastian, Roland Mathis, Matteo Manica, Heiner Giefers, Tomas Tuma, Costas Bekas, Alessandro Curioni, Evangelos Eleftheriou

arXiv:1701.04279v5cs.ET

TL;DR

Accurate solutions with computational memory are challenging because device variability and non-ideal characteristics limit precision. This paper combines in-memory computation with high-precision iterative refinement, experimentally solving a 5,000-equation system using 998,752 phase-change memory devices.

  • Problem

    Computational memory's limited precision leaves open how to obtain high-accuracy solutions using its low-precision computations.

    Source: §V · p. 7

  • Method

    The computational memory unit performs matrix-vector products, while high-precision residuals guide iterative refinement using an inexact inner solver.

    Source: §I · pp. 1–2

  • Results

    1.3 · 10^-15 was the minimum error for N = 500, limited by the high-precision unit's machine precision; convergence was also obtained for matrix sizes up to N = 10,000.

    Source: §III · p. 4

  • Takeaways & Limitations

    The demonstration supports partly performing linear analysis with computational memory without sacrificing overall computational accuracy.

    Source: §IV · p. 6

    The tested problems were well-conditioned and relatively small due to hardware size and precision limits; broader condition numbers require greater computational-memory precision.

    Source: §IV · p. 6

Abstract

from arXiv · show

As CMOS scaling reaches its technological limits, a radical departure from traditional von Neumann systems, which involve separate processing and memory units, is needed in order to significantly extend the performance of today's computers. In-memory computing is a promising approach in which nanoscale resistive memory devices, organized in a computational memory unit, are used for both processing and memory. However, to reach the numerical accuracy typically required for data analytics and scientific computing, limitations arising from device variability and non-ideal device characteristics need to be addressed. Here we introduce the concept of mixed-precision in-memory computing, which combines a von Neumann machine with a computational memory unit. In this hybrid system, the computational memory unit performs the bulk of a computational task, while the von Neumann machine implements a backward method to iteratively improve the accuracy of the solution. The system therefore benefits from both the high precision of digital computing and the energy/areal efficiency of in-memory computing. We experimentally demonstrate the efficacy of the approach by accurately solving systems of linear equations, in particular, a system of 5,000 equations using 998,752 phase-change memory devices.

I. MIXED-PRECISION IN-MEMORY LINEAR EQUATION SOLVER

The mixed-precision solver combines high-precision digital processing with low-precision in-memory computation to solve dense covariance matrix problems. Iterative refinement uses the in-memory unit for computationally intensive matrix-vector products and the digital unit to improve solution accuracy.

I. MIXED-PRECISION IN-MEMORY LINEAR EQUATION SOLVER

  • Iterative refinement computes the residual r = b−Ax at high precision, solves Az = r with an inexact inner solver, and updates the solution until the residual norm is below tol.The target problems are dense covariance matrix problems common in cognitive computing and data analytics.Source: §I · p. 1
  • Krylov methods build a basis through repeated matrix-vector multiplications and use it to approximate the error-correction term A−1r.The passage names Conjugate Gradient and GMRES as examples of Krylov subspace methods.Source: §I · pp. 1–2
  • The solver maps the computationally intensive product Av(k) to a memristive crossbar programmed with A, computing in situ without intermediate data movement.Although approximate in-memory computation perturbs the inner solver, outer iterative refinement ensures convergence to a high-accuracy solution.Source: §I · p. 2, Figure 1 · p. 2
  • The perturbations the solver can tolerate decrease as the condition number of A increases.The condition number reflects how much the solution x changes in response to a change in b.Source: §I · p. 2

II. IN-MEMORY MULTIPLICATIONS WITH PCM DEVICES

A prototype PCM crossbar was used to implement low-precision scalar and matrix-vector multiplications. Averaging device measurements reduces scalar-product error, while the matrix-vector setup performs summation off-chip and uses calibration to address conductance drift.

PCM hardware

  • The prototype chip contains a 512 × 2048 PCM crossbar array integrated in 90-nm CMOS, with each crosspoint comprising a PCM device and access transistor.The chip contains one million PCM devices whose conductance can be programmed by altering their amorphous/crystalline phase configuration.Source: §II · p. 2

Scalar multiplication

  • For 1,024 uniformly generated input pairs, β was mapped to conductance and γ to voltage, and multiplication used a pseudo-Ohm’s-law model of nonlinear PCM current.Devices were programmed and verified iteratively; the computation was realized over approximately two decades of current.Source: §II · pp. 2–3
  • The scalar-product error standard deviation scales as K^-0.5 when averaging K devices, consistent with random current perturbations.Likely sources include programming inaccuracies, device-to-device current/voltage variability, conductance variations and low-frequency noise.Source: §II · p. 3

Matrix-vector multiplication

  • Matrix-vector multiplication encodes matrix elements in PCM conductance states, but the hardware performs only elementwise products; summation occurs off-chip.The experimental hardware allows serial access to individual crosspoints.Source: §II · p. 3

Drift calibration

  • Periodic readings of summed conductance from a subset of matrix-encoding devices estimate global conductance shifts and calibrate against PCM drift without assuming how conductance changes.The procedure is designed for implementation in a crossbar array.Source: §II · p. 4

III. ACCURATELY SOLVING LINEAR EQUATIONS USING PCM HARDWARE

The section demonstrates mixed-precision PCM solving on model covariance systems and RNA-derived gene covariance data, using iterative refinement to obtain accurate solutions.

III. ACCURATELY SOLVING LINEAR EQUATIONS USING PCM HARDWARE

  • For N=500, the experiment converged after 23 refinements, and banding to 12 entries per side gave a rate almost identical to using the full matrix.The banded N=500 experiment used K=4 devices per matrix element.Source: §III · p. 4
  • N=5,000 banded systems converged with K=8 using 23 high-precision matrix-vector multiplications, versus 50 for high-precision CG at the same accuracy.The comparison reflects one high-precision CG run to obtain the same solution accuracy.Source: §III · p. 4
  • At N=500 and tol=10^-15, minimum error was ~1.3·10^-15, limited by processing-unit machine precision; convergence was also obtained up to N=10,000 with speedup methods.Iterative refinement makes maximum accuracy depend on the high-precision processing unit, not the computational memory unit.Source: §III · p. 4
  • For RNA-derived covariance matrices from 946 normal and 946 cancer samples, PCM with GMRES reached desired precision on all 40 gene systems, yielding an interactome identical to the exact-Σ result.The 40 genes were selected from the KEGG autophagy pathway.Source: §III · pp. 4–5, Figure 4 · p. 5
  • Cancer-tissue partial correlations produced a sparser, differently connected network than normal tissue, where upstream signals correlated with many downstream autophagy-formation targets.Some gene interactions were preserved between tissues; the displayed interactomes use partial-correlation magnitude greater than 0.13 as a threshold.Source: §III · p. 5, Figure 4 · p. 5

IV. PERFORMANCE ASSESSMENT AND CURRENT LIMITATIONS

Mixed-precision in-memory computing can improve solution time and energy while retaining computational accuracy, but its performance depends on problem and hardware conditions. The section reports energy comparisons, implementation requirements, and limitations on scale and conditioning.

Performance assessment

  • The demonstration points toward energy-efficient large-scale analytics, where data transfers in von Neumann architectures are identified as a major energy cost as datasets grow.Source: §IV · p. 6

Current limitations

  • Experiments covered well-conditioned, relatively small problems; scale-up and broader conditioning require larger or parallel arrays and greater computational-memory precision.Potential precision improvements include reducing device variability and conductance noise, mapping matrix bits to multiple physical columns, or using error correction; preconditioning may also improve Krylov-solver efficiency.Source: §IV · p. 6
  • The approach can address more general matrices when the chosen Krylov solver's convergence conditions hold: CG requires symmetric positive-definite matrices, while GMRES handles broader cases, including nonsymmetric matrices, at higher complexity.Source: §IV · p. 6

Performance assessment

  • For matrix (3), maximum measured dynamic energy gains were 6.8× at current-PCM-like precision and 24× assuming two orders less computational-memory noise.The system outperformed high-precision CPU- and GPU-based implementations in time and energy to solution. Measurements included transfers, conversions, and extra floating-point operations, but assumed negligible computational-memory time and energy, giving an upper bound; gains depend on the matrix and right-hand side.Source: §IV · p. 6
  • For near-optimal speed, the analysis specifies ten 1000 × 1000 crossbars in parallel at a cycle time of 1 µs or less.Only reads are used during the algorithm, so crossbar power overhead is expected to be minimal, but peripheral circuitry and I/O converters remain important design requirements.Source: §IV · p. 6
  • A prototype-like memristive crossbar could use up to 80 times less energy than a 4-bit FPGA matrix-vector multiplier, even when all FPGA coefficients are stored on-chip.The FPGA multiplier's accuracy was comparable to that of the prototype PCM chip, and the estimate excluded off-chip data transfers.Source: §IV · p. 6

V. CONCLUSIONS

Mixed-precision in-memory computing pairs low-precision computation in a computational memory unit with a high-precision processor that iteratively improves solution accuracy. The authors experimentally demonstrate the approach for linear systems and identify broader computational applications as future work.

  • The approach was experimentally demonstrated on linear systems using a PCM chip for analogue matrix-vector multiplication, with model covariance matrices and real-world RNA expression data.Source: §V · p. 7
  • A computational memory unit performs most computation, while a high-precision processor applies a backward method to iteratively improve solution accuracy.Source: §V · p. 7
  • The hybrid approach can achieve arbitrarily high solution accuracy while realizing most computation as low-precision in-memory computing.Source: §V · pp. 6–7
  • Future work will generalize the approach beyond linear systems to automatic control, optimization, machine learning, deep learning, and signal processing.Source: §V · p. 7

METHODS · Experimental platform.

The experimental platform combines a 3-million-device PCM chip with FPGA and analog-front-end boards, while Matlab coordinates experiments and implements the linear-equation algorithms and data conversions. All experiments used the chip’s 1-million-device sub-array, which supports calibrated readout and iterative program-and-verify programming.

Experimental platform.

  • The platform pairs a 3-million-device PCM chip with two FPGA boards, an analog-front-end board, and a host computer, with Matlab coordinating experiments and implementing algorithms and data conversions.The FPGAs handle system control, data management, and the data-processing-unit interface; the AFE provides power supplies and voltage and current references.Source: Experimental platform · p. 7
  • The PCM devices use doped Ge2Sb2Te5 in a 90-nm CMOS chip fabricated with the key-hole process described in Ref. 44.The bottom electrode has an approximately 20-nm radius and 65-nm length; the phase-change material is approximately 100 nm thick and extends to a top electrode with an approximately 100-nm radius.Source: Experimental platform · p. 7
  • All experiments used the 1-million-device sub-array with two parallel 240-nm-wide access transistors per device, arranged as 512 WL × 2048 BL.Devices are selected by serially addressing a word line and a bit line.Source: Experimental platform · p. 7
  • Each PCM read takes 1 µs: the selected bit line is typically biased at 100–300 mV, and an on-chip 8-bit cyclic ADC digitizes the integrated sensed current.Readout is calibrated using on-chip reference polysilicon resistors.Source: Experimental platform · p. 7
  • Programming uses off-chip voltage converted on-chip to a selected-bit-line current, delivered as a 400-ns rectangular pulse with amplitude from 0 to 500 µA.The current is mirrored into the selected bit line for the desired pulse duration.Source: Experimental platform · p. 7
  • Iterative program-and-verify steps adjust programming current according to conductance error and stop below a 1.74 µS error margin or after 20 iterations.Each pulse is followed by a conductance read at 0.2 V; a program-and-verify step takes approximately 2.5 µs. The procedure is described in Ref. 45.Source: Experimental platform · pp. 7–8

Solving the linear system for model covariance matrices.

For covariance matrices defined by Eq. (3), the study used mixed-precision in-memory computing with Conjugate Gradient (CG) as the inner Krylov-subspace solver and generated b uniformly in [0,1].

Solving the linear system for model covariance matrices.

  • CG performed matrix-vector products with the PCM-coded matrix and returned z(m+1) as the final solution.The coded matrix ˜A could be the full matrix A or a reduced banded version.Source: Solving the linear system for model… · p. 8
  • In the reduced banded version, ˜A retains A_ij when 0 ≤ |i−j| ≤ 12 and sets other entries to zero.Source: Solving the linear system for model… · p. 8
  • The solver used m = 5 iterations for N = 500 and m = 10 for N = 5,000, with iterative-refinement tolerance tol = 10^-5.Nonzero matrix elements were encoded using K devices averaged per element; drift calibration ran at the first CG iteration on S = 10,000 devices.Source: Solving the linear system for model… · p. 8

Estimation of gene interactions from RNA measurements.

The study estimated gene interactions from TCGA RNA-Seq profiles by computing an inverse covariance matrix with mixed-precision in-memory computing, then deriving and visualizing thresholded partial correlations. Its analysis compared cancer and normal samples matched in size and grouped genes by KEGG Orthology.

Estimation of gene interactions from RNA measurements.

  • The analysis used normalized RNA-Seq data for 40 KEGG autophagy-pathway genes, matching 946 cancer profiles to 946 normal samples for network comparison.The source included 11,935 cancer samples; a Kolmogorov–Smirnov test with Bonferroni correction was reported to show no evidence of a difference after subsampling (p < 0.05).Source: Estimation of gene interactions from… · p. 8
  • The inverse covariance matrix was obtained by solving 40 preconditioned linear systems with mixed-precision in-memory computing, using M = diag(A).The sample covariance A_ij was computed from gene expression values and their means; the preconditioned system was M−1Ax(n) = M−1e(n).Source: Estimation of gene interactions from… · p. 8
  • GMRES used five iterations and 10−3 refinement tolerance, with off-diagonal elements of ˜A coded using four averaged PCM devices per element.The matrix-vector product added the identity term digitally rather than coding diagonal entries, and drift calibration was performed at the first iteration.Source: Estimation of gene interactions from… · pp. 8–9
  • Gene interactions for visualization were retained when |ρij| > 0.13, a threshold corresponding to the 90th percentile of normal samples.Genes were grouped by KEGG Orthology, and group interaction strength was defined by averaging the partial correlations for qualifying interactions.Source: Estimation of gene interactions from… · p. 9

Related papers

Loading 1701.04279v5…