Source-linked AI summary
Mixed-Precision In-Memory Computing
Manuel Le Gallo, Abu Sebastian, Roland Mathis, Matteo Manica, Heiner Giefers, Tomas Tuma, Costas Bekas, Alessandro Curioni, Evangelos Eleftheriou
TL;DR
In-memory computing promises efficient computation but device variability and non-ideal behavior can limit numerical accuracy. This paper combines approximate in-memory computation with high-precision iterative refinement, demonstrating accurate linear-system solutions while retaining in-memory processing for the computationally intensive operations.
Problem
Device variability and non-ideal characteristics make it difficult for in-memory computing to reach the numerical accuracy required in data analytics and scientific computing.
Method
A low-precision computational memory unit performs bulk matrix-vector operations, while a high-precision processor computes residuals and iteratively refines the solution.
Results
∼1.3 · 10^-15 minimum error was reached for N = 500 with tol = 10^-15, limited by the high-precision processor's machine precision.
Takeaways & Limitations
Mixed-precision in-memory computing can achieve high solution accuracy while keeping the bulk of computation in low-precision in-memory hardware.
Takeaways & Limitations
The experiments used well-conditioned, relatively small-scale problems; broader condition numbers require greater computational-memory precision.
Abstract
from arXiv · showhide
As CMOS scaling reaches its technological limits, a radical departure from traditional von Neumann systems, which involve separate processing and memory units, is needed in order to significantly extend the performance of today's computers. In-memory computing is a promising approach in which nanoscale resistive memory devices, organized in a computational memory unit, are used for both processing and memory. However, to reach the numerical accuracy typically required for data analytics and scientific computing, limitations arising from device variability and non-ideal device characteristics need to be addressed. Here we introduce the concept of mixed-precision in-memory computing, which combines a von Neumann machine with a computational memory unit. In this hybrid system, the computational memory unit performs the bulk of a computational task, while the von Neumann machine implements a backward method to iteratively improve the accuracy of the solution. The system therefore benefits from both the high precision of digital computing and the energy/areal efficiency of in-memory computing. We experimentally demonstrate the efficacy of the approach by accurately solving systems of linear equations, in particular, a system of 5,000 equations using 998,752 phase-change memory devices.
I. MIXED-PRECISION IN-MEMORY LINEAR EQUATION SOLVER
Mixed-precision in-memory computing assigns approximate, high-load operations to a computational memory unit and accurate error correction to a high-precision processor. For linear systems, iterative refinement uses high-precision residuals and low-precision inner solves, with tolerable perturbations depending on matrix conditioning.
- Algorithm: The hybrid solver computes approximate corrections in memory while the high-precision unit calculates residuals and iteratively refines the solution.The residual is r = b−Ax, and refinement continues until its norm falls below tol.
- Application: The method targets dense covariance-matrix problems common in cognitive computing and data analytics.A is nonsingular, b is known, and x is updated until the residual norm reaches the desired tolerance.
- Architecture: The computational memory unit performs the bulk of computation, while the von Neumann processor manages high-precision operations and system control.Figure 1 separates digital logic, analog in-memory computation, and bus-based control, data, and addressing.
- Algorithm: Matrix-vector multiplication is the computationally intensive Krylov-subspace operation and is therefore mapped to memristive crossbar conductances.The inner solver builds a basis from repeated products w^(k) = Av^(k) and approximates A^-1r.
- Accuracy: The system can tolerate approximate inner-solver perturbations because outer iterative refinement drives the solution toward high accuracy.The tolerated perturbation magnitude decreases as the condition number of A increases.
II. IN-MEMORY MULTIPLICATIONS WITH PCM DEVICES
Phase-change memory devices implement approximate scalar products by mapping operands to conductances and voltages, then averaging device outputs. Experiments characterize error reduction with averaging and account for device drift during matrix-vector computations.
- Hardware: One million PCM devices in a 512 × 2048 crossbar prototype implement low-precision matrix-vector multiplication.The array was integrated in 90-nm CMOS, with each crosspoint containing a PCM device and access transistor.
- Scalar multiplication: PCM scalar multiplication maps β_n to conductance G_n and γ_n to voltage V_n, using a polynomial correction for nonlinear current-voltage behavior.G_n spans approximately 0–50 µS and V_n approximately 0.1–0.3 V.
- Scalar multiplication: 1024 × K devices are used to average 1024 operand combinations, and the analogue computation covers approximately two decades of current.The measured current is converted into an approximate product and compared with the double-precision result.
- Error behavior: The error standard deviation scales as K^-0.5, so averaging more devices narrows the computed-product error distribution.This behavior is consistent with averaging independent, identically distributed perturbations.
- Matrix-vector multiplication: Hardware performs element-wise matrix-vector multiplications, while summation occurs outside the chip because the prototype permits serial crosspoint access.A calibration procedure periodically reads summed conductance to compensate for global drift during experiments.
III. ACCURATELY SOLVING LINEAR EQUATIONS USING PCM HARDWARE
The mixed-precision method solves covariance-based linear systems by combining PCM-based approximate computation with iterative high-precision refinement. Experiments reached the desired accuracy for model matrices and for 40-gene cancer and normal-tissue systems.
- Model covariance matrices: 23 iterative refinements were required for the N = 500 covariance-matrix experiment to reach the desired accuracy.
- Model covariance matrices: A reduced banded matrix can serve as inexact input to the inner solver while outer refinement seeks the exact solution.
- Accuracy and scale: The minimum experimental error for N = 500 at tol = 10^-15 was ∼ 1.3 · 10^-15, limited by the high-precision unit's machine precision.
- RNA-based gene interactions: The method converged to the desired precision for all 40 cancer and normal-tissue linear systems, producing an interactome identical to the exact result.
- RNA-based gene interactions: Cancer and normal tissues exhibited different autophagy-gene connectivity patterns, with cancer showing a sparsely connected network.
IV. PERFORMANCE ASSESSMENT AND CURRENT LIMITATIONS
The assessment finds that mixed-precision in-memory computing can improve time and energy to solution relative to high-precision CPU and GPU implementations. The authors also identify hardware and problem-scale boundaries for the demonstrated system.
- Current limitations: The experiments used well-conditioned, relatively small problems because the available hardware had limited size and precision.
- Performance assessment: Mixed-precision in-memory computing outperformed high-precision CPU- and GPU-based implementations in time and energy to solution.
- Performance assessment: 6.8× to 24× dynamic energy gains were measured depending on computational-memory precision and assumed device noise.
- Scaling requirements: Ten 1000 × 1000 crossbars operating in parallel at 1 µs or less are expected to approach optimal speed performance.
- Performance assessment: A memristive crossbar could offer up to 80 times lower energy consumption than a comparable 4-bit FPGA matrix-vector multiplier.
V. CONCLUSIONS
The paper introduces mixed-precision in-memory computing, combining low-precision in-memory computation with high-precision iterative refinement. Experiments show accurate linear-system solutions while retaining in-memory computation for the bulk of the workload.
- Mixed-precision in-memory computing uses a computational memory unit for bulk computation and a high-precision processor for iterative accuracy improvement.
- The approach can achieve arbitrarily high solution accuracy while the bulk of computation remains low-precision in-memory computing.
- Experiments solved linear systems using phase-change memory for analogue matrix-vector multiplication on model covariance matrices and RNA-expression data.
- The paper identifies automatic control, optimization, machine learning, deep learning, and signal processing as future application areas.
Experimental platform.
The experimental platform combines a prototype phase-change-memory chip with FPGA and analogue front-end hardware. Experiments use a 1-million-device sub-array fabricated in 90-nm CMOS, with device programming and readout supported on-chip.
- The prototype PCM chip contains 3 million devices and interfaces with two FPGA boards and an analogue-front-end board.
- Experiments use the chip’s second sub-array, containing 1 million devices arranged as 512 word lines and 2048 bit lines.
- The PCM devices were integrated in 90-nm CMOS using doped Ge2Sb2Te5 phase-change material.
- Device reads use constant-voltage bit-line bias, capacitor-based current integration, and an on-chip 8-bit cyclic ADC.
Solving the linear system for model covariance matrices.
For model covariance matrices, the method solves linear systems with Conjugate Gradient iterations whose matrix-vector products are computed using the PCM-coded matrix. The final refined solution is obtained from the inner solver’s output.
- The experiments solve covariance-matrix systems with Conjugate Gradient as the inner Krylov-subspace solver.
- Each Conjugate Gradient iteration computes w(k) := ˜Av(k) in the memristive array.
- The algorithm updates the solution using α(k), residual vectors, and β(k) before generating the next search direction.
- The final solution is z(m+1), with ˜A denoting the matrix coded in the PCM chip.
- A reduced banded matrix retains entries satisfying 0 ≤ |i−j| ≤ 12 and sets the remaining entries to zero.
- The experiments use five Conjugate Gradient iterations for N = 500 and ten for N = 5,000, with iterative-refinement tolerance tol = 10^-5.
Estimation of gene interactions from RNA measurements.
The paper estimates gene-interaction networks from TCGA RNA-sequencing data by solving inverse-covariance systems with mixed-precision in-memory computing. Partial correlations above a threshold are then used to build gene- and pathway-level interaction graphs.
- The analysis uses RNA-Seq measurements for 40 genes from the KEGG autophagy pathway.
- 946 normal samples and 11,935 cancer samples are considered, with 946 cancer profiles subsampled for equal-size network comparisons.
- The sample covariance Aij is computed from gene-expression values Xsi and gene-specific means μi across samples.
- The inverse covariance matrix is obtained by solving 40 mixed-precision linear systems, one for each column, using diagonal preconditioning.
- The PCM-coded matrix has zero diagonal elements, and its nonzero elements are represented using four devices averaged per element.
- GMRES computes matrix-vector products in the memristive array, while the identity contribution is added separately.
- GMRES uses five iterations and iterative-refinement tolerance tol = 10^-3, with drift calibration at every first iteration.
- Interactions are retained when |ρij| exceeds τ = 0.13, corresponding to the 90th percentile of the normals.
Data availability.
The data supporting the paper’s plots and other findings are available from the corresponding author upon reasonable request.
- Data supporting the paper’s plots and other findings are available from the corresponding author upon reasonable request.
COMPETING INTERESTS
The authors declare no competing financial interests.
- The authors declare no competing financial interests.