PaperlayerView exact revision on arXiv

Source-linked AI summary

Computing in Memory with Spin-Transfer Torque Magnetic RAM

Shubham Jain, Ashish Ranjan, Kaushik Roy, Anand Raghunathan

arXiv:1703.02118v4cs.ET

TL;DR

Processor-memory data movement is a major performance and energy bottleneck, motivating computation within memory. The paper proposes STT-CiM with circuit, ECC, architectural, and data-mapping techniques, and reports system-level performance and memory-energy improvements.

  • Problem

    Processor-memory data movement creates a major performance and energy bottleneck, motivating in-memory computation that reduces accesses and transferred data.

  • Method

    STT-CiM enables multiple-wordline sensing in standard STT-MRAM to perform arithmetic, logic, and vector operations, with ECC, ISA, bus, and data-mapping extensions.

  • Results

    3.93x average system-performance improvement and 3.83x average total memory-energy improvement were achieved, with maxima of 10.4x and 12.4x, respectively.

  • Takeaways & Limitations

    STT-CiM shows promise for alleviating the processor-memory gap while preserving the standard STT-MRAM core data array.

Abstract

from arXiv · show

In-memory computing is a promising approach to addressing the processor-memory data transfer bottleneck in computing systems. We propose Spin-Transfer Torque Compute-in-Memory (STT-CiM), a design for in-memory computing with Spin-Transfer Torque Magnetic RAM (STT-MRAM). The unique properties of spintronic memory allow multiple wordlines within an array to be simultaneously enabled, opening up the possibility of directly sensing functions of the values stored in multiple rows using a single access. We propose modifications to STT-MRAM peripheral circuits that leverage this principle to perform logic, arithmetic, and complex vector operations. We address the challenge of reliable in-memory computing under process variations by extending ECC schemes to detect and correct errors that occur during CiM operations. We also address the question of how STT-CiM should be integrated within a general-purpose computing system. To this end, we propose architectural enhancements to processor instruction sets and on-chip buses that enable STT-CiM to be utilized as a scratchpad memory. Finally, we present data mapping techniques to increase the effectiveness of STT-CiM. We evaluate STT-CiM using a device-to-architecture modeling framework, and integrate cycle-accurate models of STT-CiM with a commercial processor and on-chip bus (Nios II and Avalon from Intel). Our system-level evaluation shows that STT-CiM provides system-level performance improvements of 3.93x on average (upto 10.4x), and concurrently reduces memory system energy by 3.83x on average (upto 12.4x).

I. INTRODUCTION

The paper proposes STT-CiM, which uses STT-MRAM’s multiple-wordline capability to perform in-memory computation while addressing reliability and processor-system integration. It evaluates the design’s performance and energy benefits in a programmable processor system.

  • Motivation: In-memory computing reduces processor-memory data movement, a major performance and energy bottleneck, by computing within memory structures.It also exploits the wider internal bandwidth available within memory systems.
  • STT-CiM Design: Simultaneously enabling multiple wordlines and sensing each bit-line’s effective resistance allows direct computation of logic functions over stored bit-cell values.This capability is easier in STT-MRAM than in CMOS memories because its bit-cells are resistive.
  • STT-CiM Design: STT-CiM performs arithmetic, logic, and vector operations using an enhanced STT-MRAM array without modifying its bit-cells or core data array.The design therefore maintains standard STT-MRAM memory density and read/write efficiency.
  • Reliability: Extended ECC mechanisms address reliable in-memory computation under process variations, where sensing challenges are aggravated by CiM operations.The paper identifies reliable computation under process variation as a key STT-CiM challenge.
  • System Integration: ISA and on-chip-bus extensions integrate STT-CiM as a scratchpad in an Intel Nios II processor system, with data mapping techniques exposed to software.The integration uses Intel’s Avalon on-chip bus and supports programmable processor operation.
  • Evaluation: 3.83x average total memory-energy improvement and 3.93x average system-performance improvement were achieved, with maxima of 12.4x and 10.4x, respectively.These results are reported for the evaluated STT-CiM system.

II. RELATED WORK

Related work distinguishes near-memory computing, which places processing units close to memory, from in-memory computing, which integrates logic into memory arrays. The paper positions STT-CiM as a spintronic design that preserves standard STT-MRAM bit-cell density while broadening supported operations and system integration.

  • Near-Memory Computing: Near-memory computing places logic or processing units closer to memory while keeping them distinct from memory arrays.Examples span integrated processors, page-level processing, storage, and industry efforts such as HMC and HBM.
  • In-Memory Computing: In-memory computing integrates logic operations into memory arrays, but must preserve standard-memory density and read/write efficiency.These constraints typically limit prior designs to a small number of simple operations.
  • Prior In-Memory Designs: Prior in-memory-computing proposals include application-specific vector-matrix multiplication and sum-of-absolute-difference operations, alongside specialized memory structures.The literature also includes ternary content-addressable memory, ROM-embedded RAM, AC-DIMM, and automata processors.
  • Spintronic In-Memory Computing: Earlier work explored bitwise in-memory logic in memristive memories and DRAM, whereas this paper targets spintronic memory with non-destructive operations and lower sensing margins.The lower on-to-off resistance ratio in spintronic memory creates lower sensing margins.
  • Spintronic In-Memory Computing: STT-CiM performs computation in a standard 1T-1R STT-MRAM array without changing bit-cells, unlike a 2T-1R approach that sacrifices density.The paper also distinguishes its broader operation set and architectural support from concurrent STT-MRAM AND/OR work.

III. BACKGROUND

STT-MRAM stores bits in magnetic tunnel junction resistance states and supports standard memory operations, while its write and read behavior is subject to several failure mechanisms.

  • An STT-MRAM bit-cell combines an access transistor with a magnetic tunnel junction whose parallel and antiparallel resistance states encode a bit.The parallel resistance RP is lower than the antiparallel resistance RAP.
  • Read sensing distinguishes the two current values produced by the MTJ configurations against a reference.
  • Writes switch the MTJ using a current above its critical switching current, with the write direction determining the stored logic value.Write duration and current magnitude affect the write failure rate.
  • STT-CiM is introduced as in-memory computing using standard STT-MRAM arrays.

A. STT-CiM overview

STT-CiM enables multiple wordlines and senses their combined bit-line currents to compute logic and arithmetic functions directly in a standard STT-MRAM array.

  • STT-CiM overview: STT-CiM simultaneously enables multiple wordlines, connecting multiple resistive bit-cells to each bitline for direct computation.This is feasible because STT-MRAM bit-cells are resistive and write currents are much higher than read currents.
  • STT-CiM overview: Two enabled bit-cells produce a source-line current ISL equal to the sum of their state-dependent currents, which enhanced sensing distinguishes.
  • Logic operations: Sensing references distinguish the possible combined currents to realize logic functions, including OR and NOR.For OR/NOR, the reference current is placed between the relevant antiparallel and parallel current cases.
  • Logic operations: The logic operations are symmetric, so the design need not distinguish the input orderings “10” and “01”.
  • Arithmetic operations: ADD computes each sum and carry using XOR, AND, and the previous carry, with XOR and AND performed simultaneously in one array access.

B. STT-CiM array

The STT-CiM array adds operation control and modified read peripherals while preserving standard write circuitry and supporting both regular memory and CiM accesses.

  • Array control: An additional CiMType input selects the compute-in-memory operation for each access, while the CiM decoder generates the required controls.
  • Array control: The array’s row-decoder configuration can activate arbitrary wordline locations, with roughly doubled row-decoder overhead representing 1.8% of evaluated area and power.
  • Peripheral circuits: Enhanced sensing circuitry supports the logic operations, and decoder-generated MUX controls select the desired operation.
  • Peripheral circuits: Modified reference generation uses two reference stacks containing cells programmed to RP, RAP, and RREF to produce additional reference currents.
  • Access modes: A normal access enables one wordline, whereas a CiM access enables two wordlines and sets CiMType to compute the selected function.

C. CiM operation under process-variations

CiM operations introduce distinct reliability challenges because their per-cell currents differ from standard reads; ECC mechanisms are extended to detect and correct resulting decision errors.

  • Failure mechanisms: CiM operations have different failure rates from standard STT-MRAM reads because the currents flowing through individual bit-cells differ.
  • Failure mechanisms: Process-variation simulations characterize current distributions for normal reads and CiM operations, including the effects of device parameter variation.
  • ECC design: SECDED and DECTED ECC schemes can address CiM decision failures because most ECC codeword properties are retained for CiM XOR.
  • ECC design: CiM XOR preserves augmented ECC codewords, allowing errors in CiM outputs to be detected and corrected using the encoded XOR result.
  • ECC results: CiM-operation errors occur with probability at most 0.1 in simulation, and 3EC4ED detects errors on all CiM operations.
  • Reliability extensions: Higher-TMR MTJs and improved sensing margins can complement ECC to reduce CiM read failures.

V. STT-CIM ARCHITECTURE

STT-CiM augments STT-MRAM peripheral circuits to compute logic, arithmetic, and vector operations by simultaneously enabling multiple wordlines. Vector operations execute elements concurrently and use reduction circuitry to limit output-transfer overhead, with ECC supporting reliable computation.

  • STT-CiM uses peripheral-circuit enhancements to perform arithmetic, logic, and vector operations while retaining the standard STT-MRAM core array.The unchanged core array maintains memory density and read/write efficiency.
  • Vector CiM operations: VCiM operations compute all vector elements concurrently, increasing internal memory bandwidth for data-parallel workloads.Figure 10 illustrates the vector-operation organization with 32xN-bit internal memory bandwidth.
  • Vector CiM operations: A Reduce Unit accumulates vector-operation outputs before the column multiplexer, reducing the need to transfer complete result vectors to the processor.The unit targets vector operations commonly followed by reductions, such as dot products.
  • Vector CiM operations: For sum_i=1^N A[i]+B[i], VCiM activates the operand rows together, performs addition in sensing circuitry, and accumulates the outputs.The conventional approach requires 2N memory accesses, while scalar CiM requires N accesses; the supplied passage truncates the VCiM count.
  • Error Detection and Correction: STT-CiM adds a 3EC4ED ECC-based Error Detection and Correction unit to detect and correct erroneous bits during CiM operations.The EDC unit checks errors using CiM XOR output.

B. Architectural Extensions for STT-CiM

STT-CiM is exposed to software through new processor instructions and requires bus support for its two-address memory operations. Program transformation then identifies instruction sequences that can use these operations.

  • ISA extension: The ISA adds CiMXOR, CiMNOT, CiMAND, and CiMADD instructions to invoke supported compute-in-memory operations.CiM instructions provide two memory addresses rather than the single address used by an ordinary access.
  • ISA extension: A CiM instruction supplies two operand addresses through registers and returns the computed result to a destination register.The format is shown as Opcode Reg1 Reg2 Reg3, with CiMXOR RADDR1 RADDR2 RDEST as an example.
  • Program transformation: Assembly-level program transformation converts suitable instruction sequences into CiM instructions so applications can exploit the extended ISA.The transformation operates at the application level.
  • Bus and interface support: Conventional buses must be extended because CiM operations send two addresses to memory through the address interface.The bus design must accommodate the key difference between CiM operations and ordinary load instructions.

C. Data Mapping

STT-CiM data placement must align operands within the memory organization so CiM instructions can access them together. The proposed techniques target element-wise, one-to-many, and replicated-column computation patterns.

  • Two operands can use a CiM operation only when they share a bank, occupy different rows, and reside in the same column set.These placement constraints determine whether two data elements can be sensed together.
  • Data placement is needed because target applications have well-defined computation patterns that can be mapped to CiM-compatible locations.The mapping objective is to maximize the use of CiM operations.
  • Type I: Array alignment maps corresponding elements A[i] and B[i] to compatible locations for element-to-element operations.Row-interleaved placement extends this approach across larger structures spanning multiple banks.
  • Type II: The spare-row technique copies an element of A into each bank to support one-to-many nested-loop computations.It aligns one A element with several B elements across the relevant banks.
  • Type II: Column replication copies a small-array element across a row so it aligns with every element of a larger array for CiM operations.The stated initial replication overhead is small relative to the targeted computation.

VI. EXPERIMENTAL METHODOLOGY

The evaluation combines device and circuit modeling with array- and system-level analysis of STT-CiM. System experiments integrate a 1MB scratchpad with an Intel Nios II processor, custom instructions, and an extended Avalon bus.

  • The device-to-architecture framework evaluates STT-CiM's performance and energy benefits at array and system levels.The framework and application benchmarks are introduced in Figure 13.
  • Device/Circuit modeling: SPICE simulations use MTJ models based on Landau-Lifshitz-Gilbert magnetization dynamics and Non-Equilibrium-Green’s Function electron transport.The simulations use 45nm bulk CMOS technology with modeled peripherals and extracted parasitics.
  • System level simulation: System-level evaluation models STT-CiM as a 1MB Nios II scratchpad with custom ISA instructions and an Avalon bus extended for CiM operations.Cycle-accurate RTL simulation provides execution time and memory-access traces.
  • Energy: A normal STT-CiM read incurs about 4.4% energy overhead versus an equal-capacity STT-MRAM read, primarily from added peripherals and stronger ECC.The 3EC4ED ECC scheme accounts for about 3% of the 4.4% overhead; CiMXOR energy is compared against two conventional reads.
  • Area and access time: VEC4 and VEC8 designs have 14.2% and 16.6% area overheads, respectively, while STT-CiM access-time overhead is approximately 0.8%.The area overhead comes from peripheral circuits, ECC storage, and ECC logic; the core array remains unchanged.

B. Application-level memory energy

STT-CiM reduces application-level memory energy relative to standard STT-MRAM, with larger benefits when accesses are convertible to CiM operations and vector operations are available.

  • 3.83x average energy improvement is achieved with STT-CiM+VEC8 across all benchmarks, relative to the standard STT-MRAM baseline.STT-CiM without vector operations achieves 1.26x, while vector lengths of 4 and 8 achieve 2.77x and 3.83x, respectively.
  • Memory energy is normalized to a standard STT-MRAM baseline and includes read, write, CiM, and near-memory correction components.
  • Figures 18 and 19 provide the application-level performance and memory-access breakdown used to relate energy outcomes to application behavior.
  • Applications dominated by CiM-convertible reads experience higher energy benefits from STT-CiM.The access categories are writes, non-convertible reads, and CiM-convertible reads.

C. System-level performance

STT-CiM improves system performance over standard STT-MRAM by reducing memory accesses and executed instructions, with larger gains at higher memory latency and when vector operations are used.

  • STT-CiM reduces total memory accesses and executed instructions, leading to application-level performance improvements.
  • Applications with more CiM-convertible reads and vectorization opportunities tend to show higher energy savings and performance improvements.
  • Higher memory latency increases STT-CiM’s performance benefit because fewer memory accesses have a larger impact.
  • The proposed design was evaluated using a device-to-architecture simulation framework and showed substantial energy and performance improvements.
Loading 1703.02118v4…