A Modern Primer on Processing in Memory

Motivation

• More than 62% of mobile-device system energy and more than 90% of edge-accelerator system energy can be spent on data movement or memory, underscoring the processor-centric bottleneck.Moving data from DRAM to the CPU incurs substantial latency and energy, often without reuse by processors or accelerators [10].Source: §1 · pp. 2–3

PIM concept

• PIM reduces data movement by placing computation inside or near memory, including memory chips, 3D-stacked logic layers, controllers, caches, and storage units.PIM can use memory-circuit operational properties or added processing logic such as accelerators, simple cores, and reconfigurable logic.Source: §1 · p. 3

Processing-using-memory (PUM)

• PUM exploits analog memory operations for efficient bulk and specialized computation, potentially delivering large performance and energy gains with minimal memory-chip changes [211].Examples include DRAM copy, initialization, Boolean and arithmetic operations, plus SRAM, NAND flash, PCM, STT-MRAM, and ReRAM operations such as convolutions and matrix multiplications [158–160, 161, 195, 197–209, 222–262].Source: §1 · p. 3

Processing-near-memory (PNM)

• PNM adds more general-purpose computation to controllers, memory chips or modules, or 3D-stacked logic layers, using high internal bandwidth to access nearby data.Technologies such as Hybrid Memory Cube and High-Bandwidth Memory provide logic layers and internal bandwidth enabled by through-silicon vias, while hybrid bonding and monolithic 3D integration may amplify these benefits [371].Source: §1 · p. 3

Adoption challenges

• Practical PIM adoption requires coordinated support for programming, runtime scheduling, coherence, unified virtual memory, PIM-aware data structures, and evaluation infrastructure.The article frames the shift from processor-centric to memory-centric computing as a central adoption challenge and addresses these issues across devices, architecture, systems, compilers, programming models, and applications.Source: §1 · pp. 3–4

Capacity, bandwidth, and latency

• Memory capacity requirements for large machine learning models increased by more than 10,000 times in five years, while DRAM scaling cannot cheaply maintain capacity, performance, energy efficiency, and reliability.Core counts double every two years, but DRAM capacity doubles only every three years, causing memory capacity per core to drop approximately 30% every two years.Source: §2 · p. 4

Implications for system design

• Because DRAM scaling affects capacity, bandwidth, latency, reliability, energy, and cost, intelligent memory controllers are proposed as a path toward better scaling and PIM.Such controllers can also serve as a starting substrate for processing-in-memory systems.Source: §2 · p. 4

Capacity, bandwidth, and latency

• DRAM capacity has improved approximately 128× while bandwidth improved only approximately 20× from 1999–2017, and latency reduced only 30%, creating a growing bottleneck for data-intensive workloads.Across more than 50 years, DRAM capacity improved by more than 1 million times, whereas latency improved by only approximately 8 times and remained almost constant over the past 20 years.Source: §2 · pp. 4–5

Reliability

• Smaller DRAM cells become leakier and more vulnerable to noise and disturbance, increasing memory errors and correlating higher chip density with higher server failure rates.These reliability problems arise because both the capacitor and access transistor become less reliable as cell size decreases.Source: §2 · p. 5, Figure 2 · p. 5

Robustness, security, and safety

• RowHammer and RowPress enable predictable bit flips that break physical memory isolation, creating security and safety vulnerabilities including privilege escalation, data disclosure, and corruption of programs or models.RowPress is prevalent across characterized DDR4 chips, worsens with scaling, and can reduce the activations needed for a bit flip by 1–2 orders of magnitude; existing mitigations remain insecurely and provably unresolved at low overheads.Source: §2 · pp. 6–7

Power and energy

• Main-memory energy consumption is increasing because capacity, bandwidth, parallelism, complexity, off-chip integration, and scaling-related reliability mechanisms all raise power demands.DRAM alone accounts for more than 40% of total system power in some recent CPU- and GPU-based systems, while earlier commercial servers attributed 40–50% to the off-chip memory hierarchy.Source: §2 · pp. 7–8

Reliability

• Intelligent memory controllers can provide programmable, low-overhead protection against RowHammer and RowPress by issuing probabilistic refreshes to physically adjacent rows based on chip vulnerability.PARA requires knowledge of physical row adjacency, programmable refresh probability, and controller-issued adjacent-row refreshes, while incurring much lower performance and energy overheads than refreshing all rows more frequently.Source: §3 · p. 8

Refresh management

• More than 75% of DRAM refreshes can be eliminated at low cost by profiling per-row retention times, adjusting refresh rates, or decommissioning weak rows.This approach reduces refresh-related energy and performance losses and may ease DRAM technology scaling challenges caused by worst-case retention assumptions [54, 377].Source: §3 · pp. 8–9

Latency and energy management

• DRAM read and write latencies can fall 33% and 55%, respectively, at 55 °C versus 85 °C across 115 real modules, while profiling cell heterogeneity can reduce access latency by as much as 57%.Controllers can exploit temperature, manufacturing, access-pattern, charge-level, subarray, and latency–reliability heterogeneity to improve latency, energy, refresh parallelism, random-number generation, and PUF evaluation.Source: §3 · pp. 9–10

Hybrid memory management

• Intelligent controllers can manage hybrid memories by mapping, migrating, and moving data across heterogeneous technologies according to data or application error tolerance, improving cost, energy efficiency, and inference performance.Examples include Heterogeneous Reliability Memory and EDEN, which map data elements or neural-network layers to memory regions with different reliability, latency, and voltage characteristics.Source: §3 · pp. 10–11

Controller design challenges

• Flash controllers already use intelligence for remapping, wear leveling, refresh, voltage optimization, error correction, recovery, and fairness, but comparable DRAM intelligence is harder because DRAM has lower latency and higher bandwidth.The complexity of these mechanisms reflects the scaling challenges of NAND flash and motivates more sophisticated DRAM controllers.Source: §3 · p. 11

Overall role

• Collectively, intelligent controllers can mitigate reliability, refresh, latency, and energy problems by managing diverse memory-chip characteristics, and can extend this role to special-purpose or general-purpose computation that reduces data movement.The paper identifies intelligent memory controllers as a key direction for alleviating main-memory scaling issues.Source: §3 · pp. 8, 11

• Memory operations require requests, off-chip commands, data transfer, caching, and register placement before the CPU can compute, consuming substantial time and energy [10].Source: §4 · p. 12
• More than 62% of energy for four mobile workloads and more than 90% for commercial edge neural networks is spent accessing or moving data.These measurements demonstrate the severe energy imbalance created by the processor–memory dichotomy [10].Source: §4 · p. 12
• Narrow off-chip buses, ineffective caches, irregular accesses, and power-hungry interconnects impose bandwidth, latency, energy, and area costs across modern systems.Random and streaming access patterns can limit cache, prefetching, bus, and memory effectiveness, especially in graph and sparse workloads [185].Source: §4 · pp. 12–13
• Data movement creates a vicious cycle in which latency demands complex tolerance mechanisms that add energy, latency, and system complexity, requiring a shift to data-centric computing.PIM breaks the processor–memory dichotomy by minimizing movement and performing computation where data resides.Source: §4 · pp. 12–14
• Memory accesses consume roughly 115× to 800× the energy of addition, while data movement accounts for 35%–more than 90% of total system energy across workloads.The cited values span scientific, mobile, consumer, and edge machine-learning applications.Source: §4 · p. 13

5. Processing-in-Memory (PIM): Technology Enablers and Two Approaches

• PIM addresses processor-centric systems’ major data-movement cost by enabling computation in the memory subsystem, improving prospects for high-performance, energy-efficient, and sustainable architectures.Source: §5 · p. 14

5.2. Two Approaches: Processing-Using-Memory (PUM) vs. Processing-Near-Memory (PNM)

• PUM exploits memory-cell and array operational principles, usually with small changes, to perform in-memory copy, initialization, bitwise logic, arithmetic, lookup, and user-defined complex operations across multiple memory technologies.Source: §5 · p. 14, §5.2 · p. 16
• PNM integrates accelerators, processing cores, functional units, or reconfigurable logic near or inside memory, including in 3D-stacked logic layers, memory controllers, and memory chips or modules, enabling a wider range of functions.Source: §5 · p. 14, §5.2 · p. 16
• PUM reduces data movement and exploits memory-array bandwidth and parallelism, whereas PNM supports more conventional and flexible logic; combining both approaches can maximize PIM benefits.Source: §5 · p. 14, §5.2 · p. 16

5.1.1. 3D-Stacked Memory Architectures

• 3D-stacked memories use thousands of vertical through-silicon vias to provide much greater internal bandwidth than narrow memory channels, while their logic layers offer space for PIM functionality subject to area, energy, and thermal constraints.Source: §5.1.1 · p. 15

5.1.2. Non-Volatile Memory (NVM) Architectures

• Byte-addressable NVMs—including PCM, MRAM, RRAM, and FeRAM—offer high density, nonvolatility, and DRAM-competitive or near-competitive latency and energy, while enabling memory-cell Boolean logic and analog matrix-vector multiplication.Source: §5.1.2 · pp. 15–16

6. Processing-Using-Memory (PUM)

• Processing-using-DRAM exploits DRAM’s analog operation and internal parallelism for bulk movement, bitwise operations, complex SIMD/MIMD computation, lookup tables, data reorganization, and security primitives.These approaches generally require small memory-chip modifications without dedicated processing elements or a dedicated logic layer.Source: §6 · p. 17, §6.1 · p. 17

6.1.1. RowClone

• 11.6× lower latency and 74.4× lower energy for a 4kB page copy make RowClone a low-cost mechanism for in-DRAM bulk movement, with only 0.01% DRAM area overhead.Across six copy- or initialization-intensive benchmarks, RowClone improves performance by 4%–66% and reduces DRAM energy by 15%–69%; COTS DRAM also supports related operations.Source: §6.1.1 · pp. 17–19, §6.1 · p. 17

6.1.2. Ambit

• Ambit uses triple-row activation and sense-amplifier inverters to provide functionally complete bulk AND, OR, and NOT operations, improving throughput up to 44× over Skylake and 9.7× over HMC logic-layer processing.Ambit reduces average energy by 35× versus DDR3 and lowers end-to-end query latency by 5.4×–6.6× for bitmap databases and 4×–12× for BitWeaving.Source: §6.1.2 · pp. 19–20

6.1.3. SIMDRAM

• SIMDRAM achieves 2.0× Ambit throughput and 2.6× Ambit energy efficiency, while delivering up to 88× CPU throughput and 257× CPU energy efficiency for complex operations.SIMDRAM combines vertical data layout, majority-based computation, offline logic optimization, generated DRAM-command microprograms, and ISA extensions for system integration.Source: §6.1.3 · pp. 21–22

6.1.4. MIMDRAM

• MIMDRAM addresses rigid PUD granularity and programmability barriers by allocating only needed DRAM mats, supporting independent operations, and enabling in-DRAM reductions through low-cost interconnects.Fine-grained activation reduces underutilization and energy waste while enabling multiple-instruction multiple-data execution within a subarray.Source: §6.1.4 · p. 22
• MIMDRAM delivers 13.2×/0.22×/173× performance, 0.0017×/0.00007×/0.004× energy consumption, and 582.4×/13612×/272× performance per Watt versus CPU/GPU/SIMDRAM baselines.It achieves 15.6× SIMD utilization over SIMDRAM while adding 1.11% DRAM-chip and 0.6% CPU-die area cost; compiler passes vectorize, configure, and schedule PUD operations.Source: Figure 20 · p. 23, §6.1.4 · p. 23

6.1.5. pLUTo

• pLUTo extends PUM to complex operations by bulk-querying lookup tables in a bit-parallel DRAM architecture that can query multiple LUTs stored in one subarray.Its ISA, library, compiler, and controller support programmers in expressing operations and planning in-memory data placement and alignment, overcoming prior PUD limitations for operations such as multiplication and exponentiation.Source: §6.1.5 · pp. 23–24
• pLUTo outperforms optimized CPU and GPU baselines by average factors of 713× and 1.2×, respectively, reduces energy by 1855× and 39.5×, and outperforms state-of-the-art PiM architectures by 18.3×.Its three designs trade hardware overhead against flexibility, performance, and energy efficiency, with additional DRAM area overheads between 10.2% and 23.1%.Source: §6.1.5 · pp. 24–25

6.1.6. Gather-Scatter DRAM

• GS-DRAM reorganizes data across DRAM chips and uses controller-supplied pattern IDs with column translation logic to gather strided values in one memory command.It achieves near-ideal bandwidth and cache utilization, outperforming column-store transactional workloads by 3×, row-store analytics workloads by 2×, and tiled matrix multiplication by 10%.Source: §6.1.6 · p. 25

6.1.7. In-DRAM Security Primitives

• DRAM timing violations enable runtime-accessible PUFs and true random-number generation: DRAM Latency PUF produces unique identifiers in 88.2ms across 223 LPDDR4 chips, while D-RaNGe aggregates randomly failing cells for high-throughput, low-latency randomness.The PUF requires no DRAM-chip modification, only a memory controller that changes timing parameters and identifies reliable cells; both mechanisms exploit manufacturing variation or induced random failures.Source: §6.1.7 · pp. 25–26

D-RaNGe Key Idea

• D-RaNGe extracts true random values from DRAM latency-failure probabilities and generates robust, standards-compliant random data without modifying existing DRAM chips.It was evaluated on 282 LPDDR4 devices, passed all NIST statistical tests, and achieved over two orders of magnitude higher throughput than the state-of-the-art DRAM-based TRNG.Source: D-RaNGe Key Idea · p. 26
• Commodity DRAM can also generate high-throughput, low-latency security primitives directly in memory, including true random numbers, unique keys, PUF responses, and cold-boot protection.CODIC provides 1.8× higher PUF throughput than the best state-of-the-art DRAM PUF, while its cold-boot mechanism achieves 2.0× lower latency and 1.7× lower energy than LISA and RowClone during power-on.Source: D-RaNGe Key Idea · pp. 26–28

6.2.1. Using Emerging NVMs for Computation

• Processing-using-NVMs exploit analog and resistive memory behavior to perform stateful or non-stateful bitwise operations, in-situ matrix-vector multiplication, and parallel string matching.Stateful operations consume write endurance, whereas non-stateful operations avoid writes but require additional sensing circuitry when cascading operations; crossbar MVM executes in nearly one NVM read cycle when the matrix fits.Source: §6.2.1 · pp. 28–30
• MATSA accelerates subsequence dynamic time warping by decomposing computation into MRAM bitwise operations, using linear-footprint mapping, and exploiting anti-diagonal pipelining.It improves performance by 7.35×/6.15×/6.31× and energy efficiency by 11.29×/4.21×/2.65× over server-class CPU, GPU, and processing-near-memory platforms, respectively.Source: §6.2.1 · p. 29
• NVM-based PUMs offer performance and energy benefits by minimizing data movement, but device non-idealities, endurance, sensing and conversion costs, robustness, scalability, and workload applicability remain open challenges.These issues can affect application accuracy, system cost, and scalability, motivating more robust designs and broader programming models.Source: §6.2.1 · pp. 30–31
• GenPIP integrates NVM-based in-situ MVM and string matching to combine genome basecalling and read mapping while minimizing intermediate-result movement and unused computation.Swordfish addresses NVM non-idealities through modeling, accuracy-enhancement techniques, and system evaluation; realistically, PUMA accelerates Bonito by 25.7× with a 6.01% accuracy loss versus an NVIDIA V100 GPU.Source: §6.2.1 · pp. 30–31

6.2.2. Flash-Cosmos: In-Flash Bulk Bitwise Operations

• Flash-Cosmos performs reliable bulk bitwise operations inside NAND flash using Multi-Wordline Sensing across many operands and Enhanced SLC-mode Programming for computation reliability.It reliably performs 48-input AND and 4-input NOR with less than a 10% sensing-latency increase, and tested NAND chips produced no errors.Source: §6.2.2 · pp. 31–32
• Across three real-world applications, Flash-Cosmos improves average performance and energy efficiency by 3.5×/32× and 3.3×/95× over ParaBit [233]/outside-storage processing, respectively.Evaluation on 160 real 3D NAND flash chips demonstrated all described operations without errors, while future work must expand computation, programmability, and system integration.Source: §6.2.2 · pp. 31–32
• Flash-Cosmos’s ESP technique improves processing-using-flash reliability by exploiting existing flash-controller mechanisms, motivating further robust controller-aware designs.Relevant mechanisms include data randomization, wear leveling, adaptive refresh, block remapping, garbage collection, read-reference tuning, and error correction [147, 350, 651, 657– 665, 670, 671, 673, 962, 967–985].Source: §6.2.2 · p. 32

6.3. Processing-Using-SRAM

• SRAM performs AND and NOR by simultaneously activating operand rows and sensing whether parasitic bitline voltage remains above the reference.The sense amplifier detects ‘1’ for two stored ones on the bitline, while the complementary bitline detects ‘1’ only for two stored zeros.Source: §6.3 · p. 32
• Compute Caches repurpose processor LLCs as in-memory SIMD engines, extending SRAM bitwise processing with NOR, compare, search, copy, and carryless multiplication.Each SRAM bitline becomes a SIMD lane, while Neural Cache and Duality Cache add bit-serial arithmetic inside LLC slices.Source: §6.3 · pp. 32–33
• Duality Cache supports general-purpose data-parallel SIMT execution, floating-point and transcendental operations, and CUDA/OpenACC compilation through a specialized compiler.Its control blocks execute 1,024 threads simultaneously, with each thread block containing 256 threads and each thread mapped to a cache bitline.Source: §6.3 · p. 33
• Processing-using-SRAM is easy to manufacture with commercial CMOS tools but is constrained by SRAM’s lower density and higher cost per bit.These constraints limit workloads to megabyte-scale datasets or require additional data movement, and they reduce computation density relative to DRAM and NVM.Source: §6.3 · p. 33
• Up to 52.9× faster bit-serial addition is achieved by processing-using-DRAM over processing-using-SRAM when DRAM-to-cache data movement is included.An 8 GB processing-using-DRAM engine provides up to 67,108,864 SIMD lanes, compared with 1,146,880 lanes in a 35 MB processing-using-cache engine.Source: §6.3 · p. 33
• Future memory-centric systems could collaboratively map and schedule bulk-bitwise and bit-serial SIMD operations across SRAM, DRAM, NVM, and NAND-flash PUM architectures.Such integration aims to minimize data movement across caches, main memory, and storage while improving performance, energy efficiency, and scalability for workloads with varied locality and parallelism.Source: §6.3 · p. 33

7.1. PNM in DRAM-Based Main Memory

• PNM architectures commonly use custom logic in the logic layer of high-bandwidth 3D-stacked DRAM, with application-, function-, or instruction-level offloading trading performance gains against complexity and cost.Near-memory engines obtain high-bandwidth, low-latency access through vertical connections, while coarse-grained application-level customization can maximize performance and energy benefits for memory-bound workloads.Source: §7 · p. 33, §7.1 · p. 34, §7.1.1 · p. 34

7.1.1. Tesseract: Coarse-Grained Application-Level PNM Acceleration (of Graph Processing)

• 13.8× average performance improvement and 87% average energy reduction over conventional systems demonstrate Tesseract’s effectiveness for large-scale graph processing.Tesseract places simple in-order cores in 3D-stacked memory, moves functions to data, and provides message-passing support for remote function calls.Source: §7.1.1 · pp. 34–35

7.1.2. Polynesia: Coarse-Grained Application-Level PNM Acceleration (of Hybrid Transactional/Analytical Processing)

• Polynesia outperforms three state-of-the-art HTAP systems by 1.70× transactional throughput and 3.74× analytical throughput on average, while using 48% less energy.It partitions resources into isolated transactional and analytical processing islands with workload-specific data replicas, execution engines, and hardware.Source: §7.1.2 · pp. 35–36

7.1.3. NATSA: Coarse-Grained Application-Level PNM Acceleration (of Time Series Analysis)

• NATSA improves matrix-profile performance by up to 14.2× and reduces energy by up to 27.2× over a state-of-the-art multicore implementation.It performs complete time-series analysis near 3D-stacked HBM using specialized floating-point processing units, addressing low arithmetic intensity and costly data movement.Source: §7.1.3 · p. 36

7.1.4. Function-Level PNM Acceleration (of Mobile Consumer Workloads and Edge Neural Networks)

• Function-level PNM reduces consumer-workload execution time by 54.2% and energy by 55.4% on average, while requiring only limited-area cores or fixed-function accelerators.The approach offloads selected functions such as browser tiling, ML packing, and video codecs; it is easier to adopt than full application-level customization but generally provides smaller benefits.Source: §7.1.4 · pp. 37, 39
• Mensa-G reduces inference energy by 66.0%, improves energy efficiency by 3.0×, and increases throughput by 3.1× versus a monolithic Edge TPU across 24 Google edge NN models.Its three heterogeneous accelerators are tailored to clustered layer characteristics, with memory-centric accelerators placed in 3D-stacked DRAM logic layers.Source: §7.1.4 · p. 38

7.1.5. Programmer-Transparent Function-Level PNM Acceleration of GPU Applications

• TOM improves average GPGPU performance by 30% and reduces average energy consumption by 11% without requiring programmer changes.Compiler analysis identifies profitable code blocks, runtime decisions account for logic-layer contention, and software/hardware mapping places accessed pages in the execution stack.Source: §7.1.5 · p. 39

7.1.6. Function-Level PNM Acceleration of Genome Analysis Workloads

• Algorithm–architecture co-design with function-level PNM is especially beneficial for data-intensive genome analysis workloads.The cited works identify genome analysis as a domain where reducing memory movement through near-memory acceleration can provide important benefits.Source: §7.1.6 · p. 40
• 3.65× faster full read mapping across 10 real genome read sets demonstrates GRIM-Filter’s benefit over a conventional CPU-only system.GRIM-Filter places seed filtering in the logic layer of 3D-stacked memory and integrates with read mappers through a communication protocol.Source: §7.1.6 · p. 40
• GenASM combines modified bitvector approximate string matching and traceback algorithms with 3D-stacked-memory hardware, outperforming prior accelerators and software baselines by one or more orders of magnitude.Its systolic-array compute units and on-chip SRAMs match computation with memory capacity and bandwidth, and performance scales linearly with parallel compute units.Source: §7.1.6 · p. 40

7.1.7. PEI: Fine-Grained Instruction-Level PNM Acceleration

• 47% average performance improvement and 25% average energy reduction across 10 data-intensive workloads demonstrate the benefits of PEIs over a baseline CPU.PEIs preserve conventional programming and virtual-memory models while dynamically choosing host or PIM execution using locality-aware runtime mechanisms.Source: §7.1.7 · pp. 41–42

7.2. PNM in Caches & Storage

• 1.65× average stencil-kernel speedup and 35% average energy reduction show Casper’s near-cache processing benefits over a commercial high-performance multicore processor.Casper places stencil processing units in LLC slices and exploits regular accesses and parallelism across SRAM arrays.Source: §7.2 · pp. 42–43
• 2.07–6.05× and 1.45–33.63× read-mapping speedups show GenStore outperforming software baselines across high- and low-similarity genomic read sets.GenStore filters reads inside NAND flash using SSD- and channel-level accelerators, reducing unnecessary transfers to the host.Source: §7.2 · p. 43
• MegIS enables cooperative storage-based processing through host–storage hardware/software co-design, partitions metagenomic analysis across host and SSD, overlaps transfer with computation, and leverages SSD bandwidth efficiently, achieving up to 7.2× cost-efficiency versus a processor-centric system.Source: §7.2 · p. 44
• 2.7×–37.2× and 6.9×–100.2× speedups show MegIS outperforming performance- and accuracy-optimized software tools, while matching the accuracy-optimized tool.MegIS also achieves 1.5×–5.1× speedup over a PIM-accelerated tool with significantly higher accuracy, and reduces average energy by 5.4×, 15.2×, and 1.9× versus the respective baselines.Source: §7.2 · p. 44

8. Enabling the Adoption of PIM

• PIM adoption requires programming models and compiler or library support that ease use across programmers, architectures, and offloading granularities.PIM-enabled instructions suit fine-grained computation but can incur CPU–PIM exchange overhead for large tasks, motivating seamless support for instructions, library calls, and larger code patterns.Source: §8.1 · p. 45, §8 · p. 45

8.1.1. Programming Support for PNM Systems

• PNM support combines transparent compiler offloading, runtime mapping, and high-level abstractions such as SimplePIM and DaPPA to reduce programmer responsibility.ALP uses compiler and runtime information to map segments to CPU or PIM cores while accounting for segment properties and inter-segment data movement; DaPPA automatically manages data distribution, collection, memory, and DPU parallelization.Source: §8.1.1 · p. 46
• High-level PNM frameworks such as SimplePIM and DaPPA are presented as central directions for reducing programming burden and automating PIM code generation and optimization.SimplePIM uses management, communication, and processing interfaces, while DaPPA uses data-parallel patterns, pipelines, and dynamic template-based compilation.Source: §8.1.1 · pp. 46–47
• SimplePIM requires 4.4× fewer lines of code than existing hand-optimized implementations and matches or exceeds their performance across six UPMEM applications.It performs 1.10×–1.37× faster in weak scaling and 1.15×–1.43× faster in strong scaling for vector addition, logistic regression, and K-means.Source: §8.1.1 · p. 46, Figure 43 · p. 47
• DaPPA improves end-to-end performance by 2.1× on average across six hand-optimized PrIM benchmarks while reducing programming complexity by 94.4% on average.Its optimizations include data-transfer parallelization and improved workload partitioning between CPU and PIM cores.Source: §8.1.1 · p. 47

8.1.2. Programming Support for PUM Systems

• MIMDRAM compiles C/C++ applications into binaries mixing CPU and PUD instructions through LLVM-based code identification, scheduling and data mapping, and data allocation and generation.It vectorizes suitable loops, maps operands across DRAM mats to improve SIMD utilization, and supports the energy-efficiency and throughput results reported for Ambit-like bulk bitwise PUM execution.Source: §8.1.2 · pp. 47–48

8.1.3. Summary & Future Work

• Effective, easy-to-adopt programming interfaces and compiler or library support remain central open research and development issues despite substantial progress.Future work must map high-level abstractions across PNM and PUM hardware while minimizing data movement, synchronization, and latency and maximizing parallelism.Source: §8.1.3 · p. 48
• Future compilers must automate loop parallelization and data partitioning while efficiently coordinating heterogeneous CPU, PNM, and PUM processing units with different bandwidths and compute capabilities.The section specifically highlights irregular workloads, hybrid PIM systems, varying memory access costs, and the tradeoff between offloading and synchronization overhead.Source: §8.1.1 · p. 47, §8.1.3 · p. 48

Runtime challenges

• PIM runtime systems must address code selection, offload timing, data mapping, and sharing or partitioning PIM mechanisms across threads and cores.These decisions must maximize performance and energy efficiency across varied conditions, yet substantial research remains.Source: §8.2 · p. 48

PIM offloading candidates

• 54% of TensorFlow Mobile’s data-movement energy comes from packing/unpacking and quantization, while the examined neural networks spend more than 57% of execution energy on data movement.A systematic toolflow uses whole-application CPU execution, hardware counters, and energy models to identify such PIM offloading candidates; candidate functions must exceed thresholds for workload energy, data-movement energy, and memory intensity.Source: §8.2 · p. 49, Figure 46 · p. 49

Dynamic scheduling

• Runtime systems combine compile-time analysis, online profiling, locality awareness, and monitoring to dynamically decide whether marked code segments should execute on PIM.Examples include L1-to-LLC miss-ratio profiling, an Offload Management Unit, and Monitor Units collecting runtime cache information.Source: §8.2 · pp. 49–50

Data and code mapping

• Effective PIM mapping depends on the application, system configuration, and PIM type; PiDRAM and MIMDRAM APIs align operands within the same DRAM subarray or mat.These mechanisms coordinate allocation across virtual and physical pages, while other approaches move partial results between rows, subarrays, and banks.Source: §8.2 · p. 50

Open research questions

• Programmer-transparent mapping, scheduling, resource sharing, CPU-PIM memory access, and efficient data movement remain open runtime-system questions for broad PIM adoption.Existing cooperative mechanisms support regular GPU applications across 3D-stacked memory chips, but wider application and hardware coverage is still needed.Source: §8.2 · p. 50

Challenge

• Traditional fine-grained coherence can send many messages over the narrow processor-memory bus, potentially negating PIM’s bandwidth and latency benefits.Prior approaches either restrict programming through incoherence and message passing or limit PIM performance and energy gains.Source: §8.3 · pp. 50–51

CoNDA

• CoNDA maintains cache coherence between PIM logic and CPU cores without sending coherence requests for every memory access.Its lazy approach brings performance and energy consumption within 10.4% and 4.4%, respectively, of zero-cost, zero-latency coherence.Source: §8.3 · p. 51
• CoNDA speculatively acquires permissions, batches requests into compressed coherence signatures, and rolls back and re-executes PIM code when CPU-side resolution detects conflicts.The CPU checks signatures for coherence violations and flushes dirty cache lines that caused conflicts.Source: §8.3 · p. 51, Figure 47 · p. 51

Results

• CoNDA achieves 38.3% higher performance than the highest-performance prior coherence mechanism, and 8.4×/7.7× over CPU-only/PIM-only execution.On large datasets, CoNDA comes within 10.2% of an ideal no-cost coherence mechanism across four workloads.Source: §8.3 · p. 51
• For modest datasets, CoNDA reduces energy by 18.0% versus the best prior low-memory-energy coherence mechanism and by 43.7% versus CPU-only execution.It comes within 4.4% of the energy-reduction potential of a no-cost coherence mechanism.Source: §8.3 · p. 51

Future directions

• Further work should develop coherence that handles diverse workloads, offloading granularities, and PIM approaches, including interfaces such as CXL, OpenCAPI, and OMI.Simpler and more efficient mechanisms remain important for adoption.Source: §8.3 · p. 51

8.4. Virtual Memory Support

• A unified Host–PIM virtual address space enables “a point is a pointer” semantics, but requires distributed address translation while preserving access protection for every memory access.Source: §8.4 · pp. 51–52
• CPU-side translation can nullify PIM gains through off-chip latency and extra memory traffic, while duplicated memory-side TLBs require coherence, storage, and cross-architecture compatibility.Source: §8.4 · p. 52
• IMPICA improves linked-list, hash-table, and B-tree pointer chasing by 92%, 29%, and 18%, respectively, while improving DBx1000 throughput by 16% and response time by 13%.It avoids CPU TLB lookups and CPU page walkers by using a region-based page table and separate address-calculation and memory-access engines in 3D-stacked memory.Source: §8.4 · p. 52
• IMPICA reduces energy consumption by 6% for DBx1000.Source: §8.4 · p. 52
• VBI delegates physical memory management to memory controllers and specialized hardware, offering a framework for PIM units to manage allocation and address translation beyond pointer chasing.Figure 48 contrasts VBI with conventional virtual memory.Source: §8.4 · p. 52, Figure 48 · p. 52

8.5. Data Structures for PIM

• Concurrent data structures tailored to PIM are necessary for high-performance, scalable many-core systems that otherwise rely on optimized concurrent structures.These structures are widely used in heavily optimized server systems, motivating PIM-specific designs.Source: §8.5 · p. 52
• PIM-managed concurrent data structures can outperform state-of-the-art multicore structures by exploiting vault partitioning and pipelined operations.Pointer-chasing structures can expose parallelism through partitioning across 3D-stacked memory vaults, while a PIM core pipelines FIFO enqueue and dequeue requests.Source: §8.5 · p. 53
• SISA [185] addresses graph mining’s irregular parallelism and memory access complexity by exposing vertex-set operations through cross-layer programming, ISA, algorithm, and hardware co-design.Set intersection and union provide rich, simple parallelism across graph pattern matching and graph learning workloads [704, 1193, 1194, 1195].Source: §8.5 · p. 53
• More than 10× speedup over Bron-Kerbosch [1196] shows that SISA co-design with PUM and PNM can accelerate graph mining’s set operations.SISA [185] combines a set-centric ISA with in-DRAM bulk bitwise computing for high-degree vertices and near-memory logic for low-degree vertices, supporting more than 10 algorithm formulations.Source: §8.5 · p. 53
• Future PIM data-structure research includes hash tables, search trees, priority queues, irregular and sparse structures, and designs for bulk bitwise execution.Such designs may also enable new algorithm formulations, as demonstrated by SISA.Source: §8.5 · p. 53

Benchmark requirements

• PIM assessment also depends on real-world, diverse application suites and agreed candidate workloads to support reproducibility and guide the design of PIM logic and kernels.Candidate domains include databases, graph processing, neural networks, machine learning, analytics, search, bioinformatics, security, and in-memory data stores.Source: §8.6 · pp. 53–54

Kernel identification methodology

• DAMOV’s three-step methodology identifies memory-bound functions, clusters spatial and temporal locality, and classifies hardware-dependent memory bottlenecks through scalability analysis.It uses hardware profiling, architecture-independent locality analysis in DAMOV-SIM, and simulations of CPU and PNM configurations while sweeping 1–256 cores and measuring AI, MPKI, and LFMR.Source: §8.6 · pp. 54–55

Benchmark development

• DAMOV analyzes 345 applications and 77K functions, identifying 144 memory-bound functions and six major data-movement bottleneck classes.The classes include DRAM bandwidth, DRAM latency, cache capacity, and cache contention.Source: §8.6 · p. 55
• The open-source DAMOV Benchmark Suite contains 144 representative functions and supports studies of PIM load balance, inter-vault communication, cores, accelerators, core models, and simple PIM instructions.These benchmarks enable comparative exploration of several architectural design choices.Source: §8.6 · p. 55

Simulation infrastructures

• Simulation infrastructures must model PIM hardware, memory bandwidth, and communication overheads across the compute stack, with Ramulator-PIM and DAMOV-SIM supporting diverse cores, accelerators, memory technologies, and configurations.Ramulator-PIM combines Ramulator and zsim, while DAMOV-SIM adds configurations and a more user-friendly interface.Source: §8.6 · p. 55

8.7.1. PUM Prototypes

• COTS DRAM prototypes perform Boolean operations, up to 16-input logic, and concurrent row copying with over 94% success rates and over 99.98% copy reliability.DRAM Bender experimentally demonstrates these capabilities across hundreds of COTS DRAM chips, motivating robust future PUM mechanisms and open research infrastructures.Source: §8.7.1 · p. 56

8.7.2. UPMEM PIM Architecture

• UPMEM integrates a DPU beside each DRAM bank, and its 2,560-DPU systems outperform CPUs on most PrIM benchmarks and GPUs on 10 of 16.UPMEM is compute-bound and best suited to memory-bound workloads using simple operations with little inter-DPU communication; its characterization produced the 16-workload PrIM suite and programming and hardware recommendations.Source: §8.7.2 · pp. 56–57
• UPMEM applications achieve 1.8×–28× over CPUs and 1.2×–2.7× over GPUs for sequence alignment, while optimized machine-learning training reaches 2.8×/27× over CPUs and 1.3×/3.2× over GPUs.Other evaluated workloads include SpMV at 51.7% of UPMEM peak performance versus under 1% on CPUs and GPUs, homomorphic operations with up to 300× CPU speedup, and reinforcement learning exceeding CPU and GPU implementations by at least 1.62× and 4.84×.Source: §8.7.2 · pp. 58–60

8.7.3. Samsung Function-In-Memory DRAM (FIMDRAM)

• Samsung’s FIMDRAM adds a PCU to LPDDRx memories for on-device generative-AI inference, while CXL-PNM uses PE arrays and adder trees for matrix and matrix-vector multiplication.Using eight CXL-PNM devices with LPDDR5X-based CXL memory, the architecture achieves 23% lower latency, 31% higher throughput, 2.8× higher energy efficiency, and 30% lower hardware cost than eight-A100-GPU architecture during LLM inference.Source: §8.7.3 · p. 61

8.7.4. Samsung Acceleration DIMM (AxDIMM)

• Samsung’s AxDIMM places FPGA-based near-memory accelerators in a DIMM buffer chip for recommendation inference and database operations.For DLRM, one accelerator per rank performs element-wise summation of embedding-table entries representing sparse recommendation features.Source: §8.7.4 · p. 61

8.7.5. SK Hynix Accelerator-in-Memory (AiM) & CXL-Memory Solution (CMS)

• SK Hynix’s AiM combines bank-local floating-point multiply-accumulate and activation units, while its FPGA-prototyped CMS uses load balancing and PIM engines for kNN kernels.AiM includes a global SRAM buffer supporting input storage and limited RowClone-like inter-bank copying; CMS integrates a CXL controller, DDR memory, and dot-product, kNN, and top-K units.Source: §8.7.5 · p. 61

8.7.6. Alibaba Logic-to-DRAM Hybrid Bonding with PNM (HB-PNM)

• Alibaba’s HB-PNM vertically hybrid-bonds a DRAM die containing 36 1 Gb DRAM cores with a logic die containing match and neural engines for recommendation-system matching and ranking.The DRAM cores form a 6 × 6 array with 8 banks each; Figure 61 depicts the architecture and package cross-section [1282].Source: §8.7.6 · p. 62, Figure 61 · p. 62

8.7.7. Summary

• Real PUM and PNM prototypes and solutions are emerging, providing baselines that can enable and accelerate future software and adoption research for PIM.The section characterizes industry adoption and ongoing prototype development as promising for future PUM and PNM systems.Source: §8.7.7 · p. 62

Security considerations

• Because PIM allows applications to offload code to computation units, unrestricted concurrent access can create unforeseen information leakage and confidentiality or integrity violations.The paper frames PIM security as requiring both protection of offloaded execution and specialized mechanisms that enhance system security.Source: §8.8 · p. 62

Security guarantees

• PIM must provide process isolation and secure access control comparable to processor-centric systems, while virtualization and cross-layer memory-management mechanisms may enable stronger protections.Memory partitioning can isolate processes, and mechanisms such as VBI and Expressive Memory support extensive hardware management capabilities.Source: §8.8 · p. 62

Reliability and integrity

• PIM reliability can be threatened by amplified RowHammer effects, NVM cell wearout, technology-scaling noise, and computation-induced stress, making ECC and other robustness techniques likely necessary.Intelligent memory controllers may help prevent RowHammer, but endurance limitations can reduce the reliability and effectiveness of NVM-based processing-using-memory approaches.Source: §8.8 · pp. 62–63

Security attacks

• 12.87 Mb/s and 14.16 Mb/s covert-channel throughput is achieved by IMPACT, up to 4.91× and 5.41× faster than prior main-memory covert channels, while its side channel leaks victim secrets with low error rate.IMPACT exploits shared DRAM row buffers, PIM parallelism, and direct main-memory access to establish high-throughput covert and side channels.Source: §8.8 · p. 63

Security opportunities

• Keeping data and computation within memory can reduce exposure to attacks by eliminating processor–memory movement over the main memory bus and enable more secure and private computing.PIM also enables specialized in-DRAM primitives, including DRAM-latency PUFs and true-random-number generators and QUAC true-random-number generators.Source: §8.8 · p. 63

• Courses, conference sessions, and tutorials now cover PIM concepts, research, practical aspects, applications, tools, methodologies, and challenges across academia and industry.Broader memory-centric computing tutorials are also scheduled at PPoPP 2025 and ASPLOS 2025.Source: §9 · p. 63
• Interviews on memory-centric computing provide additional resources documenting progress in the field.Source: §9 · p. 64
• Open-source simulators, prototyping platforms, benchmark suites, and application implementations can disseminate PIM knowledge and enable further developments and innovations.The paper hopes these resources help shape memory-centric computing to improve computing systems broadly.Source: §9 · p. 64

• Data movement consumes substantial system energy because processor-centric designs perform computation far from data, forcing extensive movement across the memory hierarchy.Source: §10 · p. 64
• PUM exploits memory circuitry to perform operations such as copying, initialization, bulk-bitwise computation, reorganization, and random-number generation with small chip changes.These approaches apply beyond DRAM and provide large performance and energy benefits.Source: §10 · p. 64
• PNM embeds computation beside memory banks or subarrays, in modules, controllers, or 3D-stacked-memory logic layers, improving performance and energy efficiency across applications and platforms.PNM approaches also extend to different memory types.Source: §10 · p. 64
• Widespread PIM adoption requires easy programming models, runtime support, secure hardware/software, and benchmarks, simulators, prototypes, and real hardware for accurate evaluation.Useful infrastructure includes memory-intensive benchmarks and methods for identifying PIM-suitable application regions and estimating benefits and overheads.Source: §10 · p. 64
• A data-centric combined computation-and-storage paradigm could greatly reduce data movement, deliver orders-of-magnitude energy and performance improvements, and enable new applications, but mindset and infrastructure remain the largest adoption challenge.Overcoming this shift could enable an energy-efficient, high-performance, and sustainable approach, while substantial cross-stack challenges remain.Source: §10 · pp. 64–65