PaperlayerView exact revision on arXiv

Source-linked AI summary

GUMPIM: Unitary and Malleable Memory for Processing-in-Memory with Guaranteed PIM Pages

Yilong Zhao, Fangxin Liu, Yiwei Hu, Zongwu Wang, Mingyu Gao, Onur Mutlu, He Xian, Hanlin Dong, Naifeng Jing, Xiaoyao Liang, Haibing Guan, Qiang Xu, Chixiao Chen, Shuxue Quan, Tao Yang, Li Jiang

arXiv:2610.23406v1cs.AR

TL;DR

CPU virtual memory and memory interleaving constrain the contiguous data PIM units can access, while isolated memory spaces or disabled interleaving impose transfer costs or reduce CPU bandwidth. GUMPIM combines interleaved CPU pages and non-interleaved PIM pages with dual-track management, DRAM-side mapping hardware, and PIM-assisted page migration. It reports 2.3× overall PIM speedup alongside 0.1% CPU-task performance loss.

  • Problem

    Virtual-memory mappings and memory interleaving constrain PIM units' access to large contiguous data, while existing ways to address this can add data transfers or degrade CPU bandwidth.

    Source: §1 · pp. 2–4

  • Method

    GUMPIM combines interleaved CPU pages and non-interleaved PIM pages in one memory space, using dual-track page management, DRAM-side address mapping, and PIM-assisted migration.

    Source: §1 · p. 4

  • Results

    2.3× overall PIM-task speedup accompanies 4.93× lower CPU-part time and 2.7× lower memory-allocation time, with 0.1% CPU-task performance loss.

    Source: §9 · p. 23

  • Takeaways & Limitations

    The design confines hardware modifications to DRAM and is reported as adaptable to DIMM DRAM, LPDDRx, and HBMx.

    Source: §1 · p. 4, §9 · p. 23

Abstract

from arXiv · show

DRAM-based Processing-in-Memory (PIM) addresses the "memory wall" by executing computations directly inside main memory. However, memory interleaving and virtual memory limit contiguous data size visible to PIM units, constraining PIM task granularity. To mitigate this, existing PIM systems drastically isolate PIM memory or disable memory interleaving. These design choices, however, decrease the CPU memory bandwidth and introduce extra data transfer, leading to an additional "system memory wall" that must be resolved to realize PIM's full potential. In this work, we propose GUMPIM, a PIM system that allows interleaved CPU pages and non-interleaved PIM pages to coexist in a Unitary and Malleable memory space with Guaranteed PIM page allocation. GUMPIM enables zero-copy during PIM task offloading and maintains CPU memory bandwidth while ensuring low PIM offloading overhead. First, we propose a dual-track memory management mechanism consisting of independent page allocation and address translation for CPU and PIM pages. Second, we design GUMPIM interface hardware on PIM-enabled DRAMs to provide a dynamic address mapping for the different data layouts of CPU and PIM pages. Third, we propose a PIM-assisted page migration mechanism that transparently migrates pages while preserving CPU access bandwidth, thereby enabling guaranteed and accelerated PIM page allocation. GUMPIM requires no changes to commodity DRAM standards; all hardware modifications are limited to the DRAM side, ensuring full compatibility with existing CPUs and enabling immediate deployment on current HBMx- and LPDDRx-based PIM platforms. Our results show only <0.1% performance degradation for CPU workloads on GUMPIM. For PIM workloads, GUMPIM reduces memory allocation and CPU-part computation times by 2.7x and 4.93x, respectively, yielding an end-to-end 2.3x speedup over a state-of-the-art baseline system.

1 Introduction

CPU virtual memory and memory interleaving limit the contiguous data PIM units can access, while existing workarounds add transfer overhead or reduce CPU bandwidth. GUMPIM combines CPU and PIM pages in one memory space and reports faster PIM execution with little CPU performance impact.

1 Introduction

  • Virtual-memory mapping and memory interleaving limit PIM units' access to large contiguous data, constraining PIM task granularity.For a DIMM DRAM bank, PIM units may not access even consecutive bytes within a data structure.Source: §1 · pp. 2–3
  • Isolated PIM memory requires explicit CPU-to-PIM data transfers, software re-layout can add over 90% overhead for fine-grained tasks, and disabling interleaving degrades CPU bandwidth.The re-layout result is reported for BFS and NW on a real PIM system.Source: §1 · p. 3
  • GUMPIM co-locates interleaved CPU pages and non-interleaved PIM pages, enabling zero-copy offloading while preserving CPU access to interleaved pages.The proposed unitary, malleable memory space supports distinct page layouts without software remapping or re-layout.Source: §1 · pp. 3–4
  • Dual-track page management, DRAM-side address-mapping hardware, and PIM-assisted page migration address page coexistence, differing layouts, and fragmented physical memory.The journal version also generalizes the design to LPDDRx and HBMx while retaining CPU-side compatibility.Source: §1 · pp. 3–4
  • GUMPIM's contributions also include NCCL-inspired collective communication primitives that use shared addressing and fine-grained inter-PIM connectivity to reduce synchronization and data movement.The introduction identifies isolated PIM memory spaces inaccessible to the host CPU as a source of added data-orchestration overhead.Source: §1 · p. 4

2 Background

CPU interleaving spreads adjacent data across memory devices, while general-purpose PIM needs contiguous data within its own memory. Existing transfer paths retain substantial translation, re-layout, and short-transfer overhead, and prior dynamic-mapping approaches leave constraints on PIM co-location or system changes.

2.1 Address Mapping and Memory Interleaving

  • CPU interleaving scatters contiguous data across banks and devices, whereas PIM mapping needs lower-level addresses in lower-order bits to keep contiguous physical addresses within a device.Existing software-only approaches still incur data re-layout and address-mapping overhead under byte-level interleaving.Source: §2.1 · p. 5, Figure 2 · p. 5

2.2 Data Transfer Overhead in DRAM-based PIM

  • Transfer time plateaus below 4 kB per transfer, and address translation plus data re-layout account for nearly 70% of a 256KB transfer despite disabling channel- and rank-level interleaving.Applications with more short transfers, including NW, BFS, and UNI, exhibit more transfer time when total transferred data is similar.Source: §2.2 · pp. 5–6, Figure 3 · p. 6

2.3 Dynamic Memory Mapping

  • A cited comparison reports hardware address translation reducing overhead by 4.5×, while surveyed dynamic-mapping approaches do not guarantee PIM operand co-location or require CPU-side changes.The section argues for control over all DRAM hierarchy levels using DRAM-side modifications without system-software changes.Source: §2.3 · p. 6, §2.2 · p. 6

3 Dual-Track Memory Management

GUMPIM manages interleaved CPU pages and PIM pages separately in one memory space, using chunk-based management to provide PIM units with local contiguous data. Its address schemes let the CPU and PIM units address the same PIM data, while chunk-size selection balances migration cost against chunk-list area.

3 Dual-Track Memory Management

  • GUMPIM separately manages CPU and PIM pages in one memory space, retaining global interleaving and existing CPU-page management while using chunks for PIM-page locality.The scheme is designed to optimize the two page types without compromising one another.Source: §3 · pp. 6–7

3.1 Chunk-based Memory Management for PIM Pages

  • A chunk is evenly distributed across banks when its size meets the stated organization threshold, giving each PIM unit a local PIM page.In the example configuration, a 256 MB chunk contributes 128 kB per bank, the PIM page size.Source: §3.1 · p. 7

3.2 CPU Addressing PIM Pages

  • The CPU-side layout maps adjacent addresses preferentially to the same PIM unit, with chunk numbers selecting chunks and offset bits identifying data within each chunk.The host program records virtual chunk numbers and derives a chunk and PIM offset to locate data across allocated chunks.Source: §3.2 · p. 7

3.3 PIM Units Addressing PIM Pages

  • PIM units use a dedicated virtual address space and chunk list to translate PIM chunk numbers to physical chunks, reaching the same hardware address as the CPU for the same byte.The host OS allocates PIM pages, so PIM units are not inherently aware of their physical addresses at runtime.Source: §3.3 · pp. 7–8

3.4 Chunk Size Selection

  • 256 MB chunks are selected to minimize PIM Chunk List area, balancing migration cost against the list's storage overhead.Larger chunks require more page migration, while smaller chunks require more chunks and a larger list; the selected size can be adapted to standard host-OS huge-page configurations.Source: §3.4 · p. 8

4 Hardware Support for Page Layouts

GUMPIM places address-mapping and data re-layout hardware on the DRAM side to support distinct CPU and PIM page layouts. Its mapping path preserves CPU-page mapping and translates PIM accesses, while re-layout assembles PIM data for transfer to the CPU.

4 Hardware Support for Page Layouts

  • The GUMPIM interface uses DRAM-side modules for below-bank address mapping and PIM-page data re-layout, while the CPU memory controller applies mappings at and above the bank level.The design places modules in the RCD and DIMM buffer chips and includes PIM-side address translation inside each bank.Source: §4 · pp. 8–9, Figure 6 · p. 9

4.1 CPU-side Address Mapping

  • The CPU-page mapping remains unchanged, while PIM-page mapping combines BIOS-configured above-bank mapping with DRAM-side below-bank mapping.The above-bank path uses a non-interleaved address alias to the same memory space as the default interleaved mapping.Source: §4.1 · p. 9, §4.1.1 · p. 9

4.1.1 Above-Bank Address Mapping.

  • Bit 37 selects the default TAD0 mapping for CPU pages or the non-interleaved TAD1 alias for PIM pages in the 256 GB example.The controller maps rank, channel, and bank, then sends remaining address bits to the DRAM-side interface for below-bank mapping.Source: §4.1.1 · p. 9

4.1.2 Below-Bank Address Mapping.

  • The Rank Chunk List identifies page type, and the Address Translation Module maps PIM addresses to row, column, and device addresses before command generation.For CPU pages, row and column addresses pass through unchanged; for PIM pages, column bits encode the device address and a real column address.Source: §4.1.2 · p. 10
  • The interface pipeline preserves DRAM timings tRP and tRCD, adding a fixed tUI delay that does not accumulate across pipelined commands.On a Re-layout Cache hit, the Command Generator can initiate a burst transfer directly from the cache, saving tCL.Source: §4.1.2 · p. 10, §4.3 · p. 11

4.2 PIM-side Address Translation

  • The PIM Chunk List translates a PIM-generated virtual chunk number to its allocated physical chunk number, which combines with the PIM offset to form a physical address.The list entries are indexed by PIM chunk number.Source: §4.2 · p. 10

4.3 Data Re-layout

  • A 64B PIM burst mapped to one device would otherwise require eight normal bursts, leaving 7 × 64B unused; the Re-layout Cache buffers and assembles the data on the DRAM side.Eight sub-buffers collect the bursts, then the device address selects the data sent to the memory bus.Source: §4.3 · pp. 10–11, Figure 7 · p. 11

4.4 DRAM Instruction Support for Dual-Track Memory Management

  • ACN updates the PIM and Rank Chunk Lists on allocation, CCL clears them at program termination, and CRC flushes cached data to prevent inconsistency from delayed updates.These instructions use MMIO operations or register configurations, without extending the CPU ISA or DRAM standard commands.Source: §4.4 · pp. 11–12, Table 1 · p. 12

5 Software Support for GUMPIM

GUMPIM’s software support pairs PIM-page allocation and unit-aware access with cache-conscious traversal and communication APIs. These interfaces manage chunk boundaries and organize transfers to use the Re-layout Cache effectively.

5.1 Memory Allocation

  • malloc_pim(len) allocates PIM memory in chunks, faults in pages, locks them against swapping, and registers their physical chunk numbers with the PIM hardware.CPU pages continue to use the unchanged glibc malloc().Source: §5.1 · p. 12

5.2 Unit-Aware Memory Access

  • Unit-aware APIs provide per-unit pointer, copy, and iterator operations across chunk boundaries, abstracting each PIM unit’s memory as a linear address space.This supports vector traversal and multi-operand alignment without manual address calculations.Source: §5.2 · p. 12

5.3 Performance Impact of Different Nested Loop Order

  • All-gather loop orders that obey the stated nesting rules achieve performance only 1.6× slower than the CPU-page baseline.The rules place device-address loops innermost to improve Re-layout Cache hit rates and DRAM bandwidth; the strategy also applies to general traversal.Source: §5.3 · p. 13, Figure 9 · p. 13

5.4 High-level APIs for Inter-PIM-Unit Data Transfer

  • Scatter distributes a bank’s contiguous data across banks, while broadcast copies one bank’s data to all banks using Re-layout Cache locality.The described iteration orders batch or reuse data through the cache.Source: §5.4 · pp. 13–14
  • Gather consolidates data from all DRAM banks into one bank, reading matching offsets in adjacent iterations to use Re-layout Cache locality.Source: §5.4 · p. 14

6 Guaranteed PIM Chunk Allocation

GUMPIM addresses fragmented physical memory with DRAM-side, PIM-assisted migration for chunk allocation. Row-level redirection preserves access during migration, while row-stripe granularity coordinates migration across interleaved channels.

6.1 Memory Fragmentation and Chunk Allocation

  • At 25% occupancy in a 256GB system, a 256MB chunk allocation can fail; migration is costly and unmovable pages impede consolidation, while static reservation lacks flexibility and wastes CPU memory.The cited experiments motivate dynamic migration as a path to guaranteed chunk allocation.Source: §6.1 · p. 14

6.2 Challenges in CPU-Assisted Page Migration

  • CPU-assisted migration reduces overhead by 2× but still transfers data between CPU and memory; copying accounts for 58% of total migration overhead when TLB shootdown is excluded.The LLC-buffered approach also requires CPU-side hardware changes and redirects I/O-device accesses to the CPU-side cache.Source: §6.2 · p. 14

6.3 PIM-Assisted Page Migration

  • DRAM-side address redirection and PIM-unit data movement avoid bus-crossing migration transfers, while the Row Migration Table routes access to the buffer, source, or destination row as migration proceeds.The OS later updates page tables and clears the table; new chunk allocation waits until that table is cleared.Source: §6.3 · pp. 14–16

6.4 Justification of Migration Granularity

  • Migration operates on complete DRAM rows because the Row Migration Table must select a row at ACT time, before the column address arrives.Across interleaved channels, the OS coordinates co-aligned rows as row stripes to preserve page contiguity; experiments found over 5,000 free stripes even above 90% occupancy.Source: §6.4 · pp. 16–17

6.5 DRAM Instruction Support for Guaranteed PIM Chunk Allocation

  • SRMT adds source-destination row pairs to the Row Migration Table, and CRMT clears it after the OS updates page-table entries.Source: §6.5 · p. 17

7 GUMPIM on Other DRAM Systems

GUMPIM’s dual-track management extends to HBMx and LPDDRx, with reduced DRAM-side hardware where those architectures lack device-level hierarchy. The approach retains necessary CPU-side mapping and PIM-side translation.

7 GUMPIM on Other DRAM Systems

  • HBMx and LPDDRx lack device-level hierarchy, allowing GUMPIM to omit below-bank mapping and data re-layout hardware while retaining above-bank CPU mapping and PIM address translation.The architecture-specific support is illustrated for LPDDRx and HBMx.Source: §7 · p. 17, Figure 11 · p. 17
  • LPDDRx configurable address decoders and customizable HBMx controller logic enable the required mapping without host-side changes, with reduced DRAM-side hardware complexity.Adjacent addresses are mapped to the same bank whenever possible to preserve spatial locality.Source: §7 · pp. 17–18

8 Evaluation

GUMPIM preserves CPU workload performance and reduces PIM-side CPU computation and allocation time in the evaluated configurations. Its evaluation also examines scaling, migration granularity, DRAM technology, and hardware area.

8.1 Experiment Setup

8.2 CPU Workloads

  • CPU workload performance degradation on GUMPIM is less than 0.1% across the evaluated benchmarks, compared with the CPU system.PIM-Ioff degradation averages 88.2%, 83.8%, and 74.2% across benchmarks for 2-, 4-, and 8-core configurations, respectively.Source: §8.2 · pp. 19–20, Figure 12 · p. 20

8.3 PIM Workloads

  • GUMPIM reduces CPU computing and data transfer time by 4.93× on average versus PIM-Ioff, with a 1.96× overall workload speedup when allocation and data staging are excluded.The evaluation uses one CPU thread and PIM units from all 16 ranks.Source: §8.3 · p. 20, Figure 13 · p. 20
  • BFS and PageRank better exploit added PIM-unit parallelism with GUMPIM, whose lower data-transfer overhead makes scaling beyond 8 PIM ranks less detrimental.Source: §8.3 · pp. 20–21, Figure 14 · p. 21

8.4 Chunk Allocation Overhead

  • 2.7× lower chunk allocation time yields 2.3× lower average end-to-end PIM workload latency at a 50% migration ratio.When allocation alone exceeds 95% of end-to-end latency for several workloads, the allocation mechanism achieves a 2.65× average end-to-end speedup.Source: §8.4 · p. 21, Figure 15 · p. 22
  • At a computation-to-allocation ratio of 100:1, GUMPIM still averages 2.6× speedup for the named large-footprint, modest-computation workloads.The speedup decreases as computation time becomes a larger share of total time.Source: §8.4 · p. 21, Figure 15 · p. 22

8.5 Migration Granularity

  • At high memory occupancy, chunk allocation latency approaches 55 ms as occupied pages force migration of their full granularity; finer granularity increases Row Migration Table area.The design choice is the finest migration granularity within the acceptable table-area budget.Source: §8.5 · pp. 21–22, Figure 16 · p. 22

8.6 GUMPIM on LPDDR5 and HBM3

  • Across five typical benchmarks, GUMPIM reduces CPU-part time (CPU computing and data transfer) by 6.0× on average; versus PIM-Ioff, reductions average 5.9× on LPDDR5 and 5.2× on HBM3.The PIM Chunk List and Row Migration Table add 0.13% overhead relative to the 130 mm2 DRAM die area; the Re-layout Cache occupies 0.21 mm2.Source: §8.6 · p. 23, §8.7 · p. 23

9 Conclusion

GUMPIM combines CPU and PIM pages in a unitary, malleable memory space. It reports a 2.3× overall speedup for PIM tasks while limiting CPU-task performance loss to 0.1%.

9 Conclusion

  • 2.3× overall speedup for PIM tasks, alongside only 0.1% performance loss for CPU tasks.The reported speedup comes with reductions of 4.93× in CPU-part time and 2.7× in memory allocation time.Source: §9 · p. 23
  • 4.93× reduction in CPU-part time and 2.7× reduction in memory allocation time for PIM tasks.Source: §9 · p. 23
  • GUMPIM provides a unitary and malleable memory space for CPU and PIM units, with fine-grained CPU-page interleaving and no data-transfer overhead between CPU and PIM units.The method can be adapted to DIMM DRAM, LPDDRx, and HBMx.Source: §9 · p. 23
Loading 2610.23406v1…