Source-linked AI summary
PipeDRAM: A Data-Transposition-Free In-DRAM Architecture with Hardware/Software Pipelining
Geraldo F. Oliveira, Ataberk Olgun, Ismail Emir Yüksel, F. Nisa Bostancı, Pedro H. E. Becker, Mohammad Sadrosadati, Saugata Ghose, Juan Gómez-Luna, Onur Mutlu
TL;DR
Existing PUD architectures rely on vertical data layouts, while conventional systems use horizontal layouts, creating a need for costly runtime transformation. PipeDRAM maps bits for PUD-friendly placement within a horizontal layout and pipelines in-DRAM operations. It reports higher performance and lower energy consumption than three state-of-the-art PUD systems, with low reported area costs.
Problem
Existing bit-serial PUD architectures use vertical layouts that conflict with conventional horizontal memory organization and require runtime data transformation.
Source: §1 · pp. 1–2Method
PipeDRAM maps bits within memory requests for PUD-friendly placement in a horizontal layout and pipelines bit-dependent and bit-independent operations across DRAM mats.
Source: §1 · p. 2Results
11.8×, 11.8×, and 80.4× higher performance and 25.4×, 3.0×, and 38.0× lower energy consumption than SIMDRAM, MIMDRAM, and Proteus, respectively.
Source: §1 · p. 2Takeaways & Limitations
PipeDRAM enables bit-serial and bit-parallel PUD execution directly on horizontally laid-out data without runtime layout transformation.
Source: §8 · p. 16The evaluation excludes independent performance and energy benefits that fine-grained DRAM, SALP, and LISA could provide beyond supporting PipeDRAM.
Source: §6.6 · p. 15
Abstract
from arXiv · showhide
Processing-using-DRAM (PUD) architectures exploit the analog operational properties of DRAM to perform bulk bitwise Boolean and arithmetic operations inside memory arrays by organizing data in a vertical layout, where operand bits are stacked along DRAM columns. However, modern computing systems natively employ a horizontal data layout that preserves the cache line abstraction, leverages spatial locality in row buffers, and enables high memory throughput. This fundamental mismatch forces existing PUD architectures to frequently perform data layout transformations between horizontal and vertical formats, incurring significant performance, energy, and system integration overheads. Our goal is to eliminate data transposition overheads in PUD systems at low cost. To this end, we propose PipeDRAM, a PUD architecture that eliminates the need for runtime data layout transformation, enabling PUD operations directly over horizontally laid-out data. PipeDRAM's key ideas are to (i) deterministically reorganize bits inside each memory request to enable a PUD-friendly data placement within a DRAM array in a horizontal data layout, and (ii) employ a pipeline-based execution model that overlaps bit-dependent and bit-independent in-DRAM operations to exploit bit-level parallelism across the memory array. We compare PipeDRAM to different computing platforms. PipeDRAM provides (i) 11.8x, 11.8x, and 80.4x higher performance and (ii) 25.4x, 3.0x, and 38.0x lower energy consumption than three state-of-the-art PUD systems. PipeDRAM incurs low area cost on top of a DRAM chip (1.86%) and CPU die (0.05%). To enable further research on PUD systems, we open-source PipeDRAM at https://github.com/CMU-SAFARI/PipeDRAM.
1. Introduction
PipeDRAM addresses the mismatch between vertical-layout PUD and conventional horizontal memory by enabling in-DRAM operations on horizontally laid-out data. Its bit mapping and pipelined execution remove runtime transposition while delivering substantial gains over other PUD systems.
1. Introduction
- Vertical layout localizes all bits of each operand in one DRAM column, but horizontal placement scatters bits across the hierarchy and lacks native column-wise carry propagation.Existing approaches address these constraints with vertical organization and row-copy operations, but this conflicts with conventional horizontal memory organization.Source: §1 · p. 1
- Runtime transposition reduces performance and energy efficiency by 3.9× and 3.7×, respectively, versus an idealized PUD system without transposition.Vertical layouts also amplify cache-line traffic and require applications to manage two physical data layouts.Source: §1 · pp. 1–2
- MABM permutes bits within each cache line so each word stays within one chip while its bit positions are distributed across mats, without throughput loss or format-conversion overhead.It applies the permutation on writes and its inverse on reads, keeping operands within a single subarray.Source: §1 · p. 2
- PipeDRAM makes DRAM mats pipeline stages, overlapping bit-independent primitives across bit positions while carry-dependent primitives advance between mats.It statically schedules primitives offline and uses hierarchical in-DRAM copying to keep carry propagation from stalling pipeline stages.Source: §1 · p. 2
- 11.8×, 11.8×, and 80.4× higher performance and 25.4×, 3.0×, and 38.0× lower energy consumption than SIMDRAM, MIMDRAM, and Proteus, respectively.PipeDRAM also achieves 356× and 11.7× higher energy efficiency than a CPU and GPU, respectively, on average across twelve applications.Source: §1 · p. 2
- PipeDRAM incurs area costs of 1.86% on a DRAM chip and 0.05% on a CPU die.The paper reports evaluation across twelve real-world applications and open-sources PipeDRAM.Source: §1 · p. 2
2. Background
The background introduces DRAM organization and operation, then reviews PUD architectures, their execution primitives, and the parallelism available within PUD operations.
2.1. DRAM Organization & Operation
- A DRAM bank contains subarrays and global control logic; each subarray contains mats with local row decoders, row buffers, and helper flip-flops.DRAM requests activate a row, read or write a cache line, and precharge; a 64 B cache line is transferred over an eight-beat burst.Source: §2.1 · p. 3
2.2. Processing-Using-DRAM
- Ambit implements in-DRAM majority logic through triple-row activation, while SIMDRAM exposes offline-generated primitive sequences and Proteus adds bit-parallel operations.Fine-grained MIMDRAM supports mat-level activation and inter-/intra-mat data movement for independent operations.Source: §2.2 · pp. 3–4
- PUD operations compose primitives such as row copies and majority operations; bit-level parallelism is available among primitives independent of carry propagation.A 2-bit addition illustrates how copying can dominate computation and how fine-grained activation can execute independent primitives concurrently.Source: §2.2 · p. 4
- Prior PUD substrates use SIMD, MIMD, or pipelined execution and differ in parallelism and granularity, but all require transposition from horizontal to vertical data layout.SIMDRAM uses coarse full-row computation, MIMDRAM adds fine-grained mat-level execution, and Proteus pipelines bit-independent primitives.Source: §2.2 · p. 4, Table 1 · p. 5
3. Motivation
The motivation section details the costs of vertical-layout PUD: transposition overhead, amplified cache traffic, and the need to manage two physical layouts. These costs vary across applications and operation complexity, motivating computation directly on horizontally laid-out data.
3. Motivation
- Vertical-layout cache-line reads or writes require N separate DRAM command sequences for an N-bit data element, amplifying memory access cost because row activations dominate latency and energy.In a horizontal layout, a cache line is serviced by one sequence of DRAM commands.Source: §3 · p. 5
- Quadratic operations reach low transposition overhead at lower operational intensities than logarithmic operations because their higher per-element compute cost improves the compute-to-transposition ratio.The analysis spans logarithmic, linear, and quadratic PUD complexity classes.Source: §3 · p. 5
- 77%–100% of execution time and 73%–100% of energy consumption are due to transposition in bp, fdtd, and km, while compute-intensive applications still incur up to 14.6% time and 12.3% energy overhead.Figure 4 compares transposition overhead across 16 bit-serial operations and 12 real-world applications.Source: Figure 4 · p. 5, §3 · p. 5
- Evicting vertically organized data requires assembling N cache lines into a full slice before issuing N DRAM writes, amplifying data movement and coupling PUD execution to cache behavior.A single element access or eviction can trigger writeback of its entire slice.Source: §3 · p. 5
- Maintaining horizontal and vertical physical layouts complicates software and makes performance depend on when transformations occur relative to cache residency and future accesses.Transforming too late places conversion on the PUD critical path; transforming too early can force further conversions after host accesses.Source: §3 · p. 5
- The stated goal is high-throughput in-DRAM computation on horizontally laid-out data, eliminating runtime transposition and avoiding dual physical layouts.Source: §3 · p. 5
4. PipeDRAM Overview
PipeDRAM runs bit-serial PUD operations directly on horizontally laid-out data by combining deterministic cache-line bit mapping with pipelining across DRAM mats. Its design also uses fine-grained mat access, subarray-level parallelism, and hierarchical data copying.
4.1. Subarray Organization
- PipeDRAM builds its subarray organization on MIMDRAM mat access, LISA inter-subarray connections, and the SALP row-decoder latch.It logically designates neighboring subarrays for copying without adding circuitry substrate.Source: §4.1 · p. 6
4.2. Mat-Aware Bit Mapping (MABM)
- MABM permutes each cache line’s bits so scalar elements remain within one chip while their bit-positions are distributed across mats.The controller applies the mapping before writing; the same cache-line abstraction is preserved.Source: §4.2 · pp. 6–7
4.3. Software-Assisted Pipelined PUD Execution
- Successive bit-positions of one operation run in neighboring mats, which overlap execution when their bit-dependencies permit.This converts mat-level parallelism into pipeline parallelism within a subarray.Source: §4.3.1 · p. 7
4.3.1. Pipelined Execution Flow.
- 18 to 13 cycles: PipeDRAM overlaps PUD primitives for different bit-positions across mats in the illustrated addition.The example proceeds through initialization, carry generation and propagation, steady state, and pipeline drain.Source: §4.3.1 · p. 7
Software-Assisted Instruction Scheduling.
- The static scheduler maps each primitive to a mat and cycle while respecting data dependencies and available-mat constraints.It computes a feasible initiation interval and increases it when a candidate schedule violates either constraint.Source: Software-Assisted Instruction Scheduling · pp. 7–8
4.4. Hierarchical In-DRAM Data Copy Scheme
- Inter-mat transfers are limited by helper flip-flop width, so propagating a mat-wide intermediate value requires repeated transfers and can bottleneck throughput.The cited organization gives 4–8 helper flip-flop bits as a typical example.Source: §4.4 · p. 8
1 Construct DDG G from PUD primitives in U
The scheduler begins by constructing a data-dependency graph from the PUD primitives in the input µProgram.
1 Construct DDG G from PUD primitives in U
- The first scheduling step constructs dependency graph G from the PUD primitives in µProgram U.The scheduler then derives recurrence- and resource-constrained initiation intervals.Source: §1 · p. 8
5 repeat
PipeDRAM separates latency-critical carry communication from non-critical propagation using a hierarchical in-DRAM copy scheme. The discussion also describes its parallelism and controller support.
5 repeat
- The hierarchical copy scheme sends dependency-critical values over a high-throughput inter-subarray path and uses a low-cost inter-mat path for asynchronous propagation.A neighboring copying subarray stages intermediate values; it does not reduce effective DRAM capacity because it uses only the local row buffer during PUD execution.Source: §5 · p. 9
4.4.1. Asynchronous Carry Propagation.
- In the baseline, inter-mat carry copying blocks subsequent bit-independent work; PipeDRAM stages the carry in a neighboring copying subarray before forwarding it asynchronously to the next computing mat.The sequence uses LISA-RISC for the initial transfer, GB-MOV for asynchronous propagation, and LISA-RISC to deliver the carry to the computing subarray.Source: §4.4.1 · p. 9
5 repeat
- Mat-width helper flip-flops are prohibitively costly: widening them to a mat’s width would add 20–26% area to a 4–8 Gbit DDR4 chip.The area estimate is based on CACTI analysis; the existing 4-helper-flip-flop example requires 128 serialized transfers across a 512-column mat.Source: §5 · p. 8, §4.4.1 · p. 9
4.5. Discussion
- PipeDRAM converts mat-level parallelism into pipeline parallelism within one operation while retaining chip-level MIMD parallelism across a logical subarray.Because MABM interleaves operand bits across chips, different chips can execute independent PUD operations concurrently.Source: §4.5 · p. 9
- MABM applies the same fixed-size mapping to every cache line without needing its data type or bit-precision; bit-precision is supplied for PUD execution through inherited ISA extensions.The mapping is defined over 64-bit words in this design.Source: §4.5 · pp. 9–10
- The controller retrieves each µProgram’s precomputed schedule and streams its primitive-to-mat, cycle, and stage assignments to the processing engine.The schedule table stores the static schedule generated by the scheduler.Source: §4.5 · p. 10
5. Methodology
The evaluation uses a cycle-level simulator to compare PipeDRAM with prior PUD systems and processor-centric systems on selected workloads. It also describes the simulator validation, configurations, and workload inputs.
5. Methodology
- PipeDRAM is evaluated with an in-house cycle-level simulator against SIMDRAM, MIMDRAM, and Proteus, with reported comparisons also using CPU and GPU measurements.The simulator was validated against gem5 implementations of SIMDRAM and MIMDRAM and accounts for SALP-MASA ACT latency.Source: §5 · p. 10
- CPU and GPU measurements come from a prior open-source artifact, using the same workloads, compilation settings, and input datasets for comparison.They were not collected in this work.Source: §5 · p. 10
- Three PipeDRAM configurations distinguish FIFO scheduling, the proposed scheduler, and the scheduler combined with hierarchical copying.The configurations are PipeDRAM-Seq, PipeDRAM-Pipe, and PipeDRAM-Hier.Source: §5 · p. 10
- The real-workload analysis covers twelve PUD-friendly applications from SPEC 2017, Phoenix, Polybench, and Rodinia.It uses the largest available input datasets and native data precisions: 32-bit except for x264 at 8-bit.Source: §5 · p. 10
6. Evaluation
PipeDRAM is evaluated against PUD systems and processor platforms across operations and real-world workloads, alongside analyses of utilization, pipeline trade-offs, carry overhead, and area. PipeDRAM-Hier improves execution time and energy consumption over three PUD baselines, with low reported DRAM-chip and CPU-die area costs.
6.1. Throughput Analysis
- 2.0× higher average throughput than SIMDRAM/MIMDRAM with transposition; at 32-bit precision PipeDRAM-Hier reaches 1.06× their transposition-free throughput despite 32× fewer SIMD lanes.PipeDRAM-Hier's steady-state throughput is independent of bit precision, while SIMDRAM/MIMDRAM throughput scales with precision according to operation class.Source: §6.1 · p. 11, Figure 9 · p. 11
- PipeDRAM-Hier outperforms Proteus by 2–6× without transposition and 6–12× with transposition across four bit-parallel operations.Modulo scheduling overlaps bit-independent primitives across pipeline stages, while Proteus serializes primitives within each bit-position.Source: §6.1 · p. 11, Figure 9 · p. 11
6.2. Real-World Applications
- 11.77×, 11.75×, and 80.4× lower execution time, and 25.4×, 3.0×, and 38.0× lower energy consumption, versus SIMDRAM, MIMDRAM, and Proteus, respectively.These are reported across twelve real-world applications.Source: §6.2 · p. 12, Figure 10 · p. 12
- 356×, 11.7×, 25.4×, 3.0×, and 37.9× higher performance per Watt than CPU, GPU, SIMDRAM, MIMDRAM, and Proteus, respectively, on average across twelve applications.GPU measurements are available for eight workloads.Source: §6.2 · pp. 12–13, Figure 11 · p. 13
6.3. Load Balance & Pipeline Utilization
- 87% average mat utilization (93% across eleven 32-bit workloads) and less than 0.04% inter-chip imbalance; division-heavy workloads reach 78–88% mat utilization.The reported division utilization reflects the deployed algorithm's per-bit work distribution.Source: §6.3 · p. 13, Figure 12 · p. 13
6.4. SIMD-Width Trade-Off Analysis
- 1.35× faster than SIMDRAM on average across twelve applications, even when SIMDRAM's transposition overhead is ignored.All twelve applications fall in the PipeDRAM-favored region of the precision and data-size trade-off.Source: §6.4 · p. 14, Figure 13 · p. 14
6.5. Carry-Propagation Overhead Analysis
- Below 0.1% of execution time remains exposed to carry propagation for quadratic operations in steady state; eleven applications have B ≥ 5 and at most 2.9% exposed overhead.The hierarchical in-DRAM copy scheme makes the per-batch inter-subarray carry transfer 15× cheaper than a direct inter-mat transfer.Source: §6.5 · p. 14, Figure 14 · p. 14
6.6. Area Analysis
- PipeDRAM adds 1.86% area overhead on a DRAM chip and 0.05% on a CPU die; the evaluation excludes independent benefits from fine-grained DRAM, SALP, and LISA.Those schemes' costs could be amortized further if enabled for general deployment.Source: §6.6 · pp. 14–15
7. Related Work
PipeDRAM uses mat-level pipelining and a horizontal PUD-friendly layout to distinguish its approach from prior DRAM and NVM designs. Across twelve applications, it outperforms MIMDRAM and Proteus in execution time and energy consumption.
Processing-Using-DRAM
- PipeDRAM assigns each mat a successive bit-position and uses modulo scheduling to overlap bit-independent primitives, unlike Proteus's serialized per-bit-position primitives.Proteus also operates multiple subarrays simultaneously, constrained by DRAM power-delivery limits such as tFAW.Source: §7 · p. 15
- 11.8× and 80.4× higher execution performance, and 25.4× and 38.0× lower energy consumption, than MIMDRAM and Proteus, respectively, across twelve applications.PipeDRAM exploits bit-level parallelism within one operation, unlike MIMDRAM's data-level parallelism across independent operations; Proteus uses subarrays rather than mats as its finer-grained units.Source: §7 · pp. 15–16
NVM-Based PUM Architectures
- RACER is the closest NVM-based comparison, but its fixed hardware micro-op queues cannot exploit operation-specific overlap as PipeDRAM's software modulo scheduler does.NVM-based PUM systems often use dedicated accelerators and custom layouts, whereas PipeDRAM targets commodity DRAM shared with the host and must preserve horizontal layout.Source: §7 · p. 16
Avoiding Dual-Data Layout in PNM Systems
- PNM relayout hardware and huge-page allocation mitigate dual layouts, but do not resolve PUD's need to align data through the internal DRAM hierarchy.For PUD, mapping across subarrays and mats is not visible to the memory controller or operating system; PipeDRAM uses MIMDRAM's allocation routine with huge pages and reverse-engineered address mapping.Source: §7 · p. 16
- PUD's vertical layout requires multiple DRAM row activations to reconstruct a CPU-friendly cache line, so a DRAM-side relayout accelerator cannot eliminate transposition overhead.PipeDRAM instead uses MABM to place PUD-friendly data horizontally.Source: §7 · p. 16
8. Conclusion
PipeDRAM combines horizontal PUD-friendly placement with scheduled execution across DRAM mats and hierarchical in-memory data copying. Its evaluation reports improved performance and energy efficiency over three state-of-the-art PUD systems at low DRAM-chip and CPU-die area cost.
8. Conclusion
- PipeDRAM combines MABM, software-assisted scheduling across DRAM mats, and hierarchical in-memory copying to execute directly on horizontally laid-out data without runtime transformation.The copy scheme separates latency-critical carry propagation from non-critical intermediate transfers.Source: §8 · p. 16
- PipeDRAM significantly outperforms three state-of-the-art PUD systems in performance and energy efficiency while incurring low area cost on a DRAM chip and CPU die.The conclusion characterizes PipeDRAM as bridging modern horizontal memory layout and the vertical layout required by PUD architectures.Source: §8 · p. 16
- The PipeDRAM infrastructure is freely open-sourced to support further research, design, and adoption of PUD systems.The paper provides the project repository at https://github.com/CMU-SAFARI/PipeDRAM.Source: §8 · p. 16
A.1. Abstract
The artifact packages PipeDRAM evaluation materials and provides quick and full workflows for reproducing figures and results.
A.1. Abstract
- The artifact includes analytical PUD, instrumented workloads, immutable paper inputs, plotting programs, MABM RTL, and validation tools.It compares SIMDRAM, MIMDRAM, Proteus, and three PipeDRAM configurations using the submitted timing, energy, batching, and transposition methodology.Source: §A.1 · p. 23
- The quick workflow regenerates and validates figures from frozen paper CSVs, while the full workflow rebuilds and runs redistributable paper-scale workloads.A one-command workflow reproduces an individual figure, and a guided notebook separates experiments into cells.Source: §A.1 · p. 23
- The artifact supports reproduction of PipeDRAM's evaluation results through its included inputs, tools, and workflows.Its stated scope includes all figures from the frozen inputs or, in full mode, rebuilt workloads.Source: §A.1 · p. 23
A.2. Artifact Checklist (Meta-Information) •
The checklist specifies the artifact's implementation, inputs, runtime requirements, workflows, outputs, and reproduction resources.
A.2. Artifact Checklist (Meta-Information)
- The artifact implements per-primitive PUD simulation, dependency-graph construction, modulo scheduling, hierarchical carry transfer, and MABM bit permutation.It uses Python, C/C++, Verilog, and shell scripts, with GCC/G++, GNU Make, Yosys, and Icarus Verilog.Source: §A.2 · p. 23
- Evaluation uses an analytical DRAM latency-and-energy model with fixed submitted parameters and an instrumented application-level PUD cost model.Metrics include latency, throughput, energy, normalized performance per watt, transposition overhead, utilization, carry overhead, crossover point, and area.Source: §A.2 · p. 23
- Inputs include frozen CSVs, deterministically generated kmeans data, optional checksum-verified heartwall video, and frozen x264 results.Restricted SPEC source and media, as well as the heartwall video, are excluded.Source: §A.2 · p. 23
- Quick mode needs a conventional x86-64 system; full mode is recommended on a 32-thread host with 64 GB RAM, and neither requires a GPU.The runtime environment is Ubuntu 24.04 x86-64 with Python 3.12.Source: §A.2 · p. 23
- Automated quick, full, area-only, and per-figure workflows produce figures, CSVs, JSON metrics, and textual validation results.Outputs include one-page PDF figures and auxiliary PNGs.Source: §A.2 · p. 23
- The artifact covers Figures 4 and 9–14, workload evaluation, dataset verification, and area/RTL evaluation.A guided Jupyter notebook and shell scripts automate the workflows.Source: §A.2 · p. 23
- Quick-mode setup and disk use are approximately 5–15 minutes and 1 GB; full mode reserves 10 GB and may take up to 24 hours.Quick experiments normally finish in under two minutes; full-mode duration depends on the evaluator machine.Source: §A.2 · p. 23
- Project-authored code uses the MIT license, and the artifact has a Zenodo record with DOI 10.5281/zenodo.21530459.The record provides a tokenized preview; third-party benchmark notices and licenses are retained.Source: §A.2 · p. 23
A.3. Description
The artifact is independently distributable, with README.md as its entry point and a guided, figure-by-figure notebook interface.
A.3.1. How to Access.
- Quick reproduction needs a few CPU cores, 2 GB RAM, and about 1 GB free disk; the full workflow is recommended with 32 threads, 64 GB RAM, swap, and 10 GB free disk.CPU RAPL and GPU measurements are immutable paper inputs rather than remeasured results.Source: §A.3.2 · p. 23
A.3.2. Hardware Dependencies.
- The supported environment is Ubuntu 24.04 x86-64, and setup checks required build, PDF, and Verilog tools after installing pinned Python dependencies.After prerequisite packages are installed, no administrator privileges are needed.Source: §A.3.3 · p. 23, §A.4 · p. 23
A.5. Experiment Workflow
- Full mode rebuilds and runs ten named workloads; heartwall runs only when its verified video is supplied, while x264 always uses its frozen result.Generated files go under generated/, and immutable references are never overwritten.Source: §A.5 · p. 23
A.5.3. Individual Figures.
- Figures 4, 10, and 11 use application CSVs, while Figures 9 and 12–14 run analytical simulator experiments directly.The notebook caches full workload preparation to avoid repeating it across data-dependent figure cells.Source: §A.5.3 · p. 24
- Validation checks the 1.86% area sum, 1,350-entry schedule's 5,400-byte fit in 16 KiB, zero-logic-cell MABM synthesis, and permutation/inverse round trip.It also checks numerical claims, PDFs, labels, rasterized renderings, reference checksums, and path isolation.Source: §A.5 · p. 23, §A.5.3 · p. 24
A.5.4. Area and RTL.
- The expected-metrics file contains plotted workload values, geometric means, crossover points, mat utilization, and exposed-carry fractions.The artifact also regenerates Figures 4 and 9–14 and the area comparison from immutable paper inputs.Source: §A.5 · p. 23, §A.5.4 · p. 24
A.7. Experiment Customization
- The simulator CLI exposes DRAM, timing, energy, precision, scheduling, transposition, subarray, and bank parameters for sensitivity analysis.Customized parameters are not expected to pass the paper-reference validator.Source: §A.7 · p. 24
A.8. Troubleshooting and Notes
- On constrained systems, quick mode is recommended because full workloads use paper-scale dimensions and may allocate several gigabytes.Troubleshooting notes also identify reference-input, rendering, and environment issues that can cause validation mismatches.Source: §A.8 · p. 24