PaperlayerView exact revision on arXiv

Source-linked AI summary

TileDRAM: Compiler-Controlled Tile-to-DRAM Layouts for 3D DRAM GPGPUs

Guangyuan Xu, Lanxuan Zhang, Ziyang Li, Zhaoshi Li, Yajun Ha

arXiv:2610.23767v1cs.AR

TL;DR

GPU compilers provide tile-level layouts for compute-side mapping but lack a corresponding abstraction for DRAM placement, where conventional placement can cause row-buffer conflicts. TileDRAM extends Linear Layouts to tile-to-DRAM placement and selects layouts per recurring tile access signature. Across 108 MoE FFN cases, it reduces the request-weighted row-conflict rate from 75.7% to 15.2% and achieves a 1.77× geomean speedup.

  • Problem

    GPU layout abstractions do not specify how operand-tile data are arranged in DRAM, leaving placement outside the compiler’s tile-level layout abstraction.

    Source: §1 · p. 2

  • Method

    TileDRAM extends GPU Linear Layouts to reusable tile-to-DRAM placements and selects a layout per tile access signature by evaluating ordered accesses’ row-buffer behavior.

    Source: §1 · p. 2

  • Results

    Across 108 full-trace MoE FFN cases, the request-weighted row-conflict rate falls from 75.7% to 15.2% and geomean kernel speedup is 1.77×.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Compiler-selected layouts alone improve row conflicts and performance under an unchanged DRAM address mapping, without new remapping hardware or ISA support.

    Source: §1 · p. 2

    The evaluation focuses on tiled GEMM within one CU–stack pair and trace-driven Accel-Sim/Ramulator2 replay.

    Source: §8 · p. 12

Abstract

from arXiv · show

Modern GPU compilers expose rich layout abstractions for mapping tensors to threads and on-chip resources, but compute-unit-local (CU-local) 3D DRAM architectures lack a corresponding abstraction for specifying how operand-tile data are arranged within the DRAM address space. Without such an abstraction, conventional placement under the existing DRAM address mapping can subject structured tile accesses to severe row-buffer conflicts. We present TileDRAM, a compiler-runtime toolchain that introduces tile-to-DRAM layouts as a compiler-visible abstraction for DRAM placement. TileDRAM extends the GPU Linear Layout abstraction from compute-side mapping targets to DRAM placement coordinates and selects layouts per tile access signature, using recurring tile-local access behavior as the specialization unit. Since row-buffer locality depends on ordered request history, TileDRAM evaluates candidate layouts through their row-buffer behavior under fixed DRAM address mappings. Accesses under the selected layouts are realized through ordinary address arithmetic, without new address-remapping hardware or ISA support. In full-trace replay on Accel-Sim/Ramulator2 across 108 mixture-of-experts (MoE) feed-forward network (FFN) cases, TileDRAM reduces the request-weighted row-conflict rate from 75.7% to 15.2% and achieves a 1.77x geomean speedup. Retargeting six traces to six 3D DRAM organizations yields a 2.20x geomean speedup under the same compiler abstraction. On an RTX 3090, address transformation adds 0.17% geomean overhead over identity.

1 Introduction

CU-local 3D DRAM offers a high-bandwidth path, but compiler-visible GPU layouts do not specify operand-tile placement in DRAM. TileDRAM adds this placement abstraction and selects layouts for recurring tile access signatures, improving row-buffer behavior under a fixed address mapping.

Motivation

  • GPU layout abstractions cover threads and on-chip resources but leave operand-tile placement in the DRAM address space unspecified.The section frames DRAM placement as a missing compiler-visible tile-level layout decision.Source: §1 · p. 2

Approach

  • TileDRAM extends GPU Linear Layouts into reusable tile-to-DRAM maps that place logical operand-tile coordinates in DRAM coordinates.The toolchain selects layouts per tile access signature, combining tile configuration, operand properties, and ordered tile-local accesses; it simulates row-buffer state transitions to choose layouts with fewer conflicts.Source: §1 · p. 2

Implementation

  • The compiler applies selected placements through ordinary address arithmetic without new address-remapping hardware or ISA support.Retargeting to a different 3D DRAM organization reselects layouts while reusing the same abstraction and compilation path.Source: §1 · p. 2

Evaluation

  • 1.77× geomean kernel speedup accompanies a request-weighted row-conflict-rate reduction from 75.7% to 15.2% across 108 MoE FFN cases.Retargeting six traces to six 3D DRAM organizations achieves a 2.20× geomean speedup; RTX 3090 address transformation adds 0.17% geomean overhead over identity.Source: §1 · p. 2

2 Background and Motivation

The paper models a CU-local stack and uses MoE FFN tiled GEMM to study structured operand streams at its DRAM boundary. It argues that recurring tile-local behavior is a more stable specialization unit than a heterogeneous full trace.

2.1 3D DRAM and the CU-local Architecture

  • The modeled CU-local organization connects each CU to a nearby 3D DRAM stack through a local controller and analyzes one CU–stack pair at a time.The software-managed aperture reaches the local controller without virtual-address translation, warp-level coalescing, or caching; other addresses use the conventional hierarchy.Source: §2.1 · p. 3

2.2 MoE FFNs Present Structured Operand-Tile Streams

  • Grouped GEMM CTAs repeatedly load activation and weight tiles, producing separate ordered tile-local access sequences at the CU-local boundary.Dynamic scheduling can interleave resident CTAs, while persistent CTAs may process multiple tile instances.Source: §2.2 · p. 4
  • Across 108 full-trace MoE FFN cases, conventional placement yields a 75.7% request-weighted row-conflict rate as repeated tile-local differences map to repeated row changes.The stack remains banked, row-buffered DRAM, so placement-induced row sequences condition how its local bandwidth is realized.Source: §2.2 · p. 4

2.3 Stable Specialization Granularity in MoE FFNs

  • One controller mapping must accommodate distinct activation and weight address-difference distributions and runtime-varying expert workloads.Trace-wide exhaustive synthesis is costly, while tile configuration and operand properties fix a recurring ordered access sequence across instance origins and evaluated M values; TileDRAM selects per signature.Source: §2.3 · p. 4

3 Overview

TileDRAM organizes compiler-controlled placement as a target-scoped PLAN–BIND–APPLY lifecycle. PLAN selects reusable layouts for a DRAM configuration; BIND and APPLY construct and use tile-specific placement descriptors.

PLAN

  • PLAN fixes the DRAM organization and controller-owned address mapping, evaluates candidate layouts over ordered signature requests, and packages the selected layout as a reusable template.Candidates are ranked by fewer row conflicts, greater parallelism, and lower address-generation complexity.Source: §3 · p. 5

BIND

  • BIND looks up the PLAN-produced template for a tile access signature and combines it with the tile ID, source interval, and DRAM region to construct a descriptor..Source: §3 · p. 5

APPLY

  • During execution, APPLY transforms each source address with a GF(2)-based address transform and uses the descriptor’s region base to form the backend address.A new DRAM configuration reruns PLAN, while BIND and APPLY remain unchanged.Source: §3 · p. 5

4 Tile-to-DRAM Linear Layouts

TileDRAM extends GPU Linear Layouts to represent operand-tile placement in DRAM and selects layouts for recurring tile access signatures. It models ordered row-buffer behavior and synthesizes legal layouts under a fixed target address mapping.

4 Tile-to-DRAM Linear Layouts

  • A tile-to-DRAM Linear Layout maps each layout unit to a row-buffer domain, a row within that domain, and a within-row position.The row-buffer domain is the hardware unit that independently holds one open row; in the evaluated organization, it is a tier-bank pair.Source: §4.1 · p. 5, §4 · p. 5

4.1 Tile-to-DRAM Linear Layout Object

  • Each layout is selected for a tile access signature combining tile configuration, operand properties, and the ordered tile-local access sequence.A representative execution supplies the normalized source sequence used for selection.Source: §4.1 · p. 5, §4 · p. 5

4.2 Ordered Row-Buffer Semantics and Row-Reuse Kernel Reduction

  • Row-buffer outcomes depend on request order and open-row state; in the Figure 5 projection, the layout turns three conventional conflicts into three row hits.An opening miss finds a closed row, while a conflict requires a precharge and activation; a hit requires neither.Source: §4.2 · p. 5, Figure 5 · p. 6
  • For a fixed domain, same-domain predecessor differences in K are hits, reducing selection to a maximum-weight c-dimensional subspace under sequence-recurrence, mapping-consistency, and binding-legality conditions.The source-order hit count is the sum of difference weights over K.Source: §4.2 · pp. 6–7
  • TileDRAM reverses the control variable of prior mapping synthesis: it selects data placement while keeping the controller mapping fixed, making placement a compiler-visible decision.Taking differences over tile-local coordinates lets one solution serve tiles sharing a signature.Source: §4.2 · p. 7

4.3 Layout Synthesis and Selection

  • The synthesis process enumerates row-buffer-domain sublayouts, constructs and completes row-reuse kernels, realizes executable layouts, and ranks retained candidates by conflicts, domain coverage, and address-map complexity.Legality filtering includes injectivity, mapping consistency, alignment, capacity, aperture bounds, and allocation-policy constraints.Source: §4.3 · p. 7

5 Compiler and Runtime Realization

TileDRAM realizes target-specific layouts through PLAN–BIND–APPLY, with compiler and runtime stages sharing a template and address transformation. Its compiler lowering and simulator replay apply that transformation before fixed address mapping and row-buffer service.

5 Compiler and Runtime Realization

  • PLAN selects and realizes a target-specific layout template, while compiler-side and runtime BIND connect it to resident tiles for APPLY.The compiler marks accesses for APPLY, and runtime BIND constructs a descriptor for the resident tile.Source: §5 · p. 7

5.1 Semantic Extraction and Offline Template Construction

  • The compiler derives tile configuration and operand properties, combines them with a recorded active-lane sequence into a tile access signature, and publishes a reusable template for each admitted signature and target contract.The programmer-facing annotation supplies problem, tile, execution, and operand properties; the kernel remains ordinary Triton/Gluon code.Source: §5.1 · p. 7

5.2 Template Binding and Address Lowering

  • Compiler-side BIND looks up the template and records binding metadata; runtime BIND combines it with the tile ID, source interval, and DRAM region to form a tile descriptor.APPLY transforms each active lane’s source address through extraction, layout transformation, and address reconstruction using ordinary integer and pointer operations.Source: §5.2 · p. 8, Figure 6 · p. 8

5.3 Runtime Replay and Simulator Execution

  • Simulator replay applies the same address transformation to asynchronous-copy events, preserving their metadata and dependencies as Ramulator2’s fixed mapper and controller return service latency.The production compiler lowering and simulator-side transformer implement one common address function, checked against an independent reference evaluator.Source: §5.3 · p. 8

6 Evaluation

Across the evaluated MoE FFN cases, TileDRAM improves row-buffer locality and performance, and retains locality when retargeted across six 3D DRAM organizations. The evaluation also measures setup conditions and the costs of layout synthesis and address transformation.

6.1 Experimental Setup

  • The evaluation replays NVBit traces from Triton/Gluon tiled GEMM kernels through Accel-Sim/Ramulator2 while holding each baseline–TileDRAM pair’s logical trace and DRAM conditions fixed.The suite covers 108 MoE FFN cases across six models, up/gate and down projections, and eight tile shapes.Source: §6.1 · p. 9

6.2 RQ1: Row-Buffer Cost of Conventional DRAM Placement

  • Conventional placement yields a 75.7% request-weighted row-conflict rate across 108 cases, with rates varying by model family, projection type, and tile configuration.Average read latency ranges from 41.1 to 101.9 GPU cycles and closely tracks row-conflict rate (Pearson r=0.97).Source: Figure 7 · p. 9, §6.2 · p. 9

6.3 RQ2: Row-Buffer Locality and Performance

  • The request-weighted row-conflict rate falls from 75.7% to 15.2%, and speedup reaches a 1.77× geomean across 108 cases.Row-hit rate rises from 19.3% to 81.6%; speedup ranges from 1.10× to 2.95×.Source: §6.3 · pp. 9–10, Figure 8 · p. 10
  • Request-weighted read latency falls from 81.0 to 40.8 GPU cycles, while pooled shader memory-stall cycles fall from 2.086 to 0.962 billion.The performance range reflects baseline memory-stall share: cases using 64×128×64 achieve a 2.17× geomean speedup, versus 1.25× for 128×256×64.Source: §6.3 · p. 10

6.4 RQ3: Retargeting Across 3D DRAM Organizations

  • Across 36 paired replays, the pooled row-conflict rate falls from 84.3% to 17.6%, and speedup reaches a 2.20× geomean.Six full-trace FFN streams use the 64×64×32 tile across six organizations; six-stream geomean speedups range from 1.52× to 2.77×.Source: §6.4 · p. 10, Figure 9 · p. 11

6.5 RQ4: Cost of Layout Synthesis and Address Transformation

  • Address transformations add 0.17% geomean overhead relative to identity address generation across 108 hardware-measured configurations.Offline synthesis has a median per-case runtime of 349.0 s and median peak resident memory of 227.0 MB.Source: §6.5 · p. 11

7 Related Work

Prior work addresses 3D DRAM organization, compute-side layouts, and DRAM mapping or placement at other granularities. TileDRAM instead makes operand-tile placement a compiler-visible decision while keeping the controller mapping fixed.

3D DRAM Architectures and Accelerators

  • Prior 3D DRAM systems use global-memory or GPU-wide LLC abstractions, while TileDRAM focuses on compute-unit-local stacks and compiler-visible operand-tile placement.The related work describes global-memory routing through a package-level endpoint and LLC distribution that incurs NoC area and power.Source: §7 · p. 11

Compute-Side Tile Layouts and Algebraic Layout Abstractions

  • GPU systems expose tiled computation and compute-side layout abstractions, which TileDRAM extends to DRAM placement coordinates.Linear Layouts represent compute-side mappings between hardware-resource and logical tensor coordinates over F2.Source: §7 · p. 11

Controller Mapping and Software-Directed DRAM Placement

  • Unlike controller-mapping approaches, TileDRAM keeps the controller mapping fixed and specializes placement by recurring ordered tile-local access behavior.Its placements use ordinary address arithmetic, allowing many tile access signatures to coexist without controller or ISA changes.Source: §7 · p. 11

8 Conclusion and Future Work

TileDRAM makes DRAM placement a compiler-controlled tile layout decision under a fixed controller mapping, using recurring tile-local access behavior to select reusable placements. The conclusion reports improvements across MoE FFN cases and six organizations; the appendix formalizes the layout semantics under a fixed target contract.

8 Conclusion and Future Work

  • TileDRAM extends GPU Linear Layouts to reusable tile-to-DRAM placements selected per tile access signature and realized through ordinary address arithmetic.Across 108 MoE FFN cases, row-conflict rate falls from 75.7% to 15.2% with 1.77× geomean speedup; six-organization retargeting yields 2.20× geomean speedup.Source: §8 · p. 12
  • The evaluation is limited to tiled GEMM within one CU–stack pair, using trace-driven Accel-Sim/Ramulator2 replay; physical 3D DRAM validation remains future work.The authors also identify attention and convolution, and coordination across multiple CU–stack pairs, as future extensions.Source: §8 · p. 12

A Tile-to-DRAM Layout Details

  • The appendix formalizes tile access, layout semantics, resident tiles and DRAM regions, ordered row-buffer trajectories, and reconstruction under a fixed target contract.It treats the access quantum, request-formation policy, target mapping, selection parameters, and placement constraints as standing assumptions.Source: A Tile-to-DRAM Layout Details · p. 15

A.1 Tile-to-DRAM Semantics

This section defines tile-to-DRAM layouts as reusable linear maps with certified physical realizations that preserve ordered row-buffer behavior, and develops a conflict-first procedure for selecting executable layouts via weighted row reuse. It establishes exact selection relative to a finite certified catalog and specifies when source-order semantics and the fixed-domain kernel reduction apply.

A.1 Tile-to-DRAM Semantics

  • A reusable semantic layout maps a fixed ordered tile-access signature to DRAM coordinates, and mapping-consistent placements preserve its source-order row-buffer events.The semantics separate intrinsic row-buffer behavior from physical address labels, subject to the stated mapping-consistency and request-formation conditions.Source: §A.1 · p. 16

A.2 Source-Sequence Domain Coverage

  • After conflicts are minimized, source-sequence domain coverage counts distinct exercised domains and favors the layout with the largest count among tied candidates.Under the closed-domain initial state, this count also equals the opening-miss count.Source: §A.2 · p. 17

A.3 Legal Placement and Allocation Invariance

  • Legal placements require unique in-bounds addresses and mapping consistency; certified base shifts by the target-map period preserve legality when ranges remain in the CU-local aperture.Simultaneously resident tiles must also occupy disjoint reserved regions.Source: §A.3 · p. 17

A.4 Projected Row-Reuse Kernel Reduction

  • For a fixed domain block, maximizing the weighted same-domain predecessor-difference kernel is equivalent to minimizing source-order row conflicts.Opening misses and domain coverage are fixed by the domain block; only the hit/conflict split varies with the row block.Source: §A.4 · pp. 17–18

A.5 Fixed-Domain Solver Interface

  • TileDRAM optimizes weighted same-domain predecessor differences with Rotaru et al. [43] exact or greedy subspace routes; the exact route yields a fixed-domain optimum, while the greedy route is heuristic.Reduced-coordinate solutions are lifted to feasible fixed-domain subspaces while preserving objective values.Source: §A.5 · p. 18

A.6 Kernel-to-Layout Reconstruction

  • A selected kernel determines a row block by basis extension, and residual-address rows complete the semantic map to full column rank.For wider target spaces, retained completions must also satisfy target-embedding and sparsity constraints.Source: §A.6 · pp. 18–19

A.7 Certified-Catalog Selection

  • The catalog selector ranks candidates lexicographically by minimizing source-order conflicts, maximizing source-sequence domain coverage, then minimizing executable-map nonzero-term complexity.Candidates are generated and retained through target-realization, mapping-consistency, and binding-legality checks.Source: §A.7 · p. 19
  • For a nonempty certified catalog, exhaustive comparison is lexicographically exact; each kernel is optimal for its fixed-domain problem with exact routing, or feasible with greedy routing.The exact-route result applies on the complete positive-weight support; catalog exactness is relative to the emitted candidates.Source: §A.7 · p. 19

A.8 Applicability Conditions

  • Recurring ordered accesses, linear live-coordinate representation, and certified target realization justify the row-buffer semantics and fixed-domain kernel reduction.The conditions require consistent ordered request formation, injective live-unit coordinates, and affine domain and row labels under the fixed mapping; tiled GEMM is the evaluated instance. For an emitted kernel, captured same-domain predecessor-difference weight equals source-order row reuse.Source: §A.8 · p. 19

B Controller-Side DRAM Address Mapping: Configuration Feasibility, Construction Scale, and Performance

Controller-side mappings apply one global interpretation to every request, while the section evaluates fixed mappings and full-trace synthesis for configuration feasibility and replay performance.

B Controller-Side DRAM Address Mapping: Configuration Feasibility, Construction Scale, and Performance

  • Controller-side address mapping applies one global interpretation to every request, while TileDRAM selects a layout for each tile access signature.The evaluated fixed global hashes apply the same static bit-mixing rule regardless of operand properties or tile signature.Source: B Controller-Side DRAM Address… · p. 19
  • The fixed-hash replay uses a Llama 4 up/gate projection with tile (64, 128, 64) under the t8/b16/row13/col3 organization and open-row FR-FCFS.The conventional branch uses the identity RoBaTiCoCh mapping.Source: Table 2 · p. 20

B.1 Fixed Global Mappings: Closure and Replay Feasibility

Fixed global recipes apply one frozen address-field rule to both operand streams; the best replay-complete recipes make only negligible changes in the evaluated case.

B.1 Fixed Global Mappings: Closure and Replay Feasibility

  • The fixed-hash family includes row-to-bank, column-to-bank, bank-to-row, and column-to-row XOR mappings, compound row/bank swizzling, and a MOP4CLXOR-styled field swizzle.Each recipe applies one frozen rule to both operand access sequences.Source: §B.1 · p. 20
  • The best evaluated replay-complete fixed global recipes reduce conflicts by only 0.0892% and improve performance by 0.0144%.The replay evaluates a Llama 4 up/gate projection with tile (64, 128, 64), using the refresh-on t8/b16/row13/col3 organization and cycle-level Ramulator2 results relative to identity.Source: §B.1 · p. 20

B.2 Cost of Rotaru-Style Exhaustive Full-Trace Synthesis

Exhaustive full-trace synthesis scales prohibitively because each outer mapping candidate requires trace-dependent construction. The compiler interface exposes tiled-kernel semantics while retaining ordinary Triton/Gluon code.

B.2 Cost of Rotaru-Style Exhaustive Full-Trace Synthesis

  • Each outer mapping candidate changes request-domain membership, same-domain predecessors, and the resulting difference histogram, requiring reconstruction of a separate fixed-domain solver instance.The full-trace approach specializes on the workload trace, while TileDRAM keeps the controller mapping fixed and reuses placement selected per recurring tile access signature.Source: §B.2 · p. 20
  • Approximately 8.31 × 10^14 candidate-copy-event evaluations are required for the evaluated Llama 4 full trace before fixed-domain solves or physical mapping evaluation.The trace contains 7,864,320 active-lane copy events, and the outer search has 105,662,774 candidates before rank filtering and row-space deduplication.Source: §B.2 · p. 20
  • At 10 billion candidate-copy-event evaluations per second, outer construction takes approximately 23.1 hours; at 1 billion per second, it takes approximately 9.6 days.These estimates exclude fixed-domain solves, map completion, and physical address-mapping evaluation.Source: §B.2 · p. 20

C Compiler Interface

  • The programmer-facing annotation exposes tiled-kernel semantics to TileDRAM, while the kernel body remains ordinary Triton/Gluon code with tiled loads and matrix multiplication.The listing is an excerpt from the compiler path; its output-store path is omitted for brevity.Source: C Compiler Interface · p. 21

D Artifact and Evaluation Configuration

The evaluation combines full-trace MoE FFN replay, retargeting across 3D DRAM organizations, and hardware calibration of address-transformation cost. Its artifact documents the configuration and checks compiler–simulator address equivalence.

D Artifact and Evaluation Configuration

  • Across 108 validated cases, the reported transformation-versus-identity overhead was 0.17% geomean, below 0.01% median, 0.57% at the 95th percentile, and 4.93% maximum.Source: D Artifact and Evaluation Configuration · p. 22
  • 349.0 s median per-case synthesis time and 227.0 MB median peak memory accompany an artifact that pins the software and run configuration and validates address equivalence with zero mismatches.The artifact’s equivalence report covers all 108 main-suite configurations and both identity and selected-layout branches.Source: D Artifact and Evaluation Configuration · pp. 21–22
  • Each baseline–TileDRAM pair holds the access sequence, target organization, initial closed-row state, and refresh/precharge policy fixed while varying DRAM placement.The replay protocol also fixes kernel and tile shapes, execution staging, source trace, and DRAM address mapping.Source: D Artifact and Evaluation Configuration · p. 22
  • The primary suite comprises 108 full-trace FFN cases across six model families, collected from Triton/Gluon tiled GEMM kernels and replayed with Accel-Sim/Ramulator2.The platform models GPU execution context with Accel-Sim and the 3D DRAM backend with Ramulator2.Source: D Artifact and Evaluation Configuration · p. 22
  • The evaluation defines request-weighted row-conflict rate from pooled conflict and row-event counts and reports per-case geometric-mean speedup separately from organization-level pooled-cycle speedup.Replay records row hits, misses, conflicts, latency, queue wait, memory-stall cycles, and cycles.Source: D Artifact and Evaluation Configuration · pp. 22–23
  • 75.7% baseline and 15.2% TileDRAM row-conflict rates yield an 80.0% pooled reduction and 1.77× geomean speedup across 108 MoE cases; 36 retargeted cases yield 79.2% pooled reduction and 2.20× geomean speedup.Across the same 108 cases, shader-memory stalls fall by 53.9% and request-weighted Ramulator2 read latency by 49.6%.Source: D Artifact and Evaluation Configuration · p. 23

Related papers

Loading 2610.23767v1…