PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory Profiling and Migration for Heterogeneous Memory Architectures

Marios Asiminakis, Polydoros Petrakis, Manolis Marazakis

arXiv:2609.10554v1cs.DCcs.PF

TL;DR

Heterogeneous HBM–DRAM systems need better tier-aware placement than manual tuning or generic OS heuristics provide. SHAMBLES combines kernel-integrated low-overhead profiling with transparent migration and pluggable policies for unmodified applications. Across HPC workloads, it preserves near-ideal performance while using substantially less HBM, with effective fast-tier selection depending on workload and problem size.

  • Problem

    HBM–DRAM systems make placement across fast and slow tiers difficult because manual tuning is costly and OS heuristics are not designed for heterogeneous-tier dynamics.

  • Method

    SHAMBLES combines kernel-integrated access profiling, a lightweight user-space runtime, and pluggable static or dynamic policies to migrate data transparently without application changes.

  • Results

    SHAMBLES retains near-ideal performance while reducing HBM usage: up to 93.75% of all-in-HBM performance for HPCG at 40% HBM and up to 99% for DGEMM at one third of its footprint.

  • Takeaways & Limitations

    Effective tiering requires workload- and size-aware fast-tier selection, while policy-driven migration offers a practical path to performance portability on tiered memory systems.

  • Takeaways & Limitations

    The prototype selects hot pages from sampled page faults without feedback on whether migration decisions improve application or system performance.

Abstract

from arXiv · show

Heterogeneous memory systems that combine high-bandwidth memory (HBM) with commodity DRAM can accelerate bandwidth-bound HPC workloads, but current page placement largely depends on manual tuning or OS heuristics not designed for multi-tier dynamics. We present SHAMBLES, a kernel-integrated framework that profiles application memory behavior at low overhead and migrates data across tiers without requiring application changes. SHAMBLES exposes a policy-agnostic interface and a lightweight user-space runtime with pluggable policies (e.g. recency and frequency based) as well as static placement for controlled studies. A logging mode provides reproducible timelines of allocations and migrations to aid analysis. We implement SHAMBLES on a commodity Linux system with HBM and DDR exposed as NUMA nodes and evaluate it with the HPCG, DGEMM benchmarks and Himeno stencil mini-app. Our design and methodology show how transparent, policy-driven migration can respond to changing access locality and concentrate hot data in HBM without developer intervention, offering a practical path to performance portability on tiered memory. Results from HPCG show that we can maintain up to 93.75% of the all-in-HBM baseline performance, while keeping only 40% of the problem size in the HBM. DGEMM experiments show that dynamic policies in SHAMBLES sustain up to 99% of the all-in-HBM performance, while keeping only one third of the DGEMM matrix footprint in HBM. For Himeno, SHAMBLES shows that fast-tier selection must be both workload-aware and size-aware: with a 50% fast-tier budget, it can outperform fixed all-in-HBM and all-in-DDR placements for the L size, while the XL size shifts back toward HBM.

1 INTRODUCTION

Heterogeneous HBM–DRAM systems offer performance and capacity benefits, but tier-aware placement remains difficult under manual tuning and generic OS heuristics. SHAMBLES addresses this with transparent profiling, migration, and pluggable policies for unmodified HPC applications.

  • Motivation: HBM–DRAM architectures provide fast tiers for bandwidth-sensitive workloads while retaining larger, slower memory pools, but placement remains challenging.
  • Motivation: Manual placement requires extensive profiling and tuning, while AutoNUMA primarily targets cross-node balancing rather than heterogeneous-tier optimization.
  • Contribution: SHAMBLES integrates low-overhead profiling and transparent tier-aware migration through a kernel framework and lightweight user-space runtime.Its policy-agnostic design supports configurable placement without specialized hardware or application changes.
  • Evaluation: SHAMBLES evaluates policy-driven placement on HPC workloads and reports near-peak performance while reducing HBM footprint by 50–66.6%.The evaluation uses an Intel Xeon Max HBM+DDR platform and practical baselines.

2 RELATED WORK

Prior tiering systems address different hardware, deployment, and workload settings, often with fixed policies. SHAMBLES instead targets HBM+DDR HPC nodes with pluggable policies and low-overhead, policy-driven experimentation.

  • Positioning: SHAMBLES targets HBM+DDR HPC nodes with low-overhead sampling, pluggable policies, and support for both static and online placement.
  • Comparison: DAMON and DAMOS provide low-overhead region sampling and user-defined actions, while SHAMBLES specializes similar policy-driven mechanisms for heterogeneous HPC memory.
  • Scope: SHAMBLES operates within a single node and does not address memory pooling, virtualization-aware remapping, or broader deployment models.
  • Comparison: AutoNUMA performs transparent fault-driven migration across DRAM-only NUMA nodes, whereas SHAMBLES targets HBM+DDR hierarchies and richer tiering policies.
  • Comparison: Existing systems commonly target DRAM with NVM, CXL, remote, or virtualized memory and are therefore not like-for-like baselines for SHAMBLES.

3 DESIGN AND IMPLEMENTATION OVERVIEW

SHAMBLES separates kernel sampling, allocator integration, and user-space policy logic into modular components. The kernel samples accesses at low overhead while plugins consume this information to drive profiling, migration, and logging.

  • Architecture: SHAMBLES combines kernel patches, a modified jemalloc allocator, and dynamically loaded plugins that implement profiling and migration functionality.
  • Architecture: The modular design allows policies and architecture ports to change without modifying or recompiling the other user-space components.
  • Kernel component: Kernel instrumentation samples accesses in a reserved virtual-address band, keeping ordinary process mappings separate from profiling activity.
  • Kernel–user interface: A lightweight kernel–user interface streams sampled addresses and triggering instruction PCs through debugfs, leaving the kernel policy-agnostic.
  • Portability: The implementation supports four-level page tables, 4KB base pages, and huge pages, but not five-level x86 tables or 16KB and 64KB ARM base pages.
  • Runtime: The LD_PRELOAD jemalloc allocator enables SHAMBLES without application recompilation and invokes plugins on allocation, reallocation, and deallocation events.

4 IMPLEMENTED POLICIES

SHAMBLES supports static and dynamic chunk placement policies configured through a common NUMA-based interface. Dynamic policies adapt placement using recency or access frequency, while tunable budgets constrain fast-tier capacity.

  • Configuration: Policies treat selected NUMA nodes as fast and the remainder as slow, with node masks and allocation thresholds configurable through environment variables.
  • Policy types: Static placement never migrates allocations, whereas LRU and Window dynamically adapt chunk locations to observed access patterns during execution.
  • Configuration: Dynamic policies can cap fast-tier capacity and divide it into configurable chunks, allowing experiments that intentionally restrict the available fast memory.
  • Migration: Initial placement and migration use mbind with MPOL_BIND and MPOL_MF_MOVE, while processing each sample has constant time complexity.
  • Static placement: The static-fractional plugin specifies the desired percentage of each allocation placed in fast versus slow memory without dynamic migration.
  • LRU: LRU ranks chunks by recency, moving recently accessed chunks toward fast memory and less recent chunks toward slow memory.
  • Window: Window ranks chunks by access frequency within a configurable sliding window, placing frequently accessed chunks in faster tiers.

5 EVALUATION METHODOLOGY AND RESULTS

SHAMBLES is evaluated on an HBM+DDR Xeon Max system using HPCG, DGEMM, and Himeno, comparing dynamic migration with static and hardware-placement baselines. Results show near-HBM performance with reduced HBM usage, but effective tier selection depends on sampling, chunking, workload, and problem size.

  • HPCG: HPCG performance degrades with coarse chunking and high sampling: two chunks fall below 36 GF/s at 10 Hz and at or below 27 GF/s at 100 Hz.For 96 ranks, increasing sampling from 10 Hz to 100 Hz can raise migration from 20.6 to 364 GiB while reducing hit rate below 80% in one configuration.
  • HPCG: 93.75% of the best HBM baseline is reached for HPCG at 96 ranks while using 40% of the problem size in HBM.At 48 ranks, SHAMBLES reaches 92.13%; sufficiently fine chunks and low sampling rates perform best.
  • DGEMM: 99.0% of the all-in-HBM DGEMM baseline is achieved at 10 Hz while using only one third of HBM capacity.The best dynamic policy reaches 763.6 GF/s versus 770.9 GF/s all-in-HBM and migrates 1 GiB over 80 iterations.
  • Himeno: 43% higher performance than all-in-DDR is obtained by the best selective Himeno placement, reaching 32.7 GF/s, while the worst layout falls to 17.0 GF/s.The result comes from evaluating 148 static placements, showing that selective HBM use helps only with suitable placement.
  • Himeno: Himeno’s preferred fast tier changes with problem size: L benefits from DDR-as-fast placement, whereas XL favors HBM-as-fast placement.For XL, HBM-fast dynamic placement reaches 23.4 GF/s, or 96.3% of all-in-HBM, while the best static layout reaches 25.9 GF/s; cache mode drops to 11.8 GF/s.
  • Sampling overhead: 0.08% is the maximum sampling overhead at SHAMBLES’s default 10–100 Hz rates for DGEMM, while overhead remains below 0.6% up to 1,000 Hz.The reported overhead is zero for applications that do not preload SHAMBLES; higher rates are mainly useful in full-system simulators.

6 FUTURE EXTENSIONS

Future work focuses on making SHAMBLES more feedback-driven, automatically tuned, and better matched to platform-specific latency and bandwidth characteristics.

  • 6 FUTURE EXTENSIONS: Hardware performance counters could guide migration policies using IPC, memory stalls, or bandwidth utilization feedback.The current prototype selects hot pages from sampled page-fault activity without learning whether migrations improve application performance.
  • 6 FUTURE EXTENSIONS: Automated tuning could optimize policy selection, sampling rate, and environment variables for each platform and application.This may require microbenchmarks and on-the-fly application profiling to extract machine-dependent characteristics.
  • 6 FUTURE EXTENSIONS: Advanced policies that model low-latency and high-bandwidth tiers could address Himeno’s tier-selection sensitivity more effectively than fixed fast and slow labels.The proposed direction replaces the current generic tier ordering with models of actual platform characteristics.
  • 6 FUTURE EXTENSIONS: Alternative samplers such as Intel PEBS, ARM SPE, or DAMON could replace the current sampling method, but portability or overhead may worsen.These approaches trade broader hardware assistance or established facilities against implementation cost and runtime overhead.

7 CONCLUSIONS

SHAMBLES provides transparent, policy-driven migration for heterogeneous HBM+DDR systems, reducing fast-memory footprint while retaining most idealized-baseline performance. Its results show that effective placement depends on workload, problem size, and the platform’s latency–bandwidth asymmetry.

  • 7 CONCLUSIONS: SHAMBLES combines kernel page-fault sampling and a jemalloc-based runtime to track hot regions, migrate them across tiers, and log placement activity for unmodified applications.Its policy-agnostic interface separates profiling and migration mechanisms from configurable user-space policy logic.
  • 7 CONCLUSIONS: Up to 93.75% of all-in-HBM HPCG performance is retained with only 40% of the problem size in HBM, while DGEMM reaches up to 99% with one third of its footprint.These results show substantial HBM-footprint reduction across memory-bound and compute-bound workloads.
  • 7 CONCLUSIONS: Himeno’s L workload gains 12–27% over all-in-DDR when DDR is treated as fast, whereas XL shifts toward HBM-as-fast placement.The results reflect both latency–bandwidth trade-offs and platform asymmetry rather than a single fixed memory hierarchy.
  • 7 CONCLUSIONS: Cache mode suffers a severe performance hit when the memory footprint exceeds available HBM capacity.This limitation appears when comparing Himeno problem sizes and constrains cache-based tiering for larger working sets.
  • 7 CONCLUSIONS: SHAMBLES offers a practical path to performance portability by supporting transparent migration, configurable placements, and rich traces for analysis.Future extensions include hardware-counter feedback and policies for latency-sensitive codes and multi-tenant settings.
Loading 2609.10554v1…