PaperlayerView exact revision on arXiv

Source-linked AI summary

TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory

Hasan Al Maruf, Hao Wang, Abhishek Dhanotia, Johannes Weiner, Niket Agarwal, Pallab Bhattacharya, Chris Petersen, Mosharaf Chowdhury, Shobhit Kanaujia, Prakash Chauhan

arXiv:2206.02878v2cs.DCcs.OS

TL;DR

Rising memory demand and cost make CXL-enabled tiered memory attractive, but heterogeneous latency and inefficient placement create a memory-management challenge. The paper characterizes production workloads and proposes transparent page placement, with TPP reaching near-all-local-memory performance and outperforming Linux and existing tiering mechanisms.

  • Problem

    Tiered memory can provide cheaper capacity, but higher-latency technologies and inefficient page placement can significantly degrade performance.

  • Method

    The paper uses Chameleon to characterize production memory behavior and proposes TPP, an OS-level transparent mechanism that demotes cold pages and promotes hot pages across CXL-enabled tiers.

  • Results

    TPP performs within <1% of an ideal all-local-memory system, improves default Linux by up to 18%, and outperforms NUMA Balancing and AutoTiering by 5–17%.

  • Takeaways & Limitations

    TPP provides a globally deployable, application-transparent approach to managing CXL-enabled tiered memory, with most patches merged in Linux v5.18.

  • Takeaways & Limitations

    In multi-tenant clouds with differently prioritized applications and QoS requirements, TPP may provide sub-optimal performance without QoS-aware memory management.

Abstract

from arXiv · show

The increasing demand for memory in hyperscale applications has led to memory becoming a large portion of the overall datacenter spend. The emergence of coherent interfaces like CXL enables main memory expansion and offers an efficient solution to this problem. In such systems, the main memory can constitute different memory technologies with varied characteristics. In this paper, we characterize memory usage patterns of a wide range of datacenter applications across the server fleet of Meta. We, therefore, demonstrate the opportunities to offload colder pages to slower memory tiers for these applications. Without efficient memory management, however, such systems can significantly degrade performance. We propose a novel OS-level application-transparent page placement mechanism (TPP) for CXL-enabled memory. TPP employs a lightweight mechanism to identify and place hot/cold pages to appropriate memory tiers. It enables a proactive page demotion from local memory to CXL-Memory. This technique ensures a memory headroom for new page allocations that are often related to request processing and tend to be short-lived and hot. At the same time, TPP can promptly promote performance-critical hot pages trapped in the slow CXL-Memory to the fast local memory, while minimizing both sampling overhead and unnecessary migrations. TPP works transparently without any application-specific knowledge and can be deployed globally as a kernel release. We evaluate TPP in the production server fleet with early samples of new x86 CPUs with CXL 1.1 support. TPP makes a tiered memory system performant as an ideal baseline (<1% gap) that has all the memory in the local tier. It is 18% better than today's Linux, and 5-17% better than existing solutions including NUMA Balancing and AutoTiering. Most of the TPP patches have been merged in the Linux v5.18 release.

1 Introduction

Memory demand and cost motivate CXL-enabled tiered memory, but effective placement is needed because memory technologies differ in latency and application performance depends on fast-memory service. TPP combines transparent page demotion, promotion, and allocation management, achieving near-local-memory performance while outperforming existing approaches.

  • Motivation: Memory demand and rising DRAM costs make tiered memory attractive, but inefficient page placement can significantly degrade performance.Non-DRAM tiers offer cheaper capacity but have higher latency, while application performance depends on the fraction of memory served from fast memory.
  • Workload characterization: Chameleon profiling finds meaningful warm and cold working-set portions, hotter anon memory, colder file-backed memory, and stable access patterns.These observations support moving cold pages to slower memory and making placement decisions in kernel space.
  • TPP design: TPP transparently places hot pages in fast memory and cold pages in slow memory through reclamation, decoupled allocation, and reactive promotion.It also supports optional page-type-aware allocation, favoring sensitive anon pages in fast memory and file caches in slow memory.
  • Evaluation: 18% better than default Linux and 5–17% better than NUMA Balancing and AutoTiering, TPP makes tiered memory perform like an ideal all-local-memory system.The evaluation uses diverse production workloads on a CXL 1.1-capable system.

2 Motivation

Datacenter memory demand and cost are increasing while homogeneous server designs restrict memory technology, capacity, and bandwidth choices. CXL relaxes these constraints, and workload characterization indicates that cold memory can be moved to a cheaper slow tier to expand effective capacity.

  • Increased Memory Demand in Datacenter Applications: Memory is becoming a larger share of rack power and total cost of ownership as in-memory computation and memory demand grow.The trend is described across newer CPU and DRAM generations.
  • Scaling Challenges in Homogeneous Server Designs: Homogeneous servers restrict memory technology mixing, fine-grained capacity sizing, and independent bandwidth scaling.These constraints can force excess capacity and strand compute, network, or memory resources.
  • CXL for Designing Tiered-Memory Systems: CXL provides coherent, byte-addressable, cache-line-granular memory expansion with about 50–100 nanoseconds of extra latency over normal DRAM.Its NUMA-like behavior and main-memory-like access semantics make CXL-Memory suitable as a slow tier.
  • Scope of CXL-Based Tiered-Memory Systems: 55–80% of allocated memory remains idle within any two-minute interval across four characterized production applications.Moving this cold memory to a slower tier can create space for hot pages and support smaller fast-memory tiers.
  • Lightweight Characterization of Datacenter Applications: Chameleon addresses limitations of access-bit characterization by providing lightweight user-space working-set analysis without requiring kernel modifications.Existing IPT-based tools provide coarse access information and cannot track allocation and deallocation behavior adequately.

3 Characterizing Datacenter Applications

Chameleon characterizes memory access behavior in production workloads, revealing substantial cold memory and page-type-dependent temperature patterns that motivate tiered placement.

  • Characterization Method: Chameleon is a lightweight user-space tool for characterizing application memory access behavior without modifying the underlying kernel.It uses a Collector and Worker to gather sampled memory-access information and generate insights.
  • Production Workloads: The study covers long-running, memory-bound production workloads across four diverse service domains, including Web, Cache, Data Warehouse, and Ads.These workloads serve live traffic and represent a significant portion of the server fleet.
  • Memory Temperature: 95–98% of system memory is allocated by Web, Cache, and Ads, but only 22–80% is used within a two-minute interval on average.This indicates substantial portions of allocated memory remain cold over short intervals.
  • Memory Temperature: Only 20% of accessed memory is hot within a two-minute interval for the Data Warehouse workload, despite operations spanning terabytes and nearly all available server memory.The workload therefore also contains substantial memory suitable for colder-tier placement.
  • Page Types: Anonymous pages are generally hotter than file pages, with Web anonymous pages at 35–60% hot versus 3–14% for files within two minutes.Data Warehouse and Ads similarly use anonymous pages for computation while file pages hold intermediate data and remain comparatively cold.
  • Re-access Behavior: Cold-page re-access times vary by workload: nearly 80% of Web pages are re-accessed within ten minutes, while only 20% of Data Warehouse hot file pages were previously accessed.Placement should therefore account for both page temperature and workload-specific re-access behavior.

4 Design Principles of TPP

TPP is designed as a kernel-level, transparent mechanism that places pages across local and CXL memory according to temperature while limiting overhead.

  • Implementation Layer: TPP is implemented in the kernel because user-space placement requires context switches, history management, and additional memory overhead.The authors expect the kernel implementation to be less complex and more performant.
  • Page Temperature Detection: TPP combines lightweight LRU-based cold-page detection in local memory with minor-fault-based detection for pages in CXL-Memory.CXL-Memory is expected to hold warm and cold pages, keeping temperature-detection overhead low.
  • Page Temperature Detection: TPP uses reclamation and NUMA Balancing together to detect most hot CXL-Memory pages at virtually zero overhead.The experiments report that kernel LRUs work well for on-the-fly profiling without a more sophisticated detector.
  • CXL-Memory Abstraction: TPP treats CXL-Memory as directly accessible memory rather than swap space, preserving CXL’s cache-line-granular load/store semantics.TMO’s feedback-driven reclamation can complement TPP, but TMO operates above TPP and pushes for memory reclamation.

5 TPP for CXL-Memory

TPP transparently places hotter pages in local memory and colder pages in CXL-Memory through lightweight demotion, decoupled allocation and reclamation, and activity-aware promotion. Its design preserves local-memory headroom while limiting unnecessary migration traffic.

  • TPP design overview: TPP places hotter pages in local memory and moves colder pages to CXL-Memory through four coordinated design areas.These areas are lightweight demotion, decoupled allocation and reclamation, hot-page promotion, and page-type-aware allocation.
  • Lightweight demotion: TPP asynchronously migrates reclamation candidates to CXL-Memory instead of swapping them, using Linux’s default LRU mechanism to select demotion candidates.Migration to a NUMA node is described as orders of magnitude faster than swapping.
  • Decoupled allocation and reclamation: TPP decouples allocation from reclamation so background demotion continues beyond the allocation threshold, maintaining free-page headroom on the local node.The demotion watermark is set above the allocation and low watermarks, allowing new allocations while reclamation continues.
  • Activity-aware promotion: Activity-aware promotion reduces unnecessary traffic by avoiding ping-pong migrations caused when infrequently accessed pages are promoted and then rapidly demoted.The issue arises because default NUMA Balancing promotes pages immediately without checking their active state.
  • Activity-aware promotion: TPP limits promotion candidates to faulted pages on active LRU lists, while first activating inactive pages to add hysteresis before promotion.This avoids immediately promoting infrequently accessed pages that may soon be demoted again.
  • Design consequence: The policy supports applications with infrequent cache accesses on systems combining limited local memory with larger, cheaper CXL-Memory.The stated goal is maintaining performance while using a small local-memory tier.

6 Evaluation

Across production workloads and configurations, TPP keeps effective hot memory close to the local tier and substantially reduces performance loss relative to default Linux and competing tiering systems. Its component mechanisms improve promotion success, local traffic, and throughput under memory pressure.

  • Default production environment: In the Web workload, TPP raises local-node service from 22% to 90% of memory accesses and limits throughput drop to 0.5%.Default Linux’s throughput drop is 16.5% in the same configuration.
  • Severe memory constraint: With local memory at 20% of working-set size, TPP achieves only a 0.5% throughput regression for Cache1.TPP serves 85% of total memory accesses from the local node after promoting hot anonymous pages and demoting less latency-sensitive file pages.
  • Severe memory constraint: For Cache2 in the 1:4 configuration, TPP reduces throughput loss to 5% despite 41% of memory traffic coming from CXL-Memory.Default Linux experiences an 18% throughput loss in this configuration.
  • Impact of TPP components: Active LRU-based detection reduces promotion rate by 11×, improves promotion success by 48%, and increases throughput by 2.4%.It also reduces subsequently re-promoted demoted pages by 50%, while adding five minutes to peak local-traffic convergence.
  • Overall effectiveness: TPP keeps throughput close to the all-local-memory baseline, with throughput drops of only 0.2–2.5% across evaluated workloads.This reflects TPP’s ability to keep most effective hot pages on the local node.
  • Comparison with existing systems: TPP outperforms NUMA Balancing and AutoTiering under memory pressure, where failed promotion or fixed buffers leave substantial traffic on CXL-Memory.AutoTiering serves 70% of traffic from CXL-Memory in one comparison, while TPP’s throughput drop is 0.5%.

7 Discussion and Future Research

TPP is positioned as production-ready for initial CXL-enabled tiered-memory systems, while several research opportunities remain as technologies and deployment scenarios evolve.

  • TPP is production-ready for onboarding the first generation of CXL-enabled tiered-memory systems.
  • Tiered Memory for Multi-tenant Clouds: In multi-tenant clouds, TPP can enable tenants to share memory tiers, but differing application priorities and QoS requirements may produce sub-optimal performance.A QoS-aware memory management mechanism is identified as a potential remedy.
  • Allocation Policy for Memory Bandwidth Expansion: Bandwidth-bound applications may require placing bandwidth-heavy, latency-insensitive pages in CXL-Memory rather than only offloading cold pages.Identifying the ideal working-set fraction may require hardware support.
  • Allocation Policy for Memory Bandwidth Expansion: Transparent memory management for memory-bandwidth expansion remains future work.
  • Hardware Support for Effective Page Placement: Hardware-side caches, prefetchers, and data-movement support could further improve CXL-Memory latency and migration overheads.In the authors’ environment, steady-state migration bandwidth is 4–16 MB/s, or 1–4K pages/second.

8 Related Work

Related work spans heterogeneous-memory systems, hardware-assisted and application-guided placement, transparent temperature detection, swapping, and NVM-based tiering. TPP differs by using application-transparent placement while avoiding page-fault overheads and critical-path migration exchange.

  • Tiered Memory System: CXL provides an intermediate, DRAM-like low-latency memory tier and supports flexible heterogeneous server designs.
  • Page Placement for Tiered Memory: Prior page-placement systems use hardware assistance or application guidance, limiting scalability when they require hardware support or application redesign.
  • Page Placement for Tiered Memory: Application-transparent approaches profile physical or virtual addresses, but frequent TLB invalidations or interrupts can impose high overhead.The paper finds in-kernel LRU-based temperature detection sufficient for CXL-Memory.
  • Page Placement for Tiered Memory: In-memory swapping is ineffective for workloads with varied access frequencies because CXL-node accesses incur page faults and immediately return pages to main memory.CXL-Memory integrated into main memory avoids page-fault overhead for less frequently accessed pages.
  • Page Placement for Tiered Memory: Nimble’s page-exchange migration can worsen performance because demotion waits for promotion on the critical path.AutoTiering and Huang et al. use background demotion and optimized NUMA-balancing promotion.
  • Page Placement for Tiered Memory: The evaluation finds optimized promotion critical for memory-bound applications to maintain performance under memory pressure.
  • Disaggregated Memory: Network-based disaggregated-memory management is orthogonal to TPP because its latency characteristics differ substantially from CXL-Memory.The two approaches can be used together across CXL- and network-enabled memory tiers.

9 Conclusion

The paper combines lightweight memory characterization with transparent OS-level page placement for CXL-enabled tiered memory. Across diverse production workloads, TPP improves on default Linux and existing tiered-memory mechanisms.

  • Chameleon characterizes datacenter memory usage, while TPP provides application-transparent OS-level page placement without prior application access-pattern knowledge.
  • 18%: TPP improves application performance over default Linux, and 5–17% over NUMA Balancing and AutoTiering.
Loading 2206.02878v2…