PaperlayerView exact revision on arXiv

Source-linked AI summary

CMM-MP: A Novel Memory Pooling Architecture using Purpose-built Memory Device with CXL and Ethernet

Michael Wasef, Gongjin Sun, Edmund Au, Ranjitha Gopalakrishna, Elham Movahedian, Dongyang Li, Senthil Murugesapandian, Caroline Kahn, Shuyu Lyu, Brian Luu, Jason Scott, Namita Kumar, Mehdi Nik, Kwan Huen, Xuebin Yao, Amir Beygi, Luis Vitorio Cargnini, Ramdas Kachare

arXiv:2610.22716v1cs.AR

TL;DR

Memory-intensive workloads strain fixed server memory capacity and make independent scaling difficult. CMM-MP combines a CXL host interface with an Ethernet-connected memory pool, with device firmware handling pool discovery. Evaluations report modest Valkey+YCSB throughput degradation despite lower raw bandwidth and DiskANN throughput above disk-based storage.

  • Problem

    Memory-intensive workloads can exceed a single node’s memory limits, while power, space, and thermal constraints hinder independent memory scaling.

    Source: §I · p. 1

  • Method

    CMM-MP is a purpose-built device combining CXL and Ethernet, with device firmware autonomously discovering available pool devices and capacity.

    Source: §I · p. 1, §III · p. 3, E. ARM Subsystem · p. 7

  • Results

    With 10x less raw bandwidth (32 GB/s vs 304 GB/s), Valkey+YCSB throughput degradation stayed well below 30%; MP-R DiskANN throughput was about 3x disk-based storage.

    Source: C. Real World Application Use Cases · p. 10

  • Takeaways & Limitations

    The aggregated-pool architecture allows compute and memory to scale independently and may permit different cooling methods for each.

    Source: §V · p. 8

Abstract

from arXiv · show

CXL Memory Module - Memory Pooling (CMM-MP) introduces a novel memory pooling architecture to meet the unique requirements of emerging memory-intensive workloads. This industry-first, purpose-built memory-pooling device combines the benefits of Compute Express Link (CXL) with the scalability and cost advantages of Ethernet. Multiple CMM-MP devices together provide a flexible, composable, and disaggregated memory expansion solution. The CMM-MP enables 100s of TB capacity for GPU/CPU applications without modifications. The memory pool is exposed as a large-capacity logical aggregate memory, with all pool discovery, composition, and backend network datapath tasks fully offloaded to the device. CMM-MP supports both memory pooling and sharing usage modes. It provides an application latency of 2.7 us and up to 23 GB/s of memory throughput. Our evaluations demonstrate that CMM-MP significantly outperforms RDMA-based solutions in latency and throughput. We also evaluated real-world use cases of Valkey+YCSB and DiskANN using CMM-MP.

I. INTRODUCTION

Memory-intensive workloads expose limits in fixed, compute-centric servers: capacity can be scarce or underused, while physical constraints make memory difficult to scale independently. CMM-MP combines CXL host access with an Ethernet backend to present a composed memory pool to hosts.

I. INTRODUCTION

  • Memory-intensive applications can exceed one node’s capacity, while fixed server memory and power, space, and thermal constraints complicate independent scaling.Operators also face difficulty sharing idle or lightly loaded memory across hosts without compromising performance.Source: §I · p. 1
  • Memory disaggregation separates memory from compute, and pooling combines devices into a shared addressable resource that can adapt to workload variation.The passage links pooling to reduced under-utilization, over-provisioning, TCO, and power consumption.Source: §I · p. 1
  • CMM-MP combines a CXL load/store host interface with Ethernet as the transport fabric, exposing the pool as a logical system-memory range without application changes.Its device architecture includes a CXL Endpoint, local DRAM, memory-pooling logic, and network ports.Source: §I · pp. 1–2
  • Multiple CMM-MP devices connect through a dedicated Ethernet switch, and device firmware discovers pool capacity while each host accesses the pool through its local device.A host-management utility manages memory-allocation policy.Source: §I · p. 2

II. BACKGROUND AND RELATED WORK

Existing approaches trade software complexity, reach, scalability, cost, and sharing capability differently. The related work contrasts RDMA, CXL-based designs, and other interconnect approaches against these constraints.

II. BACKGROUND AND RELATED WORK

  • RDMA can scale memory resources but requires explicit remote-memory calls and intrusive software changes –.Prior work also reports improved latency and throughput in key-value systems and performance gains from aggregating server DRAM.Source: §II · p. 2
  • CXL load/store semantics support unmodified applications,,, while dedicated CXL switches offer pooling but face limited scalability, higher cost, and deployment constraints,.Source: §II · p. 2
  • CXL MLD sharing provides transparent cache-line-granular access, but remains constrained by physical reach, switching latency, cost, and scalability.The later Multi-Headed Device concept removes the physical switch, while retaining a physical-reach limitation.Source: §II · p. 2
  • Hybrid CXL–RDMA designs seek both technologies’ benefits but retain RDMA overheads and an additional host software layer.Source: §II · p. 2

III. DEVICE ARCHITECTURE

CMM-MP is a purpose-built CXL Type 3 device that uses Ethernet for backend connectivity. Its device logic handles host requests and maps pool addresses to local or remote memory.

III. DEVICE ARCHITECTURE

  • CMM-MP integrates CXL and Ethernet in a purpose-built CXL Type 3 memory device with four dedicated logic subsystems.The subsystems are CXL, Ethernet, DDR, and ARM; the CXL and Ethernet subsystems direct requests toward local memory or the backend network, respectively.Source: §III · p. 3
  • The CXL subsystem identifies whether host requests target local or remote memory and forwards them to the DDR or Ethernet subsystem.Source: §III · p. 3
  • The ARM subsystem handles pool discovery, memory-map composition, and internal mapping-table programming alongside host traffic.Source: §III · p. 3

A. Memory Pool Address Resolution

  • The device normalizes host addresses, looks up the normalized address in PAMT, and uses the returned device ID and offset to access pool data locally or remotely.With 64 GB PAMT granularity, the passage states a maximum pool capacity of 256 TB.Source: A. Memory Pool Address Resolution · p. 3

B. CXL Subsystem

The CXL subsystem accepts host memory transactions, normalizes their addresses, and routes requests through pool-composition and response-handling logic. The mapping supports a consistent logical pool view across hosts.

B. CXL Subsystem

  • The CXL subsystem implements CXL 1.1/2.0 partially over PCIe Gen5 x8 and handles memory reads, writes, invalidation, and speculative reads.It maps received host addresses to pool-memory addresses.Source: B. CXL Subsystem · p. 3
  • Host addresses are normalized into the logical aggregate pool address so hosts mapping the pool at different system-memory locations share a consistent view.The CXL Request Manager performs this normalization using equation 1.Source: B. CXL Subsystem · p. 4
  • The Tag Table stores request-header fields under an internal tag so the response path can recover the original CXL request information.Source: B. CXL Subsystem · pp. 4–5
  • PAMT translates a normalized address into a target device ID and device-memory offset, and the request router serves matching device IDs locally or sends others to Ethernet.Source: B. CXL Subsystem · pp. 4–5
  • The response manager returns DRS with payload data for reads and NDR status and completion codes for writes and invalidations.Source: B. CXL Subsystem · p. 5

C. Ethernet Subsystem

The Ethernet subsystem transports memory requests and responses across the pool and exchanges control packets among devices. Its transport logic tracks packet sequence state and implements recovery mechanisms for reliable data movement.

C. Ethernet Subsystem

  • The Ethernet subsystem has transport, data-link, and physical layers, carrying remote memory data and firmware control packets across the pool.Its reliability functions include automatic retransmission, duplicate-packet handling, and packet-loss detection and recovery.Source: C. Ethernet Subsystem · p. 5
  • Transport-layer transmit logic tracks outstanding remote requests in RIFT entries and assigns per-device sequence numbers for response validation and loss or duplicate detection.A timed-out request is retransmitted; completed entries are cleared for reuse.Source: C. Ethernet Subsystem · p. 5, Figure 10 · p. 5, Figure 9 · p. 5
  • The receive sequence table uses a base sequence number, packet bitmap, and overflow counter to track ingress packet order and identify received or expected packets.A timeout can advance the base sequence number when packet loss leaves bitmap bits unset.Source: C. Ethernet Subsystem · pp. 5–6, Figure 11 · p. 6
  • ETH-TX encapsulates transport and control packets in Ethernet frames, while ETH-RX processes ingress frames; MAC addresses map target device IDs to destinations.The MIC module exchanges frames with MAC IP and handles clock-domain crossing; Ethernet MAC, PHY, and SERDES implement IEEE 802.3.Source: C. Ethernet Subsystem · p. 6, Figure 12 · p. 6, Figure 13 · p. 6

D. DDR Subsystem

The DDR subsystem arbitrates local and remote memory requests and stripes addresses across three DDR4 channels to support high-bandwidth, low-latency access.

D. DDR Subsystem

  • Three DDR4 channels, each with 16 GB of DRAM, provide memory access through a controller, clock-domain-crossing logic, and requester interfaces.The subsystem arbitrates concurrent reads and writes from CXL local-host and Ethernet remote-host requesters.Source: D. DDR Subsystem · p. 6, Figure 14 · p. 6
  • Address striping across the DDR channels maximizes throughput while preserving request-response association so data returns to the correct requester.The subsystem provides high-bandwidth, low-latency access to DDR memory.Source: D. DDR Subsystem · p. 6

E. ARM Subsystem

The ARM subsystem acts as the pool control plane, autonomously discovering devices, composing the memory map, and configuring the datapath. Discovery assigns device IDs from sorted chip identifiers and programs the global pool map.

E. ARM Subsystem

  • ARM firmware performs pool discovery, memory-map composition, and datapath configuration without host intervention.The subsystem includes an ARM core, CXL GP-CSR interface, and Ethernet packet interface.Source: E. ARM Subsystem · p. 7, Figure 15 · p. 7
  • Host software communicates with ARM firmware through CXL.io GP-CSRs, while firmware sends and receives Ethernet control frames through datapath-connected packet FIFOs.After discovery, firmware programs PAMT entries, device IDs, and MAC addresses.Source: E. ARM Subsystem · p. 7
  • During autonomous discovery, devices broadcast messages, sort unique responses by chip identifier, and use identifier rank to assign device IDs.The IDs and device capacities program the global pool map in PAMT before the pool enters runtime.Source: E. ARM Subsystem · p. 7, Figure 16 · p. 7

IV. HOST INTERFACE

CMM-MP exposes pool memory as CXL Type 3 NUMA memory for exclusive pooling, concurrent sharing, or both. The paper describes aggregated pools shared by hosts and distributed pools assembled from host-attached devices.

IV. HOST INTERFACE

  • CMM-MP exposes pool memory as CXL Type 3 NUMA memory for exclusive allocation, concurrent sharing, or both modes simultaneously.Shared ranges guarantee atomicity but not request ordering, so participating hosts must handle data coherency when needed.Source: §IV · p. 7, Figure 17 · p. 7

V. AGGREGATED MEMORY POOL

  • In an aggregated pool, hosts access memory through local CMM-MP devices connected to chassis devices over dedicated Ethernet switches.Segments may reside on different devices, allowing compute and memory to scale independently and potentially use different cooling methods.Source: §V · pp. 7–8, Figure 18 · p. 8, Figure 19 · p. 8

VI. DISTRIBUTED MEMORY POOL

  • A distributed pool combines host-attached CMM-MP memories, and hosts can be added or removed as needed.Compared with aggregation, this approach tightly couples compute and memory scaling while retaining scalability through added hosts.Source: §VI · p. 8, Figure 20 · p. 8

VII. PERFORMANCE EVALUATION

The evaluation uses a two-host CXL setup with a 288 GB pool, then characterizes CMM-MP latency, bandwidth, and stability against remote-memory baselines.

A. Experiment Setup

  • The evaluation used an Intel Granite Rapids development server with CXL support; its setup connected two host servers to four standalone CMM-MP devices, forming a 288 GB pool of six devices using a 32-port switch at 200 Gbps per port.Source: A. Experiment Setup · p. 8

B. Device Characterization

  • 800 ns of port-to-port round-trip delay is attributed to the Ethernet switch; local CMM-MP latency is close to GNR cross-socket latency.Latency measurements include local memory and remote memory with and without a switch.Source: Table II · p. 9, B. Device Characterization · p. 9
  • Table III reports highly consistent bandwidth across read/write mixes for remote memory accessed through the switch.The measurements use one device for MP-L and five remote devices for MP-R (WS).Source: Table III · p. 9, B. Device Characterization · p. 9
  • RDMA small transfers (64 B) show microsecond-level latency, while 8 MB transfers achieve approximately 23 GB/s bandwidth.These measurements provide a baseline for evaluating remote memory access over conventional RDMA-based architectures.Source: Table IV · p. 9, B. Device Characterization · p. 9
  • 24 hours of GSAT completed with no detected errors, supporting CMM-MP system stability.GSAT continuously exercises the memory subsystem to detect potential memory errors.Source: Figure 21 · p. 9, B. Device Characterization · p. 9

C. Real World Application Use Cases

The real-world evaluation uses Valkey microbenchmarks, Valkey with YCSB workloads A–F, and DiskANN vector search. Local pooled memory tracks system memory closely, while remote pooled memory incurs network-related performance costs but remains competitive with disk.

Valkey benchmark

  • For Valkey SET/GET, throughput degradation stays below 50% despite CMM-MP having 10x less raw bandwidth than system memory (32 GB/s vs 304 GB/s).MP-L remains close to the system-memory baseline across 64–4096-byte values; MP-R throughput is lower as value size and network data transfer increase.Source: Figure 22 · p. 9, C. Real World Application Use Cases · p. 9
  • For Valkey, MP-L latency stays close to system memory, while MP-R latency degradation is around 20% when the dataset is small.MP-R's higher latency is attributed to the additional network transport for remote accesses.Source: C. Real World Application Use Cases · p. 9, Figure 23 · p. 10

Valkey with YCSB

DiskANN

  • On SIFT (1 million records, one CPU core, ramdisk), MP-L closely follows system memory; MP-R trails it but remains about 3x faster than disk.DiskANN query throughput is evaluated across search list sizes; the ramdisk setup is used for a fair comparison.Source: Figure 26 · p. 10, C. Real World Application Use Cases · p. 10

VIII. CONCLUSION

CMM-MP combines a CXL Type 3 host interface with Ethernet backend connectivity and offloads pool composition and management from the host OS. The authors report lower latency and higher throughput than RDMA-based solutions, while identifying dynamic memory management and newer backend protocols as future work.

VIII. CONCLUSION

  • CMM-MP presents a standard CXL Type 3 interface while using Ethernet to connect memory devices, preserving load/store and traditional software interfaces without application changes.Pool composition and management functions are offloaded and abstracted from the Host OS.Source: §VIII · p. 10
  • Experiments report significantly lower latency and higher throughput than traditional RDMA-based solutions.The paper characterizes CMM-MP as an easy-to-use, cost-effective, scalable memory-pooling architecture.Source: §VIII · p. 10
  • Future work includes newer scale-up and scale-out backend protocols, device-side caching, and dynamic rather than static pool-memory management.The authors also plan to explore fine-grained dynamic allocation and management.Source: §VIII · p. 11
Loading 2610.22716v1…