Source-linked AI summary
MIND: In-Network Memory Management for Disaggregated Data Centers
Seung-seob Lee, Yanpeng Yu, Yupeng Tang, Anurag Khandelwal, Lin Zhong, Abhishek Bhattacharjee
TL;DR
Prior memory-disaggregation designs trade performance against elastic resource sharing, and none of the known approaches meet the requirements simultaneously. MIND puts memory management in programmable switches and redesigns its mechanisms for the network setting. Its prototype enables transparent resource elasticity while matching prior proposals’ performance for real-world workloads.
Problem
Prior memory-disaggregation approaches trade performance against elastic resource sharing, and none of the known approaches simultaneously meet requirements for access performance, resource elasticity, and unmodified applications.
Source: §1 · p. 1Method
MIND places memory-management logic and metadata in programmable switches and redesigns translation, protection, and MSI coherence for the disaggregated setting.
Source: §1 · pp. 1–2Results
MIND’s prototype enables transparent resource elasticity for real-world workloads while matching the performance of prior memory-disaggregation proposals.
Source: §1 · p. 2, §10 · p. 15Takeaways & Limitations
MIND’s prototype demonstrates that rack-scale memory disaggregation can provide transparent resource elasticity while matching prior proposals’ performance on real-world workloads.
Source: §10 · p. 15MIND’s implementation supports stronger TSO consistency but precludes weaker PSO consistency models because current x86 or ARM architectures cannot realize the required page-fault behavior.
Source: §6.1 · p. 9
Abstract
from arXiv · showhide
Memory-compute disaggregation promises transparent elasticity, high utilization and balanced usage for resources in data centers by physically separating memory and compute into network-attached resource "blades". However, existing designs achieve performance at the cost of resource elasticity, restricting memory sharing to a single compute blade to avoid costly memory coherence traffic over the network. In this work, we show that emerging programmable network switches can enable an efficient shared memory abstraction for disaggregated architectures by placing memory management logic in the network fabric. We find that centralizing memory management in the network permits bandwidth and latency-efficient realization of in-network cache coherence protocols, while programmable switch ASICs support other memory management logic at line-rate. We realize these insights into MIND, an in-network memory management unit for rack-scale memory disaggregation. MIND enables transparent resource elasticity while matching the performance of prior memory disaggregation proposals for real-world workloads.
1 Introduction
Memory disaggregation must combine low-latency, high-throughput access with elastic resources and unmodified applications, but prior designs do not meet all three requirements simultaneously. MIND places memory-management logic in programmable switches, redesigns address translation, protection, and coherence for switch constraints, and matches prior proposals’ performance while enabling transparent elasticity.
1 Introduction
- Disaggregation must provide low-latency, high-throughput memory access, elastic memory and compute, and support for unmodified applications, yet known approaches do not satisfy all three.Transparent designs restrict application compute elasticity to one blade to avoid network coherence traffic.Source: §1 · p. 1
- MIND places memory-management logic and metadata in programmable switches, which act as the MMU and process that logic at line rate.The network fabric is positioned between compute and memory resources in the access path.Source: §1 · p. 1
- Programmable switch ASICs constrain MIND with only a few megabytes of on-chip memory, limited per-packet computation, and physically staged processing pipelines.These constraints make traditional page tables for terabytes of memory challenging to store and complicate placement of management logic and metadata.Source: §1 · pp. 1–2
- MIND combines a shared, range-partitioned virtual address space and load-balanced physical allocation with decoupled domain-based protection and in-network MSI coherence using switch multicast.Its Bounded Splitting algorithm dynamically sizes memory regions to bound switch storage and false-invalidation overheads.Source: §1 · p. 2
- MIND’s prototype enables transparent resource elasticity for real-world workloads while matching the performance of prior memory-disaggregation proposals.The system is realized on a disaggregated cluster emulated with traditional servers and a programmable switch.Source: §1 · p. 2
- High read-write contention workloads scale sub-linearly with more threads because of limitations in current hardware.The cited passage identifies this as a limitation despite MIND’s competitiveness with compared systems.Source: §1 · p. 2
2 Background
The paper assumes rack-scale, partial memory disaggregation and describes programmable switches and shared-memory coherence as enabling technologies. It then explains why existing transparent and non-transparent designs trade performance against elastic resource sharing.
2 Background
- The paper scopes its design to rack-scale systems connected by one programmable switch, with most memory network-attached and a few GBs of local DRAM per compute blade as cache.This is partial memory disaggregation, consistent with the cited prior work.Source: §2 · p. 2
2.1 Enabling Technologies
- RMT switches use P4-programmable parser and match-action pipeline stages, and MIND adopts them for performance, availability, power, and cost efficiency.Switches also include a general-purpose CPU with DRAM for processing beyond ASIC resources and hosting controllers.Source: §2.1 · pp. 2–3
- In directory-based MSI coherence, each cached block is Modified, Shared, or Invalid, while a directory tracks its state and sharers to coordinate access and invalidations.MIND restricts its focus to MSI for simplicity.Source: §2.1 · p. 3
2.2 Disaggregated Memory Designs and Challenges
- Transparent elastic compute scaling requires shared memory, but coherence for application data and management metadata imposes performance overheads in existing disaggregated designs.This tradeoff motivates placing management in the network fabric, whose central position avoids metadata coherence and permits in-network coherence logic.Source: §2.2 · p. 3
- Transparent compute-centric and memory-centric DSM adaptations require sequential remote requests, while caching metadata adds coherence overhead; non-transparent proposals avoid network coherence by limiting processes to one compute blade.The resulting single-blade restriction limits transparent compute elasticity and can require application modification to scale beyond that blade.Source: §2.2 · p. 3
3 MIND Overview
MIND places memory-management logic and metadata in the network fabric, using programmable switches to support shared memory while addressing metadata-coherence overhead. Its design separates memory-management functions and uses the switch’s global view and network hardware primitives to realize them.
3 MIND Overview
- Central network placement avoids metadata coherence, and programmable switch ASICs can execute memory-management logic at line rate, even for multi-terabit traffic.The switch also hosts cache-coherence logic and directory, enabling specialized protocols with reduced network latency and bandwidth overheads.Source: §3 · pp. 3–4
3.1 Design Principles
- MIND’s principles are to decouple functions, use a centralized control plane’s global view to compute policies, and exploit switch hardware primitives to implement them.The design applies these principles across allocation, translation, protection, cache access, and coherence.Source: §3.1 · p. 4
3.2 Design Overview
- MIND places memory-management logic and metadata in the network rather than on compute or memory blades or in a separate global controller.Programmable switches act as the MMU in the transparent virtual-memory abstraction.Source: §3.2 · p. 4
- Compute blades intercept allocations and handle loads and stores through local DRAM caches; cache misses trigger page faults and remote RDMA fetches.The switch checks permissions and coherence, translates virtual addresses when needed, and routes requests; memory blades serve physical-page RDMA requests without general-purpose CPU cycles.Source: §3.2 · pp. 4–5
4 In-Network Memory Management
MIND separates allocation, translation, protection, and coherence functions, then implements them in the network under switch storage and processing constraints. Its design combines compact translation and protection metadata with adaptive coherence regions and network-efficient invalidation handling.
4.1 Memory Allocation & Addressing
- A single global virtual address space is range-partitioned across memory blades, allowing one translation entry per blade and reducing switch data-plane storage.The control plane allocates memory to the blade with the least allocated memory; one-to-one virtual-to-physical mapping within a blade supports traditional allocation schemes and reduces fragmentation.Source: §4.1 · p. 5
4.2 Memory Protection
- MIND separates protection from translation: protection domains identify who may access a vma, while permission classes specify the allowed access.A data-plane table stores the mapping; TCAM range matches check requests, and MIND aligns allocations and coalesces compatible entries to save TCAM space.Source: §4.2 · pp. 5–6
4.3 Caching & Cache Coherence
- Compute-blade caches coexist with a switch-resident coherence directory and logic, letting common MSI transitions use one round-trip while the most expensive transitions use two.The switch’s central placement gives directory access within half a round-trip.Source: §4.3 · p. 6
4.3.1 Storage vs. performance tradeoff
- MIND decouples fine-grained cache accesses from coarser directory tracking; invalidating a tracked region invalidates its cached dirty pages.A larger region saves directory entries but can cause false invalidations of unrelated dirty pages, wasting bandwidth and cache capacity.Source: §4.3.1 · p. 6
- MIND sizes directory regions by starting large and splitting when false-invalidation overhead exceeds a threshold, stopping at the threshold or 4 KB pages.This bounded splitting adapts region sizes to balance coherence performance against directory storage.Source: §4.3.1 · pp. 6–7
4.3.2 In-Network Coherence Protocol
- MIND uses directory-based MSI coherence and switch multicast for invalidations, embedding the sharer list so egress drops requests not destined for sharers.This avoids sending invalidations to non-sharers while using native switch multicast.Source: §4.3.2 · p. 7
4.4 Handling Failures
- For switch failures, MIND replicates the control plane at a backup switch and reconstructs data-plane state; for communication failures, ACKs, timeouts, and reset messages prevent coherence deadlocks.After a predefined number of retransmissions without an ACK, the compute blade asks the switch control plane to flush that address from caches and remove its directory entry.Source: §4.4 · p. 7
5 Bounded Splitting: Algorithm & Analysis
Bounded Splitting adapts cache-directory region sizes to false invalidation counts, keeping those counts below a threshold while limiting directory growth. Its analysis bounds the regions generated by splitting and motivates a practical tradeoff between directory storage and performance overhead.
5.1 Algorithm
- Each epoch, the algorithm tracks false invalidations per region and splits any region whose count exceeds threshold t, stopping at 4 KB.Splitting reduces false invalidations across the resulting regions; at 4 KB, the count is zero.Source: §5.1 · p. 7, §5.2 · p. 8
- The algorithm assumes access patterns and allocated pages remain stable for O(log2 M) epochs; the authors report suitable epoch sizing for evaluated workloads.Source: §5.1 · p. 7
5.2 Performance Bounds
- The worst-case bound for total directory regions depends on the per-region bound and threshold t, which is set as a fraction of average false invalidations.The threshold setting is intended to ensure fairness across M-sized regions.Source: §5.2 · pp. 7–8
- An M-page region generates at most (⌈f/t⌉−1) · (1 + log2 M) sub-regions, bounding directory growth as regions are split.The bound follows from limiting the smallest region to 4 KB, so an M-sized region can be split at most log2 M times.Source: §5.2 · p. 8
- MIND defaults to 16 KB regions as a favorable storage-versus-performance-overhead tradeoff for evaluated workloads.The algorithm can also be implemented by merging from 4 KB regions, or by starting at an intermediate size and splitting or merging.Source: §5.2 · pp. 8–9
- At c = 1, dynamic resizing reduces worst-case directory storage from M·N entries to (1 + log2 M)·N, an exponential decrease.In practice, the analysis is worst-case; MIND dynamically adjusts c to keep switch data-plane storage utilization below 95%.Source: §5.2 · p. 9
6 Implementation Details
MIND divides implementation across compute blades, memory blades, and a programmable switch, with Linux-compatible interfaces supporting transparent remote-memory access. The switch control plane manages processes and directory entries, while its ASIC performs packet-time memory-management work.
6.1 Compute Blade
- Compute-blade kernel changes intercept memory and process operations so the switch can manage shared address spaces while returning Linux-compatible system-call results.Threads of one process can share a PID across blades; scheduling is not addressed and placement is round-robin.Source: §6.1 · p. 9
- On a cache miss, the compute blade sends a one-sided RDMA read; after receipt, it populates local page tables, while invalidations flush writable pages and remove local mappings.The received page is registered as the NIC buffer, avoiding additional data copies.Source: §6.1 · p. 9
- MIND's implementation uses TSO and excludes weaker PSO because x86 and ARM cannot trap reads without also trapping writes; this limits scalability under high read/write contention.PSO would require buffering writes without a page fault and trapping on a subsequent read to the same page.Source: §6.1 · p. 9
6.2 Memory blade
- Memory blades need no compute logic or data-plane processing for RDMA requests, but the prototype uses a kernel module for RDMA initialization; removing that CPU-dependent step requires new hardware.After initialization, the memory-blade NIC handles compute-blade RDMA requests without CPU involvement.Source: §6.2 · pp. 9–10
6.3 Programmable Switch
- The switch CPU manages processes, memory, and cache directories, while the ASIC performs address translation, protection, directory transitions, and RDMA virtualization.The prototype uses a 32-port switch with a 6.4 Tbps Tofino ASIC.Source: §6.3 · p. 10
- A single match-action unit cannot perform directory lookup, state-based translation, and entry update together, so MIND splits the work across two units and recirculates requests for updates.The second unit stores a materialized state-transition table, trading data-plane memory capacity for limited compute cycles; invalidations are multicast to current sharers.Source: §6.3 · p. 10
7 Evaluation
MIND is evaluated for transparent, elastic memory access using real-world workloads and comparisons with GAM and FastSwap. The evaluation examines scaling, switch and memory bottlenecks, and bounded splitting.
7.1 Performance Scaling for Real-World Workloads
- For TF, doubling compute blades improves MIND performance by ~1.67×, and MIND is 59× faster than GAM at 8 blades.GC performance declines beyond 2 blades amid more shared writes and invalidations; M𝐴 and M𝐶 scale poorly beyond one blade, with over 10× the invalidations and page flushes of TF or GC.Source: Figure 5 · p. 11, §7.1 · p. 12
- MIND and FastSwap scale almost linearly with more threads on one blade, while GAM scales linearly only through 4 threads before software overheads limit scaling.The evaluation compares GAM, a DSM-based elastic design, with FastSwap, which does not transparently scale beyond one compute blade; workloads include TF, GC, Memcached YCSB-A/C, and Native-KVS.Source: §7 · pp. 10–11, §7.1 · p. 11
7.3 Evaluating MIND’s Bounded Splitting Algorithm
- Bounded splitting balances directory size against false invalidations: smaller tracked regions reduce false invalidations but require more directory entries.Increasing the epoch from 1 to 100 ms has little effect on false invalidations and reduces control-plane overhead; 16 KB is the selected initial region size, and neither parameter noticeably affects directory entries at stable state.Source: Figure 9 · p. 13, §7.3 · p. 14
7.2 MIND Performance and Resource Bottlenecks
- Read-only or unshared accesses sustain high throughput, but increasing both writes and sharing reduces throughput by ~10× at sharing ratio 1.Invalidations also add latency: modified-state transitions take ~18 μs, while read-only shared-state requests take ~9 μs; TLB shootdowns and queueing further increase latency under lower read ratios.Source: §7.2 · pp. 12–13
- MIND keeps address-translation and protection match-action rules nearly constant as workload size increases, while balancing allocation load across memory blades.The directory remains below its 30 k-entry limit for TF and GC, but stays close to the limit for M𝐴 and M𝐶, where coarse tracking causes false invalidations; 2 MB pages can balance load but require more translation entries, while 1 GB pages balance poorly for allocation-intensive workloads.Source: §7.2 · pp. 13–14, Figure 8 · p. 13
8 Limitations and Future Research
The discussion identifies limitations of the current MIND implementation and outlines possible architectural and protocol changes to address them.
Thread management
- Remote memory access remains at least two orders of magnitude slower than local access; co-locating threads with more shared-memory accesses could reduce network invalidations.This is presented as an orthogonal way to improve end-to-end application performance.Source: §8 · p. 14
Other coherence protocols
- MOESI could improve scalability by reducing broadcasts and write-backs, but requires larger state-transition tables and more transient-state handling.The authors say the required TCAM entries would remain small relative to switch capacity.Source: §8 · p. 14
Weaker consistency models
- The x86 page-fault-based implementation cannot realize weaker consistency models such as PSO; page faults on reads but not writes could enable them.The proposed compute-blade redesign could facilitate higher throughput to disaggregated memory.Source: §8 · p. 14
Scaling beyond a rack
- Scaling transparently beyond one rack would require extending MIND from a single switch to a data-center-wide network topology.The discussion compares this shift to moving from single-node CPUs to multi-node NUMA architectures.Source: §8 · p. 14
Virtualization
- Virtual-memory-level protection does not provide full user isolation; virtualization extensions are needed, including isolation of shared compute-memory interconnect resources for performance isolation.The listed resources include network bandwidth, switch resources, and NIC resources.Source: §8 · p. 15
9 Related Work
Related work places MIND alongside in-network computing, application-driven disaggregation, key-value systems, and interconnect standards, while distinguishing where memory-management logic resides.
In-network computing
- Unlike in-network caches and metadata offloads, MIND places memory management completely in the network; NOPaxos could complement it with consistent replication.Concordia uses a programmable switch as a cache for DSM directory entries, whereas MIND realizes memory management in the network.Source: §9 · p. 15
Application-driven memory disaggregation
- MIND’s transparent disaggregated shared memory can coexist with OS-level libraries that expose memory placement and failure notifications to applications.The passage describes application-facing abstractions as compatible additions layered atop MIND.Source: §9 · p. 15
- Clover uses atomic RDMA verbs for lock-free access to key-value pairs in disaggregated persistent memory, but does not place coordination in the fabric or address protection, caching, or coherence.A centralized coordinator was one possible design considered for Clover.Source: §9 · p. 15
Emerging industry standards
- Unlike MIND, interconnect standards place MMU functions at endpoints; Gen-Z targets inter-server fabrics and defines operations usable as building blocks for software coherence.The passage says no publicly available realization was known to the authors.Source: §9 · p. 15
10 Conclusion
MIND redesigns memory-management mechanisms for rack-scale disaggregation under programmable-switch constraints. Its prototype provides transparent resource elasticity while matching prior proposals’ performance on real-world workloads.
10 Conclusion
- MIND achieves resource elasticity, performance, and transparency through a redesign of traditional memory-management mechanisms that accounts for programmable-switch ASIC constraints.The prototype facilitates transparent resource elasticity while matching the performance of prior memory-disaggregation proposals for real-world workloads.Source: §10 · p. 15