PaperlayerView exact revision on arXiv

Source-linked AI summary

Beaver: Elastic GPU Sharing between ML and Latency-Critical vRAN Workloads

Yuncheng Yao, Zhenzhou Qi, Junyao Zheng, Chung-Hsuan Tung, Danyang Zhuo, Tingjun Chen

arXiv:2610.02522v1cs.DCcs.NI

TL;DR

Latency-critical vRAN and best-effort ML workloads share GPUs, but protecting vRAN deadlines while preserving co-tenant throughput is challenging. Beaver adapts SM allocations by slot and coordinates HBM access by rewriting compiled kernels; on an H200, it keeps vRAN p99.9 latency within its deadline while retaining 74% of Llama-3.3-70B serving throughput.

  • Problem

    Shared-GPU co-tenancy must protect the latency-critical vRAN deadline while preserving as much best-effort ML throughput as possible, despite shared HBM and slot-varying SM needs.

    Source: §1 · p. 1

  • Method

    Beaver predicts deadline-safe SM allocations per slot, repartitions SMs at slot granularity, and rewrites compiled co-tenant kernels to yield HBM bandwidth during vRAN memory-critical phases.

    Source: §1 · p. 2

  • Results

    With 13 memory-heavy vRAN cells and LLM serving on H200, Beaver keeps p99.9 latency within the deadline while retaining 74% of Llama-3.3-70B serving throughput.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across evaluated cellular traces, production inference kernels, downlink processing, and other GPUs, Beaver maintains deadline protection while supporting co-tenant workloads.

    Source: §1 · p. 2, §8 · p. 13

Abstract

from arXiv · show

Within the shared industry vision of AI-RAN, AI-and-RAN seeks to co-locate virtualized radio access network (vRAN) workloads and AI services on shared GPUs. This sharing is inherently asymmetric: vRAN workload is latency-critical, whereas the machine learning (ML) workload is a throughput-oriented, best-effort co-tenant. We present Beaver, a GPU sharing system that jointly manages compute and memory resources while protecting the vRAN's strict processing deadline. Beaver sizes the vRAN's streaming multiprocessor (SM) allocation from each slot's scheduled workload, repartitions SM allocations at slot granularity, and rewrites compiled ML kernels to yield HBM bandwidth during the vRAN's memory-critical phases. We implement Beaver and evaluate it using NVIDIA Aerial with heterogeneous multi-cell workloads, real-world cellular traces, production inference kernels, and full-stack LLM serving. On an H200 GPU, Beaver keeps the vRAN's p99.9 latency within its 1.5ms uplink deadline while retaining 74% of Llama-3.3-70B serving throughput. It also incurs no observed deadline misses under replayed cellular traces, protects a 375us downlink deadline, and generalizes to other GPUs including A100, GB10 and GH200.

1 Introduction

Beaver addresses asymmetric GPU co-tenancy: vRAN must meet strict processing deadlines, while best-effort ML should use safely spare capacity. It coordinates SM allocation and HBM access to preserve vRAN latency while supporting co-tenant throughput.

1 Introduction

  • Sequential, small-grid Aerial kernels structurally underutilize the GPU; a 20-cell H200 workload uses about 60% of peak SM throughput and less than 10% of peak HBM bandwidth.This leaves potential co-tenant capacity, provided vRAN p99.9 latency stays within its three-slot deadline.Source: §1 · p. 1
  • Existing partitions leave L2 cache and HBM shared, so bandwidth-intensive ML can delay vRAN kernels, while static allocations cannot track slot-varying demand.MIG or MPS reconfiguration takes milliseconds to seconds, versus a 500 μs radio slot.Source: §1 · p. 2
  • Safe sharing must find the minimum deadline-safe SM budget for heterogeneous cell loads, repartition at sub-slot timescales, and regulate opaque compiled ML kernels’ HBM demand without disrupting synchronization.The evaluation spans production inference kernels, full-stack LLM serving, cellular traces, and GPU architectures.Source: §1 · p. 2
  • Beaver predicts the smallest deadline-safe SM allocation for each slot, switches SM partitions at slot granularity, and rewrites compiled co-tenant kernels to yield HBM bandwidth during vRAN memory-critical phases.Its three mechanisms operate at complementary timescales, coordinating HBM access over time rather than dividing bandwidth into fixed slices.Source: §1 · p. 2
  • With 13 memory-heavy vRAN cells and LLM serving, Beaver keeps p99.9 latency within the deadline while retaining 74% of Llama-3.3-70B serving throughput.It also incurs no deadline misses under two replayed cellular traces, protects a 375 μs downlink deadline, and generalizes to A100, GB10, and GH200 GPUs.Source: §1 · p. 2

2 Background

GPU-based vRAN processing is sensitive to SM occupancy, memory traffic, and competition from concurrent ML kernels. Shared-GPU designs must protect uplink tail latency while accounting for the limits of compute partitioning and the compiled-kernel pipeline.

2 Background

  • PUSCH uplink processing is the representative protected workload because it is generally more compute-intensive, and missing a slot deadline can trigger retransmission and consume air-interface capacity.The paper targets a 1.5 ms uplink deadline.Source: §2 · p. 3
  • Kernel blocks run on SMs in successive waves when they cannot reside concurrently, and blocks remain on an SM until completion; concurrent ML kernels can contend for compute and other resources.Disjoint SM partitions prevent head-of-line blocking and compute contention, but do not isolate shared memory resources.Source: §2 · p. 3, Figure 3 · p. 3
  • MIG isolates SMs and proportional memory slices but takes seconds to rebuild; MPS and CUDA green contexts leave HBM shared and cannot resize on the slot clock.The cited text identifies no existing system combining slot-granular adaptation with HBM protection.Source: Figure 3 · p. 3
  • CUDA code is lowered to PTX and assembled into architecture-specific SASS in a cubin; PTX may be bundled with cubins or exposed through an interceptable JIT path.This provides the compilation pathway for rewriting compiled kernels.Source: Figure 3 · p. 3

3 Sharing Opportunities and Challenges

The section identifies substantial vRAN GPU headroom but shows that safe sharing must address compute and HBM interference as well as rapidly changing workloads. Meeting deadlines therefore requires workload-sensitive SM allocation and control of shared memory bandwidth.

3.1 GPU Utilization of vRAN Workloads

  • At 32 cells, p99.9 uplink latency reaches 1.5 ms before compute or HBM saturates; at 20 cells, 72 SMs is the minimum deadline-safe allocation.Compute utilization plateaus near 60% beyond 15 cells, while HBM utilization remains below 10% even at 40 cells.Source: §3.1 · p. 4

3.2 Resource Contention in GPU Sharing

  • HOL blocking delays vRAN kernel starts behind running blocks, while compute contention slows concurrent blocks; disjoint SMs remove compute contention.A 72:60 SM split keeps the 20-cell uplink within its deadline against the compute stressor.Source: Figure 4 · p. 4, §3.2 · p. 5
  • With a 72:60 SM split, compute stress causes no deadline misses, but HBM flooders, FP8 GEMM, and vLLM still cause misses up to 99.8%.Disjoint SMs do not partition the shared caches or HBM fabric.Source: §3.2 · p. 5

3.3 Design Challenges

  • Cell count alone cannot set the deadline-safe SM budget: heterogeneous workloads and nonlinear GPU scaling require selecting the smallest sufficient discrete partition.Too few SMs cause deadline misses; too many reduce co-tenant resources unnecessarily.Source: §3.3 · p. 5
  • Slot-to-slot workload changes make static SM partitions either unsafe at peak load or wasteful at light load; GC reconfiguration is too costly for sub-millisecond control.GC assignments are fixed at context creation, and changing partitions requires switching contexts and draining old work.Source: §3.3 · p. 5
  • Opaque ML kernels can saturate shared HBM during vRAN-critical phases, so Beaver must regulate them transparently at sub-slot granularity without disrupting semantics or needlessly cutting throughput.Issued memory requests cannot be recalled, making coarse or reactive throttling inadequate.Source: §3.3 · p. 5

4 Beaver’s Design

Beaver combines deadline-driven vRAN SM sizing, slot-boundary repartitioning, and cooperative control of co-tenant HBM traffic. Its design allocates compute from scheduled workload demand and protects memory-critical intervals while allowing ML progress elsewhere.

4.1 Workload-Aware SM Sizer

  • Beaver predicts heterogeneous-slot p99.9 latency from coefficients fitted on homogeneous workloads, then selects the smallest SM allocation that meets the 1.5 ms deadline.The model composes per-cell work terms, takes the largest configuration-specific setup term, and adds a safety reserve.Source: §4.1 · p. 6

4.2 Slot-Granular SM Repartitioning

  • ≈1.8 μs is enough to switch slots: Beaver rebinds launches to a pre-created GC pair rather than creating contexts on the critical path.Previously launched work finishes in its original context; the rebind is less than 0.5% of a 0.5 ms slot.Source: §4.2 · pp. 6–7

4.3 Cooperative HBM Bandwidth Guard

  • The cooperative HBM guard lets vRAN blocks signal memory-critical regions in a shared bitmap, causing the ML co-tenant to defer new HBM-intensive regions while flags are set.Per-block bits keep protection active until the last protected block clears its flag.Source: §4.3 · p. 7
  • Profile-guided signaling leaves HBM-light kernels unprotected, protects streaming kernels for their full execution, and protects LDPC decoding only during each wave’s reads and writes.The wave-aware schedule clears protection during belief propagation to let the co-tenant progress.Source: Figure 7 · p. 7, §4.3 · p. 8
  • Beaver rewrites compiled co-tenant PTX by profiling HBM traffic, selecting high-traffic basic blocks, and inserting safe polling gates before memory batches.The rewritten binary retains the original arguments and launch interface, and a load-time shim redirects targeted kernels and binds the shared bitmap.Source: §4.3 · p. 8

5 Experimental Setup

The evaluation uses Aerial uplink workloads on several GPU platforms, alongside stress tests, production inference kernels, and full-stack LLM serving. It compares protection configurations and measures vRAN latency and deadline misses as well as co-tenant throughput.

Implementation and hardware platforms

  • Experiments primarily use an H200, with cross-GPU tests on A100, GB10, and GH200; vRAN stress tests use compute-heavy tv-21 and memory-heavy tv-28.The study includes 28 cell configurations and uses 5G numerology 1 with 0.5 ms slots.Source: §5 · p. 8

Co-tenant workloads

  • Co-tenants span tunable FMA and HBM stressors, production inference kernels, and vLLM serving Llama-3.3-70B and Mistral-Small-24B.The production kernels include GEMM, FlashAttention-3, fused MoE, and FlashInfer.Source: §5 · p. 9

Baseline configurations

  • The baselines include contention-free vRAN-only, GC-only SM isolation without the HBM guard, Beaver with both protections, and two mechanism ablations.The ablations are unprotected co-scheduling and the HBM guard without GC-based SM isolation.Source: §5 · p. 9

Performance metrics

  • vRAN latency and deadline misses are measured over 50,000 slots; kernel throughput is reported in kernels/s and vLLM serving throughput in tok/s.The reported latency distribution includes p50 and p99.9, with uplink deadline misses measured against 1.5 ms.Source: §5 · p. 9

6 Evaluation Results

Beaver combines SM isolation with HBM bandwidth control to protect vRAN deadlines while sharing GPU capacity with ML co-tenants. Evaluations cover inference kernels, serving workloads, changing traffic, heterogeneous sizing, downlink processing, and multiple GPU architectures.

6.1 Microbenchmarks

  • With an HBM flooder, Beaver’s combined SM isolation and HBM regulation reduced p99.9 latency to 1.380 ms and the deadline miss rate below 0.1%.Neither mechanism alone met the deadline: GC-only reached 1.608 ms, while HBM-guard-only reached 1.769 ms.Source: §6.1 · p. 9
  • With production inference kernels, Beaver kept p99.9 latency within deadline with no observed misses, while kernels retained 38%–86% of unthrottled throughput.Under GC-only, six of seven kernels pushed median latency beyond the 1.5 ms deadline, with miss rates of 58%–99%.Source: §6.1 · p. 10, Figure 9 · p. 10
  • Against prior GPU-sharing systems, Beaver had no observed deadline misses while retaining 64.1% of co-tenant throughput; even the safest uniform-throttling point exceeded a 0.1% miss rate.Unprotected sharing and GC-only missed 98% and 79% of deadlines; TGS reached 1.7% misses only below 10% retained throughput.Source: §6.1 · p. 10

6.2 End-to-End Evaluations

  • With full-stack LLM serving, GC-only produced deadline miss rates of 93% with Llama-3.3-70B and 84% with Mistral-Small-24B.The vRAN workload comprised 13 memory-heavy cells with a 56:76 SM split.Source: §6.2 · p. 10, Figure 10 · p. 11
  • Across Madrid and TRACTOR replays, Beaver kept p99.9 latency at 1.467 and 1.426 ms, with miss rates below 0.1%, retaining 74% and 73% of throughput; GC-only missed 37%–51% of slots.The 30-second replays used time-varying vRAN workloads with a live Llama3.3-70B vLLM co-tenant.Source: Figure 11 · p. 11, §6.2 · p. 11

6.3 Ablation Studies

  • Across 4,000 heterogeneous workloads, the latency model’s median relative prediction error was 2.4% and its 95th-percentile error was 9.2%; the sizer never underprovisioned.It selected the exact minimum SM allocation or one or two eight-SM tiers above it; with GC-only, 62 of 100 workloads exceeded the 1.5 ms deadline, while Beaver kept all within it.Source: §6.3 · p. 11, Figure 12 · p. 11, Figure 13 · p. 12
  • Beaver switched SM partitions in 1.79 μs, versus 564 ms with MPS and 1.06 s with MIG, and kept p99.9 latency within deadline across A100, GB10, and GH200.Across those platforms it retained 78%–85% of unthrottled vLLM throughput; for downlink, HBM guard eliminated all observed misses under the tested memory-intensive co-tenants.Source: Figure 13 · p. 12, Figure 14 · p. 12

7 Related Work

Prior work makes software radio processing scalable and accelerates individual L1 operations, while Aerial provides production-grade GPU PDSCH and PUSCH pipelines. GPU-sharing systems vary in scheduling granularity and isolation, but the cited vRAN systems use MPS or MIG to colocate workloads.

Baseband processing and virtualization

  • CPU and heterogeneous CPU–eASIC systems make software L1 feasible and scalable, but do not protect a latency-critical GPU L1 pipeline from concurrent ML workloads.The cited GPU work includes LDPC acceleration and cross-platform benchmarking, while Aerial implements complete production-grade PDSCH and PUSCH pipelines.Source: §7 · p. 12

GPU sharing systems

  • GPU-sharing designs schedule or resize jobs at different granularities, while MIG isolates SMs and memory slices and MPS or CUDA green contexts isolate compute but leave HBM shared.REEF offers microsecond-scale preemption but requires a custom AMD ROCm driver and is incompatible with the paper’s CUDA pipeline.Source: §7 · p. 12
  • For vRAN co-tenancy, YinYangRAN uses MPS, while CAORA and RAN-LLM use MIG to colocate vRAN with AI or LLM workloads.Source: §7 · p. 12

8 Conclusion

Beaver enables elastic co-tenancy between ML and latency-critical vRAN workloads by combining workload-aware SM sizing, slot-granular repartitioning, and an HBM bandwidth guard. Evaluations with Aerial report deadline protection for uplink serving and extension to downlink and other GPUs.

  • Beaver combines workload-aware SM sizing for heterogeneous multi-cell workloads, slot-granular repartitioning across pre-created GC pairs, and an HBM bandwidth guard injected into compiled co-tenant kernels.Source: §8 · p. 12
  • Evaluated with NVIDIA Aerial, Beaver keeps vRAN p99.9 processing latency within deadline alongside full-stack vLLM serving, and extends to downlink and other GPUs.Future directions include coordinating multiple ML co-tenants, multi-GPU resource management, and finer-grained HBM bandwidth control.Source: §8 · p. 13

A Aerial Uplink Test Vector Selection

The appendix describes the Aerial uplink test vectors used to benchmark one cell’s work in one slot. It selects separate compute-heavy and memory-heavy vectors so experiments stress the resource each mechanism controls.

  • An Aerial test vector specifies reproducible input for one radio-pipeline execution, representing one 0.5-ms uplink slot for one cell rather than a multi-slot traffic trace.The benchmark replays that slot and replicates it across cells.Source: A Aerial Uplink Test Vector Selection · p. 14
  • The selected family contains 35 vectors covering every MCS value from 0–27, on a 100-MHz carrier with 273 resource blocks and one user.Each vector captures a complete 0.5-ms uplink slot with 14 PUSCH OFDM symbols.Source: A Aerial Uplink Test Vector Selection · p. 14
  • Vector 21 is the most compute-intensive at 113.5 μs per cell, while vector 28 is the most memory-intensive, moving 17.2 MB through HBM per cell.Vector 28 sustains 217 GB/s across the slot and reaches 660 GB/s in its busiest kernel.Source: A Aerial Uplink Test Vector Selection · p. 14
  • The evaluation uses vector 21 for compute-contention and SM-partitioning tests, and vector 28 for HBM-contention and bandwidth-protection tests.Each mechanism is tested with the vector that places the most pressure on the resource it controls.Source: A Aerial Uplink Test Vector Selection · p. 14

B Aerial Downlink Test Vector Selection

The downlink evaluation selects Aerial PDSCH vector 3942 as a demanding configuration because available downlink vectors have similar HBM bandwidth demand. It replicates that vector across 52 cells on an H200, dividing SMs between the downlink and co-tenant.

C SM Repartitioning Latency

Beaver changes SM allocations within a 500 μs radio slot, unlike MPS and MIG. Its pre-created partitions let it follow slot-to-slot radio demand without delaying subsequent radio processing.

D SM Sizing Model Calibration

Beaver calibrates its sizing model on homogeneous workloads, then composes per-vector coefficients to predict heterogeneous slots. A fixed tail-latency reserve accounts for the gap between predicted median latency and the p99.9 sizing target.

  • 1,008 homogeneous combinations across 28 test vectors form the calibration sweep, with each combination run for 3,000 slots.The sweep discards 50 warm-up slots per combination and computes median and p99.9 latency from the remaining 2,950 slots, totaling 3.024 million slots.Source: D SM Sizing Model Calibration · p. 16
  • Per-vector least-squares fits estimate three coefficients using only combinations with measured median latency at most 2.2 ms.Source: D SM Sizing Model Calibration · p. 16
  • For heterogeneous slots, the controller takes the largest per-slot setup term and sums per-cell terms, then adds the largest test-vector reserve once.The reserve is 85–215 μs for a vRAN-only slot, combining a fixed 24.6 μs median-to-p99.9 offset with a 60–190 μs fitting-error allowance.Source: D SM Sizing Model Calibration · p. 16
  • Across test vectors, the median fitting error is 31 μs and the maximum is 76 μs, or 5% of the deadline.The SM-parallel term rises from a median of 21 μs per cell for QPSK to 49 μs for 256-QAM; the SM-insensitive term stays below 11 μs per cell.Source: D SM Sizing Model Calibration · p. 16
Loading 2610.02522v1…