PaperlayerView exact revision on arXiv

Source-linked AI summary

ScaleRAN: A Runtime for Elastic GPU Resource Management in AI-RAN

Arthur Hamard, Jérémie Leguay, Gopalasingham Aravinthan, Alberto Conte, Julien Lallet, Abdelkader Outtagarts

arXiv:2610.23072v1cs.NIcs.DC

TL;DR

Coarse or static GPU partitioning limits adaptation to changing RAN load while preserving strict slot deadlines. ScaleRAN combines Green Context SM partitioning with autoscaling policies to redistribute GPU resources between RAN and AI workloads. Its proactive policies improve deadline success over the reactive controller, with higher prediction quantiles trading greater reliability for more SM regret.

  • Problem

    Existing GPU-sharing approaches often use coarse or static partitioning, leaving the challenge of reallocating GPU resources at fine granularity while meeting strict RAN slot deadlines.

    Source: §I · p. 1

  • Method

    ScaleRAN combines Green Context-based SM partitioning with reactive and prediction-based autoscaling to redistribute GPU resources between RAN and co-located AI workloads.

    Source: §I · p. 2, D. GPU Resource Management in AI-RAN · p. 3

  • Results

    Proactive prediction substantially improves deadline success over the reactive Kubernetes-inspired controller, and higher prediction quantiles approach 100% deadline success at the cost of larger SM regret.

    Source: D. Autoscaling Results · p. 9

  • Takeaways & Limitations

    The work demonstrates the feasibility of fine-grained GPU sharing for AI-RAN platforms and identifies O-RAN O-Cloud deployments as a suitable setting.

    Source: §VII · p. 9

    The prototype switches profiles at synchronization points rather than individual kernel launches; transparent CUDA interception could enable kernel-granularity switches but would require extensive API interception and object migration across Green Contexts.

    Source: E. Discussion · p. 5

Abstract

from arXiv · show

AI-RAN is driving the convergence of radio access network (RAN) processing and artificial intelligence (AI) workloads on GPU-enabled computing platforms, ranging from GPU-accelerated base stations to cloud-native infrastructures. While GPU acceleration improves the efficiency of virtualized RAN functions, the computational demand of latency-critical baseband processing varies with traffic load, creating transient spare GPU capacity that can be reclaimed by co-located AI applications. Existing GPU-sharing mechanisms rely on coarse-grained hardware partitioning or static resource allocation, limiting their ability to adapt to slot-level workload fluctuations while preserving strict timing requirements. This paper presents ScaleRAN, a node-local runtime for elastic GPU resource management in AI-RAN. ScaleRAN leverages NVIDIA Green Contexts to dynamically partition Streaming Multiprocessors (SMs) between RAN and AI workloads through a ladder of pre-created GPU partitions, enabling low-overhead resource reallocation at RAN slot granularity. The proposed runtime naturally supports Open-RAN (O-RAN) O-Cloud deployments, one representative AI-RAN deployment scenario. To drive these reallocations, we propose a reactive controller inspired by Kubernetes autoscaling and a proactive controller based on quantile regression and conformal prediction. We also develop LDPCSim, a GPU-accelerated workload generator built on NVIDIA Aerial for reproducible evaluation. Results show that predictive fine-grained GPU partitioning satisfies slot-level RAN deadlines while increasing the GPU resources available to co-located AI workloads, demonstrating the practicality of elastic GPU sharing for future AI-RAN platforms.

I. INTRODUCTION

AI-RAN co-locates RAN processing and AI workloads on shared GPUs, but existing sharing approaches struggle to exploit transient capacity while meeting strict slot deadlines. The paper motivates a fine-grained runtime and autoscaling policies for this setting.

I. INTRODUCTION

  • O-RAN promotes open, disaggregated cloud-native deployments, and its O-Cloud vision includes platforms hosting virtualized RAN functions and AI applications.The paper describes O-Cloud as a potential AI-RAN deployment platform.Source: §I · p. 1
  • GPU sharing must exploit transient capacity during lower RAN load while prioritizing RAN processing when load rises, without compromising deterministic performance.The paper identifies 0.5 ms transmission slots as a timing constraint for commonly used 5G numerologies.Source: §I · p. 1

II. RELATED WORK

  • Prior approaches include MPS concurrency, MIG hardware isolation, hybrid MIG/MPS systems, and kernel- or runtime-level scheduling; Green Contexts add driver-level SM-granular partitioning.The paper frames combining strict RAN latency guarantees with fine-grained, low-overhead GPU reallocation as an open challenge.Source: §I · p. 1

I. INTRODUCTION

  • ScaleRAN dynamically reallocates GPU SMs between latency-critical RAN and co-located AI workloads, with reactive and prediction-based autoscaling policies.The paper also introduces LDPCSim for reproducible evaluation.Source: §I · p. 2

A. Background on GPU Execution Model

  • GPU kernels are launched through CUDA streams, which are ordered queues for asynchronous execution.GPU threads are organized into CTAs; threads within a CTA cooperate, while CTAs can be scheduled in parallel.Source: A. Background on GPU Execution Model · p. 2
  • SMs provide GPU compute resources, and assigning more SMs can reduce completion time when a workload exposes enough independent CTAs.The paper uses SM allocation as a proxy for available GPU compute capacity.Source: A. Background on GPU Execution Model · p. 2

B. Fundamental GPU Partitioning Primitives

GPU-sharing mechanisms trade off concurrency, isolation, and reallocation flexibility. ScaleRAN builds on fine-grained Green Context partitions while avoiding the complex interception or kernel-transformation approaches used by some alternatives.

1) Temporal and hardware-level sharing.:

  • MPS supports concurrent execution across processes, while MIG provides isolated hardware instances but is limited to a small number of fixed-size partitions without runtime resizing.MPS and MIG offer different trade-offs in sharing and isolation.Source: 1) Temporal and hardware-level sharing · p. 2

2) Driver-Level dynamic sharing.:

  • Green Contexts provide runtime-managed, dynamic SM-level partitions, while MPS enables concurrent CUDA processes within each partition.The paper describes GCs and MPS as complementary mechanisms.Source: 2) Driver-Level dynamic sharing · p. 2

1) Static and coarse-grained approaches:

  • Systems combining MIG and MPS improve utilization but remain constrained by static or slowly adapting partitioning decisions.Examples include ParvaGPU, MIGER, MISO, BOER, and KACE.Source: 1) Static and coarse-grained approaches · p. 2

2) Dynamic and fine-grained approaches:

  • Some dynamic sharing systems rely on CUDA interception, application restructuring, or kernel transformation; ScaleRAN instead uses a lightweight runtime library with minimal application modifications.The cited approaches include MMK, Draco, HAMI, and Tally.Source: 2) Dynamic and fine-grained approaches · pp. 2–3

D. GPU Resource Management in AI-RAN

AI-RAN resource management seeks to share GPUs between latency-critical RAN processing and co-located AI applications. LDPC decoding is a representative RAN workload whose changing demand makes deadline-aware SM allocation important.

D. GPU Resource Management in AI-RAN

  • AI-RAN research has used MIG-based co-location and orchestration, while the O-RAN Alliance identifies dynamic allocation and proactive autoscaling among O-Cloud orchestration needs.ScaleRAN addresses these needs with node-local Green Context partitioning and reactive and predictive autoscaling.Source: D. GPU Resource Management in AI-RAN · p. 3

A. RAN Workload: Predominant LDPC Decoding

  • LDPC decoding dominates the PUSCH GPU execution timeline, and its workload scales with offered traffic while code blocks must finish within the slot duration.The paper focuses its simulator and experiments on LDPC behavior because it dominates RAN compute dynamics and has a regular, parallel structure.Source: A. RAN Workload: Predominant LDPC… · p. 3
  • LDPC latency decreases as more SMs are allocated, but gains diminish after available parallelism is exhausted; the SM count needed to meet the slot deadline depends on offered load.Over-provisioning wastes capacity for AI, while under-provisioning risks RAN timing violations.Source: A. RAN Workload: Predominant LDPC… · pp. 3–4

B. AI Workloads

AI inference latency varies with model and input characteristics as GPU SM allocation changes. ScaleRAN shares SMs through pre-created Green Context profiles and selects among them at runtime.

B. AI Workloads

  • CNN and LLM inference latency generally decreases with additional SMs, but scaling depends on model architecture and input characteristics.Long, non-preemptive kernels reduce scheduling opportunities, whereas smaller kernels create more scheduling points at some cost to execution efficiency.Source: B. AI Workloads · p. 4

A. Elastic SM Partitioning with Green Contexts

  • ScaleRAN uses Green Contexts for elastic GPU sharing and MPS for concurrent CUDA execution across RAN and AI processes.The workloads execute in separate SM partitions.Source: A. Elastic SM Partitioning with Green… · p. 4
  • ScaleRAN represents GPU allocations as a finite ladder of profiles, each defining disjoint RAN and AI Green Context partitions, with higher indices assigning more SMs to RAN and fewer to AI.Profiles are created during initialization and runtime changes switch subsequent GPU work to the selected context.Source: A. Elastic SM Partitioning with Green… · p. 4

IV. SCALERAN RUNTIME ARCHITECTURE

  • The runtime architecture routes per-slot RAN telemetry through a ring buffer to a profile selector, which publishes a profile index for the containers to apply.The ScaleRAN library switches the active Green Context and CUDA streams before subsequent GPU work launches.Source: Figure 3 · p. 4

B. Autoscaling Problem

ScaleRAN seeks the minimum RAN GPU allocation that meets real-time deadlines, while allowing the remaining resources to serve a co-located AI workload. Its split-process architecture uses slot telemetry to select and apply GPU profiles.

B. Autoscaling Problem

  • The autoscaler minimizes RAN GPU allocation subject to a decoding-latency deadline and an acceptable deadline-violation probability.The paper focuses on LDPC decoding and one co-located AI workload, while noting the framework’s broader applicability to latency-critical GPU-accelerated RAN functions.Source: B. Autoscaling Problem · p. 4

C. Architecture

  • A RAN worker handles slot-level processing, an AI worker runs best-effort inference, and an autoscaler adjusts SM partitions between them.A lightweight coordinator supports this split-process architecture.Source: C. Architecture · p. 4
  • The coordinator creates the Green Context profile ladder and shared-memory controls during initialization, allowing workers to reconstruct profiles independently.Deterministic profile definitions avoid a centralized CUDA context manager.Source: C. Architecture · p. 4
  • The RAN worker publishes slot reports, and the Profile Selector writes the chosen profile index to shared memory for both workers to apply to subsequent kernels.Reports include measured LDPC decoding latency and may include radio and workload descriptors; profile changes take effect at scheduling boundaries because kernels are non-preemptive.Source: C. Architecture · p. 5

D. Implementation

ScaleRAN connects workers and its autoscaler through shared memory, with a library that supports cooperative GPU-profile switching. The design extends to O-RAN as a node-local O-Cloud service, while the discussion identifies application changes and profile-combination growth as design considerations.

D. Implementation

  • Shared-memory ring-buffer telemetry and a compact control block connect the RAN worker to the Profile Selector without inter-process messaging on the critical control path.The shared-memory regions are hosted by the RAN container and accessed through the ScaleRAN library.Source: D. Implementation · p. 5
  • Applications wrap GPU calls with a library that selects among profile-specific CUDA streams and synchronizes when a profile changes.The cooperative interface requires minor modifications and supports AI libraries including PyTorch, TensorRT, ONNX Runtime, and Hugging Face Transformers.Source: D. Implementation · p. 5

E. Discussion

  • The cooperative library interface is intrusive, and profile switching is limited to synchronization points rather than individual kernel launches.Transparent CUDA interception could enable kernel-granularity switching, but the paper describes it as complex and fragile for modern AI frameworks.Source: E. Discussion · p. 5
  • The architecture is described as extensible to more co-located workloads, with profiles representing complete GPU partitions across workers.The number of profile combinations grows rapidly, so the discussion suggests defining profiles by worker class rather than individual worker.Source: E. Discussion · p. 5

F. O-RAN Integration

  • ScaleRAN runs as a node-local O-Cloud service beside the O-DU and AI containers, keeping slot-level GPU control local while O-RAN interfaces support management.The SMO can configure policies and monitor telemetry through O1, while O2 deploys and manages the service and containers.Source: F. O-RAN Integration · p. 6

V. AUTOSCALING POLICIES

ScaleRAN supports reactive and proactive autoscaling policies. The Kubernetes-inspired reactive controller adjusts RAN GPU allocation using decoding latency and applies different timing rules to upscaling and downscaling.

V. AUTOSCALING POLICIES

  • ScaleRAN considers a Kubernetes-inspired reactive controller and a proactive prediction-based controller as complementary autoscaling approaches.The policies are described in separate subsections.Source: §V · p. 6

A. Kubernetes-Inspired Policy

  • The reactive controller scales RAN SM allocation proportionally to the ratio of measured LDPC decoding latency to the target deadline.It adapts the HPA principle by using SM allocations instead of replicas and decoding latency instead of utilization.Source: A. Kubernetes-Inspired Policy · p. 6
  • Upscaling is immediate, while downscaling waits through a stabilization window and selects the largest profile requested during that window.The controller omits the HPA deadband to respond immediately to increased decoding latency; the window guards against releasing resources after transient load reductions.Source: A. Kubernetes-Inspired Policy · pp. 6–7

B. Proactive Policy

The proactive policy predicts next-slot LDPC latency from radio descriptors and candidate RAN profiles, using a high conditional quantile and conformal calibration to select resources against deadline risk.

B. Proactive Policy

  • The policy selects the smallest candidate profile satisfying its calibrated-latency criterion.The conformal correction is computed from sorted calibration scores for a target miscoverage probability α.Source: B. Proactive Policy · p. 7
  • The predictor estimates a high conditional latency quantile at τ = 1 − ϵ rather than expected latency, using radio descriptors to predict the next slot.The descriptors include allocated PRBs, active UEs, and MCS statistics.Source: B. Proactive Policy · p. 7
  • The model maps radio descriptors and candidate RAN profiles to decoding-latency quantiles, with features intended to characterize workload independently of traffic scenario.It uses a LightGBM gradient-boosted decision tree for nonlinear prediction and low-latency inference.Source: B. Proactive Policy · p. 7
  • The quantile model is trained with Pinball loss, with τ denoting the target quantile level.Source: B. Proactive Policy · p. 7
  • One-sided split conformal calibration adjusts quantile predictions using calibration samples, with coverage of at least 1 − α to limit latency underestimation.Positive calibration scores represent underestimation of decoding time, the direction relevant to deadline protection.Source: B. Proactive Policy · p. 7

VI. PERFORMANCE EVALUATION

The section evaluates the ScaleRAN framework and compares the performance of its autoscaling policies.

VI. PERFORMANCE EVALUATION

  • The evaluation compares the performance of ScaleRAN's autoscaling policies.Source: §VI · p. 7

A. LDPCSim: RAN Workload Generation

LDPCSim generates slot-level GPU decoding workloads from configurable 5G NR scenarios using NVIDIA Aerial cuPHY, then measures decoding performance for ScaleRAN evaluation.

A. LDPCSim: RAN Workload Generation

  • LDPCSim is a Python workload generator built on NVIDIA Aerial that submits configurable slot workloads to cuPHY and collects GPU performance measurements.Source: A. LDPCSim: RAN Workload Generation · p. 7
  • The simulator derives transport-block sizes and LDPC parameters from scheduled-user radio settings, including PRBs, MCS, and MIMO layers.PyAerial performs 3GPP-compliant sizing and segmentation, so workloads arise from valid physical-layer configurations.Source: A. LDPCSim: RAN Workload Generation · pp. 7–8
  • Dynamic scenarios vary traffic patterns and PRB allocation across UEs, changing transport-block sizes, code-block counts, and GPU decoding demand.Fig. 6 depicts aggregate uplink throughput and compares timed peak decoder SM demand with the minimum allocation meeting the 0.5 ms slot deadline; the shaded gap represents potential over-allocation.Source: A. LDPCSim: RAN Workload Generation · p. 8, Figure 6 · p. 8
  • For timed execution, LDPCSim asynchronously dispatches compatible code-block groups on CUDA streams and measures CPU-side latency through kernel completion.This measurement includes kernel-launch and host-device synchronization overheads and serves as autoscaler telemetry and a deadline-compliance measure.Source: A. LDPCSim: RAN Workload Generation · p. 8

B. Evaluation Setup

The evaluation uses a Green Context SM-allocation ladder and disjoint traffic-trace datasets to assess prediction and autoscaling. Conformal calibration restores target coverage, and proactive allocation improves the reliability–overprovisioning trade-off over the reactive controller.

B. Evaluation Setup

  • The DGX Spark evaluation uses a 0.5 ms slot deadline and an 11-profile Green Context ladder assigning RAN 4–44 SMs in increments of 4.Both RAN and AI workloads receive at least 4 SMs.Source: B. Evaluation Setup · p. 8
  • Policy evaluation uses eight unseen 10,000-slot traces, reserving each trace’s first 500 slots for conformal calibration and the remaining 9,500 for evaluation.The separate training dataset contains 105 traces of 3,000 slots and 3,465,000 supervised samples.Source: B. Evaluation Setup · p. 8

C. Decoding-latency Predictor

D. Autoscaling Results

  • K8S-React uses prior-slot measured latency, K8S-Proact substitutes predicted latency under the same proportional control, and Proactive selects SM allocation directly from predicted latency.Predictive variants use mean, quantile, or conformal-quantile predictions.Source: D. Autoscaling Results · p. 9
  • 99.96% deadline success with 4.80 mean SM regret: Proactive nearly matches K8S-Proact’s 99.95% success while reducing regret 45% from 8.69 to 4.80.The comparison uses conformal P99 prediction; the proportional controller and stabilization window provide little benefit in this setting.Source: D. Autoscaling Results · p. 9

E. AI and RAN interferences

AI–RAN interference depends on both AI-kernel SM footprint and execution duration relative to slot boundaries. ScaleRAN’s conclusion presents fine-grained GPU sharing as feasible for AI–RAN, particularly in O-RAN O-Cloud deployments.

E. AI and RAN interferences

  • Interference depends on the combination of AI-kernel SM footprint and execution duration relative to ScaleRAN’s control timescale, not inference latency or footprint alone.The 40-layer MLP with 2048 hidden units causes severe interference, while the 3-layer, 2048-unit MLP maintains high deadline success because its inference time is below the 0.5 ms slot duration.Source: E. AI and RAN interferences · p. 9, Figure 10 · p. 10

VII. CONCLUSION AND PERSPECTIVES

  • ScaleRAN combines Green Context-based SM partitioning with reactive and prediction-based autoscaling; the paper concludes fine-grained GPU sharing is feasible for AI–RAN and suited to O-RAN O-Cloud deployments.LDPCSim, built on NVIDIA Aerial, supports reproducible evaluation with representative 5G NR LDPC decoding workloads.Source: §VII · p. 9
  • The authors identify memory management, AI-workload profiling, and Kubernetes integration as future extensions beyond the current SM-allocation focus.Profiling kernel granularity and SM footprint is intended to help synchronize profile transitions with kernel launch boundaries.Source: §VII · p. 10
Loading 2610.23072v1…