PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic RAN Slicing for Service-Oriented Vehicular Networks via Constrained Learning

Wen Wu, Nan Chen, Conghao Zhou, Mushu Li, Xuemin Shen, Weihua Zhuang, Xu Li

arXiv:2012.01991v1cs.LGcs.NI

TL;DR

The paper addresses RAN slicing for vehicular networks by considering both resource allocation and computation workload distribution. It proposes the two-layer constrained RL algorithm RAWS, which reduces system cost while satisfying QoS requirements with high probability, including under heavy traffic.

  • Problem

    RAN slicing must account for both resource allocation and computation workload distribution among slices.

  • Method

    The paper presents a dynamic RAN slicing framework and proposes the two-layer constrained RL algorithm RAWS.

  • Results

    RAWS effectively reduces system cost, especially under heavy traffic, while satisfying QoS requirements with high probability.

  • Takeaways & Limitations

    Unlike traditional optimization-based methods, RAWS adapts to time-varying vehicle traffic density while supporting QoS requirements.

Abstract

from arXiv · show

In this paper, we investigate a radio access network (RAN) slicing problem for Internet of vehicles (IoV) services with different quality of service (QoS) requirements, in which multiple logically-isolated slices are constructed on a common roadside network infrastructure. A dynamic RAN slicing framework is presented to dynamically allocate radio spectrum and computing resource, and distribute computation workloads for the slices. To obtain an optimal RAN slicing policy for accommodating the spatial-temporal dynamics of vehicle traffic density, we first formulate a constrained RAN slicing problem with the objective to minimize long-term system cost. This problem cannot be directly solved by traditional reinforcement learning (RL) algorithms due to complicated coupled constraints among decisions. Therefore, we decouple the problem into a resource allocation subproblem and a workload distribution subproblem, and propose a two-layer constrained RL algorithm, named Resource Allocation and Workload diStribution (RAWS) to solve them. Specifically, an outer layer first makes the resource allocation decision via an RL algorithm, and then an inner layer makes the workload distribution decision via an optimization subroutine. Extensive trace-driven simulations show that the RAWS effectively reduces the system cost while satisfying QoS requirements with a high probability, as compared with benchmarks.

I. INTRODUCTION

The paper addresses dynamic RAN slicing for heterogeneous IoV services by jointly allocating resources and distributing workloads under coupled QoS and capacity constraints. It proposes RAWS, a two-layer constrained RL method that reduces long-term system cost while maintaining QoS satisfaction under changing traffic density.

  • Motivation: IoV services have heterogeneous QoS requirements, motivating multiple logically isolated slices over shared roadside infrastructure.Examples include delay-sensitive cooperative sensing, delay-tolerant HD-map creation, and high-throughput video streaming.
  • Problem: Existing RAN slicing work focuses mainly on slice resource allocation, while computation workload distribution remains insufficiently considered.The paper identifies workload distribution as necessary when uneven traffic overloads some BSs and leaves others underutilized.
  • Problem: The slicing decisions are coupled because resource allocation changes BS resource availability, while workload distribution changes task assignments and future resource needs.The problem also involves heterogeneous QoS constraints, spatial-temporal traffic dynamics, and decisions made without a priori future traffic information.
  • Approach: RAWS decouples the problem into resource allocation and workload distribution subproblems, using an outer RL layer and an inner convex optimization routine.This design addresses the constraint-coupling difficulty that can cause traditional RL algorithms to violate constraints.
  • Approach: The paper formulates constrained RAN slicing as a stochastic optimization problem minimizing long-term overall system cost.The formulation jointly targets resource allocation and workload distribution while satisfying coupled and resource-capacity constraints.
  • Results: Trace-driven simulations show that RAWS reduces overall system cost relative to DDPG and TD3 while satisfying QoS constraints with high probability, including under heavy traffic.The method is designed to adapt to time-varying vehicle traffic density.

II. RELATED WORK

Prior work spans core and RAN slicing, resource sharing, caching, computing, and RL-based network control. The paper distinguishes its approach by addressing workload balancing and coupled constraints in dynamic vehicular RAN slicing.

  • RAN slicing background: RAN slicing differs from core network slicing and remains an emerging area for vehicular networks.Prior systems include standardized and proof-of-concept RAN slicing architectures.
  • Resource allocation: Prior studies also investigate communication-resource allocation, computing-resource allocation, and hierarchical resource sharing among slices.Examples include spectrum allocation for delay-oriented services, fog-computing allocation, and soft hierarchical RAN slicing.
  • Existing vehicular studies: Existing vehicular RAN slicing studies address diverse services, SDN orchestration, caching, and resource multiplexing across roadside infrastructure.These works include architectures jointly orchestrating communication, caching, and computing resources, plus soft slicing for roadside caching.
  • Workload distribution: Unlike existing work focused on resource allocation among slices, this paper addresses spatially uneven traffic through workload distribution for BS workload balance.The mechanism distributes workload across BSs to account for uneven vehicle density.
  • Learning-based approaches: Machine-learning research has applied prediction, virtual-network-function deployment, traffic scheduling, caching, and mode selection to network slicing.RL methods have been used for time-varying channels, unknown content popularity, and dynamic resource allocation.
  • Positioning: This work targets constrained RAN slicing with coupled constraints that traditional RL cannot directly solve, proposing a two-layer constrained RL algorithm.Its broader framework models zones, BS associations, workload fractions, and MEC-enabled offloading along a road segment.

B. Dynamic RAN Slicing Framework

The framework supports two IoV service slices with distinct QoS requirements and dynamically coordinates resource allocation and workload distribution across time-varying traffic conditions.

  • Scope: The framework is presented for two services but can be extended to multiple delay-sensitive and delay-tolerant services by increasing decision dimensions.The paper specifically considers K = |K| = 2 services in its core formulation.
  • Service and QoS model: Two services are modeled: a delay-sensitive service with a maximum-delay requirement and a delay-tolerant service requiring queue stability.The framework mainly uses delay as the QoS metric and constructs separate slices for the two service types.
  • Time-slotted operation: The framework operates in slicing windows where traffic density is treated as stationary and decisions remain unchanged within each window.The SDN controller observes current average vehicle density, makes slicing decisions, and evaluates performance from base-station feedback.
  • Resource allocation: Resource allocation covers radio spectrum and computing resources for every slice and base station.Spectrum is allocated in subcarriers, while computing capacity is allocated in VM instances with specified CPU frequency.
  • Workload distribution: Workload distribution assigns each service's zone workloads across base stations, and these decisions are dynamically updated as traffic density changes.The framework represents allocation and distribution decisions with matrices and updates them in each slicing window.

1) Delay-sensitive service:

The QoS model represents service delay through offloading, processing, and handover components, while delay-tolerant services are constrained by offloading and processing queue stability.

  • Delay-sensitive service: For delay-sensitive service u, total service delay comprises task offloading delay, task processing delay, and handover delay.The offloading and processing components are modeled using M/M/1 queueing analysis, while handover delay accounts for movement across base stations.
  • Modeling assumptions: The delay-sensitive queue model assumes Poisson task arrivals and exponential transmission and computation-related service times.The paper also assumes negligible downlink latency because output results are much smaller than input tasks.
  • Delay-sensitive service: The delay-sensitive service requires average service delay to remain below its maximum tolerable delay threshold.Radio spectrum allocation and workload distribution are coupled because both affect task-offloading queue stability.
  • Delay-tolerant service: For delay-tolerant service e, QoS is enforced by maintaining stability of both task-offloading and task-processing queues.The corresponding constraints limit allocated radio and computing resources relative to workload arrival rates.

D. Overall System Cost Model

The overall system cost combines resource usage, slice reconfiguration, delay-violation penalties, and service revenue to evaluate dynamic RAN slicing policies under spatial-temporal traffic variation.

  • Cost components: The system cost includes operation cost, slice reconfiguration cost, delay constraint violation cost, and system revenue in each slicing window.These components jointly evaluate the performance of a RAN slicing policy.
  • 1) Operation cost:: Operation cost accounts for allocating subcarriers and VM instances, with separate unit costs for radio spectrum and computing resources.The unit costs are intended to be equivalent across the two resource types.
  • 2) Slice reconfiguration cost:: Slice reconfiguration cost measures increases in resource allocation between consecutive slicing windows and discourages frequent reconfiguration.Resource release is omitted because it is assumed to be fast and have negligible cost, while reconfiguration unit costs exceed usage costs.
  • 3) Delay constraint violation cost:: Delay constraint violation cost penalizes service delay exceeding the maximum tolerable threshold through an indicator-based penalty.The violation unit cost is set extremely high to penalize constraint violations.
  • Problem formulation: The optimization jointly allocates spectrum and computing resources and distributes workloads to minimize average long-term system cost online under capacity and QoS constraints.The problem is stochastic because vehicle density varies spatially and temporally, and resource and workload decisions are coupled.
  • Problem formulation: Traditional RL may violate the coupled constraints because it makes resource allocation and workload distribution decisions separately.The paper therefore proposes a two-layer constrained RL algorithm after decoupling the original problem.

IV. TWO-LAYER CONSTRAINED RL ALGORITHM

The two-layer constrained RL algorithm decouples resource allocation from workload distribution, using an outer RL layer and an inner optimization procedure to address their coupled constraints.

  • Algorithm structure: The proposed algorithm separates the problem into an outer resource allocation subproblem and an inner workload distribution subproblem.The two layers are solved together as a constrained RL procedure.
  • Inner layer: The workload distribution objective minimizes instantaneous service delay while satisfying the applicable queue-stability and feasibility constraints.Separate workload variables can be optimized for each considered service.
  • Implementation: The workload distribution subproblem can be solved with convex optimization solvers such as CVX and Gurobi.For the delay-sensitive service, the one-shot problem is identified as convex; the delay-tolerant case is a simplified version.
  • Feasibility handling: The inner workload distribution problem becomes infeasible when allocated resources cannot satisfy queue stability constraints.The outer-layer algorithm addresses this infeasibility through a penalty mechanism.
  • Inner layer: Given a resource allocation decision, the inner layer solves workload distribution through convex optimization when the feasible region is non-empty.The workload distribution subproblem can be converted into independent one-shot problems because its relevant costs and constraints are independent across slicing windows.

B. Outer Layer: Resource Allocation Subproblem

The outer layer formulates resource allocation as an infinite-horizon MDP that allocates radio spectrum and computing resources under dynamic vehicle-density conditions. Model-free RL is used because future traffic information and state-transition probabilities are unavailable, while continuous states prevent conventional model-based solutions.

  • The resource allocation subproblem allocates radio spectrum and computing resources to minimize long-term system cost while satisfying constraints.
  • The SDN controller observes the current network state, selects a resource allocation action, receives a reward, and transitions to a new state.
  • The state represents vehicle density across all zones in the slicing window and is continuous.
  • The reward combines long-term system-cost minimization with service-delay evaluation and penalties when workload distribution is infeasible.
  • Unknown traffic dynamics and continuous state space make conventional model-based methods unsuitable, motivating model-free RL for resource allocation.

C. RAWS Algorithm

RAWS decouples coupled resource and workload decisions into two layers: an actor network chooses resource allocation, and an optimization subroutine computes workload distribution. A softmax-based actor enforces resource-capacity constraints while neural networks learn from replayed experience.

  • RAWS uses a two-layer constrained RL architecture to make joint decisions while satisfying coupled constraints.
  • The outer actor network selects resource allocation a1, while the inner optimization subroutine computes workload distribution a2 from the state and a1.
  • The softmax-based actor activates a (K + 1)-dimension vector and uses K outputs as resource decisions satisfying capacity constraints.
  • RAWS generates experience tuples, stores them in a replay buffer, and updates critic, actor, and target networks from minibatches.

V. SIMULATION RESULTS

The evaluation uses real-world highway and urban traffic traces, a roadside network with five base stations, two IoV services, and benchmarks including shaped DDPG, shaped TD3, a simplified RAWS, and random feasible decisions.

  • Simulation traces: The simulations use highway and urban vehicle traffic traces converted to vehicle density for evaluation.
  • Network setting: The simulated road segment is 5 km long, divided into 25 zones, with five base stations separated by 1 km.
  • Service setting: The services are cooperative sensing with a 100 ms delay limit and HD map creation as a delay-tolerant service.
  • Benchmarks: RAWS is compared with decision-shaped DDPG, decision-shaped TD3, RAWS without workload distribution, and random feasible decisions.

B. Performance Evaluation Over Highway Vehicle Traffic Flow Trace

Across highway-trace experiments, RAWS converges to lower system cost and delay-violation probability than the benchmarks. Its cost advantage grows with longer evaluations and heavier task traffic, while operation cost remains lower.

  • Convergence: RAWS achieves lower overall system cost than all benchmarks after the learning-based algorithms converge.
  • Convergence: The delay-constraint violation probability decreases from more than 20% to approximately 0.6% during training, below the benchmark.
  • Cumulative cost: RAWS reduces cumulative one-day system cost by approximately 11.5% relative to the best benchmark.
  • Arrival-rate analysis: The performance gain over benchmarks increases with task arrival rate, especially under heavy traffic such as λu = 1.2.
  • Operation cost: RAWS attains about 13% average performance gain over different task arrival rates for λe = 1 compared with DDPG.

3) Service delay:

RAWS reduces system cost while maintaining QoS with high probability, using workload distribution to balance uneven workloads across base stations. Across highway and urban traces, it achieves lower cost than benchmarks and improves as traffic becomes heavier.

  • Service delay: RAWS has an average service delay of 70.9 ms, compared with 56.8 ms for DDPG and 61.8 ms for TD3.Its delay is closer to the 100 ms maximum tolerable constraint because network resources are more efficiently utilized.
  • QoS constraint satisfaction: RAWS reduces QoS constraint violation probability to 0.35% on average across task arrival rates, below the benchmark algorithms.A violation occurs when delay exceeds 100 ms or queue-stability constraints are violated.
  • Workload distribution: Workload distribution balances spatially uneven workloads by moving vehicle workloads from overloaded to underutilized base stations.Without workload distribution, RAWS performs worse than the learning-based benchmarks.
  • Workload distribution: When λe = 1, workload distribution raises the performance gain from 15.2% at λu = 0.8 to 24.2% at λu = 1.2.The gain increases with task arrival rate because workload distribution better utilizes network resources, especially under heavy traffic.
  • Urban traffic evaluation: On the urban traffic trace, RAWS achieves the minimum cumulative system cost and reduces one-day cost by approximately 9.2% versus the best benchmark.It also guarantees the 100 ms service-delay constraint with very high probability.
  • Algorithm design: RAWS combines reinforcement learning and optimization to adapt to time-varying vehicle traffic density without future information.Its two-layer design makes resource-allocation and workload-distribution decisions for dynamic RAN slicing.
  • Future work: Future work includes optimizing the slicing-window size and developing a distributed, low-complexity learning algorithm for large-scale vehicular networks.These directions define the stated scope boundary for the current implementation.

APPENDIX

The appendix establishes that the resource-allocation subproblem is convex by showing a nonnegative second derivative under stability constraints and linear problem constraints.

  • Convexity proof: The objective function is rewritten using the definitions of ωn,s,u and ωn,c,u before analyzing its second derivative.This reformulation supports the convexity proof for resource allocation problem P1,u.
  • Convexity proof: Because βm,u ≥ 0, the objective has a nonnegative second derivative when the stability constraints are satisfied.This establishes convexity of the objective over the feasible region.
  • Convexity proof: Since the constraints are linear, P1,u is a convex optimization problem.The appendix concludes the stated theorem from the convex objective and linear constraints.
Loading 2012.01991v1…