PaperlayerView exact revision on arXiv

Source-linked AI summary

SLA-Safe Energy Control for AI-Native NG-RAN Using Stability-Aware Constrained PPO

Dharmendra Kumar

arXiv:2609.05861v1cs.NIcs.LGeess.SP

TL;DR

Energy-saving control in NG-RAN must reduce power without compromising QoS or SLA requirements under changing traffic. The paper proposes stability-aware constrained PPO with mixed-regime training. In a seven-cell simulation, it reports energy reductions across nominal, stress, and unseen-stress traffic, with zero SLA violation and throughput loss in the latter two scenarios.

  • Problem

    Aggressive energy-saving actions can reduce power while degrading throughput, increasing delay, violating SLAs, or destabilizing mode switching, especially under bursty or shifting traffic.

    Source: §I · p. 1

  • Method

    The framework formulates energy control as a constrained problem and uses adaptive-penalty PPO, nominal-and-stress training, and a switching-stability penalty.

    Source: §I · pp. 1–2

  • Results

    Energy consumption fell by approximately 41.4% under nominal, 10.5% under stress, and 22.9% under unseen-stress traffic relative to Always-On; stress and unseen-stress had zero SLA violation and zero throughput loss.

    Source: §I · p. 2, A. Overall Performance · p. 5

  • Takeaways & Limitations

    In the seven-cell simulation, the proposed controller balances energy saving with QoS and SLA preservation across nominal, stress, and unseen-stress traffic.

    Source: A. Overall Performance · p. 5, §IX · p. 6

Abstract

from arXiv · show

One important AI-for-RAN use case is energy saving, in which radio resources and cell energy modes must be dynamically controlled without violating user quality-of-service (QoS) or service-level agreement (SLA) requirements. However, aggressive sleep-state or deactivation decisions may reduce energy consumption at the cost of throughput degradation, delay increase, SLA violations, and unstable mode switching, especially under time-varying and bursty traffic conditions. This paper proposes a stability-aware constrained reinforcement learning framework for SLA-safe energy control in 5G NG-RAN. The problem is formulated as a constrained Markov decision process in which an AI-native controller selects closed-loop energy-saving actions based on cell load, queue status, active-user information, current energy mode, and SLA-related indicators. The proposed framework uses constrained proximal policy optimization with adaptive Lagrangian penalties to account for throughput, delay, and SLA constraints. To improve operation under traffic distribution shift, the controller is trained using mixed nominal and stress traffic regimes, while a switching-stability penalty is introduced to reduce oscillatory transitions between active and low-power modes. Simulation results in a seven-cell NG-RAN environment show that the proposed controller reduces energy consumption by approximately 41.4% under nominal traffic, 10.5% under stress traffic, and 22.9% under unseen-stress traffic relative to the Always-On baseline. Under stress and unseen-stress traffic, the controller preserves zero SLA violation and zero throughput loss, indicating service-preserving operation under challenging conditions. The proposed method also reduces switching activity compared with basic threshold-based energy saving.

I. INTRODUCTION

Energy-saving control must balance power reduction with QoS, SLA, and switching stability under changing traffic. The paper proposes constrained, stability-aware learning and reports energy savings across nominal, stress, and unseen-stress scenarios.

I. INTRODUCTION

  • Energy control must avoid throughput degradation, delay increases, SLA violations, and unstable mode transitions as traffic becomes bursty or shifts from nominal conditions.Aggressive sleep or deactivation decisions can reduce energy while worsening service outcomes.Source: §I · p. 1

II. RELATED WORK AND GAP

  • Fixed-threshold and reward-only methods may miss changing traffic patterns or save energy while violating delay, throughput, or SLA requirements.The paper motivates explicit constraints and switching stability as part of closed-loop energy control.Source: §I · p. 1, §II · p. 2

I. INTRODUCTION

  • The framework combines constrained PPO with adaptive Lagrangian penalties, mixed nominal-and-stress training, and a switching-stability penalty.The constraints address throughput, delay, and SLA requirements.Source: §I · p. 1
  • Approximately 41.4%, 10.5%, and 22.9% energy reductions were achieved under nominal, stress, and unseen-stress traffic, respectively, relative to Always-On.The evaluation uses a seven-cell NG-RAN simulation.Source: §I · p. 2

II. RELATED WORK AND GAP

  • The framework extends switching-cost energy-saving work by treating SLA, delay, and throughput as explicit constraints and training across nominal and stress conditions.The related-work discussion identifies traffic-shift robustness and mode-switching stability as challenges for constrained RAN energy control.Source: §II · p. 2

III. AI-NATIVE RAN CONTROL ARCHITECTURE

The architecture uses telemetry-driven closed-loop control to select cell energy modes and evaluate their energy, service, and stability effects. It combines explicit QoS and SLA constraints with switching-aware control and mixed-regime training.

A. Closed-Loop Control View

  • At each decision epoch, telemetry including cell load, queues, active users, current energy mode, and QoS or SLA indicators informs per-cell energy-mode actions.Actions include active, light-sleep, and deep-sleep modes; feedback covers energy, throughput, delay, SLA violations, throughput loss, and switching activity.Source: A. Closed-Loop Control View · p. 2
  • The controller treats throughput, delay, and SLA requirements as explicit operational constraints rather than only as soft reward penalties.A low-power choice may reduce energy immediately while increasing later delay or SLA risk if demand rises.Source: A. Closed-Loop Control View · p. 2
  • Switching-aware regularization discourages unnecessary transitions between active and low-power modes.The architecture includes switching stability among the feedback measures.Source: §III · p. 2, A. Closed-Loop Control View · p. 3
  • Training on nominal and stress traffic episodes targets traffic distribution shift while supporting energy reduction in regular-load conditions and service-preserving operation under stress.Source: A. Closed-Loop Control View · p. 3

IV. PROBLEM FORMULATION

The formulation treats NG-RAN energy saving as a closed-loop decision problem that seeks lower long-term energy consumption while preserving QoS, SLA compliance, and switching stability.

IV. PROBLEM FORMULATION

  • The controller’s objective is to reduce long-term RAN energy consumption while preserving QoS, maintaining SLA compliance, and avoiding unstable energy-mode switching.Source: §IV · p. 3

A. System Model

  • The system model considers a multi-cell NG-RAN with B controllable cells and discrete decision epochs indexed by t.Source: A. System Model · p. 3
  • Each cell b has an energy mode m_b(t), and its energy use depends on that mode and its load L_b(t).Source: A. System Model · p. 3

B. State and Action Spaces

C. Objective and Service Constraints

The objective and service constraints frame energy saving as a trade-off: energy-minimizing actions must remain within throughput, delay, and SLA requirements.

C. Objective and Service Constraints

D. Reward, Constraint Costs, and CMDP

The framework balances energy efficiency, service utility, and switching stability while expressing service requirements as cumulative constraints with allowable budgets.

D. Reward, Constraint Costs, and CMDP

V. STABILITY-AWARE CONSTRAINED PPO FRAMEWORK

The proposed framework combines PPO-based policy learning with adaptive constraint handling, mixed-regime training, and switching-aware regularization.

V. STABILITY-AWARE CONSTRAINED PPO FRAMEWORK

  • The stability-aware constrained PPO framework combines PPO-based policy learning, adaptive Lagrangian constraint handling, mixed-regime training, and switching-aware regularization.Source: §V · p. 4

A. Framework Overview

The framework treats NG-RAN energy saving as closed-loop control, using network feedback to select cell energy modes. Its PPO update combines explicit QoS/SLA constraints with a switching penalty.

A. Framework Overview

  • The controller selects energy modes from network states and updates its policy using closed-loop interaction feedback.Each applied action returns a reward, constraint costs, switching cost, and next state.Source: A. Framework Overview · p. 4
  • Adaptive Lagrangian penalties handle QoS and SLA requirements as explicit constraints, while the reward discourages unnecessary transitions between active and low-power modes.The multipliers increase penalties when empirical constraint costs exceed their budgets and reduce pressure when the policy remains within feasible limits.Source: A. Framework Overview · p. 4, B. Constrained PPO Update · p. 4

B. Constrained PPO Update

  • The constrained PPO objective uses empirical constraint costs, corresponding budgets, and Lagrange multipliers alongside the standard PPO clipped surrogate objective.The multiplier update adjusts constraint pressure according to whether costs exceed allowable budgets.Source: B. Constrained PPO Update · p. 4

C. Switching Stability and Mixed-Regime Training

The framework penalizes unnecessary energy-mode changes and trains across nominal and stress traffic to address shifting demand. Its evaluation uses a seven-cell simulation, three traffic scenarios, and energy, service, SLA, and switching measures.

C. Switching Stability and Mixed-Regime Training

  • A switching penalty discourages unnecessary transitions because frequent active-to-low-power changes may create oscillatory behavior and service fluctuation.The controller is evaluated in closed loop without further parameter updates under nominal, stress, and unseen-stress scenarios.Source: C. Switching Stability and… · p. 4
  • Mixed-regime training samples nominal and stress episodes so the controller encounters regular load variation and sudden demand increases.Unseen-stress traffic, with different burst intensity, duration, or hotspot location, is reserved for evaluation.Source: C. Switching Stability and… · p. 4, A. Simulation Environment and Traffic… · p. 5

A. Simulation Environment and Traffic Scenarios

B. Energy, Service, and Baseline Models

  • Deep-sleep uses 0.30 normalized energy with a 0.35 capacity factor, while SLA violations are based on delay or throughput-loss thresholds.Throughput loss is measured relative to Always-On; comparisons include rule-based baselines and learning-based ablations for constraints, switching penalty, and training regime.Source: B. Energy, Service, and Baseline Models · p. 5

C. Training Configuration and Evaluation Metrics

The evaluation reports normalized energy, throughput, delay, SLA violations, throughput loss, and switching activity across traffic scenarios. The paper also reports training time and per-cell inference latency for its controller.

C. Training Configuration and Evaluation Metrics

  • The learning-based baselines use the same network architecture and training budget, with two hidden layers of 64 units each.Unless otherwise stated, the reported setup uses a 3×10^-4 learning rate and a 0.99 discount factor.Source: C. Training Configuration and… · p. 5
  • All methods are evaluated on energy, throughput, delay, SLA violation rate, throughput loss versus Always-On, and switching activity, averaged over five random seeds.The stated effectiveness criteria are nominal energy reduction, near-Always-On QoS under stress and unseen-stress traffic, and avoiding excessive switching.Source: C. Training Configuration and… · p. 5

D. Computational Considerations

  • The two-layer, 64-unit MLP actor has a measured mean inference latency of 0.74 ms per cell on CPU.Training for 200,000 timesteps takes approximately 14 minutes on a single CPU core; inference latency is reported as negligible relative to the 1-second control interval.Source: D. Computational Considerations · p. 5

VII. RESULTS AND DISCUSSION

  • The results discussion evaluates the tradeoff among energy consumption, QoS preservation, SLA compliance, throughput loss, and switching activity across nominal, stress, and unseen-stress traffic.Source: §VII · p. 5

A. Overall Performance

The proposed controller saves 41.4%, 10.5%, and 22.9% energy versus Always-On across nominal, stress, and unseen-stress traffic; stress and unseen-stress retain zero SLA violations and throughput loss. Ablations show unsafe or failing alternatives, while the evaluation remains bounded by simplified traffic and channel assumptions.

A. Overall Performance

  • 41.4%, 10.5%, and 22.9% energy saving versus Always-On under nominal, stress, and unseen-stress traffic, respectively; stress and unseen-stress retain zero SLA violations and zero throughput loss.Nominal traffic has an SLA violation rate below 1%; unseen-stress delay equals the Always-On reference.Source: A. Overall Performance · p. 5, Table II · p. 6

B. Ablation Study

  • Unconstrained-PPO is unsafe despite lower energy and Nominal-Only-PPO fails under stress and unseen-stress; constrained variants retain zero SLA violations, and switching regularization reduces switching versus Constrained-PPO-NoSwitch.Source: Table III · p. 6, B. Ablation Study · p. 6

A. Overall Performance

  • The proposed method reduces switching versus Threshold-ES across scenarios while maintaining SLA-safe operation; threshold methods may save more energy in some cases but are less adaptive.Threshold methods may switch frequently or behave conservatively under stress traffic.Source: A. Overall Performance · p. 6

IX. CONCLUSION

  • The framework combines adaptive Lagrangian penalties, mixed-regime training, and switching-cost regularization in constrained closed-loop energy control.The evaluation is described as achieving a balanced energy-QoS-stability tradeoff across nominal, stress, and unseen-stress traffic.Source: §IX · p. 6
  • The evaluation uses simplified traffic and channel assumptions; future work will test more realistic 5G NR channels, including fading, dynamic uplink variation, and dense-urban interference.Source: §IX · p. 6
Loading 2609.05861v1…