ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms

Michele Polese; Leonardo Bonati; Salvatore D'Oro; Stefano Basagni; Tommaso Melodia

Source-linked AI summary

ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms

Michele Polese, Leonardo Bonati, Salvatore D'Oro, Stefano Basagni, Tommaso Melodia

arXiv:2112.09559v2Dec 17, 2021cs.NIcs.LG

TL;DR

ML-based Open RAN automation is constrained by limited large-scale datasets and experimental infrastructure. The paper introduces ColO-RAN, an SDR-enabled O-RAN framework for developing and evaluating DRL xApps, and reports benefits of adaptive control alongside challenges of live online training.

Problem
Large-scale datasets and testing infrastructure for ML-based O-RAN control remain unavailable, limiting experimental development of DRL agents on real networks.
Method
The paper builds ColO-RAN by integrating virtualized O-RAN components, Colosseum’s wireless data factory, SDRs, and DRL-based xApps for closed-loop control.
Results
The large-scale evaluation finds adaptive control more effective than static configurations and examines DRL design choices and online training across scheduling and slicing xApps.
Takeaways & Limitations
ColO-RAN and its dataset provide an experimental foundation for studying scalable ML-enabled closed-loop control in next-generation RANs.
Takeaways & Limitations
Online training on a production RAN may temporarily increase throughput variability and reduce end-user quality of service.

Abstract

from arXiv · show

In spite of the new opportunities brought about by the Open RAN, advances in ML-based network automation have been slow, mainly because of the unavailability of large-scale datasets and experimental testing infrastructure. This slows down the development and widespread adoption of Deep Reinforcement Learning (DRL) agents on real networks, delaying progress in intelligent and autonomous RAN control. In this paper, we address these challenges by proposing practical solutions and software pipelines for the design, training, testing, and experimental evaluation of DRL-based closed-loop control in the Open RAN. We introduce ColO-RAN, the first publicly-available large-scale O-RAN testing framework with software-defined radios-in-the-loop. Building on the scale and computational capabilities of the Colosseum wireless network emulator, ColO-RAN enables ML research at scale using O-RAN components, programmable base stations, and a "wireless data factory". Specifically, we design and develop three exemplary xApps for DRL-based control of RAN slicing, scheduling and online model training, and evaluate their performance on a cellular network with 7 softwarized base stations and 42 users. Finally, we showcase the portability of ColO-RAN to different platforms by deploying it on Arena, an indoor programmable testbed. Extensive results from our first-of-its-kind large-scale evaluation highlight the benefits and challenges of DRL-based adaptive control. They also provide insights on the development of wireless DRL pipelines, from data analysis to the design of DRL agents, and on the tradeoffs associated to training on a live RAN. ColO-RAN and the collected large-scale dataset will be made publicly available to the research community.

1 INTRODUCTION

Future cellular networks must support heterogeneous service requirements through programmable, closed-loop, data-driven control. The paper addresses practical ML deployment challenges by introducing ColO-RAN and evaluating DRL-based xApps at scale.

Heterogeneous applications impose different latency, reliability, throughput, and packet-loss requirements on cellular networks.Examples include URLLC for autonomous driving and high-rate multimedia services that tolerate greater latency and packet loss.
Future RANs combine programmable virtualized protocol stacks, closed-loop control, and data-driven ML.Telemetry enables reconfiguration of cellular nodes according to current network conditions and traffic.
Open interfaces connect disaggregated RAN units to RICs, where xApps execute custom control logic.The architecture replaces monolithic RAN control with open, programmable, virtualized components.
ML-driven xApps face unresolved challenges in collecting large-scale datasets and deploying control solutions across complex wireless systems.The paper identifies dataset collection and meaningful feature selection as practical challenges for scalable RAN automation.
ColO-RAN provides an open, large-scale experimental O-RAN framework that combines SDR-based data collection with DRL testing.It uses Colosseum as a wireless data factory and deploys ML models as xApps through O-RAN interfaces.
Three xApps control RAN scheduling and slicing and support online DRL training on live environments.The agents and autoencoders are trained using 3.4 GB of data containing more than 73 hours of live RAN traces, with evaluation involving 49 nodes.
The evaluation highlights adaptive control over static configurations, DRL design effects on network performance, and the importance of online training.These conclusions are presented as key takeaways from the large-scale experiments.

2 MACHINE LEARNING FOR THE OPEN RAN

O-RAN supports an ML workflow spanning data collection, model design, training, deployment, and runtime control. Its open interfaces and RICs organize this workflow, but large-scale datasets and testing facilities remain essential.

ML deployment in wireless networks includes data collection, model design, offline or online training, and runtime inference and control.O-RAN is designed to support this multi-step deployment process through open interfaces.
O-RAN uses open standardized interfaces to connect interoperable equipment with RAN controllers.The architecture introduces non-real-time and near-real-time RICs for control procedures at different time scales.
The non-real-time RIC collects bulk data through O1, while the near-real-time RIC uses E2 for KPM reporting and parameter control.The two interfaces support distinct data and control functions in the O-RAN architecture.
O-RAN’s ML workflow has five steps: data collection, model design, training and testing, xApp deployment, and runtime inference and control.These steps define the paper’s organizing workflow for ML applications in O-RAN.
Model design selects RAN input parameters and uses collected data organized by the non-real-time RIC.The workflow begins with data from different RAN configurations and setups, including metrics such as throughput and latency.

3 COLO-RAN: ENABLING LARGE-SCALE ML RESEARCH WITH O-RAN AND COLOSSEUM

ColO-RAN addresses the shortage of scalable ML datasets and testing infrastructure by integrating virtualized O-RAN components with Colosseum. Its containerized design supports large-scale experimentation and portability across testbeds.

Researchers lack comprehensive datasets and scalable testing facilities, often relying on small lab datasets that may not generalize to real deployments.Commercial networks are difficult to dedicate to ML training and testing because testing must be separated from service without degrading performance.
ColO-RAN is a large-scale infrastructure built on Colosseum for training, deploying, and testing wireless ML solutions.The platform combines an emulated wireless environment with an O-RAN implementation and designed xApps.
Colosseum combines programmable SDR hardware, emulated propagation, traffic generation, and substantial computing resources as a wireless data factory.Its 256 USRP X310 SDRs, channel-emulation FPGAs, and over 170 servers support heterogeneous scenario generation and dataset collection.
ColO-RAN provides an end-to-end environment for large-scale data collection, heterogeneous testing, and deployment on a 3GPP-compliant RAN.Its virtualized, container-based implementation has a lightweight footprint and can be ported to platforms such as Arena.
The infrastructure includes SMO resources for ML training and storage, a near-real-time RIC, E2 connectivity, and RAN base stations.The near-real-time RIC is deployed as a standalone Linux container and supports multiple xApps interacting with base stations.
RAN slicing supports eMBB, MTC, and URLLC services by controlling PRB masks and selecting among three scheduling policies.The SCOPE framework implements slicing during scheduling and permits independent scheduling-policy choices for each slice.

4 XAPP DESIGN FOR DRL-BASED CONTROL

The xApps connect O-RAN control interfaces with ML infrastructure, using PPO-based DRL agents and autoencoders to process RAN data for closed-loop control. Different xApps vary their observations, actions, rewards, training data, and offline or online training setup.

xApp architecture: Each xApp receives RAN data and controls nodes through custom service models, selecting slicing and scheduling policies.The available actions include choosing PRB allocations among slices and selecting the scheduler used for each slice.
xApp architecture: The xApp implementation contains a RIC interface for service-model communication and an ML infrastructure containing autoencoders and DRL agents.The RIC interface performs ASN.1 encoding and decoding of RAN data and control.
DRL agent design: PPO agents use actor-critic learning, with the actor selecting actions from network states and the critic evaluating actions using observed rewards.This separates action selection from reward evaluation and supports learning policies based on long-term effectiveness.
DRL agent design: PPO is on-policy: its training buffer contains experience collected with the current actor and is emptied after actor updates.The approach uses fresh data rather than experiences collected by older policy versions, though it is usually slower than off-policy alternatives.
DRL agent design: Autoencoder encoders reduce high-dimensional RAN observations, mitigating redundant information and outliers before DRL processing.The encoder produces a lower-dimensional latent representation that the decoder can reconstruct when properly trained.
xApp configurations: The experiments use distinct DRL configurations for joint slicing and scheduling, slice-specific scheduling, offline training, and online training with live RAN data.The sched-slicing xApp includes baseline, reduced-action, and no-autoencoder models; online training specializes models to a deployment despite not being recommended by O-RAN.

5 DRL-BASED XAPP EVALUATION

The evaluation examines KPM feature selection and compares DRL-based scheduling and slicing strategies. Results show that joint adaptive control, suitable action spaces, and autoencoder-based state representation affect performance across slices.

RAN KPM and Feature Selection: More than 400 standardized KPMs can be available, making feature selection necessary to limit state dimensionality and reporting overhead.The paper advocates combining expert knowledge with correlation analysis from large-scale datasets.
RAN KPM and Feature Selection: −0.56 correlation links downlink MCS and buffer occupancy, while downlink TBs and symbols have a 0.998 positive correlation for eMBB.TBs and buffer occupancy correlate weakly because the data carried per TB varies with MCS.
RAN KPM and Feature Selection: URLLC correlation patterns differ from eMBB because its lower traffic rate produces lower load and quickly drained buffers even at lower MCSs.The comparison uses 36 PRBs for eMBB and 11 PRBs for URLLC under slice-based traffic.
Comparing Different DRL-based RAN Control Strategies: Jointly controlling slicing and scheduling improves each slice’s relevant metrics, especially PRB ratio and throughput below the 40th percentile.The comparison uses the sched xApp with fixed slicing and the sched-slicing xApp with joint control.
Comparing Different DRL-based RAN Control Strategies: DRL-driven control outperforms static baselines, while restricting the action space changes the preferred allocation and slice-specific outcomes.The reduced-actions agent favors MTC performance but fails to maintain a small buffer and high PRB ratio for URLLC.
Comparing Different DRL-based RAN Control Strategies: Autoencoder-equipped agents generally outperform the no-autoencoder agent by reducing input dimensionality and tolerating missing or inconsistent live-RAN telemetry.The autoencoder maps T = 10 input vectors with N = 3 metrics to a single N-dimensional vector and is trained with randomly zeroed entries.

6 ONLINE TRAINING FOR DRL-DRIVEN XAPPS

The online-training xApp adapts a pre-trained DRL agent to changing traffic, but exploration temporarily reduces RAN performance. Experiments compare convergence, action distributions, throughput, and portability across Colosseum and Arena.

Training setup: Online training continues a model pre-trained on slice-based traffic while evaluating convergence, exploration effects, and adaptation on a live RAN.The model is deployed on a ColO-RAN base station and trained online with exploratory actions.
Convergence: Entropy regularization loss and average reward track convergence, while the Colosseum online phase requires 30% fewer steps than the initial offline training.Arena’s average reward is halved because its configuration uses three users per base station instead of six; offline exploration supports parallel learning environments.
Action selection: The DRL agent’s action distribution evolves from uniform exploration to skewed, multi-modal distributions by the ends of offline and online training.The highlighted snapshots correspond to steps 2260, 17460, and 29820, with offline training stopping at step 17460.
RAN performance: The trained agent improves throughput relative to the exploratory training phase, although online training increases throughput variability for the two eMBB users.The trained agent no longer explores, while the training process can temporarily worsen performance.
Adaptability: 2 Mbit/s in the 80th percentile separates the online- and offline-trained agents under uniform traffic, favoring the online-trained agent.The online agent adapts to updated traffic and changes its action profile toward more balanced PRB allocations.
Summary: Online training helps pre-trained models adapt to deployment environments, but its training-time performance cost motivates alternating training and control according to network needs.ColO-RAN provides a controlled setting for pre-training and testing, including online training without affecting production environments; models pre-trained on Colosseum can also work over the air.

7 RELATED WORK

Related work applies ML and DRL across wireless-network layers and cellular control tasks. Existing studies cover diverse applications, but the cited work spans reviews, protocol-stack methods, and theoretical or task-specific DRL evaluations.

ML for wireless networks: ML research in wireless networks spans channel modeling, PHY and MAC layers, routing, transport, and data-driven applications.Several surveys review these learning techniques and open issues.
Cellular-network applications: Cellular-network studies apply ML to modulation and coding classification and link adaptation for eMBB under URLLC preemptive puncturing.The cited modulation-and-coding approach addresses robustness to modulation parameters absent from its training set.
DRL for RAN control: DRL studies examine centralized wireless control, handover, medium access, user association, and scheduling, including theoretical and task-specific evaluations.The cited examples include multi-agent control, actor-critic scheduling, and Q-learning for radio access.

8 CONCLUSIONS

The paper presents ColO-RAN as a large-scale platform for evaluating ML-driven O-RAN xApps and extracting design lessons for closed-loop DRL control. Its conclusions emphasize careful state and action design, adaptive strategies, and live-training tradeoffs.

Platform: ColO-RAN integrates an E2 termination, near-real-time RIC with three xApps, and non-real-time RIC functions for storage and ML training in Colosseum.The authors pledge to release the framework and collected dataset publicly.
Design lessons: Meaningful network-state features and a properly designed action space are crucial to avoid unnecessarily high-dimensional DRL inputs and ineffective control choices.The comparison also reports that autoencoders can help handle unreliable real-RAN data.
Evaluation: Comparative xApp evaluations support adaptive scheduling and slicing strategies while highlighting insights and tradeoffs in live DRL training.The paper positions these results as a large-scale evaluation of ML-driven O-RAN xApps for cellular-network management and control.