Source-linked AI summary
Mitigating Concept Drift in QoS Prediction for Teleoperation of Autonomous Vehicles Using Historic Data
Xiyan Su, Jianning Gao, Mahmoud Ashri, Frank Diermeyer
TL;DR
Teleoperation needs mobile-network resources that are not always available, while QoS models can degrade on previously unseen data. The paper predicts uplink data-rate and round-trip latency using historic references, and reports improved unseen-data prediction with similar first-route performance.
Problem
Mobile-network resources needed for reliable teleoperation are not always available, and prediction models can degrade on previously unseen data because of concept drift.
Source: A. Teleoperation · p. 1, A. General Prediction Performance · p. 5Method
The paper predicts uplink data-rate and round-trip latency using an online framework that trains models to predict differences from historic reference values.
Source: C. Related Works · p. 2, A. Prediction Framework · p. 3Results
Historic references greatly improve prediction on unseen data while maintaining similar first-route performance, and critical-scenario F1 scores are higher for both KPIs on the second route.
Source: A. General Prediction Performance · p. 5Takeaways & Limitations
Evaluating teleoperation pQoS requires critical-scenario detection because general prediction performance alone does not represent its use case.
Source: B. Critical Scenario Detection for… · p. 5The approach still needs evaluation across more routes, devices, and data distributions, and the teleoperation countermeasures for critical network scenarios remain to be defined and investigated.
Source: §V · p. 6
Abstract
from arXiv · showhide
Teleoperation serves as the fallback solution to autonomous driving but reliable functions of the teleoperation require a certain amount of mobile network resources, which cannot be guaranteed at all times. Therefore, predictive quality of service (pQoS) is introduced as a concept to increase the resilience of the teleoperation. In this paper, based on a data measurement campaign, we propose a prediction framework to prediction two important network KPIs of teleoperation: uplink data-rate and round-trip latency. Furthermore, we introduce a method to alleviate the performance degradation of machine-learning-based prediction models on previously unseen data due to concept drift by incorporating historic data into the prediction pipeline. Additionally, we introduce the metric of critical scenario detection to evaluate the prediction performance specifically for teleoperation.
I. INTRODUCTION
Teleoperation depends on mobile-network resources that are not always available, motivating predictive QoS as a way to anticipate network quality and adapt proactively. The paper focuses on an over-the-top pQoS service because mobile-network-operator support is not ubiquitous.
A. Teleoperation
- Reliable teleoperation requires network resources that commercial mobile networks cannot provide at all times.The cited 5GAA safe-scenario target is 32 Mbps uplink data-rate and 99% reliability.Source: A. Teleoperation · p. 1
- Even data-rate reduction and multiple-link approaches cannot guarantee safe operation through the network at all times; critical scenarios may instead require degraded video quality or velocity restrictions.Source: A. Teleoperation · p. 1
B. Predictive Quality of Service
- pQoS lets autonomous-driving vehicles predict upcoming network quality and adapt proactively, increasing teleoperation resilience to network fluctuations.Source: B. Predictive Quality of Service · p. 1
- Because pQoS is not ubiquitously supported by mobile-network operators, the paper focuses on an over-the-top service provided on the vehicle by a third party.Source: B. Predictive Quality of Service · p. 1
C. Related Works
Prior work studies QoS prediction with machine-learning models and historic network maps, but distribution shifts can undermine predictions on new data. This paper combines these approaches by using historic data to reduce concept-drift degradation without explicit retraining.
C. Related Works
- Mobile QoS prediction is complex because radio access, core-network, and routing factors affect it; prior work identifies PHY indicators as useful for uplink prediction,.Source: C. Related Works · p. 1
- Earlier studies used LSTM models for throughput prediction and bidirectional LSTMs for downlink data-rate prediction across multiple horizons.Source: C. Related Works · p. 1
- Prior work finds that trained ML models generalize from training-data distributions, so predictions can degrade in another area with different data distributions and require time- and resource-consuming retraining [12],.Source: C. Related Works · p. 2
- Historic QoS data can be stored in Radio Environmental Maps, and prior work used data from a vehicle that had driven nearby to predict current-vehicle QoS,, –,.Source: C. Related Works · p. 2
- The paper combines QoS prediction with historic-data methods, using a Radio Environmental Map to train models to predict differences from historic reference values and alleviate concept drift without explicit retraining.Source: C. Related Works · p. 2
II. DATA MEASUREMENT
The authors collected real-world teleoperation network data on two routes with different operational design domains. The first route supplies training data, while the second is held out and used to create historic reference data.
A. Measurement Setup
- Two routes with different operational design domains were measured, with data from the first route used for training and none from the second route included in training.Source: A. Measurement Setup · p. 2, Figure 1 · p. 2
- Measurements covered physical-layer and application-layer KPIs, including round-trip latency measured by ping during 4500 Kbps video streaming and uplink data-rate measured separately with iperf3.The data-rate run used UDP packets; it was separated from latency measurement because it consumed practically all uplink bandwidth and would make latency unrealistically high.Source: A. Measurement Setup · p. 2
- Timestamp synchronization aligned records from different sources; first-route data were split by runs for evaluation, while second-route data supplied historic data for a Radio Environmental Map.Source: A. Measurement Setup · p. 2
B. Static Data Analysis
Physical-layer indicators relate to uplink data-rate but do not guarantee high data-rate, and similar correlations are not observed for latency. Across the two routes, uplink data-rate is more susceptible to change, making it more likely to experience concept drift and prediction degradation on unseen data.
B. Static Data Analysis
- RSRP, RSRQ, and SINR correlate more strongly with uplink data-rate, but the relationship is not monotonic: good indicators make low data-rate unlikely, not high data-rate certain.Source: B. Static Data Analysis · pp. 2–3
- Similar PHY-KPI correlations are not observed for latency.Source: B. Static Data Analysis · p. 3
- The route ECDF comparison shows uplink data-rate is more susceptible to change, suggesting greater concept-drift risk and possible prediction degradation on unseen data without retraining.Source: B. Static Data Analysis · p. 3, Figure 3 · p. 3
III. QUALITY OF SERVICE PREDICTION
The framework predicts uplink data-rate and round-trip latency from online vehicle measurements, using historic network data as references. It predicts target-minus-reference differences and adds the references back after inference.
A. Prediction Framework
- The onboard framework predicts uplink data-rate and round-trip latency from current PHY and application-layer KPIs, excluding uplink rate as an input because measuring it would consume uplink capacity.The listed online input feature KPIs are identified in Table II.Source: A. Prediction Framework · p. 3, Table II · p. 4
- A KNN-selected set of historic REM measurements supplies reference values, and the model predicts target-minus-reference differences that are added back to those references after inference.Historic references also cover input-feature KPIs.Source: A. Prediction Framework · p. 3
- The framework uses recent timestamp measurements as lagged inputs and predicts future frames using a step parameter that can be tuned to the use case.Both lag and step are model hyperparameters.Source: A. Prediction Framework · pp. 3–4
- XGBRegressor with MultiOutputRegressor supports multi-step, multihead prediction; inputs are standardized and measurement outliers are eliminated.The input-feature KPI list is shown in Table II.Source: A. Prediction Framework · p. 4, Table II · p. 4
B. Usage of Historic Data
Historic measurements are selected as references using a KNN distance that combines spatial proximity with two temporal distances. The selected neighbors are averaged to form the reference values.
B. Usage of Historic Data
- Historic measurements are ranked by a distance combining Euclidean location distance with two temporal distances: within-week time difference and week difference.The within-week component reflects the assumed regular weekly pattern of mobile-network fluctuations reported in.Source: B. Usage of Historic Data · p. 4
- Each distance component is normalized and weighted by α, β, and γ before KNN selects the nearest neighbors and averages them as the reference.The hyperparameters are chosen through parameter searches.Source: B. Usage of Historic Data · p. 4
- The distance uses locations represented by latitude and longitude and timestamps measured in seconds; the week-length constant is sw = 604800 seconds.The hyperparameters for the ML model and KNN are listed in Table III.Source: B. Usage of Historic Data · p. 4, Table III · p. 4
- Figure 4 presents MAE for uplink data-rate and round-trip latency predictions across prediction horizons on Route 1.The supplied caption gives the figure's scope but no numerical comparisons.Source: Figure 4 · p. 4
A. General Prediction Performance
Using historic data as reference points improves prediction on the unseen route while maintaining similar performance on the training route. Without historic data, performance degrades on unseen data, whereas the effect of longer horizons differs between routes.
A. General Prediction Performance
- Historic references greatly improve prediction on unseen Route 2 while maintaining similar performance on Route 1, as compared in Figure 5.Route 1 is the training route; Route 2 contains completely unseen data.Source: Figure 5 · p. 5, A. General Prediction Performance · p. 5
- Without historic data, the model predicts well on Route 1 but degrades quickly on unseen Route 2, whose data distribution differs drastically from training data.The passage states that this traditional pipeline usually needs retraining to cope with the degradation, at a time and resource cost.Source: A. General Prediction Performance · p. 5
- For the first route, prediction error slightly degrades at higher prediction horizons, but using historic data reduces error there by acting as an anchor point; this relationship is not observed for the second route.Source: A. General Prediction Performance · p. 5
B. Critical Scenario Detection for Teleoperation
Critical-scenario detection evaluates pQoS where network conditions cross thresholds that can destabilize teleoperation. On the unseen second route, historic data improves F1 scores for both uplink data-rate and latency.
B. Critical Scenario Detection for Teleoperation
- On the unseen second route, F1 scores are higher for both uplink data-rate and latency with historic data.For the first route, uplink data-rate F1 is slightly higher without historic data, while latency F1 is lower without it; predicting latency on unseen data remains difficult.Source: Table IV · p. 5, B. Critical Scenario Detection for… · pp. 5–6
- Critical scenarios are evaluated when uplink data-rate falls below 14 Mbps or round-trip latency exceeds 100 ms, thresholds associated with teleoperation becoming unstable.F1 incorporates both false negatives and false positives.Source: B. Critical Scenario Detection for… · p. 5
V. CONCLUSION
The paper presents a measurement-based pQoS framework for predicting uplink data-rate and round-trip latency, using historic data to address concept drift. It reports reduced prediction error and more detected uplink-related critical scenarios, while identifying broader evaluation and countermeasures as open questions.
V. CONCLUSION
- The historic-data method reduces prediction error versus the traditional approach and detects more uplink data-rate-related critical scenarios.The reported prediction evaluation uses mean absolute error.Source: §V · p. 6
- The framework predicts uplink data-rate and round-trip latency by training the model to predict differences from reference target values derived from historic data.The measurements come from a campaign with an experimental teleoperated vehicle and include physical-layer and application-layer KPIs.Source: §V · p. 6
- The approach needs evaluation on more routes and devices across a wider range of data distributions, including settings where historic data or REM is shared between devices.Source: §V · p. 6
- Future work must define and investigate teleoperation software countermeasures for critical network scenarios.Source: §V · p. 6