PaperlayerView exact revision on arXiv

Source-linked AI summary

ResLearn-XR: Residual Learning for Network Traffic and Quality-of-Experience-Aware Modeling in Extended Reality

Yoga Suhas Kuruba Manjunath, Jie Gao, Lian Zhao

arXiv:2609.04493v1cs.AIcs.LGcs.NI

TL;DR

XR traffic prediction must capture bursty frame-level dynamics, while QoE-risk estimation lacks frame-timing representations and XR datasets with session-level user labels. ResLearn-XR combines task-specific residual learning, causal descriptors, and paired traffic/QoE data; it reports SMAPE improvements in both traffic prediction and QoE-risk estimation.

  • Problem

    XR traffic predictors can underpredict frame-level bursts, while existing XR datasets lack session-level QoE labels and QoE models lack frame-level timing representations.

    Source: §I · p. 1, C. XR Traffic-QoE Datasets · p. 2

  • Method

    ResLearn-XR uses task-specific value-space and logit-space residual heads, DDA frame-timing descriptors, and a dataset pairing traffic traces with session-level QoE annotations.

    Source: §I · pp. 1–2

  • Results

    SMAPE improves by up to 17.84% across frame-count, frame-size, and inter-arrival-time prediction and by up to 87.8% for QoE-risk estimation.

    Source: §I · p. 2

  • Takeaways & Limitations

    ResLearn-XR is presented as a predictive modeling component for supporting proactive bandwidth allocation.

    Source: §VII · p. 12

    The two-participant QoE dataset and session-level weak labels make participant-independent results preliminary cross-user evidence, not broad user-level generalization.

    Source: F. Limitations · p. 12, §VII · p. 12

Abstract

from arXiv · show

We present ResLearn-XR, a residual learning framework for predicting eXtended Reality (XR) network traffic and estimating Quality-of-Experience (QoE) risk. ResLearn-XR adopts a two-stage temporal learning structure comprising a base sequence prediction model augmented with task-specific residual learning components to improve adaptability to bursty, non-stationary XR traffic dynamics. The residual learning stages operate in the value space for continuous XR traffic forecasting and in the logit space for probabilistic QoE risk estimation. \rev{For the QoE-risk branch, we introduce a Data Descriptor Algorithm (DDA), a causal feature-construction module that converts packet-level application-layer observables into frame-timing-aware descriptors suitable for encrypted traffic analysis. We also construct an XR Traffic-QoE dataset that pairs continuous XR traffic traces with session-level user-reported QoE labels. ResLearn-XR reduces SMAPE by up to 17.84% across frame-count, frame-size, and inter-arrival-time prediction, while reducing QoE-risk estimation SMAPE by up to 87.8% over single-stage baselines.

I. INTRODUCTION

XR traffic’s burstiness and timing-sensitive QoE risk motivate joint prediction; ResLearn-XR combines residual learning, causal descriptors, and paired traffic/QoE data, with reported gains across both tasks.

A. XR Traffic Prediction

  • Traffic prediction targets frame-level bursts alongside long-range trends, since conventional sequence predictors can underpredict high-variance bursts critical to MTP requirements.Prior Transformer-based models can oversmooth frame-scale bursts, motivating explicit residual modeling of short-timescale errors.Source: §I · p. 1, A. XR Traffic Prediction · p. 2

B. XR QoE Risk Estimation

  • Existing QoE approaches use static or aggregated network indicators, while XR QoE risk requires frame-level timing variability linked to traffic evolution and delivery behavior.The cited LSTM and digital-twin approaches do not explicitly link XR traffic evolution to frame-level delivery behavior.Source: §I · p. 1, B. XR QoE Risk Estimation · p. 2

I. INTRODUCTION

  • ResLearn-XR combines a Transformer with value-space residual correction for traffic prediction and logit-space correction for QoE-risk estimates.DDA constructs frame-timing-aware causal descriptors from packet-level observables, and the XR Traffic-QoE dataset pairs traffic traces with session-level QoE labels.Source: §I · pp. 1–2
  • Up to 17.84% SMAPE improvement across frame-count, frame-size, and inter-arrival-time prediction accompanies up to 87.8% improvement for QoE-risk estimation.The traffic-prediction average gain is approximately 6.7%.Source: §I · p. 2

C. XR Traffic-QoE Datasets

  • Existing XR traffic datasets lack associated session-level subjective QoE labels, and no open dataset jointly captures diverse XR traffic and user QoE outcomes.ResLearn-XR introduces a dataset pairing continuous traffic traces with session-level QoE annotations.Source: C. XR Traffic-QoE Datasets · p. 2, §I · p. 2

III. PROBLEM FORMULATION

  • The formulation learns traffic forecasts from observed packet sequences and estimates elevated QoE-risk probability, assigning each session label to its temporal windows.Traffic targets aggregate frame count, average frame size, and frame inter-arrival time; those future-window metrics are unavailable at prediction time because they require later packet arrivals.Source: §III · pp. 2–3

IV. RESIDUAL LEARNING FRAMEWORK

ResLearn-XR derives traffic-prediction targets and QoE descriptors from shared packet traces, then uses a two-stage residual model for frame-aligned traffic forecasting. Its traffic branch adds value-space corrections to a base predictor to address burst-related errors.

IV. RESIDUAL LEARNING FRAMEWORK

  • Shared packet traces support two paths: frame-aligned traffic targets for prediction and DDA-derived descriptors for QoE risk estimation.Traffic targets are frame count, average frame size, and average inter-arrival interval; the QoE path uses application-layer observables aligned with the session refresh rate.Source: §IV · p. 3

A. XR Traffic Prediction

  • The traffic model forecasts the next frame-aligned traffic vector from a look-back sequence, using a Transformer encoder by default.The vector contains average frame size, frame count, and average frame inter-arrival interval; the residual formulation also supports LSTM, GRU, or Stacked LSTM encoders.Source: A. XR Traffic Prediction · p. 4
  • MSE can attenuate short-term bursts, so a value-space residual learner models the remaining prediction error after the base stage.The described bursts involve increased frame size and count, often with reduced inter-arrival interval; these patterns relate to MTP-latency risk.Source: A. XR Traffic Prediction · p. 4
  • Using only the most recently observed residual preserves temporal causality; the residual learner predicts a correction that is added to the base forecast.A global bias computed from valid training residuals shifts residual inputs, reducing mixed-sign cancellation while the supervised target remains in the original residual space.Source: A. XR Traffic Prediction · p. 4

1) DDA:

DDA converts packet-level observables into causal, frame-timing-aware descriptors for QoE risk estimation. Its descriptors represent delivery, pacing, offered load, and temporal variability over successive windows.

1) DDA:

  • DDA aggregates packet size, direction, inter-arrival time, and timestamp into frame-aligned statistics, then maps them to temporal descriptors for QoE risk estimation.The descriptors encode frame-rate deviation from the display target, timing and throughput instability, and offered load as a congestion-delay proxy.Source: 1) DDA · p. 5
  • Each window descriptor uses only packet observations timestamped up to that window’s end, with rolling statistics based on a fixed look-back.The look-back is independent of window duration; descriptor sequences include causal dispersion and trend statistics.Source: 1) DDA · pp. 5–6
  • The descriptors jointly characterize delivery rate, payload intensity, pacing regularity, offered load, and timing variability relevant to QoE degradation risk.The composite instability indicator weights inter-frame timing volatility, frame-rate volatility, and display-target deviation.Source: 1) DDA · pp. 5–6
  • The evaluation ablates base per-window, pacing, short-term stability, composite-instability, and full DDA representations.Source: 1) DDA · p. 6

2) XR QoE Risk Estimation:

The QoE branch estimates elevated risk from causal DDA descriptors using a two-stage logit-space model. It calibrates the resulting probabilities within descriptor-based strata, with hierarchical pooling when a fine-grained stratum is unavailable.

2) XR QoE Risk Estimation:

  • The base estimator uses class-weighted binary cross-entropy with logits, with the class weight compensating for elevated-risk label imbalance.Source: 2) XR QoE Risk Estimation · p. 6
  • A residual head maps the base QoE model’s final-step latent state to a bounded logit correction, which is added before converting the result to an elevated-risk probability.The branch uses only the causal descriptor look-back and corresponding latent state at inference; future descriptors and ground-truth QoE labels are excluded.Source: 2) XR QoE Risk Estimation · p. 6
  • Stratum-wise isotonic regression calibrates risk probabilities using DDA-derived covariates, backing off to pooled strata when a finer stratum is unavailable.The strata condition on inferred refresh rate, offered load, pacing ratio, and composite instability; the most specific available calibrator is used, with a global fallback.Source: 2) XR QoE Risk Estimation · p. 7

A. Experimental Platform and Dataset Collection

The dataset platform captures XR traffic between a Quest 2 headset and remote rendering services while traffic shaping emulates bandwidth-limited access conditions. Two participants provide session-level QoE ratings across applications and bandwidth regimes.

A. Experimental Platform and Dataset Collection

  • The platform combines a Quest 2 headset, Xbox console, programmable traffic shaper, and remote XR rendering system.The traffic shaper constrains uplink and downlink rates to emulate bandwidth-limited access conditions; remote rendering serves several VR, AR, and MR applications.Source: A. Experimental Platform and Dataset… · pp. 7–8
  • Two participants recorded sessions across five bandwidth regimes: 15, 30, 60, and 120 Mbps, plus adaptive bandwidth.Repeated sessions used identical application and network configurations.Source: A. Experimental Platform and Dataset… · p. 8
  • Session ratings are binarized into elevated and non-elevated risk, providing weak session-level supervision rather than per-window discomfort onset, duration, or intensity.Moderate/discomfort and severe/cybersickness ratings map to elevated risk; none/comfortable and slight/tolerable map to non-elevated risk.Source: A. Experimental Platform and Dataset… · p. 8

B. Dataset and Experiments

The section describes the datasets, split procedures, and baselines used to evaluate XR traffic prediction and QoE risk estimation. Evaluation preserves chronology for traffic prediction and separates QoE sessions to limit leakage.

B. Dataset and Experiments

  • Traffic prediction uses Dataset I, SteamVR Dataset II, and Dataset III; Dataset I also supports QoE estimation because the other datasets lack paired traffic traces and session-level QoE labels.Dataset III includes diverse VR applications and a replay under varied network conditions.Source: B. Dataset and Experiments · p. 8
  • Traffic prediction uses chronological 40%/10%/50% train/validation/test splits within each session, with no shuffling or split-boundary crossing.Sequences are generated separately within each segment to evaluate next-step prediction on later unseen traffic and prevent look-ahead leakage.Source: B. Dataset and Experiments · p. 8
  • QoE splits keep complete sessions intact: participant-independent evaluation separates participants, while mixed-user evaluation assigns sessions from both participants to disjoint splits.This prevents leakage of session-specific patterns and repeated labels; no participant-dependent QoE setting is reported.Source: B. Dataset and Experiments · p. 8
  • Dataset I evaluation represents five bandwidth regimes, with stratification where possible and matched regimes across participants for participant-independent QoE evaluation.The design limits dominance by any single bandwidth setting.Source: B. Dataset and Experiments · p. 8
  • Informer, FEDformer, and Temporal Fusion Transformer are traffic-forecasting baselines; logistic regression, random forest, and gradient-boosted trees are QoE baselines.Models use task-specific inputs, splits, and metrics.Source: B. Dataset and Experiments · p. 8

1) Traffic Prediction:

2) QoE Risk Estimation:

Residual correction improves XR traffic forecasts across evaluated tasks, while QoE-risk results vary by evaluation split and metric. The section also reports proxy-risk detection, inference costs, and limits on deployment and user-level generalization.

A. Traffic Prediction

  • Across 16 paired backbone–task comparisons, every ResLearn-XR variant reduces SMAPE by 0.35%–17.84%, with an average gain of approximately 6.7%.The largest reported gain, 17.84%, is for Transformer-based inter-arrival-time prediction; the GRU gain there is 0.35%.Source: A. Traffic Prediction · p. 9
  • Against forecasting baselines under the same chronological split, ResLearn-XR has the lowest RMSE, MAPE, and SMAPE, improving the best competing values from 4818.62 to 4760.99, 0.0085 to 0.0076, and 0.85 to 0.76.The controlled residual-learning comparison is based on paired results in Tables IV–VII.Source: A. Traffic Prediction · p. 9, Table VIII · p. 10
  • Residual correction raises queueing-proxy risk recall from 0.63 to 0.81 and lowers the missed-risk rate from 0.37 to 0.19; direct closed-loop MTP-violation measurement remains future work.The proxy marks predicted offered load above a predefined threshold as MTP-risk positive.Source: A. Traffic Prediction · pp. 9–10, Table IX · p. 10

B. QoE Risk Estimation

  • In participant-independent QoE evaluation, Transformer residual models reduce SMAPE from 1.250 to 0.259 for U2→U1 and from 1.245 to 0.152 for U1→U2, with AUCs of 0.874 and 0.930.In mixed-user evaluation, residual models also reduce SMAPE; LSTM has the lowest error (0.429), while Transformer has the highest AUC (0.909).Source: B. QoE Risk Estimation · p. 10
  • QoE calibration is split- and backbone-dependent: the non-residual Transformer and Stacked LSTM have lower ECE (0.052) than the residual Transformer (0.076) in Table XIII, while the residual Transformer improves ECE in Table XII.SMAPE, AUC, and ECE are treated as complementary rather than uniformly aligned properties.Source: B. QoE Risk Estimation · p. 10

A. Traffic Prediction

  • The residual head adds about 4K parameters and 0.02 MB; Transformer inference rises from 0.42 ms to 0.47 ms, but feasibility excludes packet capture and downstream control delays.Full deployment still requires system-level profiling and closed-loop validation.Source: Table X · p. 10, A. Traffic Prediction · p. 10

B. QoE Risk Estimation

  • ResLearn-XR attains the highest AUC and Macro-F1 and lowest ECE among non-deep-learning QoE baselines under participant-independent evaluation.This comparison indicates value beyond the DDA-derived descriptors alone.Source: Table XI · p. 10, B. QoE Risk Estimation · p. 10
  • With only two participants and session-level weak QoE labels, participant-independent results are preliminary cross-subject evidence, not broad user-level generalization.The paper calls for larger cohorts, temporally localized feedback, and broader XR scenarios for robust user-independent validation.Source: B. QoE Risk Estimation · p. 10

C. Effect of the DDA Descriptors

The section reports gains from DDA descriptors and residual learning across XR traffic prediction and QoE-risk estimation, alongside deployment costs and limits on cross-user generalization. It also frames ResLearn-XR as a predictive component rather than a closed-loop controller.

C. Effect of the DDA Descriptors

D. Effect of Residual Learning

  • Traffic residuals yield positive SMAPE gains across backbone–task pairs, peaking at 17.84% for Transformer inter-arrival prediction; training residual bias lowers frame-size SMAPE from 0.97 to 0.76.For QoE estimation, residual learning improves accuracy and discrimination, while calibration depends on backbone and split; Fig. 5 illustrates one better-calibrated model pair.Source: D. Effect of Residual Learning · p. 12, Table XVII · p. 12, Figure 5 · p. 12

E. Deployment Feasibility

  • The Transformer configuration adds approximately 4K parameters, 0.02 MB of FP32 trainable-weight memory, and 0.05 ms per prediction window over the base model.Reported model-only inference excludes packet capture, descriptor extraction, controller communication, scheduling, and actuation delay.Source: E. Deployment Feasibility · p. 12

F. Limitations

  • The QoE evaluation uses two participants, session-level weak labels assigned to temporal windows, and a binary reduction of four ordinal ratings, limiting cross-user evidence.The authors characterize participant-independent results as preliminary rather than broad user-level generalization.Source: F. Limitations · p. 12

VII. CONCLUSION AND FUTURE WORK

  • Traffic prediction gains reach 17.84% in SMAPE, averaging approximately 6.7%, while QoE-risk estimation improves SMAPE by up to 87.8% over single-stage baselines.These corrected strictly causal traffic-prediction results span the updated traffic tables.Source: §VII · p. 12
  • ResLearn-XR is presented as a predictive component for proactive resource management, not an end-to-end closed-loop controller; heterogeneous devices, 5G/6G conditions, and controller-in-the-loop deployment remain future work.The stated applications include bandwidth allocation, scheduling, rate adaptation, admission control, and edge-resource orchestration.Source: §VII · pp. 12–13

APPENDIX A XR TRAFFIC SPIKES AND MTP VIOLATION

The appendix relates bursty XR traffic to network-delay growth and the MTP latency budget, then motivates proxy-based risk analysis using residual-corrected predictions. It also identifies an implementation and reproducibility configuration table.

APPENDIX A XR TRAFFIC SPIKES AND MTP VIOLATION

  • Under M/M/1 queueing, expected delay steepens as utilization approaches one; burst-driven traffic changes can exhaust the network’s 20 ms−Lproc budget.The appendix connects frame-size and frame-count increases or shorter inter-arrival times with disproportionate latency increases.Source: Appendix A · p. 13
  • The appendix motivates windowed proxy MTP-risk analysis; residual correction reduces burst underestimation and improves proxy-risk detection, while direct closed-loop MTP-violation measurement remains future work.The burst-risk indicator uses an effective service-capacity threshold.Source: Appendix A · p. 13

APPENDIX B IMPLEMENTATION AND REPRODUCIBILITY

  • An implementation and reproducibility configuration is documented in Table XVIII.Source: Table XVIII · p. 13

Related papers

Loading 2609.04493v1…