PaperlayerView exact revision on arXiv

Source-linked AI summary

DIA: A Deep Reinforcement Learning-Based Interest-Aware ABR Scheme for Cloud Gaming

Zhe Zhang, Haiwei Dong, Marc St-Hilaire, Abdulmotaleb El Saddik

arXiv:2610.13698v1cs.MM

TL;DR

Cloud gaming ABR must transmit gameplay at high resolution and low latency over fluctuating wireless networks. DIA allocates bitrate at the object level, prioritizing key objects while reducing non-key-object resolution; it outperforms baseline approaches across QoE metrics in 5G and WiFi scenarios.

  • Problem

    Cloud gaming ABR must maintain QoE under fluctuating wireless conditions, while existing schemes often allocate bitrate too coarsely to account for key objects.

    Source: §I · p. 1, D. Summary · p. 3

  • Method

    DIA uses semantic segmentation and object-level bitrate allocation to prioritize key objects based on user movement, object features, and network conditions.

    Source: §I · p. 2

  • Results

    DIA outperforms all baseline approaches on all QoE metrics under 5G and WiFi networks.

    Source: D. Evaluation Results · p. 10

  • Takeaways & Limitations

    Increasing object update frequency from 4 Hz to 8 Hz brings limited MOS improvement, so 4 Hz is adopted to balance perceptual quality and computational overhead.

    Source: F. Summary · p. 17

    Training data is mainly from GTA V, so genres with distinct key objects may need additional data and fine-tuning; real-time deployment validation remains future work.

    Source: F. Limitations · pp. 13–14

Abstract

from arXiv · show

The continuous evolution of cloud gaming has enabled users to play high-quality cloud games on thin clients, such as smartphones and tablets. However, how to effectively transmit the gameplay frames with high resolution and low latency, especially over wireless networks, remains a significant challenge. To address this challenge, we propose a Deep Reinforcement Learning (DRL)-based Interest-Aware Adaptive Bitrate (ABR) scheme, called DIA, that guarantees the quality of perceptually key objects in each frame, while dynamically allocating lower resolutions for non-key objects to conserve bandwidth under constrained network conditions. By leveraging an off-the-shelf semantic segmentation model, the proposed DIA scheme can adjust the bitrate at the object-level. We also design a transformer-based feature extractor and an interest-aware reward function to enable the proposed DIA scheme to identify key objects and dynamically adjust their bitrates. This adaptation is guided by users' in-game movement speed, object features, and prevailing network conditions and operates without requiring additional hardware, such as gaze trackers. Extensive experiments show that the proposed DIA scheme outperforms existing ABR schemes in dynamic wireless network environments. Notably, DIA achieves at least a 20% higher normalized average Quality of Experience (QoE) than the State-Of-The-Art (SOTA) ABR algorithms in 5G scenarios.

I. INTRODUCTION

Cloud gaming ABR must maintain QoE over fluctuating wireless networks while meeting stricter latency and bandwidth demands than regular video streaming. DIA addresses coarse, scene-wide bitrate allocation by prioritizing key objects, and the paper reports improved QoE over SOTA ABR schemes.

I. INTRODUCTION

  • Cloud gaming must sustain QoE under fluctuating wireless conditions despite strict latency, reliability, and bandwidth requirements.Unlike regular streaming, it needs real-time rendering and cannot rely on pre-encoded resolution versions; chunk-level ABR is too coarse for its latency needs.Source: §I · p. 1
  • DIA uses object-level bitrate allocation to preserve key-object quality while lowering non-key-object resolutions under constrained bandwidth.It uses semantic segmentation and considers user movement speed, object features, and network conditions to identify key objects.Source: §I · p. 2
  • DIA improves users’ QoE over SOTA ABR schemes, especially in dynamic wireless environments.The paper also introduces QoE metrics that account for latency and the quality of perceptually key objects.Source: §I · p. 2

A. Rule-based ABR Schemes

  • Rule-based ABR schemes choose bitrates using fixed rules based on network throughput, buffer occupancy, or rebuffering events.Examples include FESTIVE, BBA, BOLA, and RobustMPC, which combines predicted throughput and buffer occupancy in QoE optimization.Source: A. Rule-based ABR Schemes · p. 2

B. Learning-based ABR Schemes

Learning-based ABR schemes use AI and DRL for bitrate decisions, while key-object identification methods range from visual heuristics to AI-based detection and saliency. The paper frames coarse-grained allocation and automatic identification of important objects as open challenges, then models wireless delivery from a cloud server to users.

B. Learning-based ABR Schemes

  • Existing learning-based ABR schemes include DRL methods such as Pensieve and Stick, as well as approaches for generalization and perceptual QoE.Examples include self-play RL, meta-learning, and reward functions based on perceptual quality or inferred user preferences.Source: B. Learning-based ABR Schemes · pp. 2–3

C. Identifying Key Objects

  • Key-object identification methods use visual features, object detection, semantic segmentation, or saliency, but often rely on predefined rules to mark user-interested regions.The cited discussion notes that gameplay interest can also depend on movement speed, semantic labels, and network conditions.Source: C. Identifying Key Objects · p. 3

D. Summary

  • Most existing ABR schemes allocate bitrate at the chunk level, overlooking key objects and potentially wasting bandwidth on perceptually unimportant regions.Automatically identifying key objects without gaze-tracking hardware or predefined rules remains a challenge.Source: D. Summary · p. 3

A. Network Model

  • The system model considers users receiving gameplay frames from a cloud server over time-varying wireless networks.Rendering and encoding occur at the server, and frames travel to user devices through base stations or access points.Source: A. Network Model · pp. 3–4

B. User Mobility Model

The model represents user mobility, gameplay content, and user interest to support object-level bitrate decisions. Objects receive different resolution levels according to estimated interest, which depends on object features, relationships, user movement, and network conditions.

B. User Mobility Model

  • Users move randomly in a 2D space, with speed sampled uniformly from 0 to a maximum to represent stationary through high-mobility scenarios.The mobility model supplies user position and speed at each time slot.Source: B. User Mobility Model · p. 4

C. Content Model

  • Object-level ABR assigns each object a resolution level and encoded bitrate, avoiding uniform allocation when only a few objects attract attention.Each object is represented by its semantic label, pixel area, and motion speed; speed is calculated from successive centroid positions.Source: C. Content Model · pp. 4–5

D. User Interest Model

  • A neural network estimates object interest from semantic labels, area, position, object and user movement speeds, while latent object relationships also matter.The model selects the top-K objects by estimated interest as key objects; the relevant factors can vary with the gameplay scenario.Source: D. User Interest Model · p. 5

E. Transmission Model

The transmission model accounts for end-to-end latency, buffering, and rebuffering when frames traverse variable-bandwidth wireless networks. It constrains buffered bitrate while recognizing that buffering can add latency.

E. Transmission Model

  • Because wireless bandwidth varies and cloud gaming uses a small buffer, frames may rebuffer more often than in regular video streaming.The buffered-frame bitrate is constrained by buffer size, and a larger buffer can increase buffer latency.Source: E. Transmission Model · p. 5
  • The model tracks bitrate of frames played and received, the received-frame count, and buffer size..Source: E. Transmission Model · p. 6
  • End-to-end latency includes frame transmission, propagation, queuing, processing, buffering, and rebuffering components.The model describes frame transmission and propagation latency, then decomposes total latency into processing, buffer, and rebuffering terms.Source: E. Transmission Model · pp. 5–6

F. QoE Model

The QoE model combines object-level bitrate quality with temporal smoothness and system-level latency penalties. DIA uses object-level features and network conditions to allocate resolutions, assigning higher resolutions to objects that contribute more to QoE.

F. QoE Model

  • Frame-sequence QoE weights object-level bitrate quality, bitrate variation, rebuffering time, and end-to-end latency.The model can produce different QoE metrics by changing the bitrate-quality function and weights.Source: F. QoE Model · p. 6

IV. DIA SCHEME

  • DIA applies semantic segmentation and PPO to use object features, user movement speed, and network conditions for per-object resolution selection.The object features include semantic labels, pixel areas, centroid positions, and object motion information.Source: A. Overview · p. 6

A. Overview

  • Objects contributing more to overall QoE receive higher resolutions, while less important objects receive lower resolutions within the available bandwidth budget.The workflow is shown in Fig. 2.Source: A. Overview · p. 6, Figure 2 · p. 6

B. Design of DIA Scheme

DIA frames object-level resolution selection as a QoE-maximizing problem under changing network conditions. Its state combines object, user-movement, and network information, while its discrete actions select a resolution for each object.

B. Design of DIA Scheme

  • At each time slot, DIA seeks to maximize QoE by allocating bitrate to individual objects using object features, user movement speed, and network conditions.Object motion comes from positional changes across frames, user movement speed from background changes, and network conditions from the device; user speed is categorized as low, medium, or high.Source: B. Design of DIA Scheme · p. 6
  • The object-level selection problem is formulated as an MDP, and model-free PPO learns a policy because the state-transition probability is unknown.The authors choose PPO for training stability and sample efficiency relative to DQN and actor-critic models.Source: B. Design of DIA Scheme · p. 6

1) State Space:

  • The state includes object-level features, user in-game movement speed, and network conditions, including available bandwidth and buffer occupancy.Object features include semantic labels, pixel areas, and motion speed; the state also tracks the percentage of objects not yet assigned a bitrate.Source: 1) State Space · pp. 6–7

2) Action Space:

  • For each object, the agent selects one of six discrete resolution levels, which represents relative encoding quality rather than whole-frame resolution.The listed levels range from 320×240 to 3840×2160; the selected level determines the object's bitrate.Source: 2) Action Space · p. 7

3) Interest-aware Reward Function:

DIA rewards assigning higher bitrates to human-labeled key objects and penalizes rebuffering, bitrate variation, and latency. Network feedback supplies the penalty signals, which are normalized and clipped for training.

3) Interest-aware Reward Function:

  • The interest-aware reward favors higher resolutions for key objects when bandwidth is constrained, using object-level ground-truth interest labels.The labels come from a human-labeled dataset; assigning a higher bitrate to a key object earns a positive bitrate reward, while otherwise the agent receives a penalty.Source: 3) Interest-aware Reward Function · p. 7
  • The reward combines bitrate reward with penalties for rebuffering, bitrate variation, and latency.The bitrate reward uses resolution and ground-truth interest labels, while the other penalties come from network-environment feedback after an action.Source: 3) Interest-aware Reward Function · p. 7
  • Penalty terms are normalized and clipped to [0, 1] using environment feedback for rebuffering, end-to-end latency, and aggregated bitrate variation.Table II maps resolution and interest label to a scalar bitrate-reward score.Source: 3) Interest-aware Reward Function · p. 7, Table II · p. 7

C. Neural Network Architecture

DIA’s lightweight DRL model uses a transformer feature extractor with actor and critic networks. The reported training implementation uses annotated GTA V gameplay data, specified hardware, and PPO hyperparameter settings.

C. Neural Network Architecture

  • The lightweight DRL architecture pairs a transformer-based feature extractor with an actor-critic architecture to support bitrate allocation under cloud gaming’s low-latency requirements.The transformer captures latent relationships among objects, and its outputs feed the actor-critic policy alongside user movement speed and network conditions.Source: C. Neural Network Architecture · p. 7, Figure 3 · p. 8

1) Transformer-based Feature Extractor:

  • A shared feature extractor embeds object features and processes them through two transformer encoder blocks with four-head self-attention.The actor and critic networks jointly train the shared extractor through policy and value-function losses.Source: 1) Transformer-based Feature Extractor · p. 8

2) Actor-Critic Architecture:

  • The actor outputs a probability distribution over discrete resolution levels, while the critic estimates a scalar state value.Both networks use three fully connected layers with ReLU activations; normalization follows the second ReLU.Source: 2) Actor-Critic Architecture · p. 8

1) Dataset:

  • The dataset comprises GTA V gameplay clips spanning varied scenes, objects, and gameplay patterns, with about 500 clips and over 1,500 images.Five annotators label key objects, and objects identified consistently by multiple annotators become ground-truth key objects; the dataset uses 22 predefined classes based on COCO classes.Source: 1) Dataset · p. 8

2) Hardware Environment:

  • Training ran on a Linux server with dual Intel Xeon Platinum 8352V CPUs and four RTX 4090 GPUs.Each GPU has 24 GB of memory.Source: 2) Hardware Environment · p. 8

3) Hyperparameter Settings:

  • The PPO training uses the hyperparameters summarized in Table III and exponentially decays entropy weight δ over training episodes.The decay is intended to encourage early exploration followed by a gradual shift toward exploitation.Source: 3) Hyperparameter Settings · p. 8, Table III · p. 8

4) Inference Strategy:

DIA limits inference overhead by updating decisions periodically without blocking streaming and connects to standard production encoders. Its evaluation uses cloud-gaming-specific QoE metrics alongside QoS measures and subjective ratings.

4) Inference Strategy:

  • DIA makes resolution decisions every 250 ms without blocking the rendering and streaming pipeline; inference overhead affects adaptation timeliness rather than end-to-end latency.On an RTX 4090, segmentation averages 90 ms and DRL inference 0.96 ms.Source: 4) Inference Strategy · pp. 8–9

E. Integration with Production Encoders

  • DIA maps resolution decisions to a QP map or Delta-QP values passed to standard encoders, without modifying codec standards.The design uses encoder ROI APIs and supports encoders such as H.264/HEVC and x265.Source: E. Integration with Production Encoders · p. 9

V. EVALUATION

  • Evaluation combines QoE metrics with frame-level QoS measures and a subjective study of perceived visual quality.Reported QoS measures include rebuffering time, bitrate difference, bitrate, and end-to-end latency.Source: §V · p. 9, A. Evaluation Metrics · p. 9

A. Evaluation Metrics

  • The three proposed QoE metrics replace uniform frame-level utility with object-weighted utility, with the HD-oriented metric emphasizing high resolution for important gameplay objects.The metrics vary bitrate quality functions and weights; QoE formulations retain bitrate utility, rebuffering, quality-variation, and latency terms.Source: A. Evaluation Metrics · p. 9

B. Baseline Approaches

DIA is compared with conventional chunk-based and ROI-based ABR schemes. A custom simulator evaluates these approaches across three wireless scenarios with modeled user mobility and changing network conditions.

B. Baseline Approaches

  • The comparison includes conventional chunk-based ABR schemes BBA, BOLA, RobustMPC, and Pensieve, alongside ROI-based cloud-gaming schemes DeepGame and T-Gaming.Source: B. Baseline Approaches · p. 9
  • BOLA uses Lyapunov optimization, RobustMPC predicts throughput and buffer occupancy, and Pensieve learns bitrate decisions from current observations.Pensieve is described as a DRL-based ABR scheme.Source: B. Baseline Approaches · p. 9
  • DeepGame predicts ROI with an LSTM, T-Gaming uses saliency maps, Nebula adapts encoder parameters and frame-level redundancy, and MetaABR uses meta-reinforcement learning.Source: B. Baseline Approaches · pp. 9–10

C. Simulation Settings

  • The Python simulator models dynamic network conditions and user mobility in 5G, 4G, and WiFi scenarios, with 160 users over 300-second runs averaged across 10 repetitions.Each scenario uses the same geographical area and user count but different mobility patterns and network parameters.Source: C. Simulation Settings · p. 10
  • The scenarios set maximum user bandwidth to 300 Mbps for 5G and WiFi and 100 Mbps for 4G; modeled bandwidth also reflects path loss, interference, background traffic, mobility, and handovers.User speeds range from 0–30 km/h in 5G and 4G, and 0–5 km/h in WiFi.Source: C. Simulation Settings · p. 10

D. Evaluation Results

Across the evaluated wireless scenarios, DIA generally outperforms baseline ABR schemes on QoE, while the results also show setting-specific trade-offs and support from key-object validation and a subjective study.

D. Evaluation Results

  • DIA achieves 90.58% average key-object identification accuracy over 10 evaluation runs using human-annotated ground-truth labels.Source: D. Evaluation Results · p. 10
  • DIA significantly outperforms baselines across QoE metrics and network settings, with comparisons shown for 5G, 4G, and WiFi.Source: Figure 4 · p. 11, D. Evaluation Results · p. 11, Figure 5 · p. 12
  • As mobility increases, DIA’s QoElog and QoEHD advantages become more significant as bandwidth grows more constrained.DIA prioritizes high resolutions for key objects under limited bandwidth.Source: D. Evaluation Results · pp. 10–11
  • Under 4G, MetaABR beats DIA on QoElin and QoElog but loses on QoEHD, while DIA’s error bars are smaller than RobustMPC’s and Pensieve’s.Source: D. Evaluation Results · p. 11
  • In 5G and WiFi, more than 80% of DIA users have QoElin above 80, while more than 90% of baseline users are below 60; all DIA users’ QoEHD values exceed 14.Only a small amount of baseline users’ QoEHD values exceed 14.Source: D. Evaluation Results · p. 11, Figure 5 · p. 12, Figure 7 · p. 12
  • In 4G, DIA balances bitrate, rebuffering, bitrate variation, and latency, while learning-based baselines trade lower latency and rebuffering for reduced frame bitrate.RobustMPC’s bandwidth prediction errors can lead to aggressive bitrate decisions and higher rebuffering, latency, and bitrate difference.Source: D. Evaluation Results · pp. 11–12
  • Under WiFi, approaches achieve similar high average bitrate and low latency, while DIA has the lowest bitrate difference and second-lowest average rebuffering time.Source: D. Evaluation Results · p. 12
  • The relative performance trends from all three proposed QoE metrics are consistent with the subjective MOS evaluation.Source: D. Evaluation Results · p. 12

E. Subjective User Study

The subjective study found higher average ratings for DIA than for the evaluated baselines, while the paper identifies training-data generalization and deployment validation as limitations. The conclusion describes DIA’s object-level bitrate allocation and its performance across network scenarios.

E. Subjective User Study

  • DIA averaged 4.16 MOS across the evaluated clips, compared with Pensieve at 2.49, T-Gaming at 3.74, and MetaABR at 2.86.The study reports that the three proposed QoE metrics followed the same performance trend as subjective MOS.Source: E. Subjective User Study · p. 13
  • DIA’s subjective results, together with its QoE and QoS results, indicate improved user-perceived QoE, especially under bandwidth-constrained conditions.The paper connects this outcome to prioritizing key objects for bitrate allocation.Source: E. Subjective User Study · p. 13
  • Increasing object update frequency from 2 Hz to 4 Hz raised average MOS from 3.70 to 4.05 significantly, while the increase from 4 Hz to 8 Hz was not significant.The paper adopts 4 Hz as the default because it improves perception over 2 Hz while avoiding the substantially higher inference workload associated with 8 Hz.Source: E. Subjective User Study · p. 13

F. Limitations

  • DIA’s training data is mainly from GTA V, so its learned object-importance pattern may not directly generalize to genres with distinct key objects, such as sports games.The paper notes that applying DIA to such genres may require additional data collection and model fine-tuning.Source: F. Limitations · pp. 13–14
  • The subjective study used processed clips to isolate perceptual effects; validation with real-time cloud gaming systems and online encoder adaptation remains future work.The conclusion also identifies full interactive implementation with dynamic encoder adaptation and real-time user control as future work.Source: F. Limitations · p. 14, §VI · p. 14

VI. CONCLUSION AND FUTURE WORK

  • DIA uses semantic segmentation, a transformer-based feature extractor, and an interest-aware reward to prioritize key-object resolution and reduce non-key-object bitrate when bandwidth is limited.Its key-object decisions use in-game movement speed, semantic labels, and network conditions; the conclusion reports better proposed-QoE metrics than SOTA approaches across 5G, 4G, and WiFi.Source: §VI · p. 14
  • The paper plans broader evaluation across games with different visual styles and interaction dynamics, with fine-tuning or transfer learning potentially needed for domain-specific content.Sports games with ball-centric attention patterns are given as an example.Source: §VI · p. 14

APPENDIX

The appendix reports subjective comparisons, checks how its objective QoE metrics align with MOS, and examines update frequency. DIA leads the evaluated baselines in MOS, while 4 Hz is selected as a practical update setting.

APPENDIX

  • The appendix evaluates whether object-aware bitrate allocation matches user perception and examines the effect of object update frequency on MOS.The two evaluations were conducted independently, so their absolute MOS values should be interpreted within their respective contexts.Source: APPENDIX · p. 16

A. Study Protocol

  • The study used processed clips from five game genres under the same time-varying bandwidth-constrained trace, with 34 participants rating outputs on a 1–5 MOS scale.Clip order was randomized; participants could use side-by-side viewing, A/B switching, and wipe comparisons.Source: A. Study Protocol · p. 16

B. MOS Comparison Among DIA and Baseline Approaches

C. Consistency Between the Proposed QoE Metrics and MOS

D. A Representative Example

  • Under the same frame and network condition, DIA preserves higher quality for the enemy character and assigns lower quality to the background vehicle.The comparison contrasts DIA’s semantic object-level allocation with frame-level adaptation by Pensieve and MetaABR and ROI-level differentiation by T-Gaming.Source: D. A Representative Example · p. 17, Figure 9 · p. 18

E. MOS Comparison Under Different Object Update Frequencies

  • Average MOS was 3.70 at 2 Hz, 4.05 at 4 Hz, and 4.17 at 8 Hz; 4 Hz and 8 Hz both significantly outperformed 2 Hz, but not each other.The 4-to-8 Hz difference was not significant (p = 0.096), and 4 Hz was selected to balance responsiveness and computational overhead.Source: E. MOS Comparison Under Different… · p. 17, Table VIII · p. 17

F. Summary

  • The study’s two main findings are that DIA exceeds baseline MOS across evaluated clips and that 8 Hz offers limited improvement over 4 Hz.The paper adopts 4 Hz as a practical balance between perceptual quality and computational overhead.Source: F. Summary · p. 17
Loading 2610.13698v1…