PaperlayerView exact revision on arXiv

Source-linked AI summary

Virtual Reality over Wireless Networks: Quality-of-Service Model and Learning-Based Resource Management

Mingzhe Chen, Walid Saad, Changchuan Yin

arXiv:1703.04209v2cs.IT

TL;DR

Wireless VR over small cell networks must account for tracking accuracy, processing delay, and transmission delay while jointly managing uplink and downlink resources. The paper models this management as a noncooperative game and uses echo state networks to learn resource-allocation actions, reporting improved delay and convergence behavior over comparison methods.

  • Problem

    Wireless VR requires accurate, low-delay tracking and wireless connectivity because user movement affects VR-video generation and wired control restricts immersive applications.

  • Method

    The paper combines a multi-attribute VR QoS model with a noncooperative game and an echo-state-network algorithm that lets SBSs autonomously choose actions based on user and network states.

  • Results

    The proposed algorithm achieves up to 19.6% lower average delay than Q-learning with 6 SBSs, meets a typical 20 ms VR delay requirement, and improves delay CDF gains over Q-learning and proportional fairness.

  • Takeaways & Limitations

    The ESN-based framework supports distributed SBS resource management with faster convergence and low-delay wireless VR transmission.

Abstract

from arXiv · show

In this paper, the problem of resource management is studied for a network of wireless virtual reality (VR) users communicating over small cell networks (SCNs). In order to capture the VR users' quality-of-service (QoS) in SCNs, a novel VR model, based on multi-attribute utility theory, is proposed. This model jointly accounts for VR metrics such as tracking accuracy, processing delay, and transmission delay. In this model, the small base stations (SBSs) act as the VR control centers that collect the tracking information from VR users over the cellular uplink. Once this information is collected, the SBSs will then send the three dimensional images and accompanying surround stereo audio to the VR users over the downlink. Therefore, the resource allocation problem in VR wireless networks must jointly consider both the uplink and downlink. This problem is then formulated as a noncooperative game and a distributed algorithm based on the machine learning framework of echo state networks (ESNs) is proposed to find the solution of this game. The use of the proposed ESN algorithm enables the SBSs to predict the VR QoS of each SBS and guarantees the convergence to a mixed-strategy Nash equilibrium. The analytical result shows that each user's VR QoS jointly depends on both VR tracking accuracy and wireless resource allocation. Simulation results show that the proposed algorithm yields significant gains, in terms of total utility value of VR QoS, that reach up to 22.2% and 37.5%, respectively, compared to Q-learning and a baseline proportional fair algorithm. The results also show that the proposed algorithm has a faster convergence time than Q-learning and can guarantee low delays for VR services.

I. INTRODUCTION

Wireless VR over small cell networks requires modeling tracking and delay while jointly allocating uplink and downlink resources. The paper proposes a comprehensive framework combining a multi-attribute VR QoS model, SBS resource-allocation game, and ESN-based learning algorithm.

  • Motivation: Wireless VR differs from conventional HD video because content depends on users’ movement and requires accurate, timely tracking information.Tracking accuracy and tracking-transmission delay affect VR video creation and transmission.
  • Motivation: Existing VR studies largely focus on wired systems or single-user WiFi experiments and generally do not capture wireless SCN challenges.Prior work also commonly optimizes only a single VR quality-of-service aspect.
  • VR QoS model: The proposed VR model jointly captures tracking accuracy, transmission delay, and processing delay to quantify wireless users’ VR QoS.Tracking information travels over the cellular uplink, while VR images are delivered over the downlink.
  • Resource management: Resource blocks are allocated jointly over uplink and downlink, with SBSs modeled as players in a noncooperative game maximizing VR QoS utility.Uplink blocks carry sensor data, whereas downlink blocks carry displayed images.
  • Learning algorithm: An ESN-based learning algorithm predicts VR QoS from resource-allocation actions and is designed to converge to a mixed-strategy Nash equilibrium.The algorithm allows SBSs to make autonomous decisions using users’ and networks’ states.
  • Evaluation: 22.2% and 37.5% VR QoS utility gains are reported over Q-learning and proportional fair, respectively.The paper also reports up to 24.2% faster convergence than Q-learning.

2) Delay:

The model separates transmission and processing delay in the VR pipeline. Processing delay depends on tracking accuracy, associated-user load, and allocated resources, while transmission delay depends on link rates and resource-block allocation.

  • Transmission delay: Transmission delay is defined for each user and depends on the user’s link rates and downlink resource-block vector.The downlink vector determines image transmission, while uplink and downlink rates characterize the corresponding links.
  • Processing delay: Processing delay represents the SBS time required to update a VR image using corrected image content.Motion search identifies changed pixels, which the SBS can replace directly.
  • Processing delay: The model computes changed image content from motion search, subject to a maximum correction size L.The motion-search output depends on image size, pixel representation, and VR-image content.
  • Tracking and delay coupling: Tracking accuracy depends on uplink transmission errors because the sensors are assumed to collect user movement accurately.Uplink SINR and allocated uplink resource blocks affect the tracking-vector transmission.
  • Processing delay: Processing delay depends on tracking accuracy, the number of users associated with the SBS, and resource blocks allocated to the user.The model also allocates computation resources to each VR session.

B. Utility Function Model

The paper uses multi-attribute utility theory to jointly model VR tracking and delay. This avoids assuming that the two attributes are independent and linearly related, while tying utility to both uplink and downlink allocation.

  • Utility construction: The proposed utility model jointly captures tracking accuracy, transmission delay, and processing delay for wireless VR users.It is based on multi-attribute utility theory rather than a simple sum of attributes.
  • Utility construction: Directly summing delay and tracking is valid only when they are independent and linearly related, assumptions the VR model does not make.The paper therefore constructs a total utility function through conditional utility functions.
  • Delay utility: The conditional delay utility uses the maximal tolerable delay γD as the VR system’s delay requirement.When delay remains below γD, the conditional utility value remains 1.
  • Total utility: Delay and tracking are treated as dominant components that jointly determine the total VR QoS utility.Each component receives a distinct utility value within the total utility function.
  • Resource dependence: The uplink and downlink resource vectors jointly determine each user’s total utility value.The formulation assigns a unique value to each tracking and delay component.

C. Problem Formulation

The paper formulates joint user association and uplink/downlink resource allocation as a noncooperative game among SBSs. Each SBS selects resource-allocation actions to maximize average VR QoS under interaction with other SBSs.

  • Optimization objective: The optimization jointly considers user association, uplink resource allocation, and downlink resource allocation to maximize users’ VR QoS.The coupled problem is formulated as a resource-management objective.
  • Game formulation: An SBS’s VR QoS depends on both its own resource allocation and the decisions of other SBSs.This interaction makes centralized optimization largely intractable and motivates a distributed game.
  • Game formulation: Each SBS is a player whose action includes downlink resource allocation and additional allocation decisions represented in its action profile.Actions are selected from an SBS-specific action set.
  • Constraints: Resource-block constraints allow each downlink or uplink resource block to be allocated to at most one VR user.The formulation also restricts association to users within an SBS’s coverage area.
  • Equilibrium objective: The mixed-strategy Nash equilibrium gives each SBS a strategy that maximizes its average VR QoS given opponents’ actions.Utility is averaged over a period of T time slots.

III. ECHO STATE NETWORKS FOR SELF-ORGANIZING RESOURCE ALLOCATION

The paper introduces an echo state network learning algorithm for the VR resource-allocation game. The larger joint uplink/downlink action space makes traditional fictitious-play approaches difficult for SBSs to use.

  • Learning motivation: Joint uplink and downlink allocation creates many more actions than conventional one-direction resource-allocation scenarios.The increased action count challenges traditional game-theoretic learning approaches.
  • Learning approach: The proposed ESN-based learning algorithm is used to solve the VR game and find its Nash equilibrium.The algorithm is introduced specifically to address the enlarged action space.

A. ESN Components

The ESN resource-allocation architecture represents SBSs as agents whose actions jointly allocate uplink and downlink resource blocks. It estimates utilities from strategy distributions while using exploration and VR-QoS gain analysis to guide action selection.

  • Each SBS is an agent whose action jointly specifies uplink and downlink resource-block allocations.
  • ε-greedy exploration ensures every action retains nonzero selection probability while balancing exploration and exploitation.
  • Tracking accuracy and uplink resource blocks directly affect VR-QoS gains, while downlink gains depend on users’ data rates.
  • The ESN input encodes each SBS’s probability-distribution index and estimates its average VR QoS.
  • The ESN maps strategy distributions to utility values using output weights and a dynamic reservoir that stores historical information.
  • The number of actions increases with users and resource blocks, with uplink actions exceeding downlink actions because uplink allocations change interference.

B. ESN-Based Learning Algorithm for Resource Allocation

The learning algorithm uses reservoir states containing past mixed strategies to estimate conditional utilities. It trains the output weights by gradient descent and uses those estimates to update action selection.

  • The dynamic reservoir state combines past reservoir states with the SBS’s mixed strategy at the current time.
  • The ESN combines the reservoir state with output weights to estimate the utility associated with each candidate action.
  • The output weight matrix is trained through linear gradient descent using the learning rate and the difference between actual and estimated utility.

C. Convergence of the ESN-Based Learning Algorithm

The paper analyzes convergence of the ESN algorithm under conditions on its input weights or learning rate. It establishes convergence to a mixed-strategy Nash equilibrium, while noting that the equilibrium need not be unique.

  • The ESN algorithm converges to its utility value when the learning rate is constant under the stated input-weight condition or satisfies the Robbins–Monro conditions.
  • Convergence depends on the input-weight matrix and input sequence, which also determine the ESN’s memory capacity.
  • The algorithm can converge to the conditional utility function even when the input weights and inputs are generated randomly.
  • One ESN is sufficient for the proposed convergence guarantee, unlike prior work that used two echo state networks.
  • The ESN-based learning algorithm converges to a mixed-strategy Nash equilibrium for each SBS.
  • The resulting mixed-strategy Nash equilibrium is not unique because the equilibrium depends on utility values that are not unique.

D. Implementation and Complexity

The algorithm is distributed: SBSs exchange compact mixed-strategy indices while updating their local ESN states and outputs. Its worst-case complexity depends on the joint action space, although prediction from stored history can reduce the need to test every action.

  • Each SBS updates its ESN locally and broadcasts its mixed-strategy index to other SBSs during each iteration.
  • The exchanged strategy index can use fewer than 16 bits, making SBS interactions independent of network size relative to VR content and tracking data.
  • The worst-case complexity is O(|A1|×· · ·×|AB|), corresponding to traversing all actions across SBSs.
  • Stored ESN information lets SBSs predict utility without implementing all possible actions, reducing the algorithm’s complexity during convergence.

IV. SIMULATION RESULTS AND ANALYSIS

Simulations evaluate the proposed VR QoS model and ESN-based resource-management algorithm across network sizes, user loads, delay, utility, and convergence. The proposed approach improves utility and delay outcomes over Q-learning and proportional fair baselines.

  • Simulation setup: The simulations use a 100 m-radius SCN with 25 users, 4 SBSs, HTC Vive imagery at 1920 × 1080 pixels, 60 images per second, and compression factor 150.Each user requires 25.3125 Mbit/s after compression.
  • QoS model evaluation: VR QoS utility is zero when either tracking or delay utility is zero, and is maximized only when both utilities equal 1.With tracking accuracy 1, utility is maximized when a downlink resource-block group exceeds 4 MHz.
  • Delay performance: As the number of SBSs increases, average delay utility increases and transmission delay decreases, although interference causes utility gains to slow.The proposed algorithm achieves up to 19.6% average-delay gain over Q-learning with 6 SBSs and meets a typical 20 ms VR delay requirement.
  • Delay performance: As user count increases, average delay rises because fewer resource blocks are allocated per user, while the proposed algorithm achieves a 29.8% average-delay gain over proportional fair.The ESN approximates QoS values and records more action utilities than Q-learning’s matrix as the action space grows.
  • Utility performance: 22.2% and 37.5% gains in total VR QoS utility are achieved over Q-learning and proportional fair, respectively, with 35 users.Total utility values increase with user count, although users beyond SBS service capacity receive zero VR QoS.
  • Convergence: The proposed ESN approach reaches its best mixed-strategy Nash equilibrium in 19 iterations versus 25 for Q-learning, a 24.2% convergence gain.The worst proposed mixed-strategy equilibrium still achieves a 14% total-utility gain over Q-learning.

V. CONCLUSION

The paper develops a VR QoS model and an ESN-based resource-allocation framework for wireless SCNs. It formulates SBS interactions as a noncooperative game and reports improved utility and convergence performance.

  • Conclusion: The multi-attribute VR model jointly captures tracking accuracy, transmission delay, and processing delay.The model quantifies VR QoS for wireless users through these tracking and delay components.
  • Conclusion: Resource allocation is formulated as a noncooperative game in which SBSs autonomously select actions based on users’ and network states.The objective is to maximize average VR QoS utilities for users.
  • Conclusion: The ESN-based learning algorithm updates mixed strategies during training and converges quickly to a mixed-strategy Nash equilibrium.Each SBS uses the learned model to decide actions autonomously.
  • Conclusion: Simulation results show that the proposed framework captures wireless VR QoS while providing significant performance gains.The conclusion summarizes gains from both the QoS model and the resource-management algorithm.

APPENDIX

The appendix analyzes resource-allocation action gains and establishes convergence properties for the ESN-based learning algorithm. It also argues that finite reservoir memory makes the learning process a finite-order MDP under stated conditions.

  • Resource-allocation analysis: The appendix derives utility gains from increasing a user’s uplink resource blocks and from changing its downlink resource blocks.These gains support the analysis of how resource-allocation actions affect VR utility.
  • Action-space analysis: Because all downlink resource blocks are allocated to associated users, interference remains unchanged when block identities change, so actions depend on allocation counts.The appendix reduces the downlink action analysis to the number of blocks assigned to each user.
  • Action-space analysis: The uplink action count is derived sequentially by allocating resource blocks to users and deriving later users’ actions from earlier allocations.The appendix completes the combined uplink and downlink action-count derivation.
  • ESN convergence: ESN convergence is analyzed by proving nondivergence and deriving the value to which the ESN converges under convergence conditions.The stated conditions include k-step unambiguity and an appropriate learning process.
  • ESN convergence: Finite reservoir memory means the dynamic reservoir state depends only on finitely many past states, making the proposed process a finite-order MDP.The appendix also notes that the learning rate must satisfy Robbins–Monro conditions.
Loading 1703.04209v2…