PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond D2D: Full Dimension UAV-to-Everything Communications in 6G

Shuhang Zhang, Hongliang Zhang, Lingyang Song

arXiv:2004.01920v1eess.SPcs.NI

TL;DR

UAV sensing and transmission are coupled, while their performance goals may conflict, motivating flexible communication modes. The paper proposes U2X modes and a reinforcement-learning framework; its evaluation reports more valid transmissions than conventional cellular communication and higher utility than policy gradient.

  • Problem

    UAV trajectory affects both sensing and transmission, whose performance may not improve concurrently.

    Source: B. UAV Trajectory Design · p. 5

  • Method

    U2X offers U2N, U2U, and U2D modes, with joint sensing and transmission protocols, trajectory design, and resource management studied alongside an RL framework.

    Source: §I · p. 2

  • Results

    Over 30% more valid data transmissions on average than conventional cellular mode; proposed-algorithm total utility was over 15% higher than policy gradient.

    Source: D. Performance Analysis · p. 8

  • Takeaways & Limitations

    The proposed U2X framework is presented as supporting varied UAV sensing applications with high transmission rate.

    Source: §VI · p. 10

Abstract

from arXiv · show

In this paper, we consider an Internet of unmanned aerial vehicles (UAVs) over cellular networks, where UAVs work as aerial users to collect various sensory data, and send the collected data to their transmission destinations over cellular links. Unlike the terrestrial users in the conventional cellular networks, different UAVs have various communication requirements due to their sensing applications, and a more flexible communication framework is in demand. To tackle this problem, we propose a UAV-to-Everything (U2X) networking, which enables the UAVs to adjust their communication modes full dimensionally according to the requirements of their sensing applications. In this article, we first introduce the concept of U2X communications, and elaborate on its three communication modes. Afterwards, we discuss the key techniques of the U2X communications, including joint sensing and transmission protocol, UAV trajectory design, and radio resource management. A reinforcement learning-based mathematical framework for U2X communications is then proposed. Finally, the extensions of the U2X communications are presented.

I. INTRODUCTION

Conventional cellular links struggle to serve UAVs with diverse sensing requirements and destinations, particularly when UAVs operate near the cell edge. The paper proposes U2X modes and supporting techniques, then presents an RL-based framework and possible extensions.

I. INTRODUCTION

  • Cell-edge UAVs with destinations beyond the BS cannot reliably meet transmission-rate needs through conventional cellular uploads alone.Sensing UAVs may need to communicate with BSs, terrestrial UEs, or other UAVs.Source: §I · pp. 1–2
  • U2X offers U2N, U2U, and U2D modes so UAVs can choose links for application-specific sensing and transmission requirements.U2N routes data through the BS; U2U enables direct UAV links; U2D sends data directly to a destination device.Source: §I · p. 2
  • U2X differs from V2X in its sensing-and-transmission goals, three-dimensional UAV trajectories, and air-to-ground channel models.The paper argues that these differences call for studying U2X independently.Source: §I · p. 2
  • Key techniques include jointly coordinating sensing and transmission, designing UAV trajectories, and managing shared spectrum and power resources.Mode selection and trajectory design are decentralized, while radio-resource management is centralized.Source: §I · p. 2
  • The proposed RL-based framework seeks to maximize the probability of successful sensing and transmission with U2X communications.The paper also evaluates trajectory design and radio-resource management within this framework.Source: §I · p. 2
  • The paper discusses UAV cooperation, MEC, and NOMA as U2X extensions, alongside open problems and potential solutions.Cooperative UAVs can converge sensory data at one UAV for transmission; MEC and NOMA are presented as additional scenarios.Source: §I · p. 2

II. BASICS OF U2X COMMUNICATIONS

This section introduces the cellular Internet of UAVs and its three communication modes.

II. BASICS OF U2X COMMUNICATIONS

  • The section introduces the cellular Internet of UAVs and the three communication modes used in its network.Source: §II · p. 3

A. The Cellular Internet of UAVs

UAVs sense assigned targets and transmit the resulting data to BSs or UEs through application-selected communication modes. The section establishes the cellular network model and introduces U2N as one transmission option.

A. The Cellular Internet of UAVs

  • The cellular Internet of UAVs comprises a BS, multiple UAVs, and terrestrial UEs that perform sensing and transmission in successive cycles while UAVs follow designed trajectories.Each cycle begins with sensing and is followed by transmission to corresponding destinations.Source: A. The Cellular Internet of UAVs · p. 3

1) UAV Sensing:

  • A UAV collects data at a task-specific sensing target, and successful sensing is necessary to complete the task.Sensing performance is evaluated with a probabilistic model related to the UAV's distance from the target.Source: 1) UAV Sensing · p. 3

2) UAV Transmission:

  • After sensing, a UAV transmits data to a BS or UE using a mode selected according to its location and application requirements.The three modes are U2N, U2U, and U2D.Source: 2) UAV Transmission · p. 3

B. U2X Communication Modes

1) UAV-to-Network Mode:

  • In U2N mode, a UAV uploads data to the BS over cellular uplink spectrum, supporting high data rate and low latency when it is near the BS.The BS can forward collected data to terrestrial UEs over the existing infrastructure.Source: 1) UAV-to-Network Mode · p. 3

2) UAV-to-UAV Mode:

U2U and U2D provide direct transmission options, while U2X operation requires coordinated sensing, mode selection, and resource allocation. The proposed protocol divides those decisions between UAVs and the BS.

2) UAV-to-UAV Mode:

  • U2U links connect adjacent UAVs while sharing spectrum with cellular and U2N communications, reducing latency and supporting data diffusion.Underlay transmission also improves network spectrum efficiency.Source: 2) UAV-to-UAV Mode · p. 3

3) UAV-to-Device Mode:

  • U2D lets a UAV send data directly to a destination UE, bypassing the BS and reducing transmission latency.It is especially efficient when the UAV is near its destination UE.Source: 3) UAV-to-Device Mode · p. 3

III. KEY TECHNIQUES FOR U2X COMMUNICATIONS

  • U2X techniques address coupled sensing and transmission, trajectory design under limited real-time CSI, and shared-spectrum resource management.The section identifies subchannel allocation and power control as means to reduce co-channel interference.Source: §III · p. 4

A. Joint Sensing and Transmission Protocol

  • The protocol combines decentralized UAV mode and trajectory choices with centralized BS radio-resource management in synchronized cycles.Each cycle includes mode selection, channel assignment, sensing, and transmission; UAVs set application-specific QoS thresholds and select the highest-rate mode that meets them.Source: A. Joint Sensing and Transmission… · p. 4

B. UAV Trajectory Design

UAV trajectory design must account for sensing and transmission together, since positioning for one can trade off against performance of the other. The paper identifies decentralized design challenges and presents multi-agent deep reinforcement learning as a promising approach.

B. UAV Trajectory Design

  • The UAV trajectory determines future UAV locations, and the model considers large-scale fading and average transmission QoS because real-time small-scale fading is difficult to obtain.UAV location affects both sensing success probability and transmission SNR.Source: B. UAV Trajectory Design · pp. 4–5
  • When sensing targets and transmission destinations lie in different directions, trajectories must trade off successful sensing probability against transmission QoS according to application requirements.Sensing success depends on UAV location, as does the received SNR at the transmission destination.Source: B. UAV Trajectory Design · p. 5
  • Decentralized design must handle application-specific modes and requirements, predict other UAVs’ performance on U2U links, and address non-convexity from co-channel interference.The passage says centralized BS trajectory design is especially difficult and that each UAV should design independently.Source: B. UAV Trajectory Design · p. 5
  • Multi-agent deep RL is presented as a promising approach in which each UAV independently learns trajectory actions from environmental interactions and application-determined rewards.The approach learns from actions and rewards in prior cycles rather than solving the non-convex problem directly.Source: B. UAV Trajectory Design · p. 5

C. Radio Resource Management

Radio resource management allocates shared spectrum and transmission power despite difficulty obtaining real-time CSI from mobile UAVs. The section describes distinct allocation approaches for overlay and underlay links, alongside power control that accounts for interference and application-specific utility.

C. Radio Resource Management

  • Because the BS has difficulty obtaining real-time CSI from highly mobile UAVs, it uses large-scale fading—mainly determined by UAV and target locations—for radio resource management.The UAVs share terrestrial cellular spectrum resources.Source: C. Radio Resource Management · p. 5

1) Subchannel Allocation:

  • U2N overlay allocation can be relaxed to a convex problem, while underlay U2U and U2D allocation is non-convex and can use iterative matching based on UAV and subchannel utilities.The allocation must account for air-to-ground channels in U2N and U2D links, unlike conventional ground-to-ground D2D methods.Source: 1) Subchannel Allocation · p. 5

2) Power Control:

  • Power control trades improved link QoS from higher transmit power against increased co-channel interference, with link utilities determined by sensing applications.With fixed allocation and trajectories, single-link subchannels yield linear power problems; shared subchannels yield non-convex problems addressed using difference-of-convex approximations and successive convex problems.Source: 2) Power Control · pp. 5–6

IV. REINFORCEMENT LEARNING FRAMEWORK FOR U2X COMMUNICATIONS

The paper proposes a reinforcement-learning algorithm for trajectory design and radio resource management, evaluated by simulation. Its example aims to maximize valid transmissions by jointly accounting for successful sensing and transmission, with trajectory, allocation, power, and operational constraints represented in the model.

IV. REINFORCEMENT LEARNING FRAMEWORK FOR U2X COMMUNICATIONS

  • An RL-based algorithm addresses UAV trajectory design and radio resource management, and its performance is evaluated through simulations.The section presents an example of sensing and transmission design in the cellular Internet of UAVs.Source: §IV · p. 6

A. Problem Description

  • The example maximizes average valid data transmissions per cycle, counting a transmission as valid only when the UAV both senses its target and transmits the data successfully.The objective is expressed using successful sensing and successful transmission probabilities.Source: A. Problem Description · p. 6
  • The model makes sensing success depend on UAV trajectory and transmission success depend on allocation, trajectory, power, and application-specific data amount.The variables represent trajectory, subchannel allocation, and transmission power; allocation is binary and the model includes velocity and maximum-power constraints.Source: A. Problem Description · p. 6

B. Brief Introduction to Reinforcement Learning

The section outlines reinforcement learning as repeated agent-environment interaction guided by rewards. It describes Q-learning for small state-action spaces and deep neural networks for estimating Q-values in larger spaces.

B. Brief Introduction to Reinforcement Learning

  • Each RL agent selects actions from a discrete action space, observes state transitions, and learns through repeated interaction with the environment and reward signals.The framework contains multiple agents with independent data-processing capability.Source: B. Brief Introduction to Reinforcement… · p. 6
  • A Q-value represents expected discounted long-term reward for a state-action pair, and agents update a Q-table to derive a policy.The policy selects actions to maximize long-term reward.Source: B. Brief Introduction to Reinforcement… · p. 6
  • For large state-action spaces, DQN uses a deep neural network to estimate Q-values, whereas small spaces can use Q-learning lookup tables.The network learns mappings between UAV states and corresponding Q-values from training data.Source: B. Brief Introduction to Reinforcement… · pp. 6–7

C. Algorithm for Valid Data Transmission Maximization

The trajectory optimizer treats each UAV as an agent in a multi-agent deep reinforcement-learning framework, choosing movements to maximize discounted valid transmissions. Nested Markov chains represent sensing and transmission outcomes, while the base station performs radio-resource management after trajectory design.

C. Algorithm for Valid Data Transmission Maximization

  • Nested Markov chains represent sensing across cycles and transmission across frames, with sensing-success or failure states and mode-specific transmission-success or failure states.The outer chain has two states; the inner chain has four states for U2N, U2U, U2D, and transmission failure.Source: C. Algorithm for Valid Data… · p. 7
  • Each UAV has 27 candidate trajectories per cycle, and its reward is the discounted value of expected valid transmissions across future cycles.The 27 moves correspond to discrete locations in three-dimensional space; discounting reflects the timeliness requirements of sensing tasks.Source: C. Algorithm for Valid Data… · p. 7, Figure 4 · p. 7
  • A DQN approximates Q-values because the trajectory problem’s state-action space is too large for lookup tables.Training uses a separate target network, whose weights are periodically updated from the DQN.Source: C. Algorithm for Valid Data… · p. 7
  • After UAVs design their trajectories, the base station jointly manages radio resources, iteratively decoupling subchannel allocation and power control until convergence.Subchannel allocation uses a matching-based algorithm, while power control uses a difference-of-convex algorithm.Source: C. Algorithm for Valid Data… · pp. 7–8

D. Performance Analysis

In the simulated cell, U2X improves valid data transmissions over conventional cellular communication and the proposed algorithm outperforms a policy-gradient comparison. Results also illustrate location-dependent mode selection and successful sensing and transmission across much of the cell.

D. Performance Analysis

  • The simulation uses five UAVs and two terrestrial UEs, with a 10 dBm maximum UAV transmission power and a 1 bit/s/Hz transmission QoS threshold.The trajectory step length is 25 m.Source: D. Performance Analysis · p. 8
  • Over 30% more valid data transmissions on average are achieved by U2X than by conventional cellular communication.The performance gap is larger when the number of subchannels is small, as U2U and U2D links can underlay terrestrial links.Source: D. Performance Analysis · p. 8
  • Over 15% higher total utility is achieved by the proposed algorithm than by the policy-gradient algorithm, with convergence within 70 episodes.Using three rather than one hidden layer improves network utility by about 10% but requires more episodes to converge.Source: D. Performance Analysis · p. 8
  • UAV mode selection varies with location: U2D is preferred near the destination, U2N near the base station, and U2U when far from both.In the U2U case, an adjacent UAV relays the sensory data to the destination.Source: D. Performance Analysis · pp. 8–9
  • With U2X, sensing and transmission are feasible in most cell locations, and successful sensing is the main factor affecting their joint probability.With conventional cellular transmission, sensing and transmission can occur only when the UAV is close to both the sensing target and the base station.Source: D. Performance Analysis · p. 9

V. EXTENSIONS WITH U2X COMMUNICATIONS

The paper presents UAV cooperation as one of three possible U2X extensions and identifies routing as an open problem. Cooperation uses multi-hop transfers to serve remote, high-rate sensing tasks.

V. EXTENSIONS WITH U2X COMMUNICATIONS

  • The extensions section outlines three possible U2X scenarios and discusses open problems and potential solutions.The supplied discussion describes UAV cooperation as one of those scenarios.Source: §V · p. 9, A. UAV Cooperation with U2X… · p. 9

A. UAV Cooperation with U2X Communications

  • Cooperative UAVs can send data through a UAV with the best channel to the destination, which forwards it to the base station or destination UE; routing must account for rapidly changing, sparse UAV networks and onboard constraints.The paper notes that existing routing protocols are mostly heuristic and few consider onboard energy and buffer; it calls for protocols jointly considering trajectories and physical constraints.Source: A. UAV Cooperation with U2X… · p. 9

B. Mobile Edge Computing with U2X Communications

The paper discusses mobile edge computing and non-orthogonal multiple access as U2X extensions, then concludes by recapping the framework and its proposed optimization technologies.

B. Mobile Edge Computing with U2X Communications

  • With U2X mobile edge computing, cell-edge UAVs can process offloaded sensory data and send results directly to destination UEs, reducing base-station computation pressure and transmission distance.Open issues include allocating tasks between the base station and UAVs and accounting for the limited batteries of computing UAVs.Source: B. Mobile Edge Computing with U2X… · p. 10

C. Non-Orthogonal Multiple Access for U2X Communications

  • NOMA can let UAVs in different communication modes share a channel through power- or code-domain multiplexing, with successive interference cancellation proposed to address co-channel interference.The paper says NOMA-based cellular U2X design remains to be studied in spectrum management, power control, and signaling control.Source: C. Non-Orthogonal Multiple Access for… · p. 10

VI. CONCLUSIONS

  • The conclusion recaps U2N, U2U, and U2D modes, three key technology areas, and an RL-based framework for maximizing valid data transmissions.It also lists UAV cooperation, mobile edge computing, and NOMA among the discussed extensions and open problems.Source: §VI · p. 10
Loading 2004.01920v1…