PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach

Liang Wang, Kezhi Wang, Cunhua Pan, Nauman Aslam

arXiv:2007.08380v2eess.SP

TL;DR

Building obstructions can block UAV–UE communication, motivating IRS assistance. The paper jointly optimizes UAV trajectories and IRS phase shifts with DQN for discrete trajectories and DDPG for continuous trajectories. DDPG achieves higher reported energy efficiency than DQN, while DQN trains faster.

  • Problem

    Buildings can block UAV–UE signals, while the paper considers communication with a moving UE and IRS selection in the system design.

    Source: §I · p. 2

  • Method

    The paper jointly optimizes UAV trajectory and IRS phase shifts, applying DQN to discretized trajectories and DDPG to continuous trajectories.

    Source: §I · p. 2, A. DQN-based Algorithm for Discrete… · p. 5

  • Results

    DDPG's average UAV energy efficiency rises from 52 bps/J to 70 bps/J; DQN performs slightly worse than DDPG, which also outperforms FF and RR.

    Source: §V · pp. 8–9

  • Takeaways & Limitations

    DQN's shorter training time may suit time-sensitive cases, while DDPG offers better performance by optimizing system variables continuously.

    Source: §VI · p. 10, §V · p. 10

Abstract

from arXiv · show

In this paper, the intelligent reflecting surface (IRS)-aided unmanned aerial vehicle (UAV) communication system is studied, where the UAV is deployed to serve the user equipment (UE) with the assistance of multiple IRSs mounted on several buildings to enhance the communication quality between UAV and UE. We aim to maximize the energy efficiency of the system, including the data rate of UE and the energy consumption of UAV via jointly optimizing the UAV's trajectory and the phase shifts of reflecting elements of IRS, when the UE moves and the selection of IRSs is considered for the energy saving purpose. Since the system is complex and the environment is dynamic, it is challenging to derive low-complexity algorithms by using conventional optimization methods. To address this issue, we first propose a deep Q-network (DQN)-based algorithm by discretizing the trajectory, which has the advantage of training time. Furthermore, we propose a deep deterministic policy gradient (DDPG)-based algorithm to tackle the case with continuous trajectory for achieving better performance. The experimental results show that the proposed algorithms achieve considerable performance compared to other traditional solutions.

I. INTRODUCTION

UAV mobility and IRS reflection are studied as ways to improve communication when buildings block links. The paper combines trajectory and IRS phase-shift optimization with discrete- and continuous-action reinforcement learning.

I. INTRODUCTION

  • UAV mobility has been studied for improving communication performance, while UAV applications also span edge computing, data collection, and wireless power transfer.The cited examples include trajectory and resource-allocation optimization for throughput, energy, and other objectives.Source: §I · pp. 1–2

A. IRS-aided UAV Communications

  • Building obstructions can block UAV–UE signals; passive IRS elements adjust reflection phase shifts to improve data rate and are more energy-efficient than traditional relay-aided techniques.IRSs may be mounted on building exteriors to improve the channel condition between UAV and UE.Source: §I · p. 2

I. INTRODUCTION

  • The paper jointly optimizes UAV trajectory and IRS phase shifts while accounting for UE movement and IRS selection, using DQN for discretized trajectories and DDPG for continuous trajectories.The authors report that DQN trains faster but has a performance loss, while DDPG targets better performance.Source: §I · p. 2, B. DRL Background · p. 3

A. IRS-aided UAV Communications

  • Prior IRS-aided UAV work used convex optimization, which may be suboptimal and time-consuming, with complexity that may grow with the number of reflecting elements.The introduction also identifies prior DRL work that did not consider both IRS selection and UE movement.Source: A. IRS-aided UAV Communications · p. 3, B. DRL Background · p. 3

III. SYSTEM MODEL

The system model considers one rotary UAV serving a moving UE in a crowded area with help from IRSs mounted on buildings. The direct UAV–UE link is omitted, and each IRS has a reflecting array.

III. SYSTEM MODEL

  • The modeled system has one rotary UAV, K building-mounted IRSs, and one moving UE in a crowded area where buildings and trees cause severe path loss and attenuation.The direct UAV–UE link is not considered; IRSs are deployed to enhance UE communication quality.Source: §III · p. 3
  • The UAV operates over a rectangular target area and within an altitude range for a finite sequence of time slots.The time-slot duration is denoted by td.Source: §III · p. 3
  • Each IRS uses a uniform rectangular array with Mr × Mc passive reflecting elements whose phase shifts can boost useful signal power.Source: §III · p. 4

A. UAV model

The UAV model represents motion through bounded horizontal and vertical flying actions in each time slot. It accounts for propulsion energy while omitting communication circuitry and signal-processing energy.

A. UAV model

  • In each time slot, the UAV's flying action consists of bounded horizontal distances ax_t and ay_t and a bounded vertical distance az_t.The UAV's coordinates are determined from its initial position and these actions.Source: A. UAV model · p. 4
  • The UAV's motion is constrained not to cross the target-area border.Source: A. UAV model · p. 4
  • Communication-circuitry and signal-processing energy are ignored relative to propulsion energy, which is modeled per time slot.The model's fixed constants are obtained from.Source: A. UAV model · p. 4

B. Channel model

The channel model accounts for the moving UE, UAV–IRS and IRS–UE links, and passive IRS phase shifts. A TDMA schedule selects one IRS per time slot to save IRS energy.

B. Channel model

  • The UE location varies over time, and the model defines the UAV–IRS and IRS–UE distances for each time slot.Source: B. Channel model · p. 4
  • The UAV–IRS link uses a 3-D path-loss model with elevation-angle and LoS/NLoS-related parameters.The passage identifies the carrier frequency and speed of light, and environmental constants, in the channel description.Source: B. Channel model · p. 4
  • Each IRS element passively adjusts its phase shift, forming a diagonal phase-shift matrix used for the achievable UAV–IRS–UE link data rate.The IRS–UE link description includes path loss and a LoS component.Source: B. Channel model · p. 4
  • TDMA selects only one IRS in each time slot, allowing other IRSs to be switched off or placed in sleep mode to save energy.The schedule factor is 1 for the selected IRS and 0 otherwise; alternative selection schemes may also be used.Source: B. Channel model · pp. 4–5

C. Problem Formulation

The paper formulates an energy-efficiency objective that accounts for UE data rate and UAV energy consumption. Because the problem is non-convex, it proposes discrete- and continuous-trajectory reinforcement-learning methods and a phase-alignment scheme.

C. Problem Formulation

  • The objective maximizes UAV energy efficiency by accounting for UE data rate and UAV energy consumption.The paper formulates this as an optimization problem over the indicated variables.Source: C. Problem Formulation · p. 5
  • Because the problem is generally non-convex, the paper proposes DQN with a discretized trajectory and DDPG with continuous actions.The paper notes that discretization has a training-time advantage but may incur some performance loss; continuous actions are proposed for better performance.Source: C. Problem Formulation · p. 5
  • A low-complexity phase-alignment scheme is also proposed to optimize the reflecting-element phase shifts.Source: C. Problem Formulation · p. 5

A. DQN-based Algorithm for Discrete Cases

The DQN-based method treats the IRS-aided UAV system as an environment and uses discrete actions to optimize the UAV trajectory. An agent uses state, action, and reward information to learn its decisions, while replay memory and target-network updates support training.

A. DQN-based Algorithm for Discrete Cases

B. DDPG-based Algorithm For Continuous Cases

For continuous trajectory optimization, the paper uses a DDPG actor-critic approach. The actor generates actions from states, while the critic estimates state-action values and the networks are trained from replayed transitions.

B. DDPG-based Algorithm For Continuous Cases

C. Phase Shift Optimization

The paper presents a low-complexity scheme for optimizing the phase shifts of reflecting elements at the selected IRS. It uses the UAV and UE coordinates in a time slot to align phase shifts for maximizing UE data rate.

C. Phase Shift Optimization

V. SIMULATION RESULT

The simulations compare DQN and DDPG with fixed and random baselines across reward, energy efficiency, UAV trajectories, and training time. DDPG achieves stronger performance, while DQN trains faster.

V. SIMULATION RESULT

  • Simulations use three IRSs and 100 time slots with a UE moving at 6 m/s; FF and RR provide fixed and random movement and phase-shift baselines.The UE moves from [0, 150] to [600, 150]; the UAV starts at [0, 300, 50].Source: §V · p. 7
  • With three IRSs, rewards initially remain negative, then rise and stabilize; more reflecting elements increase rewards, and DDPG out-rewards DQN at equal counts.The authors attribute DDPG's reward advantage to continuous rather than limited discrete actions.Source: §V · p. 8, Figure 4 · p. 8
  • DDPG energy efficiency rises from 52 to 70 bps/J and exceeds DQN, FF, and RR across reflecting-element counts.Fig. 5 presents average energy efficiency for the four algorithms as the number of reflecting elements varies.Source: Figure 5 · p. 8, §V · pp. 8–9
  • As the UE moves, the UAV flies toward the selected IRS and stays close; DDPG produces a better trajectory than DQN.Fig. 6 depicts the DQN and DDPG trajectories, with dots marking the UE and triangles marking IRSs.Source: §V · p. 9, Figure 6 · p. 9
  • Training time increases with the number of IRS reflecting elements, but DQN consistently trains faster than DDPG because its structure is simpler.The paper notes that training time varies with the hardware platform.Source: Figure 7 · p. 9, §V · pp. 9–10

VI. CONCLUSION

The conclusion presents joint trajectory and passive phase-shift optimization for a moving-UE, IRS-aided UAV system. It contrasts DQN's training-time advantage with DDPG's better performance.

VI. CONCLUSION

  • The paper jointly optimizes UAV trajectory and IRS passive phase shifts while accounting for UE movement and IRS selection.Source: §VI · p. 10
  • DQN discretizes the trajectory and offers a training-time advantage, but incurs performance loss; the authors identify it as suitable when training time is sensitive.Source: §VI · p. 10
  • DDPG optimizes system variables continuously for better performance, and the experiments report that both proposed algorithms outperform traditional solutions.Source: §VI · p. 10

Related papers

Loading 2007.08380v2…