Reflections in the Sky: Millimeter Wave Communication with UAV-Carried Intelligent Reflectors

• mmW networks face severe blockage from common objects, making reliable links a major barrier to wide-scale deployment.Source: §I · p. 1
• Intelligent reflectors bypass obstacles by connecting LOS links, while passive components provide energy-efficient beamforming and RF energy harvesting.Source: §I · p. 1
• Fixed reflectors cannot adapt to dynamic mmW channels, whereas a UAV-IR can adjust its location to maintain LOS links with moving transmitters and receivers.Source: §I · p. 1
• The introduction identifies real-time IR-UE CSI estimation as a central challenge for efficient UAV-IR deployment.Source: §I · p. 2
• The paper uses Q-learning and neural networks to optimize UAV-IR location and reflection coefficients, while harvesting RF energy to power the IR.Source: §I · p. 2

• A UAV-IR assists NLOS users by reflecting BS mmW signals toward them when blockage substantially increases channel attenuation.Source: §II · p. 2
• The UAV adjusts the IR position to maintain LOS links with both the BS and UE, replacing one NLOS path with two connected LOS links.Source: §II · p. 2
• UE feedback is relayed through UE-BS and BS-UAV control links, while the IR harvests energy from the unreflected beamforming signal fraction.Source: §II · p. 2
• The optimization targets UAV-IR deployment for maximum downlink capacity while harvesting RF energy to power the reflector.Source: §II · p. 2

• The BS-IR channel CSI is fully known, whereas the IR-UE CSI is treated as variable because mmW links are susceptible to blockage.Source: A. Channel Model and Communications… · p. 2
• Each IR component reflects part of the incident signal with adjustable phase and amplitude while harvesting the remaining signal through a rectifier.Source: A. Channel Model and Communications… · p. 2
• The UAV location, UE position, and UE-body angle determine the received reflected signal, with UE state values updated through control links.Source: A. Channel Model and Communications… · p. 2
• When received SNR falls below threshold τ, the UE cannot decode; otherwise, downlink capacity is c(x, y, ω, Θ) = b log2 (1 + η(x, y, ω, Θ)).Source: A. Channel Model and Communications… · p. 3

• UAV-IR power consumption includes hovering, mobility, and reflection, with mobility modeled as the largest cost and transmission allowed only while hovering.Source: B. Energy Consumption and Harvesting… · p. 3
• During hovering, the IR harvests RF energy from the unreflected incident beamforming signal and converts it through a rectifier.Source: B. Energy Consumption and Harvesting… · p. 3
• Because harvested power is much smaller than hovering and mobility costs, it is mainly used to satisfy reflector power pr and enable self-powering.Source: B. Energy Consumption and Harvesting… · p. 3
• The net UAV power model combines hovering cost, reflector cost minus harvested power, and mobility cost according to the UAV speed state.Source: B. Energy Consumption and Harvesting… · p. 3

C. Problem Formulation

• The UAV-IR tracks UE movement using deployment intervals ∆T formed from grouped channel coherence times ∆t.The CSI is treated as constant within each ∆t, while deployment is optimized every ∆T = k∆t.Source: C. Problem Formulation · p. 3
• Communication and mobility stages alternate: the UAV hovers while SNR exceeds threshold τ, then moves after blockage to rebuild a LOS link.Communication-stage length depends on the IR–UE link state, whereas mobility-stage duration depends on travel distance and UAV speed.Source: C. Problem Formulation · p. 3
• The objective is to maximize total downlink transmissions before onboard energy E is exhausted by jointly selecting UAV-IR location and reflection parameters.The service ends with residual energy ϵ, where ϵ ∈ [0, pm∆T).Source: C. Problem Formulation · p. 3
• Real-time IR–UE CSI is difficult to estimate because UE and UAV movement and human-body blockage continually change the channel.Exhaustively measuring CSI across candidate UAV locations would consume substantial power and time, motivating a learning-based deployment approach.Source: C. Problem Formulation · p. 3

III. OPTIMAL DEPLOYMENT OF UAV-IR

• Q-learning with neural networks enables the UAV-IR to learn its optimal location and efficiently measure CSI during service.The learned model uses measurements and feedback from communication stages rather than prior knowledge of the dynamic environment.Source: §III · pp. 3–4

A. Stationary UE and UAV-IR

• For stationary UE conditions and known CSI h, the optimal reflection coefficient is Θ∗ = arg maxΘ c(Θ).The coefficient maximizes downlink capacity once the IR–UE channel is measured.Source: A. Stationary UE and UAV-IR · p. 4
• The optimization is equivalent to maximizing |hΘr|^2, with equality when every reflected complex component has the same argument.This phase alignment yields Θ∗ with element phases θ∗_n = −Arg(h_nr_n).Source: A. Stationary UE and UAV-IR · p. 4
• Once CSI h is available, the optimal reflection coefficient Θ∗ is uniquely determined for the served UE.The deployment framework thereafter treats Θ as its optimal value whenever h is available.Source: A. Stationary UE and UAV-IR · p. 4

B. Mobile UE and UAV-IR

• The RL framework selects a new UAV-IR position when blockage changes the CSI and requires rebuilding the LOS channel.Each communication stage supplies UE location, CSI, optimized reflection, and capacity feedback for environment modeling.Source: B. Mobile UE and UAV-IR · p. 4
• The CSI is the RL environment, the UAV is the agent, its movement is the action, and downlink data amount is the reward.Because UE mobility changes the environment even without UAV action, the process is nonstationary.Source: B. Mobile UE and UAV-IR · p. 4
• A Gaussian movement model based on location history predicts the UE’s next position and supports an MDP for future deployment decisions.The model uses mean µ = y(t)+ve(t)∆T and variance Λ = Var(Y^t).Source: B. Mobile UE and UAV-IR · p. 4
• The policy chooses the location x that maximizes expected long-term downlink transmission from the current CSI, UE position, and movement prediction.The value function measures expected cumulative reward during the next communication stage after a mobility duration K.Source: B. Mobile UE and UAV-IR · p. 4
• The discount ratio γ reflects movement-prediction accuracy, reaching γ = 1 under perfect prediction when the value function equals the original objective.The resulting optimal policy identifies the next communication-stage deployment location.Source: B. Mobile UE and UAV-IR · p. 4
• Q-learning updates the value estimate from received rewards using learning rate β, while an LSTM-based neural network approximates the value function over the infinite state space.The method can converge to a local optimum when service time is sufficiently long and the UE movement pattern is properly modeled.Source: B. Mobile UE and UAV-IR · p. 5
• At each stage, the algorithm measures CSI, optimizes reflection, keeps hovering when η(t) ≥ τ, or moves toward the learned best location after blockage.The neural value function is trained by minimizing the squared difference between the target q(t) and its estimate, while onboard energy is updated each slot.Source: B. Mobile UE and UAV-IR · p. 5

IV. SIMULATION RESULTS AND ANALYSIS

• The simulations use a 3D urban environment with two streets, four buildings, randomly distributed trees, persistent BS-UE blockage, and a moving UE modeled by an MDP.The setup includes a 30 GHz carrier, 40 dBm BS transmit power, and comparisons against static and non-learning UAV-IR baselines.Source: §IV · p. 5, Figure 2 · p. 5
• UAV-IRs significantly improve average data rate over a static IR at 20 m, but their performance declines with height and becomes similar to static deployment above 100 m.Increasing height raises path loss across the BS-IR-UE channels despite making LOS links more likely.Source: §IV · p. 5, Figure 3 · p. 5
• Over 90% LOS probability is achieved by the RL-based UAV-IR, compared with above 70% without RL and below 5% for a static IR.The learning-based advantage is attributed to predicting UE movement and estimating locations over a longer term.Source: Figure 3 · p. 5, §IV · p. 6
• As BS transmit power rises from 1 to 20 W at 40 m, UAV-IR data rates increase faster than the static IR, with RL outperforming non-learning deployment because of its higher LOS probability.At low power, all cases can have zero data rate when downlink SNR falls below threshold τ.Source: §IV · p. 6, Figure 4 · p. 6
• RF energy harvested by the UAV-IR increases with BS transmit power and reaches 1 mW, which is adequate to self-power the IR.The comparison uses a fixed UAV height of 40 m.Source: §IV · p. 6, Figure 4 · p. 6

V. CONCLUSION

• RL-based UAV-IR deployment optimizes reflector location and reflection coefficient through Q-learning and neural networks to maximize downlink capacity while serving a mobile outdoor UE.The approach models the propagation environment to maintain a downlink LOS channel.Source: §V · p. 6
• UAV-IRs outperform static IRs in average data rate and achievable downlink LOS probability, while RL-based deployment further improves performance over UAV-IR without learning.The conclusion concerns downlink mmW transmission with energy harvesting in the IR.Source: §V · p. 6
• RF energy harvested from mmW beamforming signals is sufficient to power the IR for signal reflections.Source: §V · p. 6