Source-linked AI summary
Optimizing Age of Information Through Aerial Reconfigurable Intelligent Surfaces: A Deep Reinforcement Learning Approach
Moataz Samir, Mohamed Elhattab, Chadi Assi, Sanaa Sharafeddine, Ali Ghrayeb
TL;DR
The paper addresses fresh information delivery from capability-limited IoTDs through a UAV-mounted RIS when activation patterns are unknown. It formulates joint control of altitude, scheduling, and RIS phases as an expected-sum-AoI problem and solves it with PPO, which outperforms the evaluated baselines in AoI.
Problem
Unknown IoTD activation patterns make joint optimization of UAV altitude, communication scheduling, and RIS phases a stochastic mixed-integer non-convex problem for minimizing expected sum AoI.
Method
The paper uses a UAV-mounted RIS for passive relaying and applies proximal policy optimization to learn control without prior activation-pattern knowledge.
Results
The proposed PPO algorithm outperforms random-walk and heuristic-greedy baselines in minimizing AoI.
Takeaways & Limitations
Jointly learning IoTD activation patterns, UAV altitude, and communication scheduling is important for reducing average sum AoI.
Abstract
from arXiv · showhide
We investigate the benefits of integrating unmanned aerial vehicles (UAVs) with reconfigurable intelligent surface (RIS) elements to passively relay information sampled by Internet of Things devices (IoTDs) to the base station (BS). In order to maintain the freshness of relayed information, an optimization problem with the objective of minimizing the expected sum Age-of-Information (AoI) is formulated to optimize the altitude of the UAV, the communication schedule, and phases-shift of RIS elements. In the absence of prior knowledge of the activation pattern of the IoTDs, proximal policy optimization algorithm is developed to solve this mixed-integer non-convex optimization problem. Numerical results show that our proposed algorithm outperforms all others in terms of AoI.
I. INTRODUCTION
The paper addresses the challenge of maintaining fresh and reliable IoT status updates when devices have limited communication capabilities. It proposes an aerial RIS relay and PPO-based control to optimize freshness under unknown device activation patterns.
- Smart-city applications depend on fresh sensory status updates for reliable monitoring, control, and transportation decisions.Outdated information may be inconsistent with the monitored physical process and lead to erroneous decisions.
- IoTDs have limited capabilities and may struggle to communicate reliably over long distances, making timely service difficult.Delivery latency includes data-generation, transmission-scheduling, and intermediate-processing delays.
- RIS elements enhance desired signals and mitigate interference, while UAVs improve network connectivity and coverage.The paper combines these technologies to support passive relaying for IoT communications.
- ARIS relaying can reduce intermediate delay relative to active UAV relays because RIS-based forwarding requires one time slot instead of two.
- Prior UAV-RIS studies optimized utilities such as spectral efficiency, energy efficiency, fairness, coverage, or power consumption rather than information freshness.The paper identifies this gap as motivation for optimizing RIS configuration from an AoI perspective.
- The proposed framework uses PPO to control UAV altitude, RIS phase shifts, and communication scheduling without prior knowledge of IoTD activation patterns.
II. SYSTEM MODEL AND PROBLEM FORMULATION
The system models IoTD status updates relayed through a UAV-mounted RIS to a BS over time slots. The formulation minimizes expected sum AoI by jointly controlling altitude, scheduling, and RIS phases under channel, reliability, and motion constraints.
- A UAV equipped with F RIS reflecting elements passively relays status updates from capability-limited IoTDs to the BS.The network is organized over multiple frames divided into equal time slots.
- The UAV altitude varies within aviation-authority bounds, starts at the BS height, and is constrained by its maximum per-slot vertical movement.
- The RIS phase-shift matrix is controlled per slot, while TDMA schedules at most one IoTD for transmission.The scheduling variable α_i[n] indicates whether IoTD i is selected in a slot.
- Successful delivery requires the IoTD-BS SNR to meet the decoding threshold, after which the corresponding AoI decreases; otherwise, AoI increases.The queue replaces an older status packet with a newly arrived sample.
- The optimization minimizes expected sum AoI over altitude decisions, scheduling variables, and RIS phase shifts.The objective uses ESA as the freshness metric and imposes the AoI and transmission constraints.
- Unknown stochastic IoTD activation patterns make the problem mixed-integer and non-convex, motivating an MDP reformulation and model-free PPO.The proposed PPO policy does not require prior activation-pattern knowledge.
III. PROPOSED SOLUTION
The proposed solution addresses the coupled hybrid discrete-continuous control problem involving UAV altitude, IoTD scheduling, and RIS phase configuration. The action-space design is challenging because jointly discretizing these controls creates many possible actions.
- III. PROPOSED SOLUTION: The control policy jointly governs UAV altitude, scheduling, and RIS phase shifts in a hybrid discrete-continuous action space.
- III. PROPOSED SOLUTION: Discretizing altitude and RIS phases would still create a large combined action space that is difficult for DRL algorithms to learn efficiently.For M IoTDs, K altitude actions, and Q phase choices, the stated action count is K×M×F×Q.
- III. PROPOSED SOLUTION: Because only one IoTD is scheduled per slot, the RIS can be configured for constructive signal combination at that selected IoTD.
A. MDP Formulation
The problem is modeled as an MDP in which a PPO agent jointly controls UAV altitude and IoTD scheduling under unknown activation patterns, using AoI as the negative reward.
- The MDP state comprises IoTD AoI, SNR observations, and the UAV altitude at each time-slot.
- Each action combines binary IoTD scheduling decisions with upward, downward, or hovering altitude adjustments.
- The agent receives a negative sum-AoI reward, motivating policies that reduce information age through scheduling and altitude control.
- Unknown IoTD activation patterns prevent dynamic programming with predefined transition probabilities, motivating a PPO-based deep reinforcement learning solution.
B. Proposed Solution Description
The proposed solution uses PPO to learn UAV control from channel and environment observations while adapting scheduling and altitude under unknown IoTD activation patterns. Its convergence is assessed computationally, with analytical convergence difficult to establish.
- The algorithm obtains channel information and RIS phases for coherent signal combination before observing AoI, SNR, and altitude states.
- PPO executes sampled policies for altitude and IoTD scheduling, penalizes altitude-constraint violations, and stores trajectories for training.
- The clipped PPO objective uses an advantage estimate to reduce gradient variance, then updates the policy through stochastic gradient ascent with ADAM.
- The algorithm’s multiplication complexity is expressed in terms of neural units across hidden layers.
- Convergence is limited to simulations because analytical convergence is difficult to analyze.
IV. SIMULATION AND NUMERICAL ANALYSIS
The simulations evaluate PPO against random-walk and hovering-greedy baselines across convergence, IoTD population, average AoI, and communication-resource settings. PPO learns IoTD activation patterns while adapting UAV altitude and scheduling, achieving lower expected sum AoI than the baselines.
- Convergence: PPO converges quickly to a steady point while learning IoTD activation patterns and adapting UAV altitude and scheduling to minimize ESA.The improvement diminishes once the agent is well trained about activation patterns.
- IoTD population: PPO minimizes ESA more effectively than both random-walk and hovering-greedy policies as the number of IoTDs varies.ESA increases with more IoTDs because more scheduling is needed, while hovering-greedy outperforms random walk.
- Average AoI: PPO achieves lower average sum AoI per IoTD than the baseline policies.The reported average-age gap is relatively high, highlighting the role of learning activation patterns and jointly adapting altitude and scheduling.
- Average AoI: Hovering-greedy can reduce AoI for some IoTDs but raise it to the maximum for others because it schedules only IoTDs satisfying the SNR threshold.This uneven behavior contrasts with PPO’s reported robustness for minimizing average AoI.
- Communication resources: Increasing IoTD transmit power or the number of RIS reflecting elements improves SNR and affects ESA.The simulations identify increasing RIS elements as an alternative when higher transmit power is not allowable.
V. CONCLUSION
The paper proposes a UAV-RIS passive relaying system that jointly optimizes UAV altitude, transmission scheduling, and RIS phase shifts to minimize expected sum AoI. PPO addresses the resulting mixed-integer non-convex problem, and numerical results show lower AoI than the evaluated baselines.
- Conclusion: The proposed system integrates UAVs and RIS elements for passive relaying while optimizing altitude, transmission scheduling, and RIS phase shifts.Its objective is minimizing expected sum Age-of-Information.
- Conclusion: PPO is proposed to solve the mixed-integer non-convex optimization problem.The optimization concerns maintaining information freshness in the relaying system.
- Conclusion: Numerical results show that PPO significantly minimizes AoI compared with random-walk and heuristic-greedy baselines.The conclusion reports the comparison at the algorithmic level rather than giving a specific numerical value.