Source-linked AI summary
Heterogeneous Services in Multi-UAV Networks with No-Fly Zone and User Mobility
Bin Li, Senning Wan, Decheng Zhang, Yonglei Yao, Yu Xue
TL;DR
The paper addresses how UAVs should adapt communication when mobile users enter no-fly zones and have heterogeneous service needs. It jointly optimizes UAV trajectories, transmission power, and user associations using SAC-PER; simulations report higher efficiency and priority rates than benchmarks across the considered environments.
Problem
NFZ-aware trajectory studies largely overlook how UAVs should adapt communication when users enter restricted areas, despite user mobility and heterogeneous service demands.
Source: §I · p. 1, B. Motivations and Contributions · p. 2Method
The paper jointly optimizes UAV trajectories, transmission power allocation, and binary UAV–user associations to maximize priority-weighted rates, using SAC-PER.
Source: B. Motivations and Contributions · p. 2, §IV · p. 5Results
Across considered environments, the scheme achieves higher efficiency and priority rates than benchmarks while UAVs approach users inside NFZs and avoid NFZ boundaries.
Source: B. Motivations and Contributions · p. 2Takeaways & Limitations
The simulations support adaptive connectivity for users inside NFZs under the proposed trajectory constraints and heterogeneous priority scheme.
Source: B. Motivations and Contributions · p. 2The model assumes fixed UAV altitude and perfect channel state information; altitude dynamics and CSI uncertainty are described as possible extensions.
Source: E. Limitations and Extensions · p. 5
Abstract
from arXiv · showhide
In this paper, we investigate a wireless communication network supported by multiple uncrewed aerial vehicles (UAVs). To ensure reliable connectivity, UAVs are desired to approach the boundaries of designated no-fly zones (NFZs) without violating them when mobile users (MUs) enter NFZs. To support heterogeneous communication services, MUs are assigned different priority levels based on their individual communication types. Considering the practical constraints arising from user mobility, heterogeneous communication requirements, and NFZs, the objective is to maximize the weighted sum of MU communication rates, where weights correspond to MU priorities. This is achieved by jointly optimizing trajectory planning, UAV transmission power control, and the binary association factor between UAVs and MUs. To address this challenging optimization problem under dynamic constraints, we develop a soft actor-critic algorithm with prioritized experience replay, which learns an adaptive control policy. Numerical results show that the proposed algorithm outperforms benchmark schemes in terms of convergence speed and stability. Moreover, it effectively meets diverse communication requirements, satisfies UAV energy constraints, and ensures reliable operation under NFZ constraints.
I. INTRODUCTION
The paper addresses joint adaptation to mobile users, heterogeneous service demands, and no-fly zones, where users may enter restricted areas while UAVs must remain outside. It combines joint resource optimization with an SAC-PER control approach and reports better efficiency and priority rates than benchmark schemes in the considered environments.
I. INTRODUCTION
- UAV-based communication is presented as a flexible response to limits in terrestrial deployment, including emergency and unexpected scenarios –.The introduction cites applications including autonomous driving, target localization, disaster relief, and public emergencies.Source: §I · p. 1
1) UAV Communications:
- Prior UAV communication work explores trajectory optimization with convex methods and deep reinforcement learning, including multi-UAV control and throughput maximization –.These studies address scenarios such as RIS-assisted interference mitigation, energy efficiency, and system throughput or coverage.Source: 1) UAV Communications · p. 1
2) NFZ Constraints:
- NFZ-aware trajectory studies [15]– largely overlook how UAVs should adapt communication when mobile users enter restricted areas.The paper identifies this as a gap between NFZ-aware flight design and adaptation to user mobility and service heterogeneity.Source: 2) NFZ Constraints · p. 2, B. Motivations and Contributions · p. 2
3) Heterogeneous Requirements:
- Prior studies addressed heterogeneous user requirements through per-user QoS-constrained sum-rate maximization, QoS-sensitive/tolerant groupings by data rate, and UAV deployment/selection for machine-type devices with heterogeneous data requirements.Source: 3) Heterogeneous Requirements · p. 2
B. Motivations and Contributions
- The problem jointly optimizes transmission power, UAV trajectories, and binary UAV–user associations to maximize priority-weighted rates under mobility and NFZ constraints.Mobile users receive priority levels according to their communication types, and cylindrical NFZs constrain trajectories.Source: B. Motivations and Contributions · p. 2
- The proposed solution formulates priority-rate maximization as an MDP and applies soft actor–critic with prioritized experience replay (SAC-PER).The method is motivated by channel dynamics, heterogeneous service requirements, and user mobility.Source: B. Motivations and Contributions · p. 2
- Across the considered environments, UAVs adaptively approach users inside NFZs while avoiding NFZ boundaries, and the scheme achieves higher efficiency and priority rates than benchmarks.The reported simulations evaluate the method under the proposed constraints.Source: B. Motivations and Contributions · p. 2
II. SYSTEM MODEL
The system model describes multiple aerial base stations serving mobile users within a bounded flight area containing cylindrical no-fly zones. It specifies UAV movement, safety, and energy constraints, including a fixed flight altitude and special trajectory requirements when a served user enters an NFZ.
A. Network Model
- The network has N UAVs serving K spatially distributed mobile users, with UAVs restricted to a rectangular flight region and prohibited from entering cylindrical NFZs.Each UAV carries an aerial base station; the service cycle is divided into equal time slots.Source: A. Network Model · pp. 2–3
B. UAV Mobility Model
- UAVs fly at fixed altitude H and must satisfy speed, acceleration, boundary, and inter-UAV safety constraints.NFZ constraints also require trajectory adjustment when a served user enters an NFZ, with a safety distance from its boundary.Source: B. UAV Mobility Model · p. 3
- To prevent discrete-time paths from crossing an NFZ between slots, the model checks spatial constraints at M micro-slots within each time slot.The propulsion-energy model accounts for rotor-blade and induced power, while communication-related energy is treated as negligible and ignored.Source: B. UAV Mobility Model · pp. 3–4
C. Communication Model
The communication model assigns mobile users priority weights by communication type and associates each user with one UAV. It models probabilistic LoS/NLoS channels and calculates rates under inter-UAV interference.
C. Communication Model
- Each user is assigned one communication type, whose priority level weights that user’s rate in the total priority-rate objective.The binary UAV–user association factor indicates the serving UAV, and each user is associated with exactly one UAV.Source: C. Communication Model · p. 4
- The UAV–user channel incorporates probabilistic line-of-sight and non-line-of-sight components to reflect obstacles such as trees and buildings.The channel model includes distance-dependent path loss and a Rician factor.Source: C. Communication Model · p. 4
- With OFDMA eliminating intra-UAV interference, the SINR model accounts for inter-UAV interference and supports calculation of each UAV–user transmission rate.The interference term reflects power allocated by other UAVs to their associated users.Source: C. Communication Model · p. 4
D. User Mobility Model
User movement is modeled with a recurrent self-similar Gauss-Markov process that combines historical states and Gaussian variation. Its speed update uses a persistence factor to control how strongly current speed carries over.
D. User Mobility Model
- The recurrent self-similar Gauss-Markov (RSSGM) model combines historical states with Gaussian variation to represent user trajectories across environments.Gaussian noise is also used to smooth trajectories toward typical human movement patterns.Source: D. User Mobility Model · p. 4
1) Speed Update:
- MU speed is updated intermittently over time to represent realistic movement behavior.The speed update rule is specified for each user and time slot.Source: 1) Speed Update · pp. 4–5
- The speed relevance factor η controls speed persistence: larger values produce smoother, more continuous movement, while η = 0 gives memoryless random-walk-like velocity.The update also includes a randomly sampled speed and zero-mean Gaussian noise; the noise contribution is normalized by 1 − η^2.Source: 1) Speed Update · p. 5
2) Direction Update:
The mobility model constrains directional changes and derives user positions from updated speed and direction; the optimization jointly chooses UAV acceleration, power, and association while observing operational assumptions.
2) Direction Update:
- MU direction follows a Markov process, with Gaussian perturbations clipped to [−∆θmax, ∆θmax] to avoid abrupt or physically unrealistic turns.The clipped variations are described as producing smoother trajectories aligned with human mobility patterns.Source: 2) Direction Update · p. 5
3) User Position Update:
- The RSSGM mobility framework updates MU position using speed and direction, capturing spatial continuity and temporal correlation.The direction vector is defined from the MU’s movement angle.Source: 3) User Position Update · p. 5
E. Limitations and Extensions
- The model assumes fixed UAV altitude and perfect CSI; accommodating altitude control and imperfect CSI would require added state dynamics and robust or risk-sensitive control.These simplifying assumptions are adopted for analytical tractability.Source: E. Limitations and Extensions · p. 5
III. PROBLEM FORMULATION
- The optimization jointly chooses UAV acceleration, transmission power, and binary UAV–MU associations to maximize the sum of priority rates under flight, NFZ, association, power, and energy constraints.The constraints include a specific trajectory restriction when a served MU enters an NFZ.Source: §III · p. 5
IV. PROBLEM SOLUTION
The paper casts the constrained joint-control problem as an MDP and uses SAC-PER to learn actions from UAV and user states, including energy and service-type information.
IV. PROBLEM SOLUTION
- The mixed-integer nonlinear program is NP-hard, combining continuous power and acceleration decisions with binary associations and nonlinear objective and constraints.The paper says convex reformulations carry computational overhead that degrades performance in UAV-assisted communication scenarios.Source: §IV · p. 5
- SAC-PER is selected for sample efficiency and exploration in high-dimensional continuous control, with replay-buffer reuse supporting adaptation to dynamic channels and time-varying states.The approach is presented as suitable for continuous decisions and discrete components through action-space design.Source: §IV · pp. 5–6
A. Modeling of MDP
- The MDP uses state, action, transition, reward, and discount components, with the design intended to maximize the sum of priority rates.At each time slot, the agent selects an action from the policy and receives an environment reward.Source: A. Modeling of MDP · p. 6
1) State:
- The observed state includes UAV and MU positions, UAV residual energy and speed, and each MU’s communication-type matching factor.These state variables are observed at the current time slot.Source: 1) State · p. 6
2) Action:
- Actions comprise UAV acceleration magnitude and direction, transmit power, and MU association, with the policy output converted to an association assigning each MU to one UAV per slot.The continuous association score is mapped to a binary association variable.Source: 2) Action · p. 6
3) Reward:
The reward aligns learning with priority-rate maximization while incorporating penalties and rewards for operational constraints and NFZ-related behavior.
3) Reward:
- The reward includes the instantaneous sum-rate of all MUs to align learning with the objective of maximizing priority rates.The reward is also defined to account for penalty scenarios.Source: 3) Reward · p. 6
- The reward design includes boundary and NFZ penalties, with the NFZ penalty defined in relation to UAV distance from an NFZ boundary.The text also introduces a collision penalty and an NFZ-approaching reward.Source: 3) Reward · p. 6
- The NFZ-approaching indicator is one when an MU is inside an NFZ and zero otherwise, and reward parameters must be tuned for convergence and desired performance.The indicator uses the MU’s position relative to the NFZ center and radius.Source: 3) Reward · p. 6
B. SAC-Based Algorithm
The SAC-based algorithm combines actor and critic networks, an exploration–exploitation temperature, and replay storage; its framework uses entropy regularization and twin Q-networks.
B. SAC-Based Algorithm
- SAC comprises an actor, critic networks, a temperature parameter balancing exploration and exploitation, and a replay buffer for storing and reusing experience.The training framework is illustrated in Fig. 3.Source: B. SAC-Based Algorithm · p. 6, Figure 3 · p. 7
1) SAC-based Framework:
- SAC optimizes an entropy-regularized return, with the temperature parameter regulating policy stochasticity to balance exploration and exploitation.The entropy term is described as discouraging premature exploitation and improving robustness to reward shaping and dynamics variations.Source: 1) SAC-based Framework · pp. 6–7
- Twin Q-learning uses the minimum of two Q-values in the target to mitigate overestimation risk, while the log-policy term acts as an entropy regularizer for exploration.The target also incorporates the discount factor for weighting future rewards.Source: 1) SAC-based Framework · p. 7
- The policy objective combines entropy regulation through log πϕ(at|st) with Q(st, at), the estimated expected return under the current policy.The replay buffer stores historical data for training.Source: 1) SAC-based Framework · p. 7
2) Temperature Parameter Adjustment:
SAC adjusts its temperature to balance exploration and exploitation, while prioritized experience replay (PER) samples important transitions more often. Integrating PER with SAC is reported to improve convergence efficiency, policy stability, and adaptability.
2) Temperature Parameter Adjustment:
- SAC’s automatic temperature adjustment balances reward maximization against policy entropy, with a relatively high temperature early in training that is gradually reduced.The adjustment minimizes an objective, and the temperature is updated by gradient descent.Source: 2) Temperature Parameter Adjustment · p. 7
3) Prioritized Experience Replay (PER):
- A larger temporal-difference error indicates greater potential impact on policy improvement, so those experiences are replayed more frequently to accelerate convergence.PER assigns priorities to sampled experiences and uses them to set sampling probabilities.Source: 3) Prioritized Experience Replay (PER) · p. 7
- PER assigns each experience a priority based on its absolute TD error plus a small positive constant, keeping every experience eligible for sampling.Importance sampling adjusts weights to mitigate bias from non-uniform sampling; its correction parameter is typically annealed from 0 to 1.Source: 3) Prioritized Experience Replay (PER) · p. 7
- PER integration is reported to improve convergence efficiency, policy stability, and adaptability in dynamic environments.The authors identify trajectory planning and power allocation as complex tasks that particularly benefit from the integration.Source: 3) Prioritized Experience Replay (PER) · p. 7
C. Complexity Analysis
SAC-PER’s training complexity mainly comes from actor and critic network gradient computation and parameter updates. The simulation comparison includes PPO, DDPG, fixed UAVs, and a no-priority scheme.
C. Complexity Analysis
- Actor and critic gradient computation and parameter updates are the main sources of SAC-PER computational complexity.The gradient cost depends on minibatch size and the respective network architectures, including their layer neuron counts.Source: C. Complexity Analysis · p. 8
V. SIMULATION RESULTS
- The simulation benchmarks SAC-PER against PPO and DDPG, as well as fixed-UAV and no-priority schemes.The fixed-UAV scheme keeps UAVs at predetermined optimal locations, while the no-priority scheme treats all MUs as one type.Source: §V · p. 8
A. Simulation Setup
In the reported simulations, SAC-PER converges more effectively than PPO and DDPG, while UAVs adapt their positions to mobile users and NFZ constraints. The evaluation uses randomly initialized positions and specified mobility, training, and network settings.
A. Simulation Setup
- Simulations randomly initialize UAV and MU positions, place UAVs at 100 m altitude in a [500, 500] m horizontal area, and model MU movement using RSSGM.Training uses 500 episodes, a replay buffer of 5 × 10^4 transitions, batch size 256, and actor and critic networks with two hidden layers of sizes [256, 128].Source: A. Simulation Setup · pp. 8–9
1) Convergence:
- SAC-PER begins converging around 100K steps and stabilizes, while PPO converges more slowly with a smaller final reward and DDPG does not effectively converge.SAC-PER is also reported to converge faster and achieve a slightly better final reward than SAC, with greater stability.Source: 1) Convergence · p. 9, Figure 4 · p. 9
2) Trajectories:
- With 15 MUs and 3 UAVs, UAVs reposition toward active-user clusters and maneuver along NFZ boundaries when users enter restricted zones.In the extended scenario, UAVs avoid NFZs while gravitating toward MU-dense regions, sustaining aggregate communication rate.Source: 2) Trajectories · pp. 9–10, Figure 5 · p. 10
3) Scalability:
Across the reported scenarios, the proposed scheme outperforms benchmarks, while sum-rate responds to user load, UAV deployment, altitude, NFZ constraints, and communication-type diversity. The conclusion reports improved sum-rate, convergence speed, and training stability versus baseline algorithms.
3) Scalability:
- As the MU population grows, sum-rate growth slows and eventually tends to saturate because transmit power is divided among more users, reducing per-MU SINR.With few MUs, each can receive a larger share of total transmit power, enhancing link quality.Source: 3) Scalability · p. 10
- Sum-rate increases with UAV count, and the proposed scheme consistently outperforms benchmarks across the evaluated scenarios.Denser UAV deployment reduces average UAV–MU distance and improves spatial diversity, link quality, and spectrum utilization.Source: 3) Scalability · p. 10, Figure 7 · p. 10
4) Sensitivity:
- At a UAV altitude of 60 m, the proposed scheme has a clear performance advantage over conventional methods.The passage attributes the improvement to reduced UAV–MU distance and enhanced channel quality; fixed-position UAVs are less adaptive to changing conditions.Source: 4) Sensitivity · p. 10, Figure 8 · p. 10
- Increasing NFZ density or radius consistently degrades sum-rate by shrinking feasible airspace and displacing UAVs from spatially optimal communication locations.Higher NFZ density also increases the probability of NLoS links.Source: 4) Sensitivity · p. 11, Figure 9 · p. 11
- Across varying communication-type counts, the proposed scheme achieves the highest total transmission rate, while unweighted sum-rate declines as priority classes grow.Favoring high-priority MUs shifts UAV proximity and transmit power away from locations that maximize average channel gain.Source: 4) Sensitivity · p. 11, Figure 10 · p. 11
VI. CONCLUSION
- The proposed method significantly enhances sum-rate and outperforms baseline algorithms in convergence speed and training stability.The conclusion also notes future work on joint 3D path planning, decentralized multi-agent DRL, and strict service-specific QoS constraints.Source: §VI · p. 11