PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Secure Communications for a UAV Swarm with Movable Antennas in SAGIN: CKM-Enabled Multi-Agent Reinforcement Learning Framework

Jiayang Wan, Yafei Wang, Jiawei Zhuang, Wenjin Wang, Tony Q. S. Quek

arXiv:2608.27973v1eess.SP

TL;DR

Secure SAGIN communication for UAV swarms requires joint optimization of movable antennas, trajectories, and heterogeneous link selections under CSI, action-space, and system-constraint challenges. The paper proposes a CKM-assisted MAPPO framework with rigid-body MA shifts, collaborative rewards, and action masks; it improves SEE while reducing CSI-acquisition and online-inference overhead relative to considered baselines.

Abstract

from arXiv · show

Space-air-ground integrated networks (SAGINs) can provide ubiquitous and reliable connectivity for unmanned aerial vehicles (UAVs). However, air-to-ground links, which are typically dominated by line-of-sight (LoS) propagation, are vulnerable to passive eavesdropping due to the broadcast nature of wireless channels. To enhance physical-layer security, we investigate a SAGIN-enabled secure downlink communication system in which UAVs select service links among satellite, aerial, and terrestrial networks while adjusting the positions of the movable antenna (MA) array to fully exploit connectivity and spatial degrees of freedom for improved secrecy communication performance. Specifically, we maximize the secrecy energy efficiency (SEE) of a UAV swarm by jointly optimizing the MA positions, UAV trajectories, and link selections, subject to UAV mobility, MA movement, and link connectivity constraints. To reduce the real-time channel state information (CSI) acquisition overhead, we propose a channel knowledge map (CKM)-assisted multi-agent reinforcement learning framework. Specifically, the CKM is first constructed from sparse channel measurements via Kriging interpolation and is then leveraged together with satellite ephemeris information to enable efficient storage and retrieval of CSI. To reduce the action-space dimensionality and computational complexity, we model the MA array using rigid-body kinematics and adjust its position through global rigid-body translation, thereby constructing a low-dimensional hybrid action space for the joint optimization decisions. To align local decisions with system-wide performance under system constraints, we design an individual-team collaborative reward mechanism and introduce action masks to enforce constraints on UAV mobility, collision avoidance, MA regions, and connectivity capacity.

I. INTRODUCTION

SAGIN addresses coverage gaps for highly mobile UAVs, while secure communication requires jointly exploiting heterogeneous links, trajectories, and movable-antenna spatial degrees of freedom. The paper proposes CKM-assisted MARL with reduced-dimensional actions and constraint-aware coordination.

  • SAGIN integrates terrestrial, low-altitude, and satellite networks to provide pervasive connectivity for low-altitude airspace.Source: §I · p. 1
  • Highly mobile UAVs experience rapidly varying link conditions, while fixed-position arrays cannot fully exploit local spatial channel variations.Movable antennas reposition elements within confined regions to obtain more favorable channel conditions.Source: §I · p. 1
  • The paper jointly optimizes MA positions, UAV trajectories, and satellite, aerial, or terrestrial link selections to maximize swarm secrecy energy efficiency under system constraints.Source: B. Motivation and Main Contributions · p. 2
  • Sparse offline channel measurements are interpolated into a CKM and combined with satellite ephemerides to reduce online CSI-acquisition overhead.The framework enables UAVs to query CSI for their locations and candidate MA configurations during decision-making.Source: B. Motivation and Main Contributions · p. 2
  • Rigid-body MA modeling, action masking, and individual-team rewards reduce action-space complexity while coordinating local decisions with swarm performance.The unified hybrid action space jointly generates link, trajectory, and MA decisions while excluding infeasible actions.Source: B. Motivation and Main Contributions · pp. 2–3

B. Channel Model

The A2G channel model combines geometry-dependent LoS probability and path loss with a movable receive array whose channel response depends explicitly on antenna positions. Optimizing those positions can improve channel conditions.

1) MA-Enabled A2G Links:

  • Each UAV uses a rectangular-panel M-element MA array for A2G links, with element positions constrained to a two-dimensional movable region.Source: 1) MA-Enabled A2G Links · p. 3, B. Channel Model · p. 3
  • The model computes link distance, horizontal distance, and elevation angle from GBS or ABS and UAV coordinates.Source: 1) MA-Enabled A2G Links · p. 4
  • LoS probability is modeled as an environment-dependent function of elevation angle, with average path loss combining free-space loss and additional LoS/NLoS losses.Source: 1) MA-Enabled A2G Links · p. 4
  • The small-scale A2G MIMO channel uses a geometric model with L effective paths and transmit and receive direction parameters.Source: 1) MA-Enabled A2G Links · p. 4
  • MA positions continuously adjust steering-vector phase terms, providing additional spatial degrees of freedom and enabling channel-condition optimization.Source: 1) MA-Enabled A2G Links · p. 4

2) A2S Links:

The system models A2S communication through far-field single-path LoS channels and discretizes UAV operation into time slots. It also imposes fixed-altitude, trajectory, collision-avoidance, link-selection, capacity, and signal-model constraints.

2) A2S Links:

  • A2S links use a far-field single-path LoS channel between each LEO satellite and UAV, with distance determined by their three-dimensional positions.Source: 2) A2S Links · p. 4
  • The satellite uses a K_x × K_y UPA, while each UAV uses a K-element upper-side ULA for A2S reception.Source: 2) A2S Links · p. 4
  • Mission time is divided into N equal slots, producing N + 1 discrete time instants for modeling time-varying UAV mobility.Source: C. Signal Model · p. 4
  • UAV trajectories are restricted to fixed altitude and prescribed initial and final positions, with a minimum inter-UAV safety distance preventing collisions.Source: C. Signal Model · p. 4
  • Each UAV selects at most one serving base station per slot, while each base station serves no more than its capacity limit.Source: C. Signal Model · pp. 4–5

III. PROBLEM FORMULATION

The paper formulates SEE maximization for a UAV swarm by jointly optimizing UAV trajectories, MA positions, and heterogeneous-network link selections. The resulting problem combines non-convex mixed-integer decisions, CSI overhead, and strict operational constraints.

  • SEE is maximized by jointly optimizing UAV trajectories, MA positions, and satellite, aerial, and terrestrial link selections.Source: §III · p. 5
  • The optimization is a non-convex mixed-integer nonlinear programming problem whose global optimum is generally difficult to obtain directly.Source: §III · p. 5
  • Each UAV is assumed to fly at a prescribed constant horizontal speed, while propulsion power alone is retained in the energy model.MA actuation and communication energy are left for future work.Source: §III · p. 5
  • Instantaneous CSI required for secrecy-rate evaluation incurs substantial signaling overhead.Source: §III · p. 6
  • The framework addresses the formulation challenges through CKM-based CSI retrieval, rigid-body MA translation, and action masking for feasible decisions.These mechanisms target CSI acquisition, hybrid action-space complexity, and constraint enforcement.Source: §III · p. 6

IV. PROPOSED CKM-ASSISTED MARL FRAMEWORK

The proposed framework uses an offline-constructed CKM and satellite ephemerides to support CSI-aware training without online physical-environment interaction. It jointly handles link selection, UAV trajectories, and MA positioning through hybrid actions and feasibility masking.

  • The CKM-assisted MARL framework uses an offline-constructed CKM to support CSI-aware training.Satellite ephemerides characterize the training environment alongside the CKM.Source: §IV · p. 6
  • Agents acquire CSI through data-driven interactions without requiring online interactions with the physical environment during training.Source: §IV · p. 6
  • Link selection, trajectory planning, and MA positioning are formulated as a unified hybrid discrete action space.Action masking ensures feasibility under hard constraints.Source: §IV · p. 6

A. CKM Construction for MARL Algorithm

The CKM represents channel knowledge as a function of UAV location, propagation environment, and MA configuration. Sparse joint position–configuration measurements are interpolated with Kriging and combined with satellite geometry for CSI retrieval.

2) Observation Space:

The framework uses centralized training with decentralized execution and a low-dimensional hybrid action space for UAV heading, link selection, and rigid-body MA translation. Action masks restrict decisions to feasible trajectory, link, and MA choices.

Observation Space

  • During decentralized execution, each UAV makes decisions from its local observation, including information about neighboring UAVs within communication range.Source: 2) Observation Space · p. 7

Low-Dimensional Action-Masked Hybrid Action Space

Algorithm

4) Individual-Team Collaborative Reward Function:

The framework uses an individual-team collaborative reward to balance each UAV’s SEE with swarm-level cooperation, while masked hybrid policies and centralized training support feasible decentralized decisions.

Individual-Team Collaborative Reward Function

  • The individual-team reward combines each UAV’s local performance with a team reward to balance individual optimization and system-level cooperation.The SEE is the core reward component guiding UAV trajectory optimization during flight.Source: 4) Individual-Team Collaborative… · p. 8

MAPPO Training and Execution

  • MAPPO is trained under centralized training with decentralized execution, using a global-state critic and locally observed actor decisions.Homogeneous UAVs share actor parameters, improving scalability and preserving permutation invariance.Source: 5) MAPPO Algorithm · p. 8

Hybrid Action Policy

  • The hybrid policy factorizes decisions into conditionally independent trajectory, link-selection, and MA-position sub-actions.The joint log-probability and entropy are computed as sums over the three policy heads.Source: 5) MAPPO Algorithm · p. 8

Constraint Enforcement

  • Action masking assigns infeasible actions a logit of −∞ before softmax, while a joint resolver handles collision and residual-capacity conflicts.The same masked-distribution structure is used for trajectory, link, and MA action heads.Source: 5) MAPPO Algorithm · p. 8

Optimization

  • PPO stabilizes learning through clipped policy and value objectives, normalized advantages, and entropy regularization for exploration.The value-loss and entropy terms are weighted by c1 and c2, respectively.Source: 5) MAPPO Algorithm · pp. 8–9

V. NUMERICAL RESULTS

The numerical-results section evaluates the proposed method in a 20-UAV SAGIN simulation and specifies the network, training, and optimization settings used for comparison.

Evaluation Design

  • The experiments assess secrecy energy efficiency as the primary metric and compare proposed algorithms with learning and convex-optimization-based alternating-optimization baselines.The benchmark methods are adapted to the joint optimization problem considered in this work.Source: §V · p. 9, Table III · p. 9, 2) Neural Network and Training… · p. 9

Simulation Setup

  • The evaluation uses a 20-UAV swarm flying at fixed altitude through a 500 m×500 m area toward designated destinations.The heterogeneous network contains one GBS, one ABS, and two LEO satellites, with full frequency reuse and 1 MHz per UAV serving link.Source: 1) Communication Parameter Settings · p. 9

Training Configuration

  • The proposed MAPPO-RBHR-CKM configuration trains shared actors with separate flight-direction, link-selection, and MA-position heads.The actor uses two 128-neuron hidden layers, the critic uses three 256-neuron layers, and training lasts 10,000 episodes of 40 slots.Source: 2) Neural Network and Training… · p. 9

B. Benchmarks

The benchmark suite compares the proposed MAPPO-RBHR framework with alternating-optimization variants, independent and deterministic multi-agent learning baselines, and fixed-design schemes.

Alternating-Optimization Baselines

  • The AO-Joint-CSI baseline jointly updates UAV trajectories, link selections, and MA positions.Other AO variants isolate fixed antennas, fixed ground-network links, or predefined straight-line trajectories.Source: B. Benchmarks · p. 9

Learning Baselines

  • IPPO-CSI and MADDPG-CSI use perfect instantaneous CSI to optimize the joint decisions through independent or deterministic multi-agent learning.IPPO makes independent UAV decisions, whereas MADDPG jointly optimizes the swarm decisions.Source: B. Benchmarks · p. 9

Proposed Methods

  • The proposed MAPPO-RBHR framework is evaluated with either CKM information or perfect instantaneous CSI while jointly optimizing trajectories, links, and MA positions.This separates the framework’s CKM-assisted and perfect-CSI operating conditions.Source: B. Benchmarks · p. 9

Evaluation Metric

  • SEE is the primary comparison metric, with transmit beamforming and receive combining implemented using the MMSE criterion.The benchmark evaluation is conducted during algorithm training and across the proposed and baseline schemes.Source: B. Benchmarks · p. 10

C. Performance Comparison

The proposed method exhibits favorable training convergence and higher SEE than key baselines, while ablations and complexity analysis assess its reward, rigid-body, joint-optimization, and CKM-assisted designs.

Training Performance

  • Cumulative return and SEE improve rapidly during training and then stabilize, while critic loss remains low throughout.These trends indicate favorable convergence behavior for the proposed algorithm.Source: C. Performance Comparison · p. 10

Performance Comparison

  • MAPPO-RBHR-CSI achieves approximately 4.5% higher converged SEE than IPPO-CSI, 43% higher SEE than MADDPG-CSI, and 12% higher SEE than AO-Joint-CSI.MAPPO-RBHR-CKM outperforms most baseline schemes, while MAPPO-RBHR-CSI achieves the highest SEE during training.Source: C. Performance Comparison · p. 10

Hyperparameter Analysis

  • A learning rate of 1 × 10^-4 provides the highest stable final SEE, and 100 fast-fading realizations provide the best convergence and SEE.Increasing the number of realizations further raises computational cost while providing only marginal additional gains.Source: C. Performance Comparison · p. 11

Mechanism Ablation

  • The proposed algorithm achieves approximately 16.7% higher converged SEE than MAPPO-RB-CSI and 5.8% higher SEE than MAPPO-HR-CSI.Removing either the rigid-body shift mechanism or hybrid reward mechanism causes noticeable SEE degradation.Source: D. Ablation Experiment · p. 11

Scenario Ablation

  • Disabling trajectory optimization reduces SEE by approximately 13.8%, while variants removing MA or link optimization also achieve lower performance.The ablations support jointly optimizing UAV trajectories, link selection, and MA configuration.Source: D. Ablation Experiment · p. 11

Complexity and Deployment

  • Rigid-body shifts reduce the MA action-output dimension from MKgrid to Nsh, lowering actor computational load and online inference complexity.During deployment, the CKM-assisted scheme avoids real-time CSI acquisition, whereas AO methods require CSI and per-slot outer iterations.Source: 1) Computational Complexity · p. 12, Table IV · p. 12

2) CSI Acquisition Overhead:

The proposed MAPPO-RBHR-CKM eliminates real-time CSI acquisition by using offline CKM and satellite ephemeris data during training and deployment. This contrasts with CSI-based learning and optimization baselines that repeatedly acquire or evaluate instantaneous CSI.

  • No real-time CSI acquisition is required by MAPPO-RBHR-CKM during training or deployment.The method replaces real channel probing with offline CKM and ephemeris data.Source: 2) CSI Acquisition Overhead · p. 12
  • CSI-based MARL baselines acquire instantaneous CSI for all U agents at every training step, resulting in O(EepisodeTU) acquisitions.Source: 2) CSI Acquisition Overhead · p. 12
  • AO-Joint/FPA/GN-CSI methods require O(RUTndir) real-CSI evaluations across restarts, time slots, and candidate directions.Source: 2) CSI Acquisition Overhead · p. 12
  • AO-Straight-CSI has lower acquisition complexity, requiring O(UT) because it follows a fixed straight path.Source: 2) CSI Acquisition Overhead · p. 12
  • The resulting CKM-assisted policy improves SEE while reducing online inference and CSI-acquisition overhead relative to the considered baselines.Source: §VI · p. 12

Related papers

Loading 2608.27973v1…