Safe World Model for Joint Trajectory and Beamforming Design of Mobile Base Stations

I. INTRODUCTION

• MBSs offer flexible wireless coverage through adaptive mobility, but joint trajectory and beamforming decisions are complicated by changing channels and temporal dependencies.The introduction notes their potential use in areas with limited infrastructure or temporary communication needs.Source: §I · p. 1
• World-model-based reinforcement learning uses learned environmental dynamics and imagined trajectories to support sample-efficient policy optimization and capture long-term effects of trajectory and beamforming decisions.The paper identifies joint long-term planning and safety-aware learning for dynamic MBS networks as largely unexplored.Source: §I · p. 1
• The proposed SafeDreamer-based framework jointly optimizes trajectories and digital beamforming over imagined trajectories, incorporating constraints through feasibility-preserving action mappings and safety-cost signals.It learns network dynamics from experience to reduce the environment interactions required for policy learning.Source: §I · pp. 1–2

II. SYSTEM MODEL AND PROBLEM FORMULATION

• The optimization targets time-averaged sum SINR while accounting for time-varying channels, mobility, restricted-zone avoidance, and transmit-power constraints.The formulation also accounts for temporal dependencies in MBS trajectories.Source: §I · p. 1

I. INTRODUCTION

• The framework improves time-averaged sum SINR, sample efficiency, and constraint satisfaction compared with the considered baselines.The introduction identifies PPO and DQN as comparison methods.Source: §I · p. 2

II. SYSTEM MODEL AND PROBLEM FORMULATION

• The MBS must stay outside a circular restricted zone and reach its prescribed final position while obeying its movement constraints.The initial and final positions must permit reaching the destination within the mission duration while avoiding the restricted zone.Source: §II · p. 2

A. Time-Varying Channel Model

• The small-scale fading follows a first-order complex Gauss-Markov process to capture temporal correlation across consecutive time slots.The passage gives the correlation coefficient as ρ, with 0 ≤ ρ < 1.Source: A. Time-Varying Channel Model · p. 2
• The channel vector combines distance-dependent gain, with μ denoting reference-distance channel power gain and D_k[t] the MBS-to-user distance.The vector also includes the corresponding small-scale fading vector.Source: A. Time-Varying Channel Model · p. 2
• The received signal at each user is formulated using its desired transmission, other users’ transmissions, and noise.The digital precoding vector for user k is w_k[t], and the noise is modeled as additive white Gaussian noise.Source: A. Time-Varying Channel Model · p. 2

B. Problem Formulation

• The objective jointly optimizes the MBS trajectory and digital beamforming vectors to maximize time-averaged sum SINR over T_s time slots.The optimization is under time-varying channel conditions and includes initial and final position constraints.Source: B. Problem Formulation · p. 2
• The transmit-power constraint limits the MBS power by P_max during time slots 1 through T_s.P_max denotes the maximum transmit power at the MBS.Source: B. Problem Formulation · p. 2
• The problem is a non-convex stochastic optimization because channels vary over time, decision variables are coupled, and trajectory decisions have temporal dependencies.Mobility, restricted-zone, and transmit-power constraints add further optimization challenges.Source: §III · p. 2
• The proposed SafeDreamer-based world-model framework learns latent network dynamics from experience and jointly optimizes trajectory and beamforming through imagined trajectories.The approach is intended to support long-term planning while reducing extensive environment interactions.Source: §III · p. 2
• Feasibility-preserving action mappings and safety-cost signals incorporate constraints into policy learning.These mechanisms are presented as part of the proposed framework for constrained optimization.Source: §III · p. 2

A. Observation, Action, and Reward Formulation

• The observation vector is defined at the beginning of each time slot.The passage introduces the observation-space formulation.Source: 1) Observation Space · p. 2
• The observation includes normalized effective channel power between the MBS and each user.This quantity is denoted by G_k[t].Source: 1) Observation Space · p. 3
• Final-position information characterizes the relative destination position and whether it remains reachable within the remaining mission duration.The remaining travel-distance bound is B[t] = (T_s − t)V_maxδ.Source: 1) Observation Space · p. 3
• The maximum distance the MBS can travel in the remaining time slots is represented by B[t] = (T_s − t)V_maxδ.This quantity is used in the final-position information.Source: 1) Observation Space · p. 3
• Restricted-zone information is included in the observation formulation.The passage denotes this component as o_RZ[t].Source: 1) Observation Space · p. 3
• The observation-space dimension is d_o = 9 + 5K.Source: 1) Observation Space · p. 3

2) Action Space:

• The action vector combines horizontal MBS displacement with per-user transmit powers, and digital beamforming uses RZF directions scaled by allocated power.The action-space dimension is K + 2.Source: 2) Action Space · p. 3

3) Reward Function:

• The discounted cumulative sum-SINR reward accounts for sequential effects of trajectory and digital-beamforming decisions across the mission horizon.Source: 3) Reward Function · p. 3

B. Constraint Handling

• Initial and final positions are enforced by starting at the predefined location and projecting candidate positions to preserve destination reachability while avoiding the restricted zone.Source: B. Constraint Handling · p. 3

1) Action Level Constraints:

• Movement actions exceeding unit norm are normalized and scaled by Vmaxδ, while power actions are transformed into allocations satisfying positivity and the total transmit-power constraint.Source: 1) Action Level Constraints · p. 3

2) Feasibility-Preserving Trajectory Projection:

• Candidate positions that cannot preserve destination reachability within the remaining slots while avoiding the restricted zone are projected to feasible positions.Source: 2) Feasibility-Preserving Trajectory… · p. 3

3) Safety-Cost Learning:

• The operating-region cost is zero inside the allowable region, and the reachability cost is zero when the destination remains reachable within the remaining mission duration.Source: 3) Safety-Cost Learning · p. 3
• Restricted-zone cost is zero outside the zone, while final-position error cost measures deviation from the prescribed destination at mission end.Source: 3) Safety-Cost Learning · p. 3
• The four safety-cost components are summed into total cost, and policy optimization maximizes long-term sum SINR subject to an expected safety-cost budget.Source: 3) Safety-Cost Learning · p. 3

• A world model predicts future states, rewards, and safety costs from compact latent representations; critics estimate long-term performance and cost, and a safe actor selects constrained actions.Source: C. Learning Method · p. 3

• The observation encoder and recurrent state-space model form compact latent representations and model temporal dynamics, enabling imagined trajectories without direct environment interactions.Source: 1) World Model Learning · p. 4
• The world model uses prediction heads to reconstruct observations and predict rewards, safety costs, and continuation signals from latent states.Source: 1) World Model Learning · p. 4
• Prediction, transition, and representation losses train the model, with weighting coefficients βpred = 1, βtran = 0.5, and βrep = 0.1.The prediction loss jointly trains observation, reward, safety-cost, and continuation predictors.Source: 1) World Model Learning · p. 4

2) Critic Learning:

• TD(λ) returns combine predicted immediate reward or cost with corresponding future value estimates.The paper defines reward and cost TD(λ) returns for training the critics.Source: 2) Critic Learning · p. 4
• The critics use Dreamer-style two-hot categorical targets in the symlog domain.The symlog transform is sign(x) log(|x| + 1).Source: 2) Critic Learning · p. 4
• Reward and cost critics estimate long-term communication performance and cumulative safety costs, respectively.The reward critic uses predicted reward and the cost critic uses predicted safety cost.Source: 2) Critic Learning · p. 4

3) Safe Actor Learning:

• The actor optimizes trajectory and beamforming decisions over imagined trajectories, using predicted rewards and costs to calculate TD(λ) returns and reward and cost advantages.Each rollout starts from a replay-buffer latent state; the world model predicts the next latent state, reward, and safety cost.Source: 3) Safe Actor Learning · p. 4
• Reward and cost advantages are combined using a non-negative Lagrange multiplier, and imagined time slots are weighted by cumulative predicted continuation.The actor loss also includes entropy regularization.Source: 3) Safe Actor Learning · p. 4
• A PID mechanism adjusts the Lagrange multiplier from safety-cost error, accumulated error, and increases in safety cost between consecutive iterations.The derivative term is ζ = max{0, Jc−Jc,prev}; the update clips the multiplier to its maximum allowable value.Source: 3) Safe Actor Learning · pp. 4–5, Figure 1 · p. 5

IV. SIMULATION RESULTS

• The simulation uses a 10-antenna MBS serving two users over 100 half-second slots, with a 120 m-radius restricted zone in a 500 × 500 m2 region.The MBS travels from (40, 40) m to (460, 460) m at a maximum speed of 25 m/s and fixed height of 100 m.Source: §IV · p. 5
• WMBRL achieves higher time-averaged sum SINR and faster convergence than PPO and DQN while rapidly reducing constraint cost toward its prescribed budget.DQN requires more interactions to converge, while PPO converges more slowly to a lower SINR; DQN cost fluctuates more and PPO reduces cost more slowly.Source: §IV · p. 5
• The total world-model loss decreases rapidly and stabilizes, while reward- and cost-critic losses generally decrease, indicating improved long-term return estimation.Decoder, reward, cost, and continuation prediction losses show similar trends on the training data.Source: §IV · p. 5
• 7%, 9%, 5%, and 8% average time-averaged sum SINR improvements over PPO, DQN, MSM, and SL, respectively, with performance close to the upper bound.SINR increases with maximum transmit power for all considered schemes; the learned trajectory avoids the restricted zone and reaches the prescribed final location.Source: §IV · p. 5

V. CONCLUSION

• The conclusion reports improved time-averaged sum SINR, sample efficiency, and constraint satisfaction compared with the considered baselines.The framework jointly optimizes MBS trajectories and digital beamforming under mobility, restricted-zone, and transmit-power constraints.Source: §V · p. 5