World Model for Energy Efficiency Maximization in Dynamic Metasurface Antennas

• DMAs are presented as a lower-hardware-cost and lower-power way to implement large antenna arrays for next-generation wireless networks.Source: §I · p. 1
• Optimization-based methods depend on accurate network models and repeated non-convex solutions, while heuristics can lack scalability and robustness in dynamic networks.Source: §I · p. 1
• Existing RL approaches face data inefficiency and limited long-term planning, leaving joint beamforming and DMA-weight optimization under changing channels largely unexplored.Source: §I · pp. 1–2
• The framework targets time-averaged EE under changing channels, subject to transmit-power, Lorentzian-phase, and minimum-rate constraints.Source: §I · p. 2
• Feasibility-preserving action mappings handle power and Lorentzian-phase constraints, while safety costs address minimum-rate requirements during policy learning.Source: §I · p. 2
• The proposed WMBRL learns system dynamics and uses imagined trajectories to jointly optimize digital beamforming and DMA weights with fewer environment interactions.Source: §I · p. 2
• Approximately 20% higher time-averaged EE than the DQN baseline is reported for the proposed framework, alongside higher sample efficiency and more reliable constraint satisfaction.Source: §I · p. 2

II. SYSTEM MODEL AND PROBLEM FORMULATION

• The modeled base station serves K single-antenna users with N = N_d × N_e elements arranged over N_d microstrips, and transmission spans T time slots.Source: §II · p. 2

A. Time-Varying Channel Model

• The channel model combines a free-space gain with small-scale fading represented by a first-order complex Gauss–Markov process following Jakes fading.The fading model includes an initial vector, an independent innovation process, and temporal correlation coefficient ρ.Source: A. Time-Varying Channel Model · p. 2

B. DMA Architecture and Signal Model

• Each DMA element has a frequency response, while propagation along each microstrip contributes position-dependent attenuation and phase shift to the transmitted signal.The model assumes a common attenuation coefficient and propagation wavenumber across microstrips.Source: B. DMA Architecture and Signal Model · p. 2
• The microstrip input is formed from digital precoding vectors and normalized, mutually uncorrelated user symbols.Source: B. DMA Architecture and Signal Model · p. 2
• The received signal includes additive white Gaussian noise, and each user's achievable rate is expressed using its signal-to-interference-plus-noise ratio.Source: B. DMA Architecture and Signal Model · p. 3
• System energy efficiency is defined using radiated power and fixed circuit power, with the DMA's total radiated power represented separately.Source: B. DMA Architecture and Signal Model · p. 3

C. Problem Formulation

• The objective is to maximize time-averaged EE across T slots by jointly optimizing digital beamforming vectors and the DMA weight matrix.Source: C. Problem Formulation · p. 3
• The constraints bound total radiated and circuit power, enforce the DMA elements' Lorentzian phase response, and require every user's rate to meet its minimum at each slot.Source: C. Problem Formulation · p. 3

III. PROPOSED SOLUTION

• Time-varying channels and coupling between digital beamforming and DMA weights make the optimization problem non-convex and stochastic.Source: §III · p. 3
• WMBRL learns latent system dynamics and optimizes both beamforming and DMA weights through imagined trajectories, supporting long-term planning with fewer environment interactions.Source: §III · p. 3
• Feasibility-preserving action mappings and safety-cost signals handle operational constraints during policy learning.Source: §III · p. 3

A. Observation, Action, and Reward Formulation

• The agent interacts with the wireless environment through observation, action, and reward representations.These representations are introduced to apply the WMBRL framework to problem P1.Source: A. Observation, Action, and Reward… · p. 3

1) Observation Space:

• The observation vector is presented as the agent’s view of the wireless environment at the beginning of each time slot.Its dimension is d_o = 2NK + 3K + 1.Source: 1) Observation Space · p. 3

A. Observation, Action, and Reward Formulation

• The section specifies that the observation is taken at the start of each time slot and the action is defined at each time slot.The supplied passages do not give the complete observation or action vectors.Source: 1) Observation Space · p. 3, 2) Action Space · p. 3

2) Action Space:

• The action vector includes controls for DMA element weights, per-user power allocation, and total radiated power.The action-space dimension is d_a = N + K + 1.Source: 2) Action Space · p. 3
• Each user’s digital beamforming vector combines its allocated transmit power with a unit-norm regularized zero-forcing direction.The direction is constructed from the channel and DMA-weighted channel matrix, with regularization parameter δ > 0.Source: 2) Action Space · p. 3
• The beamforming normalization ensures the DMA-weighted beamforming direction has unit norm, yielding radiated power equal to the sum of user allocations.The passage states ∥MQ[t]l_k[t]∥ = 1 and gives the resulting expression for P_rad[t].Source: 2) Action Space · p. 3

3) Reward Function:

• The expected discounted cumulative reward over T slots is used to encourage long-term energy-efficiency improvement through joint beamforming and DMA-weight optimization.The policy is π and γ is the discount factor; the immediate reward is defined to aim at maximizing time-averaged EE.Source: 3) Reward Function · p. 3

B. Constraint Handling

• Transmit power and Lorentzian phase constraints use feasibility-preserving action mappings, while minimum-rate requirements use a safety-cost function.These constraint-handling approaches are selected based on the constraints’ characteristics.Source: B. Constraint Handling · p. 3

1) Transmit Power Constraint:

• Per-user power allocations are produced by a softmax mapping so their total does not exceed the radiated-power budget P_max − P_c.The passage states that the resulting transmit-power constraint is satisfied at each time slot.Source: 1) Transmit Power Constraint · p. 4

2) Lorentzian Phase Constraint:

• Normalized phase-control actions are mapped to phases from 0 to 2π, after which DMA weights are obtained from the Lorentzian response.The resulting weights are arranged into the DMA weight matrix Q[t].Source: 2) Lorentzian Phase Constraint · p. 4

3) Minimum Rate Requirements Constraint:

• The per-slot safety cost quantifies minimum-rate violations, and its discounted cumulative value is constrained by a predefined budget b.The formulation encourages maximizing time-averaged EE while reducing minimum-rate violations.Source: 3) Minimum Rate Requirements Constraint · p. 4
• The policy optimization combines expected reward maximization with a constraint on expected discounted safety cost.The safety-cost budget is predefined.Source: 3) Minimum Rate Requirements Constraint · p. 4

C. Learning Method

• The framework combines a world model, reward and cost critics, and a safe actor that optimizes expected cumulative reward while satisfying safety constraints.The world model predicts future states, rewards, and safety costs from learned latent representations.Source: C. Learning Method · p. 4

1) World Model Learning:

• An observation encoder and recurrent state-space model learn compact latent dynamics, with deterministic and stochastic states evolving over time.The transition prior predicts future latent states, enabling imagined trajectories without further environment interactions.Source: 1) World Model Learning · p. 4
• Prediction heads reconstruct observations and predict rewards and safety costs from latent states.The world-model prediction, transition, and representation losses train these predictions and latent distributions to support policy optimization through imagined trajectories.Source: 1) World Model Learning · p. 4

2) Critic Learning:

• Two critics estimate expected long-term energy-efficiency reward and safety costs from constraint violations using world-model imagined trajectories.The reward critic estimates cumulative reward; the cost critic estimates cumulative safety cost.Source: 2) Critic Learning · p. 4
• TD(λ) targets recursively combine predicted immediate rewards or costs with estimates of future returns.The reward return uses the imagination horizon and TD(λ) parameter.Source: 2) Critic Learning · p. 5
• Both critics are trained against their respective targets using Dreamer-style two-hot categorical representations in the symlog domain.The paper defines symlog(x) as sign(x) log(|x| + 1).Source: 2) Critic Learning · p. 5

3) Actor Learning:

• The actor evaluates imagined actions using reward and cost advantages derived from future return estimates from the reward and cost critics.The advantages represent the effects of selected actions on energy efficiency and constraint satisfaction, respectively.Source: 3) Actor Learning · p. 5
• A combined advantage weights the cost advantage by an effective Lagrange multiplier, which is updated from observed safety cost relative to a budget.The multiplier update uses a dual step size and is clipped to a specified allowable range.Source: 3) Actor Learning · p. 5
• The actor loss encourages long-term energy-efficiency maximization while penalizing minimum-rate violations and regularizing policy entropy.The entropy term is scaled by a regularization coefficient.Source: 3) Actor Learning · p. 5

IV. SIMULATION RESULTS

• Approximately 20% higher time-averaged EE than DQN across the considered user distances is the headline result.The comparison is shown as time-averaged EE versus maximum user distance in Fig. 1d.Source: Figure 1 · p. 6, §IV · p. 6, §V · p. 6
• WMBRL reaches the prescribed safety-cost budget in approximately 2 × 10^3 steps, versus approximately 1.4 × 10^4 steps for DQN, and fluctuates less.Fig. 1b plots violation rate versus training steps.Source: §IV · p. 5, Figure 1 · p. 6
• Across 50–200 m, WMBRL stays below the safety budget, while DQN exceeds it at 200 m.Fig. 1e plots violation rate versus maximum user distance.Source: Figure 1 · p. 6, §IV · p. 6
• World-model loss falls rapidly and stabilizes around 4.3; observation, reward, cost, and critic losses also settle or fluctuate moderately after early declines.Fig. 1c shows world-model and critic losses over training.Source: Figure 1 · p. 6, §IV · p. 6
• As maximum user distance rises from 50 to 200 m, both schemes’ time-averaged EE decreases with increased path loss and reduced channel gains.The distance sweep varies the maximum user distance from 50 to 200 m; it is set to 150 m for Figs. 1a–1c.Source: §IV · pp. 5–6

V. CONCLUSION

• The conclusion attributes sample-efficient long-term policy optimization to learned latent dynamics and imagined trajectories, with action mappings and safety-cost signals handling operational constraints.The conclusion reports approximately 20% higher time-averaged EE than DQN, faster convergence, and more reliable constraint satisfaction.Source: §V · p. 6