Source-linked AI summary
Deep Reinforcement Learning for Energy Efficiency Maximization in RSMA-IRS-Assisted ISAC System
Zhangfeng Ma, Ruichen Zhang, Bo Ai, Zhuxian Lian, Linzhou Zeng, Dusit Niyato
TL;DR
The paper tackles EE optimization for RSMA-IRS-assisted ISAC under practical fading and QoS, beamforming, and IRS-phase constraints. It builds a 3D geometry-based channel model and solves the resulting problem with PPO-based DRL, achieving improved EE while meeting QoS requirements; EE decreases with higher carrier frequency and varies with fading and IRS configuration.
Problem
Limited prior work optimizes RSMA-assisted IRS-ISAC under practical channel fading, despite path loss, blockage, and resource-sharing challenges.
Method
The paper develops a 3D geometry-based channel model, formulates constrained EE maximization, and solves it by converting the problem into an MDP for PPO-based DRL.
Results
The proposed PPO scheme improves EE, satisfies communication and sensing QoS requirements, and outperforms SDMA-ISAC; EE decreases as carrier frequency increases and rises by approximately 50% when target RCS increases from 10 m2 to 20 m2.
Takeaways & Limitations
The results support RSMA-IRS-assisted ISAC with PPO as an effective approach for EE optimization under the modeled practical fading conditions.
Abstract
from arXiv · showhide
This paper proposes a three-dimensional (3D) geometry-based channel model to accurately represent intelligent reflecting surfaces (IRS)-enhanced integrated sensing and communication (ISAC) networks using rate-splitting multiple access (RSMA) in practical urban environments. Based on this model, we formulate an energy efficiency (EE) maximization problem that incorporates transceiver beamforming constraints, IRS phase adjustments, and quality-of-service (QoS) requirements to optimize communication and sensing functions. To solve this problem, we use the proximal policy optimization (PPO) algorithm within a deep reinforcement learning (DRL) framework. Our numerical results confirm the effectiveness of the proposed method in improving EE and satisfying QoS requirements. Additionally, we observe that system EE drops at higher frequencies, especially under double-Rayleigh fading.
I. INTRODUCTION
The paper addresses limited optimization of RSMA-assisted IRS-ISAC systems under practical channel fading. It proposes a geometry-based channel model and PPO-based DRL transmission design for energy-efficiency maximization with QoS and beamforming constraints.
- ISAC combines data transmission and environmental monitoring, but path loss, blockage, and increasing resource-sharing complexity hinder practical deployment.
- Only a limited number of studies optimize IRS-assisted ISAC systems using RSMA, and prior work mainly relies on ideal channel models.
- The paper develops a 3D geometry-based channel model whose parameters represent fading effects through distance, velocity, and angle information.
- It formulates EE maximization with QoS requirements, transceiver beamforming, and practical IRS phase shifts, then applies PPO-based DRL.
- Simulations report that the proposed algorithm outperforms traditional SDMA and examine carrier-frequency and target-RCS effects on EE.
A. System Model
The system uses a dual-function BS, an IRS, multiple users, and a moving target in a 3D urban geometry. RSMA communication streams and a radar stream are jointly modeled through cascaded channels, beamformers, and IRS phase shifts.
- A dual-function BS serves K single-antenna users and detects a target through IRS-assisted BS-IRS-user and BS-IRS-target links.
- The 3D geometry specifies antenna, user, target, and IRS-element positions together with array spacings, velocities, and movement angles.
- The channel model includes LoS and NLoS components, with carrier frequency, wavelength, target RCS, Rician factors, propagation distances, and Doppler terms.
- One-layer RSMA splits each user message into common and private parts, which are encoded into common and user-specific streams.
- The transmit signal combines common, private, and radar streams through corresponding beamformers, while the IRS applies a diagonal phase-shift matrix.
- Communication and sensing signals are represented by user received-signal and BS radar-echo models, and EE is defined from achievable rate and power consumption.
B. Problem Formulation
The problem maximizes system EE by jointly optimizing rate allocation, transmit and receive beamforming, and IRS reflection coefficients under communication, sensing, power, and phase-shift constraints. Its time-varying channels motivate a DRL solution for improved adaptation.
- The optimization jointly selects common-rate portions, transmit beamforming, IRS reflection coefficients, and the echo receiving beamformer to maximize EE.
- The constraints enforce common-stream decodability, nonnegative common-rate portions, total BS transmit power, user rate thresholds, sensing SNR, and discrete IRS phase shifts.
- Because the channel model is time-varying, conventional optimization may incur computational delays and inadequate real-time performance.
III. ALGORITHM DESIGN
The algorithm design converts the EE optimization into a Markov decision process and applies PPO to solve it.
- The proposed method first reformulates the optimization problem as a Markov decision process, then applies PPO to solve it.
A. MDP
The MDP defines actions for beamforming, common-rate allocation, and IRS phase control, while states combine channel information, actions, and rewards. Its reward maximizes EE while penalizing violations of communication, QoS, power, and sensing constraints.
- Action space: The action space contains common and private beamformers, an echo receiving beamformer, common-rate portions, and IRS phase shifts.Beamforming directions use MRT and ZF schemes, while power components and other outputs are produced through bounded neural-network activations.
- State space: The state space includes current channel information, the selected action vector, and the instantaneous reward.These inputs provide the agent with environmental information and feedback about its effectiveness in addressing the optimization problem.
- Reward function: The reward function combines an EE-related reward term with penalties for constraint violations.Penalty coefficients correspond to communication, QoS, transmit-power, and echo constraints.
- Reward function: The penalty terms enforce adherence to the communication, QoS, power, and echo constraints during optimization.The formulation uses separate coefficients for the corresponding constraints.
B. PPO
The PPO framework learns transmission actions from state transitions and rewards using a clipped surrogate objective. Algorithm 1 initializes the policy and experience pool, collects transitions, computes advantages, and updates the neural network with SGD.
- PPO objective: PPO optimizes a clipped surrogate objective based on the policy probability ratio and the advantage function.Clipping constrains policy updates within a trust-region range controlled by ε.
- Algorithm: Algorithm 1 outputs beamformers, common-rate portions, and IRS phase shifts from the corresponding channel inputs.The output action is A = {v_c, v_r, u, {C_k, v_k}, {ϕ_n}}.
- Algorithm: The algorithm initializes the action, neural-network parameters, and experience pool before interacting with the environment.At each step it takes an action, observes a reward and next state, and stores the transition.
- Optimization update: PPO computes the advantage function from rewards and state values, then updates the framework using SGD over collected transitions.The advantage uses the reward, discounted next-state value, and current-state value.
- Complexity: The algorithm’s time complexity is primarily influenced by the neural-network size.The network structure is characterized by its number of layers and neurons per layer.
IV. NUMERICAL RESULTS
The numerical results show that PPO converges while satisfying communication and sensing QoS requirements, and achieves higher EE than the evaluated baselines. EE depends strongly on carrier frequency, fading conditions, IRS size, and target RCS.
- Convergence: EE and reward stabilize after a finite number of iterations, while their gap decreases during training.The improved quality of samples in the experience pool is identified as the reason for the decreasing gap.
- Convergence: Achievable rate and radar SNR converge while meeting their corresponding QoS requirements.
- Baseline comparison: The PPO scheme achieves the highest performance against Random and Greedy baseline approaches.Random selects actions uniformly, whereas Greedy maximizes EE at each step without accounting for long-term outcomes.
- Channel conditions: For M = 8, system EE decreases by approximately 67% as fc increases from 1.4 GHz to 2.4 GHz.The decline is attributed to larger path loss at higher carrier frequency.
- Channel conditions: Double-Rician channels provide higher EE than double-Rayleigh channels, while increasing IRS elements improves EE in both RSMA and SDMA schemes.The comparison uses SDMA-IRS-assisted ISAC as the benchmark.
- IRS and target effects: System EE improves by approximately 50% as σ increases from 10 m2 to 20 m2.EE increases for both RSMA and SDMA schemes as target RCS grows.
V. CONCLUSIONS
The paper proposes a 3D geometry-based channel model for RSMA-IRS-assisted ISAC and develops PPO to maximize EE. Its conclusions emphasize the effects of fading, IRS scaling, and RSMA over SDMA.
- Contributions: The proposed 3D geometry-based channel model characterizes RSMA-IRS-assisted ISAC propagation environments.
- Contributions: The PPO algorithm is developed to maximize the system's EE.
- Conclusions: Fading conditions have a strong impact on EE, while increasing the number of IRS elements yields higher EE.
- Conclusions: The RSMA-ISAC system outperforms conventional SDMA-ISAC systems.