PaperlayerView exact revision on arXiv

Source-linked AI summary

Movable-Element Multi-Functional RIS-Assisted Hybrid RSMA-NOMA: A Constrained Hybrid Reinforcement Learning Approach

Li-Hsiang Shen, Chi-Te Kuo

arXiv:2610.34701v1cs.AIeess.SP

TL;DR

The paper addresses joint optimization challenges arising from movable-element multifunctional RISs and hybrid RSMA-NOMA transmissions under practical constraints. It proposes MM-RIS and CRAFTS, combining hybrid multi-agent learning with federated transformer features and adaptive penalties. The reported simulations show improved energy efficiency and constraint satisfaction against benchmarks, alongside trade-offs from element movement and deployment choices.

  • Problem

    Integrating movable elements into multifunctional RISs for hybrid RSMA-NOMA creates a highly coupled mixed continuous-discrete optimization problem with movement overhead, interference, and power, QoS, and hardware constraints.

    Source: B. Challenges and Contributions · p. 3

  • Method

    The paper jointly optimizes MM-RIS-assisted hybrid RSMA-NOMA using CRAFTS, which combines MAPPO and MADQN with transformer features, federated learning, bidirectional parametrized sharing, and adaptive Lagrangian penalties.

    Source: B. Challenges and Contributions · p. 3, §VI · p. 18

  • Results

    The proposed MM-RIS achieves superior energy efficiency across several configurations, while CRAFTS substantially outperforms centralized and decentralized DRL, conventional precoding, and heuristic benchmarks.

    Source: §VI · pp. 18–19

  • Takeaways & Limitations

    Energy efficiency depends on balancing mobility’s spatial adaptation gains against its movement overhead, which can make fixed elements more efficient at larger element counts.

    Source: D. Different Types of Movable Structures · pp. 15–16

Abstract

from arXiv · show

In this work, we propose a movable-element (ME) multi-functional reconfigurable intelligent surface (MF-RIS) architecture, termed MM-RIS, which integrates element mobility with adaptive signal reflection, transmission, amplification, and energy harvesting (EH) to enhance communication flexibility and energy self-sustainability. A hybrid rate-splitting multiple access (RSMA) and non-orthogonal multiple access (NOMA) transmission scheme is further designed to improve spectral efficiency. We formulate an energy efficiency (EE) maximization problem by jointly optimizing transmit beamforming, power and rate allocation, MM-RIS configurations, panel deployment, and ME positions. To address the resulting non-convex mixed continuous-discrete optimization problem, we propose constrained hybrid deep reinforcement learning with federated transformer and parametrized sharing (CRAFTS). CRAFTS integrates multi-agent proximal policy optimization (MAPPO) and multi-agent deep Q-networks (MADQN) through bi-directional parametrized sharing to handle continuous and discrete actions, respectively. Moreover, a transformer-based architecture and federated learning (FL) are incorporated to enhance state representation and distributed knowledge sharing, whilst a Lagrangian-based adaptive penalty mechanism facilitates the constraint satisfaction. Simulation results have demonstrated that the proposed MM-RIS architecture and CRAFTS framework substantially improve EE and constraint satisfaction compared to the benchmarks in the open literature. Moreover, the results validate the benefits of spatially reconfigurable MM-RIS, hybrid RSMA-NOMA transmissions, and intelligent collaborative learning, highlighting the potential of the proposed framework for energy-efficient and adaptive next-generation wireless networks.

I. INTRODUCTION

The introduction motivates combining movable elements and multifunctional RIS capabilities with hybrid RSMA-NOMA, then presents a joint energy-efficiency optimization and a constrained hybrid-learning approach.

A. Background and Literature Reviews

  • Hybrid RSMA-NOMA combines RSMA message splitting with NOMA power-domain multiplexing and SIC to improve multi-user separation and interference mitigation.The introduction notes that fading and inter-user interference can still degrade signal quality and limit practical deployment.Source: A. Background and Literature Reviews · p. 1
  • Conventional RIS is described as limited to half-space coverage and dependent mainly on external power, while MF-RIS adds STAR coverage and energy harvesting.Active components can also amplify signals to compensate for severe path loss.Source: A. Background and Literature Reviews · p. 1
  • Prior ME/RIS work leaves the integration of element mobility with MF-RIS reflection, transmission, amplification, and self-sustaining energy harvesting largely unexplored.The introduction also identifies coupled phase-shift effects as a practical hardware constraint for active MF-RIS.Source: A. Background and Literature Reviews · pp. 1–2

B. Challenges and Contributions

  • The paper proposes an MM-RIS framework combining movable elements with reflection, transmission, amplification, and energy harvesting for hybrid RSMA-NOMA downlink transmissions.The stated aim is to improve adaptability, coverage, and energy self-sustainability.Source: B. Challenges and Contributions · p. 3
  • The proposed energy-efficiency problem jointly optimizes transmit beamforming, power and rate allocation, MM-RIS settings, panel deployment, and element positions.The formulation includes mixed continuous-discrete variables and practical coupling constraints.Source: B. Challenges and Contributions · p. 3
  • CRAFTS combines MAPPO and MADQN for continuous and discrete decisions, with transformer features, federated learning, bidirectional parametrized sharing, and adaptive Lagrangian penalties.The penalty mechanism dynamically updates multipliers to address constraint satisfaction.Source: B. Challenges and Contributions · p. 3

A. Channel Model

The channel model uses Rician fading for the BS-to-MM-RIS link and accounts for spatial correlation induced by movable elements.

A. Channel Model

  • The BS-to-MM-RIS channel uses a Rician fading model with line-of-sight and non-line-of-sight components.The passage identifies the reference-distance path loss, path-loss exponent, and Rician factor.Source: A. Channel Model · p. 4
  • Movable-element spatial correlation is modeled with Jake’s model using a zeroth-order Bessel function of the distances between elements.The correlation matrix is represented through its eigenvectors and diagonal eigenvalue matrix.Source: A. Channel Model · p. 5
  • The channel description includes a direct BS-to-user link with a Rayleigh-fading non-line-of-sight component and an MM-RIS-to-user channel.The MM-RIS-to-user model specifies line-of-sight channel structure and elementwise steering vectors.Source: A. Channel Model · pp. 4–5

B. Signal Model

The signal model groups users by side and channel direction, then uses common and private streams with SIC to define achievable rates. It accounts for element-movement time by separating movement and communication phases.

B. Signal Model

  • Users are grouped by reflection or transmission side and then by effective channel direction, with highly correlated channels preferentially assigned together.The grouping procedure uses channel correlation to support beamforming.Source: B. Signal Model · p. 5
  • Each group receives a common stream and user-specific private streams, with a shared group-level private precoder reducing the number of beamforming variables.Private messages are superimposed using user-specific power coefficients.Source: B. Signal Model · p. 5
  • Two-stage SIC separates common and private signals before NOMA private-message decoding, with stronger users decoding and cancelling weaker users’ private messages.Successful SIC requires each decoding user to meet the intended user’s SINR threshold.Source: B. Signal Model · p. 5
  • The achievable user rate combines allocated common-message rate and private-message rate, subject to a common-stream rate condition across users.The effective transmission block accounts for time reserved for ME repositioning, during which data transmission is halted.Source: B. Signal Model · pp. 5–6
  • Panel deployment and element repositioning operate on different timescales, with panel positions updated slowly and elements repositioned faster for channel adaptation.The EE movement-overhead term includes element reconfiguration but excludes transient energy and delay from relocating an entire panel each cycle.Source: B. Signal Model · p. 6

C. EH and Power Consumption Model

The power model accounts for per-element energy harvesting, RIS control and amplification consumption, and ME movement; self-sustainability requires consumption not to exceed harvested power.

C. EH and Power Consumption Model

  • Each MM-RIS element’s harvesting coefficient determines whether it operates in signal mode or contributes to the harvested-power model.The model adopts a nonlinear energy-harvesting conversion function.Source: C. EH and Power Consumption Model · p. 6
  • MM-RIS power consumption includes control circuitry, RF-to-DC conversion, amplification, and ME movement costs.The model counts PIN diodes used to quantize harvesting ratio, amplitude, and phase settings.Source: C. EH and Power Consumption Model · p. 6
  • Each MM-RIS is self-sustainable only when its total consumed power does not exceed the power harvested by its elements.The power accounting also identifies BS beamforming power as a separate term.Source: C. EH and Power Consumption Model · pp. 6–7

D. Problem Formulation

The formulation maximizes system energy efficiency while enforcing user-rate, MM-RIS, power-allocation, and deployment constraints. The resulting problem is non-convex and mixes discrete and continuous variables.

D. Problem Formulation

  • The objective maximizes system EE while guaranteeing minimum user rates, valid MM-RIS configurations, self-sustainability, and operational power limits.MM-RIS power consumption cannot exceed harvested power, and transmit power must remain below the BS limit.Source: D. Problem Formulation · p. 7
  • Constraints cover MM-RIS coefficients, user-rate requirements, BS power, hybrid NOMA-RSMA allocation, minimum ME spacing, and MM-RIS deployment boundaries.The allocation constraints bound power factors and rate allocation.Source: D. Problem Formulation · p. 7
  • The optimization is highly challenging because it is non-convex, nonlinear, high-dimensional, and mixes discrete and continuous variables.The constraints include coupled practical limitations.Source: D. Problem Formulation · p. 7

III. SCALING-LAW ANALYSIS

The scaling analysis shows that adding MEs increases signal power but yields at most logarithmic rate growth, while residual interference and power overhead can limit EE. Under the stated assumptions, EE need not improve monotonically with ME count.

Proposition 1

  • Under coherent phase configuration, bounded amplification coefficients, and finite channel moments, desired signal power scales as O(M^2) while amplification noise scales as O(M).With bounded ME movement, effective achievable rate retains O(log M) scaling.Source: §III · p. 7

Corollary 1

  • With aggregate residual interference PI(M) = O(M^ρ) for 0 ≤ ρ ≤ 2 and effective total power Ptot(M) = O(M^δ), where 0 ≤ δ ≤ 1, ρ = 2 yields interference-limited SINR γ(M) = O(1).Source: §III · p. 8

Scaling-law results

  • For δ > 0, EE tends to zero as M grows; with δ = 1, EE scales as O(log M/M) for ρ < 2 and O(1/M) for ρ = 2.A finite favorable operating region generally depends on the balance among array gains, interference, amplification noise, harvested energy, and hardware or movement overhead.Source: §III · p. 8
  • For ρ = 2, rate becomes bounded in the interference-limited case, while uncompensated polynomial power growth with δ > 0 eventually dominates rate improvement, so increasing MEs need not improve EE monotonically.Source: §III · p. 8

IV. PROPOSED CRAFTS FRAMEWORK

CRAFTS models the BS and each MM-RIS as agents in a hybrid DRL framework. It combines discrete and continuous actions, with a shared reward based on EE and its constraints.

IV. PROPOSED CRAFTS FRAMEWORK

  • Each agent represents either the BS or one MM-RIS, and the framework combines DQN and PPO to handle discrete and continuous actions.The paper notes that using only DQN or PPO would require discretizing continuous variables or recovering them, adding overhead and potential quantization errors.Source: A. Multi-Agent Constrained Hybrid DRL · p. 8
  • MM-RIS agents select discrete element modes and continuous amplitude, phase, location, and element-position actions; the BS agent outputs continuous beamforming, power-split, and common-rate actions.The action-space description is distributed across the MM-RIS and BS agent definitions.Source: A. Multi-Agent Constrained Hybrid DRL · p. 8
  • All agents receive the same instantaneous shared reward, defined as overall EE in conjunction with system constraints.Figure 2 depicts the proposed multi-agent hybrid constrained DRL framework.Source: A. Multi-Agent Constrained Hybrid DRL · p. 9, Figure 2 · p. 9

1) Lagrangian-based Constrained DRL:

The constrained DRL method uses an augmented Lagrangian reward and alternates policy optimization with dual-variable updates. It also makes multiplier learning rates part of the agents’ actions.

1) Lagrangian-based Constrained DRL:

  • The augmented Lagrangian combines the shared reward with constraint penalties, avoiding penalty coefficients that require labor-intensive, configuration-sensitive tuning.The paper says hierarchical RL or metaheuristic hyperparameter optimization can add computational and training overhead and does not guarantee constraint satisfaction.Source: 1) Lagrangian-based Constrained DRL · p. 9
  • An alternative optimization alternates the policy Ξ as primal variables with updates to dual variables for the constraints.The dual variables are updated to minimize constraint violations.Source: 1) Lagrangian-based Constrained DRL · p. 9
  • Unlike conventional Lagrangian methods with fixed dual learning rates, CRAFTS jointly optimizes actions and multiplier learning rates.The learning rates for inequality- and equality-constraint multipliers are included in each agent’s continuous action.Source: 1) Lagrangian-based Constrained DRL · p. 9

2) MADQN for Discrete Policy:

MADQN handles discrete actions with a Q-network, while MAPPO uses actor–critic policy optimization for continuous actions.

2) MADQN for Discrete Policy:

  • MADQN estimates action values with a Q-network and selects discrete actions using an ε-greedy strategy that balances exploration and exploitation.Its exploration rate decays over time.Source: 2) MADQN for Discrete Policy · p. 9
  • MADQN trains from replay-buffer mini-batches and uses a periodically softly updated target network to stabilize learning.The Q-network is updated by minimizing temporal-difference error.Source: 2) MADQN for Discrete Policy · pp. 9–10

3) MAPPO for Continuous Policy:

  • MAPPO uses per-agent actor–critic networks, sampling continuous actions from a state-conditioned multivariate Gaussian policy.The policy uses a clipped surrogate objective and generalized advantage estimation; the reward includes penalty constraints.Source: 3) MAPPO for Continuous Policy · p. 10

B. Intra-Agent Bi-Directional Parametrized Sharing

CRAFTS couples discrete and continuous policy branches within agents, then coordinates agents through transformer-based state features and federated model aggregation.

B. Intra-Agent Bi-Directional Parametrized Sharing

  • Bi-directional parametrized sharing feeds each branch the other branch’s previous action, coupling PPO’s continuous and DQN’s discrete decisions.The mechanism is applied only to MM-RIS agents with hybrid discrete-continuous action structures.Source: B. Intra-Agent Bi-Directional… · p. 10

C. Inter-Agent Federated Transformer

  • Each agent applies multi-head self-attention to projected state features to produce an attention-aware state representation.The attention heads are concatenated and passed through a trainable output projection.Source: C. Inter-Agent Federated Transformer · pp. 10–11
  • Agents keep local PPO and DQN modules, periodically upload their parameters for separate aggregation, and receive updated global parameters.The aggregation server may be implemented at the edge or BS controller; the process is described as preserving local data privacy and reducing communication overhead.Source: C. Inter-Agent Federated Transformer · p. 11
  • Training repeats local action selection, policy and multiplier updates, and periodic federated aggregation and broadcast.Agents use a shared reward, while MAPPO determines continuous Lagrangian-related actions for adaptive dual-variable updates.Source: C. Inter-Agent Federated Transformer · p. 11

V. SIMULATION RESULTS

In the simulated multi-MM-RIS downlink, CRAFTS leads the reported EE comparisons and achieves the highest constraint satisfaction among the tested constraint strategies.

V. SIMULATION RESULTS

  • Simulations evaluate a hybrid RSMA-NOMA network with Q = 3 MM-RISs, each equipped with M = 16 elements, and a BS with N = 6 antennas serving Jk = 2 users associated with K = 2 directions.Source: §V · pp. 11–12

A. Convergence

  • CRAFTS achieves the highest EE among the reported DRL benchmarks, while its variants show gains from intra-agent sharing and transformer-plus-FL enhancements.The comparison includes PPO, DQN, HDRL, and MA-HDRL; the paper attributes CRAFTS’s result to its combined hybrid DRL, FL, transformer, and bi-directional sharing components.Source: A. Convergence · p. 12, Figure 4 · p. 12
  • CRAFTS achieves the highest constraint satisfaction ratio among the tested strategies, while fixed penalty coefficients yield the highest EE at the expense of adaptive constraint coordination.The compared alternatives include partial dual optimization, fixed dual hyperparameters, fixed penalty coefficients, and Twin-Critic.Source: A. Convergence · pp. 12–13, Figure 5 · p. 13

B. Effect of MF-RIS Elements

Across the evaluated configurations, energy harvesting, hybrid access, and spatial movement can improve energy efficiency, but their benefits depend on balancing communication gains against operational overhead. Movement speed, surface deployment, and user grouping likewise involve trade-offs rather than uniformly increasing EE.

B. Effect of MF-RIS Elements

  • EH raises EE by offsetting element power and supporting amplification, but added elements can eventually reverse gains; reflection-only operation also limits coverage to one side.The proposed MM-RIS supports reflection and transmission, amplification, and EH, providing full-space coverage.Source: Figure 6 · p. 13, B. Effect of MF-RIS Elements · pp. 13–14

C. Various Multiple Access Mechanisms

D. Different Types of Movable Structures

E. Impact of ME Movement

  • Faster ME movement raises sum rate but also power consumption, making EE non-monotonic; longer communication durations help amortize repositioning, while both D = λ and D = λ/4 degrade performance relative to D = λ/2.Source: E. Impact of ME Movement · p. 16

F. Effect of MM-RIS Deployment Strategy

G. Impact of User Grouping Mechanisms

The results favor channel-aware user grouping and CRAFTS over the evaluated benchmarks, while ablations and complexity measurements expose trade-offs in optimization and implementation. The conclusion also notes that EE gains must be balanced against operational overhead and that generalization can be limited by distribution shifts.

G. Impact of User Grouping Mechanisms

H. Benchmark Comparison

  • Full optimization yields the highest EE; omitting individual optimizations degrades performance, and omitting BS beamforming produces the lowest EE at around 0.4 bits/J.EE also peaks at N = 6 in the reported antenna ablation.Source: H. Benchmark Comparison · pp. 16–17, Figure 14 · p. 17
  • CRAFTS achieves the highest EE across the evaluated antenna numbers, ahead of HDRL-BiPS, HDRL, conventional beamforming, genetic-algorithm optimization, and random configurations.Source: Figure 15 · p. 17, H. Benchmark Comparison · p. 18

I. Computational Complexity Analysis

  • CRAFTS uses 1.11 MB and takes 0.026 s per step, compared with 0.44 MB and 0.017 s for CRAFTS without transformer and FL.The paper reports these added costs alongside competitive execution speed.Source: §I · p. 18

VI. CONCLUSIONS

  • The conclusion identifies a trade-off: excessive amplification, movement flexibility, or surface deployment may increase power consumption and reduce EE, while distribution shifts may degrade CRAFTS generalization.Source: §VI · pp. 18–19

Related papers

Loading 2610.34701v1…