PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Multiagent Soft Actor Critic for IRS Assisted Cognitive Radio Networks under Partial Observability

Satvika Pullisani, Usama Mir, Ubaid Abbasi

arXiv:2610.22884v1cs.NI

TL;DR

Distributed SUs in IRS-assisted cognitive radio networks must coordinate spectrum access from local observations while protecting primary users, but existing approaches do not unify this setting with federated multi-agent SAC. The paper proposes FMASAC, presents illustrative trends against decentralized baselines, and identifies evaluation questions that remain open.

  • Problem

    IRS-assisted CRNs require decentralized SUs to make local-observation transmission decisions while protecting PUs, a setting not jointly addressed by existing federated multi-agent IRS control approaches.

    Source: §I · pp. 1–2

  • Method

    FMASAC has each SU learn locally from its observations while a federated server periodically aggregates model parameters without raw data exchange.

    Source: §I · p. 2, §III · p. 3

  • Results

    Illustrative trends show FMASAC above decentralized baselines in throughput, smoother in convergence, and expected to reduce action variance, pending simulation validation.

    Source: B. Illustrated Trends · pp. 4–5, §VI · p. 5

  • Takeaways & Limitations

    Parameter averaging avoids sharing CSI or observations, while increasing local interaction steps K trades fewer aggregation rounds for greater model drift between aggregations.

    Source: C. Communication Cost · p. 4

    The evaluation remains planned, including tests varying the aggregation interval K against throughput, constraint satisfaction, and communication cost.

    Source: §V · p. 5

Abstract

from arXiv · show

Intelligent reflecting surfaces (IRSs) can improve spectrum efficiency and interference control in cognitive radio networks (CRNs) by reconfiguring the propagation environment. This paper presents FMASAC, a federated multi-agent soft actor-critic framework for IRS-assisted CRNs under partial observability, in which distributed secondary users (SUs) jointly select their transmit power and the phase shifts of an assigned subset of IRS elements using only local observations. Primary user (PU) protection is incorporated through an interference-penalized reward, and SUs coordinate by periodically averaging model parameters at a federated server, without exchanging raw observations, channel state information, or replay data. We formulate the problem as a partially observable Markov decision process, specify the local learning and aggregation procedure, and characterize its per-round communication cost. Illustrative performance trends contrast FMASAC with centralized SAC, independent SAC, IRS-fixed multi-agent reinforcement learning, and a heuristic CRN strategy. We also outline the simulation-based evaluation and the open challenges, namely constraint handling, agent heterogeneity, and scalability, that are needed to validate the framework.

I. INTRODUCTION

IRS-assisted CRNs must coordinate multiple SUs using limited local observations while protecting PUs, yet prior approaches do not jointly address federated learning, multi-agent SAC, and partial observability. The paper proposes FMASAC and outlines its formulation, learning procedure, illustrative trends, and evaluation challenges.

I. INTRODUCTION

  • Prior IRS-assisted CRN approaches lack a unified treatment of decentralized multi-agent learning, spectrum sharing, and IRS-aware control under partial observability.Existing MARL approaches may rely on centralized training, shared observations, or frequent information exchange.Source: §I · pp. 1–2
  • FMASAC combines federated learning, multi-agent SAC continuous control, and partial observability for IRS-assisted CRNs without raw-data sharing or global CSI.Each SU trains locally while a federated server aggregates model parameters.Source: §I · p. 2
  • The paper jointly formulates SU transmit power and IRS phase control under a PU interference constraint, using an explicit IRS-element partition for multiple agents.The problem is modeled as a POMDP, with the reward, SAC updates, and federated aggregation rule specified.Source: §I · p. 2
  • The paper presents illustrative trends against four benchmarks and identifies simulation-based evaluation and open challenges needed to validate the framework.The stated challenges include constraint handling, agent heterogeneity, and scalability.Source: §I · p. 2

II. SYSTEM MODEL AND PROBLEM FORMULATION

The system model considers a primary link and multiple distributed secondary links sharing licensed spectrum, with an IRS assisting secondary transmissions. Secondary access is constrained by aggregate interference at the primary user.

II. SYSTEM MODEL AND PROBLEM FORMULATION

  • The model includes one primary base station serving one PU and M distributed SU transmitter–receiver pairs accessing licensed spectrum.An IRS with N passive reflecting elements assists the secondary transmissions.Source: §II · p. 2
  • The topology depicts direct, IRS-reflected, and interference paths in the IRS-assisted CRN.Source: Figure 1 · p. 2
  • The SUs may access the licensed spectrum while keeping aggregate interference at the PU below a prescribed threshold.Source: §II · p. 2

A. Channel and IRS Model

The channel model combines direct and IRS-reflected propagation, with IRS elements applying phase shifts. SU link quality and PU interference are described through channel-dependent quantities.

A. Channel and IRS Model

  • The IRS provides a controllable reflected path that can strengthen useful SU signals while reducing interference toward the PU.Channels follow Rayleigh small-scale fading with log-distance path loss.Source: A. Channel and IRS Model · p. 2
  • Each IRS element applies a phase shift θ_n(t) in [0, 2π) to configure the reflected channel.Source: A. Channel and IRS Model · p. 2
  • The effective SU-to-SU channel combines a direct link with transmitter-to-IRS and IRS-to-receiver channels; the PU channel is defined analogously.Source: A. Channel and IRS Model · pp. 2–3
  • The model defines each SU’s achievable rate from its SINR and its interference at the PU from transmit power and the SU-to-PU channel.Source: A. Channel and IRS Model · p. 3
  • The IRS elements are divided into disjoint subsets, with each SU controlling only the phases of its assigned elements.This avoids conflicting phase requests and keeps each agent’s action space fixed.Source: A. Channel and IRS Model · p. 3

B. POMDP Formulation

The formulation treats decentralized SU control under local information as a POMDP, with PU interference handled through a reward penalty and federated coordination.

B. POMDP Formulation

  • Each SU observes local information rather than the global state, making the control problem a POMDP over joint actions, observations, transitions, and rewards.Local information includes received signal strength, partial CSI for the SU's links, a PBS-broadcast PU interference indicator, and limited IRS feedback.Source: B. POMDP Formulation · p. 3
  • The aggregate PU interference constraint is enforced through the reward, where a weighted penalty discourages violations without ruling them out during learning.The violation term is available to each SU through the PBS interference indicator, without requiring other SUs’ channels or actions.Source: B. POMDP Formulation · p. 3
  • SUs learn from their own interaction data and periodically upload model parameters for server aggregation, as depicted in the POMDP decision loop.The figure caption identifies the loop as combining POMDP decisions with federated aggregation.Source: B. POMDP Formulation · p. 3, Figure 2 · p. 3

III. PROPOSED FEDERATED MASAC FRAMEWORK

FMASAC combines local SAC learning by each SU with periodic federated aggregation to improve policies using local observations and interaction data.

III. PROPOSED FEDERATED MASAC FRAMEWORK

  • Each SU interacts with the IRS-assisted environment using local observations and replay data, while a server periodically aggregates and broadcasts model parameters.This enables distributed policy improvement without exchanging raw data.Source: §III · p. 3

A. Local SAC Learning

  • Local SAC agents use actors and twin critics conditioned on local observations and actions, with entropy regularization supporting exploration and stability under partial observability.SAC is used because transmit power and phase shifts are continuous decisions.Source: A. Local SAC Learning · p. 3
  • Training repeats local interaction, replay-buffer storage, critic and actor updates, and target-critic soft updates before parameter upload and server aggregation.The server broadcasts aggregated parameters to the SUs after each aggregation step.Source: A. Local SAC Learning · p. 4

B. Federated Aggregation

After local learning, the federated server averages SU model parameters and broadcasts the shared result, under an assumption that SUs are sufficiently similar.

B. Federated Aggregation

  • After K local steps, the server forms a weighted average of uploaded actor parameters and applies the same aggregation rule to each critic before broadcasting the result.The weights are based on each SU’s replay-buffer size.Source: B. Federated Aggregation · p. 4
  • A shared averaged model assumes sufficiently similar SUs; observation normalization supports this assumption, while heterogeneous settings may call for personalized aggregation.The paper identifies personalized aggregation as a topic for heterogeneous settings.Source: B. Federated Aggregation · p. 4

C. Communication Cost

Communication cost scales with the number of SUs, rounds, and model parameters, but not with replay-buffer size or local interaction steps.

C. Communication Cost

  • Each round sends and receives |φ| + 2|ψ| parameters per SU, for total traffic of 2M(|φ|+2|ψ|) per round and 2MR(|φ|+2|ψ|) over R rounds.|φ| and |ψ| are actor and per-critic parameter counts, respectively.Source: C. Communication Cost · p. 4
  • Communication cost is independent of replay-buffer size and local interaction steps K, and no CSI or observations leave an SU.The exchanged payload consists of model parameters.Source: C. Communication Cost · p. 4
  • Increasing K reduces the rounds needed for a given amount of local experience but allows local models to drift further between aggregations.The paper identifies this as the trade-off examined in its planned evaluation.Source: C. Communication Cost · p. 4

IV. ILLUSTRATIVE PERFORMANCE TRENDS

The section presents illustrative, not simulation-trained, performance trends for FMASAC against four benchmarks in a fixed IRS-assisted setting. The trends portray throughput, PU interference, convergence, and action-variance trade-offs, but require simulation to validate.

A. Scope of This Section

  • The curves use controlled synthetic traces, not agents trained in a simulated network, so they illustrate intended trade-offs rather than measured performance.The planned simulation-based evaluation is described in Section V.Source: A. Scope of This Section · p. 4
  • The illustrated setting uses one PBS, one PU, distributed SUs, a fixed-position IRS, Rayleigh fading with log-distance path loss, and 300 training episodes.Source: A. Scope of This Section · p. 4
  • FMASAC is compared with centralized SAC, independent SAC, IRS-fixed MARL, and a heuristic CRN strategy across throughput, normalized PU interference, cumulative reward, and action variance.Centralized SAC is a global-information reference; the heuristic uses conservative transmission under PU protection.Source: A. Scope of This Section · p. 4

B. Illustrated Trends

  • FMASAC is illustrated above independent SAC, IRS-fixed MARL, and the heuristic in SU throughput, while centralized SAC serves as an upper reference using unavailable global information.Source: B. Illustrated Trends · p. 4, Figure 3 · p. 4
  • Learning-based methods are illustrated starting above the normalized PU-interference threshold and approaching it, while only the heuristic remains below throughout at lower throughput.FMASAC fluctuates less than independent SAC and IRS-fixed MARL; penalty-based enforcement is soft, so early violations are expected.Source: B. Illustrated Trends · pp. 4–5, Figure 4 · p. 4
  • FMASAC is illustrated converging more smoothly than decentralized baselines and showing lower action variance than uncoordinated learning, but whether these trends hold in practice remains to be established.Periodic aggregation is expected to smooth learning; the action-variance trend is intended to indicate policy consistency across SUs.Source: B. Illustrated Trends · p. 5, Figure 5 · p. 5, Figure 6 · p. 5

V. LIMITATIONS AND EVALUATION PLAN

The evaluation plan calls for simulation-based validation and targeted tests of communication overhead, scalability, constraint handling, agent heterogeneity, and IRS control architecture.

  • Implement the channel model with SAC agents and report means and confidence intervals over independent random seeds for stated system and network parameters.Source: §V · p. 5
  • Vary the local aggregation interval K to measure throughput and constraint satisfaction against the communication cost derived for the method.Source: §V · p. 5
  • Vary M and N to test whether element partitioning and parameter averaging remain effective as the network grows.Source: §V · p. 5
  • Compare the interference penalty with primal–dual or Lagrangian SAC, reporting violation probability rather than only average interference.Source: §V · p. 5
  • Evaluate non-identically distributed SU locations and channels, comparing plain averaging with personalized federated aggregation.Source: §V · p. 5
  • Compare element partitioning with a dedicated IRS controller agent that sets all N phases from aggregated SU feedback.Source: §V · p. 5

VI. CONCLUSION

The conclusion recaps FMASAC for IRS-assisted CRNs under partial observability and its federated learning procedure. It presents performance trends as illustrative and identifies simulation validation and stronger constraint guarantees as ongoing work.

  • FMASAC has each SU learn transmit power and assigned IRS-element phases from local observations, with periodic federated parameter averaging and no raw-data exchange.Source: §VI · p. 5
  • The framework formulates the problem as a POMDP with an interference-penalized reward and characterizes the communication cost of learning and aggregation.Source: §VI · p. 5
  • Illustrative trends indicate intended trade-offs in throughput, PU protection, convergence smoothness, and multi-user consistency relative to decentralized baselines.Source: §VI · p. 5
  • Simulation validation, the aggregation-frequency trade-off, and stronger constraint guarantees remain ongoing work.Source: §VI · p. 5
Loading 2610.22884v1…