PaperlayerView exact revision on arXiv

Source-linked AI summary

DRL-Based AoI Minimization for RSMA in Finite-Blocklength MU-MISO

Enes Kaya, Mehmet Alp Demircioğlu, Elif Tugce Ceran, Melda Yuksel

arXiv:2610.11396v1cs.ITmath.IT

TL;DR

In FBL MU-MISO systems, jointly optimizing RSMA resources for fresh status updates is analytically intractable. The paper applies actor–critic DRL to learn dynamic resource-allocation policies, and reports lower AoI than benchmarks, especially at short blocklengths. At high SNR and long blocklengths, benchmarks approach the proposed methods, which achieve similar AoI at shorter blocklengths.

  • Problem

    Jointly optimizing RSMA parameters for long-term AoI in dynamic FBL environments is analytically intractable, despite the need to manage freshness under short-packet decoding errors.

    Source: §I · p. 1

  • Method

    The paper formulates FBL-RSMA AoI minimization as an MDP and uses DDPG and TD3 actor–critic agents to learn resource-allocation policies from AoI and channel states.

    Source: §III · p. 3

  • Results

    Across the considered two-user scenarios, RSMA-RL consistently outperforms benchmarks, with largest gains at short blocklengths, while at high SNR and long blocklengths benchmarks approach its performance.

    Source: §IV · p. 4

  • Takeaways & Limitations

    The learned methods show smooth AoI minima over a range of blocklengths, reducing sensitivity to blocklength choice compared with more sensitive benchmarks.

    Source: §IV · p. 4

    The reported evaluation considers a two-user setting; scalability to larger numbers of users and antennas is left for future work.

    Source: §IV · p. 4, §V · p. 4

Abstract

from arXiv · show

This letter investigates age-of-information (AoI) minimization in multi-user wireless networks operating in the finite-blocklength (FBL) regime, which is critical for low-latency transmission of short state-update packets. While rate-splitting multiple access (RSMA) provides a powerful and flexible framework for interference management in multi-user FBL systems, the joint optimization of its parameters, such as precoding vectors, power allocation, and rate-splitting ratios, to guarantee information freshness results in analytically intractable complexity. To address this challenge, we propose an actor--critic deep reinforcement learning (DRL) framework to learn dynamic resource-allocation policies in multi-user multiple-input single-output (MU-MISO) broadcast channels. Simulation results show that the proposed RSMA-RL framework achieves consistently lower AoI than the state-of-the-art benchmarks, with substantial gains observed at low signal-to-noise ratio (SNR) and short blocklengths, while matching benchmark performance at high SNR with significantly lower online complexity via a single neural-network forward pass at execution.

I. INTRODUCTION

Short-packet FBL communications make decoding errors consequential for reliability and latency, while AoI captures the freshness of status updates. The introduction identifies joint AoI optimization for FBL-RSMA in MU-MISO as unresolved and proposes actor–critic DRL to address its complexity.

I. INTRODUCTION

  • RSMA splits each user’s message into common and private parts, allowing partial interference decoding while treating the remainder as noise.This approach bridges full interference decoding and treating all interference as noise.Source: §I · p. 1
  • FBL short packets have non-negligible decoding error, and AoI captures status-update freshness beyond transmission latency alone.Under FBL constraints, shorter blocklengths reduce transmission time but increase BLER.Source: §I · p. 1
  • Prior AoI work either uses single-antenna DRL or MU-MISO SCA without FBL or rate-splitting, leaving joint FBL-RSMA optimization unresolved.The joint optimization is analytically intractable in dynamic environments, while conventional methods repeat optimization for each channel realization.Source: §I · p. 1
  • The letter proposes a unified DDPG/TD3 framework that jointly controls common/private stream powers and precoders to minimize AoI, with one forward pass at execution.The authors report gains over RSMA benchmarks, especially at low SNR and short blocklengths.Source: §I · p. 1
  • The introduction presents this joint optimization problem as posed for the first time and motivates learning policies from channel and AoI states.The stated contribution targets the latency–reliability–freshness tradeoff.Source: §I · p. 1

II. SYSTEM MODEL AND PROBLEM FORMULATION

The system uses one-layer RSMA in a downlink MU-MISO channel, with a common stream and user-specific private streams. Its FBL rates, decoding errors, and power allocation define the resource-allocation setting for AoI optimization.

II. SYSTEM MODEL AND PROBLEM FORMULATION

  • Each user’s message is split into a common part and a private part; the common parts are jointly encoded, while each private part has its own stream.The base station serves K single-antenna users using M antennas and one common stream.Source: §II · p. 2
  • The streams form a symbol vector that is linearly precoded, with unit-power symbols and total transmit power constrained by Pmax.The received signal includes each user’s channel and receiver noise, and channels remain constant within a block but vary independently across blocks.Source: §II · p. 2
  • FBL theory approximates common and private stream rates using blocklength n and target BLERs; the common rate is limited by the weakest user.Each user’s private rate is r_k I_k, where r_k is the fraction allocated to the private part, and its common portion requires (1−r_k)I_k.Source: §II · p. 2
  • A user’s overall BLER is the probability that either its common or private stream is decoded in error.The overall error probability combines common- and private-stream BLERs as ǫ_k = ǫ_c,k + ǫ_p,k − ǫ_c,kǫ_p,k.Source: §II · p. 2

A. AoI Model

The AoI model tracks time since each user’s latest update was generated and links its evolution to decoding success or failure. The optimization seeks a transmission policy over rate-splitting ratios and precoding that minimizes long-term average AoI across users.

A. AoI Model

  • AoI for user k is the elapsed time since its most recent update was generated, under a generate-at-will model with one transmission block per slot.The model considers fixed information bits and packet size for a given system.Source: A. AoI Model · p. 2
  • After a successful decode, user k’s AoI resets to n; after an error, it increases by n.The transition probabilities are 1−ǫ_k for reset and ǫ_k for increase.Source: A. AoI Model · p. 2
  • The objective is to minimize long-term average AoI across users by choosing the rate-splitting ratio vector r and precoding matrix P.Source: A. AoI Model · p. 2

III. DRL-BASED AOI MINIMIZATION FOR FBL-RSMA

The framework casts FBL-RSMA AoI minimization as a continuous-action MDP and uses DDPG or TD3 to learn resource-allocation policies. The actor produces constrained precoding, power, and rate-splitting choices, while execution requires only one actor forward pass.

III. DRL-BASED AOI MINIMIZATION FOR FBL-RSMA

  • DDPG and TD3 solve the AoI-minimization MDP, whose state contains users’ AoIs and channel information and whose reward is the negative sum of next-state AoIs.Decoding outcomes governed by BLER determine state transitions.Source: §III · pp. 2–3
  • The actor maps the flattened AoI and channel state to precoding, power-allocation, and rate-splitting actions, with normalized precoders and total power constrained by construction.The precoding vectors are parameterized in polar coordinates; rate-splitting ratios lie in [0, 1].Source: §III · p. 3
  • TD3 uses twin critics and the minimum critic estimate to mitigate overestimation, whereas DDPG uses a single critic.Both algorithms use the same state-action representations, rewards, actor architectures, and constraint handling.Source: §III · p. 3
  • DRL inference requires one actor forward pass, O(Cactor), while SCA-based benchmarks incur high computational complexity at each channel realization.The DRL approach shifts computational burden to offline training.Source: §III · p. 3

IV. NUMERICAL RESULTS AND DISCUSSION

Across the evaluated SNR and channel-knowledge settings, RSMA-RL achieves lower AoI than the benchmarks, with its largest gains at short blocklengths. Its AoI also remains relatively robust to blocklength choice, while benchmark performance approaches it at high SNR and long blocklengths.

IV. NUMERICAL RESULTS AND DISCUSSION

  • The proof-of-concept simulations use two antennas and two users, with agents trained under perfect CSIT and tested under imperfect CSIT.Imperfect-CSIT channels follow a Gauss–Markov model with correlation coefficient 0.98, and the base station has imprecise knowledge of previous channel realizations.Source: §IV · pp. 3–4
  • The comparison includes SCA-based RSMA-LowLatency and RSMA-MaxMinFair, greedy AoI-based MRT without RSMA, and an RL-based NOMA method.The benchmark methods are evaluated using the same FBL BLER and AoI model.Source: §IV · p. 4
  • RSMA-RL consistently achieves substantially lower AoI than the RSMA, NOMA-RL, and greedy-AoI/MRT benchmarks, with especially pronounced gains at short blocklengths.At large blocklengths, it also significantly outperforms greedy-AoI with MRT.Source: §IV · p. 4
  • At high SNR and long blocklengths, RSMA-MaxMinFair and NOMA-RL approach the proposed methods, which achieve similar AoI at much shorter blocklengths.The passage attributes the benchmark convergence to sufficient channel capacity for reliably serving both users and reduced resource contention.Source: §IV · p. 4
  • Both RSMA-RL agents show smooth, stable AoI with a U-shaped or broad, nearly flat minimum, while benchmarks degrade rapidly away from their optimal blocklength.RSMA-DDPG slightly outperforms RSMA-TD3; the authors suggest TD3 target-policy smoothing may be less effective near the sharp FBL decoding threshold.Source: §IV · p. 4
  • Under perfect CSIT, minimum AoI falls from approximately 100 at 6 dB to 60–70 at 12 dB; under imperfect CSIT, it falls from about 250 to 110.Increasing SNR also decreases the optimal blocklength, while imperfect CSIT increases AoI across schemes and SNR values.Source: §IV · p. 4

V. CONCLUSION

The paper presents a DRL framework for AoI minimization in RSMA-enabled MU-MISO systems under finite-blocklength constraints. It reports effective dynamic resource-allocation policies from DDPG and TD3 and identifies scalable RL architectures for future work.

V. CONCLUSION

  • The work establishes a DRL framework for AoI minimization in RSMA-enabled MU-MISO systems under finite-blocklength constraints.It reports that DDPG and TD3 learn effective dynamic resource-allocation policies.Source: §V · p. 4
  • Future work will investigate modular, hierarchical, and multi-agent RL architectures for systems with more users and antennas.Source: §V · p. 4
Loading 2610.11396v1…