PaperlayerView exact revision on arXiv

Source-linked AI summary

A Multiarmed Bandit Approach for LTE-U/Wi-Fi Coexistence in a Multicell Scenario

Iago Diógenes do Rego, José M. de Castro Neto, Sildolfo F. G. Neto, Pedro M. de Santana, Vicente A. de Sousa Jr., Dario Vieira, Augusto Venâncio Neto

arXiv:2610.20222v1cs.NI

TL;DR

Static duty-cycle settings may not suit LTE-U/Wi-Fi coexistence when offered traffic changes, while Q-learning can require increasingly complex state-action models. This paper applies multiarmed bandits to adaptive duty-cycle selection and reports a simpler approach with improved throughput balance and gain over its prior Q-learning solution.

  • Problem

    Duty-cycle settings that remain fixed may not maximize throughput when offered data rates change, while Q-learning’s state-action modeling can become complex.

    Source: §1 · p. 1, §4 · p. 6

  • Method

    The paper uses a multiarmed-bandit framework to select LTE-U duty-cycle values dynamically for LTE-U/Wi-Fi coexistence.

    Source: §4 · p. 6, §5 · p. 8

  • Results

    The MAB solution offers higher gain than the previous Q-learning solution and more similar throughput for LTE and Wi-Fi.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that dynamic duty-cycle selection is needed when offered data rates frequently change.

    Source: §4 · p. 6

    In simulations where duty cycle 0.7 is optimal during the first half, the simulator warm-up leaves the framework less time to act, though reported performance is unaffected.

    Source: §6 · p. 10

Abstract

from arXiv · show

Recent studies and literature reviews have shown promising results for 3GPP system solutions in unlicensed bands when coexisting with Wi-Fi, either by using the duty cycle (DC) approach or licensed-assisted access (LAA). However, it is widely known that general performance in these coexistence scenarios is dependent on traffic and how the duty cycle is adjusted. Most DC solutions configure their parameters statically, which can result in performance losses when the scenario experiences changes on the offered data. In our previous works, we demonstrated that reinforcement learning (RL) techniques can be used to adjust DC parameters. We showed that a Q-learning (QL) solution that adapts the LTE DC ratio to the transmitted data rate can maximize the Wi-Fi/LTE-Unlicensed (LTE-U) aggregated throughput. In this paper, we extend our previous solution by implementing a simpler and more efficient algorithm based on multiarmed bandit (MAB) theory. We evaluate its performance and compare it with the previous one in different traffic scenarios. The results demonstrate that our new solution offers improved balance in throughput, providing similar results for LTE and Wi-Fi, while still showing a substantial system gain. Moreover, in one of the scenarios, our solution outperforms the previous approach by 6% in system throughput. In terms of user throughput, it achieves more than 100% gain for the users at the 10th percentile of performance, while the old solution only achieves a 10% gain.

1. Introduction

The paper addresses LTE-U/Wi-Fi coexistence in unlicensed spectrum, where performance depends on traffic and duty-cycle settings. It extends prior Q-learning work with a multiarmed-bandit approach intended to improve throughput balance and gain.

1. Introduction

  • Mobile data traffic growth and the cost of licensed spectrum motivate operators to consider unlicensed spectrum alongside other capacity approaches.The introduction cites projected traffic growth and describes unlicensed bands as avoiding spectrum-license purchasing costs.Source: §1 · p. 1
  • Unlicensed-band solutions must account for regional regulations, including whether medium-access sensing or transmission-time limits are required.The paper identifies LTE-LAA, LTE-U/LTE-DC, and NR-U as 3GPP solutions for these requirements.Source: §1 · pp. 1–2
  • The authors’ earlier work proposed centralized Q-learning to increase system throughput in multicell LTE-U/Wi-Fi coexistence and reported system- and user-throughput gains.The present paper builds on that approach.Source: §1 · p. 2
  • The paper applies a simpler multiarmed-bandit algorithm to LTE-U/Wi-Fi coexistence and reports higher gain and more similar throughput for both technologies than its previous Q-learning solution.The paper compares the two approaches across traffic scenarios.Source: §1 · p. 2

2. Related Works

Prior coexistence work studies LTE access mechanisms and reinforcement-learning methods for adjusting duty cycle. The paper motivates multiarmed bandits as a simpler alternative to Q-learning’s state-action modeling burden and presents an adaptive multicell strategy.

2. Related Works

  • Prior Q-learning studies adjust duty cycle using offered data rate, channel occupancy, or queue lengths, while other work explores deep Q-learning for throughput and fairness.The cited approaches include, –.Source: §2 · p. 3
  • Q-learning’s state-action model can grow in complexity as the number of pairs increases, making convergence in dynamic multicellular settings problematic when convergence time is strict.The authors identify MAB as an alternative.Source: §2 · p. 3
  • Other MAB studies address LTE-U/Wi-Fi coexistence in CBRS by tuning power and duty cycle, and M2M/Wi-Fi coexistence by exploiting Wi-Fi transmission gaps.The cited work includes and.Source: §2 · p. 4
  • The authors present an adaptive duty-cycle strategy with better performance and fairness than their Q-learning strategy, without defining state-action pairs and with fewer configuration parameters.They also describe it as flexible enough to allow more duty-cycle choices.Source: §2 · p. 4

3. Evaluation Scenario and Reference Results

The evaluation uses a 3GPP indoor multicell coexistence scenario modeled in ns-3, with simulations based on UDP full-buffer traffic. The section recalls earlier findings that the best duty cycle changes with offered rates and that fixed settings can be insufficient in dynamic conditions.

3. Evaluation Scenario and Reference Results

  • The 3GPP indoor scenario has two operators with four cells each and 40 randomly placed stations in a 120 × 50 m room without walls.Twenty stations belong to each operator; distances distinguish cross-RAT spacing d from same-RAT BS_space.Source: §3 · p. 4
  • The authors model coexistence using ns-3 and its ns3-lbt modification, which implements LTE-U duty-cycle transmission with a 40 ms Almost Blank Subframe window.Each 1 ms subframe is represented by one bit in a 40-bit ON–OFF pattern.Source: §3 · p. 5
  • Earlier results found that the duty cycle maximizing system throughput changes with offered data rates, so a fixed value is insufficient when rates vary dynamically.For unbalanced offered rates, the best duty cycle was reported to be approximately proportional to the imbalance.Source: §3 · p. 5
  • The simulations use 20-second rounds for each fixed duty-cycle value, average operator throughput, and UDP full-buffer offered rates recommended by 3GPP.The section also identifies Table 1 as listing simulation parameters adapted from.Source: §3 · p. 5, Table 1 · p. 6

4. Adaptive Duty Cycle Selection with Reinforcement Learning

Because the throughput-maximizing duty cycle changes with offered data rate, fixed settings may be suboptimal. The paper uses a multiarmed bandit to select duty-cycle values dynamically for LTE-U/Wi-Fi coexistence.

  • A fixed duty-cycle value may not maximize throughput when operators’ offered data rates change, because the maximizing value also changes.Source: §4 · p. 6
  • The MAB framework dynamically selects LTE-U duty-cycle values to maximize aggregated LTE-U and Wi-Fi throughput under changing conditions.The framework selects a value for different combinations of offered data rates and adapts it to real-time conditions.Source: §4 · p. 6
  • A multiarmed bandit simplifies the learning approach by repeatedly choosing from the same actions, without defining multiple states.The paper describes MAB as a simple and flexible alternative to other reinforcement-learning methods.Source: §4 · p. 7
  • The estimated value of each action approximates its expected reward using the average rewards received for that action over time.The paper expects this average to converge to the action’s expected value as the number of selections grows.Source: §4 · p. 7
  • An ε-greedy policy balances exploiting the highest-valued action with randomly exploring alternatives, since greedy-only selection can get stuck on a suboptimal action.The paper describes selecting the greedy action with probability 1 − ε and a random action with probability ε.Source: §4 · p. 7

5. MAB-ADC: Multiarmed Bandit for Adaptive Duty Cycle Selection

MAB-ADC maps duty-cycle choices to actions and aggregate throughput to reward, with decisions made every 40 ms. Its incremental update stores action values and selection counts to reduce memory needs.

  • The agent chooses duty-cycle actions from 0.1 to 0.8, receives aggregated LTE-U and Wi-Fi throughput as reward, and makes decisions every 40 ms of ABS time.A duty-cycle value of 0.4 reserves 40% of ABS time for LTE-U and 60% for Wi-Fi.Source: §5 · p. 8
  • The algorithm initializes ε at 0.3 and decays it by a factor of 1.015, reaching a 1.5% exploration probability after 200 random selections.This configuration provides more exploration initially and more exploitation after learning the best action.Source: §5 · p. 8
  • An incremental update stores only Q(a) and N(a), the action value and selection count, to minimize memory requirements.N(a) tracks how many times each action has been chosen.Source: §5 · p. 8

11 else

The passage gives the reward as the sum of LTE and Wi-Fi throughput.

  • The reward is written as LTE throughput plus Wi-Fi throughput.Source: §13 · p. 8

19 end loop

MAB-ADC can run independently in each LTE-U cell or centrally with coordination across cells. In centralized operation, the node distributes one duty-cycle value to all connected cells for each interval.

  • In independent mode, each LTE-U cell runs its own MAB-ADC and independently calculates ε, selects an action, and updates Q(a).Source: §19 · p. 8
  • In centralized coordinated mode, a central node exchanges information from all LTE-U cells and calculates ε, updates variables, and selects the next action.Source: §19 · p. 8
  • After choosing an action centrally, the node sends the same duty-cycle value to all connected LTE-U cells until the next ABS time.Source: §19 · p. 8

6. Performance Evaluation of MAB-ADC

The evaluation tests MAB duty-cycle adaptation in a short simulation with inverted traffic demands and a longer simulation with changing offered rates. Results vary with coordination and traffic balance, with independent MAB often providing more similar throughput for LTE and Wi-Fi.

6. Performance Evaluation of MAB-ADC

  • Nearly 15 Mbps (25% improvement): Q-Learning delivered the largest aggregated-throughput gain over the best fixed DC value of 0.5; independent and coordinated MAB gained 9 Mbps (15%) and 12 Mbps (20%).Figure 2 compares operator throughput across fixed DC values, Q-Learning, and the two MAB modes.Source: Figure 2 · p. 9, §6 · p. 9
  • Independent MAB improved LTE by over 30% and Wi-Fi by approximately 2%, while coordinated MAB and QL reached maximum system throughput but LTE suffered a 20% loss.The independent mode gives each base station its own DC value, whereas coordinated operation uses one shared value.Source: §6 · pp. 9–10
  • In the short simulation, rates invert after 20 s; coordinated MAB selected 0.7 for 70% LTE time, then 0.3 for 70% Wi-Fi time, with both values most frequent in the histogram.Simulator warm-up reduced the number of 0.7 selections but did not affect performance.Source: §6 · pp. 9–10, Figure 3 · p. 10
  • Approximately 3.5 Mbps: independent MAB exceeded the best fixed DC value of 0.6 in the 250-s dynamic-rate simulation; gains were smaller when offered rates were more equalized.The simulation repeated 100 times with offered rates changing at randomly selected intervals.Source: Figure 4 · p. 11, §6 · p. 11
  • More than 100% gain over fixed DC 0.6: independent MAB improved 10th-percentile user throughput, with less than 10% loss for Wi-Fi users.Figure 5 compares relative gain at the 10th percentile against the best fixed DC value of 0.6.Source: §6 · pp. 11–12, Figure 5 · p. 12
  • The CDF shows independent MAB gains for more than 50% of the worst LTE-U users with almost no Wi-Fi loss; coordinated MAB gains all Wi-Fi users at LTE-U users’ expense.The comparison uses fixed DC 0.6 as the reference and separates LTE-U and Wi-Fi user throughput.Source: Figure 6 · p. 12, §6 · pp. 12–13

7. Final Comments and Future Investigation

The study compares independent and coordinated MAB with Q-Learning for multicell LTE-U/Wi-Fi coexistence, and identifies independent MAB as the best solution across the simulations. It also reports user-throughput gains for the lowest-performing LTE-U users and outlines further coexistence and configuration studies.

7. Final Comments and Future Investigation

  • The study develops independent and coordinated MAB approaches for multicell LTE-U/Wi-Fi coexistence and compares them with Q-Learning.Source: §7 · p. 13
  • Across both simulation sets, independent MAB improves aggregated throughput and provides the best fairness among operators, with throughput gain similar to Q-Learning.The authors also report lower implementation complexity than Q-Learning.Source: §7 · p. 13
  • Independent MAB uses fewer parameters and simpler calculations than Q-Learning; without state definitions, it permits more actions without compromising convergence.Source: §7 · p. 13
  • Less than 10% throughput loss for Wi-Fi users accompanies over 100% gain for the worst-performing LTE-U users under independent MAB.Source: §7 · p. 13
  • Future work will analyze coexistence with LTE-LAA and NR-U, develop RL solutions for those technologies, and investigate dynamically configured Wi-Fi parameters.Source: §7 · p. 13

Related papers

Loading 2610.20222v1…