PaperlayerView exact revision on arXiv

Source-linked AI summary

Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning

Qiong Wu, Yu Zhao, Qiang Fan, Pingyi Fan, Jiangzhou Wang, Cui Zhang

arXiv:2208.01219v1cs.DCcs.LG

TL;DR

Vehicular edge caching must handle mobility, privacy-sensitive user data, and limited RSU capacity. CAFR combines asynchronous federated learning, content prediction, and dueling-DQN cooperative caching; simulations report that it outperforms other schemes.

  • Problem

    Vehicle mobility, privacy-sensitive user data, and limited RSU capacity complicate accurate popularity prediction and cooperative content caching.

  • Method

    CAFR uses asynchronous federated learning for global-model training, predicts popular contents from that model, and applies dueling DQN for cooperative caching.

  • Results

    CAFR outperforms other caching schemes in simulations.

  • Takeaways & Limitations

    The scheme combines privacy-preserving model training with mobility-aware cooperative cache placement to reduce content transmission delay and improve cache hit ratio.

Abstract

from arXiv · show

The vehicular edge computing (VEC) can cache contents in different RSUs at the network edge to support the real-time vehicular applications. In VEC, owing to the high-mobility characteristics of vehicles, it is necessary to cache the user data in advance and learn the most popular and interesting contents for vehicular users. Since user data usually contains privacy information, users are reluctant to share their data with others. To solve this problem, traditional federated learning (FL) needs to update the global model synchronously through aggregating all users' local models to protect users' privacy. However, vehicles may frequently drive out of the coverage area of the VEC before they achieve their local model trainings and thus the local models cannot be uploaded as expected, which would reduce the accuracy of the global model. In addition, the caching capacity of the local RSU is limited and the popular contents are diverse, thus the size of the predicted popular contents usually exceeds the cache capacity of the local RSU. Hence, the VEC should cache the predicted popular contents in different RSUs while considering the content transmission delay. In this paper, we consider the mobility of vehicles and propose a cooperative Caching scheme in the VEC based on Asynchronous Federated and deep Reinforcement learning (CAFR). We first consider the mobility of vehicles and propose an asynchronous FL algorithm to obtain an accurate global model, and then propose an algorithm to predict the popular contents based on the global model. In addition, we consider the mobility of vehicles and propose a deep reinforcement learning algorithm to obtain the optimal cooperative caching location for the predicted popular contents in order to optimize the content transmission delay. Extensive experimental results have demonstrated that the CAFR scheme outperforms other baseline caching schemes.

I. INTRODUCTION

The paper addresses mobility, privacy, and limited RSU capacity in vehicular edge caching by combining asynchronous federated learning with deep reinforcement learning for cooperative caching.

  • High vehicle mobility can make previously requested contents outdated, motivating advance prediction of popular contents.
  • Users’ privacy concerns make direct data sharing difficult, while vehicles may leave VEC coverage before uploading local models.
  • The proposed asynchronous FL algorithm uses vehicle mobility to improve global-model accuracy and predict popular contents.
  • Because predicted contents can exceed local RSU capacity, cooperative caching across RSUs is optimized for content transmission delay.
  • CAFR combines asynchronous FL, autoencoder-based content prediction, and a dueling DQN framework for cooperative cache placement.
  • The dueling DQN determines cooperative caching decisions intended to minimize content transmission delay.

II. RELATED WORK

Prior work studies vehicular caching, cooperative placement, mobility prediction, privacy, and DRL, but the paper identifies a gap in jointly considering vehicle mobility and user privacy.

  • Existing vehicular caching studies address security, response time, digital twins, offloading, and resource management.
  • Several cooperative VEC schemes optimize access cost, transmission delay, content placement, or caching resources using optimization and learning methods.
  • Other studies incorporate mobility prediction, content popularity, social similarity, vehicle-aided caching, or cache replacement.
  • DRL-based approaches have been used for intelligent caching, request searching, and cooperative resource utilization.
  • The paper states that prior work had not simultaneously considered vehicle mobility and VU privacy in cooperative VEC caching.

III. SYSTEM MODEL

The system model describes a three-tier urban VEC with an MBS, RSUs, and vehicles, where mobility and limited RSU storage determine how requested contents are fetched.

  • A. System Scenario: The modeled VEC has an MBS at the top tier, RSUs in the middle tier, and vehicles at the bottom tier.
  • A. System Scenario: Vehicles store local historical user data containing personal information, possible requests, and content ratings.
  • A. System Scenario: The MBS is assumed to cache all available contents, whereas each RSU can store only part of them.
  • A. System Scenario: A requested content is fetched from the local RSU, neighboring RSU, or MBS depending on cache availability.
  • B. Mobility Model of Vehicles: Vehicles move in one direction, arrive according to a Poisson process, and retain mobility characteristics within each round.

C. Communication Model

The communication model accounts for wired RSU-to-RSU links and mobility-dependent wireless V2R/V2B rates, while the caching procedure combines asynchronous FL, prediction, and DRL.

  • C. Communication Model: RSUs communicate with vehicles through V2R links, while the MBS communicates through V2B links.
  • C. Communication Model: Vehicle mobility changes distances and channel impairments across rounds, producing different transmission rates.
  • C. Communication Model: The transmission-rate model uses bandwidth, transmit power, channel gain, and noise power for local-RSU and MBS links.
  • The proposed procedure first obtains an accurate model with asynchronous FL, then predicts popular contents and selects cooperative caching through DRL.
  • 1) Select Vehicles: Vehicles are selected for asynchronous FL when their predicted staying time is sufficient to complete training and inference.

2) Download Model:

Vehicles download the global model from the local RSU, initialize local training with it, and iteratively optimize an autoencoder-based local model. The procedure includes regularization and delayed-gradient handling for mobile participants.

  • Each participating vehicle initializes its local model with the global model downloaded from the local RSU.
  • Vehicles train an autoencoder consisting of an encoder and decoder on sampled local training data.The encoder extracts hidden features and the decoder reconstructs the input.
  • Vehicles repeatedly sample training data, calculate local gradients, and update their local models until the iteration threshold is reached.
  • The local loss is regularized to reduce deviation between the vehicle’s local model and the global model.ρ is the regularization parameter.
  • Delayed local gradients from stragglers are aggregated into the current round to address unsuccessful uploads caused by delayed training.Stragglers are vehicles whose previous-round uploads failed because training was delayed.

4) Upload Model:

After local training, vehicles upload updated models to the RSU, which performs weighted asynchronous aggregation. The aggregation weights account for mobility-related participation time and content transmission delay.

  • Each vehicle uploads its updated local model to the local RSU after completing local training.
  • The local RSU updates the global model when an upload is successful and drops an unsuccessful local model.
  • Asynchronous aggregation weights are calculated using local data size, vehicle position, and transmission-related factors.The weight χ_i considers traversed distance within RSU coverage and content transmission delay.
  • A vehicle with greater traversed distance may receive a larger aggregation weight because it may have more time available for training.
  • The completed asynchronous FL round produces an updated global model, which the RSU sends to vehicles for popular-content prediction.

B. Popular Content Prediction

The popular-content prediction procedure reconstructs sparse ratings with the learned autoencoder, identifies similar users, and selects contents with the highest popularity among neighboring-user preferences.

  • The algorithm uses four steps to predict popular contents from vehicles’ rating and personal-information data.
  • Because a zero rating may indicate either disinterest or no request, treating every zero as disinterest can bias prediction.
  • The learned autoencoder reconstructs each vehicle’s rating matrix to expose hidden preference information.
  • For each active user, the algorithm selects the K most similar users using cosine similarity and uses their ratings as neighboring preferences.
  • Content popularity is counted across neighboring users’ nonzero ratings, and the Fc contents with the largest counts are selected as predicted interested contents.

4) Popular Contents:

The cooperative caching stage distributes predicted popular contents across local and neighboring RSUs, using deep reinforcement learning to minimize transmission delay. Its state, actions, and rewards represent cache placement and access costs across possible serving locations.

  • The local RSU collects vehicles’ predicted interested contents and selects the Fc contents with the largest popularity as the popular-content set.
  • Because each RSU can store fewer than Fc contents, cooperative caching determines where the predicted popular contents should be placed.
  • The DRL framework seeks a cooperative caching policy that minimizes content transmission delay under powerful RSU computation.
  • The state is the local RSU cache sorted by predicted popularity, while the binary action determines whether local contents are relocated.
  • When relocation is selected, lower-popularity local contents are exchanged with sampled uncached popular contents, while the neighboring RSU caches complementary contents.
  • The reward is designed from transmission delays when requested contents are served by the local RSU, neighboring RSU, or MBS.These serving locations have distinct transmission paths and rates.

2) DRL Algorithm:

The cooperative caching algorithm uses a dueling DQN to learn caching actions from observed states, rewards, and transitions. It separates state value from action advantage and trains prediction and target networks with replay-buffer updates.

  • Dueling DQN architecture: The dueling DQN decomposes each Q-value into a state-value function V and an action-advantage function A for the available actions.This separation is used to address cases where partial actions have no relevant effects on subsequent states.
  • Dueling DQN architecture: The algorithm contains a prediction network, a target network, and a replay buffer, with each network comprising feature, state-value, and advantage layers.The prediction network evaluates current state-action values, while the target network generates target Q-values.
  • Cooperative caching: Across episodes, RSUs randomly initialize cached contents, iteratively optimize cooperative caching, and finally cache popular contents according to the learned policy.After training terminates, the local and neighboring RSUs apply the optimal cooperative caching policy before vehicles fetch contents.
  • State-action process: At each slot, the local RSU observes the state, computes Q-values, selects an action with ε-greedy policy, receives a reward, and transitions to the next state.The neighboring RSU samples uncached popular contents, and the resulting transition is stored in the replay buffer.
  • Network training: The replay buffer stores state-action-reward-next-state tuples, and minibatches are used to calculate target Q-values, loss gradients, and prediction-network updates.The target network is periodically synchronized with the prediction network after a specified number of slots.

V. SIMULATION AND ANALYTICAL RESULTS

The experiments simulate a VEC environment using 3GPP C-V2X communications and the MovieLens 1M dataset. Vehicle-local data are divided into training and testing portions for evaluating the proposed scheme.

  • Experimental setup: The experiments simulate a VEC environment on an urban road using the 3GPP C-V2X architecture for vehicle-to-RSU and vehicle-to-MBS communications.Simulation parameters are listed in Table I.
  • Dataset: MovieLens 1M supplies the real-world ratings dataset used in the experiments, with each vehicle receiving randomly assigned local data.The dataset also contains vehicle-user information including ID number, gender, age, and postcode.
  • Dataset: Each vehicle randomly uses 99.8% of its local data for training and 0.2% for testing.For each round, vehicles randomly sample part of the testing-set movies as requested contents.

B. Performance Evaluation

Performance is evaluated using cache hit ratio and content transmission delay across cache capacities, vehicle densities, rounds, and DRL episodes. CAFR generally achieves stronger caching and lower delay than the compared schemes, while asynchronous aggregation substantially reduces training time relative to FedAVG.

  • Cache capacity: Cache hit ratio increases with cache capacity, and CAFR outperforms random, Thompson sampling, and c-ε-greedy schemes.The paper attributes CAFR’s advantage over c-ε-greedy to learning hidden features for more accurate popularity prediction.
  • Cache capacity: Content transmission delay decreases as cache capacity increases, with CAFR achieving lower delay than the other evaluated schemes.Higher cache capacity increases the probability that vehicles fetch contents from local or neighboring RSUs.
  • Vehicle density: As vehicle density increases, CAFR’s cache hit rate increases and its content transmission delay decreases.The paper relates this trend to more training data and more vehicles fetching contents directly from the local RSU.
  • DRL evaluation: During the first ten DRL episodes, cache hit rate increases and content transmission delay decreases before both converge around episode 10.The reported convergence corresponds to the local RSU learning an effective cooperative caching policy.
  • DRL evaluation: CAFR has higher cache hit ratio and lower content transmission delay than CAFR without DRL under different cache capacities.The paper attributes these differences to DRL selecting more suitable cooperative caching locations for predicted popular contents.

VI. CONCLUSIONS

CAFR combines mobility-aware asynchronous federated learning with deep reinforcement learning for cooperative caching. It predicts popular contents, selects cooperative caching locations, reduces transmission delay, and outperforms baseline caching schemes.

  • VI. CONCLUSIONS: CAFR combines asynchronous federated learning for accurate popular-content prediction with cooperative caching based on a dueling DQN algorithm.The scheme first obtains an accurate global model and predicts popular contents, then optimizes their cooperative placement.
  • VI. CONCLUSIONS: Simulation results demonstrate that CAFR outperforms other baseline caching schemes.The paper reports this conclusion from its theoretical analysis and simulation results.
  • VI. CONCLUSIONS: CAFR learns useful hidden features from vehicles’ local data and can predict accurate popular contents.The conclusions associate local-data learning with hidden-feature extraction and accurate content prediction.
  • VI. CONCLUSIONS: CAFR reduces training time by aggregating one vehicle’s local model per round while selecting and weighting vehicles using mobility characteristics.Vehicle positions and velocities are considered during vehicle selection and local-model aggregation.
  • VI. CONCLUSIONS: The DRL policy places suitable popular contents in local and neighboring RSUs to reduce content transmission delay.Cooperative placement is determined according to the predicted popular contents and the caching objective.
Loading 2208.01219v1…