PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Deadline- and Energy-Aware Caching Algorithms in Dynamic Multi-UAV Edge Networks

Jian Kim, Ki-Il Kim

arXiv:2610.30768v1cs.LGcs.NI

TL;DR

UAV caching offers a way to maintain delivery when terrestrial infrastructure is unavailable, but existing approaches do not jointly address dynamic demand, content-specific deadlines, and uneven energy use. The paper formulates a joint caching problem and develops rule-based DE-UBAC and GRU-assisted GRU-DEPPO. In simulation, GRU-DEPPO has the lowest deadline-miss rate and total communication energy among the considered schemes.

  • Problem

    Existing caching approaches often focus on cache-hit performance or average delay without jointly accounting for dynamic heterogeneous demand, content-specific deadlines, and inter-UAV energy imbalance.

    Source: §1 · p. 2

  • Method

    The paper develops interpretable rule-based DE-UBAC and a GRU-assisted PPO policy using local and global temporal representations for adaptive cache placement and replacement.

    Source: §1 · p. 2

  • Results

    GRU-DEPPO achieves the lowest deadline-miss rate and total communication energy consumption among the considered schemes.

    Source: §6 · p. 16

  • Takeaways & Limitations

    The study concludes that GRU-DEPPO provides an adaptive caching approach for dynamic multi-UAV conditions.

    Source: §1 · p. 2

    The present study focuses on cache placement and replacement rather than jointly optimizing UAV positioning.

    Source: §6 · p. 16

Abstract

from arXiv · show

Content caching via unmanned aerial vehicles (UAVs) becomes one of the solutions to maintain content delivery when terrestrial infrastructure is unavailable or disrupted. However, existing caching approaches often focus primarily on cache-hit performance or average delivery delay, without jointly addressing spatially heterogeneous and time-varying demand, content-specific deadlines, and uneven communication-energy consumption across UAVs. To address these challenges, we formulate a dynamic deadline- and energy-aware cache placement and replacement problem that jointly considers deadline misses, UAV-side communication energy, and inter-UAV energy imbalance. To achieve this, we first develop deadline- and energy-aware UBAC (DE-UBAC), an interpretable hierarchical rule-based realization that incorporates deadline-aware prioritization, cooperative placement, energy-aware coordination, and cache-update cost. However, predefined priorities and thresholds limit its adaptability under dynamically coupled network conditions. We therefore propose GRU-assisted Deadline- and Energy-aware Proximal Policy Optimization (GRU-DEPPO), which combines local and global GRU-based temporal representations with a centralized PPO actor-critic for adaptive cache placement and replacement. Simulation results show that GRU-DEPPO achieves the lowest deadline-miss rate and total UAV-side communication energy among the considered schemes. Compared with DE-UBAC, GRU-DEPPO reduces the deadline-miss rate by approximately 61% and inter-UAV energy imbalance by approximately 40%.

1. Introduction

UAV caching can support edge delivery where terrestrial infrastructure is unavailable, but dynamic demand, content deadlines, and constrained UAV resources complicate cache decisions. The paper formulates a joint deadline- and energy-aware caching problem and presents both a rule-based design and a GRU-assisted learning policy.

1. Introduction

  • UAV-assisted edge computing can be deployed where fixed base stations or edge infrastructure are unavailable, insufficient, or slow to respond to abrupt demand changes.The paper identifies remote areas, temporarily crowded venues, and disaster-affected areas as relevant settings.Source: §1 · p. 1
  • UAV caching can reduce reliance on backhaul retrieval, which may increase delivery latency and communication energy while accelerating battery depletion.Local cache hits serve users directly, while neighboring UAVs can cooperatively provide content absent from the serving UAV’s cache.Source: §1 · pp. 1–2
  • Limited UAV storage, changing regional demand and mobility, heterogeneous deadlines, replacement costs, and uneven service loads make popularity-only or cache-hit-focused strategies insufficient.The resulting misses can add delay and energy and may cause requests to exceed their deadlines.Source: §1 · p. 2
  • The proposed formulation jointly addresses deadline-miss rate, request-delivery and cache-update communication energy, and inter-UAV energy imbalance under cache-capacity and cache-update feasibility constraints.It targets spatially heterogeneous and time-varying content demand with content-specific delivery deadlines.Source: §1 · p. 2
  • DE-UBAC is an interpretable hierarchical rule-based design that combines deadline-aware prioritization, persistent-miss feedback, cooperative placement, energy-aware coordination, and cache-update-cost-aware replacement.Its predefined rules, thresholds, and priorities limit adaptability under dynamic multi-UAV conditions.Source: §1 · p. 2
  • GRU-DEPPO combines shared local and global GRU temporal representations with a centralized PPO actor–critic to adapt cache placement and replacement to dynamically coupled network conditions.Its representations capture UAV-specific evolution and network-wide context; the policy learns feasible cache updates from deadline, demand, connectivity, and energy conditions.Source: §1 · p. 2

2. Related Work

Prior multi-UAV caching approaches incorporate selected demand, network, or coordination information, while learning-based work improves adaptation to changing conditions. The paper identifies a gap in jointly handling request-level deadlines, communication energy from delivery and cache updates, and inter-UAV energy imbalance.

2. Related Work

  • Traditional LFU and LRU policies use frequency- or recency-based statistics with low computational overhead, but their fixed rules omit dynamic conditions, cooperation, content deadlines, and communication costs.The cited comparison of representative approaches appears in Table 1.Source: §2 · p. 2
  • Optimization and placement approaches incorporate network information, localized requests, or predicted mobility, but do not jointly consider cache-update energy, inter-UAV energy imbalance, and content-specific deadline misses.Examples include joint caching and V2V power optimization, UAV deployment and cache placement, and proactive updates using request patterns and predicted mobility.Source: §2 · p. 3
  • Game-theoretic approaches coordinate caching and resource allocation but generally use predefined utility or preference models and do not learn temporal demand dynamics from historical observations.The cited approaches include coalition formation, a Stackelberg–Bertrand game, and Gibbs sampling with a matching game.Source: §2 · p. 3
  • Learning-based UAV caching studies improve adaptability but focus on objectives such as average delay, QoE, trajectory, or transmission power rather than jointly modeling deadline misses, communication energy, and inter-UAV energy imbalance.Existing schemes also do not directly encode recent UAV-local states for temporal dynamics in adaptive cache updates.Source: §2 · p. 3
  • Existing schemes typically address only subsets of local demand awareness, dynamic adaptation, cooperation, communication energy, and heterogeneous delivery requirements.This study instead makes periodic cache updates considering request-level deadline misses, energy from request delivery and cache updates, and inter-UAV energy imbalance under time-varying demand.Source: §2 · p. 3

3. System Model and Problem Formulation

The section specifies a multi-UAV network, its request, link, energy, and cache models, then formulates cache placement and replacement as a joint optimization problem. The problem includes content-specific deadline misses, UAV communication energy, and energy imbalance.

3. System Model and Problem Formulation

  • Users associate with the nearest UAV offering a feasible access link, and requests are served locally, cooperatively, or through the MBS according to a strict source-selection priority.The system models a failed local base station and a remote MBS supplying backhaul access; cache placement updates occur at predefined intervals, with mobility and local demand shifting after a random change point.Source: §3 · pp. 3–4

3.1. Transmission Model; 3.1.1. User–UAV Access Link

  • User–UAV access links model elevation-dependent LoS/NLoS attenuation, while link feasibility depends on SINR and achievable-rate thresholds.Achievable rates derive from bandwidth and SINR; the link model also determines transmission delay.Source: §3.1 · p. 4, §3.1.1 · p. 4

3.1.2. UAV–UAV Relay Link; 3.1.3. MBS–UAV Backhaul Link

  • UAV–UAV relays and MBS–UAV backhaul use distance-dependent path-loss models, and multi-hop relay-path delay and energy accumulate across hops.The MBS–UAV model adds an attenuation term; cache updates can use direct MBS transfer or relayed links.Source: §3.1.2 · p. 4, §3.1.3 · p. 4

3.2. Content Request Model; 3.2.1. Content Characteristics

  • Content items share an assumed identical size for tractability, while their types map to different delivery deadlines.The model partitions the catalog into type-specific subsets; requests can therefore have different delivery-time requirements.Source: §3.2.1 · p. 5

3.2.2. Dynamic Request Generation

  • Requests follow a Zipf-like global popularity distribution perturbed by group preferences, regions, and time-varying demand, with a later episode change point shifting local demand.The shift magnitude is controlled by β_shift.Source: §3.2.2 · p. 5

3.3. Energy Consumption Model

  • The energy model counts UAV-side communication for successful request delivery and cache-update transmissions, while excluding MBS and user communication energy.Request-delivery energy includes multi-hop relays; local cache evictions have negligible communication energy.Source: §3.3 · p. 5

3.4. Deadline Metric, Cache Constraints, and Problem Formulation; 3.4.1. Deadline-Miss Metric; 3.4.2. Cache Constraints

  • A delivered request misses its content-specific deadline when end-to-end delay exceeds that deadline, and target cache states must satisfy per-UAV item-capacity constraints.The previous cache remains available during required backhaul transfers; the target state activates after the update delay, and cache evictions are local operations.Source: §3.4.1 · p. 6, §3.4.2 · p. 6

3.4.3. Problem Formulation

  • The joint caching problem minimizes deadline misses, total communication energy, and inter-UAV energy imbalance, but is NP-hard and impractical to solve exactly at scale or repeatedly as conditions change.The formulation includes cache-capacity constraints and stochastic, time-varying network conditions.Source: §3.4 · p. 5, §3.4.3 · p. 6

4. Adaptive Deadline- and Energy-Aware Cache Algorithms

This section develops rule-based DE-UBAC and then formulates adaptive cache placement and replacement as an MDP handled by GRU-DEPPO. The approach uses local and network-wide temporal representations with a centralized PPO actor–critic.

4. Adaptive Deadline- and Energy-Aware Cache Algorithms

  • DE-UBAC extends UBAC with deadline-, cooperation-, and energy-aware rules, while GRU-DEPPO learns joint cache-update decisions from temporal representations.The section presents the overall scheme in Figure 2 and collects notation for DE-UBAC, the MDP, and GRU-DEPPO in Table 2.Source: §4 · p. 6, Figure 2 · p. 7, Table 2 · p. 7

4.1. Deadline- and Energy-Aware Rule-Based Caching

DE-UBAC extends UBAC with a hierarchical set of deadline-aware, cooperative, and energy-aware cache rules, including cache-update cost. Its predefined rules and thresholds are difficult to adapt as network conditions interact and change.

4.1. Deadline- and Energy-Aware Rule-Based Caching

  • DE-UBAC extends UBAC with deadline-, cooperation-, and energy-aware rules while retaining its user-behavior-aware counting and coordinated cache management.The design also retains handover-aware history updates and MBS-level cache coordination.Source: §4.1 · p. 7
  • Deadline-missed requests accumulate caching priority more rapidly, and persistent misses further raise priority for repeatedly unsatisfied UAV–content pairs.The additional weight for a missed request does not distinguish the magnitude of its deadline violation.Source: §4.1 · p. 8
  • When an existing cooperative copy can meet a content deadline, DE-UBAC avoids extra replication; otherwise, it considers feasible local or nearby placement.The target UAV is preferred when feasible; otherwise, a deadline-feasible nearby UAV is considered.Source: §4.1 · p. 8
  • Energy burden resolves competing placements when normalized priorities differ by less than threshold τ_H, without overriding a clearly stronger deadline-aware demand.If the target UAV cannot accommodate the content, a deadline-feasible alternative with a relatively lower energy burden is preferred.Source: §4.1 · p. 8
  • Cache replacement accounts for update energy, and the current cache configuration is retained when the replacement condition is not met.The update-energy penalty is controlled by λ_up.Source: §4.1 · p. 8
  • Predefined rules and thresholds limit scalability and adaptability as demand, mobility, connectivity, cooperative service, and UAV energy states evolve together.The paper therefore proceeds to formulate cache placement and replacement as an MDP so these coupled trade-offs can be learned through network interactions.Source: §4.1 · pp. 8–9

4.2. MDP Formulation

The caching process is modeled as an MDP at cache-decision epochs, with joint cache updates chosen from local and global network conditions. Its reward combines service outcomes and energy costs across each interval.

4.2. MDP Formulation

  • The MDP acts at cache-decision epochs spaced by Δ_c, with state space, action space, transition distribution, and reward function.The network continues evolving at each simulation step between decisions.Source: §4.2 · p. 9

4.2.1. State space

  • Each UAV’s local state includes cache contents, associated-user demand, association and connectivity, communication-energy state, and deadline pressure.Deadline pressure aggregates recent requests by content type, weighting them by deadline urgency; shorter deadlines receive larger urgency values.Source: §4.2.1 · p. 9

4.2.2. Action space

  • At each decision epoch, the centralized agent jointly selects content to remove from and add to each UAV cache, including a no-operation action.Feasible updates must satisfy the cache-update constraints, which the environment checks when decoding the sampled action.Source: §4.2.2 · p. 9

4.2.3. State–transition probability distribution

  • The transition distribution models the next global state after a joint cache update, accounting for execution, mobility, requests, wireless variation, service, and energy use.The resulting state summarizes conditions at the next cache-decision epoch.Source: §4.2.3 · p. 9

4.2.4. Reward function

  • The interval reward combines deadline-miss rate, service failure ratio, total communication energy, and inter-UAV energy imbalance.The service-failure penalty addresses dropped requests when no feasible delivery path exists and prevents artificially lowering deadline misses through excessive drops.Source: §4.2.4 · pp. 9–10
  • Reward normalization uses fixed 95th-percentile reference values from offline calibration rollouts, while the service-failure penalty applies above an admissible threshold.The normalization values are fixed before PPO training and are not updated using evaluation episodes.Source: §4.2.4 · p. 10
  • Because local and network-wide conditions evolve over time, GRU-DEPPO uses UAV-specific and network-wide temporal representations in a centralized policy for joint cache updates.The representations also provide network context for cooperative caching and inter-UAV energy balancing.Source: §4.2.4 · p. 10

4.3. GRU-based Temporal Representation

GRU-DEPPO encodes recent local observations and network-wide context into compact temporal representations for centralized cache decisions. The local GRU is pretrained and then fixed during PPO training, whereas the global GRU is trained jointly with the policy.

4.3. GRU-based Temporal Representation

  • Local demand, connectivity, cache, deadline, and energy observations are temporally correlated, so GRU-DEPPO encodes histories rather than relying only on the current state.Compact local and global GRUs also avoid sending all UAV histories directly to the centralized controller.Source: §4.3 · p. 10
  • A shared local GRU encodes each UAV’s recent observations into a separate representation, preserving UAV-specific temporal characteristics.The same GRU parameters are shared across UAVs.Source: §4.3 · p. 10
  • The global GRU processes recent network-wide context summarizing activity, service quality, service modes, communication energy, demand variation, and energy imbalance.The MBS maintains the compact context history.Source: §4.3 · p. 10
  • Before PPO training, the local GRU is pretrained on observation sequences with semantic summary targets, then frozen; the global GRU is trained jointly with the policy.After pretraining, only each UAV’s compact local representation is provided to the MBS.Source: §4.3 · p. 10

4.4. GRU-assisted PPO optimization

GRU-DEPPO combines local and network-wide temporal representations with centralized PPO to make feasible cache placement and replacement decisions. Its factorized actions are optimized from service, deadline, and communication-energy outcomes.

4.4. GRU-assisted PPO optimization

  • Local and global GRUs encode recent UAV observations and network context, which are fused for a centralized PPO actor–critic to select cache updates.The actor’s decisions are followed by service, deadline, and communication-energy outcomes used to update the policy.Source: §4.4 · p. 10

4.4.1. PPO input

  • The MBS input concatenates each UAV’s local temporal representation with the global representation used for cache-update decisions.The global representation is the temporal network context at epoch ξ.Source: §4.4.1 · p. 10

4.4.2. Factorized cache-update action

  • Each UAV’s cache-update action is factorized into an operation, a cache slot, and a content item, with a no-operation choice.A feasibility mask excludes invalid component choices before sampling, and the decoded update must satisfy cache-capacity and update-feasibility constraints.Source: §4.4.2 · p. 11

4.4.3. PPO-based policy optimization

  • PPO optimizes the policy from interval-level service, deadline, failure, and communication-energy outcomes using clipped updates and generalized advantage estimation.The critic uses return estimates, while entropy regularization encourages exploration.Source: §4.4.3 · p. 11

5. Results and discussion

The simulations compare GRU-DEPPO with LRU, DE-UBAC, and TLPPO under heterogeneous mobility and demand. GRU-DEPPO leads on overall deadline-miss rate and total communication energy, while results also examine stricter deadlines, service and energy patterns, adaptation, and GRU ablation.

5.1. Simulation setup

  • The evaluation compares LRU, DE-UBAC, and TLPPO with GRU-DEPPO, averaging reported performance over 30 independent random seeds.TLPPO is trained to maximize interval-level cache-hit performance; detailed simulation parameters are listed in Table 3.Source: §5 · p. 11, Table 3 · p. 12

5.1.1. User mobility and spatial locality

  • Users follow group hotspot-aware mobility, and both hotspot locations and group-conditioned content-demand distributions shift during an episode.Different user groups have distinct mobility and demand characteristics, producing changing local traffic patterns across UAVs.Source: §5.1 · p. 12, §5.1.1 · p. 12

5.2. Simulation results

  • GRU-DEPPO has the lowest deadline-miss rate, about 51% below TLPPO, the lowest total communication energy, and an energy-imbalance value of 0.801 versus 1.256 for TLPPO (about 36% lower).The comparison covers deadline misses, total communication energy, and inter-UAV energy imbalance.Source: §5.2 · pp. 12–13, Figure 3 · p. 13
  • Under tightened deadlines without retraining, GRU-DEPPO has 24.0% lower overall DMR than TLPPO, with its advantage concentrated on the strictest 2.0-s content deadline.Its DMR is somewhat higher for the more relaxed 3.0-s and 4.0-s deadlines; it also retains advantages in total energy and inter-UAV imbalance.Source: Figure 4 · p. 13, §5.2 · p. 14
  • Among RL schemes, TLPPO uses about 36.4% more cache-update energy than GRU-DEPPO, while GRU-DEPPO also uses two- and three-hop cooperative delivery.TLPPO’s cooperative service is concentrated on one-hop delivery; LRU and DE-UBAC have larger MBS-assisted service fractions than RL schemes.Source: §5.2 · p. 14, Figure 5 · p. 15
  • GRU-DEPPO’s DMR returns to its pre-shift level after about 38 s, compared with about 108 s for TLPPO, after mobility and demand distributions shift.The experiment fixes the change point at the simulation midpoint.Source: Figure 6 · p. 15, §5.2 · p. 15
  • The GRU-based model maintains higher mean episode reward from about episode 1,200 onward and outperforms raw-observation DEPPO across the considered objectives.DEPPO removes temporal encoding while retaining the PPO-based framework and joint objectives.Source: §5.2 · pp. 15–16, Figure 7 · p. 16

6. Conclusion

GRU-DEPPO uses local and global GRU-based temporal representations with a centralized PPO policy for caching in dynamic multi-UAV networks. It achieves the lowest deadline-miss rate and total communication energy among the considered schemes, while reducing inter-UAV energy imbalance; jointly optimizing UAV positioning and caching is future work.

6. Conclusion

  • GRU-DEPPO achieves the lowest deadline-miss rate and total communication energy among the considered schemes while substantially reducing inter-UAV energy imbalance.The framework uses local and global GRU-based temporal representations with a centralized PPO policy.Source: §6 · p. 16
  • Future work will jointly optimize UAV positioning and content caching, beyond the present focus on cache placement and replacement.The authors suggest that adapting UAV positions may offer additional opportunities to improve connectivity, content accessibility, and communication-energy efficiency.Source: §6 · p. 16

A. Training Parameters

The simulator and GRU-DEPPO agent were implemented in Python with PyTorch and Stable-Baselines3/SB3-Contrib, and experiments ran on an NVIDIA GeForce RTX 4090 GPU.

A. Training Parameters

  • The simulator and GRU-DEPPO agent used Python, PyTorch, and Stable-Baselines3/SB3-Contrib; experiments ran on an NVIDIA GeForce RTX 4090 GPU.The listed PPO mini-batch size, optimization epochs, and local and global GRU hidden dimensions are denoted B_mb, K_PPO, d_loc, and d_global.Source: A. Training Parameters · p. 17

B. GRU Input Composition

GRU-DEPPO represents UAV-local operating conditions and network-wide context using separate GRU inputs. The network-wide context combines activity, service, mode, energy, demand-variation, and energy-imbalance information before global GRU encoding.

B. GRU Input Composition

  • The local GRU input combines UAV operating features with cache-type, request-type, and deadline-pressure statistics available up to the current cache-decision epoch.The UAV feature vector is 20-dimensional; each type-level vector contains |T| entries.Source: B. GRU Input Composition · p. 17
  • The network-wide context includes network activity, service quality, service-mode ratios, communication energy, demand variation, and inter-UAV energy imbalance.The MBS constructs this vector from the current network state and observations from the most recent cache-decision interval.Source: B. GRU Input Composition · p. 17, Table 6 · p. 17
  • Before global GRU encoding, each network-context vector is projected into a d_tok-dimensional token space, and the resulting token sequence is encoded by the global GRU.Source: B. GRU Input Composition · p. 17
Loading 2610.30768v1…