PaperlayerView exact revision on arXiv

Source-linked AI summary

Update for Decisions, Not Freshness: Goal-Oriented Status Updating and Selective Offloading at the Network Edge

Jianpeng Qi, Qiyang Zhang, Chao Liu, Jing Sun, Yimei Liu, Yanwei Yu, Yingjie Wang, Wei Ni

arXiv:2609.01082v1cs.DCcs.MAcs.NI

TL;DR

The paper addresses edge offloading when the EN sees remote service state only through stale, intermittent updates and freshness alone does not value updates by their decision impact. CoSMO jointly learns semantic status updating and hierarchical task handling through realized task utility, reporting higher on-time completion and capacity-aware decision accuracy across evaluated workloads. Its evaluation remains bounded to a controlled single-EN/single-SN topology.

  • Problem

    Stale remote information, partial observability, bidirectional feedback, and mismatched event horizons make status updating and task control an asynchronous coupled decision problem.

  • Method

    CoSMO uses compact SN status semantics, recurrent semi-Markov send/no-send timing at the SN, and task-terminal hierarchical gate–route value learning at the EN.

  • Results

    CoSMO achieved the highest on-time completion rate and maintained high capacity-aware decision accuracy under overload across deterministic and Poisson load settings.

  • Takeaways & Limitations

    The learned policies trade update cost against workload, using lower cost at light load and more updates under overload while outperforming freshness-driven and fixed-rule policies on task metrics.

  • Takeaways & Limitations

    Evaluation is limited to a controlled single-EN/single-SN topology with fixed-rate backhaul and fixed modulation and coding, so transfer across topologies is untested.

Abstract

from arXiv · show

In an edge--cloud collaborative edge-computing environment, an edge node (EN) must decide whether each user task should be executed locally, forwarded to a remote service (or cloud) node (SN), or rejected. The EN observes its local state directly but receives the SN state only through an intermittently refreshed cache. Status updating and task control therefore form an asynchronous closed loop under partial observability. Freshness-driven schemes, including those based on Age of Information (AoI), do not directly value an update by its effect on subsequent task decisions. We propose CoSMO (Co-design of Semantic-state Management and Offloading), a cooperative event-driven reinforcement learning (RL) framework that coordinates semantic status management and selective offloading through realized task utility. CoSMO learns a compact representation of the heterogeneous SN service state. At the SN, a recurrent semi-Markov double deep Q-network (Double DQN) agent jointly selects send/no-send and the next decision interval. At the EN, a task-terminal off-policy value-learning agent makes hierarchical gate--route decisions from local observations and stale remote semantics. The agents maintain separate observations and value targets but share the same realized task-utility stream, without centralized execution. Across the evaluated workload families, CoSMO's reported relative improvement in on-time completion rate over the best-performing competing method averages 18.6%--21.2%. For capacity-aware decision accuracy across the three strict-overload points, the corresponding reported gains average 17.6%--$17.9%.

I. INTRODUCTION

The paper frames stale remote-state offloading and status updating as one asynchronous closed-loop control problem. CoSMO jointly learns semantic status transmission and overload-aware task handling using downstream task utility.

  • The dispatcher chooses local execution, remote offloading, or rejection while observing local state directly and remote state through capacity-limited updates.
  • Freshness or state deviation alone may misvalue updates because the same cached information can yield the same action, while a small state change can alter the decision.
  • Partial observability, bidirectional feedback, and different event horizons couple status updates with task decisions and complicate joint learning.
  • CoSMO uses a compact semantic status abstraction, a recurrent semi-Markov Double DQN at the SN, and hierarchical gate–route value learning at the EN.
  • The framework evaluates information by downstream task utility and coordinates status abstraction, update timing, task acceptance, and execution routing in one asynchronous process.

1) Network Architecture

The system is a three-tier MEC architecture in which an EN dispatches UE tasks to local execution, an SN, or rejection. Wireless access, queueing, computation, and backhaul conditions determine realized completion delay.

  • Multiple UEs send tasks to an EN, which processes each task locally, offloads it to an SN, or rejects it; the EN–SN backhaul carries tasks, results, and status updates.
  • Intermittent SN updates are timestamped in an EN cache, so task decisions use local EN information and potentially stale remote status.
  • Accepted tasks follow FCFS service, with local execution or SN offloading determining the applicable execution path and delays.
  • An accepted task succeeds if and only if its realized end-to-end delay T_k is no greater than its deadline D_k.
  • Mobility is represented through effective uplink throughput, access-state trend, and residual connection time, capturing access capability, short-term change, and connection sustainability.

B. Decision-Oriented Status and Observations

CoSMO abstracts the SN service state into a compact semantic cached at the EN and maintains separate, locally available observations for the two agents. These observations include cache age, link statistics, task history, and local service information.

  • 1) SN Status Abstraction: The SN status-abstraction mapping f_θ produces a two-dimensional semantic z_t that the SN transmits and the EN caches for later task decisions.
  • Online execution preserves partial observability: the SN and EN use only their locally available observations rather than a centralized state.
  • 2) Asymmetric EN/SN-Side Observations: At each SN decision epoch, the observation includes the current semantic, the acknowledged cached semantic, its AoI, completed offloaded-task history, and update-link statistics.
  • The SN’s completed-task history contains only offloaded tasks whose terminal outcomes are known, reflecting delayed feedback.
  • 2) Asymmetric EN/SN-Side Observations: At task arrival, the EN observation combines source mobility, local EN service state, task features, cached SN semantics and age, and EN–SN link statistics.

C. Closed-Loop Event-Driven Decision Process

The system models task arrivals and SN update epochs as distinct event streams in an asynchronous semi-MDP. It jointly optimizes task decisions, semantic updates, and communication cost through realized task utility.

  • Event streams: Task arrivals trigger EN decisions, while SN epochs trigger semantic transmission and interval-selection decisions.The two event clocks are generally nonuniform, so the formulation does not impose a common artificial clock.
  • EN task control: The EN chooses local execution, SN offloading, or active rejection for each arriving task.The action indices are 0 for local execution, 1 for offloading, and 2 for active rejection.
  • SN status control: The SN jointly chooses whether to transmit the current semantic and how long to wait before the next decision epoch.A no-send decision preserves the EN’s cached semantic during the selected interval, while a send decision updates the cache after reception.
  • Coupling across events: A transmitted update can affect all EN task decisions in the subsequent interval, including when the mean update interval is comparable to or shorter than task inter-arrival time.The number of task decisions in an interval may be small, but the same model and training pipeline remain applicable.
  • Outcome utility: Task outcomes distinguish successful acceptance, failed acceptance, correct rejection, and erroneous rejection under resource and deadline feasibility constraints.These labels penalize both blind acceptance under overload and unnecessary rejection.
  • Joint objective: The objective maximizes long-term average net utility by balancing outcome-dependent task utility against semantic-update communication cost.Update cost is incurred only when the SN transmits, and one update may influence multiple subsequent tasks.

IV. COSMO: COOPERATIVE SEMANTIC-AWARE UPDATING AND OFFLOADING

CoSMO combines a compact semantic representation of SN state with separate SN updating and EN forwarding agents. Utility-guided updates coordinate the agents while preserving decentralized observations, value targets, and execution.

  • CoSMO architecture: The SN branch processes semantic and temporal information to produce update decisions, while the EN branch separately encodes local context and cached remote context before fusion.The dual-branch design keeps local task/service features distinct from delayed remote-semantic and link features.
  • Utility coordination: Utility-guided policy updates coordinate separate SN and EN replay and loss branches through their common realized task-utility stream.The agents retain separate observations and value targets, and the encoder remains frozen during coordinated training.
  • Raw-State Semantic Encoder: A frozen semantic encoder maps the raw SN service state into a compact representation used by both updating and offloading policies.The lightweight MLP representation is transmitted to and cached at the EN and also supplied to the SN scheduler.
  • Semantic pretraining: The encoder is pretrained with interval-conditioned forward-dynamics prediction of normalized SN state increments rather than raw-state reconstruction.The auxiliary prediction head uses the selected update interval and is discarded after pretraining.

C. Temporal Semantic Decision Network

The temporal semantic decision network lets the SN select send/no-send together with a variable decision interval. A recurrent dueling Double DQN evaluates interval-level task utility and update cost using semi-Markov returns.

  • Action space: The candidate intervals satisfy gmin ≤ g(1) < g(2) < · · · < g(L) ≤ gmax.These heterogeneous intervals determine the next SN decision epoch.
  • Action space: The SN scheduler chooses a paired action consisting of a transmit indicator and one candidate waiting interval.The discrete action set includes both send and no-send variants for each candidate interval when no-send is enabled.
  • Temporal observation: The scheduler forms temporal features from SN semantics, cached semantics, cache age, and recent task and link statistics before recurrent processing.A feature adapter and GRU capture temporal dependencies across SN decision epochs.
  • Value learning: A dueling Double DQN combines state-value and action-advantage branches to estimate values for the SN’s paired update actions.Double DQN separates next-action selection from evaluation to reduce Q-value overestimation.
  • Semi-Markov return: The SN interval reward sums task utilities over the following interval and subtracts semantic-update cost charged only for send actions.With no arrivals, the task-utility sum is zero; the main objective uses βSN = 1.
  • Semi-Markov return: Semi-Markov discounting assigns greater discounts to longer waiting intervals, aligning returns with heterogeneous elapsed times.The implementation uses a reference interval equal to the shortest candidate interval.

D. EN Selective-Offloading Agent

The EN agent makes selective forwarding decisions from separately encoded local observations and cached remote semantics. Its hierarchical network first gates acceptance, then routes accepted tasks locally or to the SN.

  • Selective forwarding: The EN chooses among local execution, SN offloading, and active rejection at each task arrival.Local context is current and directly observed, whereas the remote semantic is compact and delayed.
  • State encoding: A dual-branch encoder separately processes local EN context and cached SN context before feature fusion.The branches reflect different reliability, dimensions, timescales, and statistical meanings of the two information sources.
  • Hierarchical decision: The hierarchical forwarding network first determines whether to accept a task and then selects local execution or SN offloading for accepted tasks.This gate–route structure makes rejection explicit under overload.

1) Dual-Branch State Encoding

The EN constructs separate local and remote input branches, combining directly observed task and service information with cached SN semantics and link characteristics. Two MLP branches encode these inputs before feature fusion.

  • 1) Dual-Branch State Encoding: Local inputs collect directly observed access, task, and EN service state at task arrival.
  • 1) Dual-Branch State Encoding: Remote inputs contain cached SN status semantics and inter-node link information.
  • 1) Dual-Branch State Encoding: The EN-to-SN link representation includes short-term mean rate, rate standard deviation, and rate trend.
  • 1) Dual-Branch State Encoding: Separate two-layer MLPs encode the local and remote inputs into branch features.
  • 1) Dual-Branch State Encoding: The branch features are concatenated and transformed with fusion parameters to form the EN decision representation.

2) Hierarchical Forwarding Decision Network

The EN uses a hierarchical gate–route network to choose rejection or acceptance first, then local execution or SN offloading for accepted tasks. It learns expected terminal utility from delayed task outcomes while preserving consistency with stale remote observations.

  • 2) Hierarchical Forwarding Decision Network: The forwarding network separates accept/reject gating from route selection between local execution and SN offloading.
  • 2) Hierarchical Forwarding Decision Network: Gate and route heads use dueling value–advantage decompositions to score hierarchical decisions.
  • 2) Hierarchical Forwarding Decision Network: Accepted action values combine the gate’s accept value with max-normalized route scores, while rejection retains the gate’s reject value.
  • 2) Hierarchical Forwarding Decision Network: The gate controls the top-level comparison, whereas the route head determines the conditional local-versus-offload preference.
  • 2) Hierarchical Forwarding Decision Network: The decoder makes the accept/reject decision first and selects a route only for accepted tasks; auxiliary losses affect training but not execution.
  • 2) Hierarchical Forwarding Decision Network: The EN learns terminal utility from task-level feedback, allowing overload-aware rejection of tasks unlikely to meet deadlines and avoidable transmission, queueing, and compute costs.

E. Utility-Coordinated Training Algorithm

Utility-coordinated training couples separate SN and EN learners through realized task utility while preserving decentralized execution. Event-driven updates finalize EN transitions at task outcomes and train the SN across update intervals.

  • E. Utility-Coordinated Training Algorithm: The agents use separate observations, replay buffers, and value losses without a centralized critic.
  • E. Utility-Coordinated Training Algorithm: At each SN event, the scheduler selects semantic transmission and the next decision interval, then schedules the subsequent event.
  • E. Utility-Coordinated Training Algorithm: The EN stores a transition prefix when utility is unavailable and completes it with realized utility after the task reaches a terminal outcome.
  • E. Utility-Coordinated Training Algorithm: The EN and SN parameters are updated from their respective replay buffers using task-terminal and semi-Markov Double DQN losses, with the encoder frozen.
  • E. Utility-Coordinated Training Algorithm: The SN scheduler and EN offloading agent share a realized task-utility stream while retaining agent-specific value-learning mechanisms.
  • E. Utility-Coordinated Training Algorithm: During decentralized execution, each agent uses only locally available observations, and delivered SN semantics affect the EN after reaching its cache.

F. Complexity Analysis

The evaluation analyzes training and inference costs in a controlled simulated MEC environment while testing task selection, completion, and semantic-update overhead. Execution cost is constant per event for fixed network widths and bounded action sets, but backhaul variability is outside the tested scope.

  • F. Complexity Analysis: Training cost includes EN and SN optimizer updates, batch sizes, sampled sequence length, and optional oracle-derived auxiliary supervision.
  • F. Complexity Analysis: With bounded EN and SN queues, the auxiliary supervision cost is constant with respect to episode length.
  • F. Complexity Analysis: Execution removes replay sampling, oracle labeling, target-network evaluation, and backpropagation, leaving per-decision inference costs.
  • F. Complexity Analysis: Inference time is constant per decision event for fixed network widths and bounded action sets.
  • F. Complexity Analysis: The experiments test load-aware decisions, on-time completion, and semantic-update overhead under stale remote-state information.
  • F. Complexity Analysis: The controlled single-EN/single-SN setup fixes topology, access conditions, and backhaul rates to isolate semantic updating, acceptance, and routing.
  • F. Complexity Analysis: The experiments do not assess adaptation to backhaul-rate variability or short-term rate trends.

3) Baselines

The evaluation compares COSMO with fixed, freshness-driven, and rejection-capable baselines across smooth and bursty workloads spanning underload to overload. COSMO maintains higher capacity-aware decision accuracy across the evaluated load sweep.

  • Baselines: COSMO is compared with six baselines, including fixed execution policies, AoI/AoV/AoCI freshness policies, and a hierarchical-decision ablation.The baselines use identical workloads and evaluation protocols.
  • Workloads: The workload uses deterministic and Poisson arrivals at the same mean rates, testing smooth traffic and random bursts across offered loads ρ = 0.54–1.45.The sweep covers underload, the capacity boundary, and overload.
  • Metrics: Evaluation reports system success rate, capacity-aware decision accuracy, and update-cost rate using greedy learned actions and repeated ns-3 runs.Metrics average three runs, with five evaluation episodes per run and two-sided 95% confidence intervals.
  • Capacity-Aware Decision Accuracy: At strict overload, COSMO’s decision accuracy remains 0.930–0.941 for deterministic arrivals and 0.901–0.931 for Poisson arrivals.These values span λ ∈ {240, 280, 320} tasks/s.
  • Capacity-Aware Decision Accuracy: 17.6% and 17.9% are COSMO’s average relative gains over the best competing method across the three strict-overload points under deterministic and Poisson arrivals.The per-load gain ranges are 10.7%–25.5% for deterministic arrivals and 8.3%–26.7% for Poisson arrivals.
  • Capacity-Aware Decision Accuracy: 27.7% is COSMO’s relative improvement over AoCI online at the Poisson capacity boundary, where accuracies are 0.880 and 0.690, respectively.At 320 tasks/s, the three Poisson rejection-capable baselines range from 0.813 to 0.832.

2) On-Time Task Completion

COSMO achieves the highest on-time completion rate across evaluated loads and arrival processes, while remaining near the capacity-limited upper bound under overload. It trades higher update expenditure for improved task outcomes, with hierarchical gating providing larger gains in deep overload.

  • On-Time Task Completion: COSMO attains the highest system success rate at every evaluated load under deterministic and Poisson arrivals.Success rate is the fraction of measured tasks completed before their deadlines.
  • On-Time Task Completion: COSMO achieves 0.620 success at 320 tasks/s under Poisson arrivals and remains within 89%–92% of the capacity-limited upper bound under Poisson overload.Under deterministic overload, it reaches 93%–94% of the corresponding bound.
  • On-Time Task Completion: 21.2% average improvement in Poisson on-time completion over the best-performing competing method across all seven loads, versus 18.6% under deterministic arrivals.At 320 tasks/s under Poisson arrivals, COSMO reaches 0.620 versus 0.498 for AoV gap, a 24.5% improvement.
  • Status-Update Efficiency: At 320 tasks/s, COSMO uses 46.71 cost units/s under deterministic arrivals and 46.96 under Poisson arrivals, exceeding AoI threshold but outperforming higher-cost alternatives.Its update cost is 89.1% lower than AoV gap under either arrival process and 14.1%–15.5% lower than AoCI online.
  • Training-Reward Stabilization: Median reward stabilization occurs within 20 episodes in every evaluated setting, with longer stabilization under overload and Poisson arrivals.Median t90 is 6–7 episodes through 220 tasks/s, rising to 20 episodes at 320 tasks/s under Poisson arrivals.
  • Hierarchical Decision Ablation: At 320 tasks/s, hierarchical COSMO improves success rate by 13.4% under deterministic arrivals and 8.8% under Poisson arrivals over COSMO (w/o Gate).The ablation also reports higher decision accuracy and lower false-positive rate with comparable mean update cost.
Loading 2609.01082v1…