PaperlayerView exact revision on arXiv

Source-linked AI summary

Update for Decisions, Not Freshness: Goal-Oriented Status Updating and Selective Offloading at the Network Edge

Jianpeng Qi, Qiyang Zhang, Chao Liu, Jing Sun, Yimei Liu, Yanwei Yu, Yingjie Wang, Wei Ni

arXiv:2609.01082v1cs.DCcs.MAcs.NI

TL;DR

The paper addresses edge offloading when the EN sees remote service state only through stale, intermittent updates and freshness alone does not value updates by their decision impact. CoSMO jointly learns semantic status updating and hierarchical task handling through realized task utility, reporting higher on-time completion and capacity-aware decision accuracy across evaluated workloads. Its evaluation remains bounded to a controlled single-EN/single-SN topology.

  • Problem

    Stale remote information, partial observability, bidirectional feedback, and mismatched event horizons make status updating and task control an asynchronous coupled decision problem.

    Source: §I · p. 2

  • Method

    CoSMO uses compact SN status semantics, recurrent semi-Markov send/no-send timing at the SN, and task-terminal hierarchical gate–route value learning at the EN.

    Source: §I · p. 2, 1) SN Status Abstraction · p. 4, C. Closed-Loop Event-Driven Decision… · p. 5

  • Results

    CoSMO achieved the highest on-time completion rate and maintained high capacity-aware decision accuracy under overload across deterministic and Poisson load settings.

    Source: 5) Hierarchical Decision Ablation · pp. 14–15, §VI · p. 15

  • Takeaways & Limitations

    The learned policies trade update cost against workload, using lower cost at light load and more updates under overload while outperforming freshness-driven and fixed-rule policies on task metrics.

    Source: §VI · p. 15

    Evaluation is limited to a controlled single-EN/single-SN topology with fixed-rate backhaul and fixed modulation and coding, so transfer across topologies is untested.

    Source: §VI · p. 15

Abstract

from arXiv · show

In an edge--cloud collaborative edge-computing environment, an edge node (EN) must decide whether each user task should be executed locally, forwarded to a remote service (or cloud) node (SN), or rejected. The EN observes its local state directly but receives the SN state only through an intermittently refreshed cache. Status updating and task control therefore form an asynchronous closed loop under partial observability. Freshness-driven schemes, including those based on Age of Information (AoI), do not directly value an update by its effect on subsequent task decisions. We propose CoSMO (Co-design of Semantic-state Management and Offloading), a cooperative event-driven reinforcement learning (RL) framework that coordinates semantic status management and selective offloading through realized task utility. CoSMO learns a compact representation of the heterogeneous SN service state. At the SN, a recurrent semi-Markov double deep Q-network (Double DQN) agent jointly selects send/no-send and the next decision interval. At the EN, a task-terminal off-policy value-learning agent makes hierarchical gate--route decisions from local observations and stale remote semantics. The agents maintain separate observations and value targets but share the same realized task-utility stream, without centralized execution. Across the evaluated workload families, CoSMO's reported relative improvement in on-time completion rate over the best-performing competing method averages 18.6%--21.2%. For capacity-aware decision accuracy across the three strict-overload points, the corresponding reported gains average 17.6%--$17.9%.

I. INTRODUCTION

The paper frames stale remote-state offloading and status updating as one asynchronous closed-loop control problem. CoSMO jointly learns semantic status transmission and overload-aware task handling using downstream task utility.

  • The dispatcher chooses local execution, remote offloading, or rejection while observing local state directly and remote state through capacity-limited updates.Source: §I · p. 1
  • Freshness or state deviation alone may misvalue updates because the same cached information can yield the same action, while a small state change can alter the decision.Source: §I · pp. 1–2
  • Partial observability, bidirectional feedback, and different event horizons couple status updates with task decisions and complicate joint learning.Source: §I · p. 2
  • CoSMO uses a compact semantic status abstraction, a recurrent semi-Markov Double DQN at the SN, and hierarchical gate–route value learning at the EN.Source: §I · p. 2, B. Freshness and Goal-Oriented Updating · p. 3
  • The framework evaluates information by downstream task utility and coordinates status abstraction, update timing, task acceptance, and execution routing in one asynchronous process.Source: §I · p. 2, §II · p. 2, C. Semantic-Aware Edge Offloading · p. 3

1) Network Architecture

The system is a three-tier MEC architecture in which an EN dispatches UE tasks to local execution, an SN, or rejection. Wireless access, queueing, computation, and backhaul conditions determine realized completion delay.

  • Multiple UEs send tasks to an EN, which processes each task locally, offloads it to an SN, or rejects it; the EN–SN backhaul carries tasks, results, and status updates.Source: 1) Network Architecture · p. 3
  • Intermittent SN updates are timestamped in an EN cache, so task decisions use local EN information and potentially stale remote status.Source: 1) Network Architecture · p. 3
  • Accepted tasks follow FCFS service, with local execution or SN offloading determining the applicable execution path and delays.Source: 2) Task Model and Delay Cost · p. 3
  • An accepted task succeeds if and only if its realized end-to-end delay T_k is no greater than its deadline D_k.Source: 2) Task Model and Delay Cost · p. 3
  • Mobility is represented through effective uplink throughput, access-state trend, and residual connection time, capturing access capability, short-term change, and connection sustainability.Source: 3) Mobility and Access State · p. 4

B. Decision-Oriented Status and Observations

CoSMO abstracts the SN service state into a compact semantic cached at the EN and maintains separate, locally available observations for the two agents. These observations include cache age, link statistics, task history, and local service information.

1) SN Status Abstraction

  • The SN status-abstraction mapping f_θ produces a two-dimensional semantic z_t that the SN transmits and the EN caches for later task decisions.Source: 1) SN Status Abstraction · p. 4

2) Asymmetric EN/SN-Side Observations

  • At each SN decision epoch, the observation includes the current semantic, the acknowledged cached semantic, its AoI, completed offloaded-task history, and update-link statistics.Source: 2) Asymmetric EN/SN-Side Observations · p. 4

2) Asymmetric EN/SN-Side Observations

C. Closed-Loop Event-Driven Decision Process

The system models task arrivals and SN update epochs as distinct event streams in an asynchronous semi-MDP. It jointly optimizes task decisions, semantic updates, and communication cost through realized task utility.

Event streams

  • Task arrivals trigger EN decisions, while SN epochs trigger semantic transmission and interval-selection decisions.The two event clocks are generally nonuniform, so the formulation does not impose a common artificial clock.Source: C. Closed-Loop Event-Driven Decision… · p. 5

EN task control

  • The EN chooses local execution, SN offloading, or active rejection for each arriving task.The action indices are 0 for local execution, 1 for offloading, and 2 for active rejection.Source: C. Closed-Loop Event-Driven Decision… · p. 5

SN status control

  • The SN jointly chooses whether to transmit the current semantic and how long to wait before the next decision epoch.A no-send decision preserves the EN’s cached semantic during the selected interval, while a send decision updates the cache after reception.Source: C. Closed-Loop Event-Driven Decision… · p. 5

Coupling across events

  • A transmitted update can affect all EN task decisions in the subsequent interval, including when the mean update interval is comparable to or shorter than task inter-arrival time.The number of task decisions in an interval may be small, but the same model and training pipeline remain applicable.Source: C. Closed-Loop Event-Driven Decision… · p. 5

Outcome utility

  • Task outcomes distinguish successful acceptance, failed acceptance, correct rejection, and erroneous rejection under resource and deadline feasibility constraints.These labels penalize both blind acceptance under overload and unnecessary rejection.Source: D. Problem Formulation · pp. 5–6

Joint objective

  • The objective maximizes long-term average net utility by balancing outcome-dependent task utility against semantic-update communication cost.Update cost is incurred only when the SN transmits, and one update may influence multiple subsequent tasks.Source: D. Problem Formulation · pp. 5–6

IV. COSMO: COOPERATIVE SEMANTIC-AWARE UPDATING AND OFFLOADING

CoSMO combines a compact semantic representation of SN state with separate SN updating and EN forwarding agents. Utility-guided updates coordinate the agents while preserving decentralized observations, value targets, and execution.

CoSMO architecture

  • The SN branch processes semantic and temporal information to produce update decisions, while the EN branch separately encodes local context and cached remote context before fusion.The dual-branch design keeps local task/service features distinct from delayed remote-semantic and link features.Source: A. Architecture Overview · p. 6, Figure 2 · p. 7

Utility coordination

  • Utility-guided policy updates coordinate separate SN and EN replay and loss branches through their common realized task-utility stream.The agents retain separate observations and value targets, and the encoder remains frozen during coordinated training.Source: A. Architecture Overview · p. 6

Raw-State Semantic Encoder

  • A frozen semantic encoder maps the raw SN service state into a compact representation used by both updating and offloading policies.The lightweight MLP representation is transmitted to and cached at the EN and also supplied to the SN scheduler.Source: B. Raw-State Semantic Encoder · pp. 6–7

Semantic pretraining

  • The encoder is pretrained with interval-conditioned forward-dynamics prediction of normalized SN state increments rather than raw-state reconstruction.The auxiliary prediction head uses the selected update interval and is discarded after pretraining.Source: B. Raw-State Semantic Encoder · pp. 6–7, Figure 3 · p. 7

C. Temporal Semantic Decision Network

The temporal semantic decision network lets the SN select send/no-send together with a variable decision interval. A recurrent dueling Double DQN evaluates interval-level task utility and update cost using semi-Markov returns.

Action space

  • The candidate intervals satisfy gmin ≤ g(1) < g(2) < · · · < g(L) ≤ gmax.These heterogeneous intervals determine the next SN decision epoch.Source: C. Temporal Semantic Decision Network · p. 7
  • The SN scheduler chooses a paired action consisting of a transmit indicator and one candidate waiting interval.The discrete action set includes both send and no-send variants for each candidate interval when no-send is enabled.Source: C. Temporal Semantic Decision Network · p. 7

Temporal observation

  • The scheduler forms temporal features from SN semantics, cached semantics, cache age, and recent task and link statistics before recurrent processing.A feature adapter and GRU capture temporal dependencies across SN decision epochs.Source: C. Temporal Semantic Decision Network · pp. 7–8

Value learning

  • A dueling Double DQN combines state-value and action-advantage branches to estimate values for the SN’s paired update actions.Double DQN separates next-action selection from evaluation to reduce Q-value overestimation.Source: C. Temporal Semantic Decision Network · p. 8

Semi-Markov return

  • The SN interval reward sums task utilities over the following interval and subtracts semantic-update cost charged only for send actions.With no arrivals, the task-utility sum is zero; the main objective uses βSN = 1.Source: C. Temporal Semantic Decision Network · p. 8
  • Semi-Markov discounting assigns greater discounts to longer waiting intervals, aligning returns with heterogeneous elapsed times.The implementation uses a reference interval equal to the shortest candidate interval.Source: C. Temporal Semantic Decision Network · p. 8

D. EN Selective-Offloading Agent

The EN agent makes selective forwarding decisions from separately encoded local observations and cached remote semantics. Its hierarchical network first gates acceptance, then routes accepted tasks locally or to the SN.

Selective forwarding

  • The EN chooses among local execution, SN offloading, and active rejection at each task arrival.Local context is current and directly observed, whereas the remote semantic is compact and delayed.Source: D. EN Selective-Offloading Agent · p. 8

State encoding

  • A dual-branch encoder separately processes local EN context and cached SN context before feature fusion.The branches reflect different reliability, dimensions, timescales, and statistical meanings of the two information sources.Source: D. EN Selective-Offloading Agent · p. 8

Hierarchical decision

  • The hierarchical forwarding network first determines whether to accept a task and then selects local execution or SN offloading for accepted tasks.This gate–route structure makes rejection explicit under overload.Source: D. EN Selective-Offloading Agent · p. 8

1) Dual-Branch State Encoding

The EN constructs separate local and remote input branches, combining directly observed task and service information with cached SN semantics and link characteristics. Two MLP branches encode these inputs before feature fusion.

1) Dual-Branch State Encoding

2) Hierarchical Forwarding Decision Network

The EN uses a hierarchical gate–route network to choose rejection or acceptance first, then local execution or SN offloading for accepted tasks. It learns expected terminal utility from delayed task outcomes while preserving consistency with stale remote observations.

2) Hierarchical Forwarding Decision Network

E. Utility-Coordinated Training Algorithm

Utility-coordinated training couples separate SN and EN learners through realized task utility while preserving decentralized execution. Event-driven updates finalize EN transitions at task outcomes and train the SN across update intervals.

E. Utility-Coordinated Training Algorithm

F. Complexity Analysis

The evaluation analyzes training and inference costs in a controlled simulated MEC environment while testing task selection, completion, and semantic-update overhead. Execution cost is constant per event for fixed network widths and bounded action sets, but backhaul variability is outside the tested scope.

F. Complexity Analysis

  • Training cost includes EN and SN optimizer updates, batch sizes, sampled sequence length, and optional oracle-derived auxiliary supervision.Source: F. Complexity Analysis · p. 10
  • With bounded EN and SN queues, the auxiliary supervision cost is constant with respect to episode length.Source: F. Complexity Analysis · p. 11
  • Execution removes replay sampling, oracle labeling, target-network evaluation, and backpropagation, leaving per-decision inference costs.Source: F. Complexity Analysis · p. 11
  • Inference time is constant per decision event for fixed network widths and bounded action sets.Source: F. Complexity Analysis · p. 11
  • The experiments test load-aware decisions, on-time completion, and semantic-update overhead under stale remote-state information.Source: §V · p. 11
  • The controlled single-EN/single-SN setup fixes topology, access conditions, and backhaul rates to isolate semantic updating, acceptance, and routing.Source: 1) Simulation Environment · p. 11
  • The experiments do not assess adaptation to backhaul-rate variability or short-term rate trends.Source: 1) Simulation Environment · p. 11

3) Baselines

The evaluation compares COSMO with fixed, freshness-driven, and rejection-capable baselines across smooth and bursty workloads spanning underload to overload. COSMO maintains higher capacity-aware decision accuracy across the evaluated load sweep.

Baselines

  • COSMO is compared with six baselines, including fixed execution policies, AoI/AoV/AoCI freshness policies, and a hierarchical-decision ablation.The baselines use identical workloads and evaluation protocols.Source: 3) Baselines · p. 12

Workloads

  • The workload uses deterministic and Poisson arrivals at the same mean rates, testing smooth traffic and random bursts across offered loads ρ = 0.54–1.45.The sweep covers underload, the capacity boundary, and overload.Source: 4) Datasets · p. 12

Metrics

  • Evaluation reports system success rate, capacity-aware decision accuracy, and update-cost rate using greedy learned actions and repeated ns-3 runs.Metrics average three runs, with five evaluation episodes per run and two-sided 95% confidence intervals.Source: 5) Evaluation Metrics · pp. 12–13

Capacity-Aware Decision Accuracy

  • At strict overload, COSMO’s decision accuracy remains 0.930–0.941 for deterministic arrivals and 0.901–0.931 for Poisson arrivals.These values span λ ∈ {240, 280, 320} tasks/s.Source: 1) Capacity-Aware Decision Accuracy · p. 13
  • 17.6% and 17.9% are COSMO’s average relative gains over the best competing method across the three strict-overload points under deterministic and Poisson arrivals.The per-load gain ranges are 10.7%–25.5% for deterministic arrivals and 8.3%–26.7% for Poisson arrivals.Source: 1) Capacity-Aware Decision Accuracy · p. 13
  • 27.7% is COSMO’s relative improvement over AoCI online at the Poisson capacity boundary, where accuracies are 0.880 and 0.690, respectively.At 320 tasks/s, the three Poisson rejection-capable baselines range from 0.813 to 0.832.Source: 1) Capacity-Aware Decision Accuracy · p. 13

2) On-Time Task Completion

COSMO achieves the highest on-time completion rate across evaluated loads and arrival processes, while remaining near the capacity-limited upper bound under overload. It trades higher update expenditure for improved task outcomes, with hierarchical gating providing larger gains in deep overload.

On-Time Task Completion

  • COSMO attains the highest system success rate at every evaluated load under deterministic and Poisson arrivals.Success rate is the fraction of measured tasks completed before their deadlines.Source: 2) On-Time Task Completion · pp. 13–14
  • COSMO achieves 0.620 success at 320 tasks/s under Poisson arrivals and remains within 89%–92% of the capacity-limited upper bound under Poisson overload.Under deterministic overload, it reaches 93%–94% of the corresponding bound.Source: 2) On-Time Task Completion · p. 13
  • 21.2% average improvement in Poisson on-time completion over the best-performing competing method across all seven loads, versus 18.6% under deterministic arrivals.At 320 tasks/s under Poisson arrivals, COSMO reaches 0.620 versus 0.498 for AoV gap, a 24.5% improvement.Source: 2) On-Time Task Completion · p. 13

Status-Update Efficiency

  • At 320 tasks/s, COSMO uses 46.71 cost units/s under deterministic arrivals and 46.96 under Poisson arrivals, exceeding AoI threshold but outperforming higher-cost alternatives.Its update cost is 89.1% lower than AoV gap under either arrival process and 14.1%–15.5% lower than AoCI online.Source: 3) Status-Update Efficiency · p. 14

Training-Reward Stabilization

  • Median reward stabilization occurs within 20 episodes in every evaluated setting, with longer stabilization under overload and Poisson arrivals.Median t90 is 6–7 episodes through 220 tasks/s, rising to 20 episodes at 320 tasks/s under Poisson arrivals.Source: 4) Training-Reward Stabilization · p. 14

Hierarchical Decision Ablation

  • At 320 tasks/s, hierarchical COSMO improves success rate by 13.4% under deterministic arrivals and 8.8% under Poisson arrivals over COSMO (w/o Gate).The ablation also reports higher decision accuracy and lower false-positive rate with comparable mean update cost.Source: 5) Hierarchical Decision Ablation · pp. 14–15

Related papers

Loading 2609.01082v1…