PaperlayerView exact revision on arXiv

Source-linked AI summary

Dependency- and Layer-Aware Microservice Workflow Offloading and Service Image Caching for Edge Environments

Zhongxiao Wang, Yueshen Xu, Qingshan Li, Xinkui Zhao, Wei Shao, Shuiguang Deng, Rui Li

arXiv:2609.28943v1cs.DC

TL;DR

Edge workflow offloading depends on deploying task-specific services, but existing research has not jointly addressed offloading and service-image caching. The paper proposes a dependency- and layer-aware framework, and reports stronger convergence metrics and lower completion time across its algorithm configurations.

  • Problem

    The paper identifies a gap in jointly optimizing microservice workflow offloading and image caching for edge environments, where workflows have dependencies and images share layers.

    Source: §I · p. 1, C. Microservice Offloading · p. 3

  • Method

    DLA-HDRL combines hierarchical reinforcement learning with LACA for image-layer sharing and DAMH-CA for long-range workflow dependencies.

    Source: §I · p. 2, §IV · p. 6, §VI · p. 15

  • Results

    Across four algorithm configurations, DLA-HDRL ranked highest for converged average reward and average image hit rate, and reduced average task completion time by up to 3.05%, 6.8%, and 9.19% versus the other three configurations.

    Source: G. Ablation Study for DLA-HDRL · p. 15

  • Takeaways & Limitations

    The authors publish layer-configuration data for 50 popular microservice images and the experimental code on GitHub.

    Source: §I · p. 3

Abstract

from arXiv · show

The microservice architecture has been applied broadly in many mainstream computing environments. As one of the most prevalent environments, edge computing also widely employs microservices to handle diverse requests and tasks. In practical scenarios, microservices usually constitute workflows that are built based on service dependencies to execute tasks. This offers an opportunity to explore the offloading technology to better harness resources and accelerate task execution. Unfortunately, this problem has not been paid attention to by existing research, and thus some valuable resources (e.g., microservice image cache) remain obscure. To fill this gap, we innovatively study the problem of joint optimization for microservice workflow offloading and service image caching in edge. This problem is challenging due to several issues such as the complexity of its solution space, intricate relationships among shared image layers, long-range dependencies between workflow tasks, and the absence of a real-world collection of service images. To address these issues, this paper proposes an innovative dependency- and layer-aware workflow offloading and image caching framework for microservices. We collected a real-world collection of microservice image layer data and published both this collection and experimental codes on GitHub. Extensive results demonstrate that our framework achieves superior performances, for example, a 22. 38\% reduction in average task completion time compared to baselines and significantly-increased image hit rates.

I. INTRODUCTION

Edge workflows depend on services being deployed where tasks execute, while container images can share layers that reduce image-pull delay. The paper motivates joint workflow offloading and image caching, then reviews related work and frames the system model.

I. INTRODUCTION

  • As cached-image counts rise, Random and LRU caching both reduce average image-pull delay; the preliminary experiment supports using idle edge storage for caching.The paper identifies Fig. 2 as the experiment on cached-image count and image-pull delay.Source: Figure 2 · p. 2, §I · p. 2
  • The paper jointly optimizes workflow offloading and image caching with DLA-HDRL, LACA, and DAMH-CA, and contributes a public collection of real image-layer data and experimental code.The collection covers 50 popular DockerHub images.Source: §I · pp. 2–3

A. Microservice Deployment in Edge Environments

  • Prior edge-deployment studies address placement and resource objectives, but the paper says they do not use microservices to rapidly deploy task-dependent services for offloading.The cited approaches include reward-sharing deep Q-learning, randomized rounding, attention-modified soft actor-critic, and multi-objective DRL.Source: A. Microservice Deployment in Edge… · p. 3

B. Microservice Workflows in Clouds

C. Microservice Offloading

  • Prior edge-offloading work uses microservices for deployment and task execution, but the paper identifies idle-storage image caching as underused; its comparison also highlights gaps in layer sharing, task dependencies, and offloading.Source: C. Microservice Offloading · p. 3

III. SYSTEM MODEL AND PROBLEM FORMULATION

  • The system model represents joint offloading and image caching, with layer-sharing relationships among images and dependencies among workflow tasks.The section introduces a scenario, mathematical model, and joint optimization formulation.Source: §III · p. 3

A. Overview

The overview models edge workflows as dependent tasks requiring microservice images, served by a distributed edge cluster and an image base. The communication model accounts for image pulls, task-data transfers, and execution timing.

A. Overview

  • The modeled scenario has users generating topologically heterogeneous workflows whose tasks each depend on a specific microservice.Workflows are represented as directed acyclic graphs with task-dependency edges.Source: A. Overview · p. 4
  • A microservice base caches image vectors and layer sizes, while heterogeneous edge servers have image-cache limits and differing compute, memory, and download capabilities.Source: A. Overview · p. 4

B. Communication Model

  • Each edge server has one downlink channel that handles only one data type per time slot; result-data uplink transmission is treated as negligible, and FDMA allows parallel result transmissions.Source: B. Communication Model · p. 4
  • A task starts only after its input arrives, its image is pulled, predecessor results are transferred, and execution resources are available.Image deployment pulls only layers absent from the server's local cache.Source: B. Communication Model · pp. 5–6

C. Problem Formulation

The paper formulates workflow offloading and service-image caching jointly to minimize long-term average system offloading completion time. Feasibility, assignment, caching, and maximum-delay constraints make the problem an NP-hard integer nonlinear program.

C. Problem Formulation

  • The joint optimization objective is to minimize long-term average system offloading completion time across workflow offloading and service image caching.The formulation is identified as problem P.Source: C. Problem Formulation · p. 6
  • Constraint (13) equates each time slot's system completion time with the maximum task completion time across users.Source: C. Problem Formulation · p. 6
  • The constraints require sufficient server memory, one execution server per task, and integer decisions ensuring service-image caching for task execution.Source: C. Problem Formulation · p. 6
  • The formulation is an NP-hard integer nonlinear programming problem, motivating the use of deep reinforcement learning.Source: C. Problem Formulation · p. 6

IV. DEPENDENCY- AND LAYER-AWARE HIERARCHICAL DRL APPROACH FOR MICROSERVICE WORKFLOW OFFLOADING AND SERVICE IMAGE CACHING

The paper proposes DLA-HDRL for the joint workflow offloading and service-image caching problem, combining hierarchical decomposition with attention mechanisms. The motivation is that traditional DRL struggles with large action spaces and vanilla HDRL has disadvantages for this problem.

IV. DEPENDENCY- AND LAYER-AWARE HIERARCHICAL DRL APPROACH FOR MICROSERVICE WORKFLOW OFFLOADING AND SERVICE IMAGE CACHING

  • Traditional DRL can struggle to explore solution spaces with large action spaces and diverse actions.Source: §IV · p. 6
  • HDRL decomposes solution spaces for joint optimization problems, but the paper identifies disadvantages in vanilla HDRL for this problem.Source: §IV · p. 6
  • The proposed DLA-HDRL combines a hierarchical approach with attention mechanisms to address the workflow offloading and image caching problem.Source: §IV · p. 6

A. Overview

DLA-HDRL uses a workflow offloading agent and edge-server caching agents. It queues workflow tasks in dependency-respecting order and then coordinates server selection with image caching decisions.

A. Overview

  • DLA-HDRL comprises one workflow offloading agent and several microservice caching agents deployed on edge servers.Source: A. Overview · p. 6, Figure 4 · p. 7
  • The Topological Sorting and Polling tactic converts each workflow into an ordered sequence, then polls across sequences to construct the microservice task queue.Source: A. Overview · p. 6
  • For each head task, the offloading agent uses layer-aware cross-attention to perceive layer sharing between its image and cached images across servers before selecting an execution server.Source: A. Overview · p. 7
  • The selected server's caching agent uses dependency-aware multi-head cross-attention to perceive dependencies among task images in the service chain and determine image replacement.Source: A. Overview · p. 7

B. Layer-Aware Cross-Attention-Based Workflow Offloading Agent

The offloading agent uses layer-aware cross-attention to represent image-sharing relationships, then selects a feasible edge server with a D3QN-based policy. NoisyNet exploration and prioritized replay support its learning process.

B. Layer-Aware Cross-Attention-Based Workflow Offloading Agent

  • LACA embeds the task image and cached-image representations, computes attention over cached images, and combines the attention vector with the agent state.The mechanism is designed to help the agent capture complex layer-sharing relationships among images.Source: B. Layer-Aware Cross-Attention-Based… · p. 8
  • A LACA-promoted Dueling Double Deep Q-Network serves as the offloading agent for the continuous-state, discrete-action decision problem.Source: B. Layer-Aware Cross-Attention-Based… · p. 8
  • The agent uses NoisyNet to make exploration learnable, injecting Gaussian noise into the advantage layer.The paper contrasts this with the commonly used epsilon-greedy exploration approach.Source: B. Layer-Aware Cross-Attention-Based… · p. 8
  • Offloading actions are restricted to servers with enough memory to satisfy the task's requirement.Source: B. Layer-Aware Cross-Attention-Based… · p. 9
  • The agent samples replay transitions by priority for learning and updates its evaluation-network parameters by minimizing mean squared error.Transition priority is based on temporal-difference error, with a constant preventing zero priority.Source: B. Layer-Aware Cross-Attention-Based… · p. 9

C. Denedency-aware Multi-Head Cross-Attention-Based Microservice Caching Agent

The caching agent uses dependency-aware attention to make image-caching decisions informed by the current task, cached images, and images needed by subsequent tasks. Its D3QN training incorporates the workflow-offloading agent into the environment.

C. Denedency-aware Multi-Head Cross-Attention-Based Microservice Caching Agent

  • The caching state records the current task image and the image types required by the next L tasks in the queue.The stated caching objective is to reduce image-pull delay for subsequent tasks.Source: C. Denedency-aware Multi-Head… · p. 9
  • The agent chooses a cached image to replace with the current task image, and its reward uses Eq. (19) to reflect the shared goal of minimizing total task completion time.Both image caching and task offloading are described as ultimately aiming to minimize total task completion time.Source: C. Denedency-aware Multi-Head… · pp. 9–10
  • DAMH-CA mines long-range dependencies among the task image, cached images on the execution server, and images required by subsequent tasks.The mechanism combines cross-attention and multi-head attention to address dependencies across these sequences.Source: C. Denedency-aware Multi-Head… · p. 10
  • The caching agent combines D3QN with DAMH-CA, while its training environment includes the workflow-offloading agent and records caching transitions for PER-based training.The next state is available only after the offloading agent acts in the next time slot; the evaluation network is trained with MSE loss.Source: C. Denedency-aware Multi-Head… · p. 10

D. Computational Complexity

The paper analyzes inference costs for both agents and their attention mechanisms. With fixed hyperparameters, it reports that DLA-HDRL inference time grows linearly with β·|E|.

D. Computational Complexity

  • Workflow-agent inference includes LACA at O(β·|E|·d_o·H) and a fully connected layer at O(β·d_o·d_f).The initial fully connected layer dominates the network's forward-propagation complexity.Source: D. Computational Complexity · p. 10
  • Caching-agent inference uses DAMH-CA at O(L·H·N·d_c), followed by an image-replacement action at O((1 + β)·d_c·d_f).The action cost is for selecting an image replacement; d_f is the fully connected layer's output dimensionality.Source: D. Computational Complexity · pp. 10–11
  • With fixed d_o, H, N, and d_c, DLA-HDRL single-inference time is linear with respect to O(β·|E|).The reported total is O(β·|E|·d_o·H + β·N·d_c·H).Source: D. Computational Complexity · p. 11

V. EXPERIMENT

The experiment section validates the approach using microservice and edge-environment data, and reports the software and hardware used for the experiments.

V. EXPERIMENT

  • The experimental platform is based on real-world microservice data and edge-environment data.The section introduces experiment settings before presenting results and analyses.Source: §V · p. 11
  • Experiments were implemented with PyTorch 2.0.0 and CUDA 11.8 on Ubuntu 20.04.Source: §V · p. 11
  • The reported system had 32GB RAM, an Intel Core i7 CPU, and an NVIDIA RTX4090 GPU.Source: §V · p. 11

A. Data Collection and Datasets

The authors assembled microservice image frequency and layer data, then used external datasets to construct edge networks and workflow topologies for evaluation.

A. Data Collection and Datasets

  • The released image dataset records weekly download frequencies for 50 DockerHub images and layer hashes and sizes analyzed with Dive.Identical layer file hashes indicate that distinct images share a layer.Source: A. Data Collection and Datasets · p. 11
  • The edge-network construction uses the EUA dataset's geographical distribution of real-world edge servers in Australia.Source: A. Data Collection and Datasets · p. 11
  • Microservice workflow topologies are generated from Alibaba cluster-trace call graphs, with 4,500 graphs for training and 500 for testing.The paper reports randomly selecting these graphs from more than 10 clusters.Source: A. Data Collection and Datasets · p. 11

B. Environmental and Parameter Settings

The experimental platform uses adjacent edge servers in Sydney and generates user workflows over time; Table III lists parameter settings.

B. Environmental and Parameter Settings

C. Baselines

The section describes the comparison baselines, reports DLA-HDRL convergence behavior, and compares performance as user count, cache capacity, and edge-server count vary.

C. Baselines

  • Baselines comprise joint-action D3QNNP, SAC, PPO, and HOC’s earliest-available-server offloading with LRU image caching.Source: C. Baselines · pp. 11–12

D. Convergence Performance

  • Batch size 128 and learning rate 1e-3 produced superior converged reward and hit rate; reward converged after 1200 episodes, while hit rate oscillated about 0.1.Source: Figure 5 · p. 12, D. Convergence Performance · p. 12
  • Image-size variation accompanied hit-rate oscillations; both agents’ loss values reached a minimum by training’s end.The cited examples are ZooKeeper at 313 MB and Python at 44.9 MB.Source: D. Convergence Performance · p. 12

E. Performance Comparison

F. Sensitivity Analysis to System Scale

The sensitivity and ablation analyses examine DLA-HDRL across changing system scales and configurations, while the conclusion summarizes its mechanisms and future research directions.

F. Sensitivity Analysis to System Scale

  • As users increase, DLA-HDRL still converges, its average reward falls as task queues lengthen, and its image hit rate rises.The paper attributes the hit-rate increase to DAMH-CA characterizing long-range dependencies across tasks in extended queues.Source: F. Sensitivity Analysis to System Scale · p. 14
  • As each server caches more images, converged average reward and image hit rate increase because more image layers can be stored locally, reducing image-pull delay.The analysis varies β from 2 to 6.Source: F. Sensitivity Analysis to System Scale · p. 14
  • With 4 to 6 edge servers, training converges; fewer servers lower reward and hit rate as caching capacity and computational capability decline.The paper links these declines to increased average task completion time and reduced hit rate.Source: F. Sensitivity Analysis to System Scale · pp. 14–15

G. Ablation Study for DLA-HDRL

  • As user numbers change, DLA-HDRL reduces average task completion time by up to 3.05%, 6.8%, and 9.19%, and raises average image hit rate by up to 11.92%, 38.96%, and 48.68%, respectively, versus the other three configurations.At convergence, reward and image hit rate rank DLA-HDRL first, followed by ablations without attention, without hierarchy, and without both.Source: G. Ablation Study for DLA-HDRL · p. 15

VI. CONCLUSION AND FUTURE WORK

  • DLA-HDRL combines hierarchical reinforcement learning, layer-aware cross-attention for cached-image matching, and dependency-aware attention for extracting long-range workflow features.The conclusion reports superior performance to baselines across diverse edge environments using a constructed dataset and real-world data.Source: §VI · p. 15
  • Future work will explore LLM mechanisms and applications for this problem and related microservice tasks, including deployment, orchestration, and resource scheduling.Source: §VI · p. 15
Loading 2609.28943v1…