PaperlayerView exact revision on arXiv

Source-linked AI summary

COSCO: Container Orchestration using Co-Simulation and Gradient Based Optimization for Fog Computing Environments

Shreshth Tuli, Shivananda Poojara, Satish N. Srirama, Giuliano Casale, Nicholas R. Jennings

arXiv:2104.14392v3cs.DCcs.PF

TL;DR

Volatile fog workloads and changing host capabilities complicate meeting energy and response-time requirements, while existing scheduling approaches face trade-offs between adaptation and overhead. The paper proposes gradient-based GOBI and simulation-coupled COSCO/GOBI*, and reports reductions in energy use, response time, SLO violations, and scheduling time against state-of-the-art schedulers.

  • Problem

    Fog scheduling must meet energy and response-time requirements despite volatile workloads and changing host capabilities.

    Source: §1 · p. 1

  • Method

    GOBI uses input-gradient back-propagation with neural objective models, while COSCO couples simulation and orchestration for GOBI* scheduling.

    Source: §1 · p. 2

  • Results

    GOBI and GOBI* reduce energy consumption, response time, SLO violations, and scheduling time against state-of-the-art schedulers on real-world fog applications.

    Source: §9 · p. 14

  • Takeaways & Limitations

    GOBI* gives better QoS values, while GOBI is more suitable for resource-constrained servers.

    Source: §9 · p. 14

    In real systems, limited neural-network explainability prevents formal convergence guarantees, and time-bound training yields a sub-optimal GOBI approximator.

    Source: §7.1 · p. 9

Abstract

from arXiv · show

Intelligent task placement and management of tasks in large-scale fog platforms is challenging due to the highly volatile nature of modern workload applications and sensitive user requirements of low energy consumption and response time. Container orchestration platforms have emerged to alleviate this problem with prior art either using heuristics to quickly reach scheduling decisions or AI driven methods like reinforcement learning and evolutionary approaches to adapt to dynamic scenarios. The former often fail to quickly adapt in highly dynamic environments, whereas the latter have run-times that are slow enough to negatively impact response time. Therefore, there is a need for scheduling policies that are both reactive to work efficiently in volatile environments and have low scheduling overheads. To achieve this, we propose a Gradient Based Optimization Strategy using Back-propagation of gradients with respect to Input (GOBI). Further, we leverage the accuracy of predictive digital-twin models and simulation capabilities by developing a Coupled Simulation and Container Orchestration Framework (COSCO). Using this, we create a hybrid simulation driven decision approach, GOBI*, to optimize Quality of Service (QoS) parameters. Co-simulation and the back-propagation approaches allow these methods to adapt quickly in volatile environments. Experiments conducted using real-world data on fog applications using the GOBI and GOBI* methods, show a significant improvement in terms of energy consumption, response time, Service Level Objective and scheduling time by up to 15, 40, 4, and 82 percent respectively when compared to the state-of-the-art algorithms.

1 INTRODUCTION

Volatile workloads and changing host capabilities make it difficult to meet fog applications’ energy and response-time requirements. The paper introduces GOBI for fast gradient-based scheduling and COSCO with GOBI* to use simulation in scheduling decisions.

1 INTRODUCTION

  • The motivating gap is a scheduling policy that can adapt quickly to volatile fog environments while keeping scheduling overhead low.The paper describes heuristics as quick but poorly adapting to dynamic conditions, and adaptive evolutionary or reinforcement-learning approaches as slow or difficult to scale.Source: §1 · pp. 1–2
  • GOBI applies input-gradient back-propagation to neural approximators of QoS objectives, aiming for faster and more scalable fog scheduling.It combines neural-network objective models with gradient-based optimization strategies.Source: §1 · p. 2
  • COSCO couples simulation with container orchestration, enabling single- or multi-step simulation of container migration decisions in fog environments.GOBI* uses simulated schedule outcomes to provide objective values to another neural approximator.Source: §1 · p. 2, Figure 1 · p. 2
  • The paper evaluates GOBI and GOBI* against state-of-the-art schedulers using real-world fog-application data.The introduction identifies performance evaluation as the paper’s final empirical section.Source: §1 · p. 2
  • GOBI* extends GOBI by using COSCO simulation results to improve predictions and scheduling decisions with lower scheduling overheads.The paper reports that GOBI has lower performance than GOBI*, while requiring fewer computational resources.Source: §1 · p. 2

2 RELATED WORK

Related work spans heuristic, evolutionary, MaxWeight, reinforcement-learning, and simulation-based approaches. The paper positions COSCO as a way to connect simulation with AI-model decision-making rather than using simulation only to aid search.

Heuristic methods

  • Heuristic schedulers such as LR-MMT and MAD-MC use host-load prediction or utilization correlations, but are described as failing to model stochastic environments with dynamic workloads.The paper uses these approaches as fast heuristic baselines.Source: §2 · pp. 2–3

Evolutionary models

  • Genetic-algorithm approaches use gradient-free search with neural objective models, but can converge more slowly and scale less well than gradient-based methods.The paper also notes that non-local jumps can produce substantial scheduling changes and more migrations.Source: §2 · p. 3

MaxWeight based schedulers

  • POND uses virtual host queues, UCB-updated expected rewards, and MaxWeight dispatch, but the paper describes MaxWeight policies as having high wait times in volatile scenarios.Objective scores are supplied as negative rewards in the authors’ setup.Source: §2 · p. 3

Reinforcement Learning models

  • The cited reinforcement-learning approaches include DQLCM and A3C; the paper characterizes such methods as slow to adapt and insufficiently scalable.These approaches use learned state or policy methods to schedule workloads.Source: §2 · p. 3

Coupled simulation and model-based control

  • Prior coupled-simulation work is described as supporting search methods, whereas COSCO provides an interface for simulation to supply additional data to AI models.The paper selects heuristic, learning, and evolutionary baselines for their complementary strengths and weaknesses.Source: §2 · p. 3

3 MOTIVATING EXAMPLE

The motivating example isolates scheduling time as a factor in response-time SLO violations. It compares existing scheduling times with a hypothetical GA variant that makes the same decisions in 10 seconds.

3 MOTIVATING EXAMPLE

  • 100–150 seconds of scheduling time accompanies SLO violation rates of up to 24%; with GA’s same decisions in 10 seconds, the rate is approximately 2%.GA2 is the hypothetical faster variant used to illustrate the potential effect of reducing scheduling time while holding decisions constant.Source: §3 · p. 4

4 SYSTEM MODEL AND PROBLEM FORMULATION

The model describes a heterogeneous fog system in which a broker schedules and migrates tasks across hosts over discrete intervals. The optimization problem is to choose feasible decisions that minimize QoS objectives over time.

4.1 System Model

  • A fog broker manages task scheduling, data, resource monitoring, and container orchestration across IoT, gateway, fog-resource, and cloud layers.Edge hosts are resource-constrained with lower communication latency; cloud hosts are more powerful but farther from users.Source: §4.1 · p. 4
  • Hosts have time-series CPU, RAM, disk, and bandwidth utilization, capacity limits, and average communication latency.The system assumes a fixed set of hosts and accounts for latency across layers, but ignores communication latency among devices within the same layer.Source: §4.1 · p. 4

4.2 Workload Model

  • Scheduling proceeds in equal-duration intervals, with new IoT tasks carrying SLO requirements and completed tasks recorded at interval boundaries.SLO violations are measured as the fraction of task response times exceeding stipulated deadlines.Source: §4.2 · p. 4
  • At each interval, the broker decides allocations for new and waiting tasks and migrations for active tasks that have not completed.Decisions are task-host pairs, and the first interval involves only new-task allocations.Source: §4.2 · pp. 4–5
  • Only feasible allocations or migrations are executed; tasks that cannot be allocated remain in the waiting queue, while completed tasks leave the active set.The executed decision records tasks successfully allocated or migrated.Source: §4.2 · p. 5
  • The simulator takes scheduling decisions, task-utilization characteristics, and QoS parameters to produce host characteristics and QoS values such as energy consumption and response time.Physical execution is separately denoted by F.Source: §4.2 · p. 5

4.3 Problem Formulation

  • The objective is to minimize the aggregate QoS objective over the execution horizon while allocating new, waiting, and unfinished active tasks at each interval.The objective O(Pt) may combine multiple QoS parameters into a scalar value, subject to the scheduling constraints.Source: §4.3 · p. 5

5 THE GOBI SCHEDULER

GOBI optimizes fog scheduling decisions by minimizing a learned approximation of QoS with gradients taken with respect to the decision inputs. It targets energy consumption and response time, and updates decisions and its model as workloads change.

5 THE GOBI SCHEDULER

  • GOBI uses gradients with respect to the scheduling-decision input to minimize a neural approximation of the QoS objective at each interval.The approximator represents O(Pt), and the scheduler optimizes the decision matrix rather than updating model weights during that optimization.Source: §5 · p. 5, §5.3 · p. 6

5.1 Objective Function

  • GOBI's objective focuses on average energy consumption and average response time, with both metrics normalized to lie between 0 and 1.Average energy consumption is normalized by maximum host power; average response time covers leaving tasks and is normalized by the maximum response time through the current interval.Source: §5.1 · pp. 5–6

5.2 Input Parameters

  • The model input combines task-utilization features, host-utilization and capacity features, host latency, and a task-to-host decision matrix.Task features include IPS, RAM, disk, and bandwidth; the decision matrix represents allocations for new, waiting, active, and leaving tasks.Source: §5.2 · p. 6

5.3 Model Training

  • A neural approximator is trained on scheduler traces to predict the QoS objective, using mean squared error between predictions and observed values.The training dataset pairs task, host, and scheduling-decision inputs with O(Pt); the paper uses MSE as its loss.Source: §5.3 · p. 6
  • Back-propagated input gradients and the chain rule let GOBI optimize decisions through the neural model, with gradient descent iterated to a convergence threshold or iteration limit.The presented single-layer tanh derivative is W^T × (1 − tanh^2(W · x + b)); momentum or annealing methods are also suggested to help avoid local optima.Source: §5.3 · pp. 6–7

5.4 Scheduling

  • At each interval, GOBI initializes from the prior decision, fine-tunes its model on newly generated data, then converts the optimized decision matrix into task-host allocations.Rows are mapped to hosts by taking the arg max; infeasible allocations are skipped when executing the decision.Source: §5.4 · p. 7, §5.3 · p. 7

6 THE COSCO FRAMEWORK

COSCO couples a scheduler with simulation and container orchestration for fog workloads. Its model interface supports a one-interval simulation look-ahead to estimate QoS for a candidate decision.

6 THE COSCO FRAMEWORK

  • COSCO supports hosts that are simulated or physical, and workloads represented either by resource-use time series or actual applications.At each scheduling interval, it creates new workloads and retains a waiting queue for tasks not allocated previously.Source: §6 · p. 7

6.1 Simulator

  • The simulator processes completed tasks, new and waiting workloads, and scheduling decisions for allocations or active-task migrations; tasks that cannot fit are queued.The described simulation uses trace-driven discrete-event execution rather than running tasks on physical machines.Source: §6.1 · pp. 7–8

6.2 Framework

  • In physical deployments, COSCO runs tasks as Docker containers on networked hosts and uses CRIU checkpoints to migrate containers without halting active execution.A server communicates with host-side DockerClient services through HTTP REST APIs.Source: §6.2 · p. 8

6.3 Model Interface

  • At the start of an interval, COSCO can simulate the next interval using a candidate schedule and predicted utilization to estimate QoS for GOBI*.This is a single-step look-ahead intended to inform scheduling decisions.Source: §6.3 · p. 8

7 EXTENDING GOBI TO GOBI*

GOBI* extends GOBI with utilization forecasts and simulator-derived QoS estimates to guide scheduling decisions. Its interactive training can find decisions with lower objective scores than GOBI, although formal convergence properties are unavailable.

7 EXTENDING GOBI TO GOBI*

  • Predicted utilization for the next interval and simulated QoS address GOBI’s reliance on prior-interval utilization, which can make its predictions inaccurate.GOBI* adds the utilization and estimated objective as inputs to a second neural approximator.Source: §7 · pp. 8–9
  • At each interval, GOBI* uses GOBI’s prior decision, predicted utilization, and a single-step simulation estimate to choose a decision with a lower objective value when available.The simulator estimates QoS for GOBI’s action, and GOBI* uses that estimate to predict its next action.Source: §7 · pp. 8–9

7.1 Convergence of GOBI and GOBI*

  • Formal convergence properties are unavailable because neural networks have limited explainability; the authors instead offer technical observations supporting convergence seen in practice.The real-case discussion assumes time-bounded training produces a sub-optimal GOBI approximator, so its predicted decision is not always optimal.Source: §7.1 · p. 9
  • When GOBI’s decision is better, the loss encourages GOBI* to imitate it; when GOBI* is better, the interactive loss can guide it toward still better decisions.The authors state that each update yields an objective score the same as or lower than GOBI’s, assuming the described real-case dynamics.Source: §7.1 · p. 9
  • GOBI* always performs better than GOBI in objective scores in the reported real-case experiments, while sufficiently long training is expected to keep its objective no higher than GOBI’s.The authors also report that simulation-based interactive training lets GOBI* converge quickly and adapt to volatile environments.Source: §7.1 · p. 9
  • Figure 7 shows GOBI*’s probability of outperforming GOBI increasing during training, with faster loss convergence when the additional L2 term is included.The authors associate L2’s initial contribution with a sudden drop in L1.Source: §7.1 · p. 9, Figure 7 · p. 9

8 PERFORMANCE EVALUATION

The evaluation compares GOBI and GOBI* with heuristic, learning, and optimization baselines on physical and simulated fog platforms. Across the reported results, GOBI* achieves lower energy use and response time, while GOBI offers lower scheduling overhead.

8.1 Experimental Setup

  • Experiments paired a 10-VM Azure test bed (six London, four Virginia) with a 50-host scaled simulation, using 100 intervals of 300 seconds and averaging five runs.Source: §8.1 · pp. 9–10

8.2 Model Training and Assumptions

  • GOBI training data came from 2,000 simulator intervals followed by 300 framework intervals for fine-tuning; GOBI* data also included simulated estimates using GOBI decisions and predicted utilization.Utilization predictors for GOBI* use a single-cell LSTM trained on the same dataset as the GOBI model.Source: §8.2 · p. 10

8.3 Workloads

  • Evaluation workloads include Bitbrain utilization traces and DeFog applications; final DeFog comparisons use Yolo, Pocketsphinx, and Aeneas with equal selection probabilities.Dominant-workload runs also raise one application's selection probability to 0.80 and set the other two to 0.10.Source: §8.3 · pp. 10–11

8.4 Baseline Models

  • Baselines are LR-MMT and MAD-MC heuristics, GA, DQLCM and A3C learning methods, and the max-weight allocation method POND.Source: §8.4 · p. 11

8.5 Evaluation Metrics

  • Evaluation metrics include SLO violations, Jain's Fairness Index, average migration time, scheduling time, and average wait time.Source: §8.5 · p. 11

8.6 Results

  • 15.05% lower average interval energy (188.26 KW-hr) than A3C on Azure; 9.39% lower energy than A3C in simulation; average response time is 41.43% lower (156.09 seconds) than POND.The physical results use 10 Azure hosts; the simulated results use 50 hosts.Source: §8.6 · pp. 11–12, Figure 8 · p. 12, Figure 9 · p. 13
  • GOBI* has no SLO violations on physical and simulated setups; GOBI has 0.9% and 1.1%, respectively, while scheduling times are 2.65 seconds (GOBI) and 5.88 seconds (GOBI*).GOBI* also has no violations in the 50-host simulation; GA records 0.2% there.Source: §8.6 · p. 13

8.7 Comparing GOBI and GOBI*

  • GOBI* has lower next-interval objective value (16.4%) and objective-value prediction error (14.2%), but its scheduling time is more than twice GOBI's.GOBI* adds predicted utilization and a simulated estimate of the tentative GOBI action; the authors identify GOBI as more appropriate for resource-constrained servers.Source: §8.7 · p. 13

9 CONCLUSIONS AND FUTURE WORK

The conclusion presents COSCO as a coupled-simulation framework for predicting QoS and supporting scheduling decisions in heterogeneous fog environments. It reports that GOBI and GOBI* improve QoS against state-of-the-art schedulers, with different suitability depending on server resources.

9 CONCLUSIONS AND FUTURE WORK

  • COSCO combines simulator capabilities with a container-orchestration framework, while GOBI and GOBI* use back-propagation of gradients with respect to input to model scheduling objective scores.Source: §9 · pp. 13–14
  • GOBI* gives better QoS values, whereas GOBI is more suitable for resource-constrained servers.Source: §9 · p. 14
  • Future work proposes serverless workflow models for COSCO and recurrent, convolutional, or residual layers with ReLU for the back-propagation methods.The authors state that these extensions could support fine-grained auto-scaling and modeling temporal and spatial characteristics.Source: §9 · p. 14

IMPLEMENTATION OF COSCO FRAMEWORK

COSCO is implemented as Python classes for modeling hosts, workloads, scheduling, simulation, and orchestration of physical hosts. The implementation also specifies utilization modeling and a neural-network depth choice for GOBI.

IMPLEMENTATION OF COSCO FRAMEWORK

  • COSCO is implemented in Python with classes that represent hosts, workloads, scheduling decisions, simulation, and the physical orchestration framework.Workload objects generate new tasks and maintain a waiting queue; scheduler functions select containers for migration and return target hosts.Source: Appendix A · p. 14

A.1 Modelling of Computational Utilization

  • Physical experiments estimate host IPS from SPEC benchmark measurements and workload IPS from CPU utilization and executed instructions, yielding an infrastructure-agnostic utilization proxy.Source: §A.1 · p. 14

A.2 Simulator

  • The simulator tracks host and task objects, utilization, allocation decisions, QoS, and workloads; it calculates migration time from transfer time and applicable network latency.Source: §A.2 · p. 14

A.3 Framework

  • The framework provisions Docker-container hosts with Ansible and communicates through REST APIs; migrations run in parallel and non-blockingly, with utilization synchronized through InfluxDB.Source: §A.3 · pp. 14–15

ACTIVATIONS

  • The appendix presents input-gradient derivations for the softplus and tanhshrink activation functions used in the neural models.Source: ACTIVATIONS · p. 15

C.1 Model search

  • Two hidden layers were selected for GOBI: additional layers reduce loss, but two or more layers exceed GPU memory for the input batch and increase inference time on CPU.The selection is based on the available simulation infrastructure.Source: §C.1 · pp. 15–16, Figure 11 · p. 16

C.2 Model Training

GOBI and GOBI* are trained using scheduler-generated data, with GOBI* incorporating predicted utilization and simulated QoS estimates. Framework models are pretrained on simulation data and fine-tuned on physical data; Azure power values are interpolated between measured utilization increments.

C.2 Model Training

  • GOBI is trained on random-scheduler data, while GOBI* adds LSTM-predicted utilization and simulated QoS estimates based on tentative GOBI decisions.The described GOBI training dataset contains 2,000 examples, with an 80/20 train-test split and AdamW optimization.Source: §C.2 · p. 16
  • Framework models are pretrained on 2,000 simulated examples, then fine-tuned on 200 physical examples; the simulation training runs for 600 epochs before physical-data convergence.The authors attribute the loss change at epoch 600 to this transition.Source: §C.2 · p. 16

POWER MODELS OF AZURE HOSTS

  • Azure power values are taken at 10% CPU-utilization increments from SPEC benchmarks, with linear interpolation for intermediate utilization levels.Source: Appendix D · p. 16

APPENDIX E ADDITIONAL RESULTS

Across the 10-host Azure testbed and 50-host simulation, GOBI and GOBI* have the lowest overall response times; at 50 hosts, both also reduce average cost per container versus A3C.

APPENDIX E ADDITIONAL RESULTS

  • At 50 hosts, GOBI* and GOBI have 24.7% and 26.3% lower average cost per container than A3C, respectively.For 10 hosts, all policies have negligible differences in average cost per container.Source: Appendix E · p. 16, Figure 14 · p. 17, Figure 15 · p. 17
  • POND has the lowest average CPU utilization at 8.94%; GOBI and GOBI* average 2.78%–6.12% higher utilization and have the most active containers.The passage attributes the higher active-container counts to placement that can execute multiple containers together.Source: Appendix E · pp. 16–17
  • GOBI and GOBI* have the least overall response times among the policies across the 10-host and 50-host settings.Response times are highly volatile with the non-stationary workloads.Source: Figure 14 · p. 17, Figure 15 · p. 17, Appendix E · p. 18
  • GOBI* has the highest average fairness index across applications at 0.034; GOBI* has the fewest migrations at 10 hosts (146), while A3C has the fewest at 50 hosts (486).A3C is least fair, with Jain’s indices of 0.01–0.05 per application; GOBI and GOBI* have consistently low migration counts (140–174).Source: Appendix E · pp. 17–18, Figure 14 · p. 17, Figure 15 · p. 17
  • Migration times are lowest for GOBI, GOBI*, and heuristic methods, while wait times rise for POND, A3C, and DQLCM in the 50-host simulation.POND frequently fails to converge after the 60th interval; A3C and DQLCM adapt slowly to volatile workloads and can make infeasible migration decisions.Source: Figure 14 · p. 17, Figure 15 · p. 17, Appendix E · p. 18

APPENDIX F VISUALIZATION OF UTILIZATION METRICS

Grafana visualizations of utilization metrics, generated from InfluxDB data, show substantial volatility in container resource consumption over a few hours.

APPENDIX F VISUALIZATION OF UTILIZATION METRICS

  • Grafana visualizations generated from InfluxDB data show significant volatility in container resource consumption over a few hours.The toolkit was used to visualize utilization metrics and check workload volatility in the fog environment.Source: Appendix F · p. 18
  • The utilization graphs are dynamically generated from the InfluxDB database.The visualizations use the Grafana toolkit.Source: Appendix F · p. 18
  • The visualized container resource consumption varies substantially across a few hours.The graphs are presented as screenshots of a few workload visualizations.Source: Appendix F · p. 18

Related papers

Loading 2104.14392v3…