Source-linked AI summary
ARGOS: Reinforcement Learning-Driven Multidimensional Elasticity for Service Orchestration in the Computing Continuum
Javier Mateos-Bravo, Sergio Laso, Juan Luis Herrera, Ilir Murturi, Pantelis Frangoudis, Schahram Dustdar
TL;DR
When heterogeneous Continuum nodes reach capacity limits, resource scaling alone cannot respond to changing pressure while preserving client-defined analytics ranges. ARGOS controls coverage, sample, and freshness per request through an MDP with capacity-aware admission. Deep controllers outperform deployable baselines in controlled evaluation, and frozen deep policies improve over the static midpoint in live trials.
Problem
Method
ARGOS combines capacity-aware admission with per-request MDP control of coverage, sample, and freshness under observed resource pressure.
Source: §7 · p. 21Results
Deep controllers outperform both deployable baselines in every controlled seed, recover roughly two thirds to three quarters of best-fixed reward, and improve live-trial reward and delivered fidelity over the static midpoint.
Source: §6.1 · p. 19Takeaways & Limitations
For settings resembling the evaluation, DQN and PPO are candidates when reward per request matters, with PPO showing more consistent overload behavior and lower mean service duty among deep controllers.
Source: §6.1 · p. 20Evaluation uses three heterogeneous nodes and does not characterize behavior as node and concurrent-request counts grow.
Source: §6.2 · p. 20
Abstract
from arXiv · showhide
Data-intensive services in the Computing Continuum must balance analytics quality, resource usage, and cost across heterogeneous nodes with limited and uneven capacity. This balance becomes especially difficult when resource scaling reaches capacity limits, because changes in demand and cluster pressure must then be absorbed without violating client-defined quality ranges. Existing orchestrators mainly adapt resources, placements, or replicas, while analytics requirements such as coverage, sample, and freshness remain fixed. This article presents ARGOS, the Adaptive Reinforcement Learning-Driven Governance for Orchestrated Services, an end-to-end controller that formulates multidimensional elasticity as a per-request Markov decision process over analytics quality and cluster pressure, supported by capacity-aware admission. ARGOS is evaluated under controlled workloads and time-varying multi-tenant arrivals on a heterogeneous cluster. Across the controlled scenarios, the deep reinforcement learning policies consistently outperform the non-learning baselines and approach the independently tuned best-fixed reference. A separate live evaluation reports improvements over the static midpoint under realistic and saturated arrivals, with no recorded CPU or memory violations but remaining coverage violations. These results support deep reinforcement learning as an adaptive mechanism for multidimensional elasticity when resource scaling alone is insufficient.
1. Introduction
Heterogeneous, capacity-limited Continuum nodes make resource scaling insufficient for maintaining client-defined analytics quality. ARGOS responds with per-request control of coverage, sample, and freshness under observed pressure, alongside capacity-aware admission and a reproducible evaluation design.
1. Introduction
- On constrained Continuum tiers, finite capacity can halt horizontal and vertical scaling while accepted requests still must meet client-defined SLOs.The challenge is jointly controlling coverage, sample, and freshness on heterogeneous nodes while accounting for resource pressure and per-request ranges.Source: §1 · p. 2
- Three gaps motivate ARGOS: RL often controls infrastructure rather than request analytics, online traces may not represent frozen policies, and evaluations can lack reproducibility.The proposed responses are shared-step quality and pressure control, frozen-policy comparisons under common conditions, and retained policy, decision, telemetry, and provenance records.Source: §1 · p. 2
- ARGOS models each admitted request as an MDP whose state combines cluster pressure and analytics configuration, adjusting coverage, sample, and freshness within accepted ranges.Capacity-aware admission, runtime monitoring, configuration distribution, and audit records form the end-to-end controller.Source: §1 · pp. 2–3
- The evaluation compares Q-learning, DQN, and PPO with static midpoint, reactive threshold, and independently tuned best-fixed policies in controlled and live stages.It uses an urban-mobility heatmap service over Seville and releases the evaluation scenario, scripts, result tables, and figure-generation code.Source: §1 · p. 3
2. Motivational use case
Urban-mobility analytics requests share finite infrastructure while requiring different combinations of coverage, sample, and freshness. Holding a selected quality point fixed after admission cannot adapt to changing resource pressure, whereas ARGOS adjusts requirements within client-approved ranges.
2. Motivational use case
- Concurrent municipal and private requests impose different coverage, sample, and freshness requirements on the same finite infrastructure.The examples include city-wide planning, event monitoring, emergency response, and passenger-flow optimization.Source: §2 · pp. 3–4
- Moving toward broader coverage, larger samples, or shorter freshness intervals raises processing demand and estimated operating cost.Contracts specify accepted quality ranges and may include a cost budget; minimum requirements that cannot be accommodated may be rejected.Source: §2 · p. 4
- A fixed operating point cannot use client-approved flexibility as shared resource pressure changes.A demanding point keeps consuming scarce capacity, while a conservative point cannot improve results when capacity becomes available.Source: §2 · p. 4, Figure 1 · p. 4
- ARGOS treats coverage, sample, and freshness as per-request control dimensions that can move among feasible operating points as infrastructure pressure changes.The adjustments preserve client-defined target bounds and resource-safety constraints.Source: §2 · p. 4
3. Related work
Prior work addresses infrastructure elasticity, elastic analytics, learned orchestration, and reproducible evaluation, but the paper identifies a gap in combining these elements. ARGOS controls all three analytics requirements under resource pressure and compares policies within a shared, traceable orchestration model.
3.1. Multidimensional elasticity in the Computing Continuum
- Prior elastic-analytics work formalizes coverage, sample, and freshness, while its Q-learning study leaves coverage outside the learned state and action spaces.ARGOS extends this foundation to all three requirements with observed pressure in a per-request MDP on heterogeneous nodes.Source: §3.1 · p. 5
3.2. Reinforcement-learning controllers for orchestration
- Cloud and edge RL systems have targeted resource allocation, scheduling, autoscaling, placement, or execution nodes rather than analytics requirements within individual requests.ARGOS instead controls coverage, sample, and freshness within client-defined ranges while observing shared resource pressure.Source: §3.2 · pp. 5–6
3.3. Evaluation rigor: baselines and reproducibility
- Evaluation research motivates meaningful comparison policies, repeated runs, seed-level analysis, and retained configuration, decision, and telemetry evidence.Prior work reports sensitivity to seeds and implementation choices and warns that aggregate conclusions can be unstable with limited runs.Source: §3.3 · p. 6
- ARGOS combines analytics elasticity, sequential RL control, and reproducible comparison by controlling all three requirements under resource pressure in a shared orchestration model.Table 1 summarizes representative approaches by control target, evaluation setting, and comparison design.Source: §3.3 · p. 6, Table 1 · p. 6
4. ARGOS
ARGOS mediates between tenant contracts and analytics runtimes across the Computing Continuum, admitting requests and adapting their quality settings through a per-request controller. Its lifecycle covers admission, repeated telemetry-based decisions, and policy training or frozen evaluation.
4. ARGOS
- ARGOS separates external clients, the orchestrator, and managed Continuum nodes, translating accepted quality ranges into placement and configuration instructions through a uniform node interface.Figure 2 depicts the deployment architecture and runtime interfaces; service-specific behavior remains in the node-side runtime adapter.Source: §4 · p. 7, Figure 2 · p. 7
4.1. Components
- Admission validates whether a request's minimum requirements can be placed, rejecting infeasible requests, admitting feasible ones when capacity exists, and queuing temporarily blocked requests.Admitted requests have independent controllers; the runtime monitor aggregates telemetry and the deployment manager distributes configurations and records target and realized coverage.Source: §4.1 · p. 8
4.2. Operational lifecycle
- The request lifecycle initializes targets within accepted ranges, then repeatedly observes telemetry, encodes state, masks ineffective or unsafe actions, applies an adjustment, and distributes a new plan when needed.Training can update and serialize a policy; deployment or evaluation can load it with exploration and parameter updates disabled, while the audit log retains decisions and operational records.Source: §4.2 · p. 9
4.3. Decision model
- The decision model uses a 13-dimensional operational state of resource pressure, contract position, service and workload conditions, capacity, latency, and recent violations.Its reward combines updated-target quality and cost with penalties that include pre-action resource telemetry; actions adjust quality targets and are masked at contract bounds or near capacity limits.Source: §4.3 · pp. 9–11
- Q-learning, DQN, and PPO share the MDP and are evaluated as frozen policies on held-out seeds after separate training.The comparison also includes static midpoint, reactive threshold, and a best-fixed reference selected independently by workload profile.Source: §4.3 · pp. 11–12
5. Experimental validation
ARGOS is evaluated in controlled, frozen-policy comparisons and live multi-tenant trials on a heterogeneous cluster. Deep controllers outperform deployable baselines in the controlled stage, while live trials show gains alongside coverage violations.
5.1. Experimental setup
- The testbed uses three heterogeneous nodes and five workload profiles; 720 accepted sessions span controlled training, tuning, and frozen evaluations across two runtimes.Evaluation disables exploration and parameter updates and uses held-out random streams.Source: §5.1 · pp. 12–13, Table 3 · p. 13
5.2. Controlled comparison
- DQN and PPO beat both deployable baselines in every seed on both runtimes, but all learned controllers remain below the best-fixed reference.The paired comparison reports gains over the static midpoint and reactive threshold; Table 4 summarizes paired reward advantages.Source: Table 4 · p. 14, §5.2 · p. 14, Figure 3 · p. 15
5.3. Resource-pressure analysis
- On thread, PPO reaches spatial fidelity 0.931 at 10.9 % duty, versus the best-fixed reference’s 16.8 % duty.Across runtimes, deep-controller fidelity is 0.931 to 0.940, compared with 0.909 to 0.913 for deployable baselines; Table 5 reports fidelity, duty, and CPU utilization.Source: §5.3 · p. 15, Table 5 · p. 16
5.4. Live-trial validation
- Live admission tracks overlap across controllers: all realistic-arrival requests are admitted on arrival, while overload queues about 140 of 180 submitted requests per variant.Figure 5 depicts active and queued requests over time; the overload active set saturates at roughly eight.Source: §5.4 · p. 17, Figure 5 · p. 18
- In live trials, DQN and PPO improve reward over the static midpoint under realistic arrivals; under overload, PPO improves in all five pairs while DQN improves in four.Figure 6 reports paired mean reward differences, including +0.261 reward per step for DQN and +0.173 for PPO under realistic arrivals.Source: §5.4 · p. 17, Figure 6 · p. 18
- Across 58,285 live decision epochs, 27,130 of 27,145 recorded violation events concern coverage; no CPU or memory violations are recorded.All coverage violations are upper-bound violations attributed to placement granularity, so the trials support feasibility on this testbed, not full contract compliance.Source: §5.4 · pp. 17–18, Table 6 · p. 19
6. Discussion
The discussion finds that deep controllers outperform deployable heuristics in the controlled setting and improve reward and fidelity over the static midpoint in live trials. It also bounds these findings to a small evaluation, including three-node testbeds, one workload, and fixed design choices.
6.1. Results discussion
- Both deep controllers improve on both deployable baselines in every seed of both runtimes, whereas tabular Q-learning does not improve on the static midpoint.DQN and PPO form a deep-controller tier rather than a strict ranking.Source: §6.1 · p. 19
- Deep controllers recover roughly two thirds to three quarters of best-fixed reward, trailing by about 0.13 to 0.17 reward per step while retaining online per-request control.PPO has the lowest service duty among deep controllers on both runtimes; DQN exceeds best-fixed duty only on process.Source: §6.1 · p. 19
- In live multi-tenant trials, both frozen deep controllers improve mean reward and delivered fidelity over the static midpoint, and retain positive mean reward under overload.PPO improves in every overload pair with an interval excluding zero; DQN improves in four of five pairs with a wider interval. Coverage violations dominate, while freshness violations are rare and no sample, CPU, or memory violations are recorded.Source: §6.1 · pp. 19–20
6.2. Limitations
- Comparisons rest on five paired seed means, and external evaluation covers one heatmap service, selected workload profiles, and live trials on the thread runtime.With five seeds, the exact two-sided Wilcoxon signed-rank test cannot yield a p-value below 0.0625; Student-t intervals also rely on difficult-to-assess normality assumptions.Source: §6.2 · p. 20
- Scalability remains uncharacterized beyond three heterogeneous nodes, including growth in nodes and concurrent requests and distributed leader failover.One active orchestrator was used to isolate quality-control behavior.Source: §6.2 · p. 20
- Reward coefficients are fixed design parameters, and lagged resource telemetry limits causal interpretation of an individual action’s immediate resource effect.The discrete action space improves auditability but leaves continuous control unexplored.Source: §6.2 · pp. 20–21
- The shared paired protocol reduces implementation-level confounds, while frozen-policy evaluation with held-out seeds and checked fingerprints strengthens deployment relevance and auditability.The protocol shares the seed schedule, orchestration loop, persistence path, and comparison scripts across controllers.Source: §6.2 · p. 21
7. Conclusions and future work
ARGOS couples per-request quality control with capacity-aware admission; frozen deep controllers outperform untuned baselines in controlled tests and improve live reward and fidelity over the static midpoint. Evidence remains bounded by limited seeds and the three-node urban-mobility setting.
7. Conclusions and future work
- ARGOS adjusts coverage, sample, and freshness within client-accepted ranges through per-request MDP decisions over cluster pressure and analytics configuration, paired with capacity-aware admission.Source: §7 · p. 21
- Across two runtimes, five workload profiles, and five held-out seeds, frozen DQN and PPO beat both untuned deployable baselines in every controlled comparison seed.They recover roughly two thirds to three quarters of the independently tuned best-fixed reference's reward, with near-reference fidelity at lower mean service duty in every case but one. Tabular Q-learning does not beat the static midpoint on either runtime.Source: §7 · p. 21
- In live arrivals, both frozen deep controllers improve mean reward and fidelity over the static midpoint; PPO improves in every paired trial, but coverage violations remain.No CPU or memory violations were recorded; upper-bound coverage violations remain because of three-node placement granularity.Source: §7 · p. 21
- Future work targets continuous actions, more seeds and deployment configurations, and larger Continuum topologies with additional analytics services to test finer control, statistical power, and generalization.Source: §7 · p. 21