Source-linked AI summary
A Dynamic Vertical Scaling Strategy for Distributed Stream Processing Applications in Edge Computing
Guilherme Hiago Costa dos Santos, Carlos Henrique Kayser, Tiago Coelho Ferreto
TL;DR
Edge stream processing must reconcile latency and throughput with limited, heterogeneous resources, and evidence is needed on task-level CPU adjustment for fixed placements. The paper uses PPO in a POMDP to adjust task allocations with an SLO-prioritizing reward; in simulation it lowers violation rates versus VRebalance and preserves throughput, while CPU savings vary by scenario.
Problem
Method
The paper formulates vertical autoscaling as a POMDP and uses PPO to jointly adjust per-task CPU with a reward prioritizing p95 latency SLO compliance.
Source: §1 · p. 2, §4.2 · p. 5Results
0.03–0.30% mean SLO violation rates for PPO versus 1.01–8.73% for VRebalance; throughput was preserved and CPU was lower in three of four combinations.
Source: §7 · p. 12Takeaways & Limitations
Across the evaluated profiles, workloads, and paired placements, results give a qualified affirmative answer, with CPU savings depending on application and workload conditions.
Source: §7 · p. 12Evaluation remains simulation-based; the authors call for tests on stream-processing engines in emulated and physical edge environments.
Source: §7 · p. 12
Abstract
from arXiv · showhide
Distributed Stream Processing applications at the edge must reconcile low latency and high throughput with limited and heterogeneous resources. This paper presents a dynamic vertical scaling strategy based on Proximal Policy Optimization, formulated as a Partially Observable Markov Decision Process. The policy jointly adjusts task allocations and prioritizes compliance with a p95 end-to-end latency Service Level Objective. In EdgeStreamPy simulation experiments with two application profiles, two workloads, and ten paired placements per combination, the selected policies preserved throughput, obtained mean violation rates from 0.03% to 0.30%, below VRebalance in every scenario, and used less CPU in three of four combinations. The comparison covers complete controller configurations with different decision frequencies.
1. Introduction
Edge stream processing must balance low latency and throughput against constrained, heterogeneous resources. This paper investigates whether operator-aware PPO can selectively adjust task CPU while maintaining p95 latency SLO compliance and throughput.
1. Introduction
- Limited edge capacity and heterogeneous servers make resource management central to sustaining application performance.Cloud-centric processing can also be constrained by low-response-time requirements and limited network bandwidth.Source: §1 · p. 1
- Vertical scaling changes resources for existing tasks without modifying the application graph or reconfiguring operator parallelism.This approach addresses the risk that uniform allocation wastes CPU or leaves local bottlenecks intact.Source: §1 · pp. 1–2
- The paper asks whether operator-level observations let a controller allocate CPU selectively while maintaining application performance.It formulates scaling as a POMDP controlled by PPO, with one-second relative CPU adjustments and a reward prioritizing p95 latency SLO compliance.Source: §1 · p. 2
- The evaluation tests lower SLO violation rates, lower CPU without higher violations, and workload-following throughput across two profiles and two workloads.The contributions include a task-level observation and joint-action design, an SLO-aware reward, and comparison with VRebalance.Source: §1 · p. 2
2. Related Work
Prior edge stream-scaling work spans heuristics, optimization, machine learning, and communication or control mechanisms. This paper focuses on whether joint task-level CPU adjustments can reduce p95 SLO violations while preserving throughput and avoiding unnecessary allocation on fixed placements.
2. Related Work
- Machine-learning approaches have modeled elastic parallelism with Markov decision processes and contextual bandits or combined model-based control with geo-distributed autoscaling.These approaches primarily target horizontal scaling and commonly optimize throughput or latency.Source: §2 · p. 2
- VRebalance is the closest baseline because it uses Bayesian Optimization to allocate CPU to existing workers from application-level feedback without changing parallelism.The gap addressed here is joint task-level CPU adjustment on fixed edge placements.Source: §2 · p. 3
- Most reviewed proposals change parallelism or placement, while resource consumption is not always considered alongside latency or throughput.Prior approaches include decentralized horizontal scaling and hierarchical coordination across operator, application, infrastructure, and region managers.Source: §2 · pp. 2–3
3. System Model and Problem Formulation
The system models a stream-processing application as a directed graph of tasks placed on heterogeneous edge servers. The problem is to adjust CPU while fixed placement and memory preserve an SLO-first objective.
3. System Model and Problem Formulation
- Tasks process streams on heterogeneous servers, each with CPU, processing, and RAM capacities; task placement remains fixed during an execution.Task CPU reservations can change over time.Source: §3 · p. 3
- Increasing a task's reserved CPU raises its processing capacity and lowers nominal per-event processing time, though latency effects may be delayed by queued events.Queues may exist locally or at downstream operators.Source: §3 · p. 3
- The latency measure is the 95th percentile of completed-event latency in each interval, compared with the threshold for the application profile.End-to-end latency includes queueing, network, and processing time.Source: §3 · p. 3
- Only CPU is modified online; memory demand and placement are fixed, and the objective prioritizes avoiding or recovering from p95 violations before reducing CPU.While compliant, the controller also avoids unstable reconfiguration and sustained throughput deficits.Source: §3 · pp. 3–4
4. Operator-Aware Vertical Autoscaler
The autoscaler uses a one-second observation–action–execution loop to adjust CPU across tasks while preserving topology and placement. Its POMDP observations, joint actions, and reward encode localized task pressure and an SLO-first control objective.
4. Operator-Aware Vertical Autoscaler
- At each one-second interval, the controller observes the application, selects a joint per-task CPU action, and receives a new observation and reward after simulated execution.Training updates PPO from transitions across simulator instances; evaluation uses a deterministic policy with fixed parameters.Source: §4 · p. 4, Figure 1 · p. 4
4.1. POMDP and Observation Space
- The POMDP observation combines task-, server-, and application-level information, allowing the policy to distinguish localized saturation from application-wide latency increases.There are 15 features per task, five per server, and 11 for the application; Table 1 groups features by normalization and control purpose.Source: §4.1 · p. 4, Table 1 · p. 5
4.2. Joint Action and Feasibility
- The joint action assigns a relative CPU adjustment to each task, enabling increases for bottlenecks and reductions elsewhere within task and server capacities.The action options are −500, −50, 0, 50, or 500 millicores; infeasible targets are projected onto the feasible set, with reductions processed before increases.Source: §4.2 · p. 5
4.3. SLO-Aware Reward
- The reward prioritizes p95 SLO compliance and recovery, then encourages CPU efficiency and stable reconfiguration while compliant.It combines seven components and is clipped to [−6, 2.5]; during violations, efficiency and CPU-reduction terms are zero, and its tuned coefficients encode a directional hierarchy rather than a guarantee.Source: §4.3 · pp. 5–6, Table 2 · p. 6
5. Experimental Methodology
The methodology evaluates vertical scaling in EdgeStreamPy across two stream-processing profiles and two replayed workloads. It compares PPO and VRebalance using matched placements and a repeated-run protocol.
5.1. Simulation Environment and Scenarios
- EdgeStreamPy simulates task processing, queues, network behavior, and CPU reallocation; migration and horizontal scaling are disabled.The infrastructure is a network graph of edge servers and switches, and each application is modeled as a task DAG.Source: §5.1 · p. 6
- PRED has two prediction branches, whereas ETL has nine tasks in a longer chain; their topologies are adapted from RIoTBench.The profiles run on seven heterogeneous edge servers, with task placements fixed during execution.Source: §5.1 · pp. 6–7, Figure 2 · p. 7
- TAXI uses 3,600 trace points, while INTEL’s 1,056 points are replayed cyclically to fill the 3,600-second evaluation horizon.The INTEL cycles are repeated observations of a constructed periodic workload, not independent experimental replications.Source: §5.1 · p. 7
- The p95 latency thresholds are 240 ms for ETL and 180 ms for PRED; violation rate counts intervals exceeding the profile threshold.Throughput is measured at the sink, and CPU allocation is summed across the profile’s tasks.Source: §5.1 · p. 7
5.2. Baseline, Training, and Protocol
- Independent PPO models were trained for PRED and ETL, each receiving 500,000 aggregate environment steps before a fixed policy was selected per profile.Training used a 352-point taxi segment and four scenarios per profile with varied arrival rates and placements.Source: §5.2 · p. 8
- Ten matched placement pairs per profile-workload combination compare complete controller configurations, with PPO updating every second and VRebalance every 60 seconds.The paired analysis uses percentile-bootstrap confidence intervals over ten complete pairs; VRebalance scores allocations using application p95 latency and CPU, with sink throughput selecting workload tier.Source: §5.2 · pp. 7–8
6. Experimental Results
PPO had lower p95 latency SLO violation rates than VRebalance in all four combinations, with every PPO run’s mean p95 latency below its profile-specific SLO. CPU allocations were lower in three scenarios, while throughput remained non-inferior in all four.
6.1. Tail Latency and SLO Mitigation
- PPO’s pointwise mean p95 latency stayed below its SLO throughout all four combinations, while VRebalance repeatedly exceeded thresholds, especially across PRED placements.Figure 3 shows temporal means over ten executions.Source: Figure 3 · p. 9, §6.1 · p. 9
- Every PPO execution had a mean p95 latency below its profile-specific SLO; VRebalance achieved this in every run only for ETL–INTEL.ETL–TAXI had higher run means despite lower violation rates than the PRED scenarios, because a few extreme intervals strongly affected the mean.Source: §6.1 · p. 9, Figure 4 · p. 10
- 0.30%, 0.03%, 0.10%, and 0.18% were PPO’s mean violation rates versus VRebalance’s 2.87%, 1.01%, 5.72%, and 8.73%, respectively.H1 was met in every combination; the descriptive difference ranged from 0.98 percentage points for ETL–INTEL to 8.55 for PRED–INTEL.Source: §6.1 · pp. 8–9, Figure 5 · p. 10
6.2. CPU Allocation Efficiency Under Compliance
- PPO used lower mean and median CPU in ETL–TAXI, ETL–INTEL, and PRED–TAXI, but its median was slightly higher in PRED–INTEL.The central-tendency differences were approximately 15,000–25,000 millicores in the three lower-allocation scenarios; H2 was not supported uniformly.Source: §6.2 · pp. 9, 11, Figure 6 · p. 11
6.3. Throughput Preservation
- PPO was non-inferior to VRebalance for throughput in all four combinations under the −5% margin, with no sustained throughput deficit.All lower confidence bounds exceeded the margin, and all paired differences remained within a ±5% practical-equivalence band.Source: §6.3 · p. 11
7. Conclusion and Future Work
The operator-aware PPO controller reduced SLO violations and preserved throughput, while CPU savings varied by scenario. The authors call for broader testing and evaluation beyond simulation.
7. Conclusion and Future Work
- The POMDP/PPO controller jointly adjusts per-task CPU from task, server, and application observations, prioritizing SLO compliance and recovery before resource reduction.Source: §7 · pp. 11–12
- Mean SLO violation rates were 0.03–0.30% for PPO versus 1.01–8.73% for VRebalance across four combinations; throughput was preserved, but CPU fell in only three.PRED–INTEL required slightly more CPU while its mean violation rate fell from 8.73% to 0.18%.Source: §7 · p. 12, Figure 7 · p. 12
- Future work should broaden seeds, profiles, workloads, topologies, and placements, ablate controller components, and test on emulated and physical edge environments.The proposed evaluations also include inference and actuation overheads, runtime effects, metric noise, and reallocation cost.Source: §7 · p. 12