A Unified Queueing-Based Framework for Comparative Evaluation of Serverless Autoscaling Strategies under Bursty Workloads

1. Introduction

• Bursty, hard-to-predict demand can amplify queueing near saturation, while activation delays, concurrency limits, and finite resources constrain capacity response.Delayed or unstable scaling can sustain high utilization, intensify cold-start overheads, contention, and queue accumulation, and degrade latency.Source: §1 · p. 1
• Comparisons remain difficult because workload characteristics and platform constraints affect observed behavior, while commercial platforms expose limited internal scaling details.The paper frames this as a barrier to detailed analysis and independent validation.Source: §1 · p. 1
• The framework combines QN models and QPN representations to compare SPR, CLS, and MBS under an H2 arrival process.It represents FI availability, per-instance concurrency limits, and queue accumulation under constrained processing capacity.Source: §1 · pp. 1–2
• The assessment examines throughput, mean response time, in-system population, and system power to characterize trade-offs between scaling responsiveness and stability.The paper describes this as a QoS-oriented performance assessment approach.Source: §1 · p. 2

2.1. Autoscaling in Serverless and Microservices

• Serverless and microservices autoscaling studies include predictive, reinforcement-learning, hybrid, and optimization-based mechanisms, often evaluated through experiments or workload traces.Examples include Knative scaling, RL-based policies, and methods combining horizontal and vertical scaling.Source: §2.1 · p. 2, §2.2 · p. 3

2.2. Autoscaling with Analytical and Stochastic Modeling

• Analytical and stochastic modeling work covers capacity planning, simulation, Petri nets, queueing networks, and probabilistic evaluation, often for systems or policies other than comparative FaaS autoscaling.The review includes microservices, storage, and IaaS studies alongside autoscaling models.Source: §2.2 · p. 3
• Relatively few studies compare distinct FaaS autoscaling strategies under common workload and system assumptions while jointly considering burstiness, concurrency, capacity, and queue buildup near saturation.The review argues that a unified framework can support more consistent and reproducible comparisons under dynamic workloads.Source: §2.2 · p. 3

3. Background

• A single-station M/M/∞ QN serves as the analytical baseline for SPR, idealizing each request as immediately receiving processing capacity without queueing.QN models interconnected service stations and supports analysis of throughput, response time, utilization, and population under different workload and capacity conditions.Source: §3 · p. 4
• Petri nets represent system states through places and tokens, with transitions changing those states to model concurrency, synchronization, resource sharing, and conditional behavior.Token distributions define the current marking, or state, of the model.Source: §3 · p. 4
• QPN combines Petri-net state representation with queueing and service behavior, providing the basis for modeling autoscaling dynamics.In this study, QN establishes the analytical baseline, while QPN represents conditional activation and resource-state changes alongside queueing and service dynamics.Source: §3 · p. 4

4. System Description

• SPR assigns each request to an available FI and activates another when none is idle; each FI handles at most one request at a time.After service, an FI remains warm for a predefined idle period before release if no new request arrives.Source: §4 · p. 5
• CLS assigns requests to FIs with available concurrency capacity and activates another FI when active capacity is exhausted.Completed requests free concurrency for reuse, while idle FIs are released after a retention period.Source: §4 · p. 5
• MBS monitors system metrics and, in this study, uses utilization thresholds to trigger FI activation or deactivation as demand changes.The approach also includes temporary retention of idle FIs before release.Source: §4 · p. 5

5.1. Analytical and QPN-Based Autoscaling

• The three strategies use complementary models: analytical queueing networks and queueing Petri nets for SPR, and queueing Petri nets for CLS and MBS.The representations capture SPR instance-level capacity management, CLS state-dependent concurrency control, and MBS metric-driven scaling and dispatching.Source: §5.1 · p. 6

5.1.1. SPR Model

• SPR uses an M/M/∞ analytical baseline and a QPN with parallel FI branches, where capacity tokens govern request admission and return after service.The analytical baseline assumes requests never wait for available FIs; the QPN makes request flow and instance capacity explicit.Source: §5.1.1 · pp. 6–7, Figure 2 · p. 6

5.1.2. CLS Model

• CLS uses concurrency tokens to admit requests and inhibitor arcs to direct them to later FI branches only after preceding capacity is exhausted.Completed requests release their tokens, restoring branch capacity and directing requests back to the earliest FI with available concurrency.Source: §5.1.2 · pp. 7–8, Figure 3 · p. 7

5.1.3. MBS Model

• MBS integrates metric-driven scaling and request dispatch in a Router that adjusts active FIs using thresholds and distributes requests across them.An idle FI is removed only after a predefined observation period without a new assigned request.Source: §5.1.3 · pp. 8–9, Figure 4 · p. 8

5.2. Evaluation Metrics

• The evaluation defines throughput, mean system response time, in-system population, and system power to assess responsiveness, usage, and stability under varying workloads.System power is the ratio of overall throughput to mean system response time.Source: §5.2 · p. 9

• The evaluations use common experimental settings before presenting results for SPR, CLS, and MBS and comparing the strategies.Source: §6 · p. 9

• The H2 workload uses mixing probability p=0.2 and inter-arrival-time squared coefficient of variation C²_a≈2.125.This corresponds to C_a≈1.457 and is described as reflecting bursty workload.Source: §6.1 · p. 9
• In the JMT What-if analysis, λ controls the mean H2 arrival rate while the mixing probability and arrival-process variability remain fixed.The underlying phase rates are recalculated as λ changes.Source: §6.1 · p. 9
• Reported workload levels are mean arrival rates, but the probabilistic H2 process can produce closely spaced arrivals and short-term bursts.Source: §6.1 · p. 9

6.2.1. Analytical Baseline and Model Validation

• The unlimited-capacity M/M/∞ baseline assumes Poisson arrivals and exponential service, with each arrival immediately entering service and no queueing delay.Source: §6.2.1 · pp. 9–10
• Simulation closely reproduces the analytical baseline: throughput’s maximum relative deviation is approximately 0.3%, and response-time relative error is approximately 0.49%.At λ_in=15 req/s, the analytical mean population is approximately 5 requests.Source: §6.2.1 · p. 10
• Analytical system power rises approximately linearly to 45 at λ_in=15 req/s, and simulation follows the same trend.Source: §6.2.1 · p. 10
• The agreement across throughput, response time, population, and power validates the unlimited-capacity SPR implementation as an ideal reference for the finite-capacity model.Source: §6.2.1 · p. 10

6.2.2. Capacity-Limited SPR Performance

• The capacity-limited QPN model uses 15 active FIs, each serving at most one request at a time, under the common H2 workload.Service times are exponential with μ=3 req/s; simulations use up to 500,000 samples and average 10 runs.Source: §6.2.2 · p. 10
• At λ=15 req/s, mean response time reaches approximately 62.37 seconds after a practical breaking point near λ=14.5 req/s.The capacity-limited model remains close to the ideal baseline at low and moderate workloads, but deteriorates in the high-load region.Source: §6.2.2 · pp. 10–11
• At λ=15 req/s, the capacity-limited SPR model averages approximately 416.28 requests in the system versus about 5 in the unlimited baseline, as queueing delay rises near capacity.Throughput remains close to incoming load, while requests accumulate in the Arriving Requests place.Source: §6.2.2 · pp. 10–11, Figure 5 · p. 11
• System power peaks at approximately 21.1 at λ=12.5 req/s, then declines as response-time growth outweighs throughput gains.The throughput–latency trade-off therefore worsens before the most severe queue accumulation near the capacity limit.Source: §6.2.2 · pp. 10–11

6.3.1. Dynamic QPN Performance Evaluation

• Under the dynamic H2 workload, FI1 initially handles most requests near its concurrency limit; FI2 and then FI3 are activated as demand increases.Around t=125–145 seconds, FI2 response time reaches approximately 11 seconds while FI3 becomes active.Source: §6.3.1 · pp. 11–12, Figure 6 · p. 12
• The QPN model captures time-dependent concurrency use, but sustained bursts can occupy multiple FIs and increase response time.Source: §6.3.1 · p. 12

6.3.2. Wide vs. Deep Concurrency Configurations

• With the same 12-token budget, M1–M4 range from fewer FIs with more tokens per FI to more FIs with fewer tokens per FI.The service-time characteristics remain unchanged across configurations.Source: Table 3 · p. 12, §6.3.2 · p. 12
• M4 follows offered workload over almost the entire evaluated range, while M1 reaches an effective throughput ceiling of approximately 6 req/s.Wider configurations have more FIs and higher aggregate nominal processing rates, postponing throughput saturation.Source: §6.3.2 · p. 12
• M4 reaches the highest CLS system-power peak, approximately 19.9 at λ=10 req/s, among the reported configuration peaks.The reported peaks for M1, M2, and M3 are approximately 5, 9.86, and 14.9, respectively.Source: §6.3.2 · p. 12
• At λ=12 req/s, CLS configurations M1–M4 have system populations of approximately 72,360, 30,880, 8,110, and 145 requests, respectively.M4 exhibits the most controlled behavior over the evaluated range; Fig. 7 compares population, response time, throughput, and power.Source: §6.3.2 · pp. 12–13, Figure 7 · p. 13

6.4.1. Utilization Threshold Sensitivity

• Lower utilization thresholds activate FIs earlier, preserving more headroom for bursts; higher thresholds delay activation and use existing capacity more fully.At U_th = 0.7 capacity expands earlier, while 0.9 keeps active FIs closer to their utilization limit before adding capacity.Source: §6.4.1 · p. 13, Figure 8 · p. 14

6.4.2. Routing Policy Sensitivity

• At higher loads, round-robin keeps system population below random and probabilistic routing and yields lower mean response time: 23.37 seconds versus 33.0 and 71.1 seconds at λ = 15 req/s.The three routing policies have similar response times at low arrival rates.Source: §6.4.2 · p. 14
• Throughput remains close across routing policies, while system-power curves peak near λ = 10.5 req/s and decline as response time rises.Power differences become more visible as response time increases.Source: §6.4.2 · pp. 13–14
• Round-robin with U_th = 0.7 is selected for the final MBS comparison; FI activation and request redistribution contribute to local population and response-time variations.Activating an FI temporarily distributes requests across more instances and reduces queueing; the bursty H2 workload can also contribute to variation.Source: §6.4.2 · pp. 13–14

6.5. Comparative Evaluation of Autoscaling Strategies

• The evaluation varies λ from 0 to 15 req/s; SPR can activate up to 15 FIs, CLS uses five FIs with three concurrency tokens each, and MBS uses U_th = 0.7 with round-robin routing.Population and mean-response-time plots are displayed only through λ = 12.5 req/s; higher-load results are reported in the discussion.Source: §6.5 · p. 15
• At λ = 15 req/s, system population reaches approximately 416.3 requests for SPR, 15,300 for CLS, and 315.1 for MBS.SPR has the lowest population through most of 0 ≤ λ ≤ 12.5 req/s; CLS accumulates requests earlier under its fixed concurrency configuration.Source: §6.5 · p. 15, Figure 10 · p. 16
• At λ = 15 req/s, throughput is approximately 14.92 req/s for SPR, 14.82 req/s for MBS, and 12.9 req/s for CLS.All strategies track offered workload closely at low and moderate loads; CLS levels off earlier near the upper workload limit.Source: §6.5 · p. 15, Figure 10 · p. 16
• SPR offers strong low-to-moderate-load performance, CLS is more sensitive to concurrency exhaustion, and MBS trades peak performance for more controlled behavior near capacity limits.At λ = 15 req/s, MBS has a mean response time of approximately 23.27 seconds, versus 62.4 seconds for SPR and 3240 seconds for CLS.Source: Figure 10 · p. 16, §6.5 · p. 16
• At λ = 15 req/s, mean response time is approximately 62.4 seconds for SPR, 3240 seconds for CLS, and 23.27 seconds for MBS.SPR has the lowest response time through most of the low-to-moderate workload range, while its delay deteriorates near the finite FI boundary.Source: §6.5 · pp. 15–16, Figure 10 · p. 16
• System-power peaks are approximately 21.1 for SPR near λ = 12.5 req/s, 16.5 for CLS near λ = 10.5–11 req/s, and 11.19 for MBS.The power comparison reinforces that no strategy dominates across all operating conditions.Source: §6.5 · p. 16, Figure 10 · p. 16

7. Conclusion and Future Work

• The H2 workload and analytical queueing and QPN models capture workload variability and the strategies’ different scaling mechanisms.For SPR, the M/M/∞ model provides an analytical reference for its simulation model.Source: §7 · p. 16
• With finite processing capacity, SPR maintains low response time over much of the workload range but deteriorates rapidly as available FI capacity is approached.This conclusion is consistent with the comparative evaluation’s response-time results.Source: §7 · p. 16
• For CLS, performance depends on how a fixed concurrency-token budget is distributed; more independent FIs delayed queue buildup in the wide-versus-deep experiments.Source: §7 · p. 16
• For MBS, the utilization threshold sets FI activation timing and routing affects request distribution; round-robin was the most controlled evaluated policy, with U_th = 0.7 used in the final comparison.Source: §7 · pp. 16–17
• The strategies trade off responsiveness, concurrency control, capacity, and robustness to workload variability; H2 variability can contribute to queue growth and QoS degradation before nominal capacity is reached.Future work proposes heterogeneous workloads and FI demands, shared-resource contention, resource cost, predictive mechanisms, and comparison against platform measurements.Source: §7 · p. 17