PaperlayerView exact revision on arXiv

Source-linked AI summary

CLASP: Chained-Request-Aware Scaling and Operator Placement for Serverless Stream Processing

Tianyu Qi, Maria A. Rodriguez, Rajkumar Buyya

arXiv:2608.29103v1cs.DC

TL;DR

Stateful serverless stream processors must scale operator parallelism and placement while accounting for chained-request overhead, which existing approaches may omit. CLASP estimates both execution and chained-request costs, packs operators onto the fewest sustainable workers, and migrates state with instances during scaling. Compared with FaaSFlow and Zhang24, it improves throughput by up to 3.3 × and reduces end-to-end latency by up to 76%.

  • Problem

    Existing scaling approaches overlook chained-request capacity costs, misestimating worker requirements and potentially harming throughput or latency.

  • Method

    CLASP uses observed metrics to estimate execution and chained-request costs, packs projected workloads onto workers, and migrates operator state with instances during scaling.

  • Results

    Up to 3.3 × throughput improvement and 76% end-to-end latency reduction were achieved over FaaSFlow and Zhang24.

  • Takeaways & Limitations

    CLASP provides elastic scaling and scheduling for stateful serverless stream processing while minimizing worker use and accounting for chained-request costs.

  • Takeaways & Limitations

    Slot-based placement methods can provision too few workers when chained-request cost is not included in capacity modeling.

Abstract

from arXiv · show

Stateful serverless (Function-as-a-Service) environments, whose workers host state servers, are increasingly used for stream processing. A stream application is a pipeline of operators, where each operator forwards intermediate data downstream through a chained request. As input rates fluctuate, the system should adjust operator parallelism and place instances across workers to sustain the incoming rate. Existing approaches do so without fully accounting for chained-request overhead, leading them to misestimate the required number of workers. Too few leave the cluster unable to keep up with the input rate, while too many route a larger fraction of chained requests across worker boundaries, increasing end-to-end latency. We propose CLASP, a scaling and scheduling strategy for stream processing in stateful serverless environments. At runtime, CLASP estimates execution cost and chained-request cost from observed metrics. Under a capacity model that covers the two costs, it adjusts operator parallelism and packs operators onto the fewest workers that can sustain the target input rate. Once a scaling decision is made, CLASP migrates each operator's state together with its instances, thereby minimizing execution pause time. Experiments show that CLASP improves throughput by up to 3.3x and reduces median end-to-end latency by up to 76% compared with state-of-the-art scaling strategies.

1 Introduction

CLASP addresses elastic scaling and operator placement for stateful serverless stream processing by modeling both execution and chained-request costs. It packs workloads onto the fewest workers that sustain the input rate and coordinates state migration during scaling.

  • Motivation: Existing scaling approaches tune operator parallelism but may overlook worker placement and the capacity consumed by chained requests.Slot-based models account for function execution while omitting scheduling and dispatch activity, which can saturate workers unexpectedly.
  • Motivation: Too few workers cannot sustain the input rate, whereas too many increase cross-worker chained requests and end-to-end latency.The placement problem also requires respecting worker capacity and co-locating heavily communicating operators.
  • Limitations of Existing Approaches: Slot-based placement can provision too few workers because it ignores the chained-request cost induced by a new placement.Other approaches may spread instances across more workers using user-specified weights without quantifying local versus remote request costs.
  • CLASP: CLASP estimates worker capacity online from observed metrics, accounting jointly for execution workload and chained-request cost.When scaling triggers, it packs projected workloads onto workers while evaluating the chained-request cost under the new placement.
  • CLASP: CLASP migrates operator instances, state, and in-flight requests in a coordinated transition so execution resumes with state co-located at the new placement.This avoids gradual convergence through repeated rebalancing rounds and minimizes processing interruption.
  • Evaluation: Up to 3.3 × throughput improvement and 76% end-to-end latency reduction were reported over FaaSFlow and Zhang24.The method was implemented on Faasm.

2 Background and Motivation

Stream processing in stateful serverless systems combines concurrent operator execution with local or remote chained-request and state-access costs. Experiments show that executor slots alone misestimate capacity, while excess workers reduce throughput scaling and increase latency; local state access is therefore important.

  • Capacity model: Workers have finite executor slots, but scheduling and dispatching chained requests also consume capacity.Consequently, slot count alone does not determine the workload a worker can sustain.
  • Execution model: A stream application is a DAG of concurrently executing operators whose instances forward data through chained requests.Each request invokes an operator, which can generate downstream requests dispatched by local schedulers to local or remote workers.
  • Capacity model: Existing placement approaches assign instances by available slots while overlooking chained-request communication overhead across workers.This can make workers appear capable of handling more workload than they can actually sustain.
  • Worker scaling: 3.7× and 5.5× are SYN’s maximum throughput relative to one worker at 5 and 10 workers, respectively, showing markedly sub-linear scaling.The uniform synthetic pipeline controls for workload imbalance, while added workers increase cross-worker chained requests.
  • Worker scaling: 2.3× and 1.5× are SYN’s p95 end-to-end latency increases relative to one worker at 10 and 5 workers, respectively.Across applications, smaller clusters consistently have lower p95 latency because fewer requests cross worker boundaries.
  • State access: 1400 μs is the execution latency for a stateful operator with remote state access, versus around 400 μs with local access or for stateless execution.Local access also yields about 3.5× higher throughput for WC and about 1.5× for PL, corresponding to 90% and 20% state-access rates.

3 Related Work

Related work tunes resources, parallelism, or placement, but existing approaches generally omit an explicit model of local versus remote chained-request costs. CLASP instead combines runtime cost estimation, cost-aware placement, worker minimization, and coordinated state migration.

  • Vertical resource provisioning: Vertical provisioning methods adjust executor resources or function configuration, primarily optimizing cost, latency, or performance bounds.These methods do not address the placement problem described for fixed worker capacity.
  • Batch versus stream processing: Batch-processing methods can exploit stage-by-stage execution to keep chained requests local, unlike continuously concurrent stream processing.In stream processing, downstream operators are already running, so requests must be routed immediately.
  • Platform-managed placement: Platform-managed placement methods cannot evaluate placement-dependent chained-request costs or explicitly minimize worker count.Placement is delegated to the underlying serverless platform.
  • Self-managed placement: FaaSFlow co-locates heavily communicating operators but measures worker capacity only by executor count, overlooking chained-request cost.This can cause resource imbalance across workers.
  • Self-managed placement: Zhang24 scores data affinity and load balancing, but does not quantify local versus remote chained-request costs and relies on user-specified weighting.CLASP estimates both costs from runtime observations and accounts for them during assignment while minimizing workers used.
  • State migration: CLASP derives one migration plan from placement differences and moves every operator’s complete state with its instances.This contrasts with incremental schemes that repeatedly migrate part of one operator’s state after imbalance triggers.

4 CLASP Design

CLASP continuously monitors runtime behavior, models execution and chained-request overhead, and jointly determines worker count, operator placement, and parallelism. It packs communicating operators onto the fewest capacity-satisfying workers and migrates state with queued requests during reconfiguration.

  • Architecture and runtime cycle: CLASP runs a monitor–predict–place–migrate cycle that collects metrics, projects load, computes placement, derives operator parallelism, and migrates state with queued requests.Workers and the planner report queue, execution, throughput, input-rate, and chained-request metrics to support scaling decisions.
  • Worker capacity model: The worker capacity model sums execution cost with local and remote chained-request overheads, including the number of distinct remote destination workers.Online estimation recovers the overhead coefficients and maximum worker capacity from saturated runtime observations.
  • Scaling policy: CLASP scales to the fewest workers that can sustain the current input rate while balancing saturation, latency, communication overhead, and resource use.The planner continuously checks whether the worker pool can sustain the input rate and scales out or in accordingly.
  • Operator placement: Instead of choosing each operator’s parallelism independently, CLASP places operators using processing cost and communication patterns, then derives their parallelism.Projected rates and execution times feed a bin-packing procedure that can split operator demand across workers.
  • Operator placement: The placement model co-locates heavily communicating operators, accounts for local versus remote routing, and enforces each worker’s maximum capacity.Reverse-topological placement exposes successor locations before assigning an operator; the implementation is decentralized without a global synchronization barrier or central coordinator.

5 Performance Evaluation

CLASP extends Faasm with queues, distributed scheduling, and per-worker monitoring to support stateful stream processing and scaling.

  • Implementation: CLASP adds incoming request queues, a distributed scheduling engine, and a metrics monitor on every worker and the planner.Locally scheduled chained requests use shared memory to reduce communication latency.

5.1 Experiment Setup

The evaluation uses an 11-node Kubernetes cluster, compares CLASP with FaaSFlow and Zhang24, and measures throughput and end-to-end latency across repeated runs.

  • Baselines: CLASP is compared against FaaSFlow’s invocation-rate profiling and greedy packing, and Zhang24’s nonlinear performance model with affinity-aware placement.Zhang24 scores data affinity and load balancing when assigning instances to workers.
  • Evaluation procedure: Each experiment runs five times for ten minutes, and reported results are averages across the five runs.The setup remaps operators after a 10,000-request warm-up phase before recording performance.
  • Evaluation procedure: Scaling is triggered when the input rate changes consistently for 2 seconds or throughput fails to match input rate for 3 seconds.CLASP initializes its online coefficients near values that converge across applications and cluster sizes.
  • Metrics: The primary goal is sustaining the input rate with the minimum number of workers, while throughput counts end-to-end requests processed per second.End-to-end latency includes scheduling and execution time for a request and all subsequent chained requests.

5.2 Migration Cost

Migration experiments measure planner and worker durations across applications and worker transitions. The decentralized mechanism generally resumes processing within 10 ms, while larger states and concentrated scale-out transfers take longer.

  • Measurement: Migration experiments trigger six worker transitions—1→5, 5→10, 1→10, and their reverses—and distinguish planner duration from worker duration.Planner duration runs from issuing the plan to receiving completion messages; worker duration begins when a worker receives the plan.
  • Migration results: Scaling up takes longer than scaling down on average because data spreads across more destination workers and each destination waits for all sources.Transitions from one worker to five or ten take especially long because one source worker dispatches to every destination.
  • Migration results: Across most applications and transitions, workers resume processing within 10 ms, indicating a short execution pause from decentralized migration.This measures the pause before workers resume processing, rather than total planner or worker migration duration.

5.3 Online Estimation of the Worker Capacity Model Parameters

CLASP estimates worker-capacity-model coefficients online from saturated observations, revealing that chained-request communication contributes substantially to worker load. The model distinguishes local and remote request costs and fan-out overhead from execution cost.

  • Online estimation: Online linear regression updates α, β, and γ from saturated observations after initializing them to 100, 1000, and 100000.The experiments run applications from 1 to 10 workers under unthrottled input.
  • Estimated costs: A local chained request costs about one-seventh as much as a remote one, reflecting the overhead of crossing worker boundaries.The reported local coefficient is α = 38.5, compared with β = 275 for remote dispatch.
  • Estimated costs: γ = 51500 makes opening a chained request to one additional remote worker roughly as expensive as executing 130 requests.Fan-out becomes the dominant cost when an operator’s chained requests are scattered across many workers.
  • Worker capacity: A worker’s smoothed maximum capacity is estimated at around 850000, corresponding to roughly 2000 local requests per second without communication cost.The capacity estimate is obtained via Equations (3)–(4).

5.4 Operator Placement

CLASP’s operator placement accounts for chained-request costs while packing operators across workers. On ten workers, it achieves higher throughput than FaaSFlow and Zhang24 across all applications.

  • Overall comparison: CLASP achieves higher throughput than FaaSFlow and Zhang24 across all applications when running on ten workers.The comparison uses unthrottled input.
  • Baseline behavior: FaaSFlow’s unaccounted chained-request cost leaves workloads imbalanced and reduces throughput, most prominently for WC.WC’s first operator emits more than ten chained requests per processed request.
  • Baseline behavior: WC suffers the largest throughput drop when placement packs more instances onto a worker than that worker can sustain.The overloaded worker hosts the affected instances.
  • Baseline behavior: Zhang24’s workload balancing scatters successor instances across workers, causing a worker to send data to nearly every other worker.This placement incurs heavy fan-out cost.

5.5 Worker Number Prediction

CLASP predicts worker counts close to the observed optimum across target rates and applications. Its error remains small and stable, including under large input-rate changes.

  • Prediction accuracy: CLASP closely tracks the observed optimal worker count across target input rates, with only minor deviation.The predicted count fluctuates slightly around the observed optimum.
  • Prediction accuracy: Across applications, absolute worker-count error stays below 0.25 workers relative to the observed optimal worker count.This result is reported across applications in Figure 11b.
  • Input-rate changes: Prediction error stays below 0.3 workers across input-rate gaps, including the largest 90% gap.Accuracy remains stable under large, abrupt input-rate changes.
  • Input-rate changes: CLASP scales more accurately when scaling down than when scaling up.The passage reports this directional difference without quantifying it further.

5.6 End-to-End Performance

CLASP is evaluated against scaling and placement baselines under burst and gradual input-rate changes. It consistently delivers higher throughput and lower latency, with gains attributed to both scaling and placement.

  • End-to-end results: CLASP achieves up to 2.5× FaaSFlow’s throughput and 3.3× Zhang24’s throughput, while reducing median latency by up to 75% and 76%, respectively.These results hold across burst and gradual modes in the reported Figure 14 comparison.
  • End-to-end results: CLASP consistently achieves higher throughput and lower latency than FaaSFlow and Zhang24 in both burst and gradual mode.The reported improvement comes from both the scaling method and operator placement.
  • Placement ablation: FaaSFlow-SP under-provisions at high input rates, lowering throughput and increasing latency despite sharing CLASP’s placement.Its prediction accuracy degrades when the source rate is far from the target rate.
  • Placement ablation: CLASP remains robust in burst and gradual modes, unlike FaaSFlow-SP’s lower burst-mode throughput when source and target rates differ substantially.The comparison attributes FaaSFlow-SP’s degradation to worker-count prediction accuracy.
  • Placement ablation: Against Zhang24-SP, CLASP achieves similar throughput while sustaining it with fewer workers.Both methods provision enough resources to match the input rate, but CLASP uses fewer workers.

6 Conclusions and Future Work

CLASP combines scaling, worker placement, and state migration for stateful serverless stream processing. Its evaluation reports higher throughput and lower end-to-end latency, while future work targets broader cluster and objective settings.

  • CLASP Design: CLASP jointly models execution and chained-request costs when scheduling operators onto workers.The capacity model supports placements that sustain the input rate while minimizing workers used.
  • CLASP Design: CLASP migrates operator state and pending requests according to new placements to minimize processing pauses during rescheduling.
  • Evaluation: CLASP improves throughput and reduces end-to-end latency against state-of-the-art scaling strategies under burst and gradual input-rate changes.
  • Future Work: Future work extends CLASP to heterogeneous clusters and SLO-aware objectives beyond minimizing worker count.
Loading 2608.29103v1…