Source-linked AI summary
Cloudburst: Stateful Functions-as-a-Service
Vikram Sreekanti, Chenggang Wu, Xiayue Charles Lin, Johann Schleier-Smith, Jose M. Faleiro, Joseph E. Gonzalez, Joseph M. Hellerstein, Alexey Tumanov
TL;DR
Current FaaS systems are optimized for isolated stateless functions and provide limited support for low-latency shared state, communication, and distributed consistency. Cloudburst addresses this gap with LDPC, autoscaling storage, colocated mutable caches, lattice-encapsulated state, and distributed session protocols. The paper reports practical stateful serverless computing with strong consistency guarantees and performance that rivals or beats baselines, while identifying isolation and fault tolerance as remaining challenges.
Problem
Current FaaS platforms poorly support shared state, communication, and distributed session consistency for applications beyond isolated stateless functions.
Method
Cloudburst combines logical disaggregation with physical colocation, Anna-backed autoscaling storage, mutable executor caches, lattice capsules, and distributed session-consistency protocols.
Results
Cloudburst demonstrates stateful serverless feasibility with performance that rivals and often beats baselines while providing repeatable-read and causal-consistency guarantees across function compositions.
Takeaways & Limitations
LDPC shows that autoscaling storage and compute can coexist with physically colocated state and coordination-free consistency for general-purpose stateful serverless programs.
Takeaways & Limitations
Cloudburst does not provide isolation between concurrent DAGs, and its standard failure model restarts failed DAGs while leaving non-idempotent side effects to programmers.
Abstract
from arXiv · showhide
Function-as-a-Service (FaaS) platforms and "serverless" cloud computing are becoming increasingly popular. Current FaaS offerings are targeted at stateless functions that do minimal I/O and communication. We argue that the benefits of serverless computing can be extended to a broader range of applications and algorithms. We present the design and implementation of Cloudburst, a stateful FaaS platform that provides familiar Python programming with low-latency mutable state and communication, while maintaining the autoscaling benefits of serverless computing. Cloudburst accomplishes this by leveraging Anna, an autoscaling key-value store, for state sharing and overlay routing combined with mutable caches co-located with function executors for data locality. Performant cache consistency emerges as a key challenge in this architecture. To this end, Cloudburst provides a combination of lattice-encapsulated state and new definitions and protocols for distributed session consistency. Empirical results on benchmarks and diverse applications show that Cloudburst makes stateful functions practical, reducing the state-management overheads of current FaaS platforms by orders of magnitude while also improving the state of the art in serverless consistency.
1. INTRODUCTION
Cloudburst extends serverless computing beyond isolated, stateless functions by combining autoscaling with low-latency shared state, communication, and consistency mechanisms. Its design colocates mutable state with function execution while preserving logical separation of storage and compute.
- Motivation: Current FaaS platforms provide autoscaling and usage-based pricing but work best for isolated, stateless functions.Their storage is high-latency, functions cannot communicate directly, and nested calls are slow.
- Motivation: Workarounds using fixed coordinators or Redis reintroduce scaling, fault-tolerance, and management problems.Examples include ExCamera’s coordinator and numpywren’s static Redis machine.
- Motivation: Stateful serverless computing targets low-latency services and distributed protocols requiring autoscaling, shared mutable state, and fine-grained communication.Examples include session-aware web services, forums, ad servers, distributed aggregation, membership, and leader election.
- Design: Cloudburst introduces logical disaggregation with physical colocation, keeping hot data near functions while retaining independent storage and compute scaling.The design allows updates at any invocation site and aims for wire-speed cross-function communication.
- Correctness: Colocating compute and data creates distributed session-consistency challenges when composed functions execute on different nodes.Each node can update local data independently, but multi-node compositions must preserve a consistent session.
- Contributions: Cloudburst combines Anna’s autoscaling key-value store and overlay routing with mutable executor-local caches, lattice-encapsulated state, and distributed session protocols.The contributions include LDPC architecture, repeatable-read and causal-consistency protocols, lattice capsules, and evaluation on stateful workloads.
2. MOTIVATION AND BACKGROUND
The motivation section identifies shared-state, composition, and communication limitations in current FaaS systems. It presents LDPC and coordination-free consistency as the architectural direction for addressing these limitations.
- Limitations: Current FaaS offerings are poorly suited to shared state, especially for latency-sensitive applications.The section focuses on function composition, direct communication, and shared mutable storage.
- Function Composition: Function invocation overheads can reach 20ms and compound linearly, making a five-function call stack approach interactive-service latency limits.The section argues that functional patterns for state sharing are therefore impractical on current FaaS platforms.
- LDPC: LDPC combines independently scalable storage and compute with physically nearby data and caches for low-latency access.A low-latency autoscaling KVS can provide global storage and overlay routing, while caches improve locality.
- Consistency: Coordination-free consistency fits elastic serverless membership, and Anna provides lattice-based consistency with associative, commutative, and idempotent merges.Anna also selectively replicates frequently accessed data as workloads change.
- Distributed Sessions: Cloudburst requires distributed session consistency because one request may invoke functions on separate machines, beyond Anna’s individual-client guarantees.The paper introduces protocols for multiple consistency levels and transparent state encapsulation for programmers.
3. PROGRAMMING INTERFACE
Cloudburst exposes vanilla Python functions, key-value access, message passing, and composable DAGs through a serverless programming interface. The runtime transparently handles references, serialization, consistency metadata, and result propagation.
- Python Interface: Cloudburst accepts vanilla Python functions that execute remotely while preserving regular-function syntax.Results can return synchronously to the client or be stored in the KVS and accessed through a CloudburstFuture.
- Arguments and State: Function arguments may be Python objects or KVS references, which the runtime retrieves and deserializes before invocation.The runtime attempts to place calls near cached referenced data.
- State Access: Programmers use Anna’s get and put operations for Python objects, with serialization and consistency encapsulation handled transparently.Executor caches make common-case state access fast.
- Compositions: Users register arbitrary function compositions as DAGs, with results automatically passed between successive functions.The final result is either stored in the KVS or returned directly, while resource management scales function replicas.
- Communication: The system API supports key-value operations and message passing between function invocations.Explicit messaging uses invocation identifiers and a deterministic mapping to establish executor connections.
4. ARCHITECTURE
Cloudburst implements LDPC with schedulers, executors, local caches, Anna, and resource management. Its architecture uses cache locality, consistency-aware freshness, metadata-driven scheduling, and workload-responsive autoscaling, while leaving advanced policy design for future work.
- Architecture: Cloudburst independently autoscales compute and Anna storage while mutable executor-side caches provide colocated low-latency access.The architecture uses logical disaggregation with physical colocation.
- Caching: Caches mediate executor-KVS access and asynchronously merge local updates into Anna.This design keeps frequently used data local while retaining Anna as persistent shared storage.
- Consistency: Caches publish cached-key snapshots so Anna can index which caches hold each key and direct freshness updates efficiently.This avoids the unnecessary KVS load caused by polling or blind timeout-based eviction in read-dominated workloads.
- Scheduling: Schedulers route requests to executors, cache functions and data locally, and coordinate DAG execution across selected nodes.DAG schedules are broadcast to participating executors after each function is assigned to a node caching it.
- Scheduling Policy: Scheduling uses cached-key metadata, executor load, and replication through backpressure to improve locality and avoid overloaded nodes.Hot keys and functions spread to additional executors when initially caching nodes become saturated.
- Autoscaling: Monitoring publishes metrics to Anna and scales DAG resources when request rates exceed completion rates or executor CPU utilization crosses 70%.New nodes read functions, metadata, and application state from Anna, which acts as the metadata source of truth.
- Limitations: The scheduling heuristics depend on current system dynamics, while advanced autoscaling policies remain future work.The paper identifies workload-infrastructure interactions as a direction for improving policy design.
5. CACHE CONSISTENCY
Cloudburst defines repeatable-read and causal consistency for function compositions spanning multiple executors, using lattice-encapsulated state and cache protocols to preserve cross-function consistency. Its protocols propagate read snapshots and causal dependencies because per-cache causal cuts alone are insufficient.
- Cloudburst scopes consistency to a function-composition DAG, whose reads and writes experience the selected consistency definition across function boundaries.
- Repeatable Read: Repeatable read ensures that functions in a linear DAG observe the same version of a key unless a later update occurs within the DAG.Cloudburst snapshots locally cached objects on first read and propagates cache addresses and version timestamps downstream.
- Causal Consistency: Causal consistency requires reads and writes to respect happens-before dependencies, while allowing concurrent functions within a DAG to read divergent versions.A function must not read a version older than any dependency established by previously read versions or ancestor functions.
- Lattice Encapsulation: Cloudburst encapsulates Python objects in Anna lattices, using LWW lattices by default and causal lattices with vector clocks, dependencies, and values in causal mode.Concurrent causal versions merge through vector-clock maxima and set union of dependencies and values.
- Distributed Session Protocols: Maintaining a causal cut in each cache is insufficient, so executors propagate causal dependencies and upstream caches store dependency snapshots.The downstream cache validates local versions against upstream snapshots and fetches a correct version when the local version is invalid.
6. EVALUATION
The evaluation studies Cloudburst’s mechanisms, consistency overheads, and two applications, using AWS EC2 and ElastiCache deployments in one availability zone.
- The evaluation covers microbenchmarks, consistency-mechanism overheads, machine-learning prediction serving, and a Twitter clone.
- All experiments ran in the us-east-1a AWS availability zone with schedulers on c5.large VMs and function executors on c5.2xlarge VMs.Function VMs used three cores for Python execution and one for the cache; Redis experiments used ElastiCache with two shards and three replicas per shard.
6.1 Mechanisms in Cloudburst
Cloudburst’s mechanisms target function composition, data locality, low-latency communication, and responsive autoscaling. Across these evaluations, Cloudburst reduces latency relative to serverless storage and invocation workarounds while retaining rapid resource adaptation.
- Function Composition: Cloudburst’s two-function composition latency remains roughly equal to single-function latency and is significantly faster than other measured systems.Cloudburst is about an order of magnitude faster than SAND at both median and 99th-percentile latency.
- Function Composition: Cloudburst matches state-of-the-art Python runtime latency and outperforms commercial serverless infrastructure by 1-3 orders of magnitude.AWS Step Functions was 82× slower than Cloudburst, while DynamoDB and S3 added 15ms and 40ms penalties, respectively.
- Data Locality: At 8MB, Cloudburst with cache hits improves median latency over Cloudburst Cold by 10×, Lambda on Redis by 25×, and Lambda on S3 by 79×.At 80MB, Cloudburst Hot remains 9× faster than Cloudburst Cold and 24× faster than S3.
- Low-Latency Communication: Cloudburst’s gossip-based aggregation is 3× faster than Lambda and DynamoDB gather, while Cloudburst gather is 22× faster than Redis and 53× faster than DynamoDB.Gossip is also about 10% faster than Redis gather at the median and 40% faster at the 99th percentile.
- Autoscaling: Cloudburst scales throughput from about 3,300 to 5.6K and 6.7K requests per second as resources increase, then reduces allocated threads within 20 seconds after demand drops.The implementation is bottlenecked by the latency of spinning up EC2 instances.
- Autoscaling: Cloudburst’s autoscaling policies quickly detect and react to workload changes, while high EC2 instance startup costs remain the primary limitation.The paper identifies policy and instance-startup improvements as compatible with the existing architecture.
6.2 Consistency Models
Cloudburst’s consistency models trade higher tail latency for stronger guarantees across function compositions. Median latency remains nearly uniform, while distributed session causal consistency incurs the greatest overhead and prevents anomalies that weaker models expose.
- Latency overheads: Median latency is nearly uniform across Cloudburst’s five consistency modes, but stronger modes have higher 99th-percentile latency.The evaluated modes are LWW, DSRR, single-key causality, multi-key causality, and distributed session causal consistency.
- Latency overheads: 1.8× higher 99th-percentile latency for DSRR than LWW results from exact version matching and remote fetches after cache-version mismatches.
- Latency overheads: Vector clocks and dependency metadata increase tail retrieval latency for single-key and multi-key causality, especially for hot keys.
- Latency overheads: A five-function DAG can require four extra version-snapshot round trips, producing 1.7× higher 99th-percentile latency than single- and multi-key causality and 9× higher latency than LWW.
- Consistency outcomes: Cloudburst’s stronger consistency models prevent anomalies that arise under weaker models, while median latency remains over an order of magnitude faster than DynamoDB and S3 for similar tasks.
6.3 Case Studies
Cloudburst supports practical stateful applications across prediction serving and social-network workloads. Its applications achieve smooth scaling, competitive latency, and modest overheads relative to native Python, managed services, and Redis baselines.
- Prediction serving: Cloudburst prediction serving is about 15ms slower than native Python at the median, while AWS Sagemaker is 1.7× slower than native Python and 1.6× slower than Cloudburst.The comparison uses a three-stage MobileNet image-classification pipeline running on CPUs.
- Prediction serving: Throughput scales linearly from 10 to 160 worker threads, with minimal 95th-percentile latency growth between 80 and 160 executors.Median and 99th-percentile latency rise from 10 to 20 workers, then show no significant change through 160 executors.
- Prediction serving: Cloudburst prediction serving provides smooth scaling and low, predictable latency comparable to native Python while outperforming a purpose-built commercial service.
- Retwis: Retwis in Cloudburst’s LWW mode has median and 99th-percentile latencies 27% and 2% higher than Redis, respectively.
- Retwis: Causal Retwis adds 4% median and 20% tail overhead over LWW, while scaling from 10 to 160 executor threads increases both latency percentiles by about 60%.The increase is attributed to more new tweets causing timeline reads to query the key-value store for new data.
- Retwis: Adapting Retwis was straightforward, adding modest overhead to Redis while scaling smoothly as workload increases.
7. RELATED WORK
Related work spans client-side caching, faster serverless execution, datacentric services, serverless infrastructure, storage systems, and language-level consistency. Cloudburst differs by combining autoscaling, colocated caching, and distributed session consistency for stateful serverless programs.
- Architecture: Prior client-side caching work primarily targets strong transactional consistency for static or slowly changing configurations, whereas Cloudburst pursues colocated caching for serverless workloads.
- Serverless platforms: SOCK, gVisor, and Firecracker address function provisioning, sandboxing, or startup overheads and are complementary to Cloudburst’s design.
- Serverless applications: Starling, PyWren, ExCamera, and related systems build datacentric or highly parallel applications on stateless serverless infrastructure, rather than providing Cloudburst’s stateful serverless model.
- Serverless infrastructure: Archipelago focuses on scheduling DAGs to meet per-request latency deadlines, complementing Cloudburst’s focus on state and consistency.
- Serverless storage: Shredder pushes functions into storage but is limited to a single node and does not address autoscaling or other characteristic serverless features.
- Language-level consistency: Cloudburst implements causal consistency in caches with per-key dependency metadata, avoiding cross-cache coordination but incurring dependency metadata overhead.
8. CONCLUSION AND FUTURE WORK
Cloudburst demonstrates the feasibility of general-purpose stateful serverless computing through logical disaggregation, physical colocation, lattice-based storage, and distributed session consistency. The design performs well, but isolation, fault tolerance, autoscaling policy, and streaming remain future-work areas.
- Conclusion: Cloudburst demonstrates stateful serverless computing by logically disaggregating storage and compute while physically colocating caches with compute services.
- Conclusion: Lattice capsules enable asynchronous merging of opaque state in coordination-free storage, while distributed session protocols provide consistent correctness across caches.
- Conclusion: Cloudburst provides stronger state guarantees than commercial FaaS backing storage while rivaling or exceeding inelastic server-centric baselines.
- Future work: Future work includes transactional isolation and atomicity, improved autoscaling mechanisms and policies, and streaming services.