PaperlayerView exact revision on arXiv

Source-linked AI summary

Faasm: Lightweight Isolation for Efficient Stateful Serverless Computing

Simon Shillaker, Peter Pietzuch

arXiv:2002.09344v2cs.DC

TL;DR

Data-intensive serverless workloads suffer from repeated data movement and container overheads because functions cannot directly share memory. The paper introduces Faaslets and the FAASM runtime, combining WebAssembly-based isolation with shared state and snapshot restoration. Compared with container-based deployment, FAASM improves training, inference throughput, and tail latency while reducing memory and network costs.

  • Problem

    Container-based serverless platforms prevent direct memory sharing, forcing data duplication and serialisation that add performance and resource costs.

  • Method

    Faaslets use WebAssembly-based software-fault isolation in a shared address space, while FAASM provides resource isolation, two-tier state access, POSIX facilities, and snapshot-based restoration.

  • Results

    FAASM outperforms a container-based deployment across machine-learning training and inference, including 60% shorter training runtime, over 200% higher maximum inference throughput, and 90% lower inference tail latency.

  • Takeaways & Limitations

    Faaslets provide a serverless execution model that combines memory safety and resource fairness with shared in-memory state and distributed scaling.

  • Takeaways & Limitations

    WebAssembly alone lacks mechanisms for dynamically sharing memory, and the Faaslet host interface operates outside memory-safety bounds and must be trusted to preserve isolation.

Abstract

from arXiv · show

Serverless computing is an excellent fit for big data processing because it can scale quickly and cheaply to thousands of parallel functions. Existing serverless platforms isolate functions in ephemeral, stateless containers, preventing them from directly sharing memory. This forces users to duplicate and serialise data repeatedly, adding unnecessary performance and resource costs. We believe that a new lightweight isolation approach is needed, which supports sharing memory directly between functions and reduces resource overheads. We introduce Faaslets, a new isolation abstraction for high-performance serverless computing. Faaslets isolate the memory of executed functions using software-fault isolation (SFI), as provided by WebAssembly, while allowing memory regions to be shared between functions in the same address space. Faaslets can thus avoid expensive data movement when functions are co-located on the same machine. Our runtime for Faaslets, Faasm, isolates other resources, e.g. CPU and network, using standard Linux cgroups, and provides a low-level POSIX host interface for networking, file system access and dynamic loading. To reduce initialisation times, Faasm restores Faaslets from already-initialised snapshots. We compare Faasm to a standard container-based platform and show that, when training a machine learning model, it achieves a 2x speed-up with 10x less memory; for serving machine learning inference, Faasm doubles the throughput and reduces tail latency by 90%.

1 Introduction

Existing container-based serverless platforms support massive parallelism but impose data-access and resource overheads for data-intensive applications. Faaslets and FAASM address these costs through lightweight isolation, shared state, and fast restoration.

  • Motivation: Container isolation forces external state management or inter-function data passing, causing duplication, serialisation, and network-transfer overheads.These costs produce a data-shipping architecture that moves data to computation.
  • Motivation: Container process isolation mismatches short-lived serverless functions, with cold starts lasting hundreds of milliseconds to several seconds and substantial memory footprints.The resource footprint limits the efficiency and density of high-volume workloads.
  • Faaslets: Faaslets use WebAssembly-based software-fault isolation while executing multiple dedicated-thread instances within one address space.FAASM constrains CPU cycles for each thread using resource isolation mechanisms described in the paper.
  • Faaslets: Faaslets share local memory efficiently and provide a two-tier architecture for in-memory local state and distributed global state.This co-locates data and functions, avoids serialisation overheads, and supports different consistency requirements.
  • Evaluation: 60% shorter training runtime, 70% fewer network transfers, and 90% lower memory usage were achieved against a container-based deployment.The training workload used stochastic gradient descent.
  • Evaluation: Inference achieved over a 200% increase in maximum throughput and a 90% reduction in tail latency versus the container-based deployment.The inference workload used TensorFlow Lite and MobileNet.

2 Isolation vs. Sharing in Serverless

The paper examines how serverless isolation can combine memory and resource safety with efficient state sharing. Existing mechanisms trade off sharing, isolation, resource overhead, or portability, motivating Faaslets as a combined approach.

  • Design tension: Sharing memory between functions conflicts with isolation, creating a central challenge for multi-tenant stateful serverless computing.The paper treats safe shared access to in-memory state as a design problem.
  • Existing approaches: Containers and VMs provide memory safety and resource isolation but compare poorly on initialisation time and memory footprint.Their complete POSIX or hardware virtualisation contributes to these costs.
  • Existing approaches: Existing serverless systems address data movement or container overheads separately, but some sacrifice isolation while others introduce non-serverless components.Recycled containers weaken isolation, whereas static VMs add scaling complexity and cost.
  • Software-based isolation: Software-based isolation can reduce initialisation times and memory overheads by up to two orders of magnitude relative to containers and VMs.However, software-based isolation alone does not provide resource isolation or efficient in-memory state sharing.
  • Software-based isolation: WebAssembly-based systems provide memory safety with low overheads but do not by themselves support resource isolation or efficient in-memory state sharing.Related systems also rely on data shipping or replicate container-style isolation.
  • Software-based isolation: WebAssembly’s specification does not yet include mechanisms for dynamically sharing memory, so WebAssembly alone cannot meet the paper’s requirements.The proposed synchronised shared-memory mechanism requires compile-time knowledge of shared regions and lacks a programming model for dynamic distributed state.
  • Design goals: Faaslets are proposed to combine strong memory and resource isolation with efficient shared state, multi-host scaling, low footprint, fast instantiation, and multi-language support.The design goal is to co-locate data with functions while preserving serverless elasticity.

3 Faaslets

Faaslets combine lightweight software-based memory isolation with shared state access and resource isolation for data-intensive serverless computing. Their host interface and snapshot-based startup support efficient execution across applications and hosts.

  • Overview: Faaslets combine strong memory and resource isolation with efficient shared in-memory state.Their host interface provides lightweight virtualisation for serverless-specific tasks, memory management, filesystem access, and networking.
  • Overview: Below 200 KB memory and less than 10 ms cold-start initialisation improve on containers and VMs.Functions are compiled to secure intermediate representation, supporting multiple programming languages.
  • Overview: Proto-Faaslets reduce initialisation times to hundreds of microseconds and can be restored across hosts for horizontal cluster scaling.They mitigate Faaslets’ slower startup relative to pure SFI through ahead-of-time initialisation from snapshots.
  • Memory and state: WebAssembly provides memory safety and control-flow integrity while permitting private and shared memory regions.Each function normally receives private contiguous memory, but shared regions allow access to shared in-memory state within WebAssembly’s safety constraints.
  • Resource isolation: Linux cgroups, network namespaces, virtual interfaces, and traffic shaping provide fair CPU and network access between co-located Faaslets.Dedicated runtime threads receive equal CPU shares, while virtual interfaces enforce ingress and egress rate limits.
  • Host interface: The minimal host interface replaces full POSIX virtualisation and external APIs with dynamically linked low-level calls for serverless operations.It supports chained invocation, shared-state interaction, memory management, timing, filesystem and network I/O, and dynamic linking.

4 Local and Global State

Faaslets provide a two-tier state model combining shared in-memory access on each host with distributed state synchronization across hosts. Distributed data objects expose this model while allowing selective replication, variable consistency, and direct state-API control.

  • State programming model: Distributed data objects provide language-specific classes with a convenient high-level interface to state through the key/value state API.Each DDO represents one state value referenced by a string state key.
  • State architecture: The two-tier architecture combines a local tier for shared in-memory state with a global tier for distributed access across hosts.Faaslets synchronize state between tiers using push and pull operations.
  • State programming model: Functions may access the state API directly for fine-grained consistency and synchronization control or to implement custom data structures.DDOs hide the two-tier architecture but do not prevent lower-level access.
  • Consistency: Push and pull operations support variable consistency, including delayed updates and lazy reads, while immutable objects avoid repeated synchronization.Applications can trade synchronization frequency for lower overhead when their algorithms tolerate inconsistency.
  • Selective access: DDOs can replicate only required state chunks locally, avoiding unnecessary transfer of an entire larger state value.The columns method accesses subsets of matrix state through independently managed chunks.
  • Consistency: Local replicas use implicit read and write locks, whereas strong global consistency requires global locks around pull, update, push, and release operations.Direct pointer writes bypass the implicit state-API locking and require explicit coordination.

5 FAASM Runtime

FAASM is a distributed runtime that schedules, executes, and manages Faaslets across a cluster while co-locating functions with warm state. It further reduces startup time through snapshot-based Proto-Faaslets and resets instances to protect data across tenants.

  • Runtime architecture: FAASM distributes runtime instances across servers, each managing scheduling, execution, and state management for Faaslets.The runtime is designed to integrate with existing serverless infrastructure, autoscaling, and user-facing frontends.
  • Scheduling: The local scheduler minimizes data shipping by co-locating function calls with required in-memory state and prioritizing warm Faaslets.Warm Faaslets already have their code and state loaded.
  • Scheduling: Function calls execute locally when a warm Faaslet has capacity or are shared with another warm host through the distributed shared-state scheduler.The set of warm hosts is stored in the global tier and atomically updated during scheduling decisions.
  • Instantiation: Faaslets typically initialise in under 10 ms, while Proto-Faaslets reduce new-instance startup to hundreds of microseconds by restoring snapshots.Snapshots can contain arbitrary execution state and be restored on any host in the cluster.
  • Instantiation: Proto-Faaslet snapshots can be generated from user-defined initialization code, including dynamic-language runtime initialization.Repeated initialization work can be moved out of the function and into the snapshot preparation stage.
  • Isolation: FAASM restores Proto-Faaslets after each function call, clearing private memory before reuse across tenants.This reset mechanism is intended for multi-tenant functions and prevents information from a previous call from being disclosed.

6 Evaluation

FAASM is evaluated against Knative across machine-learning, linear-algebra, language-runtime, initialization, and capacity workloads. The results show lower data-movement and memory overheads, faster initialization, higher throughput, and generally competitive execution performance, with some WebAssembly-related overheads.

  • 6.1 Experimental Set-up: FAASM is compared with Knative using shared workloads, common state-management code, and a Kubernetes testbed spanning 20 hosts.The evaluation measures execution time, throughput, latency, network transfers, and billable memory.
  • 6.2 Machine Learning Training: 60% runtime improvement is achieved by FAASM over Knative with 15 parallel training functions.At low parallelism the improvement is 10%; FAASM continues improving up to 38 functions, while Knative exhausts memory above 30.
  • 6.2 Machine Learning Training: 75 GB versus 145 GB of network transfers are recorded at two parallel functions, while FAASM uses 500 versus over 5,000 GB-seconds of billable memory at higher parallelism.The shared local tier reduces data shipping and duplication; batched shared-weight updates provide further improvements.
  • 6.2 Machine Learning Training: At 32 parallel functions, FAASM records 460 ms training time, 19 MB network transfers, and 0.01 GB-second billable memory, versus 630 ms, 48 MB, and 0.04 GB-second for Knative.The container memory overhead is 8 MB per function, compared with 270 kB per Faaslet.
  • 6.3 Machine Learning Inference: FAASM maintains under-150-ms tail latency across cold-start ratios and over 200 req/s, while Knative exceeds 2 s under a 20% cold-start workload.FAASM's optimal latency is higher because TensorFlow Lite compilation to WebAssembly adds inference overhead.
  • 6.4 Language Runtime Performance with Python: 13% less network traffic is observed for FAASM in matrix multiplication, while execution duration remains almost identical to Knative across matrix sizes.Most Polybench benchmarks are comparable to native execution, but some Python benchmarks incur 50%–60% overhead and pidigits incurs 240%.
  • 6.5 Efficiency of Faaslets vs. Containers: 10× more Faaslets than containers can run on one host, rising to twelve times more with Proto-Faaslets, while memory footprints are nearly seven and fifteen times lower, respectively.The table also reports several orders of magnitude improvement in CPU cycles and elapsed time for no-op isolation.
  • 6.5 Efficiency of Faaslets vs. Containers: 3.2 s container initialization is reduced to 0.9 ms for a restored Python Proto-Faaslet snapshot.In throughput tests, Docker initializes in about 2 s, Faaslets in about 5 ms, and restored Proto-Faaslets in about 0.5 ms.

7 Related Work

Related systems provide lightweight sandboxing, language-independent runtimes, or distributed storage, but differ from Faaslets in shared-state support and scope of isolation.

  • Isolation mechanisms: Shreds and Wedge focus on intra-process memory isolation, whereas Faaslets isolate complete executables.Light-weight Contexts and Picoprocesses sandbox complete POSIX applications but do not provide efficient shared state.
  • Common runtimes and distributed storage: Anna, Tuba, Pocket, and Infinispan-based storage systems offer alternatives for distributed state management that could support FAASM's global state tier.The passage contrasts these systems with Redis on latency, autoscaling granularity, or serverless specialization.

8 Conclusions

FAASM combines memory-safe, resource-fair Faaslets with shared in-memory state, rapid snapshot-based initialization, distributed state, and POSIX support for dynamic and traditional applications.

  • Contributions: FAASM provides memory safety and resource fairness while allowing Faaslets to share in-memory state.Proto-Faaslet snapshots reduce initialization time.
  • State architecture: FAASM's two-tier state architecture co-locates functions with required state for parallel in-memory processing while scaling across hosts.Users build stateful serverless applications with distributed data objects through the Faaslet state API.
  • Host interface: The Faaslet host interface supports dynamic language runtimes and traditional POSIX applications.
Loading 2002.09344v2…