PaperlayerView exact revision on arXiv

Source-linked AI summary

FaaSNet: Scalable and Fast Provisioning of Custom Serverless Container Runtimes at Alibaba Cloud Function Compute

Ao Wang, Shuai Chang, Huangshi Tian, Hongqi Wang, Haoran Yang, Huiba Li, Rui Du, Yue Cheng

arXiv:2105.11229v3cs.DC

TL;DR

Custom-container functions can face long cold starts, particularly with sizeable dependencies. FaaSNet decentralizes provisioning across VMs using adaptive function trees, provisioning 2,500 containers in 8.3 seconds and scaling 13.4× and 16.3× faster than Alibaba Cloud’s current FaaS platform and Kraken.

  • Problem

    Custom-container FaaS faces long cold starts because providers must fetch image data and start runtimes, with sizeable dependencies worsening the issue.

    Source: §2.2.2 · p. 3

  • Method

    FaaSNet decentralizes container provisioning across VMs organized in adaptive function trees, with topology balancing to accommodate VMs joining and leaving.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    2,500 function containers are provisioned within 8.3 seconds; FaaSNet scales 13.4× and 16.3× faster than Alibaba Cloud’s current FaaS platform and Kraken, respectively.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Scalable, fast container provisioning can support fundamental elasticity for FaaS providers offering custom-container-based functions.

    Source: §7 · p. 12

    Extremely short-lived, sub-second functions with sparse invocations may be adversarial to FaaSNet and custom-container FaaS platforms.

    Source: §5 · p. 12

Abstract

from arXiv · show

Serverless computing, or Function-as-a-Service (FaaS), enables a new way of building and scaling applications by allowing users to deploy fine-grained functions while providing fully-managed resource provisioning and auto-scaling. Custom FaaS container support is gaining traction as it enables better control over OSes, versioning, and tooling for modernizing FaaS applications. However, providing rapid container provisioning introduces non-trivial challenges for FaaS providers, since container provisioning is costly, and real-world FaaS workloads exhibit highly dynamic patterns. In this paper, we design FaaSNet, a highly-scalable middleware system for accelerating FaaS container provisioning. FaaSNet is driven by the workload and infrastructure requirements of the FaaS platform at one of the world's largest cloud providers, Alibaba Cloud Function Compute. FaaSNet enables scalable container provisioning via a lightweight, adaptive function tree (FT) structure. FaaSNet uses an I/O efficient, on-demand fetching mechanism to further reduce provisioning costs at scale. We implement and integrate FaaSNet in Alibaba Cloud Function Compute. Evaluation results show that FaaSNet: (1) finishes provisioning 2500 function containers on 1000 virtual machines in 8.3 seconds, (2) scales 13.4x and 16.3x faster than Alibaba Cloud's current FaaS platform and a state-of-the-art P2P container registry (Kraken), respectively, and (3) sustains a bursty workload using 75.2% less time than an optimized baseline.

1 Introduction

FaaSNet addresses the cost and scalability challenges of provisioning custom FaaS containers under bursty workloads and large-image contention. It uses an adaptive function tree to decentralize provisioning across host VMs and is evaluated in Alibaba Cloud Function Compute.

Introduction

  • Custom container images offer flexibility, portability, and familiar tooling for FaaS applications with large dependencies, while enabling incremental updates.The paper motivates container packaging in part by the package-size constraints of archive-based deployment.Source: §1 · p. 1
  • Bursty demand can make hundreds of VMs pull the same image at once, while large images under contention can cause cold startup latency of several minutes.The analyzed production workload had spikes above 1,000 RPS and a peak-to-trough ratio above 500×.Source: §1 · pp. 1–2
  • Existing P2P provisioning systems rely on dedicated, powerful root servers, which add cost and bottlenecks and do not fit Function Compute’s dynamic pool of resource-constrained VMs.Function Compute uses containers inside VMs for tenant isolation, and hosts may join or leave the pool at any time.Source: §1 · p. 2
  • FaaSNet decentralizes provisioning across host VMs organized in function trees and uses dynamic tree balancing to adapt when VMs join or leave.Its design targets Alibaba Cloud Function Compute’s workload and infrastructure requirements; the paper also releases the function-tree implementation and an anonymized production cold-start trace dataset.Source: §1 · p. 2
  • FaaSNet provisions 2,500 function containers in 8.3 seconds, scales 13.4× and 16.3× faster than Alibaba Cloud’s platform and Kraken, respectively, and uses 75.2% less time than an optimized baseline.The system was implemented and integrated in Alibaba Cloud Function Compute and evaluated with production workloads and microbenchmarks.Source: §1 · p. 2

2 Background and Motivation

Function Compute routes custom-container function invocations through a frontend to backend VMs that pull image data and start containers. Workload traces show bursty demand and costly cold starts, motivating fast, scalable container provisioning.

2.1 FaaS Container Workflows in Alibaba Cloud Function Compute

  • Function Compute's workflow sends deployment images to a centralized registry, then routes invocation requests through a frontend gateway to backend VMs, which pull container data and start containers.The frontend checks the request and registry image status before forwarding invocations to the backend cluster.Source: §2.1 · p. 2

2.2.1 Workload Burstiness

  • A request spike rose from 22 to 485 RPS; IoT throughput jumped from around 682 to over 1460 RPS, while VOS average throughput rose from 580 to 920 RPS.The traces show distinct burst patterns: IoT's peak lasted about 40 minutes, and VOS throughput gradually returned to an average of 560 RPS. These dynamics require scalable, resilient provisioning of large numbers of containers to smooth latency spikes.Source: Figure 2 · p. 3, §2.2.1 · p. 3

2.2.2 Cold Start Costs of Containerized Functions

  • 57% of Beijing image pulls exceed 45 seconds, while more than 86% of Shanghai pulls take at least 80 seconds; over 50%/60% of requests spend at least 80%/72% of startup time pulling images, respectively.The study analyzed 712,295 cold starts from 15-day registry logs in Beijing and Shanghai.Source: Figure 3 · p. 3, §2.2.2 · p. 3
  • About 49% of cold starts in both regions had inter-arrival times under 1 second; in Shanghai, about 80% of function durations were under 32.5 seconds, with 90th and 99th percentiles of 36.6 and 45.6 seconds.About 80% of Beijing executions lasted longer than 1 second; cold-start costs were of the same magnitude as function duration, motivating faster container provisioning.Source: Figure 3 · p. 3, §2.2.2 · pp. 3–4

2.2 Workload Analysis

  • Function Compute trace analysis focuses on the need for fast, scalable container runtime provisioning at Step 5 to support high elasticity.The analysis uses workload traces collected from Function Compute.Source: §2.2 · p. 3

3 FAASNET Design

FAASNET accelerates container provisioning by distributing work across function-specific trees of VMs, while its scheduler manages tree metadata and workers fetch image data from peers. Block-based, on-demand I/O and efficient data streaming further reduce provisioning costs.

3.1 Design Overview

  • FAASNET decentralizes and parallelizes provisioning across VMs: the scheduler manages function trees while workers fetch image layers from peers on demand.On invocation, the scheduler contacts workers in the function tree; on deployment, the gateway converts regular images into an I/O-efficient format and stores the layers and manifest.Source: §3.1 · p. 4

3.2 Function Trees

  • Each function has a dynamically managed balanced binary tree, which limits concurrent downloads per VM and keeps image propagation to at most ⌊log(N)⌋ hops.The FT manager inserts and deletes VMs, rebalancing when a node’s left and right subtree heights differ by more than one.Source: §3.2 · pp. 4–5

3.3 Function Tree Integration

  • The scheduler maintains FT metadata and coordinates workers; when a VM joins, it obtains the image manifest, fetches layers from its upstream peer, and reports container creation.The scheduler integrates the FT manager, while each VM agent runs a worker that serves provisioning commands and manages local function containers.Source: §3.3 · pp. 5–6

3.4 FT Design Discussion

  • FAASNET’s function-level tree can eliminate registry I/O when an active VM already stores the requested container, while avoiding the network and root-node bottlenecks of layer-based all-to-all designs.The paper notes that all-to-all layer trees can bottleneck on resource-constrained VMs, and that using powerful root nodes for seeding and management adds provider cost.Source: §3.4 · p. 6

3.5 Optimizations

  • FAASNET compresses fixed-size image blocks separately and uses an offset table for on-demand reads, trading possible read amplification and CPU overhead for reduced I/O cost.The same format supports code packages; a user-level zero-copy RPC library pipelines requests and forwards each completed data block to downstream peers.Source: §3.5 · p. 7

4 Evaluation

Evaluation uses production-derived FaaS traces and microbenchmarks to assess provisioning under bursts, scaling, placement, package format, and on-demand I/O.

4.1 Experimental Methodology

  • Evaluation used 500- and 1,000-VM pools of 2-CPU, 4-GB, 1-Gbps VMs, 512-KB blocks, and compared Kraken, baseline, on-demand, and DADI+P2P.Unless otherwise specified, tests used a 758-MB Python 3.8 PyStan image, 3008-MB function memory, and one function per VM; VM cold-start time was excluded.Source: §4.1 · p. 7

4.2 FaaS Application Workloads

  • FAASNET reduced peak IoT application response time from 28 to 6 seconds and restored normal response in 28 seconds, versus almost 113 seconds for baseline.The IoT trace included bursts reaching 300–400 RPS and later around 400 RPS.Source: §4.2 · p. 8
  • On the synthetic trace, FAASNET scaled its function tree from 2 to 7 levels and 82 VMs for the first burst, then added 62 VMs for a 102-VM second burst.It began parallel provisioning immediately and sustained latency spikes in about 10 seconds.Source: §4.2 · pp. 8–9

4.3 Scalability and Efficiency

microbenchmarking.

  • At 128-way concurrency, FAASNET’s provisioning pipeline spanned 1.5 seconds, versus 16.4 seconds for on-demand and 19 seconds for DADI+P2P.Workers load images locally in parallel and stream layer blocks from peers, starting containers once enough blocks arrive; FAASNET maintained almost identical latency from 8 to 128 starts.Source: microbenchmarking · pp. 9–10, Figure 15 · p. 10

4.4 Impact of Function Placement

  • DADI+P2P showed much higher provisioning-latency variation when 4 or 8 functions shared a VM, as its root VM was overloaded establishing many small layer trees.Source: Figure 18 · p. 10, §4.4 · p. 10

4.5 I/O Efficient Data Format

  • The I/O-efficient format matched .zip for the 11-KB Helloworld package and performed better for Video and AI by fetching data on demand rather than extracting everything.Its packages were larger because compression incurred extra storage overhead.Source: Figure 19 · p. 11, §4.5 · p. 11

4.6 On-Demand I/O: Sensitivity Analysis

  • With 512-KB blocks, on-demand fetching reduced network I/Os by 83.9% versus regular docker pull.Larger blocks can increase read amplification when requested offsets misalign with block boundaries; reducing read amplification is future work.Source: Figure 20 · p. 11, §4.6 · p. 11

5 Discussion

The discussion identifies deployment constraints for FAASNET and outlines future work in placement, virtualization portability, broader data sharing, workload handling, and bare-metal deployment.

FT-aware Placement

  • To limit bandwidth contention, FAASNET avoids co-locating functions when resources permit; future FT-aware placement would balance VM traffic and exploit shared layers to reduce transfers.Placement can control how many function trees involve each VM and whether it serves as a leaf or interior node.Source: §5 · p. 11

Multi-Tenancy

  • Because Alibaba Cloud isolates tenants with containers and VMs and cannot share VMs across tenants, FTs are tenant-isolated; porting to other secure, lightweight virtualization techniques remains ongoing work.Source: §5 · p. 11

FAASNET for Data Sharing

  • FAASNET’s P2P image-sharing design could generalize to data sharing in orchestrators such as Kubernetes, but requires user-facing primitives and tree management for more frequent topology changes.Centralized storage can bottleneck data-intensive applications; the paper leaves this broader data-sharing exploration for future work.Source: §5 · pp. 11–12

Adversarial Workloads

  • Sub-second functions with sparse invocations may be adversarial to FAASNET and custom-container FaaS; caching and pre-provisioning [50] can help at extra infrastructure cost.Source: §5 · p. 12

Portability

  • FAASNET is transparent to FaaS applications and infrastructure, reuses Function Compute’s VM-reclaiming policy without extra system-level costs, and is being ported to Alibaba Cloud bare metal.The paper also states that FAASNET could be applied to other FaaS platforms.Source: §5 · p. 12

6 Related Work

Prior work mitigates FaaS cold starts through caching, pre-provisioning, low-level runtime optimizations, and container distribution, but leaves provisioning costs or workload-fit limitations. FaaSNet builds on container and P2P distribution work with a design attuned to FaaS workloads.

Function Environment Caching and Pre-provisioning

  • Caching and pre-provisioning can reduce cold starts, but do not fundamentally solve the high costs of function-environment provisioning.Prior approaches include provider-side caching [55], prediction-based pre-warming [50], shared workflow runtimes, cloned Python containers, and pre-provisioned networking.Source: §6 · p. 12

Sandbox, OS, and Language-level Support

  • Catalyzer and SEUSS use checkpoint or snapshot images, while Faasm [53] and use language-based isolation; these approaches require modified OSes or limit compatibility and usability.Source: §6 · p. 12

Container Storage

  • Container storage systems can serve as FaaSNet backends, while Kraken and DADI rely on static P2P topologies and dedicated components that are ill-suited to dynamic demand and bursts.Related systems include Slacker, Wharf, CFS [44], and Bolt; P2P distribution systems face adaptability and scalability challenges under dynamicity and unpredictable bursts.Source: §6 · p. 12

AWS Lambda Containers

  • AWS Lambda’s announced image support uses microVM-local, bare-metal-server, and availability-zone caches, but its approach is unsuitable for Alibaba Cloud’s platform of thousands of small VMs.The described design suits powerful bare-metal clusters that can co-locate many microVMs.Source: §6 · p. 12

P2P Content Distribution

  • FaaSNet builds on prior P2P distribution approaches, including tree-structured VMThunder, BitTorrent-like distribution, and Bullet’s overlay mesh, with a design attuned to FaaS workloads.Source: §6 · p. 12

7 Conclusion

FAASNET provides an end-to-end, integrated solution for scalable, fast provisioning of custom FaaS container runtimes, and its FT prototype and anonymized production cold-start traces are publicly released.

7 Conclusion

  • FAASNET is the first end-to-end integrated solution for FaaS-optimized container runtime provisioning, using lightweight, decentralized, adaptive function trees and starting thousands of large function containers in seconds.The system is attuned to Alibaba Cloud’s FaaS platform requirements.Source: §7 · p. 12
  • The authors release the FAASNET FT prototype source code and an anonymized dataset of Alibaba Cloud Function Compute production FaaS cold-start traces.The materials are available at https://github.com/mason-leap-lab/FaaSNet.Source: §7 · p. 12

Related papers

Loading 2105.11229v3…