PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking, Analysis, and Optimization of Serverless Function Snapshots

Dmitrii Ustiugov, Plamen Petrov, Marios Kogias, Edouard Bugnion, Boris Grot

arXiv:2101.09355v3cs.DC

TL;DR

Cold starts are costly for short, sporadically invoked serverless functions, while keeping instances warm wastes memory. The paper uses vHive to analyze snapshot-based infrastructure and introduces REAP, which exploits stable memory working sets to prefetch pages. REAP reduces cold-start latency by 3.7× on average compared with baseline snapshotting.

  • Problem

    Cold-start latency is costly for short-running serverless functions, while keeping idle instances warm consumes substantial memory.

  • Method

    The paper introduces vHive for analyzing snapshot-based serverless infrastructure and develops REAP to record stable guest-memory pages and prefetch them on later invocations.

  • Results

    3.7× average cold-start speedup is achieved by REAP compared with baseline snapshotting.

  • Takeaways & Limitations

    Stable working sets enable proactive prefetching to reduce snapshot-based serverless cold-start delays.

  • Takeaways & Limitations

    Functions with large inputs or substantially different control flow across invocations may benefit less from REAP.

Abstract

from arXiv · show

Serverless computing has seen rapid adoption due to its high scalability and flexible, pay-as-you-go billing model. In serverless, developers structure their services as a collection of functions, sporadically invoked by various events like clicks. High inter-arrival time variability of function invocations motivates the providers to start new function instances upon each invocation, leading to significant cold-start delays that degrade user experience. To reduce cold-start latency, the industry has turned to snapshotting, whereby an image of a fully-booted function is stored on disk, enabling a faster invocation compared to booting a function from scratch. This work introduces vHive, an open-source framework for serverless experimentation with the goal of enabling researchers to study and innovate across the entire serverless stack. Using vHive, we characterize a state-of-the-art snapshot-based serverless infrastructure, based on industry-leading Containerd orchestration framework and Firecracker hypervisor technologies. We find that the execution time of a function started from a snapshot is 95% higher, on average, than when the same function is memory-resident. We show that the high latency is attributable to frequent page faults as the function's state is brought from disk into guest memory one page at a time. Our analysis further reveals that functions access the same stable working set of pages across different invocations of the same function. By leveraging this insight, we build REAP, a light-weight software mechanism for serverless hosts that records functions' stable working set of guest memory pages and proactively prefetches it from disk into memory. Compared to baseline snapshotting, REAP slashes the cold-start delays by 3.7x, on average.

1 INTRODUCTION

Serverless workloads make cold-start latency costly, while keeping infrequently used functions warm wastes memory. This paper introduces vHive to analyze snapshot-based execution and REAP to reduce snapshot cold-start delays.

  • vHive and REAP: vHive is an open-source framework that combines production-grade serverless components for experimentation across the serverless stack.The framework supports characterization of snapshot-based infrastructure built with Containerd and Firecracker.
  • Motivation and observations: 95% of actual function processing time is spent servicing page faults on the critical path, slowing snapshot-restored functions versus warm execution.Pages are brought from disk one at a time, and their accesses have poor spatial locality.
  • Motivation and observations: 97% of memory pages are the same across invocations of the studied functions, revealing a stable guest-memory working set.This recurrence motivates recording and reusing accessed pages across invocations.
  • vHive and REAP: REAP records a function’s guest-physical page trace on its first invocation and proactively prefetches the working set on later invocations.The mechanism is implemented in userspace using Linux user-level page-fault handling.
  • vHive and REAP: 3.7× average cold-start speedup is achieved by REAP compared with baseline snapshotting.The evaluation also reports that REAP eliminates 97% of page faults on average.

2 SERVERLESS BACKGROUND

Serverless functions are short-running and sporadically invoked, creating pressure to reclaim idle instances while avoiding expensive cold starts. Existing virtualization and snapshotting approaches reduce some costs but retain restoration overheads.

  • Workload characteristics: 90% of Azure Functions are invoked less frequently than once per minute, while more than 90% run for under 10 seconds.These workload characteristics make startup latency significant relative to execution time.
  • Workload characteristics: 4–8 thousand function instances are a stated AWS Lambda deployment goal for a single host, increasing the value of efficient resource use.High colocation also creates security and isolation challenges for untrusted code.
  • Cold-start challenge: Cold-start delay occurs when providers deallocate idle instances and start new ones after inactivity, making startup latency a central serverless metric.Providers commonly limit instance lifetimes to 8–20 minutes after the last invocation.
  • Hypervisors and snapshots: Firecracker reduces hypervisor boot time to 125ms and its memory footprint to 3MB, but production frameworks can take 700–1300ms to boot a VM.Additional framework setup, including mounting a function image, contributes to the longer production boot time.
  • Hypervisors and snapshots: Snapshotting stores a fully booted VM state on disk so new instances can be loaded without retaining their complete state in main memory.Firecracker snapshot restoration maps guest memory for lazy paging after loading VMM state.

3 VHIVE: AN OPEN-SOURCE FRAMEWORK FOR SERVERLESS EXPERIMENTATION

vHive provides an open-source, production-representative framework for studying serverless deployments. Its architecture combines Kubernetes, Knative, Containerd, Firecracker MicroVMs, and measurement support across control and data planes.

  • Framework overview: vHive is introduced as an open-source framework for deeper experimentation with serverless computing.It is designed to support systems research across representative serverless deployments.
  • Framework overview: Figure 1 distinguishes the data plane with solid arrows from the control plane with dashed arrows.The figure provides the architecture’s high-level plane-level organization.
  • Framework components: vHive integrates production-grade components from leading serverless providers, including Amazon and Google.The framework adopts Knative on Kubernetes and supports OCI function images.
  • Framework components: A function instance runs in a MicroVM inside a Kubernetes pod, alongside a Knative Queue-Proxy that provides queuing and health monitoring.The MicroVM isolates the worker host from untrusted developer-provided code.
  • Framework components: The vHive-CRI orchestrator integrates stock Containerd with its Firecracker-specific fork to manage container and MicroVM lifecycles.It processes Kubernetes Container-Runtime Interface requests and dispatches them to the appropriate runtime components.
  • Research support: vHive supports performance analysis through high-precision Containerd and Kubernetes logs, custom metrics, and client-side response-time evaluation.These facilities enable experiments that vary function mixes and deployment scenarios.

4 SERVERLESS LATENCY AND MEMORY FOOTPRINT CHARACTERIZATION

Using vHive, the study characterizes snapshot-based serverless latency, memory footprint, locality, and page reuse in Firecracker MicroVMs. Snapshot restoration reduces memory use but incurs substantial invocation latency because lazy, non-contiguous page faults retrieve guest memory from disk.

  • Cold-start latency: Cold invocations from snapshots take one to two orders of magnitude longer than warm invocations.Warm instances are memory-resident, while snapshot-based cold starts remain a major latency bottleneck.
  • Cold-start latency: 95% longer average function-processing time makes cold snapshot invocations much slower than warm invocations of the same functions.Load VMM and connection restoration add 156-317 ms, while function processing can reach hundreds of milliseconds even for millisecond-scale warm functions.
  • Page-fault mechanism: Lazy paging triggers thousands of serial page faults, and non-contiguous guest-memory accesses prevent disk run-ahead prefetching.Contiguous regions average 2-3 pages for most functions and up to 5 pages for lr_training, increasing page-fault delays.

5 REAP: RECORD-AND-PREFETCH

REAP records a function’s stable guest-memory working set during one snapshot-based invocation, then proactively prefetches those pages for subsequent cold invocations. Its userspace design integrates with vHive-CRI and reduces page-fault handling while preserving support for multi-tenant execution.

  • REAP overview: 97% of memory pages are the same across invocations on average, motivating prefetching of a stable working set.The compact, stable page set provides the basis for reducing cold-start delays.
  • REAP overview: REAP records a function’s working set during its first snapshot-based invocation and replays it to accelerate subsequent cold invocations.The mechanism records accessed pages and uses the resulting record to eliminate the majority of guest-memory page faults.
  • Record phase: During recording, REAP traces page faults, identifies guest-memory-file offsets, and stores accessed pages in a contiguous working-set file plus an offset trace.The trace maps accessed pages to their original positions, while the working-set file provides a compact copy for later loading.
  • Prefetch phase: During prefetching, REAP reads the working-set file into an orchestrator buffer, installs pages eagerly, and serves remaining non-working-set faults on demand.Because the working-set file captures most accessed pages, only a small number of page faults remain for monitor handling.
  • Trade-off: The record phase increases invocation time relative to baseline snapshots, so REAP penalizes the first invocation to benefit later ones.The overhead results from userspace page-fault handling and is quantified separately in the paper.
  • Implementation: 533-850MB/s of SSD read throughput is achieved by fetching the working-set file with a single large direct read.This design addresses the variability of SSD throughput across access patterns.
  • Implementation: REAP is implemented in userspace within vHive-CRI, uses parallel monitor threads, and requires no host or guest kernel modifications.The implementation is loosely integrated with Containerd via gRPC and adds less than 200 lines to Firecracker’s Rust codebase.

6 EVALUATION

The evaluation measures REAP against baseline snapshotting across representative functions and optimization stages. REAP reduces cold-start latency through working-set prefetching, while benefits vary with invocation behavior, recording overhead, and available disk bandwidth.

  • 6.1 Evaluation Platform: The evaluation uses nine Python FunctionBench functions plus helloworld on Firecracker VMs managed by Containerd.The setup uses a single-vCPU VM with 256MB guest memory and cold invocations under controlled hardware conditions.
  • 6.2 Understanding REAP Optimizations: 182ms versus 1ms for warm helloworld: vanilla snapshots spend most invocation time handling page faults.The approximately 8MB working set is served at only 43MB/s, below 5% of the platform’s peak SSD bandwidth.
  • 6.2 Understanding REAP Optimizations: 1.9× faster to 118ms: parallel page-fault handling overlaps I/O, but reaches only 130MB/s of SSD bandwidth.This design demonstrates that higher SSD read bandwidth is important for efficient page-fault processing.
  • 6.2 Understanding REAP Optimizations: 29ms with the WS file and 15ms with REAP: single-read working-set retrieval outperforms parallel page-sized reads, while bypassing the page cache further accelerates loading.REAP reaches 533MB/s, within 37% of the SSD’s 850MB/s peak.
  • 6.3 REAP on FunctionBench: 1.04–9.7× faster, averaging 3.7×: REAP lowers cold-start delays across the evaluated functions compared with baseline Firecracker snapshots.The orchestrator-to-function gRPC reconnection component shrinks 45× on average to 4–7ms.
  • 6.4 Record Overhead: 15–87% first-invocation overhead: REAP’s trace and working-set recording cost is one-time and is reported as amortizable because functions usually execute multiple times.Image_rotate is the outlier with an 87% degradation.

7 DISCUSSION

REAP’s benefits depend on workload and storage conditions: it reduces transfer bottlenecks, but can waste resources or provide little benefit for unsuitable invocation patterns. Snapshot cloning also raises security concerns involving entropy and memory-layout reuse.

  • REAP’s Efficiency and Mispredictions: REAP’s retrieval speed depends on snapshot location, transferred data volume, and network and storage latency and bandwidth.
  • REAP’s Efficiency and Mispredictions: 3-39% is the reported range for mispredicted pages during a cold invocation, causing modest SSD-bandwidth increases without affecting correctness.
  • Applicability to Real-World Functions: REAP may not justify its additional working-set and trace files for some functions.
  • Applicability to Real-World Functions: Functions invoked very rarely or more frequently than once per minute are unlikely to benefit from snapshot-based solutions.
  • Security Concerns: Snapshot-based VM clones can share random-number-generator state and guest memory layouts, creating entropy and ASLR security concerns.

8 RELATED WORK

Related work spans benchmark suites, serverless platforms, VM snapshotting, boot-time optimization, and caching. vHive combines production-oriented virtualization, snapshotting, orchestration, deployment, and benchmarking capabilities for end-to-end experimentation.

  • Open-Source Serverless Platforms: vHive uses diverse Python benchmarks from FunctionBench and contrasts with platforms focused on multi-function composition or autoscaling.
  • Open-Source Serverless Platforms: Many existing serverless platforms rely on Docker or language sandboxes whose isolation is considered insufficiently secure for public-cloud deployments.
  • Open-Source Serverless Platforms: Kata Containers and gVisor provide virtualized runtimes but lack function-deployment and end-to-end evaluation toolchains, and do not support snapshotting.
  • VM Snapshots and Cold Starts: Prior VM-cloning approaches use lazy guest-memory loading, while REAP instead targets serverless workloads with proactive working-set prefetching.
  • Caching: Caching approaches keep pre-initialized environments or processes ready for requests, including zygote-based and reusable-sandbox designs.

9 CONCLUSION

The paper identifies serial lazy page faults as the root cause of high snapshot cold-start delays and finds that functions repeatedly access a stable small working set. REAP records that working set once and eagerly prefetches it for later invocations.

  • 9 CONCLUSION: Thousands of serial page faults significantly slow function invocations restored from snapshots.
  • 9 CONCLUSION: Functions exhibit a small, stable working set of guest-memory pages across different invocations.
  • 9 CONCLUSION: REAP records a function’s working-set pages during its first invocation and eagerly prefetches them into newly loaded instances thereafter.
Loading 2101.09355v3…