Source-linked AI summary
Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, Ricardo Bianchini
TL;DR
FaaS providers must provision resources for fast, ideally cold-start-free executions while controlling cost, but public evidence about production workloads has been limited. This paper characterizes the entire production workload of Azure Functions and uses the observations to develop and evaluate a practical resource-management policy. The policy reduces cold starts at lower resource cost than fixed keep-alive policies, with evaluation through simulation and real implementation.
Problem
Public information on production FaaS workload characteristics has been limited, although providers need such understanding to manage performance and resource cost.
Method
The paper characterizes Azure Functions’ production workload and designs a policy using workload observations, including interinvocation-time histograms and adaptive pre-warming.
Results
The policy achieves fewer cold starts with fewer resources than fixed keep-alive policies in simulations and is implemented with minimal overhead in real systems.
Takeaways & Limitations
Workload-aware keep-alive and pre-warming can improve cold-start management while maintaining low resource provisioning cost.
Takeaways & Limitations
For confidentiality reasons, the characterization cannot disclose some absolute numbers, including the total number of functions and invocations.
Abstract
from arXiv · showhide
Function as a Service (FaaS) has been gaining popularity as a way to deploy computations to serverless backends in the cloud. This paradigm shifts the complexity of allocating and provisioning resources to the cloud provider, which has to provide the illusion of always-available resources (i.e., fast function invocations without cold starts) at the lowest possible resource cost. Doing so requires the provider to deeply understand the characteristics of the FaaS workload. Unfortunately, there has been little to no public information on these characteristics. Thus, in this paper, we first characterize the entire production FaaS workload of Azure Functions. We show for example that most functions are invoked very infrequently, but there is an 8-order-of-magnitude range of invocation frequencies. Using observations from our characterization, we then propose a practical resource management policy that significantly reduces the number of function coldstarts,while spending fewerresources than state-of-the-practice policies.
1 Introduction
FaaS shifts resource provisioning from users to cloud providers, making workload understanding essential for delivering fast executions at low cost. The paper characterizes Azure Functions’ production workload and uses those observations to design a practical cold-start management policy.
- Motivation: FaaS users upload functions and invoke them through events, while the provider provisions execution resources and bills actual executions.Users do not explicitly configure VMs or containers, and providers manage the resources needed for execution.
- Motivation: Warm starts avoid loading function code from persistent storage, but retaining all functions in memory can be prohibitively expensive when executions are short and infrequent.The provider seeks the illusion that functions are always warm while spending resources as if they were always cold.
- Motivation: Widely varying resource needs and invocation frequencies complicate prediction-based resource management.The workload’s broad invocation-frequency range makes it difficult to determine which resources should remain available.
- Research gap: Production FaaS workload characteristics lacked public information, while prior work primarily used benchmark functions or prototype systems.The paper identifies comprehensive characterization of users’ real workloads as the missing evidence.
- Contributions: The paper provides workload distributions and sanitized production traces for realistic research workloads.These resources are intended for researchers creating realistic traces and studying FaaS systems.
- Contributions: The proposed policy adapts keep-alive values to each workload’s invocation pattern and can pre-warm executions before predicted invocations.It uses a recent interinvocation-time histogram; simulation and OpenWhisk experiments report fewer cold starts with fewer resources than fixed keep-alive policies.
2 Background
FaaS shifts resource allocation and retention decisions to the provider, which schedules applications in response to diverse trigger types. Fixed keep-alive policies do not account for varied application behavior, motivating a more adaptive policy.
- FaaS model: In FaaS, providers decide which resources to allocate, when to allocate them, and how long to retain them after users upload executable code.The provider exposes a handle for running the uploaded code.
- Triggers: Azure Functions groups triggers into HTTP, Event, Queue, Timer, Orchestration, Storage, and other classes.These triggers include requests, message insertion, scheduled events, durable-function workflows, and data changes.
- Applications: Applications group functions and serve as the unit of scheduling and resource allocation in Azure Functions.An application may encompass multiple functions and helps organize and package software.
- Keep-alive policy: Most providers keep application instances loaded for a fixed period after execution, including OpenWhisk’s 10-minute period.This fixed keep-alive approach is simple to implement and maintain.
- Keep-alive policy: Fixed keep-alive policies can produce many cold starts while wasting resources because they ignore varied application behaviors.Users may also issue dummy invocations to keep applications warm, amplifying resource waste.
3 FaaS Workloads
The Azure Functions workload is highly heterogeneous and skewed: invocation frequencies span over 8 orders of magnitude, while most applications are invoked infrequently. Execution times, memory demands, and interarrival patterns create important cold-start and resource-management challenges.
- The characterization focuses on intrinsic application and function properties, including arrival patterns, execution times, and memory demands.
- Functions and applications: 54% of applications contain one function, while 95% contain at most 10 functions.Only about 0.04% of applications have more than 100 functions.
- Functions and applications: 64% of applications have an HTTP trigger, and 29% have a timer trigger; 43% use only HTTP triggers and 13% only timer triggers.Applications may combine multiple trigger types, so category percentages can sum to more than 100%.
- Invocation patterns: Invocation patterns include diurnal and weekly variation, while only about 20% of applications have near-zero interarrival-time coefficient of variation.Timer-triggered applications are not uniformly periodic, and only a small fraction of applications has CV close to 1.
- Invocation patterns: Invocation rates vary by over 8 orders of magnitude, while 45% of applications are invoked at most hourly and 81% at most every minute on average.Keeping infrequently invoked applications warm can cost more than their total execution time.
- Invocation patterns: 18.6% of the most popular applications account for 99.6% of all function invocations.
- Execution times: 50% of functions execute for less than 1 second on average, and 90% take at most 60 seconds.Execution times are comparable to reported cold-start times, making cold-start reduction important for overall performance.
- Memory usage: 90% of applications never consume more than 400 MB, while the first 90% show a 4× variation in maximum allocated memory.Invocation frequency does not strongly correlate with memory allocation or execution time.
4 Managing Cold Starts in FaaS
The hybrid histogram policy adapts pre-warming and keep-alive windows to each application’s invocation pattern, using histograms when representative and alternative strategies otherwise. It combines standard keep-alive, histogram-based windows, and time-series forecasting to reduce cold starts while limiting resource waste.
- Policy overview: The policy sets application-specific pre-warming and keep-alive windows to balance cold-start reduction against resource waste.A pre-warming window controls when the application image is loaded before an expected invocation; a keep-alive window controls how long it remains in memory.
- Histogram component: A compact histogram tracks each application’s idle-time distribution using 1-minute bins.The histogram counts idle times by length and uses the distribution’s head and tail to choose the pre-warming and keep-alive windows.
- Histogram component: The policy uses the distribution head for pre-warming and the tail for keep-alive, defaulting to the 5th- and 99th-percentiles to exclude outliers.Percentiles falling within bins are rounded down for the head and up for the tail.
- Fallback components: When idle-time behavior is uncertain, the policy uses standard keep-alive, and when idle times exceed the histogram range, it uses ARIMA forecasting.Uncertainty includes too few observations, changing idle-time regimes, or a low coefficient of variation across histogram bin counts.
- Fallback components: ARIMA predictions place pre-warming just before the predicted invocation and use a short keep-alive window.For example, a predicted idle time of 5 hours with a 15% margin yields a 4.25-hour pre-warming window and a 1.5-hour keep-alive window.
- Implementation: The policy was implemented in simulation and Apache OpenWhisk using real workload traces.The OpenWhisk implementation is written in Scala and uses the platform’s REST-based FaaS architecture.
5 Evaluation
The evaluation compares fixed keep-alive and hybrid policies through simulation and OpenWhisk experiments using real workload traces. The hybrid policy reduces cold starts and memory waste while adding minimal implementation overhead.
- Fixed keep-alive policy: 50.3% of invocations are cold starts for the 75th-percentile application with a 10-minute keep-alive, falling to 25% with 1 hour.Infrequent applications benefit most from longer keep-alive periods.
- Fixed keep-alive policy: A fixed 2-hour keep-alive wastes almost 30% more memory time than the 10-minute baseline.Longer keep-alive reduces cold starts but increases resource waste.
- Hybrid policy: The 4-hour histogram has roughly 2.5x fewer cold starts than the 10-minute fixed policy at the 75th percentile with the same memory use.The fixed 2-hour policy achieves roughly the same cold-start percentage as the 4-hour histogram but with greater memory waste.
- Hybrid policy: Using ARIMA reduces applications with 100% cold starts from 10.5% to 5.2%, and from 6.9% to 1.7% after excluding single-invocation applications.Only 0.64% of invocations used ARIMA, while 9.3% of applications used it at least once.
- OpenWhisk experiments: The hybrid policy reduces average and 99th-percentile function execution time by 32.5% and 82.4%, respectively.The reduction results from eliminating language-runtime bootstrap time for warm containers.
- OpenWhisk experiments: The policy adds only 835.7µs average end-to-end latency and 4–6% higher controller CPU utilization than OpenWhisk’s default policy.The measured overhead is small relative to runtime and cold-container initiation costs.
6 Production Implementation
The policy was implemented in Azure Functions for HTTP-triggered applications using in-memory invocation histograms, persistent backups, scheduled pre-warming, and per-worker keep-alive durations.
- Azure Functions implementation: Azure Functions implements the policy for HTTP-triggered applications, with production rollout beginning in stages.The implementation uses a controller, function-execution workers, and persistent system state.
- State management: Each application’s histogram contains 240 integers, or 960 bytes, and is backed up hourly to a database.A new daily histogram tracks changes in invocation patterns.
- Pre-warming: When an application becomes idle, Azure Functions schedules pre-warming 90 seconds before the computed pre-warm interval.Pre-warming loads dependencies and performs JIT where applicable, while preserving execution semantics.
7 Related Work
Prior work primarily characterized FaaS through external benchmarks or public repositories, whereas this paper studies an entire provider-side production workload and optimizes cold-start frequency and resource use.
- FaaS characterization: Prior FaaS characterization studies typically reverse-engineer provider behavior using benchmark functions from an external-user perspective.This paper instead provides a longitudinal characterization of an entire large cloud provider’s workload.
- FaaS characterization: Public application repositories cannot reveal the aggregate workload seen by a provider.Repository-based studies examine developer usage but lack provider-side workload coverage.
- FaaS serving optimization: Other systems optimize individual cold-start latency through mechanisms such as network-interface pre-allocation, library caching, and lightweight isolation.The paper’s reduction of cold-start frequency and resource use is orthogonal to those per-cold-start improvements.
- Predictive optimization: Related predictive approaches target multi-tenancy, memory allocation, or overbooking, while this work predicts function invocations.The distinction is the predicted quantity used for resource management.
8 Conclusion
The paper characterizes Azure Functions’ production workload and uses those observations to design and evaluate a practical policy for reducing cold starts at low resource cost. It also provides sanitized traces and describes an Azure Functions implementation.
- Contributions: The study characterizes the entire production FaaS workload of Azure Functions.The characterization informs the paper’s cold-start and resource-management policy.
- Results: The proposed policy achieves the same number of cold starts at much lower resource cost, or fewer cold starts at the same resource cost.Evaluation uses simulations and a real implementation driven by real workload traces.
- Data release: Sanitized traces from the characterization data were released with the paper.The traces support realistic workload studies.