Source-linked AI summary
The Serverless Computing Survey: A Technical Primer for Design Architecture
Zijun Li, Linsong Guo, Jiagan Cheng, Quan Chen, Bingsheng He, Minyi Guo
TL;DR
Serverless computing addresses inefficient resource allocation in traditional IaaS and faces challenges including cold-start latency, resource volatility, workflow complexity, and security risks. This survey organizes the field into four architectural layers, reviews representative techniques and implications, and identifies limitations and future challenges. It concludes that serverless computing remains in its infancy, with substantial potential for further development.
Problem
Traditional IaaS retains resources whether applications run or not, producing about 10% average resource utilization, while serverless systems still face cold starts, workflow volatility, and security threats.
Method
The survey decomposes serverless design into Virtualization, Encapsule, System Orchestration, and System Coordination layers, reviewing relevant works and their practical implications.
Results
The survey provides a comprehensive depiction of four serverless architecture layers and discusses representative techniques, implications, limitations, and challenges across them.
Takeaways & Limitations
Serverless computing offers an architectural basis for cloud-native microservices, but realizing its potential requires addressing compromises in performance, flexibility, resource use, orchestration, coordination, and security.
Takeaways & Limitations
The survey supplements layer-specific limitations and challenges, while directing readers to other surveys for more detailed treatment of the Virtualization layer.
Abstract
from arXiv · showhide
The development of cloud infrastructures inspires the emergence of cloud-native computing. As the most promising architecture for deploying microservices, serverless computing has recently attracted more and more attention in both industry and academia. Due to its inherent scalability and flexibility, serverless computing becomes attractive and more pervasive for ever-growing Internet services. Despite the momentum in the cloud-native community, the existing challenges and compromises still wait for more advanced research and solutions to further explore the potentials of the serverless computing model. As a contribution to this knowledge, this article surveys and elaborates the research domains in the serverless context by decoupling the architecture into four stack layers: Virtualization, Encapsule, System Orchestration, and System Coordination. Inspired by the security model, we highlight the key implications and limitations of these works in each layer, and make suggestions for potential challenges to the field of future serverless computing.
1 INTRODUCTION
The survey broadens serverless computing beyond FaaS and organizes its architecture into four layers to clarify design issues, solutions, and limitations. It reviews the layers’ roles and identifies performance and security challenges.
- Motivation: Traditional IaaS retains resources for long-term service delivery, averaging only about 10% resource utilization for applications with diurnal workloads.
- Definition: The paper argues that serverless should not be equated with FaaS: FaaS handles function isolation and invocation, while BaaS supplies backend support.
- Layered Architecture: The survey models serverless architecture with four layers: Virtualization, Encapsule, System Orchestration, and System Coordination.
- Layered Architecture: Virtualization isolates function execution in containers or virtual machines, while Encapsule middleware supports triggers, execution, metrics, sidecars, and prewarming.
- Layered Architecture: System Orchestration dynamically adjusts triggers, scheduling, and resources, while System Coordination integrates backend services through APIs and SDKs.
- Survey Scope: The survey reviews representative works, analyzes performance and limitations, and explores challenges and opportunities for serverless computing.
2 VIRTUALIZATION LAYER
The Virtualization layer provides isolated sandboxes for serverless functions, balancing isolation, flexibility, startup latency, and performance across virtualization mechanisms.
- Virtualized sandboxes isolate function processes in demarcated resource spaces while supporting testing, debugging, and runtime customization.
- VM-based isolation offers strong isolation and flexibility but typically incurs more than 1000ms of startup latency.
- Containers provide customizable, lightweight sandboxes by combining namespaces, Linux Cgroups, and layered images while sharing the HostOS kernel.
- CNTR dynamically attaches a “fat” image to a slim image and significantly improves overall performance while reducing image size in data-center use.
- Secure containers strengthen tenant isolation through MicroVMs or syscall restrictions, but trade flexibility or syscall-heavy workload suitability for overhead.
- Unikernels reduce image size and offer performance and security potential, but compile-time invariance limits runtime flexibility.
- The four mechanisms exhibit a security-performance-flexibility tradeoff: VMs favor isolation and flexibility, containers favor faster startup, and Unikernels favor performance and security.
3 ENCAPSULE LAYER
The Encapsule layer addresses cold startup through one-to-one and one-for-all prewarming strategies, each trading resource use, prediction requirements, compatibility, and privacy.
- Cold startups arise when no warm container is available or workloads burst, and software initialization can dominate sandbox preparation time.
- One-to-one prewarming uses function-specific pools or workload prediction to reduce initialization latency, but prediction degrades with scarce traces.
- One-for-all prewarming reuses cached, commonly configured sandboxes, reducing initialization work and generally requiring less additional memory than one-to-one prewarming.
- Fixed-size one-to-one pools improve service stability but can leave many idle instances consuming resources.
- Caching-aware mechanisms accelerate restoration and specialize prewarmed containers while protecting private packages through dynamic loading or access controls.
- One-for-all approaches face large template images, conflicting pre-imported libraries, and potential privacy exposure from deployment similarities.
- The preferred strategy depends on workload patterns: one-for-all suits first invocations or poor predictions, whereas one-to-one suits regular or diurnal patterns.
4 ORCHESTRATION LAYER
The System Orchestration layer coordinates elastic resource, instance, and workflow scheduling, but must manage dependencies, interference, overhead, volatility, and security risks at scale.
- Orchestration challenges: The System Orchestration layer faces common challenges in supporting diverse services, including scheduling massive functions with dependencies and predicting on-demand resources.The survey identifies efficient scheduling strategies as a central requirement for serverless orchestrators.
- Dynamic resource adjustment: Resource-level scheduling allocates CPU and memory dynamically, using monitoring, feedback, and learning-based methods to provision resources for new instances.Pigeon uses function-level scheduling and a static oversubscribed pool, while FlowCon dynamically adjusts configurations for container-based deep-learning tasks.
- Dynamic resource adjustment: Resource optimization can target performance, cost, QoS, and interference, but machine-learning estimates trade global optimality against robustness to inaccurate workload information.CherryPick searches configurations under budget or QoS constraints, while MPC reduces QoS violations and resource contention.
- Instance-level scheduling: Instance-level scheduling uses load balancers, resource monitors, and hash-based or multi-objective routing to distribute functions across cluster nodes.Monitoring provides cluster-wide resource status, but container-heavy monitoring can consume substantial memory as function counts increase.
- Application-level workflow scheduling: Workflow scheduling must handle data dependencies and communication overhead while minimizing makespan, execution cost, or resource use under constraints.The survey describes workflow scheduling as NP-hard, notes resource volatility and storage-induced latency, and calls for DAG-based benchmarks based on real applications.
- Security considerations: A single unavailable function node can degrade QoS or generate excessive charges through invocation exhaustion, known as a Denial of Wallet attack.Invocation concurrency and instance quotas are proposed as mitigations.
5 ESSENTIAL BAAS COMPONENTS IN COORDINATION LAYER
The Coordination layer integrates storage, queues, gateways, triggers, caches, and DevOps tools to support communication, invocation, deployment, and operational management in serverless systems.
- Coordination components: Serverless systems commonly integrate six coordination services: storage, queue, API gateway, trigger, data cache, and DevOps tools.The survey presents these components as essential supporting services in a serverless implementation.
- Storage: Remote stores provide a natural way for functions to share ephemeral data and save results for asynchronous invocation.This communication pattern reflects the data-sharing requirements of serverless workloads.
- Storage: Storage supports authentication, in-function API calls, and logging, with memory databases recommended for fast authentication and either disk-resident or memory databases for ephemeral data.Logging records invocation information and helps return results, especially for asynchronous functions.
- Storage: Serverless storage research uses hybrid approaches, responsibility separation, heuristics, and locality improvements to mitigate I/O bottlenecks and contention.The survey notes that heterogeneous functions and uncertain workloads still make these techniques challenging in practice.
- Queues: Queues pass messages between components and can support function communication, node-level load balancing, and transparent scaling for bursty invocations.Queue-based mechanisms may reduce performance or availability, motivating cold-queue, warm-queue, and shared-memory designs.
- Gateways, triggers, and caches: Dynamic addressing and port allocation complicate API gateway management as serverless deployments scale, while triggers bind detectable events to function invocations.HTTP, queue, timer, and event triggers support external and internal invocation patterns, and caching reduces unnecessary round trips.
- DevOps tools: DevOps tools are organized into continuous integration, continuous delivery, and continuous monitoring to automate serverless operational workflows.Continuous delivery strategies maintain availability by gradually replacing old instances with new ones.
- DevOps tools: DevOps tools improve compatibility and flexibility in production, but may introduce vulnerabilities requiring stronger detection of vulnerable containers.The survey identifies security support for container vulnerability detection as an open research need.
6 PERFORMANCE AND COMPARISON
Serverless performance varies substantially with virtualization technology, language runtime, isolation mechanism, memory limit, and platform. Production platforms likewise exhibit different strengths and weaknesses across throughput, bandwidth, I/O, and startup latency.
- Cold Startup Performance: Cold startup performance differs across virtualization and language runtimes, with HyperContainer highest and process-based Docker lowest in Catalyzer evaluations.The passage also reports that interpreted languages generally perform differently from compiled runtimes, though the supplied text is truncated.
- Cold Startup Performance: Language-runtime performance depends on the platform: .NET C# starts slower than Node.js on AWS Lambda but faster on Azure Functions.Azure’s stronger C# support and Windows-container implementation are offered as explanations for this reversal.
- Isolation Mechanisms: Native exec and fork provide the fastest sandbox isolation, while Xen MirageOS performs similarly to Docker; warm Docker exec C is faster than cold Docker run C.These comparisons come from SAND’s measurements of function-execution isolation mechanisms.
- Resource Limits: Cold startup latency increases as container memory limits decrease, with a significant improvement when increasing the limit from 128MB to 256MB.Beyond 256MB, additional memory produces less obvious optimization according to the supplied passage.
- Additional Factors: Branch misprediction, LLC size, and memory bandwidth also affect function performance, although LLC size matters mainly below 2M.The passage says short-execution functions spend more time in runtime startup and that vendors commonly pre-profile LLC settings.
- Production Comparison: Serverless vendors differ in throughput, network bandwidth, I/O capacity, and cold startup latency, producing distinct platform strengths and weaknesses.AWS Lambda has higher concurrent-invocation capacity and throughput but poorer trigger throughput, while Azure Functions offers faster sequential query reads and writes but higher cold startup latency.
7 OTHER KEY LIMITATIONS AND CHALLENGES
The survey identifies limitations involving instance reuse, concurrency density, portability, and benchmark realism. These constraints complicate resource efficiency, cross-platform compatibility, and meaningful evaluation of complex applications.
- Instance Management: Automatic instance recycling means short-lived functions are not guaranteed to process successive queries on the same function instance.On-demand loading and execution prevent long-term association between a function and a particular server or instance.
- Concurrency: Concurrent invocations either incur cold startups across multiple containers or share one container, while massive sidecars limit high-density deployment.The former approach prioritizes performance isolation by allowing only one invocation per container at a time.
- API Lock-in: Function portability is less constrained by code languages than by differing BaaS APIs and service definitions across platforms.Tools such as Apex and Sparta extend deployment to unsupported languages, but do not eliminate API differences.
- Benchmarking: Micro-benchmarks are over-emphasized in 75% of current works, motivating cross-platform real-world application benchmarks.The survey notes that decomposing large services into fine-grained function graphs also makes evaluation difficult.
8 OPPORTUNITIES IN SERVERLESS COMPUTING
The survey proposes application-level optimization through workflow support and scheduling, robust cold-start mitigation, improved accelerator integration, and better data movement. These opportunities target latency, parallelism, portability of execution, and broader serverless applicability.
- Application-Level Optimization: Application-level optimization should coordinate functions through workflow support and workflow scheduling rather than optimize isolated functions.The survey highlights hidden data dependencies and caller-callee relations as central coordination concerns.
- Workflow Support: Better storage is needed because exchanging large ephemeral files through intermediate storage can consume I/O resources and slow workflow responses.The survey prioritizes metadata exchange between functions within an application.
- Workflow Support: Dataflow execution can prewarm and partially execute downstream functions before a caller finishes, increasing parallelism and lowering response latency.Function B and C can start earlier when data dependencies, rather than function-state dependencies, permit it.
- Workflow Support: Improved data locality could reduce metadata-transfer costs and shift serverless designs from data shipping toward code shipping.Scheduling dependent functions on the same physical node can reduce data transmission, while frequent code transfer raises security and privacy concerns.
- Cold-Start Mitigation: Cold-start prediction is limited by insufficient historical data, especially for sporadically triggered functions, motivating more robust warm-up strategies.The proposed direction uses function and application context, including service category, libraries, and context diagrams.
- Accelerators: Serverless accelerator support requires accelerator-aware scheduling, virtualization, and automatic batching to address costly scaling, isolation, and I/O constraints.The survey presents these as distinct opportunities for integrating accelerators into serverless systems.
9 CONCLUSION
The survey organizes serverless design into four layers and reviews each layer’s responsibilities, relevant work, and practical implications. It concludes that serverless computing remains in its infancy, with substantial potential for future development.
- Survey Scope: The survey presents a four-layer serverless architecture: Virtualization, Encapsule, System Orchestration, and System Coordination.It reviews the responsibility and significance of each layer and discusses implications for adopting state-of-the-art techniques.
- Conclusion: Serverless computing is still in its infancy, leaving further potential to be explored in forthcoming years.This is the survey’s concluding assessment of the field’s maturity and remaining opportunity.