PaperlayerView exact revision on arXiv

Source-linked AI summary

funcX: A Federated Function Serving Fabric for Science

Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, Kyle Chard

arXiv:2005.04215v1cs.DC

TL;DR

Scientific workloads need computation that can move near data or specialized resources and can be decomposed into remotely executed functions. The paper presents funcX, a federated FaaS platform for registering and securely running Python functions across research endpoints, and reports million-function-per-second throughput alongside large-scale worker execution. Its experiences also identify boundaries involving tightly integrated applications, portability, allocation models, and state sharing.

  • Problem

    Scientific workloads require mobile, decomposable computation across diverse resources, but existing FaaS and research infrastructure do not support federated heterogeneous function execution well.

  • Method

    funcX combines a cloud-hosted service with endpoint software to register Python functions and execute them securely and reliably on authorized clouds, clusters, and supercomputers.

  • Results

    funcX supports execution across heterogeneous research resources, scales to over 130 000 concurrent workers, and achieves more than one million functions per second with batching on one machine.

  • Takeaways & Limitations

    funcX provides a research-oriented federated FaaS platform with cloud-like invocation and support for distributed scientific workloads.

  • Takeaways & Limitations

    FaaS is not suitable for some tightly integrated applications, containerization may require resource-specific compilation, and stateless decomposition can make state sharing difficult.

Abstract

from arXiv · show

Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or run remotely where resources are available. They also require new design approaches in which monolithic applications can be decomposed into smaller components, that may in turn be executed separately and on the most suitable resources. To address these needs we present funcX---a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. funcX's endpoint software can transform existing clouds, clusters, and supercomputers into function serving systems, while funcX's cloud-hosted service provides transparent, secure, and reliable function execution across a federated ecosystem of endpoints. We motivate the need for funcX with several scientific case studies, present our prototype design and implementation, show optimizations that deliver throughput in excess of 1 million functions per second, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than more than 130000 concurrent workers.

1 INTRODUCTION

Scientific workloads need computation that can move near data or suitable resources, while existing remote-computing and FaaS systems do not adequately support heterogeneous, federated research infrastructure. funcX adapts FaaS for secure, scalable function execution across clouds, clusters, and supercomputers.

  • Remote computing has remained complex and expensive because of unreliable networks, security challenges, and heterogeneous computer architectures.
  • FaaS enables scientists to decompose monolithic applications and dispatch functions to remote resources or event-triggered cloud executions.
  • 40× speedup was reported when a physics task moved from a CPU to an AWS FPGA, including 20 ms round-trip latency and 30 ms execution time.
  • Existing FaaS systems are not designed for heterogeneous research cyberinfrastructure, while existing CI lacks granular, sporadic function execution support.
  • funcX registers Python functions and invokes them on authorized remote endpoints, provisioning resources, staging code and inputs, and managing containerized execution.
  • funcX combines a distributed federated platform with performance enhancements including memoization, function warming, batching, and prefetching.
  • The platform is presented as a foundation for applications that execute functions across heterogeneous, distributed research resources.

2 REQUIREMENTS

Scientific case studies show short-duration functions operating across high-volume, specialized, and rapidly changing data workflows. These workloads require FaaS support for specialized resources, distribution, dependencies, data access, and scalable execution.

  • Six case studies illustrate short-duration scientific functions and their execution-time distributions: metadata extraction, ML inference, crystallography, neuroscience, spectroscopy, and HEP.
  • Metadata extraction: Xtract executes Python metadata extractors near edge data to reduce transfer costs, with typical execution times from 3 milliseconds to 15 seconds.
  • Synchrotron serial crystallography: SSX processes images 1–2 orders of magnitude faster than other methods, creating a need for automated seconds-scale processing and HPC resources.
  • Neuroscience: Neurocartography workflows process approximately 20GB every minute for image quality control, ML-based detection, and preview generation.
  • High energy physics: HEP workflows reduce datasets typically measuring hundreds of petabytes, and query-based analysis is being developed for real-time use.
  • Correlation spectroscopy: XPCS detectors produce megapixel frames at 60 Hz, approximately 120 MB/sec, while correlation functions take about 50 seconds and can run in parallel.
  • Scientific FaaS requires specialized compute, distribution near data, dependencies, varied data access, and authentication beyond existing solutions.

3 CONCEPTUAL MODEL

funcX presents a cloud-hosted service, registered functions, and endpoint agents as a conceptual model for asynchronously executing authorized Python functions on selected compute resources. Its SDK and REST API support registration, invocation, monitoring, and result retrieval.

  • funcX service: The cloud-hosted funcX service exposes a REST API for registering functions and endpoints, executing functions, monitoring execution, and retrieving results.
  • Functions: funcX functions are Python snippets with explicit input signatures and imported dependencies, and may use containers to package required environments.
  • Endpoints: A funcX endpoint is a logical compute-resource entity whose agent enables dispatch, authentication, authorization, provisioning, monitoring, and management.
  • Function execution: Authorized users invoke registered functions asynchronously on selected endpoints, receiving task identifiers for progress monitoring and result retrieval.
  • User interface: The Python SDK wraps the REST API so users can register a function, invoke it with an endpoint and inputs, and retrieve its result.

4 ARCHITECTURE AND IMPLEMENTATION

funcX combines a cloud-hosted management service with endpoint software that provisions resources, dispatches functions, and coordinates workers across heterogeneous research infrastructure.

  • funcX combines a cloud-hosted management service with software agents deployed on remote resources.
  • The funcX Service: The service uses a REST API, persistent registry, Redis storage, and endpoint-specific queues to manage functions, tasks, and results.
  • Reliability: Heartbeats, watchdogs, queued tasks, and re-execution provide resilience to disconnected agents and failed managers.
  • The funcX Endpoint: The endpoint architecture comprises an agent, managers, and workers for queuing tasks, managing resources, and executing containerized functions.
  • Resource Management: Agents use schedulers or cloud APIs and a pilot-job model to provision managers across clouds, clusters, and supercomputers.
  • Optimizations: The map operation partitions iterator inputs into memory-efficient batches, with batch_size controlling tasks per batch and batch_count controlling total batches.

5 EVALUATION

The evaluation measures latency, scalability, throughput, and optimization effects, including comparisons with commercial FaaS platforms. Warm funcX latency is comparable, while cold starts are substantially slower in this setup.

  • Evaluation Design: The evaluation measures latency, scalability, throughput, fault tolerance, batching, memoization, and prefetching.
  • Experimental Setup: For single-invocation comparisons, all platforms execute the same Python function with the same “hello-world” payload.
  • Latency: 111 ms warm round-trip time makes funcX comparable to Amazon Lambda, Google Functions, and Azure Functions.
  • Latency: 1497 ms cold round-trip time for funcX exceeds Amazon Lambda at 469 ms, Google Functions at 223 ms, and Azure Functions at 1360 ms.
  • Latency Breakdown: For warm containers, authentication and internal queuing and dispatching account for most funcX overhead.

5.2 Scalability and Throughput

funcX agents scale across Theta and Cori, maintaining useful scaling for longer-running functions and reaching over 130000 concurrent containers in weak-scaling experiments.

  • Strong Scaling: Completion time for 100000 concurrent requests decreases with more containers, reaching 256 no-op and 2048 sleep containers on Theta before leveling behavior is reported.
  • Weak Scaling: 131072 concurrent containers enabled more than 1.3 million “no-op” functions on Cori.
  • Throughput: 1694 requests per second on Theta and 1466 requests per second on Cori were the maximum observed funcX-agent throughputs.
  • Overall Scaling: funcX scales to 130000+ containers for a single function and provides similar scalability and throughput with Singularity and Shifter.
  • Scope: The experiments study the funcX agent rather than end-to-end funcX throughput.

5.3 Elasticity

funcX endpoints dynamically provision and scale compute resources in response to function load, then terminate unused pods after work completes.

  • The Kubernetes endpoint scales active pods in response to submitted functions with different execution durations.
  • The experiment submits one 1-second, five 10-second, and twenty 20-second sleep functions every 120 seconds.

5.4 Fault Tolerance

funcX uses heartbeat-based failure detection and recovery for managers and endpoints. Failures temporarily increase task latency, which returns toward prior levels after recovery.

  • Heartbeat messages enable funcX to detect and respond to endpoint and manager failures during execution.
  • Manager failure immediately increases task latency as queued work accumulates, then latency quickly decreases after manager recovery.Two managers process tasks at capacity; one is terminated after 2 seconds and restarted after 4 seconds.
  • Endpoint failure similarly increases task latency, which returns to previous levels after the endpoint recovers.The endpoint failure occurs after 43 seconds, with recovery after 85 seconds.

5.5 Optimizations

funcX evaluates container startup and four performance optimizations: batching, prefetching, memoization, and function warming. Batching substantially improves high-volume short-function workloads and enables million-function-per-second throughput.

  • Container initialization: Container initialization is measured across container technologies and execution resources using the baseline cold-function startup steps.The measurements include AWS EC2, Theta, and Cori resources.
  • Batching: 6.7s completion time with batching compares with 118s without batching for 10,000 concurrent no-op requests.The experiment uses four Theta nodes with 64 containers each.
  • Strong scaling: 1.2 million functions-per-second is the peak single-machine throughput for 10 million 10µs functions with batching.The client and endpoint run on one AWS EC2 c5n.9xlarge instance.
  • User-driven batching: Batching most benefits short-running functions, while larger batches add little benefit and long-running functions gain little from communication reduction.For larger batches, distributing requests to additional workers is preferable.
  • Prefetching: Prefetching sharply decreases completion time, with diminishing benefit when the prefetch count exceeds 64.The evaluation uses 10,000 concurrent requests across four Theta nodes with 64 containers each.
  • Memoization: Memoization decreases completion time dramatically as the percentage of repeated requests increases.The experiment submits 100,000 concurrent requests for a one-second function.

6 EXPERIENCES WITH fUNCX

The case studies use funcX to run scientific functions across local, cloud, and HPC resources, integrate computation into automated workflows, and share functions securely. Researchers also identify portability, decomposition, and allocation-model constraints.

  • Scientific case studies: Xtract executes metadata extractors centrally or on remote endpoints where data reside, avoiding cloud data movement in the latter mode.
  • Scientific case studies: DLHub uses funcX to run model inference on arbitrary compute resources with batching and caching optimizations.
  • Scientific case studies: SSX researchers submit the same stills-processing function to local endpoints for validation or HPC resources for full-dataset processing.
  • Reported benefits: Researchers used funcX to access diverse resources without manual management and integrate computation into automated visualization and analysis workflows through APIs.
  • Scientific case studies: A Coffea-backed funcX analysis processed 300 million HEP events in nine minutes, or 1.9 µs per event, using two endpoints.
  • Reported benefits: Secure function sharing lets collaborators apply functions to their own datasets without setting up environments.This was particularly useful when researchers shared access to the same instrument.
  • Challenges: FaaS is unsuitable for tightly integrated, data-sharing applications with large and complex code bases, and containerization may require resource-specific compilation.The XPCS case required compiling code for a target resource.

7 RELATED WORK

Related systems provide cloud, on-premise, container-based, or parallel-computing capabilities, but none directly support function execution across heterogeneous, federated research resources. funcX extends remote-function and container-orchestration ideas toward research cyberinfrastructure.

  • Hosted FaaS: Hosted FaaS platforms provide high-performance, reliable execution but are not designed for heterogeneous research CI and may be costly.
  • Open-source FaaS: Open-source FaaS platforms support local deployment and customization, but commonly depend on Docker and Kubernetes and do not easily adapt to HPC environments.
  • Research gap: No surveyed system supports remote function execution across a distributed or federated ecosystem of endpoints.
  • Remote execution: GridRPC requires developers to implement client and server code, whereas funcX extends remote execution by allowing interpreted functions to be registered.
  • Container orchestration: Container orchestration systems manage scheduling, fault tolerance, and provisioning but primarily target dedicated cloud-like infrastructure rather than most HPC resources.
  • Parallel systems: Data-parallel systems impose a map-reduce programming model on dedicated clusters, while Parsl and Dask execute selected functions within parallel programs.

8 CONCLUSION

funcX provides a distributed FaaS platform for scientific computing, combining secure execution across heterogeneous endpoints with scalable performance. Its case studies show practical benefits alongside limitations involving application suitability, allocation models, and function decomposition.

  • Conclusion: funcX supports scientific computing across HPC systems and cloud platforms by securely executing functions on heterogeneous, specialized endpoints.It provides a cloud-hosted interface and supports three container technologies.
  • Conclusion: 1M tasks over 130 000 concurrent workers demonstrate funcX agent scalability on the Cori supercomputer.The platform also elastically scales with load and responds automatically to failures.
  • Conclusion: More than one million functions per second can be executed on a single machine through user-driven batching.
  • Conclusion: Six scientific case studies identify abstraction, code simplification, portability, scalability, and sharing as advantages of the federated ecosystem.
  • Conclusion: The approach is limited by suitability for some applications, conflicts with current allocation models, and challenges decomposing applications into functions.
  • Conclusion: Future work targets dynamic container creation and staging, container sharing, and resource-aware scheduling.
Loading 2005.04215v1…