Source-linked AI summary
STORE: Self-Provisioning Storage-as-a-Service for Serverless Functions
Florian Trimmel, Cynthia Marcelino, Thomas Pusztai, Stefan Nastic
TL;DR
Serverless functions rely on externally provisioned storage, while existing provisioning approaches still require developer configuration. STORE moves self-provisioning to the platform, dynamically selects storage backends, and provides zero-touch operation; evaluation reports up to 84% lower implementation effort with low overhead.
Problem
Serverless functions depend on external storage, but provisioning and configuring those systems still requires manual setup or declarative scripts.
Method
STORE is a platform-level self-provisioning architecture that dynamically selects storage backends and embeds zero-touch configuration into serverless operation.
Results
84%: STORE reduces implementation effort by up to 84% compared with Terraform and Pulumi, with relative overheads as low as 0.19%.
Takeaways & Limitations
STORE enables self-provisioned storage while maintaining performance and scalability comparable to directly employing underlying storage solutions.
Abstract
from arXiv · showhide
Serverless computing provides on-demand elasticity, pay-per-use, and simplified deployment. However, serverless functions are typically stateless and depend on external storage services such as object stores or databases to exchange data or support stateful functions. Provisioning and configuring these storage systems still requires manual setup or declarative scripts, introducing complexity, slowing development, and increasing the risk of configuration errors. To address these challenges, in this paper, we introduce \textsc{STORE}, a self-provisioning storage architecture for serverless functions. STORE automatically selects the optimal storage backend and eliminates developer effort through zero-touch and zero-configuration provisioning, achieved by moving the self-provisioning logic to the platform level. Our evaluation results show that STORE reduces implementation effort by up to 84% compared to well-established Infrastructure-as-Code frameworks such as Terraform and Pulumi while maintaining low latency and linear scalability under realistic workloads, without introducing performance overhead.
I. INTRODUCTION
Serverless functions depend on externally provisioned storage, but existing approaches still require developer configuration. STORE moves self-provisioning into the platform through dynamic storage selection and zero-touch configuration.
- Serverless functions are typically stateless and rely on external storage services such as key-value stores, object stores, and databases.
- Existing infrastructure-as-code approaches reduce abstraction costs but still require developers to configure and maintain deployment scripts.They operate at provisioning time and can introduce complexity and configuration errors.
- STORE defines a self-provisioning storage lifecycle and architecture that automates provisioning and management while eliminating manual configuration.Its lifecycle includes preparation, resolution, policy enforcement, dynamic binding, and translation.
- 84%: STORE reduces implementation effort by up to 84% with relative overheads as low as 0.19%.
- STORE dynamically selects storage backends according to data shape, access pattern, and SLOs without developer intervention.The mechanism also supports redirecting or migrating workloads without service interruption.
- STORE embeds self-provisioning at the serverless platform level to provide zero-touch, zero-configuration storage access with fine-grained permissions.
II. RELATED WORK
Prior abstractions simplify infrastructure or portability but still depend on explicit specifications, modeling, or infrastructure-as-code. STORE instead abstracts storage at the workflow and data-shape level while provisioning and adapting it at runtime.
- Infrastructure-as-code tools remain widely used but require manual specification of the desired system state.
- Intent-based and model-based approaches reduce effort but still require explicit intents, architectures, or infrastructure definitions.Many produce infrastructure-as-code output that still requires IaC tools.
- STORE eliminates infrastructure declarations by abstracting storage at the level of workflows and data shapes.The platform handles provisioning and adaptation transparently.
- Function-centric portability frameworks simplify functions but generally restrict or omit direct interaction with backend storage.
- Portability models typically require upfront abstractions and ongoing static analysis, whereas STORE dynamically provisions and selects storage at runtime.
C. Serverless Storage
STORE shifts storage management from developers to the platform, which selects, provisions, secures, and translates access to underlying storage. Its lifecycle supports zero-touch operation and adaptation across storage types.
- Serverless databases can serve as pluggable underlying storage for STORE through a translation component.STORE itself is not a serverless database.
- STORE abstracts provisioning, scaling, and placement, allowing the platform to manage data according to access patterns, consistency requirements, and cost–performance trade-offs.Data persistence and retrieval are intended to require a single API call independent of function location or execution time.
- STORE self-provisioning proceeds through five phases from preparation to translation to automate provisioning, enforce security, and adapt storage dynamically.
- Preparation: Preparation handles deployment-time groundwork, including ensuring storage systems are available and communicable.
- Resolution: Resolution lets functions communicate with STORE without developer-supplied connection details or authentication credentials.Per-invocation data can be resolved dynamically.
- Permission Management: Permission management supports uniform STORE-level access checks and relayed permissions for systems accessing underlying storage directly.Permissions are managed by security level across functions or workflows.
- Selection: Selection chooses underlying storage using data size, usage statistics, and data structure, and can switch storage when data changes.
- Translation: Translation converts a request into one or more instructions for storing or retrieving data from the selected backend.
B. STORE Architecture Overview
STORE is implemented as modular components above an orchestrator and integrated with a serverless platform. Separate control-plane and data-plane components provision resources, route requests, and enable dynamic storage behavior.
- STORE consists of modules on top of an orchestrator and uses pluggable components to integrate with serverless platforms and storage technologies.
- Control plane: The control plane maintains system state by establishing communication, provisioning STORE and underlying storage, and routing requests.It includes a Watcher, STORE-Service Manager, and Storage Manager.
- Data plane: The data plane comprises the Client-SDK, Auto-Migration Framework, and STORE-Service, managing dynamic storage selection and enabling zero-touch configuration.
1) Components:
STORE comprises six components that provision storage, manage service instances, select and translate storage operations, integrate functions, and support migration. Together, these modules provide zero-touch configuration and compatibility across serverless platforms and storage technologies.
- Components: STORE comprises six components: Watcher, Storage Manager, STORE-Service Manager, STORE-Service, Client-SDK, and Auto-Migration Framework.
- Components: The Watcher instruments serverless resources with configuration information required for zero-touch configuration.
- Components: The Storage Manager provisions underlying storage systems through orchestrator resource definitions that represent desired state and available storage.
- Components: The STORE-Service Manager provisions service instances, supplies storage configuration, and links each tenant to its available storage systems.
- Components: The STORE-Service manages permissions, dynamically selects storage, translates requests, validates storage state, and provisions resources.
- Components: The Client-SDK enables zero-touch integration, while the Auto-Migration Framework maps existing storage-library calls to Client-SDK calls for backward compatibility.
IV. STORE MECHANISMS
STORE dynamically selects storage providers using data characteristics, usage statistics, and optional constraints. It filters incompatible or unsuitable providers, scores the remainder, and handles switching either through penalties or lazy provisioning.
- Dynamic Storage Selection: Dynamic provider selection considers data structure, size, usage statistics, and optional type preselection when choosing a storage implementation.
- Dynamic Storage Selection: Providers can be filtered by type and by a streaming threshold before scoring.
- Dynamic Storage Selection: Providers with incompatible data structures receive a translation penalty, while compatible providers retain their usage-based score.
- Dynamic Storage Selection: Provider switching can include a switch penalty or use lazy provisioning, which starts storage provisioning after the switch decision.With lazy provisioning, the next request triggers the switch after provisioning finishes, avoiding added provisioning latency.
B. Zero-Touch Configuration
Zero-touch configuration removes developer interaction by combining deployment-time instrumentation with runtime resolution and data passing. The Client-SDK retrieves configuration automatically, communicates with storage, and adds runtime-specific response information.
- Zero-Touch Configuration: Zero-touch configuration comprises deployment-time instrumentation, runtime resolution, and runtime data passing.
- Zero-Touch Configuration: At deployment time, the STORE-Service Manager informs the Watcher about active service instances and their reachability so functions can be instrumented.
- Zero-Touch Configuration: The Client-SDK can be used in a function handler without prior configuration and retrieves configuration from resolvers and the service locator.
- Zero-Touch Configuration: The Client-SDK stores and retrieves data as instructed by functions while passing identity information for access control.
- Zero-Touch Configuration: After a function returns, the response writer enriches its response body with runtime-specific information.
- Zero-Touch Configuration: The experiments cover file and tabular storage, scoped permissions, dynamic selection, concurrency control, and workflow operations supported by zero-touch configuration.
A. Experimental Setup
The evaluation measures performance, scalability, and resource usage across realistic serverless workflows and storage types. Results show minor, predictable overhead, linear scaling, and resource consumption within the defined criteria.
- Experimental Setup: The prototype runs on seven Debian 12 Kubernetes virtual machines with 32 CPUs, 378 GiB RAM, and SSD-backed ZFS storage.Knative Serving and Eventing provide autoscaling for high concurrency, with cold starts disabled.
- Results: STORE introduces minor and predictable overhead for file-based and tabular workloads while maintaining scalability across storage types.
- Results: 0.19% to 26.08% is STORE’s contribution to total execution time for file storing and retrieval, decreasing rapidly as input size grows.The service’s streaming design allows processing to begin before the full payload arrives.
- Results: 12.56% to 31.79% is STORE-Service’s contribution to total latency for tabular row storage and retrieval.Translation fully reads and converts request payloads into CQL statements, while overall latency scales linearly with input size.
2) Resource Usage:
STORE adds modest overhead across file, tabular, and workflow workloads, while overhead generally decreases or remains stable as workloads scale. The service and function contribute limited additional execution time relative to underlying storage and orchestration.
- STORE overhead remains between 3 and 514 ms across all scalability experiments.
- File-based workloads: File storing has 71 ms mean absolute overhead, averaging 1.9% relative overhead and peaking at 15.6%.
- File-based workloads: File retrieval has 4.3 ms mean absolute overhead, with relative overhead decreasing from 19% toward negligible levels.
- Tabular workloads: Tabular storing and retrieval have 4.6 ms and 4.2 ms mean overheads, respectively, with stable or decreasing relative overhead as scale increases.
- Workflow workloads: Workflow execution has 101 ms mean overhead, reaching up to 38% for small requests while decreasing consistently with scale.
2) Resource Usage:
STORE resource usage scales predictably with request rate: CPU remains at most linear, while memory is generally stable except for higher buffering needs in file storage. The service maintains low and controlled resource consumption across workloads.
- CPU consumption scales linearly with request rate for both service and function components across all experiments.
- CPU usage: The workflow scenario reaches 3306 millicores for the service and 1699 millicores for functions.
- CPU usage: File storage shows the largest service/function CPU imbalance, at 1708 versus 467 millicores.
- Memory usage: Function memory remains stable across workloads, while file-storage request buffering raises service memory to a maximum of 850 MB.
- Memory usage: Most use cases show an initial 42 MB BaaS allocation bump followed by stable scaling.
D. Line of Code Results
STORE substantially reduces implementation effort by eliminating configuration and provisioning code across serverless use cases. Its code footprint is consistently smaller than Terraform and Pulumi, although lines of code remain only a coarse effort proxy.
- STORE reduces implementation effort from 64.6% to 84.1% compared to Terraform and Pulumi.
- STORE requires 8 to 56 lines across use cases, versus 38–176 for Terraform and 39–158 for Pulumi.
- The largest gap is 120 lines in Shopping Cart Workflow, where STORE uses 56 lines versus Terraform's 176.
- Function-body improvements range from 6.2% to 50.0%, with absolute savings from 1 to 49 lines.
- Lines of code are only a coarse proxy for developer effort and do not capture all aspects of complexity or maintainability.