Source-linked AI summary
ElasTraS: An Elastic Transactional Data Store in the Cloud
Sudipto Das, Divyakant Agrawal, Amr El Abbadi
TL;DR
Web applications benefit from elastic cloud infrastructure, but their database back-ends remain difficult to scale while preserving transactional guarantees. ElasTraS proposes a cloud data store built around partitioned transaction managers, elastic reassignment, scalable storage, and managed metadata. The design provides ACID guarantees for single-partition transactions and targets efficient, scalable, elastic access to partitioned data, while remaining a system in progress with important consistency and availability trade-offs.
Problem
Cloud application databases remain scalability bottlenecks, while highly scalable key-value stores generally stop short of transactional guarantees.
Method
ElasTraS combines a Bigtable-like key-value model with hierarchical transaction managers, partition ownership, elastic reassignment, distributed storage, and a replicated Metadata Manager.
Results
ElasTraS provides ACID guarantees for transactions limited to a single partition and supports efficient, scalable, elastic transactional access to partitioned data.
Takeaways & Limitations
The design targets transactional access for common single-object and statically partitioned enterprise workloads while preserving elasticity across the web, application, and data-store architecture.
Takeaways & Limitations
ElasTraS relies on eventually consistent distributed storage and does not guarantee metadata-system liveness during failures.
Abstract
from arXiv · showhide
Over the last couple of years, "Cloud Computing" or "Elastic Computing" has emerged as a compelling and successful paradigm for internet scale computing. One of the major contributing factors to this success is the elasticity of resources. In spite of the elasticity provided by the infrastructure and the scalable design of the applications, the elephant (or the underlying database), which drives most of these web-based applications, is not very elastic and scalable, and hence limits scalability. In this paper, we propose ElasTraS which addresses this issue of scalability and elasticity of the data store in a cloud computing environment to leverage from the elastic nature of the underlying infrastructure, while providing scalable transactional data access. This paper aims at providing the design of a system in progress, highlighting the major design choices, analyzing the different guarantees provided by the system, and identifying several important challenges for the research community striving for computing in the cloud.
1 Introduction
Cloud applications can elastically scale web and application tiers, but conventional database back-ends remain scalability bottlenecks. ElasTraS proposes an elastic transactional data store combining cloud scalability with transactional guarantees.
- Cloud Computing’s elasticity lets applications allocate resources on demand and scale with changing load.
- In typical three-tier applications, the database back-end becomes the scalability bottleneck because database servers do not easily scale.
- Highly scalable key-value stores such as SimpleDB and Bigtable stop short of providing transactional guarantees even on a single row.
- ElasTraS addresses elastic data-store scalability while providing scalable transactional data access in the cloud.
- The paper presents a system-in-progress design, analyzes its guarantees and design choices, and identifies research challenges.
2 Related Work
Prior scalable cloud data stores often sacrifice distributed transactions, while related database designs do not address elastic scaling. ElasTraS targets scalable and elastic transaction management in the cloud.
- Scalable key-value stores emerged after designers sacrificed distributed transactions to meet web-application scalability requirements.
- Sinfonia demonstrates that carefully used distributed transactions and commitment can support scalable distributed systems.
- Chubby’s Paxos-based consensus supports scalable data-management systems including Google File System and Bigtable.
- Existing cloud database proposals use storage layers such as S3 or unbundle transaction and data managers, but do not address elastic database scaling.
- ElasTraS is designed specifically for scalable and elastic transaction management in the cloud.
3 System Design
ElasTraS combines a key-value data model with a two-level transaction-manager hierarchy, partition ownership, distributed storage, and strongly managed metadata. These components support transactional access and elastic operation while exposing consistency and availability trade-offs.
- 3.1 Data Model: ElasTraS uses a Bigtable-like key-value design in which values have application-specified structure.
- 3.2 Design Overview: A two-level hierarchy of HTMs and OTMs provides transactional guarantees and elastic scalability as demand increases.
- 3.2 Design Overview: HTMs receive requests and either execute them locally or route them to OTMs, which own exclusive access to relevant data partitions.
- 3.2.1 Distributed Storage: The distributed storage layer supplies fault-tolerant shared storage, handles replication, and may be asynchronously replicated and eventually consistent.
- 3.2.1 Distributed Storage: Because failures may produce stale data, the system requires data versioning from the storage layer or through explicit incorporation.
- 3.2.2 Owning Transaction Managers: OTMs use leases from the Metadata Manager to maintain exclusive partition service during reassignment and failure handling.
- 3.2.3 Metadata Manager and Master: The MMM stores partition and lease state, monitors system health, replaces failed OTMs, and performs load-balancing reassignment.
- 3.2.3 Metadata Manager and Master: Paxos-based synchronous replication gives MMM metadata strong consistency and durability, but makes MMM writes costly and does not guarantee liveness during failures.
4 Implementation Sketch
ElasTraS supports strong transactional guarantees within statically assigned partitions, while dynamic partitioning narrows transaction support and elasticity depends on reassignment and instance management.
- 4.1 Transaction Management: Within a partition, ElasTraS provides ACID guarantees, but offers no consistency across partitions or global serializability.
- 4.1 Transaction Management: Exclusive OTM access enables efficient concurrency control, isolation, recovery, and performance without distributed synchronization.
- 4.1 Transaction Management: Dynamic partitioning supports minitransactions, whereas statically partitioned setups support both minitransactions and ordinary single-partition transactions.
- 4.2 Failure Recovery: Write-ahead logging enables recovery techniques such as ARIES, while durable logs must survive loss of an OTM’s local disk.
- 4.3 Elasticity: Dynamic partition reassignment provides the basis for data-store elasticity as load changes.
- 4.3 Elasticity: Under increasing load, partitions can move to less-loaded OTMs or new OTMs can be spawned to absorb demand.
5 Discussion and Conclusion
ElasTraS provides elastic, scalable transactional access for partitioned data while supporting ACID guarantees within a single partition. It is intended to replace database servers in the cloud architecture, extending elasticity across the web, application, and data-store layers.
- ACID guarantees apply to transactions limited to a single partition.ElasTraS provides serializable transactional guarantees within a partition.
- Dynamic partition reassignment enables elasticity and scalability as system load changes.The design can also dynamically partition databases when applications do not statically partition them.
- ElasTraS replaces database servers in the cloud's web-server and application-server architecture.This provides elasticity and flexibility throughout the entire architecture.