Source-linked AI summary
Performance Evaluation of the Quorum Blockchain Platform
Arati Baliga, I Subhod, Pandurang Kamat, Siddhartha Chatterjee
TL;DR
Quorum’s performance characteristics were insufficiently documented despite enterprise demands for privacy, low latency, and throughput. The paper evaluates Quorum across workloads and consensus algorithms and uses micro-benchmarks to vary transaction and smart-contract parameters. Quorum scales linearly up to 2100 tx/sec in tested workloads, while latency depends on block time and private contracts reduce throughput at higher loads.
Problem
Performance data for permissioned ledger platforms was scarce, despite enterprise requirements for privacy, transaction latency, throughput, and scalability.
Method
The paper experimentally evaluates Quorum using workload-based throughput and latency tests plus micro-benchmarks varying transaction and smart-contract parameters.
Results
Quorum scales linearly up to 2100 tx/sec in tested workloads; RAFT and IBFT have comparable throughput, while private contracts fall below public contracts beyond 600 tx/sec and the system reaches 900 tx/sec before consensus failure.
Takeaways & Limitations
Block time is a key latency control, while payload size and large write sets can materially affect transaction latency and application design.
Takeaways & Limitations
Increasing the write-set size further caused the RAFT consensus mechanism to fail, which the authors identify as a current Quorum limitation.
Abstract
from arXiv · showhide
Quorum is a permissioned blockchain platform built from the Ethereum codebase with adaptations to make it a permissioned consortium platform. It is one of the key contenders in the permissioned ledger space. Quorum supports confidentiality and privacy of smart contracts and transactions, and crash and Byzantine fault tolerant consensus algorithms. In this paper, we characterize the performance features of Quorum. We study the throughput and latency characteristics of Quorum with different workloads and consensus algorithms that it supports. Through a suite of micro-benchmarks, we explore how certain transaction and smart contract parameters can affect transaction latencies.
1. Introduction
Enterprise blockchains address consortium needs for permissioned ledgers, privacy, low-latency transactions, throughput, and integration. This paper evaluates Quorum through platform experiments and micro-benchmarks of transaction and smart-contract parameters.
- Enterprise blockchains provide permissioned ledgers for consortia, with verified identities, admission control, and efficient consensus.
- Enterprise applications particularly require transaction and data privacy, low confirmation latency, high throughput, and horizontal scalability.
- The paper addresses limited performance evidence for permissioned ledger platforms by evaluating Quorum 2.0.
- Custom Quorum micro-benchmarks examine how transaction and smart-contract parameters affect transaction latency and application design.
- The study measures Quorum throughput and latency under controlled workloads, then reports microbenchmark results and implications for blockchain application design.
2. Quorum Overview
Quorum adapts Ethereum into a permissioned platform with controlled membership, privacy-oriented communication, and alternative consensus algorithms. Its supported consensus choices include RAFT for crash tolerance and IBFT for Byzantine fault tolerance.
- Quorum is a permissioned implementation of Ethereum adapted for enterprise blockchain use.
- Permissioned participation restricts network access to provisioned nodes that verify transactions, execute smart contracts, and maintain ledger state.
- Quorum replaces Ethereum’s proof-of-work with RAFT for crash fault tolerance and IBFT for Byzantine fault tolerance.
- Quorum’s node architecture uses Constellation for secure peer communication, with Transaction Manager and Enclave components supporting private transactions.
- RAFT: RAFT uses a leader to generate blocks, requires 2f+1 nodes to tolerate f faulty nodes, and is designed for known consortium members.
- IBFT: IBFT follows a three-phase commit process and tolerates at most f faulty nodes in a network of 3f+1 nodes.
3. Characterizing Latency and Throughput
The evaluation characterizes Quorum using transaction throughput and latency under RAFT and IBFT consensus. Throughput counts committed transactions per second, while latency measures client-observed completion time.
- The experiments summarize Quorum throughput and transaction latency under both RAFT and IBFT consensus algorithms.
- Throughput is the number of transactions per second included in a block and committed to the ledger.
- Transaction latency is the elapsed time from a client request until its response, including response time for read queries.
3.2. Experimental Setup
The experiments use small private Quorum networks with minimum peer configurations for RAFT and IBFT, while clients run on comparable dedicated machines.
- The setup uses three peers for RAFT and four peers for IBFT, matching the minimum configuration requirement for each consensus algorithm.
- Peers and clients run on dedicated machines with 8 vCPUs and 16 GB RAM under Ubuntu 14.04 LTS.
3.3. Client Setup
The evaluation extends Caliper with a Quorum plugin and separates transaction sending from confirmation-event listening to measure throughput and latency reliably.
- Caliper was extended with a Quorum plugin that sends controlled workloads and records throughput and transaction latencies.The plugin runs on client machines and uses block events to identify transaction confirmation.
- A separate client process listens for confirmation events so high-rate workloads do not lose block events.The original client was single-threaded and both sent transactions and listened for confirmations.
3.4. Load Generation
Load generation varies each client’s send rate across successive rounds, producing total network loads from 150 to 1650 tx/sec.
- 150 to 1650 tx/sec: three clients increase send rates from 50 to 550 transactions per second across experimental rounds.Each client sends at a specified rate, pauses for 5 seconds, and then starts the next round.
3.5. Workloads
The study evaluates Quorum with write, null, read, and mixed workloads built around a preloaded smart-contract key-value store.
- Four workloads cover writes, null function calls, local reads, and a 50-50 mixture of reads and writes.All workloads use a smart contract preloaded with key-value pairs.
- Read transactions query a peer’s local key-value store and do not generate blockchain transactions.All clients intentionally send read workloads to a single peer.
3.6. Throughput and Latency Measurements
The measurements examine how block time, workload type, consensus algorithm, and private-contract overhead shape Quorum throughput and latency.
- 3.6.1. Tuning Block time: Throughput remains more or less constant as RAFT block time changes at 750 tx/sec, but transaction latency increases with longer block times.The added latency reflects the increased time needed to include transactions in a block; IBFT’s minimum block time is 1 sec.
- 3.6.2. Latency and Throughput with RAFT Consensus: All workloads scale linearly across the tested transaction rates under RAFT, while latency varies substantially by workload and block time.Read workloads have the least latency because they use local lookups without transaction ordering or consensus; 1000 ms block time produces the highest write and null latencies.
- 3.6.3. RAFT versus IBFT.: IBFT provides slightly higher throughput up to 1500 tx/sec, while RAFT performs slightly better above 1650 tx/sec and IBFT latency is almost double or more for most points.The comparison uses a 1 sec block time for both algorithms.
- 3.6.4. Public versus Private Contracts.: Private-contract throughput matches public-contract settings up to 600 tx/sec, then degrades; loads beyond 900 tx/sec cause consensus failure.The degradation is attributed to messaging and cryptographic overhead, while the 100 ms RAFT setting also encountered a file-descriptor bug above 600 tx/sec.
4. Micro-benchmarking Experiments
Quorum micro-benchmarks examine how read/write set sizes, key-value store sizes, and payload sizes affect transaction latency. Latency rises with larger read/write sets and payloads, while key-value store growth up to 1 million entries has little effect on reads and writes.
- 4. Micro-benchmarking Experiments: The benchmarks vary read-set and write-set sizes, key-value store sizes, and transaction and event payload sizes while averaging latency across many runs.The experiments use a single client, and the read/write-set benchmark invokes a smart-contract method generating the desired reads or writes.
- 4.1. Read Set and Write Set Size: The read/write-set experiment uses a 1 ms RAFT block-time setting to prevent block time from dominating measured transaction latency.The minimum RAFT block-time setting was used for these experiments.
- 4.1. Read Set and Write Set Size: Write latency increases significantly when the write-set reaches 1 million entries, and larger write-set experiments failed in the RAFT consensus mechanism.Read latency also increases with read-set size, but less sharply at 1 million entries than write latency.
- 4.2. Smart contract Key Value Store Size: Read and write latencies are relatively unaffected by key-value store sizes up to 1 million entries.The benchmark prepopulated stores ranging from 1,000 to 1 million entries.
- 4.3. Transaction and Event Payload Sizes: Transaction latencies increase with each 10KB increase in payload, by 25.23% for transaction payloads and 22.4% for event payloads from 1KB to 30KB.The findings inform application design choices for transaction and event payloads.
5. Related Work
Prior work introduced tools and evaluations for benchmarking private and permissioned blockchain platforms, but comprehensive performance data for Quorum was unavailable. This paper addresses that gap with a thorough Quorum performance evaluation using a Caliper plugin.
- 5. Related Work: Blockbench benchmarks private blockchain platforms through performance comparisons of Fabric, Ethereum, and Parity.The cited Blockbench work focuses on Fabric version 0.6, Ethereum, and Parity.
- 5. Related Work: Caliper measures throughput and latency for permissioned blockchain platforms, and the authors built a Quorum plugin for the measurements in this paper.Caliper is incubated as part of the Hyperledger project.
6. Conclusions
The evaluation finds that Quorum scales linearly across tested workloads, with performance differences arising mainly in latency, consensus behavior, and privacy overhead. RAFT and IBFT have comparable throughput overall, while private contracts reduce throughput at higher loads.
- 6. Conclusions: Quorum scales linearly for tested workloads up to 2100 tx/sec, with reads having the lowest latency and null/write latency tied largely to block time.With RAFT, lengthening block time leaves throughput largely unchanged but increases latency proportionately.
- 6. Conclusions: Private contracts reduce throughput beyond 600 tx/sec because confidentiality adds secure-communication and encryption/decryption overhead, with consensus failing beyond 900 tx/sec.The maximum achievable load reported for the private-contract setting was 900 tx/sec.
- 6. Conclusions: Micro-benchmarks show that million-entry write sets sharply increase latency, payload growth from 1KB to 30KB raises latency by 22%-25%, and key-value store growth is unlikely to degrade read/write performance.The conclusion states that application developers retain flexibility in choosing data-set sizes and expanding key-value stores over time.