Source-linked AI summary
A Byzantine Fault-Tolerant Ordering Service for the Hyperledger Fabric Blockchain Platform
João Sousa, Alysson Bessani, Marko Vukolić
TL;DR
HLF 1.0 lacked a Byzantine fault-tolerant ordering service, while existing blockchain systems faced performance limitations. This paper designs, implements, and evaluates a BFT-SMART-based service with wide-area optimizations, achieving up to 10k transactions per second and half-second irrevocable writes across continents.
Problem
HLF 1.0 lacked a Byzantine fault-tolerant ordering service, while existing blockchain platforms had limited throughput and high transaction-confirmation latency.
Method
The paper designs, implements, and evaluates an HLF BFT ordering service based on BFT-SMART, including WHEAT optimizations for geo-replicated deployment.
Results
Up to 10k representative transactions per second and half-second irrevocable blockchain writes were achieved, including with consensus nodes spread across different continents.
Takeaways & Limitations
HLF with this ordering service can provide high throughput and sub-second irrevable writes in geo-distributed deployments.
Takeaways & Limitations
The evaluation does not evaluate the synchronization phase required when the leader is incorrect or the system is not synchronous.
Abstract
from arXiv · showhide
Hyperledger Fabric (HLF) is a flexible permissioned blockchain platform designed for business applications beyond the basic digital coin addressed by Bitcoin and other existing networks. A key property of HLF is its extensibility, and in particular the support for multiple ordering services for building the blockchain. Nonetheless, the version 1.0 was launched in early 2017 without an implementation of a Byzantine fault-tolerant (BFT) ordering service. To overcome this limitation, we designed, implemented, and evaluated a BFT ordering service for HLF on top of the BFT-SMaRt state machine replication/consensus library, implementing also optimizations for wide-area deployment. Our results show that HLF with our ordering service can achieve up to ten thousand transactions per second and write a transaction irrevocably in the blockchain in half a second, even with peers spread in different continents.
1 Introduction
Blockchain platforms using Proof-of-Work face severe throughput and confirmation-latency limitations, while Hyperledger Fabric targets flexible business applications through pluggable services. This paper addresses HLF 1.0’s lack of a Byzantine fault-tolerant ordering service with a BFT-SMART-based design, implementation, and evaluation.
- Motivation: 7–10s–100s transactions per second and confirmation latencies of up to one hour characterize existing Proof-of-Work systems.The passage gives Bitcoin’s throughput as 7 transactions per second and describes other systems as processing tens to hundreds.
- Hyperledger Fabric: HLF targets business applications and emphasizes flexibility, generality, non-deterministic chaincodes, and pluggable services.Its extensibility includes support for multiple ordering services that write transactions to the blockchain.
- Research gap: HLF 1.0 launched without a Byzantine fault-tolerant ordering service, providing only crash tolerance through Apache Kafka.The version 1.0 release occurred in early 2017.
- Contribution: The paper designs, implements, and evaluates an HLF 1.0 BFT ordering service based on BFT-SMART and extensions for low-latency internet consensus.The evaluation covers both a local cluster and a geo-distributed setting.
- Results: 10k representative transactions per second and half-second irrevocable writes are achieved even when consensus nodes span different continents.The service’s source code is freely available to the HLF community.
- Additional contribution: The paper identifies concerns in applying state-machine replication protocols to HLF’s service model and workload beyond conventional BFT benchmarks.It contrasts these concerns with microbenchmarks and the ZooKeeper-like client-server model used to evaluate BFT protocols.
2 Blockchain Technology
Blockchains maintain growing distributed ledgers as linked blocks protected by hashes and consensus. Permissionless systems use PoW with anonymous participation, whereas permissioned systems use trusted nodes and Byzantine consensus for improved latency and throughput.
- Blockchain structure: A blockchain is a distributed ledger formed by a continuously growing list of transaction-containing blocks.Cryptographic hashes and a consensus mechanism protect the blocks from tampering.
- Blockchain structure: Each block contains the previous block’s cryptographic hash, so forging block j requires forging subsequent blocks j+1...i.Consensus also prevents modification of the chain and decides which block is appended next.
- Permissionless blockchains: Permissionless blockchains are maintained anonymously across peer-to-peer networks and use Proof-of-Work to determine the next block.PoW limits block-production rate by requiring a computationally intensive cryptographic puzzle.
- Permissionless blockchains: PoW imposes energy and time costs that limit transaction latency, even when specialized hardware is used.The computational effort is associated with finding a valid proof.
- Permissioned blockchains: Permissioned blockchains use trusted nodes and traditional Byzantine consensus to order blocks, enabling better transaction latency and throughput.They also control which participants maintain the ledger.
3 Hyperledger Fabric
Hyperledger Fabric is a modular permissioned blockchain platform with pluggable components, including chaincode, membership, and ordering services. Its protocol separates endorsement, ordering, validation, and ledger commitment, while its original ordering options lacked Byzantine fault tolerance.
- Platform architecture: HLF is an open-source modular permissioned blockchain platform supporting pluggable ordering and membership services.Clients manage transactions through chaincodes, endorsing peers, and an ordering service.
- Platform architecture: Chaincode is deployed network code executed and validated by endorsing peers, while the ordering service creates blocks and determines their ledger order.Peers maintain the ledger and a versioned key/value database state.
- Transaction protocol: HLF transaction processing sends client requests to endorsing peers, which simulate chaincode and return read/write sets with endorsement signatures.No ledger updates occur during endorsement.
- Transaction protocol: Clients assemble endorsements into signed transaction proposals, then broadcast them to the ordering service.The proposal includes read/write sets, signatures, and the channel ID.
- Transaction protocol: The ordering service does not inspect envelope contents; it atomically orders envelopes and creates signed blocks for channel peers.Peers subsequently validate the blocks and append them to the channel blockchain.
- Pluggable consensus: HLF’s available ordering modules comprised a centralized non-replicated service and a replicated Apache Kafka/ZooKeeper service capable only of crash fault tolerance.The centralized module is a single point of failure, while Kafka is decentralized and robust but not Byzantine fault tolerant.
4 BFT-SMART & WHEAT
BFT-SMART uses Byzantine state-machine replication to agree on request batches through a leader-driven message pattern, while WHEAT adapts the protocol for geo-replicated deployments. The normal pattern assumes a correct leader and synchronous conditions; otherwise replicas enter synchronization.
- BFT-SMART message pattern: Each consensus instance begins when a leader proposes a valid request batch, after which replicas exchange WRITE messages containing its hash.Replicas then register the proposed batch after verifying the leader and proposal validity.
- BFT-SMART message pattern: Replicas deliver a batch after receiving quorum support through matching WRITE and ACCEPT messages.The supplied protocol description specifies quorum thresholds of ⌈n+f+1/2⌉ messages for both phases.
- BFT-SMART message pattern: The standard message pattern applies when the leader is correct and the system is synchronous; otherwise, synchronization forces replicas to converge on the same execution.The experiments do not evaluate this synchronization phase.
- WHEAT: WHEAT delivers requests tentatively after the WRITE phase and executes ACCEPT asynchronously, using an efficient vote-assignment scheme for geo-replicated environments.The optimization relies primarily on the fastest replicas while preserving the protocol’s safety and liveness properties.
- WHEAT: Vote assignment can improve latency by including a faster spare replica in the optimal quorum.This mechanism provides more quorum choices in systems with heterogeneous replica speeds.
5 BFT-SMaRt Ordering Service
The HLF ordering service combines frontends with a BFT-SMaRt ordering cluster that totally orders envelopes, forms signed blocks, and disseminates matching blocks back to frontends. Its implementation uses HLF consenters, a Java BFT shim, blockcutters, and BFT-SMaRt’s practical durability and reconfiguration support.
- Ordering cluster: The ordering cluster contains 3f + 1 nodes that totally order envelopes received from frontends using BFT-SMaRt replication.Nodes create blocks after collecting a predetermined number of envelopes.
- Block creation and dissemination: Each ordering node creates a block containing ordered envelopes and the previous block’s hash, signs it digitally, and disseminates it to frontends.Frontends collect 2f +1 matching blocks; f +1 suffice when they verify signatures.
- Frontend architecture: Each frontend combines an HLF consenter with a BFT shim that relays envelopes to the ordering cluster and receives blocks from it.The consenter is implemented in Go, while the shim is implemented in Java with client and receiver threads.
- Block assembly: Ordering nodes use a blockcutter to accumulate the totally ordered envelopes until the configured block size is reached, then drain them into the next block.After draining, envelopes receive sequence numbers.
- Block dissemination: A custom BFT-SMaRt replier sends generated blocks to registered frontends instead of returning them as execution results to the invoking client.This connects block creation in ordering nodes with frontend dissemination.
- Operational support: The service also supports durability and ordering-group reconfiguration, with a small log and checkpoint reducing state-transfer cost for joining nodes.The state is small enough to avoid most durability inefficiencies described for state-machine replication systems.
6 Evaluation
The evaluation measures the BFT-SMART ordering service through signature, local-area throughput, and geo-distributed latency experiments. Results show that workload configuration, CPU sharing, replication overhead, and block size shape throughput and latency.
- The experiments evaluate only the ordering service, identified as a typical HLF bottleneck.
- 6.1 Signature Generation: 8.400 ECDSA signatures/second yields a theoretical upper bound of 84.000 transactions/seconds for blocks containing 10 envelopes.
- 6.2 Ordering Cluster in a LAN: Throughput decreases as receivers increase, while larger 1k and 4 kbyte transactions are less affected because replication overhead dominates block transmission.
- 6.2 Ordering Cluster in a LAN: Approximately 50.000 transactions/second is observed with 10 envelopes/block and 1 to 2 receivers.
- 6.2 Ordering Cluster in a LAN: For 100 envelopes/block, smaller envelopes achieve significantly higher throughput across cluster sizes and up to 8 receivers, while 16 and 32 receivers converge across configurations.
- 6.3 Geo-distributed Ordering Cluster: WHEAT latency is almost 50% lower than BFT-SMART’s across frontends, but 100-envelope blocks increase latency by up to 63 milliseconds.
7 Related Work
Related work situates permissioned blockchains as alternatives to Proof-of-Work systems and lists platforms using federated consensus, BFT, or Raft variants. The section also references geo-distributed latency experiments involving Amazon EC2.
- Bitcoin and Ethereum exemplify Proof-of-Work blockchain platforms, with Ethereum additionally supporting smart contracts.
- Permissioned blockchain platforms are motivated by the performance penalty associated with Proof-of-Work.
- Chain uses Federated Consensus, Tendermint implements a BFT protocol, and Kadena uses a Raft variant adapted to Byzantine settings.
- Figure 8 reports Amazon EC2 latency results for four receivers and blocks containing 10 envelopes.
8 Conclusion
The evaluation identifies the factors limiting peak throughput and shows that tuning block size and signature generation can improve throughput. The ordering service maintains high throughput and sub-second geo-replicated latency under demanding conditions.
- Signature-generation rate or the total-order protocol’s envelope-ordering rate bounds peak throughput.
- 2200 transactions/second remains achievable with 400-kbyte blocks sent to 32 receivers in a 10-node cluster.
- Geo-replicated latency stays within half a second under moderate workload using WHEAT, including large block sizes.