From Bracha to Coded MBRB: Benchmarking Byzantine Reliable Broadcast Implementations

Algorithms and scope

• The study compares Bracha’s BRB baseline with AFRT and Coded MBRB, which tolerate bounded message omissions; Coded MBRB uses erasure-coded fragments and cryptographic evidence to reduce data movement.AFRT forwards full payloads with accumulated signatures, making AFRT versus Coded MBRB the direct like-for-like comparison.Source: §1 · p. 2
• Single-shot broadcasts isolate per-invocation costs, so the study does not measure sustained throughput of a long-running dissemination service.The design avoids confounding from batching, concurrent instances, admission control, and garbage collection across repeated broadcasts.Source: §1 · pp. 2–3

Approach

• The shared Go implementation uses a centralized controller, persistent pairwise TCP connections, and a manager layer for metrics, message routing, and fault-injection interception.Evaluation combines Shadow simulation, native profiling, GCP and supplementary FABRIC deployment measurements, with silent-node, message-omission, and one partition-based equivocating-sender scenario.Source: §1 · p. 3

Findings

• At n = 30 with a 1 MB payload, Coded MBRB transmits approximately 128 MB versus about 1.7 GB for Bracha and AFRT, roughly a 14× reduction.At an 8 MB payload, Bracha and AFRT exceed 800 MB peak heap per node while Coded MBRB remains around 350 MB; in GCP at n = 5 and 8 MB, Coded MBRB stays below 400 ms while Bracha and AFRT reach about 900 ms.Source: §1 · p. 3
• Coded MBRB uses more CPU at smaller payloads, but Bracha overtakes it around 6 MB; parser checks found no checked violations across the tested configurations.At n = 30 with a 1 MB payload, Coded MBRB executes about 800 million CPU instructions per node, compared with roughly 250 million for Bracha and AFRT; 92,190 runs covered 2,361,600 parser-checked entries.Source: §1 · p. 3

Our contributions

• The paper contributes reusable evaluation components and an open-source artifact with automation, raw-output processing, parser-based consistency tests, and plotting support.It studies executable-system behavior rather than proposing a new algorithm.Source: §1 · pp. 2, 4

2.1 Reliable-Broadcast Abstractions and Algorithmic Variants

• Reliable broadcast has expanded across network models, delivery semantics, and repeated or amortized execution; this study uses three representative BRB and MBRB designs rather than covering that full space.Bracha represents classical full-payload BRB, AFRT signature-based MBRB, and Coded MBRB coded MBRB.Source: §2.1 · pp. 4–5

2.2 Message-Adversary-Tolerant and Communication-Efficient Broadcast

• AFRT and Coded MBRB share the message-adversary model, while coded fragments, vector commitments, and threshold signatures target lower communication at added cryptographic and coding cost.Recent coded BRB designs under the classical reliable-channel model complement this comparison rather than serving as direct baselines.Source: §2.2 · pp. 5–6

2.6 Positioning of This Work

• The artifact is designed to support future comparisons, but the study does not claim Coded MBRB is the most communication-efficient BRB design or evaluate all recent low-communication algorithms.Its scope focuses on BRB and MBRB under Byzantine faults and message-adversary assumptions.Source: §2.1 · p. 5, §2.2 · p. 6, §2.6 · p. 8

2.3 Reliable-Broadcast Implementations and Artifacts

• Prior implementations and prototypes do not provide the same shared comparison framework; this artifact puts Bracha, AFRT, and Coded MBRB in one codebase with common instrumentation and fault injection.The shared infrastructure supports comparison of bytes, message counts, CPU instructions, memory, latency, delivery behavior, and parser-checked outcomes.Source: §2.3 · p. 6

2.4 Reliable Dissemination inside Larger BFT Systems

• Reliable dissemination also underpins larger BFT systems, but those systems are not direct baselines because they include batching, ordering, consensus, and other machinery absent from this standalone study.The paper evaluates the broadcast primitive under common single-shot workloads.Source: §2.4 · pp. 6–7

2.5 Benchmarking, Simulation, and Byzantine Testing

• Simulation and Byzantine-testing research motivates controlled experiments, while this paper’s silent-node, omission, and equivocating-sender scenarios remain narrower than an exhaustive Byzantine campaign.The scenarios test implementation behavior against expected thresholds and parser-checked safety conditions, not as a replacement for general-purpose BFT testing tools.Source: §2.5 · p. 7

2.6 Positioning of This Work

• The paper bridges formal broadcast algorithms and full-system BFT benchmarking by studying three existing implementations under comparable workloads and selected fault-injection scenarios.The executable-system perspective exposes costs such as serialization, cryptographic verification, erasure coding, buffering, copying, and memory allocation.Source: §2.6 · pp. 7–8

3 Methodology

• Bracha serves as a cross-abstraction BRB reference, while AFRT versus Coded MBRB isolates implementation differences within MBRB.The single-shot design excludes batching, sliding windows, concurrent-instance scheduling, and multi-shot garbage collection.Source: §3 · p. 8

3.1 Research Questions and Experiment Mapping

• The four research questions cover communication scaling, CPU and memory costs, deployment latency, and behavior under injected faults.Table 1 maps each question to its experiment, testbed, and metric group.Source: §3.1 · pp. 8–9, Table 1 · p. 9
• The evaluation tests whether implementations meet expected safety, liveness, and threshold behavior under silent-node, randomized-omission, and partition-based equivocation scenarios.The research-question mapping frames these scenarios as the fault-injection experiments.Source: §3.1 · p. 9

3.2 Orchestration and Node Architecture

• The controller distributes configuration and synchronizes runs, but is not part of the reliable-broadcast algorithm being evaluated.Its registration and readiness barriers ensure all nodes are present and connected before the sender begins.Source: §3.2 · pp. 9–10
• Each node separates transport, management, metrics, and algorithm logic, with the algorithm layer serializing and deserializing its own messages.The Manager provides broadcast and per-recipient send primitives over peer-to-peer connections.Source: §3.2 · p. 10
• The Manager records messages, bytes, and delivery timestamps and hosts the shared wrappers for suppressing selected outgoing or incoming messages.Using the same interfaces supports comparison of Bracha, AFRT, and Coded MBRB.Source: §3.2 · p. 10
• Each node reports delivery status, counts, timestamps, memory statistics, and hashes; SHA-256 hashes let the parser check for inconsistent or invalid deliveries.The sender reports the original payload hash, while nodes report a delivered-payload hash if they delivered.Source: §3.2 · p. 10

3.3 Implementation Details Affecting Evaluation

• The controller coordinates registration, readiness, completion, and metric collection, while algorithmic messages are exchanged directly among nodes.Incoming message handling was made concurrent, with shared algorithm state protected by locks and expensive work moved outside critical sections when possible.Source: Figure 1 · p. 11, §3.3 · p. 12
• Final evaluations use Go gob encoding because JSON added substantial encoding overhead and processing cost.Measured network bytes include serialized implementation messages and their algorithmic metadata, signatures, fragments, commitments, or payload data.Source: §3.3 · p. 12
• Coded MBRB uses KZG vector commitments, threshold signatures over BN254, and Reed-Solomon coding, while AFRT uses Ed25519 signatures.The paper ties Coded MBRB’s high fixed CPU cost to commitment generation, proof verification, signature operations, and fragment encoding or reconstruction.Source: §3.3 · p. 12
• Byte counts exclude TCP/IP headers, orchestration traffic, and metric uploads; native CPU measurements include initialization and termination overhead for every algorithm.Memory is measured using Go runtime statistics after the termination barrier is released.Source: §3.3 · p. 12

3.4 Single-Shot Scope

• Single-shot runs isolate one invocation of reliable broadcast from multi-instance mechanisms such as batching, sliding windows, and garbage collection.The authors describe this as a methodological choice, not a claim that deployed systems broadcast only once.Source: §3.4 · pp. 12–13
• The measured per-broadcast costs do not include sustained throughput or wrapper-level effects such as stop-and-wait limits and sliding-window buffer management.Those effects are identified as future-work concerns rather than part of the measured costs.Source: §3.4 · p. 13

3.5 Evaluation Environments

• Shadow provides controlled network and fault-injection experiments, while native profiling measures CPU instructions and Go runtime memory statistics.Unless otherwise stated, configurations are repeated 10 times; Shadow runs use 10 recorded seeds.Source: §3.5 · p. 13
• GCP is the primary deployment-oriented latency environment, using controller timing over a private VPC; FABRIC supplies supplementary evidence on a more distributed infrastructure.FABRIC results are interpreted with node placement and measured RTTs in view; RTT was measured before each broadcast.Source: §3.5 · pp. 13–14
• A shared manager produces a unified JSON schema covering configuration, timestamps, delivery status, hashes, message and byte counts, and memory statistics.The environments are complementary rather than interchangeable sources of evidence.Source: §3.5 · p. 13, Table 2 · p. 14

3.6 Data Collection Pipeline

• Each run produces a JSON file containing node metrics, experiment parameters, and controller-measured start and end timestamps.Parameters include algorithm, network size, fault thresholds, payload size, sender, and injected-fault counts.Source: §3.6 · p. 14
• Shadow outputs record the run seed, while native-profiling and deployment outputs record an iteration index.Native-profiling outputs also include parsed perf instruction counts for each node.Source: §3.6 · p. 15
• The parser checks duplicate, conflicting, invalid, and spurious deliveries, but these deterministic post-processing checks are not formal verification or exhaustive exploration.Checks use delivery counts and hashes, including whether a delivered hash matches the sender hash in correct-sender executions.Source: §3.6 · p. 15
• Plots aggregate node metrics within each execution, then aggregate execution-level values over 10 repetitions; means and one-standard-deviation bands are reported unless noted.Total transmitted data is summed across nodes, while CPU instruction counts can be averaged per node; outliers are not removed.Source: §3.6 · p. 15

3.7 Metric Definitions

• Network overhead totals nodes’ implementation-message bytes per broadcast, including serialized algorithm data but excluding TCP/IP headers, orchestration, and metric-upload traffic.Included data can comprise payloads, fragments, signatures, commitments, proofs, and threshold-signature shares.Source: §3.7 · p. 15
• Implementation-message count measures algorithm-generated messages separately from bytes, so Coded MBRB may send many lightweight messages while transmitting fewer bytes.Source: §3.7 · p. 15
• CPU cost is user-space instructions per node measured with perf, including common initialization-to-termination orchestration overhead for relative comparison.Source: §3.7 · p. 15
• Memory usage reports peak Go HeapSys and cumulative TotalAlloc, capturing maximum heap pressure and allocation churn, respectively.Cumulative allocation helps explain payload-copying and buffering costs.Source: §3.7 · p. 16
• Completion latency runs from the controller’s release of the ready barrier until it receives completion signals from all honest nodes, avoiding clock-skew artifacts but including transit time.Source: §3.7 · p. 16
• Delivery ratio is the fraction of honest nodes delivering before timeout, while safety checks flag duplicate, conflicting, invalid, and spurious deliveries.Fault-threshold results may report absolute delivery counts because MBRB guarantees are stated in terms of delivery power.Source: §3.7 · p. 16

3.8 Fault-Injection Mechanisms

• The Manager layer injects faults without changing honest algorithm code.Source: §3.8 · p. 16
• A silent node stays alive but suppresses algorithmic messages, approximating an undetectable crash rather than a process failure visible through TCP closure.Source: §3.8 · p. 16
• Random omissions suppress up to numMsgDrops recipients per implementation-message broadcast, but do not model an adaptive adversary using global real-time algorithm state.Recipients are sampled uniformly without replacement among honest nodes before messages reach transport.Source: §3.8 · p. 16
• Partition-based equivocation uses two sender instances to send different payloads to network halves, testing one reproducible scenario rather than exhaustive Byzantine behavior.The setup is Twins-inspired and uses separate SplitManager wrappers.Source: §3.8 · p. 17
• Configured bounds are n > 3t for Bracha and n > 3t + 2d for AFRT and Coded MBRB; threshold tests set t and d to the injected silent-node and message-drop counts.The configuration records tolerated parameters separately from actual injected counts; in-transit modification of correct-node messages is outside the omission-only model.Source: §3.8 · p. 17

3.9 Experimental Configurations

• Exp-1 measures communication, CPU, and memory without fault injection, scaling nodes from 10 to 30 and payloads up to 8 MB at 30 nodes.An additional native-profiling run scales payloads from 10 MB to 40 MB at 10 nodes because larger 30-node configurations exceeded host memory limits.Source: §3.9 · pp. 17–18
• Exp-2 measures deployment completion latency, with GCP payload scaling up to 8 MB and node scaling to 11, plus supplementary FABRIC runs up to 30 nodes and 40 MB.Source: §3.9 · pp. 17–18
• Exp-3 tests silent nodes, randomized omissions, and sender equivocation at 30 nodes with a 1 MB payload, comparing delivery against the stated sufficient bounds.Threshold scenarios use (t, d) = (9, 0) and (10, 0) for Bracha, and (5, 7) and (7, 5) for AFRT and Coded MBRB.Source: §3.9 · pp. 17–18
• The parser-check campaign found zero duplicate-delivery, conflicting-delivery, invalid-delivery, or spurious-delivery violations in the checked outputs.Table 4 summarizes the campaign accounting that connects raw experiment outputs to safety-related claims.Source: Table 4 · p. 18, §3.9 · p. 18

4.1 Exp-1: Controlled Scalability and Resource Costs

• At n = 30, Coded MBRB sent roughly 2500 implementation messages versus roughly 1750 for Bracha and AFRT, despite transmitting fewer bytes.Its byte advantage comes from sending more lightweight messages containing fragments and cryptographic evidence.Source: §4.1 · p. 20

4.5 Answers to the Research Questions

• At n = 30, Coded MBRB transmitted close to 0.1 GB versus Bracha and AFRT’s ca 1.7 GB; at 8 MB, latency was below 400 ms versus approximately 900 ms for Bracha and AFRT.These comparisons come from Shadow with a 1 MB payload and GCP with n = 5, respectively.Source: Figure 12 · p. 26, §4.5 · pp. 33–34

4.2 Exp-2: Deployment Latency

• In GCP, Coded MBRB reached approximately 70 ms at n = 11 with a 1 MB payload, versus about 180 ms for Bracha and nearly 150 ms for AFRT; FABRIC latency varies with geographic site placement.FABRIC is treated as supplementary evidence because inter-site RTT reaches 81.00 ms, compared with submillisecond intra-site RTT.Source: §4.2 · pp. 25–26, Figure 13 · p. 27

4.3 Exp-3: Fault Injection, Threshold Behavior, and Parser Checks

• At n = 30, Bracha delivered at 9 silent nodes but not 10, beyond its n > 3t threshold; selected scenarios also showed zero deliveries beyond bounds or under partition-based equivocation.These sufficient bounds are not claimed to be sharp empirical cutoffs.Source: §4.3 · pp. 29–30, Figure 18 · p. 30, Table 5 · p. 32
• As silent nodes increased, CPU instructions, memory use, and message volume decreased because fewer nodes participated; exceeding the threshold also bypassed protocol execution phases.At the maximum within-threshold silent-node configuration, average CPU instructions fell by approximately 30% for Bracha and AFRT and around 50% for Coded MBRB.Source: §4.3 · pp. 30, 32, Figure 21 · p. 33

4.4 Additional Parser-Check Campaign

• Across 92,190 runs, the parser checked 2,361,600 entries and found zero duplicate-delivery, conflicting-delivery, invalid-delivery, or spurious-delivery violations in tested configurations.This is implementation-level evidence, not a formal proof or exhaustive Byzantine testing.Source: §4.4 · pp. 32–33

4.1 Exp-1: Controlled Scalability and Resource Costs

• At an 8 MB payload and n = 30, Bracha and AFRT exceeded 800 MB peak heap while Coded MBRB used approximately 350 MB; CPU costs cross over only at larger payloads.At n = 30, Bracha overtakes Coded MBRB in CPU cost around 6 MB; AFRT does so around 15 MB in the extended n = 10 experiment.Source: §4.5 · p. 34

• The shared Go benchmark evaluates the three algorithms through simulation, native profiling, and deployment-oriented latency measurements, with an artifact supporting extensions to algorithms, workloads, environments, and fault-injection semantics.Source: §5 · pp. 34–35
• At n = 30 and 1 MB, Coded MBRB sends about 14× less data than AFRT (and similarly than Bracha), but executes about 3× more CPU instructions per node.Source: §5 · p. 35
• At 8 MB, Coded MBRB uses around 350 MB peak heap versus over 800 MB for AFRT and Bracha; on GCP it stays below 400 ms versus about 900 ms.Source: §5 · p. 35
• Across 92,190 runs, the parser found none of the checked violations in the tested configurations.Source: §5 · p. 35