PaperlayerView exact revision on arXiv

Source-linked AI summary

Throughput per Megabyte: A Pilot Benchmark of Language-Stack Efficiency for Self-Hosted HTTP Services on a Raspberry Pi 5

William Oliveira

arXiv:2609.11932v1cs.PFcs.DCcs.PL

TL;DR

Self-hosted HTTP services on ARM64 devices face constraints that major public benchmarks do not jointly address, including idle RAM and energy per request. This pilot compares five SQLite-backed CRUD implementations on one Raspberry Pi 5, finding distinct resource-efficiency and throughput patterns across stacks. The conclusions are scoped to this device and workload.

  • Problem

    Existing benchmarks do not jointly measure ARM64 hardware, idle RAM, throughput-to-RAM ratio, and energy per request for self-hosted HTTP services.

    Source: §1 · p. 4, §2.2 · p. 6

  • Method

    A pilot on one Raspberry Pi 5 compares five language/framework stacks implementing equivalent SQLite-backed CRUD APIs and measures RAM, throughput-to-RAM, startup, energy, concurrency scaling, and memory dynamics.

    Source: §1 · p. 4

  • Results

    Rust led weighted throughput-to-RAM at 310.25 req/s/MB (2.8× Go); .NET, Node.js, Go, and Rust formed a raw-throughput cluster, while Python reached 969 req/s.

    Source: §7 · p. 32

  • Takeaways & Limitations

    For this Pi 5 CRUD-over-SQLite pilot, the data support Rust for measured resource efficiency, .NET for absolute throughput and concurrency scaling, and Go when ecosystem breadth is prioritized.

    Source: §7 · p. 33

    No published study simultaneously measures ARM64 hardware, idle RAM, throughput-to-RAM ratio, and energy per request for self-hosted HTTP services.

    Source: §2.2 · p. 6

Abstract

from arXiv · show

Cloud-centric web benchmarks miss constraints that matter for self-hosted services on ARM64 single-board computers, especially idle RAM footprint and energy per request. We ran a pilot benchmark on one Raspberry Pi 5, measuring equivalent SQLite-backed CRUD APIs implemented in Go 1.26/net/http, Rust 1.95/Axum, Python 3.13/FastAPI+Granian, Node.js 24/Fastify, and .NET 10 Native AOT across N=50 randomized runs per stack and endpoint, with a separate concurrency sweep and memory time-series collection. Rust achieved the highest weighted throughput-to-RAM ratio (310.25 req/s/MB, 2.8x Go) because its weighted peak RSS was 7.36 MB, while .NET, Node.js, Go, and Rust formed a raw-throughput cluster with overlapping bootstrap CIs (2,125--2,461 req/s) and Python remained below that cluster at 969 req/s. The concurrency sweep on GET /items/:id showed divergent scaling: .NET reached 26,209 req/s at c=240, Go and Rust plateaued near c=120, Node.js saturated near 9,700 req/s, and Python stayed below 2,300 req/s; memory snapshots showed Rust holding a flat 7.8 MB RSS while Node.js grew by 117 MB without visible GC drops. For RAM-constrained self-hosted CRUD-over-SQLite deployments on this Pi 5 unit, the data support Rust as the most resource-efficient measured stack, .NET as the strongest high-concurrency stack, and Go as a practical option where ecosystem breadth matters; replication is needed before extending these findings to other devices or storage backends.

1 Introduction

The paper studies language-stack efficiency for self-hosted HTTP services on resource-constrained ARM64 hardware, where idle RAM matters but existing benchmarks do not cover the full deployment context. It uses a single-Pi 5 pilot to compare equivalent SQLite-backed APIs and examine resource use, performance, and deployment-related measures.

1 Introduction

  • Existing benchmarks omit key self-hosted constraints: idle RAM matters on shared-resource devices, while common benchmarks target server-grade x86 or algorithmic CPU tasks.The paper notes that practitioners consequently rely on informal comparisons and x86 extrapolations that do not transfer directly.Source: §1 · p. 4
  • Five stacks expose an identical SQLite-backed CRUD API in a single-Pi 5 pilot measuring RAM, throughput-to-RAM ratio, startup time, energy, concurrency scaling, and memory dynamics.Energy is exploratory; concurrency is measured from c = 30–240, and memory dynamics are sampled as a time series.Source: §1 · p. 4
  • The study’s scope is limited to one hardware unit, SQLite, and CRUD workloads; the onboard PMIC’s ±10% accuracy limits confirmatory energy comparisons within that band.The paper says its primary purpose is to characterize gap magnitudes and motivate larger replications.Source: §1 · p. 4
  • The six research questions cover RAM footprint, throughput-to-RAM ratio, startup and deployment size, exploratory energy, concurrency scaling, and memory-allocation dynamics.The questions are scoped to a single Raspberry Pi 5 and SQLite-backed CRUD workload.Source: §1 · p. 4
  • The contributions include a workload-mix sensitivity analysis, runtime memory-management characterization, Pi 5 deployment observations, and a replication package; the authors frame the work as a pilot, not generalizable recommendations.The pilot is intended to characterize effect magnitudes and protocol feasibility, informing rather than replacing multi-device replications.Source: §1 · p. 5

2 Background and Related Work

The background frames self-hosted software as an always-on use case for ARM64 single-board computers, then identifies gaps in existing benchmarks. It also draws on established benchmarking practices and explains that the selected stacks represent dominant choices in the self-hosted ecosystem.

2.1 Self-Hosted Software

  • The Raspberry Pi 5 is presented as a capable ARM64 target for always-on services, with peak power below 27 W.The background describes self-hosted software as applications operated on infrastructure users own or control.Source: §2.1 · p. 5

2.2 Existing Benchmarks and Their Limitations

  • No published study simultaneously measures ARM64 hardware, idle RAM, throughput-to-RAM ratio, and energy per request for self-hosted HTTP services.Table 1 summarizes relevant studies and their gaps.Source: §2.2 · pp. 5–6, Table 1 · p. 6

2.3 Benchmarking Methodology Foundations

  • Prior work motivates repeated measurements and environmental controls, while energy studies frame power as a first-class metric.The paper’s protocol uses 53 runs and controls including pinned CPU frequency, disabled background services, and reboots between language blocks.Source: §2.3 · pp. 6–7

2.4 Language Stack Selection

  • The five stacks were selected to represent dominant language choices in the self-hosted ecosystem; Table 2 documents selection rationale and rejected alternatives.Source: §2.4 · p. 7, Table 2 · p. 8

3 Methodology

The study benchmarks equivalent SQLite-backed CRUD services on one Raspberry Pi 5, using a controlled protocol to measure resource use, throughput, and runtime behavior. It analyzes endpoint-level results and workload-weighted aggregates, while treating energy as exploratory.

3.1 Hardware

  • The hardware setup places wrk on a separate machine and isolates the Pi’s four Cortex-A76 cores for the server benchmark.Sessions also disable swap and non-essential services and discard runs with detected undervoltage or throttling.Source: §3.1 · p. 9

3.2 Workload

  • All five implementations expose identical SQLite-backed CRUD endpoints, with a larger pre-seed for DELETE runs that changes their index and page-cache conditions.The database uses WAL mode; DELETE runs seed 100,000 rows versus 1,000 for other endpoints.Source: §3.2 · pp. 9–10

3.4 Measurements

  • The throughput-to-RAM ratio defines efficiency as requests per second per megabyte, and binary-size comparisons use different deployment artifacts across languages.Go and Rust use self-contained binaries; Python and Node.js include their deployment directories, while .NET uses its self-contained Native AOT output.Source: §3.4 · p. 10

3.5 Experimental Protocol

  • The primary protocol measures five operations separately across language stacks, with 50 randomized measured runs per endpoint and warmups, cold starts, and thermal cooldowns.A sixth mixed workload is descriptive and excluded from hypothesis tests and weighted aggregates.Source: §3.5 · pp. 10–11
  • The study adds concurrency and memory-dynamics phases, testing GET /items/:id at four connection levels and sampling memory during runs.The endpoint was selected to isolate CPU/runtime performance without SQLite write-lock interference; sampling was approximately every 130 ms.Source: §3.5 · p. 12

3.6 Statistical Analysis Plan

  • The analysis combines endpoint-specific tests with weighted aggregates and bootstrapped confidence intervals, and treats energy comparisons as exploratory.Energy-per-request uses a weighted median because its distribution is right-skewed; the PMIC has limited precision and the derived metric is partly tautological.Source: §3.6 · pp. 13–14
  • The weighted analysis combines five endpoint results using an illustrative workload mix, while alternative mixes test sensitivity to that modeling assumption.Endpoint-level analysis keeps operation type separate; the mixed workload is excluded from inferential tests and weighted aggregates.Source: §3.6 · p. 13, Table 7 · p. 13

3.7 Measurement Infrastructure

  • The harness resets and launches each server, measures startup and idle RSS, then collects load, memory, and power data before recording derived metrics.Implementations were reviewed against idiomatic examples and framework documentation, with the checklist and code published for scrutiny.Source: §3.7 · p. 14

4 Results

Across the measured stacks, Rust led the weighted throughput-to-RAM ratio while four compiled/JIT stacks formed an overlapping weighted-throughput cluster; endpoint type shaped results. Concurrency scaling and memory time-series also differed substantially by runtime.

4.1 RQ1: RAM Footprint

  • Rust had the lowest reported footprint, at 4.97 MB idle and 6.62–8.11 MB peak, while Python reached 236.71 MB weighted peak RSS.The 7.36 MB Rust versus 236.71 MB Python weighted peak-RSS comparison is a 32.2× per-instance memory difference on the tested Pi 5.Source: Table 9 · p. 15, §4.1 · p. 15

4.2 RQ2: Throughput-to-RAM Ratio

  • 310.25 req/s/MB was Rust’s weighted throughput-to-RAM ratio, 2.8× Go’s 111.13, while the four compiled/JIT stacks clustered at 2,125–2,461 req/s.Python reached 969 req/s; bootstrap confidence intervals overlapped substantially among the four compiled/JIT stacks.Source: Table 14 · p. 18, §4.2 · p. 18

4.3 RQ3: Startup Time and Binary Size

  • 11.0 ms was Rust’s startup time and 11.2 ms Go’s; .NET Native AOT started in 44.1 ms, while Rust had the smallest self-contained binary at 3.27 MB.Python and Node.js deployment sizes require their runtime or environment in addition to the entry-point script.Source: §4.3 · pp. 18–19, Table 16 · p. 20

4.4 RQ4: Energy Characterization (Exploratory)

  • 2.71 mJ was .NET’s weighted energy per request, compared with 17.42 mJ for Python, but energy results are exploratory and throughput-driven.The PMIC’s ±10% accuracy corresponds to roughly ±0.5 mJ uncertainty per request at these throughput levels.Source: §4.4 · pp. 20–21, Table 17 · p. 21

4.5 RQ5: Concurrency Saturation (Phase 2)

  • 26,209 req/s was .NET’s throughput at c = 240, while Rust and Go plateaued near c = 120 and Node.js near 9,700 req/s.Python remained between 1,780 and 2,236 req/s across the sweep; Node.js p99 latency reached 640 ms at c = 240.Source: §4.5 · pp. 21–22, Table 19 · p. 22

4.6 RQ6: Memory Time-Series (Phase 3)

  • Rust’s RSS stayed flat at 7.7–7.8 MB, while Node.js grew by 117 MB without observed GC cycles during the measurement window.Go showed 36 visible RSS drops; Python’s baseline was 235 MB with moderate growth.Source: §4.6 · pp. 22–23, Table 20 · p. 23

5 Discussion

The discussion identifies Rust as the most resource-efficient measured stack, while Go remains throughput-competitive and .NET offers high throughput with Native AOT. Practical guidance and energy implications are bounded by the pilot’s Raspberry Pi 5 setting and unresolved limits on extrapolation.

5.1 The Efficiency Frontier

  • Rust pairs 2,125 req/s with 7.36 MB weighted peak RAM, while .NET leads throughput at 2,461 req/s; Figure 8 places Rust at the Pareto corner.Go achieves similar throughput at 2,200 req/s but uses 19.72 MB weighted peak RAM.Source: Figure 8 · p. 24, §5.1 · p. 24

5.2 Practical Implications: Observations for Pi 5 Deployments

  • Table 21 presents candidate stacks for deployment scenarios on the tested Pi 5 under a single-process CRUD-over-SQLite workload, not as general recommendations.The table’s energy row is exploratory, and differences among four compiled/JIT stacks fall within PMIC measurement uncertainty.Source: §5.2 · p. 24, Table 21 · p. 25

5.3 Go Hypothesis Evaluation

  • Rust refutes H1 with 310.25 vs. 111.13 weighted RPS/MB and 7.36 vs. 19.72 MB weighted peak RAM.Rust also produces a smaller binary than Go, at 3.27 vs. 8.75 MB.Source: §5.3 · p. 25
  • Go’s throughput and startup are competitive among compiled/JIT stacks, while ecosystem breadth and learning curve remain unmeasured practitioner considerations.The authors state that whether those qualitative factors outweigh Rust’s measured efficiency depends on practitioner priorities.Source: §5.3 · p. 25

5.4 Native AOT .NET

  • Native AOT .NET starts in 44.1 ms, with 21.35 MB idle RAM and 47.65 MB weighted peak RAM, making it viable for constrained ARM64 deployment.Its startup is 4.1× faster than Node.js and 10.7× faster than Python; the comparison to standard JIT-based .NET is described as typical.Source: §5.4 · pp. 25–26
  • .NET reaches 2,461 req/s, but POST and DELETE p99 latency spikes to 1,765 and 2,160 ms; profiling is needed to identify the dominant mechanism.The proposed explanations are non-exclusive, and the study does not determine which mechanism dominates.Source: §5.4 · p. 26

5.5 Python Worker Configuration Trade-off

  • Four Python workers use ≈2.6× peak RAM and improve read throughput, yet one worker has higher RPS/MB on 4 of 5 endpoints; GET /items is the exception.The authors describe right-sizing worker count to available RAM as a first-order optimization for memory-constrained deployments.Source: §5.5 · p. 26

5.6 Energy Implications at Fleet Scale

  • The measured Python-to-.NET energy gap is ≈6.4×, but realistic traffic, idle power, grid intensity, and PMIC accuracy constrain fleet-scale estimates.The authors’ operational conclusion is qualitative: the measured gap does not vanish at scale, while quantitative fleet accounting requires realistic traffic traces and calibrated power profiling.Source: §5.6 · pp. 26–27

6 Threats to Validity

The section qualifies runtime comparisons by identifying database, implementation, measurement, and workload constraints. It also limits generalization to the single-device SQLite setting and notes how statistical and workload-mix choices affect interpretation.

6.1 Internal Validity

  • SQLite file-level locking serializes concurrent writers, so write-path throughput and its throughput-to-RAM ratio do not isolate language-runtime differences.The authors retain SQLite for its relevance to self-hosted applications but say a client-server database replication is needed before making pure runtime claims about write paths.Source: §6.1 · p. 27
  • A single operator may introduce implementation bias that remains unresolved without independent re-implementation by language experts.The authors used idiom reviews and open materials as mitigations, while network jitter, cache effects, and memory-sampling overhead also qualify measurements.Source: §6.1 · pp. 27–28

6.2 External Validity

  • Measurements from one Pi 5 do not characterize unit-to-unit variation, and other boards may differ in memory, microarchitecture, or thermal envelope.The study also notes a larger SQLite index for DELETE than for other endpoints and that performance may change with future framework versions.Source: §6.2 · pp. 28–29
  • The primary c=30 benchmark does not fully exercise runtime differences that emerge at higher concurrency, while the saturation sweep covers only GET /items/:id.The four-worker Python setup gains read throughput at higher RAM cost than the single-worker variant, making configuration relevant to RAM-constrained deployments.Source: §6.2 · p. 29

6.3 Construct Validity

  • Throughput-to-RAM ratio captures one dimension of deployment efficiency, while binary-size comparisons span unlike artifacts and omit developer productivity.The paper reports nine metrics across four research questions rather than relying solely on the composite score.Source: §6.3 · p. 30
  • PMIC energy readings represent board-level SoC power rather than process-level energy, and the Python and Go stacks have database or server implementation asymmetries.Granian is Rust-implemented, while Go's SQLite driver is transpiled; the authors describe these as relevant qualifications to stack comparisons.Source: §6.3 · pp. 30–31

6.4 Conclusion Validity

  • Of 400 post-hoc comparisons, 253 remain significant under family-wide Holm-Bonferroni correction; losses are predominantly among similar compiled/JIT stacks.Comparisons with large effect sizes (d > 0.8) survive the correction.Source: §6.4 · p. 31
  • Weighted rankings remain stable across read-heavy, uniform, and write-heavy mixes, though the declared mix is a modeling choice rather than an observed traffic distribution.The four compiled/JIT stacks retain overlapping CIs across the three mixes, while Python remains significantly below them in each scenario.Source: §6.4 · p. 31, Table 22 · p. 31

7 Conclusion

In this single-Pi 5, SQLite-backed CRUD pilot, Rust had the highest measured throughput-to-RAM ratio, while .NET led raw throughput and high-concurrency scaling. The authors caution against extending these observations to other devices or storage backends without replication.

7 Conclusion

  • 310.25 req/s/MB was Rust’s weighted throughput-to-RAM ratio; .NET led weighted raw throughput at 2,461 req/s, while four compiled/JIT stacks formed a cluster with overlapping bootstrap CIs.The throughput cluster comprised .NET, Node.js, Go, and Rust; Python/4-worker Granian measured 969 req/s.Source: §7 · p. 32
  • 7.36 MB was Rust’s weighted peak RAM, versus 236.71 MB for Python; memory snapshots showed Rust flat at 7.8 MB and Node.js growing by 117 MB without visible GC cycles.Go remained below 20 MB and showed 36 visible GC pauses.Source: §7 · p. 32
  • .NET reached 26,209 req/s at c = 240 in the GET /items/:id sweep, while Rust and Go plateaued near c = 120, Node.js saturated, and Python showed negligible scaling.At c = 240, Rust and Go reached 17,469 and 14,230 req/s, Node.js 9,766 req/s, and Python 2,107 req/s.Source: §7 · p. 32
  • All five core endpoints showed significant language effects; the get_list endpoint most strongly differentiated stacks, while SQLite’s file-level write lock partly capped write-endpoint throughput.Across read-heavy, uniform, and write-heavy mixes, the qualitative language ordering remained stable.Source: §7 · pp. 32–33
  • For this pilot’s Pi 5 CRUD-over-SQLite scope, the data support Rust for measured resource efficiency, .NET for absolute throughput and concurrency scaling, and Go when ecosystem breadth matters.The authors caution against extending the observations to other ARM64 boards, storage backends, or read/write mixes without replication; future work includes additional devices and calibrated energy measurement.Source: §7 · p. 33

Data Availability

The replication package provides benchmark data, memory snapshots, and materials for regenerating the paper’s tables and plots.

Data Availability

  • The replication package includes raw benchmark data, analysis scripts, server implementations, and plotting scripts.Phase 1 and Phase 2 benchmark CSV files are identified by filename and SHA-256 hash.Source: Data Availability · p. 33
  • Phase 3 memory time-series data comprise 3,314 JSON snapshots with timestamped RSS readings at approximately 130 ms resolution.Firmware, bootloader, and PMIC metadata for the tested Pi 5 are also recorded in the package.Source: Data Availability · p. 33
  • A reproducibility container regenerates all tables and plots from the published CSV, with build and run commands in analysis/README.md.Source: Data Availability · p. 33
Loading 2609.11932v1…