Source-linked AI summary
Bespoke OLAP: Synthesizing Workload-Specific One-size-fits-one Database Engines
Johannes Wehrstein, Timo Eckmann, Matthias Jasny, Carsten Binnig
TL;DR
General-purpose OLAP systems pay overhead to support arbitrary schemas and queries, while bespoke engines can specialize for fixed workloads. Bespoke OLAP synthesizes such engines from a workload contract using iterative code generation, validation, and performance feedback. It reports 11.17× and 45.33× speedups over DuckDB on TPC-H and CEB, respectively, and completes synthesis within hours at a cost of a few dollars.
Problem
General-purpose OLAP engines incur performance overhead from supporting arbitrary schemas, queries, and access patterns, while constructing workload-specific engines has historically required extensive expert engineering.
Method
Bespoke OLAP uses a workload contract and an autonomous LLM-guided pipeline to generate storage, ingestion, and query-execution code, then iteratively validates and optimizes it.
Results
11.17× and 45.33× speedups over DuckDB are achieved on TPC-H SF20 and CEB SF2, respectively, with corresponding 7.24× and 9.56× speedups over Umbra.
Takeaways & Limitations
Per-workload specialization can plausibly become routine because Bespoke OLAP synthesizes engines within hours at a cost of only a few dollars.
Takeaways & Limitations
The presented synthesis pipeline is limited to in-memory engines, excluding buffer management, page eviction, and I/O scheduling from its design space.
Abstract
from arXiv · showhide
Modern OLAP engines are designed to support arbitrary analytical workloads, but this generality incurs structural overhead, including runtime schema interpretation, indirection layers, and abstraction boundaries, even in highly optimized systems. An engine specialized to a fixed workload can eliminate these costs and exploit workload-specific data structures and execution algorithms for substantially higher performance. Historically, constructing such bespoke engines has been economically impractical due to the high manual engineering effort. Recent advances in LLM-based code synthesis challenge this tradeoff by enabling automated system generation. However, naively prompting an LLM to produce a database engine does not yield a correct or efficient design, as effective synthesis requires systematic performance feedback, structured refinement, and careful management of deep architectural interdependencies. We present Bespoke OLAP, a fully autonomous synthesis pipeline for constructing high-performance database engines tightly tailored to a given workload. Our approach integrates iterative performance evaluation and automated validation to guide synthesis from storage to query execution. We demonstrate that Bespoke OLAP can generate a workload-specific engine from scratch within minutes to hours, achieving order-of-magnitude speedups over modern general-purpose systems such as DuckDB.
1 INTRODUCTION
General-purpose OLAP engines incur unavoidable overhead because they must support arbitrary schemas, queries, and access patterns. Bespoke engines remove that overhead for fixed workloads, but producing correct and performant engines requires substantial engineering effort and iterative refinement.
- 11.17× and 45.33× lower total runtime over DuckDB are reported for TPC-H and CEB in single-threaded execution.The figure also reports 7.24× and 9.56× advantages over Umbra on the same workloads.
- General-purpose OLAP engines pay an inherent performance tax for runtime schema interpretation, generic tuple layouts, and broadly applicable storage and execution algorithms.This overhead reflects the cost of supporting arbitrary relational workloads rather than poor engineering.
- Fixed schemas and known query templates let bespoke engines tailor storage layouts, encodings, and compiled execution directly to observed workload access patterns.Dynamic planning, generic operator selection, and fallback paths can be removed from the runtime path.
- Bespoke-engine construction has historically been economically impractical because high-performance database systems require years of expert engineering.
- Correctness and performance are difficult to achieve independently because storage formats, data layouts, execution strategies, and optimizations are deeply interdependent.A correct execution strategy may still be inefficient, while changes in one layer can regress another.
2 OVERVIEW OF BESPOKE OLAP
Bespoke OLAP generates one purpose-built engine from a workload contract containing query specifications and data. Its synthesis pipeline combines workload-specific code generation with empirical optimization, while the current scope focuses on in-memory C++ engines.
- A Bespoke Engine Is Defined by the Workload: The DBMS contract fixes the design space through a database schema, parameterized query templates, and the underlying dataset.The dataset is supplied as Parquet files and informs the physical storage layout during ingestion.
- A Bespoke Engine Is Defined by the Workload: The Bespoke Agent generates storage, ingestion, table representations, and query execution code whose internals are shaped by the workload contract.The resulting engine is a purpose-built artifact rather than a configured instance of an existing system.
- Synthesis Requires Both Pipeline and Infrastructure: The synthesis pipeline moves from storage planning to basic query implementation and then through empirical multi-round optimization.Optimization uses measured performance rather than cost estimation.
- A Bespoke Engine Is Defined by the Workload: Workload-bounded scope enables the pipeline to eliminate abstractions needed only for queries the workload will never present.
- Scope of This Work: The experimental pipeline targets in-memory engines and uses C++ for control over memory layout, low-level execution, compiler feedback, and optimization headroom.Restricting the design to in-memory execution excludes buffer management, page eviction, and I/O scheduling.
3 THE GENERATION PIPELINE
The generation pipeline builds a storage plan, establishes a correct engine, and iteratively optimizes it using profiling, expert knowledge, validation, and benchmarking. Infrastructure supports branching per-query optimization, hotpatching, regression tracking, and fallback handling for workload changes.
- Storage Planning and Functional Implementation: Storage layout planning is performed before execution code because later changes to storage can require extensive downstream modifications.The initial plan is followed by generated C++ structs and ingestion code that populate in-memory storage.
- Optimization: Figure 3 organizes synthesis into repeated validated loops for storage planning, functional implementation, and iterative optimization using data summaries, profiling, expert knowledge, and engineering perspective.
- Pipeline and Infrastructure: The agent interacts with the system through compiler, benchmark, shell, and patch tools that provide actionable feedback, workload measurements, inspection, and structured code changes.Benchmarking compares results against DuckDB across parameter instantiations and reports per-query runtimes.
- Storage Planning and Functional Implementation: The functional engine first establishes correctness against DuckDB before performance optimization begins.This known-good baseline anchors subsequent optimization and regression checks.
- Optimization: Optimization combines self-directed tracing, curated database research knowledge, and workload-specific decomposition while targeting dominant runtime costs.Profiling localizes expensive loops and operators before changes are made.
- Changing Workloads: Ad-hoc queries outside the workload contract are routed through a materialized flat relational representation to a general-purpose engine such as DuckDB.The invariant preserves relational semantics despite internal compression, reordering, or reorganization.
- Infrastructure: Figure 4 combines shared preparation, per-query optimization threads, hotpatching, automated validation, and snapshot-based rollback for a recoverable synthesis loop.Hotpatching allows component replacement without restarting the database process.
4 A SYSTEM FOR SYSTEM GENERATION
System generation is an incremental, infrastructure-supported process that repeatedly compiles, validates, measures, profiles, and refines a workload-specific engine. Shared foundations are established first, then per-query agents optimize independently while live code updates, versioning, and recovery preserve validated progress.
- Synthesis proceeds through well-defined stages from workload specification to a fully optimized engine, rather than as a single generation step.
- A dedicated infrastructure repeatedly compiles, loads, queries, measures, and profiles the database engine so every change can be verified quickly.
- Shared storage, ingestion, and query-interface artifacts establish a consistent foundation before query-specific optimization begins.
- Separate agent threads optimize individual query templates from the shared history while serializing changes to shared components.
- Component-level recompilation and live swapping keep storage resident and the database process running throughout synthesis.
- Version tracking prevents validated progress from being lost when an agent produces a degradation, supporting recovery during exploration.
5 EXPERIMENTAL EVALUATION
The evaluation tests Bespoke-OLAP on TPC-H and CEB and examines its speedups, their sources, and the effects of bespoke storage and agent strategies.
- The evaluation uses TPC-H and CEB to assess speedups over a state-of-the-art general-purpose OLAP engine.
- The evaluation investigates where speedups originate, including optimization stages, bespoke storage design, and agent strategies.
5.1 Experiment Setup
The experiments use complementary analytical workloads and established generation procedures to evaluate Bespoke-OLAP reproducibly across datasets and scale factors.
- TPC-H provides a standardized industrial OLAP benchmark with analytical queries for which existing engines are heavily optimized.
- CEB complements TPC-H with templated JOB queries over the real-world IMDB dataset.
- TPC-H data follows the TPCH v3.0.1 specification, while CEB uses published query instantiations and provided scaling tools.
- Both benchmarks follow established data-generation and query-instantiation procedures to support reproducibility.
5.2 Performance and Scalability
Bespoke-OLAP substantially outperforms DuckDB and Umbra across single-threaded, multi-threaded, and increasing-scale evaluations. Its advantages persist across queries and can widen on CEB as data grows, while one unstable DuckDB query instantiation is excluded from plots.
- Overall and Per-Query Speedups: 4.4 s versus 49.2 s yields an 11.17× speedup for Bespoke-TPCH over DuckDB at TPC-H scale factor 20.
- Overall and Per-Query Speedups: 0.4 s versus 19.5 s yields a 45.33× speedup for Bespoke-CEB over DuckDB at CEB scale factor 2.
- Overall and Per-Query Speedups: Bespoke-OLAP outperforms DuckDB on every plotted query, with per-query speedups ranging from 2.83× to 102× on TPC-H and 11.6× to 1500× on CEB.
- Overall and Per-Query Speedups: CEB Query 8a is excluded from all plots because DuckDB runtimes vary from 10× to over 100× across instantiations, whereas Bespoke-CEB remains stable.
- Overall and Per-Query Speedups: Bespoke-OLAP outperforms Umbra on all but two queries across both benchmarks, with the exceptions at 0.60× and 0.94×.
- Scalability with Increasing Data Sizes: On CEB, the speedup over DuckDB grows to over 45× at scale factor 10, while TPC-H speedup remains stable across tested scale factors.
- Scalability with Increasing Data Sizes: In multi-threaded execution, Bespoke achieves 7.65× over DuckDB and 6.12× over Umbra on TPC-H, plus 23.97× and 1.87× on CEB.
5.3 Anatomy of the Speedups
Bespoke OLAP’s speedups arise from cumulative optimization stages, workload-specific storage, abstraction-free query execution, and targeted implementation strategies. Bespoke storage and query code reinforce each other, while the generated system combines classical techniques with workload-specific choices.
- Impact of Optimization Stages: Each optimization stage contributes meaningfully to cumulative speedups on TPC-H and CEB.Figure 8 measures each stage after including all preceding optimizations.
- Impact of Optimization Stages: 2.34× over DuckDB comes from bespoke storage alone on TPC-H before query-execution code is optimized.Storage choices include zone-map pruning, packed aggregates, and a precomputed discounted-price column.
- Impact of Optimization Stages: 2.10× over DuckDB but 0.44× over Umbra shows that bespoke storage alone is insufficient for CEB’s irregular access patterns.Later stages add cardinality information, self-tracing, expert knowledge, and a human reference persona.
- Impact of Optimization Stages: Query-specific layouts and algorithms produce 12.35× on TPC-H and 51.40× on CEB over DuckDB.For Query 1, grouped sorting enables binary searches and AVX-512 reductions; the final engine contains approximately 11,600 lines of C++.
- Impact of Bespoke Storage: Flat storage reaches 1.26× on TPC-H and 0.57× on CEB, while bespoke storage reaches 12.35× and 51.40×, respectively, over DuckDB.The comparison isolates abstraction-free query processing from workload-specific physical layouts.
- Analysis of Employed Strategies: The agent uses targeted strategies across encoding, query-support structures, scans, joins, aggregation, and low-level execution.Examples include dictionary encoding, bitmap semijoins, fused aggregation, hash indices, derived columns, branch hints, and software prefetching.
- Analysis of Employed Strategies: TPC-H Query 5 bounds scans using physical order and range directories, while Query 13 uses bitmasks to reject nonmatching comments before substring search.These examples illustrate column- and query-specific decisions rather than one broadly applicable strategy.
5.4 The Synthesis Process
The synthesis process traces Bespoke-TPCH’s development trajectory, including agent turns and implementation progress.
- Development Trajectory and Speedups over Time: Figure 9 traces Bespoke-TPCH’s synthesis process from start to finish across agent turns.The passage introduces the development trajectory as part of assessing the approach’s practical viability.
6 RELATED WORK
Prior systems reduce generality at the workload-class or query level, while Bespoke OLAP targets deeper architecture-level specialization through LLM-driven synthesis.
- Manual Specialization: Manual specialization improves performance by adapting execution and storage to data-intensive OLAP workloads, but generally operates at the workload-class level.Examples include MonetDB/X100, column stores, and main-memory systems such as HyPer.
- Query Compilation: Query compilation generates executable code for individual queries, reducing interpretation overhead and enabling inlining and loop fusion.HyPer, LegoBase, and DBToaster demonstrate this approach within a fixed architecture.
- LLM-Driven Synthesis: Synthesizing a workload-specific DBMS enables deeper architecture-level optimizations than methods confined to a fixed architecture.The related-work discussion positions Bespoke OLAP as combining specialization with broader system generation.
7 CONCLUSION AND FUTURE WORK
Bespoke OLAP synthesizes complete workload-specific database engines and reports large speedups over DuckDB and Umbra. Future work targets the remaining gap to in-memory performance and broader workload support.
- Conclusion: 11.17× over DuckDB and 7.24× over Umbra on TPC-H, and 45.33× and 9.56× on CEB, are reported for Bespoke OLAP.The paper reports these results for TPC-H SF20 and CEB SF2.
- Conclusion: Each engine is synthesized within hours at a cost of only a few dollars, making routine per-workload specialization plausible.The conclusion contrasts this automation with specialization historically reserved for exceptional, high-value deployments.
- Future Work: Current single-threaded disk-based execution does not yet reach in-memory performance.Future synthesis must address bespoke buffer pools, eviction policies, and block-aligned page layouts.
- Future Work: Extending bespoke synthesis to OLTP introduces consistency and isolation guarantees as a new synthesis dimension.The paper also identifies lightweight cost models as a way to prune unpromising optimization alternatives for disk-resident workloads.