Prune First, Decide Fast: Scalable Semantic Query Processing with JEVDB

• Autoregressive LLMs can take hundreds of milliseconds to seconds per tuple, making queries over thousands or millions of tuples computationally and financially expensive.Prior batching, cascades, prompt rewriting, and proxy classifiers do not resolve the underlying computational bottleneck.Source: §1 · p. 2

Relational-Semantic Algebra Mapping

• JEVDB maps semantic filters, joins, classification, and ranking to typed decision primitives instead of generating free-form text for relational outputs.This avoids token-generation overhead and syntax parsing in the semantic execution path.Source: §1 · pp. 2–3

Relational and Semantic Candidate Reduction

• For semantic joins, JEVDB combines Yannakakis-style semijoin reduction over relational regions with Semantic Bloom Filters that screen latent-edge candidates using registered necessary conditions.Extracted condition attributes can be materialized and reused across queries.Source: §1 · p. 3

Adaptive Three-Tier Execution Engine

• JEVDB uses a three-tier execution engine: candidate filtering, high-throughput Jev decisions, and selective frontier-LLM fallback when uncertainty exceeds a threshold or synthesis is requested.The fallback is reserved for uncertain decisions or explicitly requested open-ended text synthesis.Source: §1 · p. 3

Evaluation on Diverse Benchmarks

• On SemBench, JEVDB-Flash has the lowest latency on all 21 queries and lowest cost on 19; on Shelob, JEVDB completes joins with 95.7%–97.5% mean F1.Shelob joins reach 540K candidate pairs, with 87.4% pruned before inference.Source: §1 · p. 3

2.1 Semantic Database Systems and Scalability Approaches

• Batching, generative cascades, agentic rewriting, and embedding-based proxies each mitigate some query overhead, while retaining decoding, prompt-engineering, storage, or fitting trade-offs.JEVDB instead compiles operators into single-pass decision primitives without offline vector precomputation or online model fitting.Source: §2.1 · pp. 3–4

2.2 Semantic Joins and Algorithmic Reductions

• Semantic joins face quadratic pairwise comparisons; JEVDB combines Yannakakis semijoin reduction to remove nonparticipating relational tuples with SBFs screening candidates across latent semantic edges.The cited approach contrasts with embedding-based retrieval, which adds index maintenance and does not solve dangling tuples in multi-table plans.Source: §2.2 · p. 4

2.3 Non-Generative Decision Models and the Jev Ecosystem

• Jev produces calibrated probabilities for Boolean, categorical, and score decisions in one pass, matching relational operators’ typed outputs rather than open-ended conversational synthesis.The related-work discussion notes decision-only models cannot operate in isolation for agentic tasks requiring multi-step synthesis or code generation.Source: §2.3 · pp. 4–5
• JEVDB’s query workflow filters candidates first, evaluates survivors with Jev, and sends uncertain decisions to a frontier LLM.Source: Figure 1 · p. 5

3 Methodology

• JEVDB’s query lifecycle comprises compilation and rewriting, candidate reduction, then fast-path execution with selective escalation of ambiguous cases.Survivors are evaluated by the Jev decision coprocessor; cases within an uncertainty band are escalated to a frontier generative LLM.Source: §3 · pp. 5–6
• The engine isolates distinct text inputs during compilation to avoid redundant inferences and maps semantic tasks to typed decision contracts.It pushes down standard relational filters, applies Yannakakis-style reduction within relational regions, and screens candidates across semantic edges with SBFs.Source: §3 · p. 5
• Semantic SQL clauses bind tuple attributes to natural-language instructions and declare typed outputs, which JEVDB maps to Jev primitives.A model is registered once in the database catalog with endpoint, batch size, concurrency, token-budget, and confidence-threshold options.Source: §3 · p. 6, Table 1 · p. 6

3.1 Semantic Operator Rewriting via Jev Primitives

• JEVDB decomposes semantic SQL expressions into four relational operators implemented with single-pass decision inference.This replaces token-by-token autoregressive generation with native typed decision primitives.Source: §3.1 · p. 6

3.1.1 Semantic Filter (σsem)

• Single-pass Jev filters make semantic predicate pushdown a default, reducing intermediate tuples without generative-model execution penalties.A filter can return a Boolean threshold decision or preserve the raw probability as a DOUBLE attribute.Source: §3.1.1 · pp. 6–7

3.1.2 Semantic Classification (Πsem)

• Choice selects the highest-probability candidate label and exposes it as a typed VARCHAR column usable as a grouping key or join attribute.Labels may be declared in the SQL expression or registered in model options.Source: §3.1.2 · p. 7

3.1.3 Semantic Ranking (Γsem / τk)

• Jev’s Score computes a continuous expected ordinal index over ordered qualitative levels, returned as a DOUBLE for SQL sorting and aggregation.Retaining fractional values preserves probability differences for ORDER BY and downstream numeric operations such as AVG.Source: §3.1.3 · p. 8

3.1.4 Semantic Join (▷◁sem)

• Semantic joins apply Noul to candidate tuple pairs and retain those whose predicate probability meets the threshold.Because an unconstrained join requires |L|·|R| inferences, JEVDB reduces candidates before inference and batches surviving pairs.Source: §3.1.4 · p. 8

3.2 Candidate Reduction for Semantic Joins

• JEVDB reduces work at semantic joins through exact relational semijoins, SBF screening across semantic edges, and reuse of decisions for distinct surviving inputs.The matching pairs remain unknown until the semantic predicate is evaluated.Source: §3.2 · p. 9

3.2.1 Semantic Bloom Filters

• SBFs use registered necessary conditions to screen semantic candidates, with unary source-key summaries or pairwise screens and conservative treatment of unknown keys.Lossless pruning requires a sound condition and sound key extraction; model-derived components validated only empirically may trade recall for cost.Source: §3.2.1 · pp. 9–10

3.2.2 Relational Yannakakis Reduction

• For supported acyclic inner-equijoin regions, Yannakakis-style semijoin passes remove tuples without relational witnesses while stopping at the semantic boundary.Bloom-filter false positives can weaken pruning but do not change results because the exact join conditions remain in the plan.Source: §3.2.2 · p. 10

3.2.3 Input Deduplication and In-Query Caching

• JEVDB evaluates each distinct pointwise semantic input once, caches its decision, and reattaches it to tuples sharing that input.Operators whose semantics depend jointly on an entire candidate batch are not deduplicated this way.Source: §3.2.3 · p. 10

3.3 Join-Aware Execution and Tiered Escalation

• A single Jev request can pair shared context with multiple typed questions, reducing repeated context and transport overhead.JEVDB uses separate, packed, or star layouts for different candidate distributions.Source: §3.3.1 · pp. 10–11

3.3.1 Anchor-Sharing Request Layouts

• JEVDB dynamically chooses star or pack layouts: star shares a high-degree anchor and can save up to (m −1) · |x(li)| tokens.Separate requests remain an option for sparse or evenly distributed pairs; selection is bounded by the model’s prompt context budget.Source: §3.3.1 · p. 11

3.3.2 Block-Level Vectorized Batching

• Blockwise Nested-Loop Join buffers candidate pairs across outer-tuple blocks, dispatches asynchronous concurrent requests, then matches returned judgments locally.This separates model-backend concurrency from the engine’s vectorized pipelining.Source: §3.3.2 · p. 11

3.3.3 Calibrated Three-Tier LLM Escalation

• Dual thresholds fast-accept or fast-reject confident Jev decisions and escalate pairs in [ℓ, h) to a frontier LLM.Online calibration samples candidate pairs and chooses the tightest uncertainty band that stays within user-specified precision and recall loss tolerances.Source: §3.3.3 · pp. 11–12

3.3.4 Condition-index scoring

• Condition-index scoring combines stored document-level probability distributions to score pairs, shifting first-stage inference from candidate pairs to distinct documents.The index is constructed once and reused across queries.Source: §3.3.4 · p. 12

4.1 Experimental Setup

• SemBench covers 21 text-and-structured-attribute queries; raw multimodal-input queries are excluded, leaving multimodal decision execution for future work.The evaluation reports end-to-end cost, latency, and answer quality across SemBench and Shelob.Source: §4.1 · pp. 12–13, §4 · p. 12
• Shelob Q1–Q5 span 81,528 candidate pairs at XS to 1.37 million at M; timed-out runs report progress rather than F1.The protocol caps execution at 1,200 seconds and reports completed-query F1 against reference answers.Source: §4.1 · p. 13, Table 2 · p. 13

4.2 SemBench: Performance on Existing Semantic Workloads

• JEVDB-Flash is fastest on all 21 SemBench queries and cheapest on 19; aggregate latency falls 89.4× vs LOTUS and 72.0× vs Palimpzest.It is best or tied-best in quality on 14 queries and within 0.02 of the best on four more.Source: §4.2 · pp. 13–14, Table 3 · p. 14

4.3 Shelob: Scaling Semantic Joins over Analytical Data

• Both JEVDB modes finish all Shelob runs; full JEVDB sustains 95.7% mean F1 at M, while no baseline completes an M-tier query.The largest individual measured semantic join contains 540K candidate pairs.Source: §4.3 · pp. 14, 16, Table 4 · p. 15

Conclusion

• Across SemBench and Shelob, JEVDB combines fast typed decisions with pruning and selective escalation to reduce semantic-query work.For joins, exact semijoin reduction handles relational structure while SBFs screen candidates across latent semantic edges.Source: §5 · pp. 16–17
• Both JEVDB modes complete all 15 Shelob runs; full JEVDB maintains mean F1 from 97.5% at XS to 95.7% at M.JEVDB-Flash has the lowest latency on all 21 SemBench queries and the lowest cost on 19.Source: §5 · p. 16

Future Work

• Future semantic-aware optimization would use sampled selectivities, candidate degrees, and escalation rates to guide join ordering and screening.The paper also proposes refining estimates during execution for adaptive replanning.Source: §5 · p. 17
• The condition index motivates persistent reusable semantic state, with summaries refreshed as data changes and versioned against their model, prompt, and condition.Proposed uses include secondary indexes and coarse-grained data skipping.Source: §5 · p. 17
• Semantic blocking keys could route likely matches in distributed joins, while selective replication or fallback evaluation handles candidates that cannot be routed safely.The paper identifies skew, approximation, and inference cost as issues for future study.Source: §5 · p. 17
• Future multimodal work would combine representation encoders with non-generative decision heads for predicates spanning text, images, and other modalities.The paper also raises which cross-modal necessary conditions can be extracted, reused safely, and pushed into SBF pruning.Source: §5 · p. 17