PaperlayerView exact revision on arXiv

Source-linked AI summary

Semantic Views: Certified Reuse of LLM Labels Across Semantic Queries

Oktie Hassanzadeh, Dharmashankar Subramanian

arXiv:2610.12768v1cs.DBcs.AI

TL;DR

Semantic filters make costly tuple-level LLM calls, while existing optimizers either miss targets or save little and do not reuse paid labels across predicates. SemViews uses prior labels and other free signals to order tuples, then certifies with fresh samples. In replayed workloads it met targets on 99.63% of queries while saving 28%–45% of exact-evaluation calls.

  • Problem

    Semantic-filter optimizers face costly oracle calls, and prior methods either miss precision and recall targets or reuse only query-independent artifacts rather than labels paid for another predicate.

    Source: §1 · p. 1

  • Method

    SemViews fits and refines a tuple score using prior semantic views, embeddings, and helper scores, then certifies regions using fresh oracle samples.

    Source: §1 · pp. 1–2

  • Results

    0.37% of 145,000 replayed queries missed a target; SemViews saved 28%–45% of exact-evaluation calls.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Earlier labels add the most savings when predicates are strongly related, including 27%–36% fewer calls when the same question returns in different words, without relying on those labels for correctness.

    Source: §1 · p. 2

    The main relations have 5,000 tuples, and relations of millions of tuples have not been measured.

    Source: §8 · p. 12

Abstract

from arXiv · show

Semantic query engines evaluate natural-language filters with a large language model, and every evaluation is a costly oracle call. Optimizers that reduce these calls face a dilemma: cascades that trust a cheap proxy miss their precision and recall targets on many queries, methods with finite-sample guarantees save little, and all of them discard the oracle's labels when the query ends. We present SemViews, a certified semantic filter that orders tuples by signals that cost no oracle calls (the labels that earlier queries left behind as semantic views, a local embedding, and a helper model's score), refines that order with labels it must pay for anyway, and certifies its answer with fresh stratified oracle samples. The answer meets user precision and recall targets with probability at least $1-\delta$ for every workload and every catalog, so a stale or misleading view can cost oracle calls but never accuracy. On four workloads replayed against 725,000 recorded oracle labels, SemViews meets both targets on 99.63% of queries and saves 28%-45% of the oracle calls, whereas the only prior method in our study with the same guarantee needs 1.32x-1.71x as many calls. When a question returns in different words, views cut its calls by a further 27%-36%, with both targets met on at least 99.8% of such queries; similarity caches and entailment-based reuse meet them on only 17%-47%.

1 Introduction

Semantic query optimization reduces costly LLM calls, but proxy-trusting cascades often miss targets, guaranteed methods save little, and paid labels are not carried across predicates. SemViews uses earlier labels to guide ordering while fresh oracle samples certify the answer.

1 Introduction

  • Earlier predicates can provide useful signals for new, differently worded filters, but existing methods either trust proxy calibration or do not reuse labels across predicates.On the workloads, the LOTUS cascade misses targets on 30.8%–65.4% of queries, while BARGAIN with its helper score saves only 2%–10% of oracle calls.Source: §1 · p. 1
  • Uncertified similarity and entailment reuse are unsafe: 56% of asserted implications are violated by more than 10% of antecedent positives, and errors compound along chains.Similarity-cache answers miss a precision or recall target on every workload hit; SemWeave misses on up to 67.8% of queries.Source: §1 · p. 1
  • Three free signals—earlier-query labels, local embeddings, and helper-model scores—shape the tuple order; paid labels refine it, and fresh samples certify the answer.A stale, partial, or misleading view can increase calls but does not weaken correctness.Source: §1 · pp. 1–2
  • 0.37% of 145,000 replayed queries missed a target; SemViews saved 28%–45% of exact-evaluation calls, while every cheaper method missed targets on at least 32.4%.Views removed 27%–36% of calls when the same question returned in different words, while similarity caches and SemWeave missed targets on 53%–83% of queries.Source: §1 · p. 2
  • The study covers semantic filters over text; joins, ranking, and multimodal data are out of scope.SemViews is implemented as a LOTUS optimizer that rewrites semantic filters without changing the engine.Source: §1 · p. 2

2 Background and Problem Statement

The paper defines semantic filters and their precision, recall, and validity targets, then formulates cross-query reuse as minimizing new oracle calls without weakening those targets for any workload or catalog.

2 Background and Problem Statement

  • A semantic filter keeps relation tuples judged to satisfy a natural-language predicate; each tuple-level oracle evaluation is a costly call.The exact answer is defined by the oracle model's labels under the system prompt at temperature zero.Source: §2 · pp. 2–3
  • A workload is a sequence of distinct filters over the same relation and columns; exact-match caches do not hit, although predicates may share passing or failing tuples.Each filter has its own precision, recall, and failure-probability targets.Source: §2 · p. 3
  • Certified cross-query reuse must meet both targets with probability at least 1−δ for every relation, workload, and catalog while minimizing new calls for the next filter.The guarantee is over the method's sampling for the new filter, given the catalog, including catalogs whose earlier results are useless or misleading.Source: §2 · p. 3

3 Semantic Views and Statistical Containment

Semantic views retain prior predicate labels with provenance, but relationships between semantic predicates are statistical rather than logically guaranteed. Errors can compound along containment chains, so SemViews uses views to guide its search rather than assume they answer a new predicate.

3 Semantic Views and Statistical Containment

  • A semantic view is a possibly partial label column bound to a predicate, its input columns, oracle model, and prompt version, with each label marked as oracle or derived.Each query leaves its view behind at no extra cost; views are never assumed correct for another predicate.Source: §3 · p. 3
  • 56% of implications asserted by an LLM judge on the four predicate pools have ε>0.1 on oracle labels.Semantic containment depends on labels on the relation, not simply on the wording of the predicates.Source: §3 · p. 3
  • Chained containment yields ε′ ≤ ε1 + ε2 sel(q)/sel(p), so the error can grow with the selectivity ratio.When sel(q)/sel(p)=10 and ε1=ε2=0.02, the chain guarantees only ε′=0.22.Source: §3 · p. 3
  • SemViews uses predicate relations to decide where to look, while fresh oracle labels decide correctness; its procedure registers oracle and derived decisions in the new view.The algorithm refines its score order, certifies regions, and labels additional uncertain tuples when needed to close the gap.Source: §3 · p. 4

4 Certified Reuse

CertifiedFilter separates tuple ordering from certification: views and other free signals guide where to look, while fresh oracle samples establish whether the answer meets precision and recall targets. Its regions and stopping conditions are designed to preserve finite-sample validity while reducing oracle calls.

4 Certified Reuse

  • CertifiedFilter orders tuples using free signals, then certifies its answer using fresh, uniformly sampled oracle labels.The operator matches views, fits and refines a score, plans three regions, certifies them, and closes any remaining gap.Source: §4 · p. 4

4.1 Matching candidate views

  • SemViews matches the ten most text-similar views, then weights their labels by predictive value on the shaping sample rather than inferred predicate relations.Even negatively related views can help; matching errors affect call savings, not correctness.Source: §4.1 · p. 4

4.2 Shaping: views as features

  • A logistic score combines view indicators, helper-model scores, and local embedding features, with unknown view entries treated as feature values.Past answers to related predicates can be more predictive than a small helper model.Source: §4.2 · p. 4

4.3 Refining the score

  • Up to eight refinement rounds label uncertain tuples in the planned oracle region and refit the score before certification regions are fixed.These labels are already paid for and do not enter a certification bound.Source: §4.3 · p. 4

4.4 Planning the regions

  • The planner minimizes predicted oracle calls over reject, oracle, and accept regions, using pessimistic expected counts and a selected split of δ.Its plan depends on the catalog, helper, embeddings, and shaping labels, but not on later certification draws.Source: §4.4 · pp. 4–5

4.5 Certification

  • With probability at least 1 − δ, CertifiedFilter meets both precision and recall targets for every relation, workload, and catalog.The guarantee follows from exact bounds on fresh samples drawn from regions fixed before certification.Source: §4.5 · p. 5

4.6 Closing the gap

  • If planned certification falls short, gap closing labels the most uncertain unknown tuples until both conditions hold, preserving validity while avoiding a failed query or exact-evaluation fallback.The loop terminates when a region is fully labeled or the certification conditions are met.Source: §4.6 · p. 5

4.7 Discussion

  • Certification imposes a selectivity-dependent call floor that can dominate savings on small relations, while views alter ordering rather than the stratified cascade’s finite-sample guarantee.The floor applies even to a perfect score; with views disabled, CertifiedFilter is a finite-sample stratified cascade.Source: §4.7 · pp. 5–6

5 Maintaining Semantic Views

Semantic views retain labels with provenance for reuse across queries, while treating derived labels as ordering features rather than truth. The catalog supports appends and oracle upgrades without invalidating views; view selection under storage limits remains open.

Registering a view

  • A registered view keeps oracle labels with provenance and derived labels for unlabeled entries, but only the queried predicate’s own oracle labels are used exactly on a repeat query.Derived labels and other predicates’ views shape the score; a repeated predicate therefore reuses its paid oracle labels without trusting derived decisions.Source: §5 · p. 6

Appends

  • New rows enter every view as unknown, so appended data remain usable as score features without invalidating views or recomputing labels.The score model learns how to order new rows from the shaping sample.Source: §5 · p. 6

Oracle upgrades

  • After an oracle upgrade, old views remain usable as features, but their labels are not used exactly under the new oracle.A view’s identity includes its oracle model and prompt version.Source: §5 · p. 6

Storage and lifecycle

  • Views cost two bytes per tuple in memory, while choosing which partial views to complete or drop under a storage budget is left for future work.The experiments retain every view.Source: §5 · p. 6

6 System

SemViews has an engine-independent core and a LOTUS adapter that rewrites semantic-filter nodes to use CertifiedFilter with a persistent view catalog. The implementation preserves LOTUS’s configured prompt and model semantics, and the operator itself has measurable runtime overhead.

System architecture

  • The engine-independent Python core contains the catalog, view matching, certification bounds, and CertifiedFilter; an adapter needs to supply an oracle, optional helper scores, and row identities.A test enforces that the core does not import LOTUS.Source: §6 · p. 6

LOTUS integration

  • The LOTUS adapter rewrites each semantic-filter node as a view-filter node bound to a catalog, leaving other nodes, prompts, and models unchanged.Users opt in by adding one optimizer, and LOTUS source is not modified.Source: §6 · p. 6, Figure 2 · p. 6

Operator cost

  • The operator’s median runtime beyond oracle calls is 849 ms per query in the single-threaded implementation.This includes local tuple embeddings, logistic-model fitting per refinement round, and planner grid search.Source: §6 · p. 6

7 Evaluation

The evaluation tests oracle-call savings, target validity, cross-query reuse, robustness, and sensitivity across four workloads.

7.1 Experimental Setup

  • Four 5,000-tuple relations and 725,000 recorded oracle calls support replay; costs use 20 workload orders, while SemViews validity uses 1,000.The workloads include an external PubMed pool, mechanically generated GoEmotions and DBpedia predicates, analyst-written Reviews predicates, and reworded questions.Source: §7.1 · p. 7

7.2 Relations Among Real Predicates

  • Only 92 of 5,280 ordered predicate pairs were 0.1-contained, and 56% of LLM-asserted implications failed the ε>0.1 test.Taxonomy-derived implications also fail because the oracle is not logically consistent across prompts.Source: §7.2 · p. 7, Table 3 · p. 8

7.3 Oracle Calls

  • 28%–45% fewer calls than exact evaluation makes SemViews the cheapest valid method; BARGAIN with the helper score saves 2%–10% and needs 1.32×–1.71× as many calls.SemViews saves 4×–18× as many calls as BARGAIN with the helper score.Source: §7.3 · pp. 7–8, Figure 3 · p. 8
  • Views save 4%–25% of calls overall and 11%–38% after a strongly correlated predicate; refinement and views sharpen the score, reducing tuples needing labels.Refinement removes another 8%–21% of calls; refinement rounds and the oracle region together account for 47%–65% of calls.Source: §7.3 · pp. 8–9, Figure 4 · p. 9

7.4 Validity

  • SemViews misses a target on at most 0.7% of queries per dataset, below δ=0.05; every method cheaper than SemViews misses on at least 32.4%.Across 145 predicates, no predicate exceeds δ; the worst miss rate is 3.2%, with a 95% upper confidence bound of 4.5%.Source: §7.4 · p. 9, Figure 5 · p. 9

7.5 Repeated Questions in Different Words

  • On reworded questions, SemViews uses 47%–53% of exact-evaluation calls, cuts calls 27%–36% versus no views, and misses targets on at most 0.2%.Similarity caches and SemWeave miss targets on 53%–83% of these queries.Source: Table 6 · p. 9, §7.5 · p. 10

7.6 Robustness

  • Views add no measurable cost on unrelated controls; misleading, partial, appended, and changed-oracle views preserve validity, while informative coverage yields savings.SemViews meets targets on at least 99.0% of adversarial-view queries, 100.0% after appends, and 100% under an oracle change; half-coverage GoEmotions views save 13%.Source: §7.6 · p. 10

7.7 Sensitivity, Ablations, and Overheads

  • As tables grow from 1,000 to 5,000 tuples, cost falls from 81% to 57% of exact evaluation; embeddings are the most valuable free signal in ablations.With refinement, shaping-sample size hardly matters; without refinement, the first-draft operator needs 1.09×–1.27× SemViews’ calls.Source: Figure 6 · p. 10, §7.7 · p. 11, Table 9 · p. 11

8 Discussion and Lessons

Savings are bounded both by the cost of certifying rare predicates and by how well earlier answers predict new ones; the evidence also leaves real sessions and very large relations unmeasured.

8 Discussion and Lessons

  • 31% of exact-evaluation calls remain with a perfect score on 5,000-tuple relations, showing that certification itself imposes a floor, especially for rare predicates.Predicates with at most 2% selectivity cost 80% of exact evaluation, versus 14% above 10%; repeated-tuple simulations reduce the perfect-score cost to 7% at 100,000 tuples.Source: §8 · p. 11
  • Median AUROC from past predicate labels ranges from 0.82 on Reviews to 0.95 on GoEmotions and DBpedia, so those labels rarely settle a new predicate.The passage concludes that views should inform tuple ordering rather than determine the answer.Source: §8 · p. 11
  • Views help substantially when workloads contain related predicates and cost almost nothing otherwise; on these pools, even a perfectly warm catalog saves only 5%–16% more calls than replayed workloads, mostly on GoEmotions.GoEmotions’ gain is likely an upper end; analyst and agent sessions were not measured.Source: §8 · p. 11
  • Certifier and free row-embedding gains over per-query baselines appear on all four datasets.View-related savings vary with the workload’s predicates, unlike these two gains.Source: §8 · p. 12
  • The evaluation uses 5,000-tuple main relations, random predicate order, two taxonomy-generated pools, and two datasets used for hyperparameter tuning; it studies filters only.Relations of millions of tuples and real analyst or agent sessions remain unmeasured.Source: §8 · p. 12

9 Related Work

Prior work optimizes individual semantic operators, offers per-query guarantees, or reuses answers across predicates under assumptions SemViews instead certifies. Related methods also address views and sampling, but target different semantics or outputs.

Semantic operator systems

  • Semantic operator systems optimize individual queries, while SemViews plugs into them to execute filters and retain labels for later queries.Examples include LOTUS [38], Palimpzest [31], Abacus [42], DocETL [46], UQE [7], Nirvana [58], CADENZA [14], PLOP [37], SEMA-SQL [27], Cortex AISQL [28], and QUEST [49].Source: §9 · p. 12

Per-query optimization with guarantees

  • Per-query optimizers cut current-predicate label costs, whereas SemViews shrinks the set on which those methods must work.Prior approaches include LOTUS, BARGAIN [54], streaming cascades [29], ScaleDoc [56], CSV [18], Larch [57], Task Cascades [47], lightweight proxies [5], unified semantic filtering [23], FDJ [55], and plan error-budget methods [43] [17].Source: §9 · p. 12

Reuse across predicates and queries

  • SemWeave and EVA reuse across predicates but trust inferred relations; SemViews instead certifies reuse, while exact-match caches reuse only identical prompts and tuples.Similarity caches [12] [41] also lack guarantees; vCache [44] bounds per-response error rather than result-set precision and recall.Source: §9 · p. 12

Views, caching, and multi-query optimization

  • Structured-data view and multi-query methods rely on logical containment, but semantic predicates are only statistically related, so SemViews uses views to plan rather than decide.Logical relations could be declared as semantic integrity constraints [26], but would still require certification.Source: §9 · p. 12

Sampling, estimation, and statistical inference

  • Sampling and inference methods combine predictions with labeled samples for valid estimates, but generally target aggregates or selectivities rather than precision and recall of a set-valued answer.Examples include ABae [21], OLLA [19], prediction-powered inference [2] [11], and active statistical inference [61].Source: §9 · p. 12

10 Conclusion

SemViews separates planning from certification so side information and reusable labels can guide tuple ordering without entering the guarantee. The appendix proof also establishes a bound for chaining statistical containment and shows it is tight.

10 Conclusion

  • SemViews uses cost-free side information to order tuples, refines that order with already-required labels, and certifies with fresh samples; views help most when predicates are related.Uncertified similarity or entailment reuse misses targets on a large share of queries; extending certified reuse to joins and ranking remains open.Source: §10 · p. 12

A.1 Proof of Proposition 3.2

  • Chaining p-to-q and q-to-r containment bounds the p-to-r error by ε1 + ε2 sel(q)/sel(p).The proof defines the containment errors relative to the positive sets of the predicates.Source: §A.1 · p. 14
  • The chained containment bound is tight: the proof constructs sets attaining equality in both inequalities.The construction gives |P\R| = ε1|P| + ε2|Q|.Source: §A.1 · p. 14

A.2 Proof of Theorem 4.1

The proof conditions on everything chosen before certification, then uses fresh sampling bounds for the fixed regions. It shows that any stopping state meeting those bounds satisfies both targets with probability at least 1−δ.

A.2 Proof of Theorem 4.1

  • Conditional on the pre-certification information F, regions and sample sizes are fixed, and draws are uniform without replacement within each region.The positive counts therefore follow hypergeometric distributions with fixed parameters.Source: §A.2 · p. 14
  • The returned set includes known positives and uncertain tuples in A; its precision and recall are expressed using the remaining unknown positives in X.The stopping proof bounds these quantities using the certified positive counts.Source: §A.2 · p. 14
  • Fresh stratified certification samples and exact bounds ensure every stopping state meets both targets with probability at least 1−δ.The proof conditions on the pre-sampling state and uses a union bound for the region-specific coverage event.Source: §A.2 · p. 14
  • Views, helper scores, embeddings, and fitted scores do not enter the bounds; they only determine the regions and sample sizes before certification.The per-query miss probability remains at most δ conditional on everything that preceded the query.Source: §A.2 · pp. 14–15
  • The argument extends to predefined strata classified as accept, reject, or oracle using simultaneous bounds across strata and scheduled looks.The recall and precision ratios are monotone in each stratum’s positive count.Source: §A.2 · p. 15

B Larger-Scale Experiments

On relations up to 100,000 tuples, SemViews stayed within the targets while its relative oracle-call cost generally declined with size.

B Larger-Scale Experiments

  • The evaluation used two disjoint 100,000-tuple relations, labeled all tuples for eight randomly selected predicates per relation, and replayed 20 random predicate orders.The helper was off; the 5,000- and 20,000-tuple tables were prefixes of the same relations, and nothing was tuned on these data.Source: B Larger-Scale Experiments · p. 15
  • On Reviews, the share of calls for predicates with selectivity at most 2% fell from 95% to 66% as table size grew.The certification floor accounts for most of the cost of rare predicates on small tables.Source: B Larger-Scale Experiments · p. 15, Table 10 · p. 15
  • At 100,000 tuples, calls were 34% of exact evaluation on DBpedia, 55% on Reviews, and 20% with human labels; target misses stayed at 1.2%, 0.6%, and 0.6%, respectively.These are the reported 100,000-tuple costs and maximum miss shares across sizes for each setting.Source: Table 10 · p. 15, B Larger-Scale Experiments · p. 15
  • The tiling simulation predicted 51% of exact-evaluation calls at 100,000 tuples on average across four datasets; DBpedia was below that share and Reviews above it.At this scale, randomly selected predicates were rarely related, so views added nothing; gains came from the certifier and learned score.Source: B Larger-Scale Experiments · p. 15

C Reproducibility

The authors describe how they fixed and tuned experimental settings, identify which datasets were not fully held out, and disclose generative-AI use in code and text.

C Reproducibility

  • Hyperparameters were fixed before full runs, while refinement and planning settings were tuned on GoEmotions and PubMed, which were therefore not fully held out; DBpedia and Reviews were held out.The reported ablation uses the first-draft operator without refinement and with Clopper–Pearson bounds.Source: C Reproducibility · p. 15

D AI-use disclosure

  • Generative AI tools helped write parts of the code and text under the authors’ direction, with policy details listed in the repository.Source: D AI-use disclosure · p. 15
Loading 2610.12768v1…