Source-linked AI summary
No\=esis: Deterministic-First Retrieval with Two-Tier Context Hydration for Factuality-Critical Queries on Small Local Models
Nicola Cogotti
TL;DR
Small local models can fail to use retrieved evidence reliably, and factuality-critical queries risk plausible fabricated values. No¯esis shifts deterministic judgments into a query plane with model-triggered verbatim hydration; across four ablations, deterministic infrastructure is the primary determinant of answer quality within the stated scope.
Problem
Below 7B parameters, RAG's main limitation is context utilization rather than retrieval quality, and factuality-critical queries can elicit plausible values absent from the corpus despite correct evidence being present.
Source: §1 · p. 1Method
Results
Across four ablations, deterministic infrastructure is the primary determinant of answer quality; removing structure degrades factual recall at 2B by 11.4 points.
Source: §10 · p. 15Takeaways & Limitations
When an address resolves, infrastructure fetches recover fine-level content verbatim with zero additional model turns.
Source: §7 · p. 6A cross-lingual paraphrase caused a retrieval-scope gap in one run, omitting deterministic facts for one program; the model reported their absence rather than inventing them.
Source: §8.6 · p. 12
Abstract
from arXiv · showhide
A wrong number is worse than no answer. Across factuality-critical domains -- audience metrics, scheduling and rights in media; dosages and lab values in healthcare; figures and citations in finance and legal -- a confident but fabricated value is more damaging than an honest admission of uncertainty. Yet this is the dominant failure mode we observe on small local language models: even when correct evidence is present in context, models fabricate plausible numbers and timestamps. Recent work characterizes a real limit of this regime: below 7B parameters, the bottleneck of retrieval-augmented generation (RAG) is not retrieval quality but context utilization. We present Noesis, the deterministic-first query plane of the Noesis architecture, which makes every deterministic judgment before generation. Its mechanisms follow from the ingestion architecture (subject of a separate patent application): (a) a producer-side fact layer rendering precomputed metric facts verbatim without ranking; (b) positional addressing with deterministic cross-source alignment, resolved ahead of query time at zero LLM cost; (c) provenance scoping as an attribution constraint with multi-tier named-reference routing; and (d) two-tier context with model-triggered verbatim hydration. Across four ablations, a 2B model reaches parity with a 35B model on factual integrity (exact values in all runs; zero confabulated numbers on absent-entity traps); structured retrieval beats flat RAG by +11.4 points at 2B; skeleton-only context preserves quantitative answers at 20-30% smaller prompts; and hydration recovers verbatim narrative in ~8s versus ~29s. Two properties matter for regulated domains: each query resolves in a single generation call, and every reported value is traceable to its exact source and position by construction.
1 Introduction
No¯esis combines deterministic query mechanisms to address small-model context-use failures and confabulation on factuality-critical queries. Its contribution is the conjunction of these mechanisms and their measured effect on small-model answer quality.
1 Introduction
- Small models may fail to use retrieved evidence below 7B parameters, while factuality-critical queries can elicit plausible numbers absent from the corpus even when correct evidence is present.The paper identifies confabulation, rather than retrieval miss, as the dominant error for these queries.Source: §1 · pp. 1–2
- Metric facts are computed at ingestion, materialized as graph nodes, and rendered verbatim in context without thresholding.Source: §1 · p. 2
- Positional addressing aligns cross-source observations deterministically ahead of query time, at zero LLM cost.Source: §1 · p. 2
- Provenance scoping constrains attribution without restricting data access, while multi-tier named-reference routing prevents lossy signals from vetoing named references.Source: §1 · p. 2
- The system provides a structural skeleton without prose and model-triggered verbatim hydration, while epistemic tiering assembles evidence by its selection mechanism and scope.The paper presents the mechanisms as a conjunction, not as individually novel components.Source: §1 · p. 2
- For factuality-critical queries over producer-structured corpora, a 2B model matches a 35B model with deterministic infrastructure and degrades gracefully when it is removed.The introduction describes this as the finding of a four-arm ablation study.Source: §1 · p. 2
2 Architecture Overview
No¯esis separates producer-side ingestion from a query plane that activates the resulting knowledge base and assembles model context. The graph navigates to information locations, while embeddings fetch there.
2 Architecture Overview
- The query side activates the structured knowledge base for a question and assembles the context the model will see.The producer side transforms heterogeneous sources into a structured knowledge base with per-source provenance; its internals are out of scope here.Source: §2 · p. 3
- The graph navigates to where information lives, and embeddings fetch at that location, reversing the common priority of vector similarity over structure.Source: §2 · p. 3
- Table 1 summarizes the four mechanisms, where each acts, and the guarantee it provides by construction.Source: §2 · p. 3
3 The Deterministic Fact Layer
The deterministic fact layer moves metric-series computation out of the model and into ingestion. At query time, activated facts reach the model verbatim before prose.
3 The Deterministic Fact Layer
- Small models can fail at arithmetic over raw data series, where locating a maximum and its position requires scanning many rows.In standard RAG, the model receives those rows as prose chunks and must perform the computation itself.Source: §3 · p. 3
- The layer computes extrema and their positions during ingestion, materializes them as deterministic facts, and renders activated facts verbatim before prose.Facts are not scored, ranked, or filtered; producer-declared vocabulary maps metric-name variants to the same series.Source: §3 · p. 3
- 28/28 quantitative runs were correct across both models and context arms, including exact values and positions for peaks, comparisons, and minima.A 2B model explicitly showed its arithmetic for a cross-program difference.Source: §3 · p. 4
4 Positional Addressing and Cross-Source Alignment
Positional addressing makes ordered content and cross-source relations available without model-based alignment at query time. The reported battery found correct metrics and positions without positional confabulation.
4 Positional Addressing and Cross-Source Alignment
- Positional questions require interval-level grounding across sources, which embeddings cannot provide and model-based alignment makes non-reproducible and costly at query time.Source: §4 · p. 4
- Cross-source alignment is resolved deterministically before query time, so questions about what happened at a peak traverse precomputed relations rather than model reasoning.The mechanism is generic to ordered attributes, with format knowledge owned by the producer.Source: §4 · p. 4
- Open-ended questions spanning both programs returned correct per-program metrics and positions in all runs, with no positional confabulation in the battery.When a position was unavailable, the model said so.Source: §4 · p. 4
5 Provenance Scoping and Named-Reference Routing
No¯esis treats named-source scoping as an attribution constraint, while preserving access to multi-entity data. Deterministic name matching prioritizes named references even when similarity signals are weak.
5 Provenance Scoping and Named-Reference Routing
- Named-source scoping bounds attribution, not data access; deterministic inventory and manifest matching prioritizes named references despite weak similarity signals.The rendered provenance rule prevents data from one source being presented as belonging to another, while multi-entity questions remain supported.Source: §5 · p. 4
- 4/4 absent-program trap runs across both models and arms produced honest refusals, with zero corpus numbers attributed to the absent program.Per-source attribution was clean in every comparison-battery run.Source: §5 · p. 4
- A single short token cannot anchor a reference: multi-token matches corroborate it, while long distinctive tokens can anchor alone.Multi-region activation is also required to route a large, varied corpus, reducing incidental matches while preserving legitimate cross-domain routing.Source: §5 · p. 5
- Prior-turn citations become the address for anaphoric follow-ups that name no source; self-contained questions are unaffected by history.At mesh level, named sources commit routing to their owning knowledge bases.Source: §5 · p. 5
6 Two-Tier Context with Model-Triggered Hydration
No¯esis pairs a deterministic skeleton with model-triggered retrieval of verbatim spans, avoiding a static choice between compact quantitative context and narrative evidence. Hydration recovers verbatim content while skeleton-answerable questions stop without a fetch.
6 Two-Tier Context with Model-Triggered Hydration
- A static compression policy cannot reliably serve both quantitative questions and narrative questions requiring verbatim content.Eager top-k prose can bloat context and degrade small models.Source: §6 · p. 5
- Tier 1 contains deterministic facts, graph structure, and coordinates but no prose; the model can answer from it or request bounded file-specific fetches.The system validates requested files and fetches region-bounded verbatim spans; global vector fill is only a starvation valve.Source: §6 · p. 5
- The protocol is training-free and total by construction: failures fall back to the previous variant’s answer, and unknown file names are discarded.It uses a prompt contract without control tokens, reinforcement learning, or constrained decoding.Source: §6 · p. 5
- Quantitative answers remained identical in value with 20–30% smaller prompts; hydration recovered verbatim quotes with exact timestamps in ∼8 s versus ∼29 s for eager full context.Verbatim-requiring questions triggered hydration 5/6 times, while structure-answerable questions stopped at phase 1 with zero fetch cost.Source: §6 · p. 5
7 Epistemic Tiering and Address-Bounded Assembly
Epistemic tiering orders evidence by how it was selected, placing deterministic facts and addressed spans ahead of semantic fill. In the targeted battery, content-verified answers were correct across both model sizes, while addressed assembly reduced context and avoided hydration fetches.
7 Epistemic Tiering and Address-Bounded Assembly
- A ∼2B model often used unrelated positions despite the correct temporal span being present, while the 35B model succeeded in all runs.Cross-program similarity fill added same-time rows without indicating which evidence was selected for the question; position bias further complicates use of middle context [4].Source: §7 · p. 6
- No¯esis assembles evidence by epistemic status: computed facts, deterministically addressed spans, then semantic fill under the residual token budget.Addressed spans carry their source and window and appear after Tier 0; Tier-2 fill is constrained to resolved files, with global fill resuming if bounded fetch under-delivers.Source: §7 · p. 6
- A named source bounds the answer domain, while a named entity anchors within it without pruning structure; addressed references route to their owning knowledge base.The assembly constraint bounds noise without starving open questions, and unresolved addressing degrades to unconstrained behavior.Source: §7 · p. 6
- The coarse level is deterministic structure from which fine-level content is recoverable verbatim, unlike learned hierarchical compression or virtual-memory paging approaches.When addresses resolve, infrastructure fetches require zero additional model turns.Source: §7 · p. 6
- Content-verified, both models answered 24/24 targeted runs correctly; the 2B model’s sole automated-check defect was omission of the program name, with no forbidden-content needles firing.The 35B model answered all 24 strictly correctly; manual review found the 2B answers contained the correct scene’s verbatim dialogue.Source: §7 · p. 6
- The paper maps each measured production failure to a resolving mechanism and the evaluation verifying its guarantee.The authors describe the ablations as checks of specific, falsifiable claims.Source: Table 2 · p. 7
- Addressed HYDRATE runs used smaller contexts than FULL and required no hydration fetch; cross-KB routing reduced context to roughly a quarter of its prior size.Multi-domain questions without an address retained their full cross-KB route.Source: §7 · pp. 6–7
8 Evaluation: Ablation Study
The four-arm ablation study tests whether deterministic infrastructure, rather than model scale, determines factuality and stability on producer-structured broadcast corpora. It compares 2B and 35B local models across targeted and extended query batteries, with ground truth checked against source files.
8.1 Ablation D: Model Swap under Fixed Infrastructure
- Under fixed infrastructure, swapping the 35B generator for 2B changed nothing on factuality-critical questions: every quantitative answer was exact, every trap refusal honest, and attribution stayed clean.Source: §8.1 · p. 8
8.2 Ablation A: Structured vs. Flat RAG at 2B
- +11.4 points in factual recall and +7.2 in bridge score favored structured context over flat RAG at 2B, at the cost of larger prompts.The flat baseline used identical routing but removed graph structure and the fact layer.Source: §8.2 · p. 8, Table 5 · p. 9
8.3 Ablations B and C: Skeleton and Hydration
- Skeleton-only context preserved quantitative answers with prompts 20–30% smaller, while hydration recovered verbatim narrative in about 8 seconds versus about 29 seconds for eager full context.Narrative-structural questions also received explicit boundary declarations rather than confabulated scenes.Source: §8.3 · pp. 8–9, Table 6 · p. 9
8.4 Extended Robustness and Scale Battery
- On four extended query classes, the 35B model passed 20/20 runs and the 2B model 19/20; its lone dense single-fact miss was an honest refusal despite evidence in context.The full extended battery also passed all five dimensions at N = 5, including negative controls with no invented values.Source: §8.4 · pp. 9–10, Table 7 · p. 10
- 85/85 final-build runs and 60/60 certification runs passed under greedy decoding, with no absent-entity attribution in either trap question.The 17-query battery produced identical needle outcomes across five runs per question.Source: §8.4 · p. 10
8.5 Context Budgeting Ablation
- Removing the neural readout preserved all 50 multi-KB passes while reducing prompt size by 42–50% and median end-to-end latency by 37%.The single failure in the 85-run collection was on a single-KB endpoint, outside the policy's scope.Source: Table 8 · p. 11, §8.5 · p. 11
- 26% smaller prompts and 26% lower median time-to-first-token accompanied score-proportional skeleton budgeting, with the extended battery still passing 85/85.Unlike a uniform source-order cut, the allocation retained skeleton shares for every high-relevance knowledge base.Source: Table 9 · p. 11, §8.5 · p. 11
8.6 Findings
- Evidence is directional rather than statistical: evaluations used small demo knowledge bases in one broadcast-media domain, and a cross-lingual paraphrase occasionally weakened provenance scoping.The paraphrase miss was an honest absence report; the canonical form passed 10/10 across two batteries.Source: §8.6 · pp. 12–13
9 Related Work
Related work explores structured retrieval, model-triggered retrieval, context compression, hierarchical context, and graph-based assembly. Noesis is positioned around deterministic structure, producer-computed facts, and addressed, provenance-preserving retrieval and hydration.
Structure-guided and constraint-driven RAG
- Structure-first systems formulate exact retrieval as subgraph matching [11] [12]; Noesis adds interval-addressable positions, producer-computed verbatim facts, and named-reference authority across KB tiers.The comparison distinguishes positional grounding and producer-computed facts from LLM-extracted KG facts.Source: §9 · pp. 13–14
Model-triggered retrieval
- Trigger-based retrieval spans low-confidence, reflection-token, evaluator, need-driven, and constrained-decoding designs [15] [16] [17] [18] [19] [20]; Noesis uses an unmodified-chat-model prompt contract.The paper says competing approaches require training or a separate evaluator, while Noesis requires neither.Source: §9 · p. 14
Context compression and re-expansion
- Unlike compression that does not re-expand [21] [22], latent decompression [23], or static graph-selected spans [24], Noesis preserves producer-declared facts and addressed, provenance-labeled fetches.The paper acknowledges token-level predictability-based elision in LLMLingua; its distinction is semantic structure with deterministic guarantees.Source: §9 · p. 14
Hierarchical context assembly and virtual memory for LLMs
- Hierarchical summaries are lossy and require learned re-derivation, whereas Noesis keeps deterministic structure and lossless addressed verbatim content [7].Noesis’s fine level is lossless where addressing reaches.Source: §9 · p. 14
- Virtual-memory systems spend model turns on context faults [5] [6]; Noesis fetches resolved addresses in infrastructure and returns provenance-labeled windows.When an address resolves, Noesis’s fetch does not spend a model turn.Source: §9 · p. 14
Graph navigation and context construction
- GraphRAG, HippoRAG, CLAUSE, and ProgRAG use summaries, graph activation, trained decisions, or progressive retrieval [25] [26] [27] [28]; Noesis’s graph answers where information lives, and the model requests only needed content.Noesis uses deterministic activation and on-demand hydration rather than progressive expansion.Source: §9 · pp. 14–15
Small-model RAG
- Small-model work identifies context utilization, not retrieval quality, as the sub-7B bottleneck [3]; Noesis supplies precomputed verbatim answers without model training [9].The paper frames the two approaches as complementary: Noesis avoids requiring small models to extract answers from prose.Source: §9 · p. 15
10 Conclusion
For factuality-critical queries over producer-structured corpora, the conclusion is that deterministic infrastructure primarily determines answer quality and stability, while model scale is secondary. The evidence is directional and limited to small broadcast-media demo KBs.
10 Conclusion
- For producer-structured factuality-critical queries, deterministic infrastructure dominates quality and stability: 2B preserves exact values, honest refusals, and attribution versus 35B under fixed infrastructure.The conclusion states this scope explicitly and reports that swapping the 35B model for a 2B model preserves those properties in all runs.Source: §10 · p. 15
- Removing structure lowers 2B factual recall by 11.4 points; skeleton-only context preserves quantitative answers at 20–30% smaller prompts; hydration recovers narrative in ~8 s versus ~29 s.These are the reported outcomes of ablations A, B, and C, respectively.Source: §10 · p. 15
- The extended cold-stack battery passed 85/85 at N = 5 across 17 queries and 18 active KBs, with consistency across five question classes and graceful concurrent-load degradation.The conclusion reports these results as confirmation beyond the targeted set.Source: §10 · p. 15
- Removing the neural readout (42–50% of prompt) preserved measured quality and cut median latency by a third; score-proportional budgeting cut prompt size and time-to-first-token another quarter.The latter change had no measurable quality change.Source: §10 · p. 15
- Evidence is directional: batteries use N = 5, one-factor arms N = 1–3, demo KBs are small, evaluation covers broadcast media, and ablation-D latency uses warm cache.The conclusion notes that the structured-versus-flat comparison uses needle-based metrics that do not measure answer style and slightly favor flat RAG.Source: §10 · p. 15
- Future work targets defensive hydration and fallback policies, multilingual filename aliases for paraphrases, and model-calibrated elision based on target-model surprisal.These proposals include forcing hydration when a question references events absent from the skeleton.Source: §10 · pp. 15–16