PaperlayerView exact revision on arXiv

Source-linked AI summary

Stemming Hallucination in Language Models Using a Licensing Oracle

Simeon Emanuilov, Richard Ackermann

arXiv:2511.06073v1cs.CLcs.AIcs.LGcs.LO

TL;DR

Language models can produce syntactically coherent but factually false claims, and standard RAG lacks principled abstention when retrieved information is insufficient or contradictory. This paper introduces the Licensing Oracle, which validates factual claims against structured knowledge using formal constraints; it reports AP = 1.0 and FAR-NE = 0.0 for its approach.

  • Problem

    Language models can produce fluent but factually false claims, while standard RAG lacks principled abstention when retrieved information is insufficient or contradictory.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    The Licensing Oracle checks generated factual claims against a structured knowledge graph and SHACL constraints, allowing validated claims through and abstaining on failures.

    Source: §1 · p. 2, §3.1 · p. 6

  • Results

    AP = 1.0 and FAR-NE = 0.0: the paper reports perfect abstention precision and no false claims among questions the system answered.

    Source: §7 · p. 18

  • Takeaways & Limitations

    The experiments support deterministic validation as a solution to hallucinations about structured knowledge, unlike statistical approaches that do not reliably ground claims.

    Source: §1 · p. 2, §5.6 · p. 14

    The evaluation assumes perfect claim extraction, and validation accuracy for subtle distinctions depends on the knowledge graph’s predicate precision and relationship coverage.

    Source: §4 · p. 6, §6.3 · p. 17

Abstract

from arXiv · show

Language models exhibit remarkable natural language generation capabilities but remain prone to hallucinations, generating factually incorrect information despite producing syntactically coherent responses. This study introduces the Licensing Oracle, an architectural solution designed to stem hallucinations in LMs by enforcing truth constraints through formal validation against structured knowledge graphs. Unlike statistical approaches that rely on data scaling or fine-tuning, the Licensing Oracle embeds a deterministic validation step into the model's generative process, ensuring that only factually accurate claims are made. We evaluated the effectiveness of the Licensing Oracle through experiments comparing it with several state-of-the-art methods, including baseline language model generation, fine-tuning for factual recall, fine-tuning for abstention behavior, and retrieval-augmented generation (RAG). Our results demonstrate that although RAG and fine-tuning improve performance, they fail to eliminate hallucinations. In contrast, the Licensing Oracle achieved perfect abstention precision (AP = 1.0) and zero false answers (FAR-NE = 0.0), ensuring that only valid claims were generated with 89.1% accuracy in factual responses. This work shows that architectural innovations, such as the Licensing Oracle, offer a necessary and sufficient solution for hallucinations in domains with structured knowledge representations, offering guarantees that statistical methods cannot match. Although the Licensing Oracle is specifically designed to address hallucinations in fact-based domains, its framework lays the groundwork for truth-constrained generation in future AI systems, providing a new path toward reliable, epistemically grounded models.

1. Introduction

Language models can produce fluent but factually false claims, and this paper treats hallucination as a structural grounding problem. It proposes the Licensing Oracle and reports that it outperforms statistical approaches in preventing false answers.

1. Introduction

  • Hallucination is described as fluent, syntactically correct output that is empirically false, linked to generation based on statistical patterns rather than verifiable knowledge.The introduction also draws on prior work arguing that hallucination is structural, not merely a consequence of insufficient training data or poor fine-tuning.Source: §1 · p. 1
  • The Licensing Oracle embeds formal verification of factual claims into generation, validating them against a structured knowledge graph and SHACL rules.The introduction frames this architectural approach as distinct from methods relying on data patterns and probabilistic reasoning.Source: §1 · p. 2
  • AP = 1.0 and effectively no false answers: the Licensing Oracle provides a structural alternative to statistical methods that do not reliably eliminate hallucinations.The experiments compare baseline generation, factual-recall and abstention fine-tuning, RAG, and graph-based RAG with formal validation.Source: §1 · p. 2
  • The paper focuses on hallucinations about structured facts, such as claims about entities and relationships representable in a knowledge graph.It argues that these can be mitigated through architectural enforcement rather than better data or larger models.Source: §1 · p. 2

2. Related Work

The related work contrasts statistical accounts of hallucination with architectural and formal-validation approaches. It frames the Licensing Oracle as combining knowledge graphs and SHACL to place a real-time truth constraint on generation.

2. Related Work

  • The related-work review argues that statistical approaches improve performance without deterministic guarantees, while formal validation has remained disconnected from generation.It organizes the review around statistical methods, architectural solutions, and integration of formal validation into generative AI.Source: §2 · p. 2, §2.6 · p. 4

2.1 The Nature and Origins of Hallucination

  • The paper describes competing accounts: hallucinations may be treated as correctable training or inference artifacts, or as structural consequences of transformers lacking grounding in verifiable evidence.Its prior-work framing distinguishes ontological hallucinations from residual reasoning hallucinations and calls the underlying condition a “flat semantic space.”Source: §2.1 · pp. 2–3

2.2.1 Fine-Tuning for Factual Grounding

  • Fine-tuning for factual grounding is presented as limited: the paper reports accuracy falling from 16.7% to 8.5% after fine-tuning.The passage connects this result to the “fine-tuning paradox” reported by Gekhman et al., 2024, and cites documented forgetting and ripple-effect failures.Source: §2.2.1 · p. 3

2.2.2 Training for Epistemic Abstention

  • Learned abstention remains unreliable in the reported experiments, with the Gemma-Abstain model achieving only 56.7% abstention precision.The passage says this R-Tuning-based model learns linguistic patterns rather than genuine epistemic boundaries.Source: §2.2.2 · p. 3

2.3 Retrieval-Augmented Generation: Capabilities and Gaps

  • Standard RAG can express unwarranted certainty when retrieved information is insufficient or contradictory because it lacks principled abstention mechanisms.The passage names this gap “epistemic mismatch.”Source: §2.3 · p. 4

2.4 Graph-Based Retrieval and Formal Validation

  • The paper reports similar retrieval performance for embedding-RAG and Graph-RAG, and argues that knowledge graphs matter chiefly for formal verifiability.It situates its approach alongside neuro-symbolic validation and LINC, which translates natural language into First-Order Logic for theorem-prover validation.Source: §2.4 · p. 4

2.5 SHACL and Formal Constraint Validation

  • The paper’s stated SHACL distinction is its control flow: SHACL governs LLM generation in real time rather than being generated or interpreted by the LLM.The validation gate is described as mandatory before factual claims are emitted.Source: §2.5 · p. 4

2.6 Research Gap and Contribution

  • The proposed architecture integrates LLMs with formal knowledge graphs and real-time SHACL validation, moving from retrieval-augmented to constraint-governed generation.The authors present this as empirical validation and engineering implementation of their earlier architectural account.Source: §2.6 · pp. 4–5

3. Theoretical Framework and Design of the Licensing Oracle

The Licensing Oracle is designed as a control layer that checks generated factual claims against a structured knowledge graph and SHACL constraints. The design allows validated claims through and abstains on claims that fail validation.

3. Theoretical Framework and Design of the Licensing Oracle

  • The Oracle shifts control from probabilistic language patterns toward structural mechanisms that demand factual accuracy and logical consistency.It is described as a control layer that grounds claims through knowledge-graph validation and addresses the “flat semantic space” problem.Source: §3 · p. 5

3.1 The Knowledge Graph and SHACL Constraints

  • The graph validates extracted subject-predicate-object triples against SHACL rules, and a claim that fails validation leads to an abstention rather than emission.The described constraints include type, logical, and temporal checks.Source: §3.1 · pp. 5–6

3.2 Model Integration and Validation Process

  • During streaming generation, the middleware extracts potential factual claims as triples, validates them with SHACL, and licenses passing claims while abstaining on failures.Claim extraction may use a named-entity-recognition model such as GLiNER (Zaratiana et al., 2025).Source: §3.2 · p. 6

3.3 Comparison to Other Hallucination Mitigation Methods

  • The Oracle is presented as a deterministic alternative to fine-tuning and RAG, validating generated claims against formal knowledge representations to prevent false assertions through abstention.The paper says fine-tuning can improve recall but may not produce reliable abstention, while RAG remains vulnerable to hallucinations when retrieved context is inaccurate or incomplete.Source: §3.3 · p. 6

4. Experimental Setup

The experiments compare the Licensing Oracle with statistical and retrieval-based approaches for factual question answering, using knowledge-graph validation as a formal constraint. The evaluation uses U.S. river data and metrics for factual accuracy, abstention, and constraint rejection.

4. Experimental Setup

  • The evaluation assumes perfect claim extraction, although extraction errors could cause false abstentions or false validations.The authors identify independent measurement of extraction accuracy as future work.Source: §4 · p. 6

4.3 Experimental Conditions

  • Five conditions compare baseline generation, two fine-tuning regimes, RAG, and graph-based RAG with the Licensing Oracle.The graph-based approach uses the graph as a licensing gate for generated claims, rather than only as a retrieval source.Source: §4 · p. 7, §4.3 · pp. 7–8

4.1 Dataset Construction

  • 17,726 U.S.-river question-answer pairs test claims about entities and relationships represented in a knowledge graph.The source graph covers 9,538 river entities and attributes including hydrological measurements and geographic relationships; about 15% of graph attributes were LLM-augmented.Source: §4.1 · p. 7, §4.2 · p. 7

4.4 Evaluation Metrics

  • Accuracy measures correct answers among attempted questions, while abstention precision measures the share of abstentions that were appropriate.Appropriate abstentions concern claims unsupported by evidence or violating logical constraints.Source: §4.4.1 · p. 8, §4.4.2 · p. 8
  • CVRR measures correctly rejected claims violating SHACL constraints; FAR-NE measures answers given to claims that should have been withheld.These metrics assess constraint rejection and hallucinated responses despite abstention capabilities, respectively.Source: §4.4.3 · p. 8, §4.4.4 · p. 9
  • Licensed Answer Accuracy measures the share of graph-entailed questions that were licensed and answered correctly.It assesses the reliability of the Oracle's validation process.Source: §4.4.5 · p. 9

4.5 Implementation Details

  • Fine-tuning used LoRA on Gemma 3-4B-Instruct, RAG retrieved five passages, and graph-based RAG checked entailment and SHACL constraints.The graph contained 118,047 triples; pySHACL handled SHACL validation and RDFLib managed graph operations.Source: §4.5 · p. 9

4.6 Statistical Methodology

  • Each condition was evaluated once, so run-to-run variance was not measured and statistical significance testing was not performed.The authors say the 0.4pp RAG–Graph-RAG difference is within expected measurement variance.Source: §4.6 · pp. 9–10

5. Results

Graph-RAG with the Licensing Oracle achieved 89.1% accuracy while adding deterministic validation and perfect abstention precision. Across rivers and philosophers, it maintained similar accuracy and identical epistemic guarantees.

5.1 Baseline Model Performance

  • Baseline accuracy was 42% for Claude Sonnet 4.5, 50.1% for Gemini 2.5 Flash Lite, and 16.7% for Gemma 3-4B-Instruct.These models were evaluated on U.S. rivers questions without grounding mechanisms.Source: Table 1 · p. 10, §5.1 · p. 10

5.2 Fine-Tuning Results

  • Factual fine-tuning reduced accuracy to 8.5%, while abstention fine-tuning achieved 56.7% abstention precision and 63.7% abstention recall.The paper reports that abstention fine-tuning still produced incorrect answers in cases where abstention was more appropriate.Source: §5.2 · pp. 10–11, Table 2 · p. 10

5.3 RAG System Performance

  • 89.5% accuracy, a 39.4 percentage-point improvement over baseline: embedding-based RAG improved factual recall but lacked a mechanism for abstention.The paper says the 0.4-point difference from Graph-RAG is not statistically significant.Source: §5.3 · p. 12, Table 3 · p. 12

5.4 Graph-Based RAG with Licensing Oracle

  • 89.1% accuracy: Graph-RAG nearly matched embedding-based RAG while validating claims before emission to prevent false or unverified assertions.The paper reports the 0.4 percentage-point difference as within expected measurement variance.Source: Table 4 · p. 12, §5.4 · p. 12

5.5 Licensing Oracle Performance

  • AP = 1.0, CVRR = 50%, and LA = 1.0; the oracle correctly abstained, rejected constraint violations, and generated supported claims.The paper also reports zero false answers on non-entailed claims.Source: §5.5 · p. 13, Table 5 · p. 13

5.6 Summary of Findings

  • The authors characterize the oracle as providing necessary guarantees and high accuracy in structured factual domains, while complementing rather than replacing RAG.Their stated scope is knowledge that can be formally represented and verified.Source: §5.6 · p. 14

5.7 Cross-Domain Validation

  • Using the same SHACL validation methodology, the oracle achieved 89.1% accuracy on rivers and 89.0% on 595 philosopher influence questions, with AP = 1.0 and FAR-NE = 0.0 in both domains.The philosopher dataset used a knowledge graph built from structured biographical and influence data.Source: §5.7.1 · p. 15, §5.7.2 · p. 16

5.7.3 Implications for Cross-Domain Generalization

  • The authors infer scalability to domains whose knowledge can be formalized in a graph, without retraining or domain-specific tuning.They describe this as a solution for structured-knowledge domains.Source: §5.7.3 · p. 16

6. Discussion

The discussion argues that formal architectural validation offers deterministic factual reliability for structured knowledge, unlike statistical approaches, while emphasizing the Oracle’s scope and coverage trade-offs.

6.1 Architectural Enforcement vs. Statistical Learning

  • The Oracle’s performance across rivers and philosophers domains supports generalization where structured facts can be represented in a knowledge graph.The discussion identifies medicine, law, science, and business intelligence as potential high-value applications.Source: §6.1 · pp. 16–17
  • Fine-tuned models reached only 56.7% abstention precision, while the Oracle uses formal graph and SHACL validation to enforce factual constraints deterministically.Statistical methods improve factual recall but do not enforce the structural constraints needed for factual correctness.Source: §6.1 · p. 16, §6.2 · p. 17

6.2 The Necessity and Sufficiency of Licensing Oracles

  • The Licensing Oracle achieves perfect abstention precision (AP = 1.0) and 89.1% accuracy in factually correct claims supported by the knowledge graph.The discussion describes these results as demonstrating necessity and sufficiency for factual reliability in structured knowledge domains.Source: §6.2 · p. 17

6.3 Limitations and Potential Improvements

  • The Oracle can validate only facts explicitly represented in its graph, and imprecise predicates or missing relationships can undermine validation of subtle distinctions.Incomplete or dynamic knowledge can also leave the Oracle unable to answer, increasing abstention.Source: §6.3 · p. 17
  • The current Oracle lacks mechanisms for disambiguating underspecified questions and handling time-dependent facts, and it validates individual triples rather than multi-hop reasoning.Complex inferences across graph facts may therefore not be validated correctly.Source: §6.3 · p. 17
  • The Oracle prioritizes precision over coverage, so incomplete knowledge graphs can lead to more abstentions.The discussion identifies combining the Oracle with statistical retrieval or expanding the graph as possible ways to address this trade-off.Source: §6.3 · p. 17

6.4 Broader Implications for AI Reliability

  • The discussion presents architectural truth-validation as especially relevant to high-value applications where factual accuracy is critical.Examples include medicine, law, science, and business intelligence.Source: §6.4 · pp. 17–18
  • The Oracle’s epistemic governance acts as a gatekeeper, validating claims against predefined logical rules before they are emitted.The discussion compares this role to type systems that validate code before execution.Source: §6.4 · p. 18

7. Conclusion

The conclusion presents the Licensing Oracle as a formal-validation approach to hallucinations about structured knowledge. It reports perfect abstention precision and zero false answers, while noting the method’s bounded applicability and future development needs.

7. Conclusion

  • The Licensing Oracle uses structured-knowledge-graph and SHACL validation to achieve AP = 1.0 and FAR-NE = 0.0, with false abstentions remaining a precision-coverage trade-off.FAR-NE = 0.0 means no false claims were generated for questions the system answered.Source: §7 · p. 18
  • The authors describe the Oracle as applicable to domains with verifiable knowledge representations, but not to creative content, open-ended conversations, or subjective judgment.Source: §7 · p. 18
  • The conclusion identifies expanding Oracle capabilities, improving multi-hop reasoning, and exploring hybrid statistical-generation and architectural-validation systems as future work.Source: §7 · p. 18

Resources

The released resources include datasets, fine-tuned models, knowledge graphs, and evaluation outputs under CC BY 4.0.

Resources

  • The repository provides raw river datasets, evaluation Q&A pairs, and Gemma factual-recall and abstention models with LoRA adapters.The river data include 9,538 rivers with 21 attributes, and the evaluation datasets include 17,726 Q&A pairs.Source: Resources · p. 21
  • The released materials also include a knowledge graph with 118,047 triples, SHACL constraints, the full knowledge base, and complete experiment outputs.The resources are released under CC BY 4.0 to facilitate open research and reproducibility.Source: Resources · p. 21

Oracle Implementation

The implementation retrieves graph context, generates a response, extracts factual claims, validates them, and abstains if validation fails.

Oracle Implementation

  • The Oracle’s five stages are graph retrieval, LLM generation, claim extraction, validation, and an abstention decision.The implementation describes extracting factual claims as triples for validation.Source: System Overview · p. 21
  • The implementation returns “I don't know” when a claim is absent from the graph or violates constraints, and emits the response when all claims are licensed.Source: Core Algorithm · p. 21

1. Knowledge Graph

The Licensing Oracle checks claims against structured river facts using SHACL constraints, then answers only when a claim is licensed. The section describes this deterministic validation approach and reports zero hallucinations.

1. Knowledge Graph

  • The knowledge graph stores river facts as RDF triples, including length, source elevation, and traversed state.Source: §1 · p. 22

2. SHACL Constraints

Validation: Checks that :sourceElevation > :mouthElevation

3. Claim Extractor

  • GLiNER extracts entities and forms triples from text, as in converting a Colorado River length claim into a length triple.Source: §3 · p. 23

4. Consistency Auditor (The Oracle)

  • The consistency auditor licenses a claim when its subject-predicate-object triple is present in the graph.Source: §4 · p. 23

5. Abstention Policy

Loading 2511.06073v1…