Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective

1. Introduction

• Hallucinations include errors about world knowledge, self-contradictions, and failures to follow prompts or use given context.The introduction notes that hallucinations can be useful for brainstorming or artwork, but constrain fact-priority uses such as QA, retrieval, summarization, and recommendations.Source: §1 · p. 1
• KGs provide machine-readable factual representations that can supply a factual basis during inference or post-generation without full retraining.This is particularly relevant where knowledge changes quickly and time or computational resources are limited.Source: §1 · pp. 1–2
• The paper categorizes KG-based knowledge integration methods by underlying architecture and by the stage where information enters an LLM.Figure 2 presents stages at which additional information can be included to boost factuality.Source: §1 · p. 2
• Whole-hypothesis semantic similarity and entailment methods can miss subtle hallucinations because they do not capture granular word-level details.Even a single incorrect word can result in a large semantic mismatch.Source: §1 · p. 2
• Robust evaluation is needed to develop reliable hallucination mitigation methods, but the paper says such evaluation is not yet present.It points to sentence-level FELM and span-level MuShroom-2025 as finer-grained examples.Source: §1 · p. 2
• The paper identifies fine-grained hallucination detection, knowledge integration beyond textual prompting, and multiprompt, multilingual, multitask factuality evaluation as open directions.The introduction emphasizes exact hallucinated spans and an in-depth analysis of model performance.Source: §1 · p. 2

2. Available Resources for Evaluating Hallucinations

• Many datasets cover multiple domains, including law, politics, medicine, science and technology, art, and finance.The paper argues that holistic evaluation should cover different domains and tasks to test for different hallucination types and generalizability.Source: §2 · p. 2
• Most resources evaluate whether a response, sentence, or span contains a hallucination, and their data can be repurposed for evaluating LLMs.The paper notes that these datasets do not explicitly model hallucination evaluation for a given LLM.Source: §2 · p. 2
• Most listed datasets are benchmarks without dedicated training splits for training parametric knowledge-integration models.The paper also notes that all listed datasets except SemEval2025-MuShroom are English-only, and that knowledge subgraphs are not a popular context feature.Source: §2 · p. 2
• Only Rahman et al. (2024) evaluates prompt robustness in the surveyed resources, pairing each question-answer datapoint with 15 paraphrases.The paper cites prior work identifying the need for multiprompt evaluation because output can depend on input phrasing.Source: §2 · p. 2
• The paper concludes that high-quality evaluation and training resources have gaps that must be addressed before they can support hallucination evaluation and mitigation, especially with KGs.Table 1 is presented as an overview of available resources for hallucination detection and evaluation.Source: §2 · p. 2, Table 1 · p. 3

3. Feasibility of Hallucination Mitigation

• External knowledge may mitigate hallucinations, but its scalability remains unclear.The paper frames reliability improvement around interpretable outputs and conditioning on a reliable external knowledge source.Source: §3 · p. 3
• KG-based scrutiny assumes that triples are factually correct for the user query and that the LLM uses them effectively.Under those conditions, output information can be mapped back to the originating KG for checking.Source: §3 · p. 3

4. Detection of Hallucinations

• GraphEval extracts atomic claims as a subgraph and compares triple entailment with textual context, while other systems extract and verify claims against KGs or web sources.Rashad et al. compare KG subgraph alignment using named entities; KGR uses claim extraction, fact selection, and verification; Fleek generates questions for claim verification.Source: §4 · pp. 3–4
• KG-structured atomic claims can make inconsistent triples interpretable, helping identify problematic text spans for scrutiny.Manual evaluation can also help clarify problematic use cases.Source: §4 · p. 4
• None of the described methods demonstrates that hallucination detection is solved, and limited evaluation datasets do not establish generalizability.LLM-prompted multi-stage extraction and validation also raise scalability and robustness concerns; the paper calls for diverse datasets, per-submodule error reporting, and approaches less reliant on LLMs.Source: §4 · p. 4
• How KG-based detection scales when combined with approaches such as LLM uncertainty detection remains unclear.The paper proposes investigating the synergy between uncertainty and KG-based detection.Source: §4 · p. 4

5. Methods for Integrating Knowledge from KGs in LLMs

• KGs can be integrated during pretraining, inference, or post-generation, but the paper says it remains unclear how different mitigation strategies complement one another when combined.The authors hypothesize reliable systems may mix strategies and call for research on stacking them.Source: §5 · pp. 4–5
• Pretraining methods incorporate KG triples through masked entity prediction, fusion modules, or adapter-based techniques that encode KG knowledge as low-parameter add-ons.The surveyed methods suggest factuality or task-specific improvements, but pretraining encodes knowledge statically and does not address rapid updates.Source: §5 · pp. 4–5
• Inference-time prompting appends documents or KG triples to queries, but output depends on prompt phrasing, evidence quality, context limits, and model use of the prompt.The paper calls for more robust evaluation, including multi-prompt evaluations.Source: §5 · p. 4
• Context-aware decoding prioritizes in-prompt knowledge with a learnable parameter, but requires two inference passes and doubles computational cost.Prompt-based injection also may conflict with a model’s internal knowledge; DefAn is identified as the only listed dataset with multi-prompt evaluation, using ten question rephrasings.Source: §5 · p. 4
• KG-grounded S-expression generation improves factuality but leaves open generalizability when KG data is incomplete or missing.Latent-space adapter injection is described as enabling rich triple metadata encoding and rapid knowledge updates beyond prompting at scale.Source: §5 · p. 4
• Post-generation retrofitting checks extracted claims against an external KG and patches the answer, with each of the five pipeline stages relying on an LLM.The paper raises broader concerns about errors accumulating as prompted submodules increase and calls for per-module error reporting and methods less dependent on prompting.Source: §5 · p. 5
• The paper identifies multilingual knowledge integration as necessary, noting that multilingual KGs may support low-resource languages and prior adapter work encoded multilingual KG knowledge.The cited adapter approach suggests benefits for entity alignment, KG completion, named entity recognition, and question answering.Source: §5 · p. 5

6. On Hallucination Evaluation

• Multiple-choice QA evaluation can yield correct answers by chance, limiting insight into models’ internal knowledge representations.The paper presents generative-task semantic evaluation as a more robust framing, while noting that neural metrics and auxiliary LLM judges have their own error risks.Source: §6 · p. 5
• The paper advocates human evaluation on at least a subset of results and evaluation beyond English because auxiliary LLM judges can hallucinate, especially in low-resource languages.It also calls for robust claim extraction and evaluation against KGs as sources of ground truth.Source: §6 · p. 5
• FactScore breaks LLM output into atomic facts and compares them with external knowledge; the paper proposes extending this approach to KG triples for fine-grained, interpretable factuality evaluation.The authors call for robust claim extraction to enable KG-grounded evaluation.Source: §6 · p. 5

• KGs are a potential mitigation route, but current methods have not yielded a general solution.Source: §7 · p. 5
• Progress also depends on effective multilingual graph creation and completion, entity extraction, graph-embedding extraction, entity linking, and KG–LLM synergy.Source: §7 · p. 6
• Large-scale datasets with accurate KG triples and train, development, and test splits could support parametric knowledge integration and parametric entity extraction.Source: §7 · p. 6
• Robust evaluation should span languages, tasks, and prompts, unlike studies often limited to one language, task, and prompt.Source: §7 · p. 6
• Hallucination detection should identify specific hallucinatory text spans, since the paper describes detection as the first step toward mitigation.Source: §7 · p. 6
• Knowledge integration should move beyond fragile textual prompts, ideally parameter-efficiently; studies should also mix mitigation methods and extend detection, evaluation, and integration across languages.Source: §7 · p. 6