Source-linked AI summary
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Nouha Dziri, Andrea Madotto, Osmar Zaiane, Avishek Joey Bose
TL;DR
Large language-model dialogue systems can produce fluent but factually incorrect responses, motivating improved faithfulness to Knowledge Graph facts. The paper proposes Neural Path Hunter, which detects and refines hallucinated entities through KG paths without retraining the dialogue model. It reports a 20.35% relative FeQA improvement over vanilla GPT2 and finds that local subgraphs serve as useful external memory.
Problem
KG-grounded dialogue systems can generate factually invalid statements, with erroneous entity injection identified as a major hallucination source.
Method
Neural Path Hunter uses a generate-then-refine pipeline combining a token-level hallucination critic with entity retrieval over a local k-hop KG subgraph.
Results
20.35% relative improvement over vanilla GPT2 is reported on FeQA, alongside a 39.98% improvement in human evaluation.
Takeaways & Limitations
Neural Path Hunter reduces hallucination across multiple base dialogue models, while local subgraph representations provide useful external memory compared with the full graph.
Takeaways & Limitations
The method assumes that the local k-hop subgraph is provided or can be extracted from the dialogue history.
Abstract
from arXiv · showhide
Dialogue systems powered by large pre-trained language models (LM) exhibit an innate ability to deliver fluent and natural-looking responses. Despite their impressive generation performance, these models can often generate factually incorrect statements impeding their widespread adoption. In this paper, we focus on the task of improving the faithfulness -- and thus reduce hallucination -- of Neural Dialogue Systems to known facts supplied by a Knowledge Graph (KG). We propose Neural Path Hunter which follows a generate-then-refine strategy whereby a generated response is amended using the k-hop subgraph of a KG. Neural Path Hunter leverages a separate token-level fact critic to identify plausible sources of hallucination followed by a refinement stage consisting of a chain of two neural LM's that retrieves correct entities by crafting a query signal that is propagated over the k-hop subgraph. Our proposed model can easily be applied to any dialogue generated responses without retraining the model. We empirically validate our proposed approach on the OpenDialKG dataset against a suite of metrics and report a relative improvement of faithfulness over dialogue responses by 20.35% based on FeQA (Durmus et al., 2020).
1 Introduction
Neural dialogue systems must produce responses that are not only fluent and topical but also factually correct, yet KG-grounded systems can hallucinate invalid facts and entities. Neural Path Hunter addresses this problem by refining generated responses with KG-supported paths, achieving reported gains in faithfulness.
- KG-grounded dialogue systems can hallucinate factually invalid information despite plausible linguistic capabilities.
- Human analysis identifies erroneous entity injection as a major source of hallucination in generated responses.
- NEURAL PATH HUNTER uses a generate-then-refine strategy that queries a KG to correct potential hallucinations in existing responses.
- A token-level hallucination critic masks concerning entities before neural language models retrieve replacements supported by KG paths.
- 20.35% relative improvement in FeQA and 39.98% improvement in human evaluation are reported for KG-grounded dialogue systems.
2 Hallucination in KG-grounded Dialogue Systems
The paper defines hallucination in KG-grounded dialogue as producing factual content unsupported by a local k-hop KG subgraph, distinguishing extrinsic additions from intrinsic relational errors. Human evaluation finds extrinsic hallucinations dominant and links more diverse decoding with greater hallucination.
- A KG consists of directed subject-predicate-object triples, and hallucination is defined as generating a factual sentence unsupported by a valid path in a local k-hop subgraph.
- Extrinsic hallucinations introduce text that does not correspond to a valid triple in the local subgraph.
- Intrinsic hallucinations misuse subject or object entities while retaining entities from the KG.
- 2.1 Modes of Hallucination: Human assessment of 1,500 GPT2 responses evaluates hallucination presence and distinguishes extrinsic, intrinsic, and combined cases.
- 2.1 Modes of Hallucination: Top-k sampling produces the highest hallucination percentage at 40.33%, while greater response diversity is positively correlated with hallucination.
- 2.1 Modes of Hallucination: Extrinsic hallucination is dominant, with 54.80% of OpenDialKG responses containing extra entity mentions unsupported by the dialogue or 1-hop subgraph.
- 2.1 Modes of Hallucination: The paper therefore focuses on identifying hallucinations and retrieving correct KG entities rather than advancing decoding methods.
3 Neural Path Hunter
Neural Path Hunter is a generate-then-refine strategy that identifies hallucinated entity mentions and replaces them with entities retrieved through queries over a local KG subgraph. It combines a token-level hallucination critic with an Entity Mention Retriever that uses contextual representations, KG-entity memory, and contrastive training to favor faithful paths.
- Overview: Neural Path Hunter refines generated responses by flagging hallucinated entity mentions and retrieving more faithful entities from a local k-hop KG subgraph.The module is designed to operate on existing generated responses, using a critic followed by an entity retriever.
- Token-level hallucination critic: The hallucination critic consumes dialogue history, KG triples, and the generated response to identify entity mentions requiring refinement.It is trained with extrinsic entity replacements and intrinsic subject-object swaps, using a 60%/40% corruption split.
- Entity Mention Retriever: The retriever masks flagged mentions, pools their contextual hidden states, and transforms them into autoregressive query representations.The query representation uses a learnable projection over the current hidden state and the previously retrieved entity embedding.
- KG-Entity Memory: KG-Entity Memory represents local-subgraph nodes and relations in a shared embedding space using GPT2 embeddings or CompGCN message passing.The retriever scores candidate KG-entity memory triples and selects the subject or object with the highest score.
- Retriever training: Noise Contrastive Estimation trains the retriever to score the correct edge triple above corrupted triples sampled from entities outside the local subgraph.Training jointly augments the conventional maximum-likelihood objective with a contrastive loss, using uniformly drawn negative entities.
4 Experiments
Experiments evaluate NPH’s hallucination detection, refinement, entity retrieval, and component choices on OpenDialKG using classification, language-model, retrieval, faithfulness, and human-evaluation metrics. NPH consistently reduces hallucination, while ablations identify the contributions of LNCE, MLM, negative sampling, and GPT2-based entity representations.
- Experimental design: Experiments assess hallucination identification, reduction, query generation, MLM and critic effects, and global graph-structure choices.The evaluation uses F1, precision, recall, perplexity, Hits@k, Mean Rank, Mean Reciprocal Rank, BLEU, FeQA, critic scores, and human judgments.
- Q1: Identifying Hallucinations: 70.35% F1 is achieved by RoBERTa-Intrin-Extrin, which detects both intrinsic and extrinsic hallucinations and is used as critic C.The classifier is trained on examples corrupted with either hallucination strategy.
- Q2: Reducing Hallucinations: NPH consistently reduces hallucination across FeQA and the hallucination Critic; for AdapterBot, it lowers the Critic score by 8.17 points and raises FeQA faithfulness by 6.67 points.BLEU changes inconsistently across baselines, and the authors note that n-gram overlap is an imperfect faithfulness proxy.
- Q3: Query Generation: Omitting LNCE causes retrieval metrics to drop dramatically, including a 70-point decrease in Hits@1, while SANS negatives improve perplexity and retrieval.Complete NPH with GPT2 embeddings nearly saturates Hits@3 and Hits@10.
- Q4: Impact of MLM and Critic: Removing MLM worsens NPH across models and reduces Hits@1 by 26 points with GPT2 entity memory, indicating the value of masked representations.The degradation also appears with CompGCN embeddings.
- Q5: Impact of global graph structure: GPT2-based entity embeddings outperform CompGCN embeddings in perplexity and retrieval, suggesting alignment with the autoregressive LM is more beneficial than enriching embeddings with global graph structure.Both embedding approaches are updated during training.
5 Related Work
Prior work established knowledge graphs as resources for question answering, language processing, and dialogue generation, while highlighting hallucination and reasoning challenges in grounded dialogue. Existing approaches include commonsense injection, dynamic graphs, and KG path traversal, but this paper targets hallucination removal in open-domain dialogue.
- Knowledge graphs are large multi-relational repositories applied to question answering and natural language processing.
- Dialogue research has used commonsense knowledge graphs and dynamic graphs to inject or incorporate knowledge into conversational responses.
- Conversational reasoning models traverse knowledge graphs to retrieve relevant paths and predict the next response entity.
- State-of-the-art natural language generation models can hallucinate by omitting important entities, while existing mitigation methods are often task-specific.
- Open-domain knowledge-grounded dialogue hallucination has received relatively little research attention.
6 Conclusion
The paper identifies erroneous entity injection as the predominant form of extrinsic hallucination in KG-grounded dialogue and proposes Neural Path Hunter to refine responses through k-hop KG queries. It reports improved faithfulness while noting that local graph structure is important and dialogue-to-KG alignment may limit broader application.
- KG-grounded dialogue models are especially susceptible to extrinsic hallucinations caused by injecting erroneous entities.
- Neural Path Hunter identifies and refines hallucinations by querying a k-hop subgraph of the knowledge graph.
- 20.35% relative improvement over vanilla GPT2 on FeQA is reported when Neural Path Hunter is paired with dialogue models.
- The local subgraph serves as external memory and plays a crucial role compared with the full global graph.
- Dialogue-to-KG alignment may be difficult to obtain in other applications, potentially requiring use of the full graph as future work.
B Human Evaluation for Modes of Hallucination
The human evaluation used trained English-speaking workers to judge hallucination, hallucination type, faithfulness, coherence, and grammaticality in dialogue responses. Each example was assessed by three workers, with majority vote used for the result.
- Evaluation setup: The evaluation presented workers with dialogue history and knowledge triples, including gold triples and 1-hop paths from the center node.
- Human evaluation procedure: Each evaluation example was judged by three workers, and the majority vote determined the assessment.
- Evaluation questions: Workers evaluated whether responses were hallucinated relative to the gold knowledge triples.
- Evaluation questions: For hallucinated responses, workers classified the error as extrinsic, intrinsic, or both.
- Evaluation questions: For non-hallucinated responses, workers distinguished faithful responses from generic ones.
- Evaluation questions: Workers also rated coherence with the dialogue history and grammatical correctness.
D Implementation Details
The implementation used transformer-based language models and standard optimization settings for Neural Path Hunter, baselines, and the hallucination critic. Experiments included human-assessment results and were run on Nvidia V100 GPUs with greedy-search inference.
- NPH: Neural Path Hunter used a small RoBERTa model as its masked language model and base GPT2 as its autoregressive language model.
- NPH: NPH training used Adam, dropout, batch size 16, and a linearly decayed learning rate of 6.25 × 10^-5.
- Human assessment: Table 7 reports human assessment of 1,500 GPT2 dialogue responses generated from test OpenDialKG data.
- Baselines: AdapterBot and GPT2-KE followed publicly available implementations and reported average runtimes of 3 hours.
- Hardware and inference: All models were trained on an Nvidia V100 GPU with 32GB memory, and inference used greedy search.
- Hallucination critic: The hallucination critic used a pre-trained RoBERTa-large classifier trained for five epochs on one Nvidia V100 GPU.
E Hallucination Metrics
The evaluation uses hallucination detection and faithfulness metrics alongside BLEU, with FeQA adapted to assess semantic correctness against KG-supported knowledge.
- BLEU can be misleading when a response differs from the reference but remains faithful to the knowledge triples.
- A token-level hallucination critic is used as a sentence-level detector, labeling an utterance hallucinated when at least one token is identified as hallucinated.
- FeQA measures semantic correctness by generating questions from responses and answering them against the knowledge used for grounding.
- The adapted FeQA document concatenates the dialogue context with the relevant KG triples, while the candidate is the generated or refined response.
F Human Evaluation of NPH responses
Human evaluation compares baseline and refined responses using the dialogue context, KG knowledge, and retrieved paths, judging hallucination and fluency.
- Annotators evaluated baseline responses with the dialogue and KG knowledge, while NPH responses also included the retrieved path and refined response.
- Human judgments covered two questions: whether the response hallucinated relative to the KG and whether it was grammatically fluent.
- 1200 responses were evaluated, split evenly between 600 baseline responses and 600 responses after refinement.
G Error Analysis
The error analysis identifies retrieval failures caused by missing entities in the available subgraph, critic misses, and occasional fluency degradation after refinement.
- NPH requires oracle access to the k-hop subgraph for effective retrieval, so failed corrections can arise when the correct entity is unavailable there.
- When the correct entity is absent from the extracted subgraph, NPH may retrieve a wrong entity despite relevant context nodes being present.
- The authors hypothesize that reasoning over multiple subgraphs or the full KG could alleviate failures caused by insufficient subgraph coverage.
- Another error source is the critic’s imperfect identification of hallucinated entities, which can propagate incorrect content into refinement.
- Refinement can occasionally reduce grammaticality while removing hallucinations, and entity-type associations are proposed as a mitigation.
H Sample Refined Responses
The paper presents selected GPT2-KG responses before and after NPH refinement, marking hallucinated entities in red and retrieved entities in green.
- Table 8 compares selected GPT2-KG test responses before and after applying NEURAL PATH HUNTER.
- Red highlights indicate hallucinated entity mentions, whereas green highlights indicate entity mentions retrieved during refinement.