Source-linked AI summary
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models
S. M Towhidul Islam Tonmoy, S M Mehedi Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, Amitava Das
TL;DR
Hallucination produces fluent but ungrounded or erroneous content, creating a central obstacle to deploying LLMs in sensitive and critical applications. This survey organizes over 32 mitigation techniques into a taxonomy, synthesizes their features, and discusses their limitations and future directions. It positions the resulting classification as a foundation for more structured research on hallucination mitigation.
Problem
LLMs can generate factually incorrect or ungrounded information, raising reliability concerns for applications such as medical-record summarization and financial analysis.
Method
The paper surveys over 32 hallucination-mitigation techniques and organizes them by dimensions including dataset utilization, tasks, feedback mechanisms, and retriever types.
Results
The survey consolidates diverse mitigation techniques into a systematic taxonomy spanning detection, retrieval, feedback, generation refinement, and supervised fine-tuning.
Takeaways & Limitations
The classification synthesizes the field’s essential features and provides a foundation for more structured future research on hallucination and related phenomena.
Takeaways & Limitations
The surveyed mitigation landscape remains constrained by challenges inherent in existing techniques, motivating hybrid methods, reduced reliance on labeled data, and further study of societal effects.
Abstract
from arXiv · showhide
As Large Language Models (LLMs) continue to advance in their ability to write human-like text, a key challenge remains around their tendency to hallucinate generating content that appears factual but is ungrounded. This issue of hallucination is arguably the biggest hindrance to safely deploying these powerful LLMs into real-world production systems that impact people's lives. The journey toward widespread adoption of LLMs in practical settings heavily relies on addressing and mitigating hallucinations. Unlike traditional AI systems focused on limited tasks, LLMs have been exposed to vast amounts of online text data during training. While this allows them to display impressive language fluency, it also means they are capable of extrapolating information from the biases in training data, misinterpreting ambiguous prompts, or modifying the information to align superficially with the input. This becomes hugely alarming when we rely on language generation capabilities for sensitive applications, such as summarizing medical records, financial analysis reports, etc. This paper presents a comprehensive survey of over 32 techniques developed to mitigate hallucination in LLMs. Notable among these are Retrieval Augmented Generation (Lewis et al, 2021), Knowledge Retrieval (Varshney et al,2023), CoNLI (Lei et al, 2023), and CoVe (Dhuliawala et al, 2023). Furthermore, we introduce a detailed taxonomy categorizing these methods based on various parameters, such as dataset utilization, common tasks, feedback mechanisms, and retriever types. This classification helps distinguish the diverse approaches specifically designed to tackle hallucination issues in LLMs. Additionally, we analyze the challenges and limitations inherent in these techniques, providing a solid foundation for future research in addressing hallucinations and related phenomena within the realm of LLMs.
1 Introduction
The paper frames hallucination as a critical reliability problem for broadly deployed LLMs and organizes mitigation research into a systematic taxonomy. Its survey covers model-development and prompt-engineering approaches, while also examining limitations and future directions.
- Motivation: LLM hallucinations are factually erroneous or fabricated details that threaten the reliability of systems used across scholarly and professional domains.The paper highlights applications including academic research, programming, technical advisement, and skill acquisition.
- Survey scope: The survey consolidates diverse hallucination-mitigation strategies, including feedback mechanisms, external information retrieval, and early generation refinement.It presents this consolidation as a comprehensive taxonomy for the field.
- Taxonomy: The taxonomy categorizes mitigation techniques across model development and prompt engineering.Model development includes decoding strategies, knowledge-graph optimization, novel loss components, and supervised fine-tuning; prompt engineering includes retrieval augmentation, feedback, and prompt tuning.
- Contributions: The paper introduces a systematic taxonomy for hallucination mitigation techniques in LLMs, including Vision Language Models.This classification is one of the paper’s stated contributions.
- Contributions: The survey synthesizes essential features of mitigation techniques to support more structured future research.It also discusses inherent limitations and challenges alongside potential solutions and future research directions.
2 Hallucination Mitigation
Hallucination mitigation spans detection, retrieval, feedback, and model-generation interventions. The surveyed methods differ in when they retrieve information or revise outputs, and they include both plug-and-play and trained approaches.
- Detection: Hallucination detection methods include multilingual summary evaluation, contextual-information frameworks, and self-contradiction analysis.The survey presents these as approaches to identifying or understanding hallucination.
- Prompt engineering: Prompt engineering mitigates hallucination by specifying context and expected outcomes through designed instructions.The survey treats prompt engineering as experimentation with instructions to obtain better model outputs.
- Retrieval augmentation: Retrieval-Augmented Generation conditions responses on external authoritative knowledge to improve accuracy, currency, and verifiability.RAG addresses reliance on potentially outdated training data or internal model knowledge.
- Before generation: LLM-Augmenter uses plug-and-play modules to retrieve external evidence, perform reasoning, and iteratively revise prompts with utility-function feedback.The modules do not require training and are intended for immediate use with black-box LLMs.
- During generation: Sentence-level knowledge retrieval detects possible hallucinations from model logits, validates them against retrieved knowledge, and repairs affected phrases.The approach can eliminate or substitute hallucinated information, although some API-only models may not expose logits.
- During generation: EVER performs real-time, stepwise generation, validation, and rectification to address intrinsic and extrinsic hallucinations during decoding.It is contrasted with methods that correct hallucinations only after generation.
- After generation: Post-generation methods retrieve information after the complete output, while high-entropy word replacement uses open-source models to detect and replace vulnerable words.Treating consecutive high-entropy words as one masked unit is particularly effective for Generated Golem and Acronym Ambiguity cases.
- Feedback and reasoning: Feedback-based methods iteratively critique, score, and refine answers to improve factuality, consistency, and entailment.The self-reflection methodology reports effectiveness in reducing hallucinations compared with baselines through automatic and human evaluation.
2. Plans verification questions to fact-check its
This section presents verification- and retrieval-oriented approaches for reducing hallucinations, including self-checking, contextual detection, prompt tuning, and external knowledge grounding.
- CoVe generates verification questions and answers them to check whether a baseline response contains factual hallucinations.Experiments report reduced hallucinations on list-based Wikidata questions and long-form text generation.
- CoNLI detects ungrounded content through a hierarchical framework and uses post-editing to reduce hallucinations without fine-tuning or domain-specific prompts.The framework combines detection with post-editing while preserving text quality.
- Prompt Tuning: Prompt tuning adjusts instructions during fine-tuning, with soft prompts learned through backpropagation rather than predetermined manually.UPRISE extends this approach by retrieving prompts for zero-shot inputs across tasks and models.
3 Developing Models
The surveyed model-development approaches address hallucinations through decoding, knowledge grounding, faithfulness-based objectives, and supervised fine-tuning or teacher-student strategies.
- Developing Models: Novel model architectures are developed to mitigate hallucinations without relying solely on fine-tuning, reflecting an ongoing combination of algorithmic and data-quality improvements.The paper categorizes these architecture-level techniques into several approaches.
- Decoding Strategy: Decoding strategies guide generation toward authentic or context-specific outputs, including contrastive methods that can override conflicting prior knowledge.CAD compares output distributions with and without context and is usable with off-the-shelf models without additional training.
- Knowledge Graphs: Knowledge graphs support faithful dialogue generation by representing entities and relations and enabling local and global knowledge grounding.RHO also uses conversational reasoning to rerank generated responses.
- Faithfulness-Based Loss Functions: Faithfulness-based objectives measure or constrain how closely outputs reflect input data or ground truth without errors, omissions, or distortions.THAM introduces information-theoretic regularization through a Text Hallucination Regularization loss for video-grounded dialogue.
- Supervised Fine-Tuning: Supervised fine-tuning improves downstream task alignment, while knowledge injection and teacher-student approaches target factuality in smaller or weaker models.Other approaches use automated fact-checking, preference-based learning, and refusal-aware data for factuality and uncertainty handling.
- Supervised Fine-Tuning: Counterfactual and hypothesis-based methods modify training or decoding to improve factuality by distinguishing uncertain examples or ranking candidates against input facts.The surveyed methods include refusal-aware training data, FATE, and TWEAK’s Hypothesis Verification Model.
4 Conclusion
The survey organizes hallucination mitigation research into a systematic taxonomy, synthesizes the field’s techniques, and discusses limitations and future research directions.
- The paper presents a systematic taxonomy of hallucination mitigation techniques for LLMs and extends its coverage to Vision Language Models.This organization is intended to support structured future research.
- The survey synthesizes essential features of mitigation techniques and consolidates diverse strategies into a resource for researchers and practitioners.It addresses hallucination detection, mitigation methods, and the current research landscape.
- The paper examines limitations and challenges in existing techniques and proposes directions for future research.Its conclusion frames these issues as part of advancing knowledge in computational linguistics.
5 Discussion and Limitations
The discussion compares a broad range of mitigation strategies and emphasizes that future progress may require integrated methods, scalable learning, responsible implementation, and continued benchmarking.
- Post-Generation Refinement: Post-generation refinement includes automated attribution with retrieved evidence and replacement of high-entropy words using context-aware methods.RARR aligns content with retrieved evidence, while high-entropy word methods identify and replace vulnerable words.
- Self-Refinement: Feedback and reasoning methods use self-contradiction detection, iterative reflection, or structured preference prediction to reduce hallucinations.The cited applications include medical generative question answering and coherence improvement.
- Prompt Tuning: Prompt-based methods include versatile prompt retrieval and synthetic tasks for abstractive summarization, while raising questions about effectiveness relative to human feedback.UPRISE emphasizes cross-task versatility, and SynTra emphasizes scalability.
- Model-Level Approaches: Decoding strategies, knowledge graphs, and faithfulness-based losses represent complementary model-level approaches to hallucination mitigation.The discussion names CAD, DoLa, RHO, and THAM as examples.
- Supervised Fine-Tuning: Supervised fine-tuning approaches inject domain knowledge into weaker models or use counterfactual datasets to improve factuality.The discussion identifies Knowledge Injection, teacher-student methods, and HAR as examples.
- Future Directions: Future directions include hybrid models, unsupervised or weakly supervised learning, research on built-in safety features, and collaboration to improve benchmarks and standards.The discussion also highlights ethical and societal considerations for responsible implementation and user confidence.
T5 AVSD@ DSTC7, AVSD@ DSTC8
The surveyed works span dialogue, question answering, biography generation, and factuality evaluation across multiple language models, datasets, and metrics. Their limitations include language-dependent gains, narrow model or domain coverage, and reliance on automatically generated data.
- Weighted-loss gains in faithfulness are inconsistent across languages, and machine-translated training data may limit feasibility and metric quality.
- The reviewed models include BLOOMZ-P3-7.1B, Vicuna, Phoenix-7B, OpenLLaMA-3B, LLaMA-7B, and LLaMA-13B.
- The surveyed techniques cover dialogue, question answering, biography generation, and medical question answering.
- Evaluation uses metrics including BLEU, ROUGE, BERTScore, Precision, and HaloCheck.
- Some studies are constrained to models up to 3B parameters, a single weak open-source model, or the NBA domain.