Source-linked AI summary
Cognitive Mirage: A Review of Hallucinations in Large Language Models
Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, Weiqiang Jia
TL;DR
LLMs can produce inaccuracies as their encoded knowledge is constrained and may be incomplete or outdated. This review synthesizes hallucination mechanisms and task-specific taxonomy alongside detection and mitigation approaches. It concludes with future directions including stronger synergy with external knowledge bases and more efficient detection methods.
Problem
LLMs’ knowledge representation is constrained, and knowledge distortions can give rise to potential inaccuracies.
Source: §1 · p. 1Method
The paper reviews hallucination mechanisms and task-specific taxonomy, together with detection and correction methods and future research directions.
Source: §1 · p. 2, §7 · p. 12Results
The review provides an overview of hallucinations in LLMs spanning taxonomy, theoretical insight, detection and correction methods, and future directions.
Source: §7 · p. 12Takeaways & Limitations
The authors envision stronger synergy between LLMs and external knowledge bases and call for sophisticated, efficient detection methods.
Source: §7 · p. 12The review says the causes of hallucinations in multimodal LLMs have not been comprehensively investigated.
Source: §6 · p. 12
Abstract
from arXiv · showhide
As large language models continue to develop in the field of AI, text generation systems are susceptible to a worrisome phenomenon known as hallucination. In this study, we summarize recent compelling insights into hallucinations in LLMs. We present a novel taxonomy of hallucinations from various text generation tasks, thus provide theoretical insights, detection methods and improvement approaches. Based on this, future research directions are proposed. Our contribution are threefold: (1) We provide a detailed and complete taxonomy for hallucinations appearing in text generation tasks; (2) We provide theoretical analyses of hallucinations in LLMs and provide existing detection and improvement methods; (3) We propose several research directions that can be developed in the future. As hallucinations garner significant attention from the community, we will maintain updates on relevant research progress.
1 Introduction
LLMs can produce fluent but unfaithful or factually inaccurate text, a reliability challenge for knowledge-intensive uses. This review organizes prior work around hallucination mechanisms, task-specific taxonomy, detection and mitigation, and future directions.
1 Introduction
- LLMs’ knowledge can be incomplete or outdated, and limits in model scale and long-tail knowledge contribute to potential inaccuracies.The review links these challenges to reliability concerns in accuracy-sensitive fields, including medicine and finance.Source: §1 · p. 1
- The review analyzes hallucination mechanisms and task-specific taxonomy, then surveys detection and mitigation methods before proposing future research directions.Its mechanism analysis covers data collection, knowledge gaps, and optimization.Source: Figure 2 · p. 2, §1 · p. 2
- Hallucination is fluent, grammatical text that departs from source faithfulness or factual accuracy; traditional NLP emphasizes faithfulness, while LLM work prioritizes factualness.The review also describes uncritical trust in LLMs as a “Cognitive Mirage” that can contribute to misguided decisions.Source: §1 · p. 2
- The review presents a comprehensive taxonomy of hallucinations across text-generation tasks and compares task-specific benchmarks.This contribution is described in §3.Source: §1 · p. 2
- It investigates proactive detection and mitigation methods as areas for informing future research.The review covers these topics in §§4–5 and connects them to directions in §6.Source: §1 · p. 2
- Existing surveys and benchmarks were scattered across tasks, and the review aims to systematically understand methodologies, compare methods, and inspire ideas.It distinguishes its coverage from earlier work focused on task-specific research progress or particular methods.Source: §1 · pp. 2–3
2 Mechanism Analysis
The review frames generation as next-token prediction and analyzes hallucinations through data collection, knowledge gaps, and optimization. It describes distinct sources of risk within each mechanism category.
Mechanism Analysis
- A parameterized probabilistic model predicts successive tokens using task instructions, demonstrations, input, and preceding generated tokens.Demonstrations provide analogy samples, while model parameters implicitly memorize corpus knowledge.Source: §2 · p. 3
Data Collection
- Data-related risks include incomplete or outdated pretraining knowledge, demonstration-order and label disparities, and difficulties with low-resource languages and visual co-occurrence.The review also notes that unavailable memories may lead models to rely on corpus-based term-frequency heuristics.Source: §2 · p. 3
Knowledge Gap
- Knowledge gaps arise from pretraining–finetuning input differences, retrieval mismatches, biased reliance on model knowledge, and inconsistency with cached hidden states.The review describes balancing model memory with retrieved evidence as a challenge.Source: §2 · p. 3
Optimization Process
- Optimization-related risks include training to imitate data without comprehension, exposure bias in long responses, uncertain sampling, and snowballing hallucinations.The review also reports that “think step by step” prompts can still yield ineffectual reasoning chains.Source: §2 · pp. 3–4
3 Taxonomy of Hallucination
Hallucination types vary across machine translation, question answering, dialogue, summarization, knowledge-graph generation, and cross-modal tasks. The section surveys representative task-specific forms rather than treating hallucination as a single pattern.
Machine Translation
- In machine translation, LLM hallucinations include off-target translations, over-generation, and failed translation attempts, with low-resource language settings also constrained by limited annotated data.The review also describes oscillatory hallucination in multilingual models trained on different scales of monolingual data.Source: §3 · p. 4
Question and Answer; Dialog System
- Question-answering hallucinations include incomplete but plausible answers when relevant information is absent, while dialogue systems may copy evidence or produce other nuanced unfaithful responses.The review links question-answering problems to flawed external knowledge, recall cues, reasoning instructions, and inaccurate or inaccessible memorized information.Source: §3 · p. 4
Summarization System
- Summarization hallucinations are intrinsic when they distort source information and extrinsic when they add information not directly attributable to the source.The review notes that extrinsic additions may be factual or non-factual.Source: §3 · p. 4
Knowledge Graph with LLMs
- Knowledge-graph generation can add redundant details from memorized knowledge, and source-fidelity categories include subject, relation, and object hallucinations.The review distinguishes correctly generated knowledge from knowledge hallucinations in terms of knowledge creation.Source: §3 · p. 4
Cross-modal System
- Cross-modal systems can generate object descriptions absent from the target image, with reported failure cases in visual question answering, image captioning, and report generation.The review identifies these as object hallucinations in large visual language models.Source: §3 · p. 6
4 Hallucination Detection
The review groups hallucination detection methods into several approaches while noting that conventional task-specific metrics do not assess factualness. It surveys classifier, uncertainty, self-evaluation, and evidence-retrieval methods.
Detection limitations
- Conventional task-specific metrics measure reference overlap or knowledge-aware response ability, but focus on faithfulness and fail to assess factualness.The review also identifies limitations in plugin-based, QA-matching, and NLI-based reference-free methods.Source: §4 · p. 6
Inference Classifier
- Inference classifiers estimate whether an answer contains hallucinated content, with approaches using LLM-generated judgments, intermediate explanations, or a classifier distinct from the generator.The review also describes multimodal binary classifiers for radiology report reliability.Source: §4 · p. 6
Uncertainty Metric; Self-Evaluation
- Uncertainty methods use signals including logits, generated-text scoring, response consistency, and answer hedging to assess output quality or factual confidence.Examples include consistency across sampled responses and information-theoretic measures such as entropy and KL-divergence.Source: §4 · pp. 6–7
Evidence Retrieval
- Evidence-retrieval methods gather supporting evidence for generated claims, including claim-centric retrieval and FActScore’s percentage of atomic facts supported by a knowledge source.FacTool integrates tools into a task- and domain-agnostic detection framework to assemble evidence about generated content.Source: §4 · p. 7
5 Hallucination Correction
The review organizes hallucination correction into five paradigms and describes methods that adapt model parameters, edit outputs, retrieve external knowledge, use assessment feedback, or involve multiple agents. It notes that the paradigms can complement one another and are not entirely orthogonal.
Parameter Adaptation
- Parameter adaptation conditions, edits, or optimizes model parameters to address pretraining biases and improve alignment with user intent.Examples include task-specific parameter modules, language-specific modules, factuality-aware sampling, and inference-time activation shifts.Source: §5 · pp. 7–8
Correction overview
- The five correction paradigms are not entirely orthogonal and may complement one another according to the task.The review relates these paradigms to mainstream NLP tasks.Source: §5 · p. 7
Post-hoc Attribution and Edit Technology
- Post-hoc attribution and editing methods probe pretraining evidence or revise generated reasoning and responses using feedback, retrieved knowledge, or knowledge graphs.The review describes iterative self-refinement and verification-and-editing as ways to revise outputs.Source: §5 · pp. 8–9
Leverage External Knowledge
- External-knowledge methods retrieve documents, knowledge-base facts, or API information to support generation and revise low-confidence content.Some approaches interleave retrieval with chain-of-thought generation, while others retrieve non-parametric memories when needed.Source: §5 · p. 9
Assessment Feedback
- Assessment-feedback methods use human preferences, refusal behavior, or generated feedback to train models and improve response reliability.The review notes that human-annotated reward models can be inefficient in data utilization and reinforcement learning can rely on imperfect reward functions.Source: §5 · pp. 9–10
Mindset Society
- Mindset-society approaches use interaction among models or systems, including debate and peer assessment, to seek consistent or agreed-upon responses.Examples include judge-managed debate, roundtable debates, and role assignment across LLMs.Source: §5 · p. 10
6 Future Directions
The paper proposes future work on data construction, downstream-task alignment, evaluation, reasoning, and multimodal hallucinations. Its directions include improving knowledge quality and task-specific reasoning while balancing factuality with creative or diverse outputs.
Data Construction Management
- Data construction should prioritize high-quality, diverse information, entity-centered instructions, and self-curation of candidate instruction pairs.The paper also proposes manual or automated quality checks during iteration.Source: §6 · pp. 10–11
Downstream Task Alignment
- Downstream alignment calls for domain cognition, symbolic reasoning and planning, faithful knowledge injection, and balancing story creativity against factual consistency.For knowledge-intensive tasks, the paper identifies both incorporating knowledge graphs into pre-training and dynamically injecting their information during reasoning as possible directions.Source: §6 · p. 11
LLM Evaluation and Task-Specific Mechanisms
- LLM-based evaluation remains limited by model size, instruction adjustments, and input forms, while judge models face position, verbosity, and self-enhancement biases.The paper calls for further research into task-specific mechanisms for analyzing and correcting processes in emerging downstream tasks.Source: §6 · p. 11
- Task-specific mechanisms for analyzing and correcting emerging downstream-task processes warrant long-term attention.The paper does not specify a single mechanism for these emerging tasks.Source: §6 · p. 11
Reasoning Mechanism Exploitation
- Reasoning directions span multiple chains of thought, tree-structured search, cumulative decomposition, graph-based thought aggregation, and programming logic.The paper also points to human-cognition theories as a basis for studying hallucinations.Source: §6 · p. 11
Multi-modal Hallucination Survey
- Multimodal hallucination research should address visual-semantic misalignment, spurious cues, and weak visual constraints while balancing output diversity with hallucination reduction.Suggested directions include penalizing attention that deviates from images, strengthening visual common sense, and aligning local image features with text descriptions.Source: §6 · pp. 11–12
7 Conclusion and Vision
The paper concludes by presenting an overview of LLM hallucinations, including a taxonomy, theoretical insights, detection and correction methods, and future research directions. It envisions stronger synergy with external knowledge bases and continued work toward reliable information across applications.
Conclusion and Vision
- The paper's overview combines a hallucination taxonomy, theoretical insight, detection and correction methods, and future research directions.It frames this synthesis as an analysis of hallucination causes and task-oriented taxonomy to explore possible improvements.Source: §7 · p. 12
- The authors envision closer synergy between LLMs and external knowledge bases toward a credible interactive system.They also call for sophisticated, efficient detection methods and a proactive community response to hallucinations.Source: §7 · p. 12
- The authors hope creative correction methods across different aspects will support reliable and secure information in broad application scenarios.They also hope that improved detection contributes to LLM performance.Source: §7 · p. 12