Source-linked AI summary
A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models
Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, Aman Chadha
TL;DR
Foundation models can produce hallucinated outputs across language, vision, video, and audio, creating reliability concerns for critical applications. This survey synthesizes multimodal research on hallucination definitions, taxonomies, detection, mitigation, benchmarks, and future directions. It provides a structured framework for understanding and addressing the problem while acknowledging incomplete coverage through May 2024.
Problem
Foundation models can generate hallucinated outputs across language, vision, video, and audio, while prior surveys mainly focused on language models.
Method
The survey systematically categorizes multimodal hallucination research and synthesizes detection, mitigation, task, dataset, evaluation, and future-direction literature.
Results
The paper establishes a structured taxonomy and comprehensive overview of hallucination detection and mitigation across text, image, video, and audio foundation models.
Takeaways & Limitations
The framework supports broader analysis of hallucination research across modalities and identifies directions including multimodal alignment, standardized benchmarking, and improved detection and mitigation.
Takeaways & Limitations
The survey may miss relevant work and covers research published through May 2024.
Abstract
from arXiv · showhide
The rapid advancement of foundation models (FMs) across language, image, audio, and video domains has shown remarkable capabilities in diverse tasks. However, the proliferation of FMs brings forth a critical challenge: the potential to generate hallucinated outputs, particularly in high-stakes applications. The tendency of foundation models to produce hallucinated content arguably represents the biggest hindrance to their widespread adoption in real-world scenarios, especially in domains where reliability and accuracy are paramount. This survey paper presents a comprehensive overview of recent developments that aim to identify and mitigate the problem of hallucination in FMs, spanning text, image, video, and audio modalities. By synthesizing recent advancements in detecting and mitigating hallucination across various modalities, the paper aims to provide valuable insights for researchers, developers, and practitioners. Essentially, it establishes a clear framework encompassing definition, taxonomy, and detection strategies for addressing hallucination in multimodal foundation models, laying the foundation for future research in this pivotal area.
1 Introduction
Foundation models span language, image, audio, and video, but can generate plausible yet factually incoherent or contextually disconnected content. This survey organizes multimodal hallucination research around definitions, taxonomies, detection, mitigation, and future directions.
- Foundation models increasingly support diverse language, image, audio, and video tasks while raising concerns about hallucinated outputs in critical applications.
- Hallucinated outputs can range from minor inaccuracies to imaginary content and may perpetuate misinformation or incorrect conclusions in critical applications.
- Hallucinations include contextual disconnection, semantic distortion, content hallucination, and factual inaccuracy across generated modalities.
- The survey extends prior reviews beyond language models to provide a holistic perspective across language, vision, video, and audio.
- Its contributions include a structured taxonomy, multimodal detection and mitigation strategies, and open challenges for future research.
2 Hallucination in Large Language Models
LLM hallucinations undermine factual reliability, motivating detection and mitigation methods that address inconsistent, unsupported, or fabricated content. Research spans black-box detection, citation mechanisms, benchmark evaluation, and formal definitions.
- Hallucination Detection and Mitigation: LLM hallucinations can impede practical implementation by producing inaccurate responses despite strong language-model capabilities.Figure 3 illustrates generated responses containing hallucinations.
- Hallucination Detection and Mitigation: SelfCheckGPT detects hallucinations in any LLM using a zero-resource, black-box approach without external resources.Its principle is that familiarity with a topic produces more consistent responses.
- Hallucination Detection and Mitigation: Citation mechanisms are identified as a significant missing capability because they acknowledge or reference sources and evidence.The cited discussion compares this gap with established web systems.
- Hallucination Detection and Mitigation: 40% average accuracy improvement on FACTOID was achieved by a multi-task framework compared with leading textual entailment methods.The framework integrates long-text embeddings with GPT-3, SpanBERT, and RoFormer.
- Benchmark Evaluation: Hallucination snowballing occurs when models fabricate false claims to rationalize earlier hallucinations, even while recognizing some claims as incorrect.Three question-answering datasets were used to study this behavior across diverse domains.
3 Hallucination in Large Vision-Language Models
Vision-language models can hallucinate objects or other visual content that does not match an input image. Existing work evaluates these errors and mitigates them through decoding, data, training, and post-processing strategies.
- Hallucination Detection and Mitigation: LVLMs process visual and textual data simultaneously but remain susceptible to visual hallucinations.Figure 4 presents an example of visual hallucination.
- Hallucination Detection and Mitigation: Object hallucinations occur when generated descriptions contain nonexistent or inaccurate objects relative to the input image.Visual instructions and co-occurring image objects can influence which objects are hallucinated.
- Hallucination Detection and Mitigation: MARINE reduced hallucinations and enhanced output detail across evaluations of six LVLMs, with validation using GPT-4V.The cited passage describes MARINE as improving output precision.
- Hallucination Detection and Mitigation: HalluciDoctor mitigated 44.6% of hallucinations while maintaining competitive performance through counterfactual visual-instruction expansion.Its approach uses human error detection and rebalances the data distribution.
- Hallucination Detection and Mitigation: VHTest contains 1,200 visual-hallucination instances across eight modes, while another dataset contains 2,000 samples covering eight orientations.Proposed mitigation categories include data-driven approaches, training adjustments, and post-processing techniques.
4 Hallucinations in Large Video Models
Video models hallucinate when they misinterpret frames or fail to capture temporal context, affecting dense captioning and other video-understanding tasks. Research addresses these problems through contextual modeling, datasets, and specialized video-generation methods.
- Hallucination Detection and Mitigation: Large video models can misinterpret video frames, producing artificial or inaccurate visual data.Figure 5 shows observed hallucination instances in large video models.
- Hallucination Detection and Mitigation: Dense video captioning risks hallucinations when traditional pipelines detect event proposals before captioning and overlook temporal dependencies.The task requires understanding multiple events within a continuous video and their context.
- Hallucination Detection and Mitigation: Context-aware and reinforcement-learning approaches are used to incorporate contextual information and produce more coherent, accurate captions.Two-level rewards and past or future events are described as mechanisms for improving contextual handling.
- Benchmark Evaluation: Inference-time video evaluation uses keyframes with dense captions and structured FA-MOUS scene descriptions to assess temporal comprehension and anticipation.FA-MOUS represents Focus, Action, Mood, Objects, and Setting.
- Hallucination Detection and Mitigation: The Deficiency-aware Masked Transformer addresses incomplete information through dual-modality-compatible inpainting and an image-inpainting prior.The image model is pretrained to provide a prior for video-model training.
- Benchmark Evaluation: Audio-visual hallucination examples distinguish errors involving both objects and actions from errors involving only hallucinated actions or objects.Type B has accurate objects but hallucinated actions, while Type C has correct actions but hallucinated objects.
5 Hallucinations in Large Audio Models
Large audio models support recognition, synthesis, analysis, and captioning but may fabricate audio content or misrepresent acoustic features. Research examines these errors through categorized hallucinations and develops audio-text datasets and captioning methods.
- Hallucination Detection and Mitigation: Large audio models can generate unrealistic audio, fabricate snippets, inject false information, and misrepresent timbre, pitch, or background noise.These failures appear in tasks including audio synthesis and summarization.
- Hallucination Detection and Mitigation: Audio captioning can be undermined by over-reliance on visual modality during audio-text pretraining, which introduces data noise and hallucinations.AudioSet tag-guided modeling was introduced to bootstrap large-scale audio-text data.
- Hallucination Detection and Mitigation: SECap generates natural-language descriptions of speech emotions using LLaMA, HuBERT, and Q-Former components.The framework aims to capture nuanced emotional features in speech.
- Hallucination Detection and Mitigation: LP-MusicCaps contains 0.5 million audio clips and approximately 2.2 million captions for music-captioning research.A transformer-based music-captioning model was trained with this dataset.
- Benchmark Evaluation: Audio hallucinations are classified as errors involving both objects and actions, accurate objects with hallucinated actions, or correct actions with hallucinated objects.A set of 1,000 solicited audio-information sentences was annotated for auditory hallucinations.
6 Future Directions
Future work emphasizes improving data quality, evaluation, detection, mitigation, and multimodal reliability for foundation models.
- Data Resources: Curated high-quality data, knowledge-graph integration, and task- or domain-specific alignment are proposed to improve accuracy and relevance.
- Automated Evaluation: Specialized factuality and coherence metrics should be combined with human judgments, adversarial testing, and fact-checking datasets.
- Improving Detection and Mitigation: Reasoning mechanisms, knowledge graphs, fact-checking models, bias mitigation, active learning, and ethical frameworks are proposed for detection and mitigation.
- Multimodal Hallucination: Multimodal hallucination research should address data quality, cross-modal alignment, architecture, benchmarking, interpretability, and trust.
7 Conclusion
The survey organizes research on foundation-model hallucination into a structured taxonomy spanning text, image, video, and audio. It also synthesizes work on detection, mitigation, tasks, datasets, and evaluation metrics.
- The survey systematically categorizes hallucination research across critical aspects including detection, mitigation, tasks, datasets, and evaluation metrics.
- Its primary contribution is a structured taxonomy for hallucination across text, image, video, and audio foundation models.
- The paper examines detection and mitigation advances because foundation models play indispensable roles in critical tasks.
8 Limitation
The survey broadens coverage beyond language-model hallucination to vision, audio, and video modalities. It acknowledges that some relevant work may be missing because the review covers papers through May 2024.
- Earlier surveys primarily focused on hallucination in large language models rather than extensively covering vision, audio, and video modalities.
- The survey aims for comprehensive multimodal coverage but may omit relevant work, with its literature coverage ending in May 2024.
9 Appendix
The appendix summarizes methodologies for hallucination research and organizes works by modality and analytical factors. Table 1 provides a concise overview of detection, mitigation, tasks, datasets, and evaluation metrics.
- The appendix summarizes methodologies for hallucination techniques in large foundation models.
- The table is intended to offer readers a concise overview of recent advancements in the field.
- Table 1 categorizes work across text, image, video, and audio modalities by detection, mitigation, tasks, datasets, and evaluation metrics.