PaperlayerView exact revision on arXiv

Source-linked AI summary

VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models

Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, Zilong Zheng

arXiv:2406.16338v1cs.CV

TL;DR

Hallucination evaluation in video-language models remains limited despite their growing video-understanding capabilities. VideoHallucer assesses intrinsic and extrinsic hallucinations using adversarial pairs of basic and hallucinated video questions. Evaluations reveal widespread vulnerability, limited scaling benefits for extrinsic factual hallucinations, and greater model proficiency at recognizing facts than detecting hallucinations.

  • Problem

    Existing hallucination benchmarks mostly address static image attributes, leaving dynamic video content and comprehensive evaluation of LVLM hallucinations insufficiently covered.

    Source: §1 · p. 2

  • Method

    VideoHallucer is a comprehensive LVLM benchmark that uses binary video question-answering with paired basic and intentionally hallucinated questions across intrinsic and extrinsic categories.

    Source: §1 · p. 2

  • Results

    LVLMs show widespread hallucination issues; scaling improves detection of basic visual cues and counterfactuals but has limited impact on extrinsic factual hallucinations, and models recognize facts better than hallucinations.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Self-PEP integrates explanatory processes and improves hallucination resistance by an average of 5.38% across model architectures.

    Source: §1 · p. 3

    VideoHallucer remains limited in scalability, and human annotations introduce noise.

    Source: §A.7 · p. 29

Abstract

from arXiv · show

Recent advancements in Multimodal Large Language Models (MLLMs) have extended their capabilities to video understanding. Yet, these models are often plagued by "hallucinations", where irrelevant or nonsensical content is generated, deviating from the actual video context. This work introduces VideoHallucer, the first comprehensive benchmark for hallucination detection in large video-language models (LVLMs). VideoHallucer categorizes hallucinations into two main types: intrinsic and extrinsic, offering further subcategories for detailed analysis, including object-relation, temporal, semantic detail, extrinsic factual, and extrinsic non-factual hallucinations. We adopt an adversarial binary VideoQA method for comprehensive evaluation, where pairs of basic and hallucinated questions are crafted strategically. By evaluating eleven LVLMs on VideoHallucer, we reveal that i) the majority of current models exhibit significant issues with hallucinations; ii) while scaling datasets and parameters improves models' ability to detect basic visual cues and counterfactuals, it provides limited benefit for detecting extrinsic factual hallucinations; iii) existing models are more adept at detecting facts than identifying hallucinations. As a byproduct, these analyses further instruct the development of our self-PEP framework, achieving an average of 5.38% improvement in hallucination resistance across all model architectures.

1 Introduction

VideoHallucer addresses the limited evaluation of hallucinations in video-language models with a benchmark spanning intrinsic and extrinsic types. Its evaluation finds widespread model vulnerability, limited scaling benefits for extrinsic factual hallucinations, and improved performance with Self-PEP.

1 Introduction

  • Existing hallucination studies largely target static image attributes, leaving dynamic video content and comprehensive LVLM hallucination evaluation insufficiently examined.The resulting extent, severity, characteristics, and causes of LVLM hallucinations remain uncertain.Source: §1 · p. 2
  • VideoHallucer distinguishes intrinsic contradictions of source-video information from unverifiable extrinsic content, with intrinsic object-relation, temporal, and semantic-detail subtypes and factual and non-factual extrinsic subtypes.Extrinsic factual content aligns with general knowledge but is absent from the source video.Source: §1 · p. 2
  • The benchmark uses adversarial pairs of basic and intentionally hallucinated binary video questions, balances yes/no responses, and supplies explanations to reduce misinterpretation.The basic questions test LVLM ability, while hallucinated questions test hallucination detection.Source: §1 · p. 2, Figure 1 · p. 2
  • Current LVLMs show widespread hallucination issues, scaling helps detect basic visual cues and counterfactuals but has limited impact on extrinsic factual hallucinations, and models recognize facts better than hallucinations.Performance gaps between humans and models are significant across VideoHallucer settings.Source: §1 · pp. 2–3
  • Self-PEP integrates explanatory processes and yields an average 5.38% improvement in hallucination resistance across model architectures.The framework is described as plug-and-play.Source: §1 · p. 3

2 Related Work

Related work defines hallucinations by their relation to source material and develops image-language benchmarks for object, visual-feature, and factual errors. The paper identifies a lack of a clear, comprehensive framework for evaluating hallucinations in LVLMs.

Hallucination in NLG

  • Intrinsic hallucinations contradict source material, whereas extrinsic hallucinations cannot be confirmed or refuted by it and may be factual or non-factual.Factual hallucinations can be verified against real-world knowledge; non-factual hallucinations conflict with what is known about the world.Source: §2 · p. 3

Vision Hallucination Benchmark

  • Image-language hallucination work includes CHAIR for precision of hallucinated object references, POPE's binary VQA detection, and studies of relationships, attributes, other visual cues, and factual errors.CHAIR's results may be affected by instruction design and human-crafted parsing rules.Source: §2 · pp. 3–4
  • Despite image-language benchmarks, related work identifies a lack of a clear, comprehensive framework specifically designed to evaluate hallucinations in LVLMs.VideoHallucer is presented as a comprehensive benchmark for video-language understanding across five settings.Source: §2 · p. 4

3 The VideoHallucer Benchmark

VideoHallucer organizes video hallucination evaluation into five settings across intrinsic and extrinsic categories. Its benchmark pairs basic questions with hallucinated questions to test both fundamental abilities and resistance to hallucination.

3.1 Dataset Construction

  • VideoHallucer divides hallucination evaluation into intrinsic and extrinsic categories, producing five settings with separate question-answer pairs.The benchmark construction section introduces the question-pair construction for each setting.Source: §3.1 · p. 4

3.1.1 Intrinsic Hallucination

  • The intrinsic settings assess object-relation, temporal, and semantic-detail hallucinations using 400 question-answer pairs each.Object-relation questions cover subjects, relations, and objects; temporal questions test absolute and relative timing and event length; semantic-detail pairs use segments selected above a CLIP score of 0.85.Source: §3.1.1 · pp. 4–5

3.1.2 Extrinsic Hallucination

  • The extrinsic settings distinguish factual content not verifiable from the source from manually altered or counterfactual non-factual content.The factual setting contains 200 paired questions from 200 videos, while the non-factual setting contains 400 question-answer pairs linked to 200 videos.Source: §3.1.2 · p. 5

3.2 Dataset Statistics

  • The benchmark contains 1,800 question-answer pairs averaging 61.4 words, drawn from 948 videos averaging 85.6 seconds.Videos range from 7 to 187 seconds; the basic questions are identical across factual and non-factual settings.Source: §3.2 · pp. 5–6

3.3 Evaluation

  • VideoHallucer uses VideoQA rather than caption-based evaluation to avoid sensitivity to caption prompts and length, complex parsing rules, and LLM self-evaluation.The authors report a positive correlation between their QA-based evaluation and caption-based methods.Source: §3.3 · p. 6
  • Each evaluation item pairs a basic question with a hallucinated question, and counts as a hit only when both are answered correctly.The paired design is intended to limit answer-distribution and language biases without compromising performance on fundamental tasks.Source: §3.3 · p. 6
  • The benchmark also reports Yes Percentage Difference and False Positive Ratio to characterize model bias.A smaller Yes Percentage Difference indicates less language bias, while a False Positive Ratio closer to 50% indicates less bias.Source: §3.3 · p. 6

4 Experiment

The experiments evaluate LVLMs on VideoHallucer, analyze hallucination performance across settings, compare image- and video-language models, and report human evaluation. Overall, models answer basic questions better than hallucinated questions, with weaknesses varying by hallucination type.

4.1 Setups

  • The evaluation covers twelve LVLMs, using their original frame counts and generation hyperparameters; three English-proficient evaluators show moderate agreement (Pearson correlation 0.557).Source: §4.1 · pp. 6–7, §4.4 · pp. 8–9

4.2 Main Benchmark Results

  • Models’ accuracy drops on hallucinated questions, and all lack significant human-like hallucination detection; Table 3 reports basic, hallucinated, and overall scores.Models with an obvious Yes/No bias are more likely to have hallucination problems.Source: §4.2 · p. 7, Table 3 · p. 7
  • Most models detect object-relation hallucinations less reliably than other types, with performance centered around 50%.The authors interpret this as a deficiency in detecting hallucinations beyond elementary visual cues.Source: §4.2 · p. 7
  • More training data improves semantic-detail hallucination performance, and larger models outperform smaller ones on non-factual settings.The authors attribute these differences to data and parameter scaling, which appear to support visual-detail recognition and world-knowledge retention.Source: §4.2 · p. 7
  • Most models struggle with extrinsic factual hallucinations that align with world knowledge but contradict the video context.Source: §4.2 · p. 7

4.3 Comparison between Image-Language Models

  • Open-source image-language models outperform video-language models on object-relation hallucination, despite the dataset including dynamic interactions.The comparison uses the middle frame of each video for image-language models; the authors suggest differences in image/video training-data volume and video noise may contribute.Source: §4.3 · p. 8, Table 4 · p. 8

5 Self-PEP: Self-improvement with Predict-Explain-Predict

The paper investigates why models struggle with extrinsic factual hallucinations and proposes Self-PEP, combining visual-knowledge extraction with predict-explain-predict refinement. Self-PEP improves most models on VideoHallucer, though PLLaVA may be harmed outside the factual setting.

5.1 Fact Detection vs. Hallucination Detection

  • Models are more adept at detecting factuality than hallucination, and the paper creates additional QA pairs to assess factual accuracy in extrinsic-hallucination questions.Source: Figure 5 · p. 9, §5.1 · p. 9

5.2 Self-PEP Framework

  • Self-generated explanations improve hallucination detection, though less than ground-truth explanations; the improvement is related to fact-detection proficiency.Source: §5.2 · pp. 9–10, Figure 7 · p. 10
  • Self-PEP has two phases: self-improvement extracts visual knowledge, while self-explanation predicts, explains, and refines the prediction.Source: §5.2 · p. 10, Figure 6 · p. 10
  • Self-PEP significantly improves most models on VideoHallucer, with larger gains on hallucinated than basic questions and consistent gains on extrinsic factual hallucinations.Source: §5.2 · p. 10, Table 5 · p. 10
  • PLLaVA may be harmed in other settings even when its performance improves on the factual setting.Source: §5.2 · p. 11

6 Conclusion and Discussion

VideoHallucer evaluates hallucinations and faithfulness in LVLMs through paired questions. Its evaluation of 12 models finds pervasive hallucinations and scaling limitations, while Self-PEP improves most models’ benchmark performance.

6 Conclusion and Discussion

  • VideoHallucer evaluates hallucinations and faithfulness in LVLMs using paired questions designed to test whether responses are accurate and reliable in video context.The paper distinguishes this perceptual evaluation from adversarial attacks, which aim to provoke errors that may lead to security breaches.Source: §6 · p. 11
  • Other video-language benchmarks may target basic visual understanding or abstract cognitive tasks, whereas VideoHallucer focuses on hallucinated content and faithfulness of generated text.The paper characterizes its focus as direct and specific.Source: §6 · p. 11
  • VQA-based evaluation avoids caption-length and prompt effects, complex human-crafted parsing rules, and reliance on LLMs as evaluators, while the paper reports positive correlation with caption-based methods.The authors report using VQA-based evaluation and a positive correlation with caption-based methods to support the benchmark’s credibility.Source: §6 · p. 11

A Appendix

The appendices provide further details on evaluation, annotation, implementation, results, and limitations, alongside example questions spanning VideoHallucer’s hallucination types.

A Appendix

  • Appendix A.1 analyzes two evaluation methods, and A.2 describes annotation procedures.The remaining materials include implementation details, coherence analysis, quantitative results, example questions, and limitations and ethics statements.Source: A Appendix · p. 19
  • Appendix A.3 covers baseline configurations, evaluation prompts, and Self-PEP implementation details.A.4 analyzes coherence between self-generated and ground-truth explanations.Source: A Appendix · p. 19
  • Appendix A.5 presents detailed VideoHallucer and Self-PEP results, while A.6 provides example questions across the benchmark’s hallucination categories.A.7 covers limitations and ethics statements.Source: A Appendix · p. 19

A.1 Evaluation Method Analysis

This analysis compares VideoHallucer’s QA-based accuracy with caption-based metrics on 100 COCO images using LLaVA-1.5. It reports moderate positive correlation with Coverage Score and weak negative correlation with CHAIR.

A.1 Evaluation Method Analysis

  • r = 0.477 for VideoHallucer accuracy and Coverage Score, while r = −0.139 for VideoHallucer accuracy and CHAIR score.The analysis used 100 randomly selected COCO images and LLaVA-1.5.Source: §A.1 · p. 19
  • The study compares QA-based accuracy with CHAIR and Coverage Score, using Pearson correlation to measure coherence between VideoHallucer and caption-based evaluation.A rule-based method similar to CHAIR calculates Coverage Score to avoid relying on LLMs for evaluation.Source: §A.1 · p. 19
  • The authors interpret the correlations as indicating that VideoHallucer accuracy correlates positively with caption-based methods and is a reliable alternative for assessing hallucinations.The reported positive relationship is with Coverage Score; the CHAIR correlation is weak and negative.Source: §A.1 · p. 19

A.2 Annotation Details

The annotation details describe a semi-automatic construction process and several benchmark sets targeting object-relation, temporal, semantic-detail, and extrinsic hallucinations. These sets use video sources and question designs tailored to the type of hallucination being tested.

Evaluation Method Analysis

  • The appendix figure is captioned as showing the correlation between VideoHallucer accuracy and Coverage Score.The supplied caption identifies the comparison but does not state its outcome.Source: Figure 8 · p. 19

Annotation Pipeline

  • The benchmark construction pipeline begins with tools and templates for basic and hallucinated question-answer pairs, followed by human review for grounding, correctness, ambiguity, and fluency.The passage describes a semi-automatic process developed to improve quality and reduce annotation costs, with a two-stage human verification process.Source: §A.2 · pp. 19–20

Object-Relation Hallucination

  • The object-relation set uses VidOR and VidVRD annotations, divides questions into subject, relation, and object categories, and contains 200 basic-hallucination pairs across 183 videos.Hallucinated questions use semantically different but visually similar objects or altered relation words; question-answer pairs are manually checked.Source: §A.2 · p. 20

Temporal Hallucination

  • Temporal questions test absolute event position, event order, and event duration, with 50 examples for absolute understanding and 75 each for relative order and event length.The benchmark uses ActivityNet Captions validation videos with event descriptions and time spans; relative-order pairs have a time gap of at least 0.1 of video duration, and event-length pairs differ by at least a factor of two.Source: §A.2 · pp. 20–21

Semantic Detail Hallucination

  • The semantic-detail set compares clips with CLIP scores above 0.85, asks annotators to identify semantic differences, and yields 200 basic-hallucination pairs across 400 videos.The selected clips come from HawkEye, which segments long videos and merges clips with high semantic similarity.Source: §A.2 · p. 21

Extrinsic Factual Hallucination

  • Extrinsic factual hallucinations are unverifiable against the source yet factually consistent, and the benchmark tests their detection rather than their overall value.The paper notes that their usefulness depends on context, contrasting source-faithful summarization with conversational settings where varied responses may be expected.Source: §A.2 · p. 21
  • For instructional videos, hallucinated questions focus on omitted final steps; for course lectures, annotators alter summaries or add knowledge absent from the video, making summaries not solely derivable from it.The dataset contains 200 paired basic and hallucination questions and answers across 200 videos: 150 instructional videos and 50 online courses.Source: §A.2 · pp. 21–22

Extrinsic Non-factual Hallucination

  • The extrinsic non-factual set reuses the factual set’s videos and basic questions, then creates hallucinations by altering questions or adding counterfactual information.It contains 400 question-answer pairs linked to 200 videos.Source: §A.2 · p. 22

A.3.1 Setups for baselines

The baseline comparison uses 7B-level video-language models with their default hyperparameters, alongside Gemini-1.5-Pro as an upper-bound reference.

A.3.1 Setups for baselines

  • Eight 7B-level video-language models are compared using each model’s default hyperparameters, including frame-count settings.The listed settings include max_new_tokens, do_sample, temperature, num_beams, and num_of_frames.Source: §A.3.1 · p. 22
  • Gemini-1.5-Pro is added as the current upper bound, evaluated at 1 fps with a maximum of 128 frames.Source: §A.3.1 · p. 22

A.3.2 Prompt for the Overall Results

The appendix specifies binary-response scoring, details Self-PEP and its explanation analysis, and catalogs quantitative results, example questions, and benchmark limitations.

A.3.2 Prompt for the Overall Results

  • Responses are scored by accuracy after appending a yes-or-no instruction and comparing the answer with the yes-or-no target.Source: §A.3.2 · p. 22

A.3.3 Implementation Detail of the Self-PEP

  • Self-PEP first uses a generated video description with the video to answer, then asks the model to explain, verify its facts, and predict again.The qualitative example reports that explanation helped Gemini-1.5-Pro correct its answer.Source: §A.3.3 · p. 22, Figure 9 · p. 23

A.4 Explanation Analysis

  • GPT-4 consistency analysis compares self-generated and ground-truth explanations; Table 6 reports that explanation quality significantly affects hallucination identification.Figure 10 supplies the analysis prompt.Source: §A.4 · pp. 22–23, Table 6 · p. 23, Figure 10 · p. 23

A.5 Detailed Quantitative Results

  • Various LVLMs are evaluated across VideoHallucer settings in Tables 7–11 for object-relation, temporal, semantic-detail, extrinsic factual, and extrinsic non-factual hallucinations; Self-PEP results across the same settings are in Tables 12–16.Source: §A.5.1 · p. 24, §A.5.2 · p. 25

A.6 Example Questions of VideoHallucer

A.7 Limitations and Ethic States

  • Further examples cover semantic-detail hallucinations involving attribution, events, counts, OCR, camera, and scene content.Source: Figure 16 · p. 31, Figure 17 · p. 31, Figure 18 · p. 31
  • The examples also include extrinsic factual and non-factual instruction and course cases; the benchmark has human-annotation noise, limited scalability, and risk of misinterpretation in related research.The authors also state responsibility for rights violations and identify the dataset license as CC BY-NC-SA.Source: §A.7 · pp. 29–30, Figure 22 · p. 33, Figure 23 · p. 33
Loading 2406.16338v1…