PaperlayerView exact revision on arXiv

Source-linked AI summary

ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Models

Vipula Rawte, Sarthak Jain, Aarush Sinha, Garv Kaushik, Aman Bansal, Prathiksha Rumale Vishwanath, Samyak Rajesh Jain, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha, Amit P. Sheth, Amitava Das

arXiv:2411.10867v2cs.CVcs.AI

TL;DR

Text-to-video models can generate hallucinated content that creates discrepancies with prompts, a concern in applications requiring high fidelity. ViBe builds a manually annotated dataset and embedding-based classification benchmark; its 3,782 videos make it 75 times the scale of prior T2VHaluBench.

  • Problem

    Text-to-video models can produce hallucinated content that misaligns with or distorts the intended scene, threatening realism and reliability where prompt fidelity matters.

    Source: §1 · p. 1

  • Method

    ViBe uses 837 diverse MS COCO captions to generate and manually annotate videos from 10 open-source models, defining five hallucination categories and a video-embedding classification benchmark.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    3,782 manually annotated videos make ViBe 75 times the scale of prior T2VHaluBench, with a five-category taxonomy for systematic analysis.

    Source: §5 · p. 8

  • Takeaways & Limitations

    ViBe provides a foundation for evaluating text-to-video hallucinations and studying errors to improve model fidelity.

    Source: §1 · p. 1

    Videos receive a single hallucination category, which may miss overlaps, and the dataset is limited to short durations.

    Source: §6 · p. 8

Abstract

from arXiv · show

Recent advances in Large Multimodal Models (LMMs) have expanded their capabilities to video understanding, with Text-to-Video (T2V) models excelling in generating videos from textual prompts. However, they still frequently produce hallucinated content, revealing AI-generated inconsistencies. We introduce ViBe (https://vibe-t2v-bench.github.io/): a large-scale dataset of hallucinated videos from open-source T2V models. We identify five major hallucination types: Vanishing Subject, Omission Error, Numeric Variability, Subject Dysmorphia, and Visual Incongruity. Using ten T2V models, we generated and manually annotated 3,782 videos from 837 diverse MS COCO captions. Our proposed benchmark includes a dataset of hallucinated videos and a classification framework using video embeddings. ViBe serves as a critical resource for evaluating T2V reliability and advancing hallucination detection. We establish classification as a baseline, with the TimeSFormer + CNN ensemble achieving the best performance (0.345 accuracy, 0.342 F1 score). While initial baselines proposed achieve modest accuracy, this highlights the difficulty of automated hallucination detection and the need for improved methods. Our research aims to drive the development of more robust T2V models and evaluate their outputs based on user preferences.

1 Introduction

Text-to-video models can produce hallucinated content that compromises output realism and reliability, especially where prompt fidelity matters. ViBe contributes a manually annotated dataset and a video-embedding classification benchmark that establishes baseline performance and highlights the difficulty of hallucination detection.

1 Introduction

  • Hallucinated content creates visual discrepancies with the intended scene and threatens realism and reliability, particularly in applications requiring high prompt fidelity.The cited applications include content creation, education, and simulation systems.Source: §1 · p. 1
  • ViBe contains 3,782 manually annotated videos generated from 837 diverse MS COCO captions using 10 open-source text-to-video models.Annotations identify common hallucination types, including disappearing subjects, missing scene components, numerical inconsistencies, and visual distortions.Source: §1 · p. 1
  • The paper presents ViBe as the first dataset focused on systematically categorizing hallucinations in generated videos.It provides a foundation for studying errors in text-to-video generation and improving model fidelity.Source: §1 · p. 1
  • ViBe defines five major hallucination categories and provides human-annotated labels for analyzing and mitigating common text-to-video errors.The supplied passage names the count but does not enumerate the categories.Source: §1 · pp. 1–2
  • The video-embedding classification benchmark establishes baseline performance and highlights the difficulty of hallucination detection.The framework uses video embeddings and classification models to identify and categorize hallucinations.Source: §1 · pp. 2–3

2 Related Work

Prior hallucination research spans text and image generation, while video work addresses frame consistency and proposes detection and evaluation tools. ViBe targets the stated gap in large-scale, human-annotated data for studying T2V hallucinations.

2 Related Work

  • Text-generation benchmarks measure how often and severely language models produce misleading or incorrect content, while image-generation datasets assess hallucinations using token-focused, reference-free evaluation.The cited examples are the Hallucinations Leaderboard and HADES (Liu et al., 2022).Source: §2 · p. 3
  • Video hallucinations include unrealistic motion, sudden changes in object appearance, and scenes that contradict real-world physics amid the challenge of consistency across frames.Source: §2 · p. 3
  • Sora Detector (Chu et al., 2024a) analyzes key frames against knowledge graphs, VideoHallucer (Wang et al., 2024b) separates model errors from external inconsistencies, and VBench assesses video quality.Source: §2 · p. 3
  • ViBe addresses the stated lack of a large-scale, human-annotated dataset specifically for T2V hallucinations with categorized annotations from ten publicly available models.The dataset is presented as a resource for developing and testing methods to detect and reduce T2V hallucinations.Source: §2 · p. 3

3 Dataset Construction

ViBe was built from diverse MS COCO prompts applied to ten open-source text-to-video models, with generated videos manually classified using a five-category hallucination taxonomy. The section describes the dataset’s scope, annotation process, and inter-annotator agreement.

3.1 Dataset Prompt Diversity

  • 837 diverse MS COCO captions anchor ViBe’s thematic coverage, while the structured prompts span varied real-world scenarios.The dataset is organized into five thematic categories.Source: §3.1 · pp. 3–4

3.2 Models Used for Dataset Creation

  • Ten T2V models received the caption prompts; most generated one-second videos, while Show1 generated two-second videos.The models represent diverse architectures, sizes, and training paradigms.Source: §3.2 · p. 4

3.3 Hallucination Definitions

  • The taxonomy distinguishes Vanishing Subject, Omission Error, Numeric Variability, Subject Dysmorphia, and Visual Incongruity, based on recurring inconsistencies in generated videos.The categories can overlap but are treated separately because they occur frequently.Source: §3.3 · p. 4

3.4 Human Annotation Details

  • Five annotators assigned each of 3,782 videos its most prominent hallucination type, following a hierarchical classification that prioritizes specific subcategories.Figure 3 represents the classification hierarchy.Source: §3.4 · pp. 4–5

3.5 Implementation Details

  • Video generation used an NVIDIA A100 GPU, while embedding extraction and classifier training used an eight-core CPU system with 32 GB of memory per core.The section also defines per-model annotation duration as the cumulative time spent annotating that model’s videos.Source: §3.5 · p. 5

3.6 Inter-Annotator Scores

  • Cohen’s Kappa was 0.8737 for Visual Incongruity and 0.7474 for Omission Error; MORA had the longest annotation time (1561.5 minutes) and AnimateLightning the shortest (483.75 minutes).Two annotators assessed 100 shared videos against gold-standard annotations.Source: §3.6 · p. 5, Figure 7 · p. 6

4 Classification

ViBe classification uses video embeddings and classical classifiers to establish hallucination-detection baselines. TimeSFormer embeddings generally outperform VideoMAE, with the TimeSFormer–CNN configuration achieving the strongest reported accuracy and F1 score.

4 Classification

  • The benchmark addresses the limited scale of the existing T2V hallucination benchmark, T2VHaluBench, which comprises 50 videos.The authors present ViBe and classification baselines as a resource for further research.Source: §4 · p. 6

4.1 T2V Hallucination Classification

  • VideoMAE and TimeSFormer embeddings serve as features for seven classifiers, including LSTM, Transformer, CNN, GRU, RNN, Random Forest, and SVM.The setup enables comparison across distinct classifier architectures.Source: §4.1 · pp. 6–7

4.2 Experimental Setup

  • The data split was 80% training and 20% testing; classification used aggregated representations of embeddings extracted from individual frames.Adam or AdamW was used as the optimizer.Source: §4.2 · p. 7

4.3 Results and Analysis

  • With VideoMAE embeddings, Random Forest reached 0.331 accuracy, while LSTM achieved the highest F1 score at 0.299.GRU recorded 0.268 accuracy and 0.190 F1 score in that feature set.Source: §4.3 · p. 8
  • 0.345 accuracy and 0.342 F1 score: the TimeSFormer–CNN configuration leads the evaluated combinations, and TimeSFormer embeddings generally outperform VideoMAE.The comparison covers accuracy and F1 score across classifier and embedding combinations.Source: Table 6 · p. 7, §4.3 · p. 8

5 Conclusion and Future Work

ViBe provides a large annotated resource and five-category taxonomy for studying T2V hallucinations. Future work includes broader categorization, longer-video evaluation, automated classification, and RLHF training.

5 Conclusion and Future Work

  • ViBe contains 3,782 manually annotated videos, 75 times the scale of T2VHaluBench, and introduces a five-category hallucination taxonomy.The taxonomy supports systematic analysis and benchmarking of T2V models.Source: §5 · p. 8
  • Future work proposes expanding the taxonomy and evaluating longer-duration videos to study scalability and temporal coherence over extended sequences.Source: §5 · p. 8
  • The authors also propose automated classification to reduce reliance on manual annotation and RLHF training to refine alignment with human preferences.Source: §5 · p. 8

6 Limitations

ViBe’s annotation and video-duration scope constrain what its dataset can represent. The authors identify multi-category annotation and longer videos as directions for addressing these boundaries.

6 Limitations

  • Videos receive a single hallucination category, which may overlook cases where multiple categories overlap.Source: §6 · p. 8
  • The dataset is limited to short videos because of constraints in open-source T2V models and annotation feasibility.Source: §6 · p. 8
  • The authors suggest multi-category annotations and longer videos as future extensions as computational and automatic annotation methods improve.Source: §6 · p. 8

7 Ethics Statement

The authors frame the benchmark’s development around transparency and accountability while acknowledging possible misuse of realistic doctored videos. They emphasize responsible disclosure and detection and mitigation uses.

7 Ethics Statement

  • The research aims to advance understanding and evaluation of generative models while ensuring transparency and accountability in their development.Source: §7 · p. 9
  • The authors identify potential misuse in creating realistic doctored videos that could contribute to misinformation, fraud, or manipulation.Source: §7 · p. 9
  • Their stated mitigations include responsible disclosure, promoting detection and mitigation efforts, and advocating ethical AI development.Source: §7 · p. 9

A Appendix

The appendix provides supplementary examples and implementation details, then describes the annotation task and its five predefined hallucination categories.

A Appendix

  • The appendix offers supplementary examples and implementation details to aid understanding of the paper.Source: A Appendix · p. 11

B Annotation Details

  • The annotation task is to detect and classify hallucinations in T2V-generated videos, supporting assessment of prompt adherence and hallucination mitigation.Source: B Annotation Details · p. 11
  • Annotators are trained to recognize five predefined categories: Vanishing Subject, Omission Error, Numeric Variability, Subject Dysmorphia, and Visual Incongruity.Source: B Annotation Details · p. 11

2 Training and Evaluation Protocol

The protocol trains and tests annotators before annotation, then has them compare each video with its prompt and label the most prominent hallucination. Annotation took an average of 2.25 seconds per video.

2 Training and Evaluation Protocol

  • Annotators first receive example videos and category justifications, then must correctly classify at least 3 of 5 test videos to proceed.Those below the 60% agreement threshold receive targeted feedback and additional training.Source: §2 · p. 11

3 Annotation Process

  • Annotators review each video against its prompt and assign the most prominent hallucination category, choosing the most visible one when several occur.Source: §3 · p. 11
  • The selected category is entered in the annotation tool, with supplementary notes available for clarification.Source: §3 · p. 11
  • 2.25 seconds per video was the recorded average annotation time.Source: §3 · p. 11

C Dataset

This section builds on the previously defined hallucination categories and their examples by presenting additional illustrations.

C Dataset

  • The five hallucination categories were defined before this section.Source: C Dataset · p. 11
  • Examples for each category had already been provided.Source: C Dataset · p. 11
  • This section presents additional examples to illustrate the categories further.Source: C Dataset · p. 11

2. Omission Error (OE): See figs. 14 and 15

The examples show omissions and other prompt-video inconsistencies, including disappearing subjects, count changes, distorted subjects, and visually incongruous scenes. The annotation tool records a selected category and optional notes.

5. Visual Incongruity (VI): See figs. 20 and 21

2. Omission Error (OE): See figs. 14 and 15

5. Visual Incongruity (VI): See figs. 20 and 21

Loading 2411.10867v2…