Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models

1 INTRODUCTION

• The introduction frames hallucination as image-description discrepancy that raises concerns for LVLM reliability and robustness.It distinguishes discriminative evaluation, which directly asks candidate models to identify hallucinations, from generative evaluation of their produced text.Source: §1 · pp. 1–2
• The framework’s discriminative evaluation asks candidate LVLMs whether templated captions contain a specified hallucination relative to an image.Its generative evaluation trains Hal-Evaluator on a large-scale hallucinatory dataset to assess model descriptions and images without reference captions.Source: §1 · p. 2
• The paper’s findings identify event hallucinations as an overlooked category, report that hallucinations rise with output length, and find that combining evaluation paradigms gives a fuller view of LVLM tendencies.It also reports that Chain-of-Thought helps reduce hallucinations in discriminative evaluation, especially for relations and events.Source: §1 · p. 2
• Hal-Eval introduces event hallucination and a universal, fine-grained framework spanning object, relation, attribute, and event types with discriminative and generative evaluation.Its annotation pipeline uses GPT-4 to generate and filter hallucinatory data for the framework.Source: §1 · p. 2

2 RELATED WORK

• Flamingo, PaLM-E, and BLIP-2 exemplify distinct LVLM design approaches, while long visual sequences remain a constraint for direct visual-feature integration.Flamingo uses gated cross-attention; PaLM-E and related models integrate visual features through linear layers; BLIP-2 uses a Q-former to condense visual sequences.Source: §2 · p. 2
• POPE [24], NOPE [34], and CIEM [16] are discriminative benchmarks focused exclusively on object hallucinations, whereas generative benchmarks also cover attributes and relations.AMBER [48] includes both generative and discriminative tasks.Source: §2 · p. 3
• Related work classifies hallucination benchmarks by discriminative or generative evaluation, with those approaches targeting different scopes of hallucination types.Discriminative benchmarks query object presence against ground truth; generative benchmarks broaden coverage to attribute and relation hallucinations.Source: §2 · p. 3

3 PRELIMINARY: HALLUCINATION IN LVLMS

• Figure 2 separates mPLUG-owl descriptions into those under 20 tokens and those exceeding 20 tokens to show hallucination-type ratios.The paper reports that the share of event hallucinations rises with description length.Source: Figure 2 · p. 3, §3 · p. 3
• Event hallucination describes a nonexistent target around which the LVLM constructs an event involving attributes, relations, and actions.The paper distinguishes it from object hallucination (a nonexistent object), attribute hallucination (incorrect properties of an existing object), and relation hallucination (an incorrect relationship between existing objects).Source: §3 · p. 3
• In an analysis of 5,000 COCO image-caption pairs described by mPLUG-owl and LLaVA, the share of event hallucinations increased as description length increased.GPT-4 inspected ground-truth and generated descriptions and categorized hallucinations as object, attribute, relationship, or event types.Source: §3 · p. 3

4 METHOD

• Hal-Eval combines discriminative and generative evaluation to assess different hallucination types, with Hal-Data supporting evaluation-data construction and generative-evaluator training.The framework includes a fine-grained annotation pipeline and a model for generative hallucination evaluation.Source: §4 · p. 3

4.1 Automatic Fine-grained Hallucination Annotation Pipeline

• GPT-4 generates categorized hallucinations and their locations in captions; filtering targets noisy annotations, with manual verification finding over 97% met the annotation standards.Random sampling found approximately 30% of annotations still contained noise before cleanup.Source: §4.1 · pp. 3–4

4.2.1 Constructing Evaluation Dataset.

• The evaluation dataset uses COCO in-domain images and web-based out-of-domain images, with 5,000 annotations for each split covering hallucination types and content.The split addresses overlap between COCO-based evaluation data and LVLM instruction-finetuning data.Source: §4.2.1 · p. 4

4.2.2 Evaluation Process.

• A unified prompt asks whether a caption accurately reflects the image; varying the caption type tests hallucination categories, with F1 as the primary metric.Accuracy, precision, recall, and “Yes” ratio are also tracked.Source: §4.2.2 · p. 4

4.3.1 Overview .

• Hal-Evaluator is designed to avoid proprietary-model fees and costly reference annotations by evaluating an LVLM description against its image.It can identify hallucination content and categories, and modify hallucinated information to produce an accurate depiction.Source: §4.3.1 · p. 5

4.3.2 Instruction finetuning of Hal-Evaluator.

• Hal-Data contains 130,000 GPT-4-annotated image-text pairs and 2 million pairs generated by a caption model fine-tuned on the smaller curated set.The larger dataset uses LLaMA2 13B to modify captions with hallucinations and annotate them.Source: §4.3.2 · p. 5
• LLaVA 1.5 13B was fine-tuned on 2 million instruction examples from Hal-Data to create Hal-Evaluator.Source: §4.3.2 · p. 5

4.3.3 Generative Evaluation Based in Hal-Evaluator.

• Hal-Evaluator receives an image and an LVLM-generated description, then detects hallucinations, identifies their type and content, and can revise the description.Source: §4.3.3 · p. 5

5.1.1 Main Results.

• Except GPT-4, tested open-source LVLMs performed subpar on discriminative evaluation; LLaVA variants showed stronger hallucination tendencies out-of-domain.POPE showed a tendency toward “yes” responses in most models, while Hal-Eval observed it only in InstructBLIP.Source: §5.1.1 · p. 5, Table 1 · p. 6

5.1.2 Analysis of Discriminative Evaluation.

• COT reduced LLaVA1.5 discriminative hallucinations in both domains, especially those involving relationships and events.The authors suggest COT is particularly effective for these types because they rely on contextual understanding and inference.Source: Figure 4 · p. 6, §5.1.2 · p. 6

5.2.1 Main Results.

• Longer generations increased hallucination content, especially event hallucinations, while GPT-4V accuracy declined as output length increased.The generative evaluation results also report that MiniGPT-4 and InstructBLIP had robust in-domain accuracy, with InstructBLIP reaching 89.1% at approximately 10 tokens.Source: §5.2.1 · p. 6, Table 2 · p. 7

5.2.2 Analysis of Generative Evaluation.

• Hal-Eval-Generative showed a robust positive correlation with human judgments across Pearson’s r, Spearman’s ρ, and Kendall’s τ.The correlation experiment compared the evaluation metrics with human judgments of LVLM faithfulness.Source: Table 3 · p. 7, §5.2.2 · p. 7
• Hal-Evaluator outperformed GPT-4V in hallucination detection and more reliably identified event hallucinations, which GPT-4V struggled to localize.Figure 5 compares their discriminative evaluation results and ROUGE-L overlap with annotated hallucination content.Source: Figure 5 · p. 7, §5.2.2 · pp. 7–8

5.3 Comparative Analysis of Discriminative and Generative Evaluations

• 54.3 average F1 for event types across models; generative evaluation better assesses event hallucinations, while using both modes gives a fuller view.The authors report discriminative evaluation as sufficiently effective for object, attribute, and relation tendencies, citing prior work [24][34][16].Source: Table 4 · p. 7, §5.3 · p. 8

5.4 Utilizing Hal-Data for Supervised Fine-tuning

• Hal-VL, fine-tuned jointly using Hal-Data 130K and LLaVA1.5 instructional data, achieved the best Hal-Eval performance and outperformed LLaVA1.5 on MMHal-Bench.The authors also report advantages on most of the general benchmarks they evaluated.Source: Table 5 · p. 7, §5.4 · p. 8

5.2.2 Analysis of Generative Evaluation.

• Hallucination incidence, particularly event hallucinations, rose with output length, making length control important when comparing generative evaluations.Figure 6 plots hallucination-type proportions and accuracy as output length varies.Source: Figure 6 · p. 7, §5.2.2 · p. 8

6 CONCLUSION

• The framework adds Event Hallucination as a category and uses advanced LLMs for data analysis.The authors describe it as a step toward assessing and improving LVLM performance.Source: §6 · p. 8

A AUTOMATIC FINE-GRAINED HALLUCINATION ANNOTATION PIPLINE (AFHA)

• The AFHA pipeline uses GPT-4 to inject specified hallucinations into captions, generate content labels, and filter noisy annotations.Manual verification found that over 97% of the cleaned data met the annotation standards.Source: A AUTOMATIC FINE-GRAINED HALLUCINATION… · p. 11

• Table 7 presents the source proportions for Hal-Data 130K and Hal-Data 2M.The supplied passage does not state the individual proportions.Source: Table 7 · p. 11

B.1 Impact of SFT Data Scales for Hal-Evaluator

• 2M-scale SFT data yielded the highest Hal-Evaluator accuracy for identifying hallucinations on Hal-Eval-Dis.The study increased the fine-tuning data scale from 130K to 2M.Source: §B.1 · p. 11

B.2 Data source

• Hal-Data 130K draws on COCO, Conceptual Captions, SBU, LAION, and Share-GPT4.Source: §B.2 · p. 11
• The 2-million-entry Hal-Data expansion prioritizes 558,000 LLaVA data points and obtains the remaining 1,400K entries from LAION, Conceptual Captions, and SBU.Source: §B.2 · p. 11

C.1 Training Setting for Hal-Evaluator and Hal-VL

• Training follows LLaVA 1.5’s pre-training data and hyperparameters, using 16 NVIDIA A100 GPUs and DeepSpeed with a batch size of 64 per GPU.The cited passage names LLaVA 1.5 [29] and DeepSpeed [39].Source: §C.1 · p. 11

C.2 Setting for GPT-4 Annotation of AFHA

• GPT-4 annotation uses one prompt template across hallucination types, changing only the type-specific definition to flag image-inconsistent objects, events, or relationships.This standardizes annotations for evaluating the frequency and nature of hallucinations in LVLM descriptions.Source: §C.2 · p. 11

C.3 Setting for GPT-4 Filtering of AFHA

• A GPT-4 filtering prompt removes annotations that fail the dataset’s objective criteria, treating those labels as noise.The filtering step is intended to retain annotations for subsequent analyses.Source: §C.3 · p. 11

C.4 Experiment Setting of Chain-of-Thought for Discriminative Evaluation.

• The Chain-of-Thought experiment modifies the discriminative prompt by asking for step-by-step analysis of the image and caption before answering.The prompt asks whether the caption accurately reflects the image and specifies hallucination categories.Source: §C.4 · pp. 11–12
• Figure 7 depicts the AFHA pipeline, its object-hallucination annotation prompt, and examples spanning four hallucination types.The figure caption directs readers to Appendix C.2 for prompts for other annotation types.Source: Figure 7 · p. 12
• Table 8 compares hallucinatory datasets with Hal-Data, including image visibility during captioning and average caption character count.“LCS” denotes LAION [41], CC [6], and SBU [37].Source: Table 8 · p. 12

D.1 More result of Hal-VL.

• Hal-VL shows significant advantages on the vast majority of the additional benchmarks discussed, supporting reported robustness across tasks and datasets.The passage refers to Table 6, but its values are not supplied here.Source: §D.1 · p. 12

D.2 Analysis of Different Types of Hallucinations Based on GPT-4

• Event hallucinations increase significantly as description length grows in GPT-4’s categorization of model-generated captions.The analysis used 5,000 COCO image-caption pairs and considered object, attribute, relationship, and event hallucinations.Source: §D.2 · pp. 12–13

D.3 Impact of SFT Data Scales for Hal-Evaluator

• 2M-scale SFT data achieves the highest accuracy for identifying hallucinations on Hal-Eval-Dis among the evaluated data scales.The experiment enlarged the SFT data from 130K to 2M.Source: §D.3 · p. 13

D.4 Effectiveness of Hall-evaluator for Mitigating Generative Hallucination.

• Hal-Evaluator detects and rectifies hallucinated content in MiniGPT-4 and LLaVA outputs, and the corrected outputs are reevaluated on MMHal-Bench [43].The authors report that this process can effectively eliminate model hallucinations.Source: Table 9 · p. 13, §D.4 · p. 14

D.5 Effectiveness of Hall-evaluator for Mitigating Generative Hallucination.

• The repeated mitigation experiment likewise reports that Hal-Evaluator effectively eliminates hallucinations after correcting MiniGPT-4 and LLaVA outputs evaluated with MMHal-Bench [43].Table 9 is identified as the evaluation-results table for different MLLMs on MMHal-Bench.Source: Table 9 · p. 13, §D.5 · p. 14

D.6 Compared with Other Hallucination Benchmarks

• The benchmark comparison identifies Hal-Eval as providing the most extensive coverage of hallucination evaluation among the compared methods.Table 14 is captioned as a comparison of LVLM hallucination-evaluation benchmarks.Source: §D.6 · p. 14, Table 14 · p. 14

E ANNOTATION PORTAL

• The annotation portal has annotators mark sentence spans as object, attribute, relation, or event hallucinations in image descriptions.Annotators receive an image and a language-model-generated detailed description.Source: E ANNOTATION PORTAL · p. 14

E.1 Annotation Agreement

• The annotators are internal organization members with NLP and multimodal-field-study experience, and GPT-4 prompts are designed to discard labels inconsistent with the criteria.The annotators consented to research use and public release of their annotation information.Source: §E.1 · p. 14, Figure 10 · p. 16

D.7 Demo for Hal-Evaluator

• Figure 12 presents Hal-Evaluator responses to different hallucination descriptions as a demonstration of its behavior.The text frames the visualization as an intuitive view of how the model responds to different hallucination types.Source: §D.7 · p. 14, Figure 12 · p. 18