PaperlayerView exact revision on arXiv

Source-linked AI summary

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Liqiang Jing, Ruosen Li, Yunmo Chen, Xinya Du

arXiv:2311.01477v2cs.CV

TL;DR

LVLM hallucinations can produce image-inconsistent descriptions, while existing evaluations have limited coverage of free-form answers and fine-grained visual facts. FAITHSCORE identifies descriptive sub-sentences, decomposes them into atomic facts, and verifies those facts against the image. Experiments report that current LVLMs remain prone to image-unfaithful hallucinations.

  • Problem

    Existing LVLM hallucination evaluations have limited coverage of free-form answers and fine-grained visual attributes, while image-inconsistent descriptions can create misinformation and safety concerns.

    Source: §1 · p. 2

  • Method

    FAITHSCORE identifies descriptive sub-sentences in an answer, decomposes them into atomic facts, and verifies their consistency with the input image.

    Source: §1 · p. 2, §3.2 · pp. 4–5

  • Results

    Experiments across six open-source models and diverse tasks and datasets find that hallucination remains a challenge for current LVLMs.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Faithfulness varies with answer length, object count, and atomic-fact type, dimensions the authors identify in their analysis of LVLMs.

    Source: §6 · p. 8

    FAITHSCORE measures factual precision but not factual recall, so it does not penalize answers for omitting facts.

    Source: Limitations · p. 9

Abstract

from arXiv · show

We introduce FaithScore (Faithfulness to Atomic Image Facts Score), a reference-free and fine-grained evaluation metric that measures the faithfulness of the generated free-form answers from large vision-language models (LVLMs). The FaithScore evaluation first identifies sub-sentences containing descriptive statements that need to be verified, then extracts a comprehensive list of atomic facts from these sub-sentences, and finally conducts consistency verification between fine-grained atomic facts and the input image. Meta-evaluation demonstrates that our metric highly correlates with human judgments of faithfulness. We collect two benchmark datasets (i.e. LLaVA-1k and MSCOCO-Cap) for evaluating LVLMs instruction-following hallucinations. We measure hallucinations in state-of-the-art LVLMs with FaithScore on the datasets. Results reveal that current systems are prone to generate hallucinated content unfaithful to the image, which leaves room for future improvements. We hope our metric FaithScore can help evaluate future LVLMs in terms of faithfulness and provide insightful advice for enhancing LVLMs' faithfulness.

1 Introduction

LVLM hallucinations include image-inconsistent descriptions, while existing evaluations have limited coverage of free-form answers and fine-grained visual attributes. FAITHSCORE evaluates descriptive content through atomic facts and image verification; experiments find that current LVLMs still produce unfaithful answers.

1 Introduction

  • Existing evaluations focus on image captioning and often overlook fine-grained hallucinations such as counts, colors, and object relations.Free-form answers can mix image descriptions with analytical content, making it important to identify which statements require image verification.Source: §1 · p. 2
  • FAITHSCORE identifies descriptive sub-sentences, decomposes them into atomic facts, and checks their consistency with the image using a Visual Entailment Model.The metric is intended for image captioning and open-ended questions, including free-form answers.Source: §1 · p. 2
  • The authors report systematic evaluation of free-form answers and fine-grained hallucinations across diverse tasks, and release their code.They describe this as a study of six open-source models.Source: §1 · pp. 2–3
  • Experiments with six open-source models across diverse tasks and datasets find that hallucination remains a challenge for current LVLMs.The authors also conclude that current systems generate answers that are not faithful to the input image.Source: §1 · pp. 2–3

2 Related Work

Prior LVLM hallucination evaluations largely target constrained captioning settings, with metrics and benchmarks addressing object hallucination or more detailed captions. FAITHSCORE is presented as a reference-free metric for fine-grained evaluation of lengthy, free-form answers to open-ended visual questions.

2 Related Work

  • Earlier methods mainly address constrained settings such as image captioning, including object-hallucination metrics and caption-focused evaluations.CHAIR evaluates object hallucination in sentence-level captions; POPE probes objects, NOPE uses VQA, and Gunjal et al. study detailed image captions.Source: §2 · p. 3
  • These evaluation settings leave open-ended, free-form visual question answering relatively unaddressed, despite its lengthy answers and broader question types.The paper contrasts its target setting with existing work focused on image captioning.Source: §2 · p. 3
  • FAITHSCORE is proposed as a reference-free metric for fine-grained evaluation of answers in open-ended visual question answering.The paper identifies counts, colors, and inter-object relations as fine-grained visual elements relevant to hallucination evaluation.Source: §2 · p. 3

3 Estimating FAITHSCORE

FAITHSCORE estimates a reference-free scalar score from an image, prompt, and LVLM answer. Its framework identifies descriptive sub-sentences, decomposes them into atomic facts, and verifies those facts against the image.

3.1 Task and Settings

  • FAITHSCORE is a reference-free scoring function that maps the prompt, image, and generated answer to a scalar between 0.0 and 1.0.The score does not require a ground-truth answer.Source: §3.1 · p. 3

3.2 The Evaluation Framework

  • The framework has three stages—descriptive sub-sentence identification, atomic fact generation, and fact verification—implemented by a Recognizer, Decomposer, and Verifier.The evaluation framework is illustrated in Figure 2.Source: §3.2 · p. 3, Figure 2 · p. 4
  • The Recognizer uses ChatGPT to label answer sub-sentences as descriptive or analytical, then retains the descriptive sub-sentences for evaluation.The answer is split into short units by punctuation; the prompt includes in-context examples.Source: §3.2 · p. 4
  • The Decomposer breaks descriptive sub-sentences into minimal entity, attribute, or relation facts, which the Verifier checks for image support.Atomic facts cover entity existence, attributes such as color and shape, and relations such as spatial relations; the verifier's fact scores are aggregated, with all fact weights set to 1.Source: §3.2 · pp. 4–5

4 Meta-evaluate FAITHSCORE

The meta-evaluation compares FaithScore with human judgments and evaluates its descriptive-sub-sentence recognizer and atomic-fact verifier. It also reports how the human annotations were collected.

4 Meta-evaluate FAITHSCORE

  • Human evaluation used LLaVA-Bench, based on MSCOCO, with 90 questions each for detailed description, conversation, and complex question, and answers from LLaVA and InstructBLIP.The evaluation was conducted to test whether automatic evaluation correlates with human judgment of hallucination.Source: §4 · p. 5

4.2 Recognizer Accuracy on Descriptive Sub-sentence Identification

  • ChatGPT outperformed LLaMA-7B at identifying descriptive sub-sentences; adding context improved LLaMA-7B on LLaVA answers but not significantly overall.The evaluation used 1,382 sub-sentences, averaging 7.68 per answer.Source: Table 1 · p. 5, §4.2 · p. 6

4.1 Human Evaluations of Hallucinations

  • Human annotators labeled answer sub-sentences as descriptive or analytical, decomposed descriptive content into revised atomic facts, and checked each fact against the image.Facts absent from or contradicting the image were marked as hallucinations; faithfulness was rated on a 1–5 Likert scale.Source: §4.1 · pp. 5–6

4.3 Verifier Accuracy on Fact Verification

  • LLaVA-1.5 performed best among the evaluated verifiers, so the authors used it to estimate FAITHSCORE.The evaluation covered 1,380 atomic facts, averaging 2.04 per descriptive sub-sentence; only facts annotated by all three annotators were retained, with final labels determined by majority vote.Source: Table 2 · p. 6, §4.3 · p. 6

4.4 Correlations with Human Evaluations

  • FAITHSCORE achieved the best correlation with human judgments among the compared metrics, with the comparison against baselines significant at p < 0.01.The correlations were measured using Pearson’s r, Spearman’s ρ, and Kendall’s τ for LLaVA and InstructBLIP hallucinations.Source: Table 3 · p. 6, §4.4 · pp. 6–7

5 Evaluating Vision-Language Model Hallucinations with FAITHSCORE

Across six open-source LVLMs and two datasets, FAITHSCORE finds varying faithfulness, with performance also related to answer length, object count, and atomic-fact type. LLaVA-1.5 generally performs well, including at sentence level.

5.1 Models and Datasets

  • Six open-source LVLMs were evaluated on 1,000-image LLaVA-1k and MSCOCO-Cap datasets.LLaVA-1k includes detailed-description, conversation, and complex-question prompts; MSCOCO-Cap evaluates image captioning.Source: §5.1 · p. 7

5.2 Hallucination Evaluation

  • MiniGPT achieved FAITHSCORE scores of 0.5679, 0.5768, and 0.5691 across tasks, respectively; the cited passage reports these as task-specific results.Source: §5.2 · p. 7
  • Most models perform better on MSCOCO-Cap than on LLaVA-1k, where answers are usually shorter.The authors suggest shorter responses may explain the difference.Source: §5.2 · p. 7

5.3 Sentence-level Hallucination Evaluation

  • LLaVA-1.5 performs well on both FAITHSCORE and sentence-level FAITHSCORE, while Multimodal-GPT performs poorly on both.The authors interpret this as consistency between the two evaluations.Source: §5.3 · p. 7, Table 5 · p. 8

5.2 Hallucination Evaluation

  • LLaVA-1.5 outperforms other models in most situations across LLaVA-1k and MSCOCO-Cap.The table compares FAITHSCORE across models and both datasets.Source: §5.2 · p. 7, Table 4 · p. 8

5.4 Other Analysis

  • For most LVLMs, image-captioning faithfulness is higher than faithfulness on other tasks, possibly because captions are brief.Multimodal-GPT generates the longest responses, while mPLUG-Owl tends to generate shorter ones.Source: §5.4 · pp. 7–8
  • InstructBLIP’s FAITHSCORE falls from 0.895 with one object to 0.662 with ten; performance generally drops as answer object counts rise.Figure 4 depicts the relationship between object count and FAITHSCORE on LLaVA-1k.Source: Figure 4 · p. 7, §5.4 · p. 8
  • LLaVA-1.5 excels across most atomic-fact categories, but models’ faithfulness varies by category and consistently high performance remains challenging.The categories are ENTITY, COUNT, COLOR, RELATION, and OTHER.Source: §5.4 · p. 8, Figure 5 · p. 8

6 Conclusion

FAITHSCORE evaluates free-form, open-domain LVLM answers at fine granularity and is more interpretable and more closely aligned with human judgments than previous metrics. Experiments find visual hallucinations remain common and faithfulness varies with answer length, object count, and fact type.

  • FAITHSCORE evaluates free-form, open-domain LVLM answers with greater granularity and interpretability, and closer alignment with human judgments than previous metrics.Source: §6 · p. 8
  • Current LVLMs remain prone to visual hallucinations, with faithfulness varying by answer length, object count, and atomic-fact type.The authors expect FAITHSCORE to be valuable for evaluating future LVLMs.Source: §6 · p. 8

Limitations

FAITHSCORE focuses on factual precision rather than recall, so the authors recommend reporting generated-text length alongside the score. The accompanying material describes human scoring, verifier models, and GPT-4Vision examples.

Limitations

  • FAITHSCORE does not penalize answers for generating fewer facts, leaving precision–recall trade-offs unresolved.The authors recommend reporting average generated-text length and leave a more holistic approach for future work.Source: Limitations · p. 9

A Likert Scale Guideline

  • Human annotators label the number of atomic facts and hallucinations, then assign scores from all hallucinated to none.The guideline defines five Likert levels.Source: A Likert Scale Guideline · p. 12

B Details about VEMs

  • OFA-EM treats neutral entailment judgments as hallucinations; other open-source VEMs receive prompts to judge whether each fact is image-supported.Source: B Details about VEMs · p. 12

C Testing Examples of GPT-4Vision

D More benchmarks

The additional LRV-Instruction benchmark produces results consistent with LLaVA-1k and MSCOCO-Cap. Evaluation examples also show a case where FAITHSCORE diverges from human judgment because of a recognizer error.

D More benchmarks

  • LRV-Instruction FAITHSCOREs were 0.6626 for InstructBLIP, 0.4903 for Multimodal-GPT, 0.6433 for mPLUG-Owl, 0.4638 for MiniGPT-4, 0.7017 for LLaVA, and 0.7855 for LLaVA-1.5.The authors report results consistent with LLaVA-1k and MSCOCO-Cap.Source: D More benchmarks · p. 12

E Examples of Evaluation

  • One example shows FAITHSCORE diverging from human judgment, attributed to an error generated by the recognizer system.The paper also presents three examples of FAITHSCORE computation alongside CLIP-Score.Source: E Examples of Evaluation · p. 12

F More Details about Human Evaluation

Human evaluation on 180 samples provides a comparison point for automatic metrics, and FaithScore has the strongest reported correlation with human judgments. The section also reports annotation agreement and compares alternative evaluation metrics and verifier models.

G Error in Atomic Fact Decomposing

The appendix reports a 2% hallucination ratio in ChatGPT-generated atomic facts, defines recognizer accuracy, and describes the distribution of descriptive and analytical sub-sentences.

G Error in Atomic Fact Decomposing

H Experimental Detail

I Proportions of the descriptive sub-sentences and analytical sub-sentences

  • Sub-sentence distributions differ by question category, and analytical sub-sentences make up nearly half of clauses overall, supporting their identification and exclusion from fact checking.Source: I Proportions of the descriptive… · p. 14

J Cost and Time Analysis

The section contrasts FaithScore’s evaluation speed with manual annotation and illustrates descriptive versus analytical sub-sentences and the prompts used in the pipeline.

J Cost and Time Analysis

  • 4s is the average evaluation time for a long response, versus 212 seconds manually, reducing cost by about 1.2 USD.The authors describe a trade-off between evaluation speed and correlation with human judgments; they report that multiple facts can be verified in parallel.Source: J Cost and Time Analysis · p. 14

K Samples of Description/Analytics Sub-sentence

L Prompts

  • Figures 15 and 16 provide prompts for identifying descriptive sub-sentences and generating atomic facts, respectively.Source: L Prompts · p. 14

M Results on Closed-source Model

GPT-4V and Gemini surpass the evaluated open-source models on the LLaVA-1k benchmark. The accompanying figures illustrate hallucinations, FaithScore computation, and annotation materials.

M Results on Closed-source Model

Generated Answer

Related papers

Loading 2311.01477v2…