PaperlayerView exact revision on arXiv

Source-linked AI summary

Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models

Holy Lovenia, Wenliang Dai, Samuel Cahyawijaya, Ziwei Ji, Pascale Fung

arXiv:2310.05338v2cs.CVcs.CL

TL;DR

Object hallucination in VL models lacks a general measurement, despite responses sometimes including objects absent from images. This paper introduces NOPE, a VQA benchmark supported by LLM-generated NegP data, and finds that no tested VL model is exempt from hallucination. The analysis also identifies question and scene characteristics associated with greater susceptibility.

  • Problem

    Object hallucination remains difficult to assess because existing approaches face generalization and scalability challenges, and NegP cases are scarce in VQA datasets.

    Source: §1 · p. 2

  • Method

    NOPE assesses object hallucination through VQA, using a multi-turn LLM pipeline to generate NegP questions from image captions and evaluating 10 VL baselines.

    Source: §1 · p. 2

  • Results

    No VL model is exempt from object hallucination, and lexical diversity, question type, and object relevance to the scene are identified as influential factors in susceptibility.

    Source: §7 · p. 9

  • Takeaways & Limitations

    The findings identify lexical diversity, question type, and scene-relevant objects as factors associated with VL models’ susceptibility to object hallucination.

    Source: §7 · p. 9

    The measurement is limited to NegP cases; for incorrect answers on Others VQA, the boundary between misclassification and object hallucination is undefined.

    Source: Evaluation Metrics for Object… · p. 9

Abstract

from arXiv · show

Object hallucination poses a significant challenge in vision-language (VL) models, often leading to the generation of nonsensical or unfaithful responses with non-existent objects. However, the absence of a general measurement for evaluating object hallucination in VL models has hindered our understanding and ability to mitigate this issue. In this work, we present NOPE (Negative Object Presence Evaluation), a novel benchmark designed to assess object hallucination in VL models through visual question answering (VQA). We propose a cost-effective and scalable approach utilizing large language models to generate 29.5k synthetic negative pronoun (NegP) data of high quality for NOPE. We extensively investigate the performance of 10 state-of-the-art VL models in discerning the non-existence of objects in visual questions, where the ground truth answers are denoted as NegP (e.g., "none"). Additionally, we evaluate their standard performance on visual questions on 9 other VQA datasets. Through our experiments, we demonstrate that no VL model is immune to the vulnerability of object hallucination, as all models achieve accuracy below 10\% on NegP. Furthermore, we uncover that lexically diverse visual questions, question types with large scopes, and scene-relevant objects capitalize the risk of object hallucination in VL models.

1 Introduction

Object hallucination occurs when VL responses include objects absent from an image, and existing evaluation approaches face generalization and scalability challenges. The paper introduces NOPE and a NegP data-generation pipeline to measure this behavior through VQA.

1 Introduction

  • Object hallucination is an unfaithful inclusion of non-existent objects and is associated with over-reliance on unimodal priors and statistical bias.The paper connects these underlying problems to difficulty understanding non-existence.Source: §1 · p. 1
  • Existing CHAIR evaluation relies on handcrafted parsing and predefined object categories, requiring new category lists for other datasets.The lists vary in granularity across studies.Source: §1 · p. 2
  • NOPE measures object hallucination in VQA by treating responses that include non-existent objects on questions with NegP ground truth as hallucinations.It distinguishes these from incorrect answers to questions whose ground truth is not NegP.Source: §1 · p. 2
  • The benchmark evaluates 10 VL baselines on near-balanced NegP and Others data, with approximately 30k development and 36k test examples.The authors also analyze baseline performance and limitations.Source: §1 · p. 2
  • The paper proposes a multi-turn LLM pipeline to generate NegP VQA data from image captions, with list-then-rewrite producing 92% validity.The generated data is checked using automatic and human validation.Source: §1 · p. 2

3. Through extensive analysis in NOPE, we find

The analysis identifies question and object characteristics associated with greater object hallucination in VL models.

3. Through extensive analysis in NOPE, we find

  • VL models tend to hallucinate more on questions with higher lexical diversity, larger answer scopes, and objects more relevant to the scene.Source: §3 · p. 2

2 Related Work

Prior vision-language hallucination research largely centers on image captioning, while earlier automatic VQA generation work focuses on Others rather than NegP questions. The paper positions its NegP data generation as addressing that gap.

2.1 Hallucination in Vision-Language

  • Prior hallucination research in vision-language has mostly focused on image captioning, including CHAIR and concerns that standard captioning metrics under-penalize hallucination.Concurrent work frames hallucination evaluation as binary-class VQA with yes/no answers.Source: §2.1 · p. 2

2.2 Question Generation for VQA Data

  • Human annotation is commonly used to ensure visual-question quality, but its cost motivates scalable automatic VQA data generation.Source: §2.2 · p. 2
  • Only about 0.4% of existing VQA datasets consist of NegP examples, which the paper says are insufficient for assessing object hallucination.Source: Figure 2 · p. 3
  • Prior automatic VQA-generation studies focus on Others data, whereas this work targets NegP VQA data.Source: §2.2 · p. 3

3 NOPE to Overcome Limited NegP

NOPE addresses the scarcity of negative-answer VQA data by generating NegP questions from image captions and validating their quality. List-then-rewrite produces substantially more human-valid questions than generate-from-scratch, while the dataset also varies question phrasing and object-scene relevance.

3 NOPE to Overcome Limited NegP

  • Only ∼0.4% of existing VQA data are NegP, motivating NOPE’s synthetic questions whose ground-truth answers indicate object absence.NOPE denotes negative answers with terms including “none,” “nothing,” “no one,” and “neither.”Source: §3 · p. 3

3.1 Prompting Methodology

  • Generation starts from image captions: generate-from-scratch conditions on question type and NegP answer, while list-then-rewrite lists absent related objects and rewrites varied templates.The two prompting methods use caption text and object annotations as inputs.Source: §3.1 · p. 3

3.2 Human Evaluation Guidelines

  • Three annotators judge question validity, possible “Others” answers, answer mismatches, and clarity using five judgment categories.The human evaluation examines generated questions alongside the visual context, caption, and ground-truth answer.Source: §3.2 · pp. 3–4

3.3 Results and Quality Analysis

  • ±92% of list-then-rewrite questions were judged valid by human annotators, and this method produced substantially better questions than generate-from-scratch.The human evaluation results compare the two methods across the judgment categories defined in §3.2.Source: Figure 3 · p. 3, §3.3 · p. 4
  • 12% of generate-from-scratch questions included an existing object, while 14% omitted objects and were overly generic.The authors use these errors to argue that the method does not reliably perform the implicit task breakdown required for question generation.Source: §3.3 · p. 4
  • List-then-rewrite generated 29.5k NegP VQA examples from OpenImagesV7 for the NOPE dataset.The authors describe generate-from-scratch as unreliable for this task.Source: §3.3 · p. 5

3.4 Dataset Statistics

  • NOPE questions use diverse starting phrases, including openings beyond standard interrogatives, addressing models’ brittleness to linguistic variation.The paper relates NegP performance negatively to lexical-diversity measures.Source: §3.4 · p. 5
  • NOPE’s relevance categories connect object-caption text similarity with image-text similarity, measured using Sentence-Transformer and CLIPScore.The categories are based on descriptors used in the object-listing step.Source: §3.4 · p. 5

4 Experimental Settings

The benchmark combines balanced NegP and Others data across 10 VQA corpora, including NOPE, to assess object hallucination alongside standard incorrectness. Its settings describe dataset quality, model baselines, and evaluation metrics.

4 Experimental Settings

  • The benchmark covers validation and test sets from 10 VQA corpora, including NOPE, with balanced object-scene relevance proportions.It compares object hallucination on NegP data with incorrectness on Others data across 10 VL models.Source: §4 · p. 5

4.1 Datasets

  • ∼30k dev and ∼36k test examples have near-balanced NegP and Others proportions; NOPE and TDIUC show lexical diversity and fluency comparable to human-generated datasets.Figure 6 presents the question-quality comparison.Source: §4.1 · p. 5, Figure 6 · p. 6

4.2 Baselines

  • The baselines include five fine-tuned models—OFA, BLIP, ALBEF, and GIT—and zero-shot models BLIP-2, PromptCap, and InstructBLIP, plus two-shot OpenFlamingo.The supplied passages describe model approaches and identify Table 2 as the baseline-model inventory.Source: §4.2 · p. 6, Table 2 · p. 6

4.3 Evaluation Settings

  • Evaluation uses exact-match accuracy and METEOR for NegP and Others, plus rule-based NegP accuracy for outputs that are negative indefinite pronouns.Scores are computed per task and weighted by task size.Source: §4.3 · p. 6

5 Results

Across the test benchmark, zero-shot and VQA-fine-tuned baselines perform substantially worse on NegP than on Others, while GIT leads both settings. NegP performance also reveals hallucination risk that standard Others-only evaluation misses.

5 Results

  • ±22% and ±18% accuracy are the averaged NegP-versus-Others discrepancies for zero-shot and VQA-fine-tuned baselines, respectively; all perform worse on NegP.Table 3 reports weighted test-set model performances; the prose notes that fine-tuned baselines are slightly better on NegP and comparable on Others to zero-shot and few-shot baselines.Source: §5 · p. 6, Table 3 · p. 7
  • Lower incorrectness on Others does not necessarily mean less object hallucination: PromptCapBASE, PromptCap, and BLIP score lower on NegP than ALBEF despite outperforming it on Others.Thus, evaluations using only Others cases cannot effectively capture object-hallucination risk.Source: §5 · p. 7
  • GIT outperforms the other baselines on both NegP and Others and surpasses much larger models despite being the smallest baseline.The authors suggest its substantially larger number of pre-training images could account for this performance.Source: §5 · p. 7

6 Analysis and Discussions

The analyses associate greater object-hallucination vulnerability with lexical diversity, question type, answer scope, and object-scene relevance. These patterns indicate that language bias and scene relevance do not reliably establish visual grounding.

6.1 Object hallucination and lexical diversity

  • NegP performance is lower on NOPE than existing datasets, and performance has strong negative Pearson correlations with lexical-diversity measures, including r = −0.8 for METEOR and HDD.The authors attribute NOPE's greater difficulty mainly to its relatively higher lexical diversity; Figure 7 also shows lower NegP than Others accuracy across baselines.Source: §6.1 · p. 7, Figure 7 · p. 7

6.2 Object hallucination and language bias

  • All fine-tuned models fail on NegP color, object, and location questions, while most models hallucinate less on counting and person questions; Figure 9 compares fine-tuned performance across question types.The authors connect the latter pattern to narrower answer scopes in training, such as numbers ≤5 and generic person labels, and caution that language bias does not solve hallucination.Source: §6.2 · pp. 7–8, Figure 9 · p. 8

6.3 Object hallucination and object-scene relevance

  • Fine-tuned models perform worse when absent objects are closely scene-related than when they are loosely related or unrelated; zero-shot and few-shot differences are less apparent.NegP scores are generally very low for zero-shot models except BLIP-2; Figure 10 separates performance by object-scene relevance.Source: §6.3 · p. 8, Figure 10 · p. 8

7 Conclusion

The paper introduces NOPE as a VQA benchmark for assessing object hallucination and presents scalable generation of synthetic NegP data. Its experiments find vulnerability across evaluated models and identify factors associated with that vulnerability.

7 Conclusion

  • NOPE addresses the lack of a general measure of object hallucination by testing whether 10 state-of-the-art VL models discern object absence in visual questions alongside their standard performance.The paper also presents a cost-effective, scalable method for generating synthetic data with over 90% validity.Source: §7 · p. 8
  • No evaluated VL model is exempt from object hallucination, and lexical diversity, question type, and object-scene relevance are identified as influential factors.The authors state these findings provide insights into assessment and may inform future development of enhanced VL models.Source: §7 · p. 9

8 Limitation and Future Work

The paper identifies limits in both its evaluation metrics and the scope of its hallucination measure. Its Others evaluation uses balanced subsets rather than full source-dataset test sets.

8 Limitation and Future Work

  • Exact match accuracy, METEOR, and NegP accuracy can score semantically correct equivalent answers as incorrect.For example, “There is no spoon in the picture” receives zero despite the label “Nowhere.”Source: Evaluation Metrics for Object… · p. 9
  • The hallucination measure covers NegP cases; the boundary between misclassification and hallucination for incorrect Others answers remains undefined.Source: Evaluation Metrics for Object… · p. 9
  • The balanced Others dev and test subsets do not provide full per-dataset performance, which requires rerunning baselines on each source dataset’s full test set.Source: Evaluation Metrics for Object… · p. 9

9 Ethics Statement

The ethics statement frames the work around improving VL-model reliability and faithfulness while recognizing risks and commitments to responsible development. The appendices document prompts, validation procedures, implementation, and examples.

9 Ethics Statement

  • The authors connect VL-model reliability and faithfulness to applications including healthcare and autonomous driving, and acknowledge possible bias and unintended consequences.Source: §9 · p. 9
  • The authors state that synthetic data was generated with privacy and bias concerns in mind, and commit to transparency, accountability, open sharing, and ongoing impact monitoring.Source: §9 · p. 9

A Prompt Templates

B Reference Question Templates

C Automatic Validation Methodologies of NegP VQA Data Generation

  • Generate-from-scratch questions are checked against captions with an NLI model and against visual context with an image-QA classifier; list-then-rewrite validation checks object absence and question consistency.Source: C Automatic Validation Methodologies… · p. 17

D Implementation Details of NegP VQA Data Generation

D.1 Image-QA Pair Classification

  • The image-QA classifier is trained on balanced valid and invalid pairs drawn from nine VQA datasets, using a frozen CLIP feature encoder and a one-hidden-layer classifier.It achieves an F1-score of 91.29% on the test set.Source: §D.1 · pp. 17–18

E Human Evaluation Category Examples

F Automatic Validation Results of NegP VQA Data Generation

List-then-rewrite produced high-quality NegP questions, while generate-from-scratch had low automatic validity and substantial disagreement between automatic and human judgments. The appendices also show question-diversity examples, baseline variants, and object-hallucination examples.

F Automatic Validation Results of NegP VQA Data Generation

G Question Diversity of Existing VQA Datasets

H Baselines in NOPE Benchmark

I Examples of Object Hallucination in NOPE

Loading 2310.05338v2…