Source-linked AI summary
Evaluating Object Hallucination in Large Vision-Language Models
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, Ji-Rong Wen
TL;DR
LVLMs may generate objects inconsistent with their images, motivating a systematic study of object hallucination and its evaluation. The paper evaluates representative models, analyzes instruction-related patterns, and proposes POPE, which provides more stable and flexible hallucination evaluation.
Problem
LVLMs suffer from object hallucination, but its prevalence, instructional influences, and reliable evaluation require systematic study.
Method
The paper evaluates representative LVLMs with CHAIR, analyzes object frequency and co-occurrence effects, and proposes the polling-based POPE method.
Results
Most evaluated LVLMs severely hallucinate objects, while POPE evaluates hallucination more stably and flexibly than existing methods.
Takeaways & Limitations
Visual instruction object distributions affect LVLM hallucination, so hallucination evaluation should account for instruction and generation effects.
Takeaways & Limitations
The study focuses only on object hallucination and evaluates models on parts of validation sets, so scores do not measure overall LVLM capacity and may reflect data distribution.
Abstract
from arXiv · showhide
Inspired by the superior language abilities of large language models (LLM), large vision-language models (LVLM) have been recently explored by integrating powerful LLMs for improving the performance on complex multimodal tasks. Despite the promising progress on LVLMs, we find that LVLMs suffer from the hallucination problem, i.e. they tend to generate objects that are inconsistent with the target images in the descriptions. To investigate it, this work presents the first systematic study on object hallucination of LVLMs. We conduct the evaluation experiments on several representative LVLMs, and show that they mostly suffer from severe object hallucination issue. We further discuss that the visual instructions may influence the hallucination, and find that: objects that frequently occur in the visual instructions or co-occur with the image objects, are obviously prone to be hallucinated by LVLMs. Besides, we find that existing evaluation methods might be affected by the input instructions and generation styles of LVLMs. Thus, we further design an improved evaluation method for object hallucination by proposing a polling-based query method called POPE. Experiment results demonstrate that our POPE can evaluate the object hallucination in a more stable and flexible way. Our codes and data are publicly available at https://github.com/RUCAIBox/POPE.
1 Introduction
This paper systematically studies object hallucination in LVLMs, finding that these models generate image-inconsistent objects and that evaluation can be improved with POPE.
- Motivation: LVLMs inherit hallucination problems from their language and vision-language components, generating objects inconsistent with ground-truth image objects.The paper frames object hallucination as a risk to LVLM reliability and user experience.
- Empirical study: Most evaluated LVLMs severely hallucinate objects, sometimes more than smaller vision-language models.The preliminary study uses CHAIR on representative LVLMs and MSCOCO images.
- Observed causes: Objects frequent in or co-occurring with visual instruction data are more likely to be hallucinated.The paper links this pattern to object sampling strategies and instruction distributions.
- POPE: POPE converts hallucination evaluation into binary Yes-or-No object-probing questions, making assessment more stable and flexible.The method is presented as an alternative to instruction-based caption evaluation.
2 Background
LVLMs combine visual encoders, LLM-based language encoders, and cross-modal alignment to support multimodal understanding, but they can hallucinate objects absent from target images.
- LVLMs: LVLMs enhance vision-language pre-trained models by integrating powerful large language models.They are referred to here as large vision-language models.
- Architecture and training: An LVLM typically contains a vision encoder, an LLM-based language encoder, and a cross-modal alignment network.Training includes unimodal pre-training, image-text alignment, and image-text instruction fine-tuning.
- Capabilities: After alignment, LVLMs can represent visual objects, interpret their linguistic semantics, and reason over related concepts.These capabilities support varied multimodal tasks.
- Object hallucination: Object hallucination occurs when generated descriptions contain objects inconsistent with or absent from the target image.This work focuses on coarse-grained object hallucination rather than finer-grained attributes or characteristics.
- Motivation: Object hallucination can hinder safe real-world deployment by causing unexpected consequences, including incorrect autonomous-driving decisions.The paper studies the issue from an evaluation perspective.
3 Object Hallucination in LVLMs
The paper evaluates LVLM object hallucination with CHAIR, finds that results depend on instructions and caption generation, and motivates a more stable evaluation approach.
- Evaluation settings: CHAIR measures the proportion of caption objects that do not appear in the image, at object-instance and sentence levels.The two variants are CHAIRI and CHAIRS.
- Evaluation settings: Five LVLMs are prompted with two short image-description instructions and evaluated on generated MSCOCO captions using CHAIR.The instructions request either a short caption or a brief image description.
- Evaluation results: 32.7 versus 13.0 on CHAIRS using Instruction 1: LLaVA hallucinates more than OSCARbase in the reported comparison.Most instruction-tuned LVLMs show serious object hallucination; InstructBLIP hallucinates less than the other evaluated LVLMs.
- Qualitative analysis: Frequently appearing or co-occurring objects show decreasing hallucination times as their occurrence frequency decreases.The pattern is reported across MiniGPT-4, LLaVA, Multimodal-GPT, and mPLUG-Owl.
- Evaluation limitations: CHAIR results can change substantially with instruction wording and caption length, while handcrafted parsing may misclassify LVLM outputs.The paper therefore calls for a more stable and convenient evaluation method.
4 Influence of Instruction Data on Object Hallucination
The study examines whether visual instruction data shapes LVLM object hallucination through frequent objects and object co-occurrence. Qualitative and quantitative analyses support both hypotheses and motivate sampling strategies for evaluation.
- 4.1 Hypotheses: The study hypothesizes that visual instruction data makes LVLMs prone to hallucinate frequently appearing and co-occurring objects.These hypotheses are motivated by the unbalanced object distribution and recurring object groups in instruction datasets.
- 4.1 Hypotheses: Visual instruction datasets built largely from MSCOCO share an unbalanced distribution dominated by frequent objects.Fine-tuning on these datasets may make LVLMs more likely to generate or hallucinate those objects.
- 4.1 Hypotheses: Frequently co-occurring objects may be elicited by objects already present in an image, such as laptops eliciting mice and keyboards.The paper tests whether co-occurrence patterns in instruction data are associated with hallucination.
- 4.2 Qualitative Analysis: Across four LVLMs, hallucination times decreased as object occurrence frequency decreased, supporting a link between frequent objects and hallucination.The analysis covers MiniGPT-4, LLaVA, MultiModal-GPT, and mPLUG-Owl using top-frequency object bar charts.
- 4.3 Quantitative Analysis: The study measures consistency between object frequency and hallucination using top-k hit ratios for appearing and co-occurring objects.HRA and HRC quantify these two correlations, respectively.
- 4.3 Quantitative Analysis: HR@k represents the proportion of top-k frequently appearing or co-occurring objects among all hallucinated objects.The measure counts relevant hallucinated objects within each image and aggregates their relationship to frequency patterns.
- 4.3 Quantitative Analysis: HRA@10 values were near 0.5 and HRC@10(dining table) values near 0.6 across LVLMs.Approximately half of hallucinated objects were among the top 10 appearing objects, while more than half were among the top 10 objects co-occurring with dining tables.
- 4.3 Quantitative Analysis: The proportions increased when the analysis expanded from the top 10 to the top 30 objects, reinforcing the prevalence of common-object hallucinations.These findings motivate three object-sampling strategies in the evaluation pipeline.
5 POPE
POPE evaluates object hallucination by converting it into balanced Yes-or-No object-probing questions, using negative sampling to test different hallucination tendencies. Across MSCOCO experiments, it provides stable, scalable, and caption-consistent evaluation.
- Overview of POPE: POPE converts object-hallucination evaluation into a binary classification task using Yes-or-No questions about image objects.Each evaluation triple contains an image, multiple probing questions, and their standard answers.
- Overview of POPE: POPE builds balanced evaluation triples by combining ground-truth objects with equally many nonexistent objects obtained from annotations or automatic segmentation.The nonexistent objects are sampled under random, popular, or adversarial strategies.
- Evaluation on MSCOCO: On MSCOCO, InstructBLIP performs best, while LLaVA, MultiModal-GPT, and mPLUG-Owl have F1 Score values below 70.The evaluation uses 500 validation images with more than three annotated objects and six questions per image.
- Evaluation on MSCOCO: LLaVA, MultiModal-GPT, and mPLUG-Owl answer “Yes” near 99%, indicating overconfidence and reduced accuracy on questions whose correct answer is “No”.Performance consistently decreases from random to popular and adversarial sampling settings.
- Advantages of POPE: POPE is more stable across prompt forms than CHAIR, with significantly lower F1-score standard deviation under different prompts.Its closed-ended responses reduce ambiguity, and the method can also be extended to unannotated datasets using automatic segmentation.
- Advantages of POPE: POPE responses are evaluated for consistency with generated captions, linking “No” responses to rare caption mentions and frequent caption mentions to “Yes” responses.The consistency analysis examines InstructBLIP and MiniGPT-4 under the reported evaluation settings.
6 Conclusion
The paper finds that LVLMs exhibit object hallucination and that visual-instruction object distributions influence this behavior. It proposes POPE to address instruction- and generation-style sensitivity, with experiments showing more reliable hallucination evaluation.
- Conclusion: Experiments show that visual-instruction object distributions affect LVLM object hallucination, while existing methods can be affected by instructions and generated text.The paper reports that these factors can make existing evaluation results less reliable.
- Conclusion: POPE is a polling-based query method proposed to improve object-hallucination evaluation in LVLMs.The reported experiments show that POPE better evaluates the object hallucination issue.
7 Limitations
The paper’s evaluation focuses specifically on object hallucination rather than overall LVLM capability, uses partial validation sets, and has additional matching and segmentation constraints.
- Scope: The evaluation cannot measure overall LVLM performance because it focuses only on object hallucination.A higher POPE score does not necessarily indicate stronger overall model capacity.
- Data coverage: Limited computation resources required evaluating models on only part of each dataset’s validation set, so reported results may reflect data-distribution effects.The authors state that the experiments were carefully configured despite this constraint.
- Evaluation procedure: POPE’s matching-based Yes-or-No decision can produce inaccurate evaluations when LVLM outputs do not explicitly contain those words.This limitation concerns occasional failures to provide explicit binary answers.
- Automatic annotation: Automatic segmentation assigns objects according to the tool’s label set, which may differ from collected human annotations.This creates a boundary for interpreting POPE results built from automatically annotated images.
B Additional Qualitative Analysis Results
Across all 80 MSCOCO objects, hallucinations are concentrated among frequently occurring objects and objects that frequently co-occur with dining tables.
- The top 30 objects account for approximately 70% of hallucinated objects across all evaluated models.
- The top 20 objects co-occurring with dining tables account for about 80% of hallucinated objects.
C Additional Quantitative Analysis Results
Hallucination trends extend beyond the initially analyzed objects: chair and car exhibit patterns similar to those reported for the dining-table analysis.
- Chair and car show a similar hallucination trend to the dining-table analysis.
D Results of SEEM-based POPE on A-OKVQA and GQA
The study extends POPE evaluation to A-OKVQA and GQA using SEEM-derived segmentation results and evaluates several LVLMs, with ChatGPT-assisted VQA scoring for open-ended responses.
- SEEM is used to annotate A-OKVQA and GQA images, providing segmentation results for constructing POPE evaluations.
- The evaluation covers InstructBLIP, MiniGPT-4, LLaVA, and the smaller supervised-tuned BLIP model.
- ChatGPT assesses whether each open-ended LVLM response matches one of the provided correct answers for VQA scoring.
F Results of Image Captioning
MSCOCO captioning performance generally follows the POPE assessments, indicating that object hallucination is associated with LVLM effectiveness on other vision tasks.
- MSCOCO captioning results generally align with the POPE assessments.
- The reported alignment suggests that object hallucination influences LVLM efficacy in other vision tasks.