PaperlayerView exact revision on arXiv

Source-linked AI summary

HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, Tianyi Zhou

arXiv:2310.14566v5cs.CVcs.CL

TL;DR

LVLMs remain vulnerable to language bias and visual misinterpretation, motivating more diagnostic evaluation of image-context reasoning. HALLUSIONBENCH uses human-crafted control-pair questions and diverse original, edited, and absent visual inputs to analyze these failures. Across 15 models, GPT-4V achieves 31.42% Question Pair Accuracy while all other methods remain below 16%.

  • Problem

    LVLMs can over-rely on language priors or misinterpret visual inputs, but their distinct failure modes require more systematic diagnostic evaluation.

  • Method

    HALLUSIONBENCH uses human-crafted visual-question control pairs across Visual Dependent and Visual Supplement settings to quantify response tendencies, consistency, and failure types.

  • Results

    31.42% Question Pair Accuracy is achieved by GPT-4V, while all other evaluated methods remain below 16%.

  • Takeaways & Limitations

    The benchmark identifies weaknesses in current LVLMs and provides insights intended to support finetuning and model improvement.

  • Takeaways & Limitations

    Current LVLMs fail to capture temporal relations across multiple images, indicating limited temporal reasoning ability in this benchmark.

Abstract

from arXiv · show

We introduce HallusionBench, a comprehensive benchmark designed for the evaluation of image-context reasoning. This benchmark presents significant challenges to advanced large visual-language models (LVLMs), such as GPT-4V(Vision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, by emphasizing nuanced understanding and interpretation of visual data. The benchmark comprises 346 images paired with 1129 questions, all meticulously crafted by human experts. We introduce a novel structure for these visual questions designed to establish control groups. This structure enables us to conduct a quantitative analysis of the models' response tendencies, logical consistency, and various failure modes. In our evaluation on HallusionBench, we benchmarked 15 different models, highlighting a 31.42% question-pair accuracy achieved by the state-of-the-art GPT-4V. Notably, all other evaluated models achieve accuracy below 16%. Moreover, our analysis not only highlights the observed failure modes, including language hallucination and visual illusion, but also deepens an understanding of these pitfalls. Our comprehensive case studies within HallusionBench shed light on the challenges of hallucination and illusion in LVLMs. Based on these insights, we suggest potential pathways for their future improvement. The benchmark and codebase can be accessed at https://github.com/tianyi-lab/HallusionBench.

1. Introduction

HALLUSIONBENCH targets language hallucination and visual illusion in LVLMs, using human-crafted visual-question structures to diagnose failure modes quantitatively.

  • Motivation: LVLMs can let language priors override visual context, producing affirmative or otherwise erroneous answers when prior knowledge conflicts with images.The paper distinguishes language bias from visual misinterpretation as separate failure patterns.
  • Benchmark: HALLUSIONBENCH is introduced as a diagnostic suite for systematically dissecting diverse LVLM failure modes.Its visual-question design supports quantitative analysis of model failures and aims to inform more robust, balanced, and precise models.
  • Benchmark: Approximately 1129 handcrafted VQA pairs use 165 original images and 181 human-modified images.The benchmark moves beyond conventional correctness and accuracy metrics through an innovative question structure.
  • Evaluation: 15 recent methods are evaluated, with GPT-4V reaching 31.42% Question Pair Accuracy while every other method remains below 16%.The benchmark presents substantial challenges even for the strongest evaluated model.
  • Analysis: The study analyzes failure cases from GPT-4V and LLaVA-1.5 and uses benchmark-enabled quantitative analysis to characterize issues facing current LVLMs.These analyses provide insights intended to support future model improvement.

2. Related Work

Prior work studies hallucination detection, mitigation, and multimodal capabilities, but existing benchmarks provide limited diagnostic coverage of LVLM failures.

  • Hallucination Research: Hallucination research examines detecting outputs unsupported by visual content and methods for reducing such errors through data and training improvements.Examples include classifiers, answer comparisons, balanced instructions, and iterative answer generation.
  • VL Benchmarks: Traditional VL benchmarks assess skills such as visual recognition and image description, while newer benchmarks structure more complex multimodal tasks.The paper argues that conventional evaluation metrics do not provide sufficiently detailed ability assessment.
  • HALLUSIONBENCH: HALLUSIONBENCH differs from POPE and GAVIE by diagnosing both visual illusion and knowledge hallucination rather than focusing only on object hallucination.It is presented as a human-annotated analytical benchmark for these two failure types.

3. HALLUSIONBENCH Construction

HALLUSIONBENCH combines diverse human-crafted visual-question control pairs, original and edited images, and Visual Dependent and Visual Supplement question categories.

  • Dataset Scope: The benchmark contains 455 visual-question control pairs, 346 figures, and 1129 questions spanning diverse topics and visual formats.Topics include food, mathematics, geography, sports, cartoons, and illusions; formats include logos, posters, charts, tables, maps, and consecutive images.
  • Visual Dependent Questions: Visual Dependent questions require visual context for an affirmative answer and evaluate visual commonsense and reasoning skills.They ask about the image itself or content within it.
  • Visual Supplement Questions: Visual Supplement questions can be answered without visual input, which supplements or corrects information available from model memory.This category evaluates visual reasoning and the balance between parametric memory and image context.
  • Annotation Structure: Questions are organized over original, modified, and empty images, with Visual Dependent and Visual Supplement sets defined differently over these image groups.All questions use binary Yes/No annotations.
  • Dataset Statistics: 346 collected images yield 591 Visual Dependent questions and 538 Visual Supplement questions, with 16% lacking visual input, 39% using original images, and 45% using edited images.Human experts manually collect diverse images and apply manipulation strategies to test robustness.
  • Uniqueness: Compared with existing benchmarks, HALLUSIONBENCH broadens topics, image types, and modalities while introducing human-edited images and control-group analysis.It evaluates language hallucinations and visual illusions beyond the narrower scope of object hallucination benchmarks.

4. HALLUSIONBENCH Evaluation Suite

HallusionBench evaluates LVLM correctness, response bias, logical consistency, and failure types using structured visual-question sets and control pairs. Its evaluation combines GPT-4-assisted correctness scoring with quantitative tests for yes/no bias, consistency, language hallucination, and visual illusion.

  • Correctness Evaluation Metrics: GPT-4 judges each LVLM answer as incorrect, correct, or uncertain against the ground truth, with evaluations repeated three times because outputs vary even at temperature 0.The benchmark fills a prompt with each question, ground truth, and model output, then averages three GPT-4 evaluations.
  • Correctness Evaluation Metrics: Uncertain answers without visual input are accepted for Visual Supplement questions, while final correctness is converted into a binary value.The binary conversion uses the GPT-4 judgment except that uncertain answers are accepted when the image is empty under the Visual Supplement category.
  • Analytical Evaluation Criteria: Three analytical criteria—Yes/No Bias, Consistency, and Diagnostic tests—quantitatively measure response tendencies and failure modes beyond accuracy.The tests increase in complexity, making later analyses less informative when a model fails the preceding basic test.
  • Yes / No Bias Test: The Yes/No Bias Test measures the difference between predicted and actual yes counts and the false-positive ratio among incorrect visual questions.Bias increases as |dy| approaches 1, while robustness is associated with a false-positive ratio near 0.5.
  • Consistency Test: The Consistency Test uses logically correlated questions about one figure and flags inconsistency when only some questions in the set are correct.Responses that are consistently correct or consistently wrong are treated as consistent under this criterion.
  • Language Hallucination and Visual Illusion: Control pairs diagnose language hallucination, visual illusion, or mixed/uncertain failures by comparing correctness and responses across original, edited, or absent-image conditions.Language hallucination reflects prior assumptions without relevant visual input, whereas visual illusion reflects misinterpretation of accurate visual information.

5. Experimental Results

HALLUSIONBENCH evaluates 15 LVLMs with correctness, bias, and fine-grained diagnostic analyses. GPT-4V leads open-source models overall but remains challenged by edited images, visual bias, and math, illusion, and video formats.

  • 15 LVLMs, including GPT-4V, Gemini Pro Vision, Claude 3, and LLaVA-1.5, are evaluated against Random Choice as a baseline.
  • GPT-4V outperforms open-source LVLMs except on Hard Accuracy, which measures understanding of human-edited images.Its poor Hard Accuracy demonstrates the difficulty of the benchmark’s image manipulations.
  • Larger LLM backbones reduce object-existence hallucination in models such as LLaVA-1.5 and BLIP2-T5.Among models below 10B parameters, InstructBLIP and mPLUG-Owl-v2 perform best.
  • GPT-4V, BLIP2-T5, and mPLUG-Owl-v2 outperform Random Choice across question-pair, figure-pair, and question-level accuracy, whereas Qwen-VL and MiniGPT4 perform worse.
  • LLaVA-1.5, Open-Flamingo, and mPLUG-Owl-v1 show affirmative-answer tendencies, associated with low or high bias-test measures.The authors suggest unbalanced positive and negative instructions and scarce edited images in training as possible reasons for poor performance.
  • Math, Illusion, and Video are the most challenging formats, with GPT-4V failing on regular triangles, edited illusions, and reversed image sequences.GPT-4V recognizes illusion names better than LLaVA-1.5 but may rely on parametric memory rather than faithfully analyzing edited images.

6. Conclusion, Limitations and Future Work

The conclusion frames HALLUSIONBENCH as a diagnostic suite for analyzing LVLM failures and identifies language hallucination, visual illusion, manipulation sensitivity, and weak temporal reasoning. The authors propose using these diagnoses to support future model improvement.

  • HALLUSIONBENCH is introduced as a diagnostic suite for analyzing failure cases across 15 current LVLMs.
  • Language hallucination occurs when models prioritize prior knowledge over visual context, while visual illusion reflects erroneous interpretations of figures.
  • LVLMs can be misled by image flipping, order reversing, masking, optical-character editing, object editing, and color editing.
  • Existing LVLMs lack true temporal reasoning ability, failing to capture temporal relations across multiple images.
  • The benchmark is intended to identify model weaknesses and provide insights for finetuning and improvement.

A. More Case Analysis on HALLUSIONBENCH with GPT-4V and LLaVA-1.5

The case-analysis section presents self-contained HALLUSIONBENCH examples that juxtapose control pairs, model responses, failures, and corresponding answer components.

  • Each case is self-contained and highlights control pairs, GPT-4V and LLaVA-1.5 responses, model failures, and corresponding answer parts.

A.1. Visual Dependent Examples

The visual-dependent case studies show that GPT-4V and LLaVA-1.5 remain vulnerable to illusions, geometric reasoning, edited characters, and temporal sequence reversals. GPT-4V often has stronger knowledge than LLaVA-1.5 but still fails to ground answers in the presented images.

  • Visual Dependent Examples: GPT-4V recognizes famous illusion cases and their names better than LLaVA-1.5 but does not answer edited-image questions faithfully.
  • Visual Dependent Examples: GPT-4V and LLaVA-1.5 fail to recognize parallel lines, regular triangles, polygons, and other mathematical theorems.These examples identify geometry and mathematics as challenging tasks for GPT-4V.
  • Visual Dependent Examples: Editing characters in images misleads both GPT-4V and LLaVA-1.5, indicating reliance on parametric memory instead of visual reasoning.
  • Visual Dependent Examples: GPT-4V cannot distinguish positive from reversed image sequences when their meanings differ, such as appearing versus disappearing.

A.2. Visual Supplement Examples

The visual-supplement examples show that GPT-4V and LLaVA-1.5 often prioritize prior knowledge over charts, tables, maps, and other visual context.

  • Visual Supplement Examples: GPT-4V and LLaVA-1.5 cannot correctly understand the provided charts, and GPT-4V’s predictions changed completely after a chart was rotated.The authors describe chart reasoning as limited in these examples.
  • Visual Supplement Examples: GPT-4V and LLaVA-1.5 answer affirmative questions without images in several examples, then continue relying on knowledge rather than analyzing supplied visual content.The examples include charts, tables, and maps, and the authors connect this behavior to dependence on textual reasoning.

B. Decision Tree Logic and Examples

The benchmark diagnoses language hallucination and visual illusion by comparing model answers across control-pair questions and visual conditions, then assigns failure types through a color-coded decision tree.

  • Decision Tree Logic and Examples: GPT-4V’s correct answer on the original Chubb illusion but incorrect answer on the edited image is classified as Language Hallucination.The same answer across both images indicates reliance on parametric knowledge rather than the image.
  • Decision Tree Logic and Examples: GPT-4V’s original-image answer can be correct while becoming uncertain with an edited or supplementary image, producing Visual Illusion diagnoses.This pattern appears for the vertical-line example and the visual-supplement example.
  • Decision Tree Logic and Examples: LLaVA-1.5 shows Visual Illusion when it fails on the original circle image but succeeds after editing, indicating difficulty determining circle diameters.The decision-tree route is identified as (VS) R-B.
  • Decision Tree Logic and Examples: The decision tree uses correctness of two control-pair questions and response differences to diagnose failure types.GPT4 Evaluation labels responses Incorrect (0), Correct (1), or Uncertain (2), while colored paths encode decision routes.
  • Decision Tree Logic and Examples: LLaVA-1.5 exhibits Language Hallucination when answers remain incorrect with unchanged or edited supplementary images, including map examples.The examples distinguish persistent hallucination from additional visual-interpretation errors.
  • Decision Tree Logic and Examples: The examples include famous optical illusions involving size and length, plus geography questions about state area and country land area.The prompts illustrate how visual-dependent and visual-supplement cases test image-conditioned reasoning.
Loading 2310.14566v5…