PaperlayerView exact revision on arXiv

Source-linked AI summary

Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges

Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, Huaxiu Yao

arXiv:2311.03287v2cs.LGcs.CLcs.CV

TL;DR

GPT-4V(ision)’s hallucination behavior had not been systematically assessed. The paper introduces Bingo to examine bias and interference, and reports hallucinations across these challenges that self-correction and chain-of-thought do not fully resolve.

  • Problem

    GPT-4V(ision)’s hallucination behavior had not been systematically assessed, despite its visual-language capabilities.

    Source: §1 · p. 1

  • Method

    The paper introduces Bingo, a benchmark of GPT-4V(ision) failure cases organized around bias and interference.

    Source: §1 · pp. 1–2, §2 · p. 2

  • Results

    GPT-4V(ision) exhibits hallucinations related to bias and interference, and self-correction and chain-of-thought fall short of completely rectifying them.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The findings characterize hallucination challenges in GPT-4V(ision) and the limits of the tested mitigation approaches.

    Source: §1 · p. 2, §6 · p. 8

    The benchmark covers a few metrics and tasks, and human judgments in data curation may carry their own biases despite multiple researchers curating and evaluating results.

    Source: §6 · p. 8

Abstract

from arXiv · show

While GPT-4V(ision) impressively models both visual and textual information simultaneously, it's hallucination behavior has not been systematically assessed. To bridge this gap, we introduce a new benchmark, namely, the Bias and Interference Challenges in Visual Language Models (Bingo). This benchmark is designed to evaluate and shed light on the two common types of hallucinations in visual language models: bias and interference. Here, bias refers to the model's tendency to hallucinate certain types of responses, possibly due to imbalance in its training data. Interference pertains to scenarios where the judgment of GPT-4V(ision) can be disrupted due to how the text prompt is phrased or how the input image is presented. We identify a notable regional bias, whereby GPT-4V(ision) is better at interpreting Western images or images with English writing compared to images from other countries or containing text in other languages. Moreover, GPT-4V(ision) is vulnerable to leading questions and is often confused when interpreting multiple images together. Popular mitigation approaches, such as self-correction and chain-of-thought reasoning, are not effective in resolving these challenges. We also identified similar biases and interference vulnerabilities with LLaVA and Bard. Our results characterize the hallucination challenges in GPT-4V(ision) and state-of-the-art visual-language models, and highlight the need for new solutions. The Bingo benchmark is available at https://github.com/gzcch/Bingo.

1 Introduction

The introduction presents Bingo as a benchmark for examining GPT-4V(ision)’s hallucinations, grouped into bias and interference. It reports that self-correction reduces hallucinations, while chain-of-thought reasoning does not prevent them in most instances.

1 Introduction

  • Bingo is a benchmark of 190 GPT-4V(ision) failure instances, categorized by causes into bias and interference; its bias categories are region, OCR, and factual bias.Source: §1 · p. 1
  • Bias covers hallucinations on specific example types, while interference covers disrupted judgment that increases susceptibility to hallucination.The introduction describes factual bias as relying on learned knowledge over image content, and text-to-image interference as user claims disrupting recognition.Source: §1 · pp. 1–2
  • Self-correction reduces hallucinations by 16.56%, while chain-of-thought reasoning still yields hallucinations in most instances.The self-correction prompt asks GPT-4V(ision) to review and revise an erroneous answer.Source: §1 · p. 2

2 Bingo Benchmark

Bingo pairs GPT-4V(ision) failure cases with success cases and organizes them into bias and interference categories. Its data collection covers region, OCR, and factual bias, as well as image-to-image and text-to-image interference.

2 Bingo Benchmark

  • Bingo contains 190 failure instances and 131 success instances; each image is paired with one or two questions.The failures are grouped into bias and interference, with three bias types and two interference types.Source: §2 · p. 2

2.1 Bias

  • Bias examples cover five regions, multilingual text, and images that contradict factual knowledge.The region set includes East Asia, South Asia, South America, Africa, and the Western world; OCR examples use Arabic, Chinese, French, Japanese, and English text.Source: §2.1 · p. 3

2.2 Interference

  • Image-to-image examples test recognition across similar composite images, while text-to-image examples pair a correct-answer prompt with an incorrect-answer prompt.The image set includes natural and synthetic examples; the paired text prompts make opposing claims about the same image.Source: §2.2 · p. 3

3 Empirical Analysis

The analysis evaluates GPT-4V(ision) on Bingo and compares it with LLaVA-1.5 and Bard. It reports regional, language, factual-prior, and interference vulnerabilities, with similar challenges in the other models.

3.1 Analysis of Bias

  • GPT-4V(ision) performs significantly better on Western images than on East Asian and African images, including correctly naming a European cathedral but misidentifying an African one.The paper suggests distribution shift as one possible factor in the regional disparity.Source: §3.1 · p. 4, Figure 3 · p. 4
  • It handles embedded English and French better than other languages; on the translated Chinese anime image, it misidentified almost all embedded text.The paper attributes the language disparity primarily to OCR detector bias and notes distribution shift and language-specific typography as possible factors.Source: Figure 4 · p. 4, §3.1 · pp. 4–5, Figure 5 · p. 5
  • 93.1% of errors on counterfactual images stem from factual-knowledge reliance, and performance is significantly better on factual than counterfactual images.The comparison concerns images containing factual knowledge versus counterfactual knowledge.Source: Table 2 · p. 5, §3.1 · p. 5

3.2 Analysis of Interference

  • Combined similar images significantly degrade GPT-4V(ision)'s performance, while inaccurate prompt claims can lead it to disregard the image and agree with the user.When similar images are presented individually, GPT-4V(ision) can recognize them accurately.Source: §3.2 · pp. 5–6, Table 3 · p. 6

3.3 Analysis on Other VLMs

  • LLaVA-1.5 shows 17.0% vs GPT-4V(ision)'s 26.8% on non-Western region bias and 2.3% vs 28.3% on non-English OCR; Bard also exhibits both challenges but fares better.Bard still falls short of GPT-4V(ision), especially on interference, and shows stronger OCR bias mitigation than LLaVA and GPT-4V(ision).Source: §3.3 · p. 6

4 Can We Reduce Hallucination in GPT-4V(ision)?

The section tests self-correction and Chain-of-Thought (CoT) prompting as ways to reduce GPT-4V(ision)’s hallucinations. Neither effectively resolves the bias and interference challenges, and CoT’s limited success may reflect visual limitations.

Self-Correction

  • 16.9% of errors were reduced through self-correction, but a significant portion remained uncorrected.The evaluation reports average accuracy across images within each category.Source: §4 · p. 7, Table 5 · p. 8

Chain-of-Thought

  • 5.7% fewer regional-bias hallucinations resulted from CoT prompting, which still failed to rectify hallucinations in most cases.The authors note that CoT enhances language reasoning but may not address visual limitations.Source: §4 · p. 7, Table 5 · p. 8
  • In the solar-system example, step-by-step prompting improved reasoning but still overlooked that Saturn was absent.The figure marks the description of the illusion in red.Source: Figure 9 · p. 7

4 Can We Reduce Hallucination in GPT-4V(ision)?

  • Neither self-correction nor CoT effectively addresses the bias and interference challenges in Bingo.The authors conclude that further research and innovation are needed to tackle these persistent issues.Source: §4 · pp. 7–8, Table 5 · p. 8

5 Related Work

Prior work studies hallucination in VLMs through mitigation, evaluation, and detection, including concurrent studies of GPT-4V(ision). The authors identify a gap in systematic analysis of why GPT-4V(ision) hallucinates.

Hallucination in VLMs

  • Hallucination in VLMs is commonly defined as generated responses containing information absent from visual content.Prior mitigation methods include fine-grained contrastive learning, feature fusion, and data augmentation.Source: §5 · p. 8
  • Studies of autoregressive VLMs such as LLaVA address hallucination evaluation, detection, and mitigation.Concurrent work also examined hallucinations in GPT-4V(ision), including visual puzzles, cross-modal interactions, character recognition, and visual illusions.Source: §5 · p. 8

Empirical Analysis of GPT-4V(ision)

  • The cited concurrent GPT-4V(ision) studies did not systematically categorize and analyze the reasons for hallucinations.This paper positions its work as a comprehensive study of causes and introduces a benchmark for that purpose.Source: §5 · p. 8

6 Conclusion

The paper introduces Bingo to analyze hallucinations in VLMs, particularly GPT-4V(ision), and finds that bias and interference challenges persist despite two mitigation strategies. The authors describe the benchmark as an initial effort with limited tasks and metrics, and note possible bias in human curation.

6 Conclusion

  • Bingo analyzes GPT-4V(ision) hallucinations associated with specific image types and interference in judgment.The benchmark is introduced for analysis of VLM hallucinations, particularly in GPT-4V(ision).Source: §6 · p. 8
  • Self-correction and Chain-of-Thought fall short of completely rectifying GPT-4V(ision)’s hallucinations under bias and interference challenges.The findings are presented as improving understanding of reliability in GPT-4V(ision) and other VLMs.Source: §6 · p. 8

Limitations

  • The benchmark covers a few metrics and tasks as a starting point, and its human curation may carry biases.The authors say multiple researchers curated and evaluated results to try to mitigate this concern.Source: §6 · p. 8

A Additional Examples of Bias and Interference

The appendix provides additional examples of bias and interference in GPT-4V(ision), including cases involving regional, OCR, factual, image-to-image, and text-to-image challenges.

Additional Examples of Bias and Interference

Image-to-Image Interference

  • Figures 15–16 show additional image-to-image interference cases using natural and synthesized images.Figure 16 specifies synthesized images.Source: Figure 15 · p. 15, Figure 16 · p. 16

Text-to-Image Interference

Additional Examples of Bias and Interference

  • Figure 19 is captioned as showing failure cases of chain-of-thought (CoT), so the supported finding is that it documents CoT failure cases.Source: Figure 19 · p. 19
Loading 2311.03287v2…