PaperlayerView exact revision on arXiv

Source-linked AI summary

The Instinctive Bias: Spurious Images lead to Illusion in MLLMs

Tianyang Han, Qing Lian, Rui Pan, Renjie Pi, Jipeng Zhang, Shizhe Diao, Yong Lin, Tong Zhang

arXiv:2402.03757v2cs.CVcs.CLcs.LG

TL;DR

MLLMs can privilege relevant but misleading image content over question semantics, producing visual illusion. The paper introduces CorrelationQA and evaluates nine mainstream models, finding that they all suffer from this bias to varying degrees.

  • Problem

    MLLMs may answer from pictured objects rather than use question semantics in complex visual reasoning, and spurious images can make otherwise answerable questions go wrong.

    Source: §1 · pp. 1–2

  • Method

    CorrelationQA uses generated QA pairs with correct and spurious answers, paired with factual and spurious images, to assess visual illusion across 13 categories.

    Source: §5 · p. 8

  • Results

    Nine state-of-the-art MLLMs, including GPT-4V, suffer visual illusion with spurious inputs, and their accuracy drops from factual to spurious images.

    Source: §1 · p. 2, Evaluation Results on CorrelationQA · p. 5

  • Takeaways & Limitations

    The findings suggest future work could adjust training strategies so MLLMs calibrate attention to image information according to its relevance in context.

    Source: §5 · p. 9

    CorrelationQA uses questions whose answers are entities, limiting evaluation of other question types.

    Source: §6 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) have recently experienced remarkable progress, where the advent of multi-modal large language models (MLLMs) has endowed LLMs with visual capabilities, leading to impressive performances in various multi-modal tasks. However, those powerful MLLMs such as GPT-4V still fail spectacularly when presented with certain image and text inputs. In this paper, we identify a typical class of inputs that baffles MLLMs, which consist of images that are highly relevant but inconsistent with answers, causing MLLMs to suffer from visual illusion. To quantify the effect, we propose CorrelationQA, the first benchmark that assesses the visual illusion level given spurious images. This benchmark contains 7,308 text-image pairs across 13 categories. Based on the proposed CorrelationQA, we conduct a thorough analysis on 9 mainstream MLLMs, illustrating that they universally suffer from this instinctive bias to varying degrees. We hope that our curated benchmark and evaluation results aid in better assessments of the MLLMs' robustness in the presence of misleading images. The code and datasets are available at https://github.com/MasaiahHan/CorrelationQA.

1 Introduction

The paper identifies an instinctive bias in which MLLMs privilege image content over question semantics, even when the image is relevant but inconsistent with the answer. It proposes CorrelationQA to quantify this problem and reports that it affects nine mainstream MLLMs.

1 Introduction

  • MLLMs may ignore reasoning-question semantics and answer from pictured objects when visual reasoning requires using information not directly shown in the image.The paper calls this visual illusion the instinctive bias.Source: §1 · p. 1
  • LLaVA answers the question correctly in text-only conditions but chooses Eiffel Tower when shown a related image that contradicts the answer, despite the correct answer being Cairns.The example illustrates how spurious visual input can change a correct answer into an incorrect one.Source: Figure 1 · p. 1, §1 · p. 2
  • CorrelationQA contains over 7,000 question-answer pairs across 13 categories, using related but incorrect answers to create spurious images and factual images for comparison.The benchmark also includes typographic images and realistic images collected from the Internet.Source: §1 · p. 2
  • Nine state-of-the-art MLLMs, including GPT-4V, suffer visual illusion with spurious inputs, motivating assessment of their robustness to misleading images.The paper presents the benchmark as a way to quantify the seriousness of this bias across types.Source: §1 · p. 2, Figure 2 · p. 2

2 Method

The method studies MLLMs under factual, spurious, and random image conditions, then constructs CorrelationQA and measures accuracy and the accuracy drop from factual to spurious inputs.

2.1 Motivation

  • The study targets answer-correlated but answer-contradicted images, against which MLLMs can fail despite answering correctly without images.It distinguishes factual images that correspond to the answer, spurious images related to the question but not the answer, and random images unrelated to either.Source: §2.1 · pp. 2–3

2.2 CorrelationQA

  • CorrelationQA’s automated pipeline defines 13 meta-categories and creates QA pairs with one correct answer and five incorrect alternatives, then pairs them with images.Images include generated natural and typography images, plus realistic images collected through search; typography images target OCR-like scenarios.Source: §2.2 · pp. 3–4, Figure 3 · p. 3

2.3 Evaluation Metrics

  • Accuracy counts correctly answered image-text pairs over all pairs, using approximate matching that accepts an abbreviation or a response containing the answer label.Outputs are limited in word count because benchmark labels contain no more than five words.Source: §2.3 · p. 4
  • AccDrop is factual-image accuracy minus spurious-image accuracy, with larger values indicating greater sensitivity to deceptive image information.The metric compares performance with factual and spurious images.Source: §2.3 · p. 4

3 Experiments

The experiments evaluate CorrelationQA across nine MLLMs and multiple image conditions, finding that spurious images lower accuracy, with stronger bias for typography images. The results also vary by category and model, and similar behavior appears with realistic images.

3.1 Dataset Collection

  • CorrelationQA is generated through a pipeline that uses GPT-4 to create question-answer pairs and Stable Diffusion and OCR scripts to create factual and spurious images.The pipeline generates one correct answer and five incorrect answers for each question.Source: §3.1 · p. 4

3.2 Experimental Setup

  • The evaluation tests nine MLLMs using direct answers to questions paired with natural or typography images, with models tested using their latest released weights.Open-source models generally use the widely used 7B version unless otherwise specified; other parameters use authors’ defaults.Source: §3.2 · pp. 4–5

Evaluation Results on CorrelationQA

  • Typography images produce more severe instinctive bias than natural images, and OCR data have higher accuracy drops.LLaVA and GPT-4V have higher average accuracy on spurious images than other tested MLLMs, while LLaVA-7B and LLaVA-13B show almost no fluctuation between factual and spurious contexts.Source: Evaluation Results on CorrelationQA · p. 5

Results on Different Categories

  • Animals, colors, food, and plants show larger AccDrop than history and art, while typography produces larger drops than natural images across categories.Unlike natural images, typography AccDrop does not differ significantly across categories.Source: Results on Different Categories · p. 6

Spurious Information induces Visual Illusion

Results on Realistic Image

  • Qwen-VL and LLaVA-7B show similar behavior on realistic and synthetic images, supporting generalization of the synthetic-image findings to realistic images.The authors suggest that extra content in keyword-searched realistic images may explain performance differences between realistic spurious images and synthetic images.Source: Results on Realistic Image · p. 7, Table 4 · p. 7

Qualitative Analysis

  • Qualitative examples suggest that recognizable images of tangible themes are more likely to mislead MLLMs than less image-identifiable history and art topics.In some correctly answered examples, image content shifts the image-text relationship toward random or irrelevant rather than misleading the answer.Source: Qualitative Analysis · pp. 7–8, Figure 6 · p. 8

4 Related Work

Prior work studies MLLM visual understanding and hallucinations, including benchmarks and mitigation approaches. This paper focuses instead on illusion under spurious visual inputs.

  • MLLMs use LLMs as reasoning engines and textual interfaces, and are trained on image-text pairs for visual understanding and tasks such as VQA and captioning.Source: §4 · p. 8
  • Prior studies describe responses inconsistent with visual information as illusion or hallucination and propose benchmarks and instruction fine-tuning approaches.The cited examples include benchmarks for object hallucination and balancing positive and negative training samples.Source: §4 · p. 8
  • Unlike those approaches, this work examines visual illusion when MLLMs receive spurious visual inputs.Source: §4 · p. 8

5 Conclusion

The paper identifies instinctive bias toward deceptive images in MLLMs and introduces CorrelationQA to evaluate it. Its findings motivate assessing MLLMs in real-world scenarios and calibrating attention to image relevance.

5 Conclusion

  • Current MLLMs can develop instinctive bias toward deceptive images.Source: §5 · p. 8
  • GPT-4 and Stable Diffusion generate factual and spurious image-text pairs for a 13-category benchmark evaluating visual illusion.Source: §5 · p. 8
  • The authors suggest future training strategies could help MLLMs calibrate attention to image information according to its relevance.Source: §5 · p. 9

6 Limitations

The evaluation and conclusions have defined scope boundaries: CorrelationQA uses entity-answer questions, and the study omits larger parameter models such as Vicuna-33B. The paper also identifies possible directions for future investigation.

6 Limitations

  • CorrelationQA’s entity-answer questions limit evaluation of other question types.Source: §6 · p. 9
  • The study does not assess larger parameter models such as Vicuna-33B, though it reports bias appears unrelated to model scale.Source: §6 · p. 9
  • The authors suggest training data may relate to bias and leave potential bias in video- and audio-capable MLLMs for future exploration.Source: §6 · p. 9

A More Related Works

The appendix reviews adversarial attacks on language models, including relevant-image jailbreaks, and provides a GPT-4 prompt example for generating QA pairs. The prompt procedure uses continuation because of an output-token limit.

A More Related Works

B Detailed Prompts Example

C.1 Manual Verification

Manual verification samples QA pairs from every category and checks their answers against the GPT-4-provided answers. Most categories exceed 90% authenticity, with city as an exception linked to fictional-city naming conflicts.

C.1 Manual Verification

  • The verification randomly samples 20% of QA pairs from each category and checks whether answers match the GPT-4-provided answers.Source: §C.1 · p. 11
  • Most categories have authenticity rates above 90%, except city.Source: §C.1 · p. 11
  • Fictional cities from movies and novels contribute to naming conflicts in the city category.Source: §C.1 · p. 11

C.2 Accuracy Results on Spurious Image

Accuracy varies by spurious-image type and category: natural-image accuracy is higher for categories whose spurious answers are often nonvisualizable or whose images are unrecognizable, while typography yields more uniform, lower average accuracy.

C.2 Accuracy Results on Spurious Image

  • Natural-image accuracy is higher for art, technology, and history, where spurious answers are often nonvisualizable, and for human images that MLLMs cannot recognize.These images therefore fail to significantly deceive or mislead the models.Source: §C.2 · p. 11
  • Typography produces more uniform accuracy across the 13 categories and lower average accuracy per category than natural images.The authors assume OCR-only typography more directly misleads MLLMs because it does not require content understanding and is category-independent.Source: §C.2 · p. 11

C.3 Accuracy Declined Ratio

The accuracy declined ratio expresses accuracy drop relative to factual-image accuracy. Results show greater decline ratios for tangible-entity categories with natural images and ratios above 30% across categories with typography.

C.3 Accuracy Declined Ratio

  • Accuracy declined ratio identifies categories with the largest proportion of accuracy decline; higher values indicate stronger effects relative to factual-image accuracy.The ratio uses AccDrop and factual-image accuracy, emphasizing relative effects.Source: §C.3 · pp. 11, 13
  • With typography, accuracy declined ratios exceed 30% in all categories, and each category's ratio is higher than with natural images.Table 8 reports the typography ratios; the text states the across-category comparison.Source: §C.3 · p. 13, Table 8 · p. 13
  • For natural images, animal, color, and plant categories, which comprise tangible entities, have higher accuracy declined ratios.Table 7 reports the natural-image ratios.Source: §C.3 · p. 13, Table 7 · p. 13

Related papers

Loading 2402.03757v2…