PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models

Mingrui Wu, Jiayi Ji, Oucheng Huang, Jiale Li, Yuhang Wu, Xiaoshuai Sun, Rongrong Ji

arXiv:2406.16449v4cs.CV

TL;DR

Object-focused hallucination research has left relationship hallucinations without comprehensive benchmarking, despite their importance for visual comprehension. R-Bench evaluates these errors at image and instance levels, and its analysis identifies greater relationship hallucination severity alongside co-occurrence and visual-grounding failure patterns.

  • Problem

    Prior work primarily evaluated object hallucinations, leaving inter-object relationship hallucinations without comprehensive, rigorous benchmarks despite their importance for visual comprehension.

    Source: §1 · p. 1

  • Method

    R-Bench uses image-level questions about relationship existence and instance-level questions about local visual comprehension, with specific objects indicated by boxes or masks.

    Source: §1 · p. 1

  • Results

    R-Bench evaluations find relationship hallucinations more severe than object hallucinations, with relationship co-occurrences and reliance on LLM common sense among reported patterns.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors hope the reported findings will inspire further work on solutions to relationship hallucinations in LVLMs.

    Source: §6 · p. 9

    The analysis covers only a subset of relationship-hallucination causes, and the manually filtered data may retain noise from omissions or biases.

    Source: §5 · p. 9

Abstract

from arXiv · show

The issue of hallucinations is a prevalent concern in existing Large Vision-Language Models (LVLMs). Previous efforts have primarily focused on investigating object hallucinations, which can be easily alleviated by introducing object detectors. However, these efforts neglect hallucinations in inter-object relationships, which is essential for visual comprehension. In this work, we introduce R-Bench, a novel benchmark for evaluating Vision Relationship Hallucination. R-Bench features image-level questions that focus on the existence of relationships and instance-level questions that assess local visual comprehension. We identify three types of relationship co-occurrences that lead to hallucinations: relationship-relationship, subject-relationship, and relationship-object. The visual instruction tuning dataset's long-tail distribution significantly impacts LVLMs' understanding of visual relationships. Furthermore, our analysis reveals that current LVLMs tend to disregard visual content and overly rely on the common sense knowledge of Large Language Models. They also struggle with reasoning about spatial relationships based on contextual information.

1. Introduction

Prior work emphasized object hallucinations, leaving relationship hallucinations without comprehensive, rigorous benchmarks despite their importance for visual scene comprehension. The paper introduces R-Bench and uses it to analyze relationship-hallucination patterns in LVLMs.

1. Introduction

  • LVLMs combine pretrained visual encoders with LLMs to answer questions about visual inputs.Visual-language pretraining and visual instruction tuning are approaches used to integrate these components.Source: §1 · p. 1
  • Prior work addressed object hallucinations but lacked comprehensive, rigorous benchmarks for inter-object relationship hallucinations, which better reflect visual-scene comprehension.Object hallucinations had been mitigated with detection or segmentation models.Source: §1 · p. 1
  • R-Bench pairs image-level questions about relationship existence with instance-level questions about local visual comprehension, using bounding boxes or masks for specific relationships.The instance-level questions can be used with existing models without retraining.Source: §1 · p. 1
  • R-Bench uses the nocaps validation set (Agrawal et al., 2019), combining LLM question generation with manual curation to retain 11,651 questions.The retained set contains 7,883 image-level and 3,768 instance-level questions.Source: §1 · pp. 1–2
  • R-Bench evaluations find relationship hallucinations more severe than object hallucinations, with long-tail training distributions and relationship co-occurrence patterns among the reported causes.The analysis also reports that LVLMs may overlook visual content, rely on LLM common sense, and struggle with context-based spatial reasoning.Source: §1 · p. 2

2. Related Work

LVLMs combine visual and language components, but their training data often emphasize individual objects over inter-object relationships. Prior hallucination research has concentrated on objects, leaving relationship hallucinations with limited benchmarking and analysis.

2.1. Large Vision-Language Model

  • LVLM training data tend to emphasize individual objects rather than inter-object relationships, despite models combining visual encoders, language decoders, and image-text alignment modules.Visual instruction tuning is used to develop visual conversation ability.Source: §2.1 · p. 2

2.2. Hallucination in LVLMs

  • The benchmark-generation pipeline derives relationship seeds and object boxes from captions and images, then uses LLM prompts to generate and filter image-level and instance-level questions.The figure describes parsing COCO and nocaps captions and using GroundingDINO to identify significant objects.Source: Figure 2 · p. 3
  • Prior hallucination work focused mainly on objects and explored detection or segmentation mitigations; research on relationship hallucinations remains limited and lacks comprehensive benchmarks and analysis.Some studies had begun examining object attributes and relative positional relationships.Source: §2.2 · pp. 2–3

3. Relationship Hallucination Benchmark

R-Bench evaluates relationship hallucinations using image-level questions about relationship existence and instance-level questions about relations between marked objects. It uses nocaps validation data to address potential overlap concerns and filters generated questions for quality.

3. Relationship Hallucination Benchmark

  • To address potential leakage concerns around COCO and Visual Genome overlap, R-Bench is built from nocaps validation data.Visual Genome had been used extensively in pretraining or visual instruction tuning across most LVLMs.Source: §3 · p. 3

3.1. Benchmark Construction

  • Image-level questions use binary answers and LLM prompts seeded by caption-parsed relationships matched to image content.The binary format is described as more stable than instruction-based methods and compatible with POPE.Source: §3.1 · p. 4
  • Instance-level questions test relations between specified objects by marking them with colored boxes or masks, targeting local visual comprehension.The prompts ask whether a subject in one marked region has a specified relationship to an object in another.Source: §3.1 · pp. 4–5
  • LVLMs generally recognized reference objects, with most scoring higher on masks than boxes; Qwen-VL’s box-level alignment training corresponded to better box accuracy.Reference-object accuracy was computed by matching the model’s object label to the expected label.Source: §3.1 · p. 4

3.2. Data Statics

  • Filtering reduced 24,897 generated questions to 11,651: 7,883 image-level and 3,768 instance-level questions.The final set covers 4,034 images and 11,335 relationship triples; positive-negative imbalance led to equal-count subsets for evaluation.Source: Table 2 · p. 4, §3.2 · p. 5

4. Experiments

The experiments evaluate R-Bench across image-level and instance-level questions, compare relationship with object hallucinations, and examine co-occurrence and reasoning failures. They find relationship hallucinations are more severe and arise alongside difficulties in local visual discrimination, visual grounding, and contextual spatial reasoning.

4.2. Evaluating on the R-Bench

  • Existing LVLMs show more serious relationship hallucinations than object hallucinations, leaving more room for improvement on relationship hallucinations.Qwen-VL is an exception, but its tendency to answer no leads to relatively low recall.Source: Table 4 · p. 6, §4.2 · pp. 6–7

4.1. Experimental Setup

  • R-Bench evaluation covers image-level, bounding-box instance-level, and mask instance-level settings, using Accuracy, Precision, Recall, F1 score, and Yes ratio.Scores average five randomly sampled subsets with a 1:1 positive-negative ratio for each setting.Source: §4.1 · p. 5, §4.2 · p. 6

4.2. Evaluating on the R-Bench

  • LVLMs answer image-level questions well but fail to discriminate relationships between reference objects in instance-level settings.This occurs despite good reference-object recognition; fine-grained image-text alignment might help.Source: §4.2 · p. 6

4.3. Analysis of the Causes of Hallucinations

  • Frequently co-occurring relationships, such as “man swings bat” and “man standing in dirt,” are more likely to trigger relationship hallucinations.One example is LLaVA incorrectly describing a person holding a hair dryer as drying her hair.Source: §4.3 · p. 7
  • Subject-behavior associations can bias answers: LLaVA says a cat in a soccer uniform is kicking a ball when asked about a “soccer player,” but answers correctly when asked about the cat.The paper describes these subject-relationship hallucinations as widespread in LVLMs.Source: §4.3 · pp. 7–8
  • LVLMs confuse “riding bike” with “walking bike” more readily than they distinguish whether the interacted-with object is a bike or motorbike.COCO captions average 3.6 object words but 1.3 relationship words; 97,522 captions omit relationship words, versus 419 omitting object words.Source: §4.3 · p. 8

4.4. Other Relationship Hallucinations

  • Qwen-VL reaches only 50% counterfactual accuracy, while InstructBLIP's best illusion accuracy is 25%, indicating failures in visual grounding and contextual spatial reasoning.The counterfactual example involves mistaking a one-wheeled motorcycle for a conventional two-wheeled one; the illusion example involves inferring interaction between spatially dislocated people.Source: §4.4 · pp. 8–9, Table 6 · p. 9

5. Limitations

The paper limits its scope to analyzing relationship hallucinations in LVLMs and notes constraints in causal coverage, question categorization, data filtering, and model evaluation.

5. Limitations

  • The analysis covers only a subset of relationship-hallucination causes, and mitigation strategies are deferred to future work.The authors call for more comprehensive future investigation.Source: §5 · p. 9
  • A lack of an effective classification standard limits fine-grained categorization of relationship-hallucination questions.The authors state that improved categorization could enhance LVLM hallucination analysis.Source: §5 · p. 9
  • Manual filtering may leave noise because of omissions and biases, and budget constraints prevented evaluation of some powerful closed-source LVLMs, including GPT-4.Source: §5 · p. 9

6. Conclusion

The paper introduces R-Bench to evaluate relationship hallucinations in LVLMs and analyzes phenomena found with the benchmark. Its findings are intended to inspire further work on solutions.

6. Conclusion

  • R-Bench combines image-level questions about relationship existence with instance-level questions evaluating local visual comprehension.Source: §6 · p. 9
  • The authors analyze existing LVLMs on R-Bench and report several relationship-hallucination phenomena.Source: §6 · p. 9
  • The authors anticipate that their findings will inspire research into solutions for LVLM relationship hallucinations.Source: §6 · p. 9

A. Implementation Detail of LVLMs

The evaluation used the largest versions of several popular LVLMs that the authors could run, with low-bit quantization to avoid GPU memory exhaustion.

A. Implementation Detail of LVLMs

  • The authors selected the largest versions of several popular LVLMs they could run and applied low-bit quantization during inference to avoid GPU memory exhaustion.Source: A. Implementation Detail of LVLMs · p. 13
  • Table 7 lists the evaluated models and parameters of their core components.Source: Table 7 · p. 13

B. Additional Results of LVLMs on our R-Bench

Additional R-Bench results are reported for nocaps in-domain, near-domain, and out-domain subsets, using repeated balanced sampling.

B. Additional Results of LVLMs on our R-Bench

C. Impact of Color Selection.

The color-selection experiments compare red and green with swapped colors and gold and pink. Swapping red and green has little effect, while gold and pink reduce performance for most tested models.

C. Impact of Color Selection.

D. Examples of Instance-level Relationship Hallucination

The examples contrast LLaVA's performance on image-level and instance-level relationship questions and visualize reference recognition, caption coverage, and hallucination types.

D. Examples of Instance-level Relationship Hallucination

E. Visualization Examples

  • Figure 10 illustrates reference-object recognition using bounding boxes and masks.Source: E. Visualization Examples · p. 13
  • The paper presents visualization examples: Figure 11 shows a COCO image with captions, while Figure 12 shows examples of counterfactual relationship hallucination and illusion relationship hallucination.Source: E. Visualization Examples · p. 13

F. Data Static

This section presents visualizations of relationship frequencies, model hallucination frequencies, benchmark results, and examples of reference recognition and relationship hallucinations.

F. Data Static

  • Figures 13 and 14 show relationship frequencies in the questions and relationship-hallucination frequencies for LVLMs across image-level and instance-level questions.Source: F. Data Static · p. 13, Figure 13 · p. 17, Figure 14 · p. 18
  • Table 8 reports R-Bench results averaged across five random subsets, each with a 1:1 positive-to-negative question ratio.Its box and mask labels distinguish instance-level question types.Source: Table 8 · p. 14
  • Table 9 presents the impact of color selection.Source: Table 9 · p. 14
  • Figure 9 marks incorrect answers in red and correct answers in green when illustrating image-level and instance-level relationship hallucination.Source: Figure 9 · p. 15, F. Data Static · p. 15
  • Figure 10 compares bounding-box and mask references; its caption says masks provide more accurate references and the LVLM responds correctly using objects in different forms.Source: Figure 10 · p. 15
  • Figures 11 and 12 illustrate caption examples with missing or unbalanced relationship descriptions and examples of counterfactual and illusion relationship hallucinations.Source: Figure 11 · p. 16, Figure 12 · p. 16
Loading 2406.16449v4…