PaperlayerView exact revision on arXiv

Source-linked AI summary

Mitigating Hallucination in Visual Language Models with Visual Supervision

Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jinqiao Wang, Ming Tang

arXiv:2311.16479v1cs.CV

TL;DR

LVLMs can hallucinate because they may not capture detailed multimodal context, while common instruction data and next-token loss provide limited visual-relation supervision. The paper combines PSG-based relation instructions with SAM mask supervision and evaluates hallucination by type; LLaVA-13B’s RAH-Bench F1 score improves by +8.4%.

  • Problem

    LVLMs can generate responses contrary to image content, while global-description instruction data and next-token loss provide limited support for detailed visual relationships and spatial understanding.

    Source: §1 · pp. 1–2

  • Method

    The paper trains LVLMs with PSG-based relation-focused instructions and SAM mask prediction supervision, and proposes RAH-Bench to assess category, attribute, and relation hallucinations.

    Source: §1 · p. 2

  • Results

    +8.4% F1 score for LLaVA-13B on RAH-Bench, with reported improvements across different evaluation settings and for mPLUG-owl.

    Source: §4.1 · p. 6

  • Takeaways & Limitations

    The authors conclude that fine-grained instruction data and expert vision-model supervision could help mitigate vision hallucination in different LVLMs.

    Source: §5 · p. 8

    The dataset is not large, and the training approach is somewhat coupled to RAI-30k’s subject-and-object relation format.

    Source: §5 · p. 8

Abstract

from arXiv · show

Large vision-language models (LVLMs) suffer from hallucination a lot, generating responses that apparently contradict to the image content occasionally. The key problem lies in its weak ability to comprehend detailed content in a multi-modal context, which can be mainly attributed to two factors in training data and loss function. The vision instruction dataset primarily focuses on global description, and the auto-regressive loss function favors text modeling rather than image understanding. In this paper, we bring more detailed vision annotations and more discriminative vision models to facilitate the training of LVLMs, so that they can generate more precise responses without encounter hallucination. On one hand, we generate image-text pairs with detailed relationship annotations in panoptic scene graph dataset (PSG). These conversations pay more attention on detailed facts in the image, encouraging the model to answer questions based on multi-modal contexts. On the other hand, we integrate SAM and mask prediction loss as auxiliary supervision, forcing the LVLMs to have the capacity to identify context-related objects, so that they can generate more accurate responses, mitigating hallucination. Moreover, to provide a deeper evaluation on the hallucination in LVLMs, we propose a new benchmark, RAH-Bench. It divides vision hallucination into three different types that contradicts the image with wrong categories, attributes or relations, and introduces False Positive Rate as detailed sub-metric for each type. In this benchmark, our approach demonstrates an +8.4% enhancement compared to original LLaVA and achieves widespread performance improvements across other models.

1. Introduction

LVLMs can produce responses that contradict image content because they lack detailed multimodal understanding. This paper pairs fine-grained relation-focused instruction data with visual mask supervision and introduces a benchmark for more detailed hallucination evaluation.

1. Introduction

  • LVLMs may answer text questions correctly yet hallucinate when responding to images, reflecting weak understanding of multimodal context.The paper calls inconsistency between a response and image content vision hallucination.Source: §1 · pp. 1–2
  • Global, mostly positive instruction data and next-token prediction provide limited coverage of misleading questions and do not explicitly model visual relationships or spatial regions.These limitations make image-grounded answers to specific questions difficult to guarantee.Source: §1 · p. 2
  • RAI-30k uses PSG relation annotations and associated instance masks to focus instruction data on detailed relations among image instances.SAM generates masks for instruction-associated instances, and mask prediction loss encourages the LVLM to attend to relevant image content during training.Source: §1 · p. 2
  • RAH-Bench evaluates category, attribute, and relation hallucinations with type-specific sub-metrics.It uses interrogative sentences paired with images and asks whether detailed descriptions match image content.Source: §1 · p. 2

2. Related Works

Prior work builds LVLMs, combines them with vision models, and addresses hallucination through text-oriented approaches. This paper emphasizes fine-grained visual instruction data, visual training supervision, and more detailed hallucination evaluation.

2.1. Large Vision-Language Models

  • Existing instruction datasets often use straightforward construction that yields positive conversations and global descriptions, whereas this work focuses on image-specific details for fine-grained conversations.The dataset discussion uses relation-focused examples, including questions about specific image facts.Source: §2.1 · p. 3

2.2. Combine LVLMs and Vision Models

  • Unlike approaches that append vision models or add functions to the system, this method uses existing vision models and annotations to enhance LVLMs’ text responses.Prior work includes command-based model use and joint tuning, while the authors say those methods pay little attention to LVLM abilities themselves.Source: §2.2 · p. 3

2.3. Hallucination in LVLMs

  • Current LVLM hallucination-mitigation approaches enrich context with additional text inputs [32], construct high-quality instruction datasets with negative samples [16], or use iterative refinements [28]; this work suggests visual-task models and loss functions could facilitate LVLM training.Source: §2.3 · p. 3
  • RAH-Bench aims to combine diverse evaluation with easier assessment and detailed sub-metrics for specific LVLM vulnerabilities.The paper contrasts it with POPE’s consistent question structure and LRV-Instruction’s GPT-4 scoring, which it describes as costly and unstable.Source: §2.3 · p. 3

3. Method

The method combines relation-focused instruction data with training-time visual mask supervision and evaluates hallucination using a benchmark that separates three error types.

3.1. Relation-Associated Instruction Dataset

  • RAI-30k uses PSG subject–object relations to generate focused question–answer conversations, with 29,712 samples containing relationship annotations and binary masks.The dataset also appends multi-round conversations from LLaVA-Instruct 80k.Source: §3.1 · p. 4

3.2. Supervision with Visual Annotations

  • During instruction tuning, LVLM features for relation subjects and objects prompt SAM to predict masks supervised by the PSG masks.Special [SUB] and [OBJ] tokens extract the instance features; SAM is discarded after training, leaving inference unchanged.Source: §3.2 · pp. 4–5

3.3. Relation-Associated Hallucination Benchmark

  • RAH-Bench contains 3,000 yes-or-no questions and separates negative questions into category, attribute, and relation hallucinations, with a false-positive rate for each type.Its questions combine true and misleading information, and its type-specific sub-metrics assess susceptibility to different misleading queries.Source: §3.3 · p. 5

4. Experiments

Across models, the method improves hallucination-related results, while ablations report gains from richer relation-focused data, broader conversations, and mask supervision. Visualizations show masks attending to question-relevant content.

4.1. Main Results

  • LLaVA-13B’s RAH-Bench F1 improves by +8.4%; its POPE F1 improves by +10.1%, +7.4%, and +3.9% across different evaluation settings.mPLUG-owl’s F1 increases by +2.3%.Source: §4.1 · p. 6
  • InstructBLIP is especially vulnerable to relation hallucination; tuning preserves random and popular POPE performance while improving adversarial POPE and RAH-Bench results.For 7B/13B, gains are +0.8%/+4.6% on adversarial POPE and +0.5%/+4.0% on RAH-Bench.Source: §4.1 · p. 6

4.2. Ablation on Data Construction

  • Adding detailed COCO captions and Visual Genome region descriptions as GPT-4 context improves RAH-Bench F1 by +6.9%, with False Positive Rates generally decreasing across hallucination types.Using only simple PSG category names degrades performance.Source: §4.2 · p. 6
  • More diverse yes/no and wh-question prompts produce slightly higher F1 scores on every POPE split.Source: §4.2 · p. 7
  • Adding LLaVA-Instruct-80k multi-round conversations improves RAH-Bench by +6.2% and yields a more modest improvement on POPE.The generated questions focus mainly on specific relationships, unlike LLaVA’s broader conversations.Source: §4.2 · p. 7

4.3. Ablation on Vision Supervision

  • SAM and auxiliary mask prediction loss add 1.5% F1 on RAH-Bench over conventional RAI-30k tuning; excessive epochs risk overfitting the binary task and impairing general ability.One epoch is the default because it does not much affect LLaVA’s general capacities.Source: §4.3 · p. 7
  • Tuning significantly improves detailed-description ability and modestly improves complex reasoning, with the authors associating these gains with fine-grained data and mask supervision.Source: §4.3 · p. 7

4.4. Visualization

  • Figure 5 visualizes attended masks: relevant instances are identified for accurate descriptions, while misleading prompts expose contradictions or yield low scores across the output mask; inference remains ordinary next-token generation.Source: §4.4 · pp. 7–8, Figure 5 · p. 8

5. Conclusion

The paper presents RAI-30k and expert-model loss supervision to mitigate vision hallucination, and introduces RAH-Bench for evaluating its types. It also notes limits in dataset size and training-format flexibility.

  • RAI-30k pairs fine-grained image instructions with relation annotations, while expert-model loss supervision is used to mitigate hallucination in LVLMs.Source: §5 · p. 8
  • RAH-Bench evaluates categorized types of vision hallucination in LVLMs.Source: §5 · p. 8
  • The dataset is relatively small, and the training approach is coupled to RAI-30k’s subject-object relation format.The authors suggest expanding the dataset from more seed annotations and broadening support for annotation formats.Source: §5 · p. 8

Related papers

Loading 2311.16479v1…