PaperlayerView exact revision on arXiv

Source-linked AI summary

From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models

Yuying Shang, Xinyi Zeng, Yutao Zhu, Xiao Yang, Zhengwei Fang, Jingyuan Zhang, Jiawei Chen, Zinan Liu, Yu Tian

arXiv:2410.06795v2cs.CLcs.CV

TL;DR

The paper asks whether LVLM object hallucinations primarily reflect deficient visual feature extraction or inadequate cross-modal alignment, a question relevant to their reliability. It investigates these sources and proposes PATCH, a virtual-token tuning strategy. PATCH improves POPE accuracy across three LVLMs, while the paper reports a Cognition-task performance decrease as a scope limitation.

  • Problem

    The primary cause of LVLM object hallucinations remains unclear: they may arise from insufficient visual-feature extraction or inadequate cross-modal alignment.

    Source: §1 · p. 1

  • Method

    The paper compares visual-encoder extraction with modal-alignment decoupling and proposes PATCH, which uses trainable virtual tokens to connect image features and object-augmented prompts.

    Source: §1 · pp. 1–2

  • Results

    On POPE, PATCH raises accuracy from 85.17% to 90.20% for LLaVA-v1.5, 57.67% to 88.13% for MiniGPT-4, and 83.33% to 90.03% for MiniGPT-v2.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The preliminary analysis identifies inadequate cross-modal alignment rather than visual-encoder limitations as the primary source of object hallucinations.

    Source: §1 · p. 2

    PATCH exhibits an 8.21-point performance decrease on the Cognition task, and the authors note it was fine-tuned only on the POPE (AOKVQA-version) dataset.

    Source: §5.6 · p. 8

Abstract

from arXiv · show

Hallucinations in large vision-language models (LVLMs) are a significant challenge, i.e., generating objects that are not presented in the visual input, which impairs their reliability. Recent studies often attribute hallucinations to a lack of understanding of visual input, yet ignore a more fundamental issue: the model's inability to effectively extract or decouple visual features. In this paper, we revisit the hallucinations in LVLMs from an architectural perspective, investigating whether the primary cause lies in the visual encoder (feature extraction) or the modal alignment module (feature decoupling). Motivated by our findings on the preliminary investigation, we propose a novel tuning strategy, PATCH, to mitigate hallucinations in LVLMs. This plug-and-play method can be integrated into various LVLMs, utilizing adaptive virtual tokens to extract object features from bounding boxes, thereby addressing hallucinations caused by insufficient decoupling of visual features. PATCH achieves state-of-the-art performance on multiple multi-modal hallucination datasets. We hope this approach provides researchers with deeper insights into the underlying causes of hallucinations in LVLMs, fostering further advancements and innovation in this field.

1 Introduction

The paper investigates whether object hallucinations arise from visual feature extraction or cross-modal feature alignment. Its analysis identifies inadequate alignment as the primary source and motivates PATCH, a virtual-token tuning strategy evaluated across multiple models and datasets.

1 Introduction

  • The paper asks what primarily causes multi-modal hallucinations despite prior mitigation work targeting data distribution, training schemes, and decoding strategies.The motivation is that hallucinations hinder LVLM practical applications by producing factually incorrect or fictitious outputs.Source: §1 · p. 1
  • The analysis identifies insufficient cross-modal alignment, rather than visual-encoder limitations, as the primary source of object hallucinations.Augmenting the LVLM input sequence with object-related information improves resistance to hallucinations.Source: §1 · p. 2
  • PATCH uses trainable, pluggable virtual tokens between image features and enhanced prompt texts to align visual and textual features with minimal parameter tuning.At inference, fine-tuned token embeddings are added to the original vocabulary, making the method plug-and-play across application scenarios.Source: §1 · p. 2
  • On POPE, PATCH raises accuracy from 85.17% to 90.20% for LLaVA-v1.5, 57.67% to 88.13% for MiniGPT-4, and 83.33% to 90.03% for MiniGPT-v2.The evaluation covers three publicly available multi-modal hallucination datasets and three LVLMs.Source: §1 · p. 2

2 Related Work

LVLMs combine visual encoders, language models, and cross-modal alignment mechanisms, yet can generate objects absent from images. Prior work studies several possible sources and mitigation strategies; this paper revisits hallucinations architecturally and proposes parameter-efficient PATCH tuning.

2.1 Large Vision-Language Models

  • LVLMs pair a visual encoder for feature extraction with a language model, using cross-modal mechanisms such as MLPs, attention, or Q-Former to integrate visual and textual embeddings.Despite strong performance on image captioning, visual question answering, and reasoning, LVLMs can describe objects absent from the input image.Source: §2.1 · p. 2

2.2 Object Hallucinations in LVLMs

  • Prior accounts link hallucinations to visual comprehension [34], cross-modal representation misalignment [17], and beam-search uncertainty [47].Mitigation approaches target data distribution, training paradigms, human feedback, and decoding, while the paper presents PATCH as parameter-efficient architectural tuning.Source: §2.2 · pp. 2–3

3 Analysis of Object Hallucinations in LVLMs

The paper tests whether hallucinations reflect poor visual feature extraction or inadequate alignment between visual features and the LLM’s semantic space. In preliminary experiments, accurate object detections often coexist with incorrect inferences, while adding object details to prompts improves object-existence answers and motivates PATCH’s selective use of detection information.

3.1 Potential Sources of Hallucination

  • The analysis hypothesizes two sources of hallucination: insufficient visual-feature extraction and inadequate decoupling of visual features into the LLM’s semantic space.The experiments assess the visual encoder and visual projection layer as possible sources.Source: §3.1 · p. 3, §3 · p. 3

3.2 Empirical Analysis

  • POPE evaluates 3,000 object-presence samples as yes/no questions, and the preliminary MiniGPT-v2 experiment compares object detections with inference outcomes using a Cascade Mask R-CNN head.A hallucination includes answering yes for an absent object or no for a present object; disagreement between detections and inference is treated as hallucination.Source: §3.2 · p. 3
  • 413 MiniGPT-v2 samples, or 13.77% of the dataset, exhibit hallucinations; 74.58% of these occur despite accurate detection and incorrect inference.This pattern points to difficulty bridging encoded image features with the LLM’s semantic space rather than a failure to extract accurate features.Source: §3.2 · p. 3
  • Adding detected object categories and bounding boxes to Prompt2 significantly improves MiniGPT-v2 performance on object-existence questions compared with Prompt1.Because weakly question-related detections can add redundant detail and hinder reasoning, PATCH uses trainable virtual tokens to help the LLM selectively leverage detection information.Source: §3.2 · p. 3

4 Methodology

The methodology formulates LVLM response generation from image and prompt inputs, then adds object categories and bounding boxes as context for object-hallucination recognition. PATCH inserts trainable virtual tokens between image features and detection information, while keeping the visual encoder, projection layer, and LLM frozen during fine-tuning.

4.2 PATCH

  • During fine-tuning, only virtual-token parameters are updated; the visual encoder, linear projection layer, and LLM remain frozen.The architecture uses a frozen pre-trained Cascade Mask R-CNN head to obtain object-related information at test time.Source: Figure 1 · p. 4, §4.2 · p. 4

4.1 Problem Formulation

  • LVLMs decode fused visual and textual representations to generate answers, with object categories and bounding boxes added as context for assessing hallucinated objects.The task evaluates whether a queried object appears in the image.Source: §4.1 · p. 4

4.2 PATCH

  • PATCH inserts trainable virtual tokens between image features and detection information to bridge encoded images and augmented prompts.The token embeddings are optimized during training.Source: §4.2 · p. 4
  • At inference, PATCH can add special tokens before detection results when object information is supplied, or revert to standard LVLM processing without it.The special-token embeddings are initialized from the fine-tuned virtual-token embeddings.Source: §4.2 · p. 4

5 Experiments

Experiments evaluate PATCH on POPE and PhD using three LVLM backbones, with ablations and transfer tests probing its components and scope. PATCH improves object-hallucination performance, while results also show sensitivity to token quantity, initialization, and task type.

5.1 Datasets

  • POPE uses 3,000 adversarial samples from A-OKVQA and MSCOCO, while PhD v1 evaluates five object-related task types with an 80/20 train-test split.The experiments use LLaVA-v1.5, MiniGPT-4, and MiniGPT-v2; MiniGPT-v2 is used for most experiments, with Accuracy, Precision, Recall, and F1 as metrics.Source: §5.1 · p. 5, §5.2 · p. 5

5.3 Baselines

  • The experiments compare PATCH with mainstream LVLMs and hallucination-mitigation baselines, including HA-DPO, Woodpecker, and HACL.PATCH uses fine-tuning, while Woodpecker is described as training-free and based on manually crafted prompts.Source: §5.3 · p. 5, §5.4 · p. 6

5.4 Experimental Results

  • Across PhD task types, PATCH improves performance, including where Hard Prompt underperforms; under misleading contexts, PATCH also remains effective on strongly misleading questions.Hard Prompt improves object recognition, counting, and positional reasoning but can hurt attribute recognition and sentiment analysis; its added detection text may be irrelevant to those questions.Source: §5.4 · p. 6, Figure 2 · p. 6

5.5.1 Impact on Object-related Information.

  • Ablations indicate that complete detection information performs best, bounding boxes aid localization, and virtual-token placement between image features and object information matters.Removing bounding boxes lowers accuracy by 1.76%; location-only reasoning with generic object placeholders is ineffective, while PATCH fine-tuned with location information partially mitigates hallucinations.Source: Table 4 · p. 6, Table 5 · p. 6, §5.5.1 · p. 7

5.5.3 Impact on the Initialization Strategy of Virtual Tokens.

  • The cited table reports ablation experimental results for different virtual-token initializations on the POPE dataset, without specifying an outcome or metric values in the provided passage.Source: Table 7 · p. 7

5.5.2 Impact on Token Quantities.

  • On POPE, accuracy is optimal with 20 virtual tokens, improving 6.70% over baseline; more tokens can add redundancy and increase tuning costs.Without object-related detection information, simply increasing trainable token quantity is insufficient to mitigate hallucinations.Source: §5.5.2 · p. 7, Figure 3 · p. 7, Table 6 · p. 7

5.6 Further Analysis

  • PATCH exceeds MiniGPT-v2 by 34.55 points on MME Perception, but falls 8.21 points on Cognition after fine-tuning only on POPE AOKVQA-version.The study also reports a 4.92% accuracy improvement over baseline when transferring to the full PhD dataset without additional fine-tuning.Source: Table 8 · p. 8, §5.6 · p. 8, Table 9 · p. 8

6 Conclusion

The paper attributes object hallucinations primarily to insufficient cross-modal alignment and presents PATCH as a parameter-efficient strategy evaluated across three hallucination benchmarks.

6 Conclusion

  • PATCH uses learnable virtual tokens to bridge image features, object-augmented prompts, and questions, and is designed as a plug-and-play method for diverse applications.Experiments on three publicly available benchmarks report effectiveness against misleading contextual interference and object hallucinations.Source: §6 · p. 8

A Datasets

The appendix describes POPE as a binary object-hallucination testbed and PhD as a benchmark covering multiple hallucination types and object-related tasks.

A Datasets

  • POPE uses present objects as positive samples and frequently co-occurring absent objects as negative samples, creating a binary test of object-hallucination susceptibility.Its 3,000 adversarially constructed samples come from A-OKVQA and MSCOCO.Source: A Datasets · p. 10
  • PhD covers object and attribute, multimodal-conflict, and counter-common-sense hallucinations; these experiments use 35,033 samples from the first two types.Object-related tasks include recognition, attribute recognition, counting, positional reasoning, and sentiment analysis; conflict levels range from neutral to strong misleading.Source: A Datasets · p. 10

B Implementation Details

Implementation settings vary by model and dataset, including learning rates, batch sizes, training durations, and weight decay.

B Implementation Details

  • MiniGPT-4 POPE training uses a 1e-4 initial learning rate, 0.05 weight decay, 30 epochs, and batch size 1.Training takes 2 hours; weight decay is set until the learning rate decays to 1e-5.Source: B Implementation Details · p. 10
  • MiniGPT-v2 POPE training uses a 2e-3 initial learning rate, batch size 1, and 30 epochs; PhD training uses 1e-3, batch size 1, and 35 epochs.Training takes 2 hours on POPE and 2.5 hours on PhD; weight decay is 0.05 until the learning rate decays to 5e-6.Source: B Implementation Details · p. 10
  • LLaVA-v1.5 POPE training uses an 8e-4 initial learning rate, batch size 16, and one epoch, completing in under 20 minutes.Source: B Implementation Details · p. 10

C Attention Map for the Final Token in the Input Sequence

The attention visualization indicates that queried-object embeddings receive higher scores when the object appears in detection information.

C Attention Map for the Final Token in the Input Sequence

  • When the queried object is included in detection information, its embeddings receive higher attention scores for the final input token across LLM layers.The passage interprets this pattern as evidence that virtual tokens enhance semantic alignment.Source: C Attention Map for the Final Token in… · p. 10

D Analysis of the Detection Head

The analysis compares detection and inference results after replacing Prompt1 with Prompt2, finding consistent error behavior and that adding object information mitigates hallucinations. It also attributes remaining errors to detector limitations.

D Analysis of the Detection Head

  • POPE adversarial examples pair six questions with each image: three positive samples and negatives formed by replacing each positive object with a frequently co-occurring absent object.Source: Figure 5 · p. 11
  • The PhD dataset examples show question-answer pairs across five task types and across different conflict levels.Source: Figure 6 · p. 11, Figure 7 · p. 11
  • Under Prompt2, adding object information to MiniGPT-v2 mitigates hallucinations, and detection and inference errors behave consistently (Table 10).Table 10 compares sample counts in detection and inference results conditioned on Prompt2.Source: D Analysis of the Detection Head · pp. 10–11, Table 10 · p. 11
  • The authors attribute remaining errors to detector limitations and suggest a more advanced detector could further improve performance.Source: D Analysis of the Detection Head · p. 11
  • Figure 8 visualizes attention scores across LLM layers based on PATCH.Source: Figure 8 · p. 11

E Limitations

PATCH depends on accurate object detections, and repeated instances of the same object can increase LVLM input length. The authors identify detection prompting as an area for future refinement.

E Limitations

  • PATCH relies on precise object detections; the authors use Cascade Mask R-CNN based on EVA’s configuration and note that improved detectors may strengthen robustness and effectiveness.Source: E Limitations · p. 11
  • Repeated instances of the same object can produce duplicate categories, unnecessarily increasing LVLM input length; the authors plan to simplify detection prompts for complex real-world scenarios.Source: E Limitations · p. 11

Related papers

Loading 2410.06795v2…