PaperlayerView exact revision on arXiv

Source-linked AI summary

Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs

Shi Liu, Kecheng Zheng, Wei Chen

arXiv:2407.21771v1cs.CV

TL;DR

LVLMs may generate identical hallucinated descriptions with and without visual input because language context can dominate image information. PAI is a training-free inference intervention that amplifies image-token attention and subtracts text-only logits, and experiments report reduced hallucination across multiple benchmarks and models.

  • Problem

    LVLMs can produce text that contradicts images, including identical hallucinated descriptions when visual input is removed, indicating Text Inertia and image-token neglect.

  • Method

    PAI edits decoder self-attention to amplify image-token weights and subtracts logits from a pure-text input during decoding.

  • Results

    PAI mitigates hallucinations across multiple benchmarks, LVLMs, tasks, and decoding methods.

  • Takeaways & Limitations

    The results support training-free inference intervention as a way to steer LVLM generation toward image-based and more trustworthy descriptions.

Abstract

from arXiv · show

Existing Large Vision-Language Models (LVLMs) primarily align image features of vision encoder with Large Language Models (LLMs) to leverage their superior text generation capabilities. However, the scale disparity between vision encoder and language model may led to LLMs assuming a predominant role in multi-modal comprehension. This imbalance in LVLMs may result in the instances of hallucinatory. Concretely, LVLMs may generate consistent descriptions with or without visual input, indicating that certain outputs are influenced solely by context text. We refer to this phenomenon as "text inertia." To counteract this issue, we introduce a training-free algorithm to find an equilibrium point between image comprehension and language inference. Specifically, we adaptively involve adjusting and amplifying the attention weights assigned to image tokens, thereby granting greater prominence to visual elements. Meanwhile, we subtract the logits of multi-modal inputs from ones of pure text input, which can help LVLMs be not biased towards LLMs. By enhancing images tokens and reducing the stubborn output of LLM, we can let LVLM pay more attention to images, towards alleviating text inertia and reducing the hallucination in LVLMs. Our extensive experiments shows that this method substantially reduces the frequency of hallucinatory outputs in various LVLMs in terms of different metrics. Project page is available at https://lalbj.github.io/projects/PAI/.

1 Introduction

LVLM hallucinations can persist from textual context even without an image, revealing “Text Inertia” and insufficient attention to visual tokens. PAI addresses this imbalance during inference by amplifying image-token attention and reducing language-prior influence.

  • LVLM hallucinations mismatch generated text with the actual visual input.
  • “Text Inertia” occurs when an LVLM reproduces the same hallucinated description from historical response text after image removal.
  • Image tokens receive insufficient attention despite occupying a significant portion of the input, making multimodal dialogue resemble context-based completion.
  • PAI is a training-free inference intervention that amplifies image-token attention and subtracts logits from an instruction-and-history-only input.
  • PAI is evaluated across CHAIR, GPT-4V, POPE, and MMHal-Bench using three decoding methods across three models.

2 Related Work

LVLM hallucination mitigation has used alignment training, architectural changes, post-processing, external tools, and decoding interventions. These approaches can improve outputs but may require substantial computation, extra inference stages, or specialized decoding procedures.

  • Alignment-based mitigation addresses data bias and vision-language knowledge gaps through data filtering, annotation, training, or architecture changes.
  • Post-processing methods detect and regenerate hallucinated content using additional modules or external visual tools.
  • External-tool and post-processing approaches extend the inference chain and increase inference costs.
  • Training-free mitigation had primarily been explored through decoding methods such as OPERA and VCD.

3 Preliminaries

LVLMs transform images into visual tokens, map them into the language representation space, concatenate them with text, and decode responses autoregressively through self-attention.

  • An LVLM contains an image encoder, projector, and language decoder that convert images into visual tokens for language generation.
  • Projectors transform N image-encoder features into M visual tokens, with linear projectors preserving one-to-one mappings and resamplers selecting fewer cues.
  • LLaMA-family decoders concatenate projected visual tokens with text tokens and process them using autoregressive self-attention.
  • Each attention head computes weights with softmax(Q_hK_h^T/√d_k), using queries, keys, and values across the sequence.
  • Attention-weighted value mixing produces hidden states that determine the vocabulary distribution for each generated token.
  • Sequence generation repeats token-level prediction until an EOS token ends the response.

4 Method

PAI intervenes during autoregressive decoding to increase attention to image tokens and suppress predictions attributable to text-only language priors. It applies these adjustments through self-attention editing and logit subtraction.

  • 4 Method: PAI targets image neglect and Text Inertia by augmenting image attention while reducing reliance on language priors.
  • 4 Method: At each decoding step, PAI extracts attention matrices from every head and layer to represent content influence during inference.
  • 4.1 Pay More Attention to Image: The decoder input combines instruction representations, projected image representations, and historically generated response representations.
  • 4.1 Pay More Attention to Image: PAI increases attention values assigned to image tokens before softmax, then redistributes token attention for hidden-state computation.
  • 4.1 Pay More Attention to Image: Intervention strength is reduced for less-trustworthy heads with lower image attention, rather than selecting only a fixed top-k set.
  • 4.1 Pay More Attention to Image: PAI uses hidden-state similarity to determine intervention timing because attention-sink behavior emerges after semantically rich token encodings stabilize.
  • 4.2 Mitigating Language Prior: The decoding update combines image-conditioned and text-only distributions, with γ controlling the penalty applied to text-based predictions.
  • 4.2 Mitigating Language Prior: This guided generation balances outputs supported by image content against outputs supported by language logic.

5 Experiments

PAI is evaluated across multiple hallucination benchmarks, LVLMs, and decoding methods, with experiments covering long-sequence descriptions, VQA, comprehensive scenarios, and GPT-4V-assisted assessment. Ablations examine image-attention amplification, layer prioritization, and language-prior mitigation.

  • Setup: The experiments evaluate three LVLMs using CHAIR, POPE, MMHal-Bench, and GPT-4V-assisted assessments across image description and VQA tasks.The models include LLAVA and Shikra with linear projectors and Minigpt4 with a resampler; evaluations include single-turn and multi-turn POPE.
  • Benchmark & Evaluation Metrics: CHAIR measures hallucinated objects at instance and sentence levels, with lower values indicating fewer hallucinations.The metric compares mentioned objects with ground-truth object labels for each image.
  • Experimental Results: PAI mitigates hallucinations across three decoding techniques and three models, although its reduction is not significant with nucleus sampling.The authors attribute the nucleus-sampling limitation to hallucination tokens remaining in the sample set.
  • Experimental Results: PAI improves single-turn POPE performance over vanilla decoding and achieves a more significant improvement in multi-turn evaluation with longer context.The reported evaluation uses brief Yes-or-No VQA responses, where text inertia and image neglect may be less noticeable in single-turn settings.
  • Experimental Results: On MMHal-Bench, PAI improves accuracy for image-based categories and raises the overall average across eight evaluation dimensions, but shows no noticeable gain for comparison and relation questions.The image-based categories include object attributes, adversarial objects, and holistic questions.
  • Ablation Study: An appropriate image-attention amplification scale balances hallucinated-object count against information conveyed, while layer priors improve performance and γ values around 1.1–1.2 provide stable behavior.Excessive amplification reduces response information; removing layer control causes regressions or fluctuations, and Minigpt4 is highly sensitive to γ.

6 Conclusion and Limitation

The paper identifies text inertia and image-token neglect as causes of hallucination, introduces training-free PAI, and reports effectiveness across benchmarks and LVLMs. It notes limitations involving LLaMA-family decoders and dependence on the underlying model’s capabilities.

  • 6 Conclusion and Limitation: PAI is a training-free inference intervention that steers LVLMs toward image-based generation by addressing image-token neglect and text inertia.The method intervenes during inference without external tools.
  • 6 Conclusion and Limitation: Extensive experiments across multiple benchmarks and LVLMs validate PAI’s effectiveness in mitigating hallucination.
  • 6 Conclusion and Limitation: The analysis leaves open whether LLaMA-family language decoders introduce image neglect and text inertia in existing open-source LVLMs.
  • 6 Conclusion and Limitation: PAI’s upper limit depends on the capabilities of the well-trained model, motivating future training objectives that incorporate image neglect.

A Text Inertia Detection Process

The text inertia detection process identifies hallucinated objects, removes the image, and tests whether the model reproduces the same descriptions from preceding response text. A prompt-based evaluation records whether the model confirms the phenomenon.

  • A Text Inertia Detection Process: The process first uses CHAIR to identify hallucinated-object indices in generated descriptions.
  • A Text Inertia Detection Process: It then extracts tokens preceding each hallucinated object and regenerates text using only the task instruction and response history, excluding the image.
  • A Text Inertia Detection Process: The detection prompt substitutes each extracted hallucinated object and the first ten tokens of the conditioned description into a test template.
  • A Text Inertia Detection Process: A “Yes” response indicates that the text inertia phenomenon is present.

B What if Model Scales Up?

Experiments on LLaVA-1.5-13B examine PAI across increasing model scale. The method remains robust and continues reducing hallucination in both long-sequence and short-VQA tasks.

  • B What if Model Scales Up?: PAI maintains robustness as model scale increases.
  • B What if Model Scales Up?: PAI continues alleviating hallucination in both long-sequence generation and short VQA tasks as model scale increases.

C Comparative Experimental Results

PAI is compared with uniformly adding an attention value to all image tokens, while self-attention visualizations examine object-token attention with and without PAI. The reported results favor adaptive enhancement based on the original attention scale.

  • C Comparative Experimental Results: The comparison baseline adds a constant value of 1.0 to the attention mask of all image tokens.This automation follows Prompt Highlighter’s intervention approach and its open-source default.
  • C Comparative Experimental Results: Uniformly intervening across the entire image is less effective than enhancing image attention according to its original scale.
  • C Comparative Experimental Results: Self-attention maps visualize each object token with and without PAI over LLaVA, highlighting hallucinated objects in red.

D Obtaining Explainable LVLMs.

PAI makes LVLM explanations more image-centric by intervening in decoder attention, but the available attention-based explanations depend on how image features are represented. Qualitative and supplementary evaluations illustrate this intervention across several LVLMs.

  • Attention-based explanations are directly available for LLaVA and Shikra because their linear projections preserve image-patch token features.Models using resamplers require reverse-engineering of abstracted representations for explanation.
  • PAI intervenes in decoder layers to make generated descriptions more image-centric.The method updates attention toward image content during inference, producing descriptions that align better with image locations.
  • PAI’s supplementary figures report hallucination-reduction performance for LLaVA-1.5-7B, MiniGPT-4, and Shikra.
Loading 2407.21771v1…