PaperlayerView exact revision on arXiv

Source-linked AI summary

Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models

Sangmin Woo, Donguk Kim, Jaehyuk Jang, Yubin Choi, Changick Kim

arXiv:2405.17820v2cs.CVcs.AI

TL;DR

Large Vision Language Models can produce descriptions that do not accurately reflect images, and the paper hypothesizes that attention misalignment contributes to these hallucinations. AvisC identifies high-attention image tokens and recalibrates their influence at inference time; evaluations report reduced hallucinations and improved visual grounding and response accuracy across diverse LVLMs.

  • Problem

    Large Vision Language Models can produce descriptions that do not accurately reflect images, and attention misalignment is hypothesized to contribute to hallucinations.

    Source: §1 · p. 1

  • Method

    AvisC analyzes layer-wise attention to identify blind tokens, then uses contrastive decoding to recalibrate their influence at inference time.

    Source: §1 · p. 1

  • Results

    AvisC reduces hallucinations and improves visual grounding and response accuracy, surpassing existing decoding strategies across hallucination benchmarks.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    AvisC is model-agnostic and requires no additional training data or external modules, making it a plug-and-play approach for existing LVLMs.

    Source: §1 · p. 2

    AvisC's effectiveness declines on tasks requiring precise object counting, where blind tokens may carry essential quantification information.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Vision Language Models (LVLMs) demonstrate strong capabilities in visual understanding and description, yet often suffer from hallucinations, attributing incorrect or misleading features to images. We observe that LVLMs disproportionately focus on a small subset of image tokens--termed blind tokens--which are typically irrelevant to the query (e.g., background or non-object regions). We hypothesize that such attention misalignment plays a key role in generating hallucinated responses. To mitigate this issue, we propose Attentional Vision Calibration (AvisC), a test-time approach that dynamically recalibrates the influence of blind tokens without modifying the underlying attention mechanism. AvisC first identifies blind tokens by analyzing layer-wise attention distributions over image tokens, then employs a contrastive decoding strategy to balance the influence of original and blind-token-biased logits. Experiments on standard benchmarks, including POPE, MME, and AMBER, demonstrate that AvisC effectively reduces hallucinations in LVLMs.

1 Introduction

LVLM hallucinations challenge reliable visual description, and the paper hypothesizes that inference-time attention misalignment contributes to them. AvisC detects blind tokens and recalibrates their influence during decoding; the paper reports reduced hallucinations and improved factual accuracy and descriptive richness.

1 Introduction

  • The paper links hallucinations to attention misalignment: LVLMs may overfocus on query-irrelevant blind tokens and misleading cues.Blind tokens are described as receiving excessive attention while carrying little query-relevant information.Source: §1 · p. 1
  • AvisC identifies blind tokens from layer-wise attention distributions and uses contrastive decoding to adjust final token probabilities.It compares logits from all image tokens with logits after selectively zeroing blind tokens.Source: §1 · p. 1
  • The decoding-stage method recalibrates token influence without directly changing the model’s attention mechanism or requiring retraining.Source: §1 · p. 1
  • Across POPE, MME, and AMBER, AvisC reduces hallucinations and improves factual accuracy and descriptive richness.The authors describe the approach as model-agnostic and plug-and-play for existing LVLMs.Source: §1 · pp. 1–2

2 Observations

The observations show that LVLM attention can concentrate on semantically uninformative image patches, including patches unrelated to the query. Blind-token removal leaves predictions nearly unchanged, while removing non-blind tokens disrupts them; the authors propose a possible structural explanation but note that causality is difficult to establish.

2 Observations

  • Although attention is intended to highlight relevant image tokens, this principle does not always hold in current LVLMs.Source: §2 · p. 2
  • Even uniformly colored images with no meaningful query content can prompt LVLMs to concentrate attention on a few semantically uninformative patches.The paper names these excessively attended patches blind tokens.Source: §2 · p. 2
  • Zeroing out blind tokens leaves logits nearly unchanged, whereas removing non-blind tokens collapses them to near-uniform probabilities.Non-blind tokens carry essential object-discriminative information in this analysis.Source: Figure 2 · p. 3, §2 · p. 3
  • 3.7% of blind tokens overlap object regions, while 23.2% of total attention weight goes to those regions.The analysis finds that highly attended patches often have little overlap with genuine objects.Source: §2 · p. 3
  • The authors hypothesize that layered processing concentrates global information in structurally convenient but semantically irrelevant tokens, while acknowledging that a definitive causal link is difficult to establish.They suggest this imbalance may predispose generation to hallucinate.Source: §2 · pp. 3–4

3 Approach: AVISC

AVISC recalibrates the influence of blind tokens during decoding through layer selection, token identification, and contrastive decoding. It selects layers with high image-token attention, identifies unusually attended image tokens, and adjusts next-token logits using original and biased visual tokens.

3 Approach: AVISC

  • AVISC recalibrates blind-token influence during decoding through layer selection, blind-token identification, and contrastive decoding.The method adjusts output logits to mitigate the influence of blind tokens.Source: §3 · p. 4

3.1 LVLM Framework

  • LVLMs concatenate visual tokens with query tokens, then process the sequence autoregressively to produce next-token probabilities.The visual tokens are made compatible with the LLM input space through a cross-modal alignment module.Source: §3.1 · p. 4

3.2 Attentional Vision Calibration

  • AVISC selects layers with a high proportion of attention on image tokens, using top-P sampling with threshold γ.Selected layers provide the basis for token-level analysis.Source: §3.2 · p. 5
  • Within selected layers, AVISC averages attention across heads and classifies tokens exceeding μ + λσ as blind tokens.Here, λ is a hyperparameter.Source: §3.2 · p. 5
  • Contrastive decoding compares next-token logits from original visual tokens with logits from biased tokens, then adjusts the logits before sampling.The contrast strength is controlled by α; the adjustment down-weights blind-token influence and promotes a more balanced attention distribution.Source: §3.2 · p. 5

4 Experiments

Across POPE, MME, and AMBER, AVISC generally outperforms comparison methods, while ablations examine its decoding parameters and layer-selection strategy. Evaluation uses InstructBLIP and LLaVA-1.5 on benchmarks covering hallucination and broader visual tasks.

4.1 Evaluation Setup

  • Evaluation uses InstructBLIP and LLaVA-1.5, compares AVISC with reimplemented VCD and M3ID, and spans POPE, MME, and AMBER tasks.The experiments set P = 0.5 and λ = 1; α is 3 for InstructBLIP and 2.5 for LLaVA-1.5. The benchmarks cover object-presence questions, 14 MME subtasks, and AMBER generative and discriminative tasks.Source: §4.1 · pp. 5–6

4.2 Benchmark Results

  • AVISC outperforms base decoding and VCD/M3ID on POPE, leads on MME-Hallucination Total Scores, tops most MME-Fullset categories, and leads on most AMBER metrics.The reported comparisons cover datasets and query configurations on POPE, object and attribute dimensions on MME-Hallucination, 14 MME-Fullset categories, and generative and discriminative AMBER tasks.Source: §4.2 · pp. 6–7, Table 1 · p. 6, Table 2 · p. 7
  • On MME-Fullset, AVISC has top results in 7 categories for InstructBLIP and 11 for LLaVA-1.5, with Count declines and model-dependent OCR outcomes.InstructBLIP performs lower on OCR tasks, while LLaVA-1.5 shows significant OCR improvements.Source: Figure 5 · p. 7, §4.2 · p. 7
  • AVISC achieves the highest Existence scores on MME-Hallucination for both models and leads in Position and Color, yielding superior Total Scores, although VCD slightly leads in Count.The comparison covers object-level Existence and Count and attribute-level Position and Color.Source: §4.2 · p. 7, Table 2 · p. 7
  • On AMBER, AVISC improves Accuracy and F1 for both models and gains in generative tasks, particularly Existence, achieving the highest scores across most metrics.The reported comparisons include Base, VCD, and M3ID.Source: §4.2 · p. 7, Table 3 · p. 8

4.3 Ablation Study

  • On MME-Hallucination, performance generally improves with higher α: the best total scores occur at α=3 for InstructBLIP and α=2.5 for LLaVA-1.5.For InstructBLIP, increasing λ also improves performance, supporting more selective application to highly attended tokens.Source: §4.3 · p. 8
  • With LLaVA-1.5, γ=0.5 yields high accuracy and balanced POPE metrics and the highest MME-Hallucination total score; performance is robust across γ values except extreme settings such as 0.1.The results suggest limited sensitivity to γ and reduced need for extensive tuning.Source: §4.3 · p. 8
  • Targeted layer selection outperforms manually fixed early, middle, or late five-layer alternatives on POPE-COCO-Random and MME-Hallucination, with modest margins.The study attributes the gains to selecting layers with stronger image-related attention for blind-token localization and calibration.Source: §4.3 · p. 8

5 Related Work

Prior hallucination-mitigation approaches operate at input, model, or output levels. AVISC is an output-level approach that uses internal attention patterns to target blind tokens rather than contrasting whole-image representations.

5 Related Work

  • Hallucination-mitigation work spans input-level data changes, model-level visual-representation improvements, and output-level refinement.Input-level approaches use negative or counterfactual data and dataset cleansing; model-level approaches use higher-resolution processing or advanced encoders, often with additional training.Source: §5 · p. 9
  • Output-level methods include contrastive and guided decoding, training-free methods, and post-hoc corrections; this work falls within the output-level category.Guided decoding may use external models such as CLIP or DETR.Source: §5 · p. 9
  • Unlike prior contrastive decoding that contrasts whole-image representations, AVISC analyzes internal attention to locate and recalibrate blind tokens.The targeted tokens attract excessive attention but contribute little to the final output.Source: §5 · p. 9

6 Conclusion

AVISC identifies blind tokens as image tokens receiving excessive attention despite carrying little task-relevant information. It uses training-free decoding to mitigate their effect, and evaluations report improved visual grounding and response accuracy.

6 Conclusion

  • Blind tokens receive excessive attention while conveying little task-relevant information, and their misdirected focus increases the likelihood of hallucinated responses.The conclusion identifies them as a target for calibration.Source: §6 · p. 9
  • AVISC is a training-free decoding technique that detects and mitigates blind-token effects through image-wise attention analysis and contrastive decoding.The method dynamically adjusts their influence.Source: §6 · p. 9
  • Extensive evaluations on hallucination benchmarks report improved visual grounding and response accuracy, surpassing existing decoding strategies.Source: §6 · p. 9

Limitations

AVISC’s effectiveness declines on precise counting tasks, and its test-time recalibration adds overhead. The authors also note that benchmark performance varies with dataset scope and evaluation protocols.

Limitations

  • AVISC’s effectiveness declines on precise counting tasks, where blind tokens may sometimes carry essential quantification information.The cited examples are the Count category in MME and Number in AMBER.Source: Limitations · p. 9
  • Dynamic test-time recalibration adds overhead, although AVISC maintains competitive tokens-per-second throughput against other contrastive decoding methods.The authors say recalibration is currently sequential but could be parallelized to reduce wall-clock time at the expense of memory.Source: Limitations · p. 9
  • MME and AMBER performance varies with dataset scope and evaluation protocols, which are sensitive to token usage.The authors nevertheless report consistently lower hallucination rates and statistically significant gains.Source: Limitations · p. 9
  • The authors hypothesize that blind tokens may be intrinsic to large-scale transformer architectures, and propose further study of them and computational efficiency.Source: Limitations · p. 9

A Visualizations & Analysis on Blind Tokens

The visual analysis finds that LVLMs give high attention to image tokens that contribute little to prediction logits and often do not overlap with objects. The section also presents examples and supplementary implementation details for visualizing and adjusting token logits.

A Visualizations & Analysis on Blind Tokens

B.1 Further Implementation details

  • Cut-off sampling masks text tokens below β times the maximum token probability; VCD and M3ID reproductions use stated parameter settings.The cited settings are α=1.0, β=0.1, T=500 for VCD and λ=0.2 for M3ID.Source: §B.1 · pp. 13–14
  • Figure 11 pairs highlighted blind-token regions with the model’s original prediction logits and logits adjusted by AVISC.Source: A Visualizations & Analysis on Blind… · p. 13, Figure 11 · p. 15

B.2 Evaluation Benchmarks

The evaluation section describes the POPE, MME, AMBER, and LLaVA-Bench datasets and the tasks they cover. It also defines metrics for accuracy and hallucination-related performance across discriminative and generative evaluations.

B.2 Evaluation Benchmarks

  • POPE evaluates yes/no object-presence questions across random, popular, and adversarial setups, while MME covers 14 perceptual and cognitive subtasks.POPE includes 3,000 question-answer pairs; MME removes the one-word response constraint to allow natural responses.Source: §B.2 · pp. 15–16
  • AMBER combines generative and discriminative tasks, while LLaVA-Bench uses 24 images and 60 questions across varied contexts.The AMBER evaluation samples 500 generative and 5,000 discriminative questions.Source: §B.2 · p. 16

B.3 Metrics

  • CHAIR evaluates hallucinatory objects in responses; Cover measures response-object coverage, and Hal indicates hallucinations when CHAIR is nonzero.Source: §B.3 · p. 16
  • Cog measures overlap with human hallucinatory-object targets, while the AMBER Score combines generative CHAIR and discriminative F1.MME’s overall score sums ACC and ACC+; ACC+ is one when both answers for an image are correct.Source: §B.3 · p. 16

C.1 Zero-Out Experiments on POPE-COCO Benchmark

The supplementary experiments test whether blind tokens affect predictions, compare calibration choices and inference cost, and evaluate AvisC across larger, diverse, and generative settings. Results generally favor AvisC, with specific trade-offs across methods and tasks.

C.1 Zero-Out Experiments on POPE-COCO Benchmark

  • Removing blind tokens minimally alters POPE-COCO logits, whereas removing non-blind tokens shifts them drastically; AvisC reduces blind-token overemphasis, particularly for TN and FP cases.Table 7 reports average Yes and No logits and the ground-truth-versus-wrong-logit difference for TP, TN, FP, and FN cases.Source: §C.1 · pp. 16–17, Table 7 · p. 17

C.2 Inference Time and OPERA

  • OPERA achieves the best overall performance but runs approximately 72.23 times slower than AvisC; the authors note that its beam-search implementation prevents a fair comparison.The inference-speed comparison uses a TiTAN RTX GPU on POPE-COCO-Random.Source: §C.2 · p. 17, Table 8 · p. 17

C.3 Alternatives to Zero-Out

  • Across non-blind-token deactivation schemes, the strongest metric varies by model and metric, while the authors report Zeros as most effective overall.For InstructBLIP, Mask leads in Accuracy and F1, Zeros in Precision, and Ones in Recall; for LLaVA-1.5, Noise leads in Accuracy and Precision while Zeros is balanced across metrics.Source: §C.3 · pp. 17–18, Table 9 · p. 17

C.4 Results of Larger LVLM

  • On POPE-COCO with LLaVA-1.5-13B, AvisC shows more pronounced improvement than with the 7B model, while VCD and M3ID gains are slight or sometimes metric-dependent decreases.The passage describes the larger-model results as resilient to changes in LVLM size.Source: §C.4 · p. 18, Table 10 · p. 18

C.5 Additional Evaluation on off-ths-shelf LVLMs

  • On POPE, AvisC consistently outperforms prior methods across LLaVA-OneVision-Qwen2-7B and Qwen2.5-VL-7B setups without compromising the original models’ capabilities.The authors characterize this as a test-time, plug-and-play strategy applicable across off-the-shelf LVLMs.Source: §C.5 · p. 18, Table 11 · p. 18

C.6 Additional Evaluation on Generative Benchmarks

  • On MMHal-Bench and Object-Hallucination, AvisC preserves or improves generative quality while consistently reducing hallucination metrics versus existing approaches.The passage cites maintained or improved MMHal scores as an example of preserved generative quality.Source: §C.6 · p. 18, Table 12 · p. 18

C.7 POPE (Li et al., 2023b) with Single-Word Constraint

  • With a one-word response constraint, POPE performance changes notably, but AvisC still achieves the best performance; precision and recall vary substantially with the instruction in the COCO-random setup.The authors instead designed experiments allowing free judgments and explanations to better evaluate discriminative capabilities.Source: §C.7 · pp. 18–19, Table 13 · p. 19

C.8 Detailed Results on MME-Fullset; C.9 Detailed Results on AMBER Discriminative Tasks

  • On MME-Fullset, AvisC improves LLaVA-1.5 and InstructBLIP across many perception and recognition tasks; on most AMBER discriminative categories, both models also improve, with a few exceptions.AMBER categories include Existence, Attribute, State, Number, Action, and Relation.Source: §C.8 · p. 19, §C.9 · p. 19

D Comparison with "Vision Transformers Need Registers" (Darcet et al., 2023)

The comparison distinguishes LVLM blind tokens from high-norm outlier tokens in vision transformers, while reporting a partial overlap between the two. Blind tokens are described as often lacking query-relevant details, unlike the global information associated with high-norm tokens.

D Comparison with "Vision Transformers Need Registers" (Darcet et al., 2023)

  • Blind tokens come from LLM attention in LVLMs, whereas high-norm tokens arise within vision-transformer layers; the models’ tasks, architectures, and domains also differ.The passage contrasts LVLM image-based question answering with vision transformers optimized for dense prediction, and notes training-regime sensitivity for high-norm tokens.Source: D Comparison with "Vision Transformers… · pp. 20, 23, 24
  • Blind and high-norm tokens overlap only moderately: P(blind token | high-norm token) is 40.38%, and P(high-norm token | blind token) is 31.27%.Blind tokens tend to occur at the beginning and end of the image-token sequence, a pattern not clearly observed for high-norm tokens.Source: D Comparison with "Vision Transformers… · pp. 24–25
  • High-norm tokens are described as encoding global information, while blind tokens in LVLMs often lack query-relevant details; the paper says non-blind tokens typically capture essential information.The proposed contrastive decoding reduces blind-token influence while enhancing non-blind-token influence, which the authors report mitigates hallucinations without sacrificing critical information.Source: D Comparison with "Vision Transformers… · pp. 20, 25

E Qualitative Results

Qualitative comparisons across benchmarks show that AvisC helps counteract hallucinations in generated descriptions. Base decoding and VCD often include errors about elements absent from the image, whereas AvisC responses more accurately reflect image content.

  • Base decoding and VCD often describe elements absent from images, while AvisC helps counteract those hallucinations and produce more accurate image descriptions.The qualitative examples cover POPE, MME, AMBER, and LLaVA-Bench.Source: E Qualitative Results · p. 25

Related papers

Loading 2405.17820v2…