PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Modal Hallucination Control by Visual Information Grounding

Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, Stefano Soatto

arXiv:2403.14003v1cs.CVcs.CLcs.LG

TL;DR

VLMs can produce plausible but visually ungrounded answers as reliance on the visual prompt declines during generation. The paper introduces M3ID, an inference-time mutual-information decoding intervention, and pairs it with DPO when training is available. On LLaVA 13B, the methods reduce hallucinated objects and improve POPE accuracy while preserving linguistic fluency.

  • Problem

    VLMs can generate plausible-sounding answers without factual basis, motivating methods that address ungrounded outputs and declining visual-prompt dependence.

  • Method

    M3ID amplifies visual-prompt dependence by rescaling image-conditioned generation against the unconditioned distribution, with DPO providing an optional training-based extension.

  • Results

    25% and 28% reductions in hallucinated objects and 21% and 24% POPE accuracy improvements are reported for M3ID and DPO, respectively, over the base model.

  • Takeaways & Limitations

    M3ID reduces hallucinations at inference time without training, while the paper reports preserved linguistic fluency and stronger visual grounding.

  • Takeaways & Limitations

    High confidence thresholds can overcompensate, increasing multimodal hallucinations and disrupting linguistic fluency.

Abstract

from arXiv · show

Generative Vision-Language Models (VLMs) are prone to generate plausible-sounding textual answers that, however, are not always grounded in the input image. We investigate this phenomenon, usually referred to as "hallucination" and show that it stems from an excessive reliance on the language prior. In particular, we show that as more tokens are generated, the reliance on the visual prompt decreases, and this behavior strongly correlates with the emergence of hallucinations. To reduce hallucinations, we introduce Multi-Modal Mutual-Information Decoding (M3ID), a new sampling method for prompt amplification. M3ID amplifies the influence of the reference image over the language prior, hence favoring the generation of tokens with higher mutual information with the visual prompt. M3ID can be applied to any pre-trained autoregressive VLM at inference time without necessitating further training and with minimal computational overhead. If training is an option, we show that M3ID can be paired with Direct Preference Optimization (DPO) to improve the model's reliance on the prompt image without requiring any labels. Our empirical findings show that our algorithms maintain the fluency and linguistic capabilities of pre-trained VLMs while reducing hallucinations by mitigating visually ungrounded answers. Specifically, for the LLaVA 13B model, M3ID and M3ID+DPO reduce the percentage of hallucinated objects in captioning tasks by 25% and 28%, respectively, and improve the accuracy on VQA benchmarks such as POPE by 21% and 24%.

AWS AI Labs

The passage lists the AWS AI Labs contact email.

  • The listed contact email is zancato, mttrager, sidchoud, pramudi, aachille, swashwin, and soattos at amazon.com.
  • The address uses the Amazon corporate domain.
  • Seven usernames appear before the shared domain.

1. Introduction

The paper studies hallucinations in VLMs as a consequence of declining visual-prompt dependence during generation. It introduces M3ID, with an optional DPO extension, to strengthen visual grounding while preserving fluency.

  • VLMs can generate plausible-sounding answers without factual basis, producing ungrounded or fabricated information.
  • The visual prompt dependency measure decreases as more tokens are generated, an effect termed conditioning dilution or fading memory.The authors associate this decline with conditioning information being diluted, forgotten, or ignored, possibly leading to more hallucinations.
  • M3ID amplifies visual-prompt dependence at inference time by rescaling the image-conditioned component against the unconditioned distribution.It is training-free, applicable to off-the-shelf autoregressive VLMs without model-weight access, and incurs low computational overhead.
  • DPO extends M3ID into a preference-optimization objective that favors continuations with higher visual prompt dependency.This training-based extension is intended for users with access to model weights and does not require additional labels in the supplied abstract context.
  • 25% and 28% reductions in hallucinated objects were obtained with M3ID and DPO, respectively, while POPE accuracy improved by 21% and 24%.These results are reported over the base model for captioning and the POPE VQA hallucination benchmark, respectively.

2. Related work

Related work covers hallucinations in grafted VLMs, decoding methods for controlling generation, and mutual-information approaches for text-based faithfulness and dialogue relevance. The paper positions its use of mutual information as a new approach to multimodal grounding.

  • Hallucinations in VLMs: Grafted VLMs inherit strong linguistic fluency from pretrained LLMs but also their tendency to produce ungrounded or fabricated information.
  • Context-dependent decodings: Search decoding can be factually accurate but repetitive, whereas sampling improves diversity while suffering from topic drift.
  • Contextual pressure: Figure 2 compares conditioned and unconditioned token likelihoods in the string “The image ...”.The probability gap is larger for visually required objects and attributes, and smaller for punctuation, articles, and already predictable content such as “kitchen”.
  • Context-dependent decodings: Mutual-information decoding has been used to keep LLM generations faithful to input text and to promote diversity or relevance in neural dialogue.
  • Context-dependent decodings: The paper describes its contribution as the first use of mutual information to improve multimodal grounding with time-varying penalization.The time-varying design is intended to counteract progressive forgetting of the visual prompt.

3. Analysis of hallucinations in VLMs

The paper evaluates hallucinations through visual prompt dependency, finding that image conditioning fades during generation and that lower dependency accompanies more hallucinated objects. It also distinguishes this dilution from contextual pressure, where low dependency can arise for linguistically predictable tokens without implying hallucination.

  • Visual prompt dependency: Visual prompt dependency compares token likelihoods with and without the image to quantify how context-specific model outputs are.High dependency indicates stronger association with the visual prompt; low dependency indicates more prompt-neutral output, but does not necessarily imply hallucination.
  • Conditioning dilution: Hallucinated objects increase with token position, while hallucinations remain rare near the visual prompt and become more frequent at lower PDM.Figure 3 reports this relationship for synthetic LLaVA captions on the MS COCO validation split.
  • Contextual pressure: Contextual pressure makes conditioned and unconditioned likelihoods similar for linguistically required or predictable tokens without necessarily increasing hallucination risk.Examples include prepositions, conjunctions, and completing fine-grained objects such as “Peanut butter.”
  • Conditioning dilution: PDM-H decreases as more tokens are generated, indicating that visual information becomes diluted and the model increasingly approaches the language prior.The paper calls this conditioning dilution or the fading memory effect.

4. Methods

The methods model visual conditioning as a fading-memory process and counteract it by amplifying image-dependent token probabilities, while suppressing corrections for highly predictable tokens. M3ID operates at inference time, and DPO uses grounded and ungrounded continuations to learn a more grounded generation policy.

  • Preventing conditioning dilution: The fading-memory model represents conditioned log probabilities as a time-varying interpolation with an unconditioned language model, using γ_t = exp(−λt).The mixing coefficient decreases over time, reducing the modeled influence of the image during generation.
  • Preventing conditioning dilution: M3ID estimates a latent distribution that does not forget prior visual context and adds a correction derived from conditioned and unconditioned log probabilities.The intervention estimates the correction term from the fading-memory model and applies it to the conditioned distribution.
  • M3ID: M3ID increasingly emphasizes conditioned tokens that surprise the unconditioned policy as γ_t approaches zero, counteracting conditioning dilution.Early generation is approximately sampled from the conditioned distribution alone; later generation amplifies the conditioned-minus-unconditioned component.
  • M3ID: M3ID suppresses its correction when the conditioned model is highly confident, avoiding penalties on obvious tokens such as prepositions and conjunctions.This adjustment accommodates contextual pressure, where conditioned and unconditioned predictions can be similar for correct linguistic tokens.
  • M3ID+DPO: DPO trains the VLM to prefer grounded continuations over ungrounded ones by increasing the preferred continuation’s likelihood relative to the base model and decreasing the rejected continuation’s likelihood.Preferred continuations are sampled with M3ID, while negative continuations are generated from an unconditioned model and constrained with a conditioned first sentence.

5. Experiments

The experiments evaluate M3ID on captioning and VQA benchmarks, showing reduced hallucinations and improved accuracy over standard decoding while revealing trade-offs from overly strong interventions.

  • Evaluation setup: M3ID is evaluated on MS COCO captioning and POPE VQA benchmarks using LLaVA and related decoding and training baselines.Captioning uses CHAIR and Cover metrics, while POPE evaluates binary object-presence classification.
  • Captioning results: 27%/21% relative improvement over LLaVA7B on CHAIRi/CHAIRs and 26%/29% over LLaVA13B reduces caption hallucinations without large Cover losses.Cover improves for the 7B model and decreases by less than 2.2% for the 13B model.
  • Captioning results: Pairing M3ID with DPO yields fewer hallucinated objects and improved Cover compared with M3ID alone and LLaVA+LURE.The DPO approach is compared with a concurrent training-based method relying on GPT-3.5 annotations.
  • VQA methodology: On POPE, M3ID accounts for template-induced dilution by selecting t = t0, where t0 counts tokens between the output and image.Although POPE generates only Yes/No tokens, the input template separates image tokens from the VLM output and introduces non-negligible dilution.
  • VQA results: M3ID reduces the Yes-answer ratio to 72.9%/61.8% for 7B/13B models and improves POPE accuracy by 8%/21% over standard LLaVA decoding.The base LLaVA models produce disproportionately high Yes-answer rates of 84%/83.7%.
  • VQA results: M3ID+DPO further improves POPE accuracy by 15% and 24% for LLaVA 7B and 13B, respectively, while requiring no labeled information.Its performance is reported as close to training-based baselines using labeled or preference data.
  • Ablations and limitations: High forgetting factors or confidence thresholds strengthen intervention but can cause overcompensation, higher hallucination rates, reduced Cover, and disrupted fluency.M3ID may omit elements predictable from the language prior, such as a man accompanying a leashed dog.

6. Conclusions

The paper concludes that M3ID reduces hallucinations by increasing reliance on visual context, while DPO can further improve visual grounding. Its inference-time use has computational and coverage-related trade-offs.

  • Conclusions: M3ID can be paired with DPO to further reduce hallucinatory behavior when model training is feasible.The paper presents this as a training-based extension of the inference-time method.
  • Conclusions: M3ID reduces hallucinations by amplifying the visual prompt over the language prior, and it can operate without additional training.The approach is designed to maximize mutual information between generated text and visual context.
  • Limitations: Two forward passes are required at inference time: one conditioned on the image and one unconditioned.Batched queries can avoid increased inference time but require more memory.
  • Limitations: M3ID may suppress objects favored by the unprompted language prior, although hyper-parameter selection can mitigate this behavior.The authors suggest progressively more detailed, grounded captions as a future direction.
  • Future work: Human- or AI-annotated preference pairs based on grounding are identified as a promising direction for future investigation.

A. Experimental details

The experiments evaluate hallucination and grounding with COCO captioning and POPE VQA, compare context-dependent decoding baselines, and specify M3ID and DPO settings.

  • Evaluation: The setup measures caption hallucinations with CHAIR on 5,000 MS COCO validation images and VQA accuracy with POPE question sets.POPE includes random, popular, and adversarial questions.
  • Evaluation: Hyper-parameters for M3ID, PMI, and Contrastive Decoding are selected using CHAIR on a separate 500-image COCO validation subset.
  • Implementation: The experiments run on 8 NVIDIA Tesla A100 GPUs, using LoRA-based DPO settings with AdamW and DeepSpeed ZeRO-2.
  • Baselines: PMI and Contrastive Decoding serve as context-dependent baselines adapted to the multi-modal setting.Both use time-independent penalization in the described formulations.
  • M3ID: M3ID compares image-conditioned and unconditioned predictions, increasing the penalty on the language-only prior as generation proceeds.Its time-dependent penalization is controlled by γ_t.
  • DPO: The DPO extension models preferred and less-preferred continuations according to their grounding with respect to the image.Its objective increases likelihood for preferred completions and decreases it for poorly grounded ones.
  • DPO: The alignment data are self-generated: M3ID produces positive captions, while an unconditioned VLM generates negative continuations.

D. Further results

This section provides additional results that complement the experiments presented in the main text.

  • Further results: The section presents further results beyond those reported in the main text.
  • Further results: These additional results are intended to complement the main experimental findings.
  • Further results: No specific experiment or numerical outcome is stated in the supplied section passage.

D.1. Prompt dependency measures

Prompt-dependency measures track how visual conditioning influences token generation. Standard decoding increasingly resembles unconditioned language modeling, whereas M3ID preserves image-conditioned separation over longer captions.

  • Prompt-dependency measures: PDM-H and PDM-R quantify differences between conditioned and unconditioned token distributions during caption generation.The measures use Hellinger and Rank distances, respectively.
  • Interpretation: When PDM-R equals 1, greedy decoding would produce the same continuation with or without the specific conditioning signal.
  • Token-level behavior: Both prompt-dependency distances increase for visually informative tokens and decrease for punctuation, articles, and already predictable words.In the example, distances rise for objects and attributes.
  • Conditioning dilution: After approximately 30 generated tokens, standard-decoding PDM-R approaches 1, indicating similar conditioned and unconditioned predictions.
  • Decoder comparison: PMI and Contrastive Decoding increase prompt-dependency distances early, but their effects become negligible later in generation.M3ID instead maximizes both distances throughout longer captions.
  • Experimental comparison: The reported plots evaluate average PDM-H and PDM-R across 5,000 captions generated with LLaVA and multiple decoding schemes.

D.2. Captioning

M3ID reduces hallucinations in captioning by amplifying visual grounding, but excessive correction can overlook language-prior-supported content and disrupt fluency.

  • Captioning results: M3ID and M3ID+DPO achieve the best results in minimizing object hallucinations for detailed captioning.Detailed captioning raises cover from approximately 55% to about 70%, while all methods show larger CHAIR values than in standard captioning.
  • Captioning results: M3ID is evaluated across captioning settings, including standard, detailed, and examples involving introduced errors and overcompensation.The supplied tables cover hallucination modification and maintenance frequencies, detailed-captioning metrics, and qualitative overcompensation examples.
  • Captioning results: 18% of hallucinations are corrected for LLaVA 13B and 16.2% for LLaVA 7B, while newly introduced hallucinations occur in 2.4% and 3.0% of cases.
  • Overcompensation: Excessively large penalties on the language prior can cause M3ID to omit elements that are predictable from language context alone, such as a man in an image.This behavior is termed overcompensation and arises when decoding strongly favors tokens that deviate from the language prior.
  • Language fluency: A high threshold α can repeatedly activate the correction indicator, disrupting syntax and eventually producing apparently random or non-English tokens.The examples use α = 1 and intensify the correction term to illustrate progressively worse language fluency.
  • Applicability: M3ID applies to VLMs when visual conditioning can be dropped to compute a language prior, including architectures such as InstructBLIP with masked image tokens.InstructBLIP connects its vision encoder and language model through a Q-Former, requiring image-token masking for unconditioned predictions.
Loading 2403.14003v1…