PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations

Nick Jiang, Anish Kachinthaya, Suzie Petryk, Yossi Gandelsman

arXiv:2410.02762v2cs.CVcs.LG

TL;DR

Vision-language models can hallucinate image content despite scaling and training, complicating their reliability. This paper probes image representations through the language vocabulary and edits them to remove hallucinations. It reports up to 25.7% hallucination reduction on COCO2014 while preserving image-captioning accuracy, alongside zero-shot segmentation results.

  • Problem

    VLMs continue to hallucinate content absent from images despite advances in model size and training, raising reliability concerns.

    Source: §1 · p. 1

  • Method

    The paper projects intermediate image representations into the language vocabulary to identify object confidence and linearly orthogonalizes image features against target-object text features.

    Source: §1 · p. 1, §3.2 · p. 4, §4.1 · p. 5

  • Results

    Up to 25.7% hallucination reduction on COCO2014 is reported while preserving image-captioning accuracy, with comparable zero-shot segmentation performance to state-of-the-art methods.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Interpreting and editing VLM latent representations supports hallucination reduction and zero-shot segmentation within the evaluated settings.

    Source: §1 · p. 2, §6 · p. 10

    The editing approach may struggle with abstract or longer sentences involving object attributes or interactions; full-sentence removal was not assessed.

    Source: §6 · p. 10

Abstract

from arXiv · show

We investigate the internal representations of vision-language models (VLMs) to address hallucinations, a persistent challenge despite advances in model size and training. We project VLMs' internal image representations to their language vocabulary and observe more confident output probabilities on real objects than hallucinated objects. We additionally use these output probabilities to spatially localize real objects. Building on this approach, we introduce a knowledge erasure algorithm that removes hallucinations by linearly orthogonalizing image features with respect to hallucinated object features. We show that targeted edits to a model's latent representations can reduce hallucinations by up to 25.7% on the COCO2014 dataset while preserving performance. Our findings demonstrate how a deeper understanding of VLMs' latent representations can enhance reliability and enable novel capabilities, such as zero-shot segmentation.

1 INTRODUCTION

VLM hallucinations persist despite model scaling, raising reliability concerns, while existing interventions can struggle to distinguish subtle hallucinations from image details. The paper probes image representations through text and edits them to reduce hallucinations while preserving performance.

1 INTRODUCTION

  • VLMs continue to hallucinate content absent from images despite advances in scale and training, creating reliability concerns for real-world applications.Existing approaches use external interventions or fine-tuning and can struggle to distinguish subtle hallucinations from existing details.Source: §1 · p. 1
  • The logit lens projects image representations into the language vocabulary, revealing differences between real and hallucinated objects and enabling spatial object localization.The paper uses these findings to detect hallucinations and localize objects.Source: §1 · p. 1
  • PROJECTAWAY removes target objects by linearly orthogonalizing image features with respect to their text features, and can erase both real and hallucinated objects.The method intervenes directly in image representations.Source: §1 · p. 1
  • Combining editing and detection to erase hallucinations from VLM internal representations reduced hallucinations by up to 25.7% on standard benchmarks while preserving image-captioning accuracy; spatial mapping achieved comparable performance to state-of-the-art zero-shot segmentation methods.Logit-lens hallucination detection improved mAP by 22.45% and 47.17% in two VLMs.Source: §1 · p. 2

2 RELATED WORK

The related work spans interpretation of language- and vision-model representations and approaches to detecting and reducing VLM hallucinations. This paper distinguishes its focus on interpreting and editing VLM latent representations for practical use.

2.1 INTERPRETING LATENT REPRESENTATIONS IN LANGUAGE MODELS

  • The logit lens uses a language model’s unembedding matrix to obtain vocabulary distributions from intermediate representations, while related work also studies activations, attention, and latent representations.Prior applications include early exiting and editing or erasing model knowledge.Source: §2.1 · p. 2

2.2 INTERPRETING LATENT REPRESENTATIONS IN VISION MODELS

  • Prior vision-model interpretation includes saliency maps, neuron analysis, concept mapping, and explanations of latent tokens, attention heads, and neurons.Few cited works interpret VLM internal computation; this paper instead focuses on interpreting and editing latent representations for real-world applicability.Source: §2.2 · pp. 2–3

2.3 DETECTING AND REDUCING VLM HALLUCINATIONS

  • VLM hallucination methods include latent-representation classifiers, learned transition functions, final-layer logits, fine-tuned revisors, and attention-based suppression.In contrast, this paper leverages internal image representations for hallucination reduction and zero-shot segmentation.Source: §2.3 · p. 3

3 EXTRACTING KNOWLEDGE FROM VLMS

The paper applies a language-model vocabulary probe to VLM image representations and finds that object confidence can distinguish present from absent objects and localize them spatially. It examines two 7B VLMs and uses intermediate image representations to study these capabilities.

3.1 PRELIMINARIES

  • The study uses 7B LLaVA 1.5 and InstructBLIP, whose vision encoders, mapping networks, and language models differ in which components are frozen or fine-tuned.LLaVA fine-tunes its mapping network and language model, whereas InstructBLIP trains only its mapping network.Source: §3.1 · p. 3
  • The framework represents an image as n embeddings and defines each layer’s latent representation as conditioned on preceding input tokens.The unembedding matrix maps the final latent representation to a next-token vocabulary distribution.Source: §3.1 · p. 3
  • The logit lens applies the language model’s unembedding matrix to intermediate representations to obtain vocabulary logit distributions.The paper applies this probe to image embeddings as they pass through language-model layers.Source: §3.1 · p. 3, §3.2 · p. 4

3.2 APPLYING LOGIT LENS ON VLMS

  • Present objects have higher internal confidence than absent objects in the examined VLMs, a distinction the paper later uses to classify hallucinations.The comparison uses InstructBLIP and LLaVA on 5,000 COCO2014 validation images and 80 COCO objects.Source: §3.2 · p. 4, Figure 2 · p. 4
  • For each image embedding, the method takes the maximum object-token probability across language-model layers; these values localize objects to image patches.LLaVA is used for localization because its image encoder preserves spatial mapping from image patches to image features.Source: §3.2 · p. 4
  • Image representations with higher confidence for an object correspond to patches where that object is visually present, supporting zero-shot segmentation.The paper describes these representations as semantically aligned with object tokens while retaining spatial locality.Source: §3.2 · p. 4, Figure 3 · p. 5

4 ERASING KNOWLEDGE FROM VLMS

PROJECTAWAY edits image representations by orthogonalizing them against target-object text representations, aiming to remove objects from captions. Hallucinated objects are more removable than correctly detected objects, and parameter choices affect hallucination reduction and caption accuracy.

4.1 ERASING OBJECTS FROM IMAGE REPRESENTATIONS

  • PROJECTAWAY subtracts a weighted target-object text embedding from image features only when their dot product is positive; at α = 1, it orthogonalizes image representations to that text representation.The editing is applied across image embeddings at a hidden layer, without modifying other latent representations.Source: §4.1 · p. 5

4.1.1 REMOVING OBJECTS ONE BY ONE

  • PROJECTAWAY removed individual hallucinated objects more effectively than correctly detected objects for both InstructBLIP and LLaVA.Together with the lower softmax scores reported for hallucinated objects, the authors suggest hallucinations manifest more weakly in image embeddings and are easier to remove.Source: §4.1.1 · p. 6

4.1.2 MASS-REMOVING OBJECTS

  • 41.3% for InstructBLIP and 23.3% for LLaVA: mass-removing hallucinations reduced hallucinations while recall count slightly increased for both models.The authors report that removing hallucinations together does not interfere with one another.Source: §4.1.2 · p. 6
  • Mass-removing correctly detected objects yielded lower removal rates than hallucinated objects, while CHAIR scores changed minimally.The authors interpret this as object removal without eroding caption accuracy.Source: §4.1.2 · p. 6, Table 1 · p. 6

4.2 ABLATION STUDY: MASS-REMOVING HALLUCINATIONS

  • 38.5% for InstructBLIP at (lI = 1, lT = 2) and 25.7% for LLaVA at (lI = 19, lT = 21): these were the best hallucination reductions in the layer ablation.The LLaVA results favor middle layers for editing and extracting text embeddings; the authors offer alternative parameter choices if the reduction does not generalize beyond their samples.Source: §4.2 · p. 7
  • Increasing α raises hallucination removal, but high values can eliminate both correctly detected and hallucinated objects, so the authors avoid weights that reduce correctly detected objects.At high α, captions can become nonsensical.Source: §4.2 · p. 7

5 APPLICATIONS

The applications use internal confidence from image representations to detect object hallucinations, guide their removal, and localize objects for zero-shot segmentation. Across these tasks, the method improves reported benchmark results, including hallucination reductions against beam search and better segmentation than raw VLM attention.

5.1 HALLUCINATION DETECTION

  • Internal confidence beat the maximum-token-probability baseline: mAP improved 47.17% in InstructBLIP and 22.45% in LLaVA, while ROC AUC improved 50.10% and 44.68%, respectively.The approach classifies object presence using confidence from the VLM’s image representations.Source: §5.1 · pp. 7–8

5.2 HALLUCINATION REMOVAL

  • Hallucinations fell by 25.7% for InstructBLIP and 23.8% for LLaVA versus beam search on 500 COCO validation images, without training or external models.The method combines internal-confidence detection with PROJECTAWAY; some correctly detected objects were also edited out.Source: §5.2 · p. 9

5.3 ZERO-SHOT SEGMENTATION

  • On ImageNet validation, segmentation mAP improved 8.03% over raw VLM attention, with better and/or comparable performance to state-of-the-art methods using the image encoder.The method thresholds class-word confidence at image patches to produce foreground/background segmentations.Source: §5.3 · pp. 9–10, Table 3 · p. 10

6 DISCUSSION AND LIMITATIONS

The discussion presents linear orthogonalization as a way to selectively remove object information from VLM image representations, with applications to hallucination reduction and zero-shot segmentation. It also identifies limitations for multi-token objects and edits to abstract or longer descriptions.

6 DISCUSSION AND LIMITATIONS

  • Linear orthogonalization selectively removes object information from VLM image representations, supporting hallucination reduction and improved zero-shot segmentation.The paper proposes extending the approach to people, attributes, actions, object injection, and other multimodal architectures.Source: §6 · p. 10
  • Multi-token shortcuts can add confidence noise when tokens occur in multiple words and approximate object representations during editing.The approach may take maximum token confidence or use an average token embedding for editing.Source: §6 · p. 10
  • The study assesses removal of individual objects, not full sentences, and notes possible difficulty with abstract or longer descriptions involving attributes or interactions.Source: §6 · p. 10
  • The proposed future directions include extending edits to people, attributes, actions, object injection, and other multimodal architectures.Source: §6 · p. 10

A.1 MASS-REMOVING OBJECTS

This appendix describes mass-removing mentioned objects with PROJECTAWAY and varying the edit weight for InstructBLIP. The ablation tracks hallucination reduction and correctly detected objects as the weight changes.

A.1 MASS-REMOVING OBJECTS

  • As the weight factor α increases, hallucinations decline, and the figure marks a weight with no loss in correctly detected objects.Source: Figure 11 · p. 15
  • The weight ablation tracks changes in correctly detected objects, object removal rate, and hallucination reduction.Source: Figure 11 · p. 15
  • PROJECTAWAY mass-removes mentioned hallucinated and correctly detected objects, then tallies unique objects of each type.Source: §A.1 · p. 15

A.2 ABLATIONS FOR INSTRUCTBLIP

The InstructBLIP ablations find that smaller layer parameters are most effective for reducing hallucinations, while much of the parameter space is too sensitive to edit without losing correctly detected objects. The best tested parameters reduce hallucinations by 38.5%.

A.2 ABLATIONS FOR INSTRUCTBLIP

  • The best InstructBLIP parameters, lI = 1 and lT = 2, reduce hallucinations by 38.5%.Source: §A.2 · p. 15
  • Smaller lT and lI parameters are most effective, while most of the tested parameter space is too sensitive and reduces correctly detected objects.Source: §A.2 · p. 15
  • The authors suggest, as a possibility, that LLaVA fine-tuning aligns image representations with text embeddings more than InstructBLIP, enabling more precise edits.Source: §A.2 · p. 15

A.3 HALLUCINATION DETECTION

The section compares internal-confidence hallucination detection with a baseline and illustrates object classifications for LLaVA and InstructBLIP.

A.3 HALLUCINATION DETECTION

  • Internal confidence is used to detect hallucinations and is quantitatively compared with a baseline in Table 5.The table reports comparisons for the hallucination-detection approach described in Section 5.1.Source: §A.3 · p. 16
  • Qualitative examples show model-generated captions and parsed objects for both LLaVA and InstructBLIP.The examples label objects as hallucinated or correctly detected based on their internal confidence scores.Source: §A.3 · p. 16
  • The examples classify parsed objects as hallucinated or correctly detected using their internal confidence scores.Source: §A.3 · p. 16

A.4 HALLUCINATION REDUCTION

The appendix extends hallucination-intervention evaluations to additional models and attribute errors, and provides examples of localization, patch classification, and results beyond COCO 2014.

A.4 HALLUCINATION REDUCTION

  • The intervention examples show linear removal of hallucination text representations from image representations and edits to captions that remove detected hallucinations.Source: §A.4 · p. 16

A.6 OBJECT LOCALIZATION

  • Localization examples use internal confidence for specific LLaVA image representations.Source: §A.6 · p. 16

A.7 ATTRIBUTE HALLUCINATIONS

  • Attribute editing is demonstrated qualitatively on VQA 2.0 color and number inaccuracies, editing only attributes with co < 0.05 using InstructBLIP parameters lI = 1, lT = 2, α = 1.5.Source: §A.7 · p. 16, Figure 12 · p. 17

A.8 ZERO-SHOT CLASSIFICATION

  • Patch-classification accuracy varies across COCO classes, which the authors hypothesize reflects differences in how specifically class objects are represented by tokens.The reported measure is the percentage of object-containing patches identified by top-k logit-lens predictions.Source: §A.8 · p. 16, Table 8 · p. 24

A.9 QUALITATIVE EXAMPLES BEYOND COCO 2014

  • On LLaVA-Bench's 24 varied-subject images, InstructBLIP examples align with the strong hallucination reduction observed on COCO 2014.COCO 2014 is the primary analysis dataset because CHAIR is tied to it and automatically categorizes caption objects.Source: §A.9 · p. 16, Figure 13 · p. 18

A.5 QUANTITATIVE EVALUATIONS ON MORE ADVANCED MODELS

  • 27.73% for LLaVA-NeXT and 28.86% for Cambrian-1 are CHAIRS improvements, with recall of present objects maintained in intervention comparisons.Both results are reported as consistent with findings on LLaVA and InstructBLIP.Source: §A.5 · p. 16, Table 7 · p. 18

Related papers

Loading 2410.02762v2…