PaperlayerView exact revision on arXiv

Source-linked AI summary

ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models

Yeji Park, Deokyeong Lee, Junsuk Choe, Buru Chang

arXiv:2408.13906v1cs.CVcs.AIcs.LG

TL;DR

Image-inaccurate responses undermine the reliability of multimodal language models. ConVis reconstructs images from generated captions with a text-to-image model and contrasts those images with the originals during decoding. Across evaluated benchmarks and models, the paper reports reduced hallucinations while preserving core language-model capabilities.

  • Problem

    MLLM responses can fail to correspond to the given image, undermining reliability and posing challenges for real-world applications.

    Source: Introduction · p. 1, Multimodal Large Language Models · p. 2

  • Method

    ConVis uses a text-to-image model to reconstruct images from generated captions and contrasts their probability distributions with those of the original image during decoding.

    Source: Introduction · pp. 1–2, Conclusion · p. 7

  • Results

    Across five benchmarks, ConVis reduced hallucinations while maintaining overall response-generation performance across several MLLMs.

    Source: Introduction · p. 2, Conclusion · p. 8

  • Takeaways & Limitations

    ConVis provides a training-free decoding approach that uses text-to-image models as visual contrastive signals to mitigate MLLM hallucinations.

    Source: Introduction · p. 2, Conclusion · p. 7

    On POPE, randomly, popularly, or adversarially sampled object questions may not align with objects visualized from a fixed captioning prompt, limiting the method’s effectiveness.

    Source: Experimental Results · p. 5

Abstract

from arXiv · show

Hallucinations in Multimodal Large Language Models (MLLMs) where generated responses fail to accurately reflect the given image pose a significant challenge to their reliability. To address this, we introduce ConVis, a novel training-free contrastive decoding method. ConVis leverages a text-to-image (T2I) generation model to semantically reconstruct the given image from hallucinated captions. By comparing the contrasting probability distributions produced by the original and reconstructed images, ConVis enables MLLMs to capture visual contrastive signals that penalize hallucination generation. Notably, this method operates purely within the decoding process, eliminating the need for additional data or model updates. Our extensive experiments on five popular benchmarks demonstrate that ConVis effectively reduces hallucinations across various MLLMs, highlighting its potential to enhance model reliability.

Introduction

MLLMs can generate image-inaccurate responses, undermining their reliability, while existing mitigation approaches may require external feedback, additional data, or training. ConVis instead uses a text-to-image model to reconstruct captions and contrast the resulting image with the original during decoding; experiments report reduced hallucinations while maintaining response-generation performance.

Introduction

  • Image-inaccurate MLLM responses undermine reliability and can pose obstacles to use in critical applications such as medicine and autonomous systems.The cited examples include incorrect diagnoses and erroneous interpretations.Source: Introduction · p. 1
  • Some mitigation approaches post-process responses or use instruction tuning, while decoding strategies intervene during generation without additional data or model training.The passage notes that some approaches rely on costly human feedback or additional training.Source: Introduction · p. 1
  • ConVis uses a text-to-image model to reconstruct an image from an MLLM-generated caption, then contrasts its probability distribution with the original image to penalize hallucinated tokens during decoding.A hallucinated caption can produce visual discrepancies between the reconstructed and original images, which provide the contrastive signal.Source: Introduction · pp. 1–2, Figure 2 · p. 2
  • Across five benchmarks, ConVis reduced hallucinations while maintaining overall response-generation performance across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2.The paper presents this as evidence for using text-to-image models as visual contrastive signals in decoding.Source: Introduction · p. 2

Related Work

MLLM hallucinations are image-inaccurate responses that remain a reliability challenge. Prior mitigation includes post-processing, fine-tuning, and decoding strategies; ConVis uses a text-to-image model to visualize caption hallucinations and contrast responses from reconstructed and original images.

Multimodal Large Language Models

  • Despite multimodal architectures, misalignment can produce responses that do not correspond to visual information, undermining MLLM reliability in real-world applications.The section identifies hallucination as a continuing challenge despite advances in multimodal systems.Source: Multimodal Large Language Models · p. 2

Hallucination Mitigation

  • Hallucination mitigation includes post-processing and instruction-tuning approaches, while decoding methods such as OPERA, VCD, and HALC reduce hallucinations without additional training.The cited decoding approaches use token-pattern suppression or contrasts involving distorted images and varying fields of view.Source: Hallucination Mitigation · p. 2
  • ConVis differs by using a text-to-image model to visualize hallucinations in an initial caption, then contrasting responses from the reconstructed and original images.The method contrasts distributions for hallucinated tokens to mitigate hallucinations.Source: Hallucination Mitigation · p. 2

Methodology

ConVis generates an initial image caption, uses a text-to-image model to produce caption-conditioned images, and contrasts their decoding distributions with the original image. Multiple captions and images address potential reconstruction misalignment, while hallucination-associated tokens are penalized in the resulting contrastive distribution.

Preliminaries

  • MLLM responses are generated autoregressively from image and instruction tokens, and this study aims to reduce image-inaccurate outputs while preserving language-model performance.The method uses Hyper-SDXL as its text-to-image model, which progressively reconstructs an image from noise.Source: Preliminaries · pp. 2–3

Hallucination Visualization

  • ConVis prompts the MLLM to caption the original image, then generates an image from that caption; caption hallucinations may be visualized in the generated image.It uses diverse nucleus-decoded captions and corresponding images to broaden potential hallucination coverage and improve robustness to caption-image misalignment.Source: Hallucination Visualization · p. 3

Contrastive Decoding

  • During decoding, ConVis averages contrasts between the original-image distribution and distributions from generated images, then penalizes tokens associated with hallucinations.The contrastive logit distribution is distinguished from related approaches by its use of visual contrast signals from T2I-visualized hallucinations.Source: Contrastive Decoding · p. 3

Experiments

ConVis was evaluated against standard and hallucination-focused decoding baselines on five benchmarks and three MLLM backbones. Results cover hallucination reduction, general-task performance, design analyses, and a VQA-related limitation.

Experiments

  • Experiments use three hallucination benchmarks, two general-purpose benchmarks, and three MLLM backbones, comparing ConVis with standard and specialized decoding methods.Hyper-SDXL is the default T2I model; captions are generated with nucleus sampling, and four images are produced from distinct captions.Source: Experiments · p. 4

Experimental Results

  • ConVis achieves the best CHAIRS performance across three backbones and leads both Figure Accuracy and All Accuracy on HallusionBench with LLaVA-1.5.On CHAIR, its CHAIRI performance ranks first or second across backbones.Source: Experimental Results · pp. 4–5, Table 2 · p. 5
  • On POPE, ConVis reaches new state-of-the-art performance on MiniGPT-4 and outperforms prior techniques on average across backbones.Its performance is comparable to existing techniques on LLaVA-1.5 and mPLUG-Owl2.Source: Experimental Results · p. 5, Table 3 · p. 5
  • On MME with LLaVA-1.5, ConVis leads the Perception category, is competitive in Cognition, and has stronger combined performance than other methods.Its Cognition performance is comparable to OPERA and superior to HALC; VCD excels in that category.Source: Experimental Results · p. 5, Table 4 · p. 5
  • On LLaVA-Bench with LLaVA-1.5, ConVis outperforms existing techniques in complex reasoning, conversation, and detailed description.The benchmark assesses whether language-model capabilities are preserved.Source: Experimental Results · p. 5, Table 5 · p. 5

Analysis and Discussion

  • Using nucleus sampling for captions outperforms greedy search with one generated image, and increasing the number of reconstructed images improves both CHAIR scores.The authors use diverse captions and multiple images to broaden hallucination visualization.Source: Analysis and Discussion · p. 6, Table 7 · p. 6, Figure 4 · p. 6
  • As CLIPScore improves, CHAIR score improves; KL divergence is high for hallucinated token car and lower for non-hallucinated tokens, indicating visual contrastive signals from generated images.Hyper-SDXL significantly outperforms Hyper-SD1.5 in all cases; SDXL-Turbo consistently outperforms Hyper-SD1.5 across all backbones except mPLUG-Owl2.Source: Analysis and Discussion · pp. 6–7
  • ConVis depends strongly on T2I reconstructions, which may not match a VQA question when the caption’s hallucinated objects are unrelated to the queried object.The method uses a fixed captioning prompt; the authors suggest question-specific prompting as future work.Source: Experimental Results · p. 5, Analysis and Discussion · p. 7

Conclusion

ConVis mitigates MLLM hallucinations by using a text-to-image model to reconstruct images from captions and contrast probability distributions from original and reconstructed images. Experiments across five benchmarks show reduced hallucinations while preserving core language-model capabilities, with competitive or superior performance to existing techniques.

Conclusion

  • ConVis contrasts probability distributions from original and caption-reconstructed images to penalize hallucinated content during decoding, without additional data or model retraining.Source: Conclusion · p. 7
  • Across five benchmarks, ConVis consistently reduces hallucinations while preserving MLLMs’ core language-model capabilities.Source: Conclusion · pp. 7–8
  • ConVis achieves competitive or superior performance to existing techniques across various categories, supporting its effectiveness in enhancing MLLM output reliability.Source: Conclusion · p. 8

Benchmarks

The paper evaluates hallucination and language-model performance across five benchmarks, spanning image-caption alignment, visual-context use, object-presence questions, broader MLLM capabilities, and instruction following.

Benchmarks

  • CHAIR evaluates caption-image alignment at sentence and object levels using 500 MSCOCO val2014 images and the detailed-description prompt.Source: Benchmarks · p. 10
  • HallusionBench assesses language hallucinations and visual illusions; experiments use its Visual Dependent category, with paired similar images and binary-answer questions.Source: Benchmarks · p. 10
  • POPE tests object-presence hallucinations with yes-or-no VQA questions, sampling queried objects randomly, popularly, or adversarially.Source: Benchmarks · p. 10
  • MME covers perception and cognition, including fine-grained visual tasks, reasoning, translation, and code reasoning, with yes-or-no answers.Source: Benchmarks · p. 10
  • LLaVA-Bench assesses instruction following across situational questions using 60 MSCOCO val2014 images, comparing generated answers with text-only GPT-4 responses.Source: Benchmarks · p. 10

Additional Implementation Details and Experimental Results

The appendix supplies additional implementation and evaluation detail, including generation-length settings and full benchmark reports for POPE and MME.

Additional Implementation Details and Experimental Results

Related papers

Loading 2408.13906v1…