PaperlayerView exact revision on arXiv

Source-linked AI summary

MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation

Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, Huajun Chen

arXiv:2410.11779v2cs.CLcs.AIcs.CVcs.LGcs.MM

TL;DR

MLLM hallucinations remain poorly understood, including whether visual information is missed or recognized and later suppressed. The paper analyzes this behavior and proposes DeCo, which uses preceding-layer knowledge to correct final logits. Experiments report reduced hallucinations across image captioning and visual question answering benchmarks.

  • Problem

    It remains uncertain whether MLLM hallucinations arise because visual information is never correctly recognized or is recognized but later suppressed.

    Source: §1 · p. 1

  • Method

    DeCo dynamically selects preceding layers and uses their knowledge to adjust final output logits, with soft modulation to preserve response style.

    Source: §1 · p. 2

  • Results

    DeCo achieves a 10.8% average hallucination suppression rate on image captioning and outperforms baselines on POPE and MME visual question answering datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The analysis finds that MLLMs can recognize objects in preceding layers, while deeper-layer knowledge priors diminish the probability of ground-truth tokens.

    Source: §1 · p. 2

    GPU cost limited experiments to a small set of MLLMs, without additional or larger models.

    Source: A LIMITATIONS · p. 16

Abstract

from arXiv · show

Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena, but the underlying reasons remain poorly understood. In this paper, we present an empirical analysis and find that, although MLLMs incorrectly generate the objects in the final output, they are actually able to recognize visual objects in the preceding layers. We speculate that this may be due to the strong knowledge priors of the language model suppressing the visual information, leading to hallucinations. Motivated by this, we propose a novel dynamic correction decoding method for MLLMs DeCo, which adaptively selects the appropriate preceding layers and proportionally integrates knowledge into the final layer to adjust the output logits. Note that DeCo is model agnostic and can be seamlessly incorporated with various classic decoding strategies and applied to different MLLMs. We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines, highlighting its potential to mitigate hallucinations. Code is available at https://github.com/zjunlp/DeCo.

1 INTRODUCTION

The paper examines whether MLLM hallucinations reflect failure to recognize visual objects or suppression of recognized information, and proposes DeCo to correct final outputs using preceding-layer knowledge. Its experiments report reduced hallucinations across image captioning and visual question answering benchmarks.

1 INTRODUCTION

  • The paper identifies an unresolved question: whether visual information is never correctly recognized or is recognized and later suppressed.Hallucinations can involve descriptions of nonexistent image content and omissions of visible objects, with risks in high-stakes applications.Source: §1 · p. 1

Hallucinated MLLM can see (to some extent)

  • The authors find that MLLMs recognize objects in preceding layers, while recognition is suppressed in later layers and knowledge priors reduce ground-truth token probability.The paper focuses this empirical analysis on object hallucinations.Source: §1 · p. 2

Dynamic correction decoding with preceding-layer knowledge

  • DeCo dynamically selects preceding layers and uses their knowledge to adjust final output logits, with soft modulation intended to preserve response style.The method is training-free and described as compatible with different MLLMs and popular decoding strategies.Source: §1 · p. 2

Contributions

  • DeCo reports a 10.8% average hallucination suppression rate on image captioning and outperforms baselines on POPE and MME visual question answering datasets.It introduces an approximate 1.2x latency increase over basic decoding, reported as much faster than VCD and OPERA.Source: §1 · p. 2

2 WHY DO MLLMS GENERATE NON-EXIST OBJECTS?

The section investigates object hallucinations in image captioning and finds that MLLMs can recognize object existence in preceding layers, while ground-truth token probabilities can be suppressed in final outputs.

2 WHY DO MLLMS GENERATE NON-EXIST OBJECTS?

  • The analysis focuses on how MLLMs generate objects in image-captioning scenarios.Source: §2 · p. 2

2.1 FINDING 1: MLLM KNOWS TO SOME EXTENT WHETHER AN OBJECT EXISTS

  • Despite generating non-existent objects, the MLLM maintains around 80% accuracy in probing object existence.Probing accuracy is higher in preceding layers; increasing visual resolution from 224px to 336px enhances accuracy for non-existing objects, and preceding-layer information may enable self-correction.Source: §2.1 · pp. 3–4

2.2 FINDING 2: LANGUAGE MODEL PRIORS SUPPRESS THE VISUAL INFORMATION THAT MLLM ALREADY SEE.

  • Ground-truth tokens are primarily activated between layers 20 and 28 but suppressed in the final output; in examples, their probabilities decline from layer 30 onward, eventually falling below hallucinated tokens’ probabilities in the final layer.Source: §2.2 · p. 4

3 PROPOSED APPROACH: DYNAMIC CORRECTION DECODING WITH PRECEDING-LAYER KNOWLEDGE

DeCo mitigates hallucinations by dynamically selecting a preceding layer and using its knowledge to correct final-layer logits. It combines candidate-token filtering and layer selection with correction that controls the amount of early-layer information.

3 PROPOSED APPROACH: DYNAMIC CORRECTION DECODING WITH PRECEDING-LAYER KNOWLEDGE

  • DeCo dynamically selects preceding-layer knowledge and uses it to correct the final output logits during inference.Its framework comprises dynamic preceding-layer selection and decoding correction.Source: §3 · p. 5

3.1 DYNAMIC PRECEDING-LAYER SELECTION

  • Top-p truncation restricts tracking to candidate tokens and is intended to exclude semantically incorrect tokens that score highly in preceding layers but poorly in the final layer.The paper describes this as a way to keep candidate tokens high quality.Source: §3.1 · p. 5
  • Within intervals such as layers 15–28, the highest-probability candidate token has a high probability of being ground truth, measured as the hit rate.The method dynamically selects the layer where that token has its highest probability; the experiments use layers 20–28 by default to limit search computation.Source: §3.1 · p. 6, Table 1 · p. 6

3.2 DECODING CORRECTION WITH PRECEDING-LAYER KNOWLEDGE

  • DeCo incorporates selected preceding-layer information into the final layer, with α controlling its proportion and dynamic soft modulation intended to preserve the original model’s generative style.The modulation coefficient defaults to the maximum probability and can prevent hard logit changes when candidate-token probabilities in preceding layers are close.Source: §3.2 · p. 6

4 EXPERIMENT

The experiments evaluate DeCo across MLLMs, decoding strategies, and hallucination benchmarks, then analyze its efficiency and design choices. DeCo reduces hallucinations across image captioning and VQA settings, with analysis of latency, layer selection, and error accumulation.

4.1 SETUP; 4.2 BENCHMARK AND METRICS

  • The evaluation uses four 7B MLLMs, compares DeCo with decoding baselines, and assesses captioning and VQA with CHAIR, POPE, MME, and GPT-4o.CHAIR evaluates instance- and sentence-level caption hallucinations; POPE uses object-presence questions across random, popular, and adversarial splits; GPT-4o rates accuracy, detailedness, and coherence.Source: §4.1 · pp. 6–7, §4.2 · pp. 7–8

4.3 EXPERIMENTAL RESULTS

  • Approximately 10.8% average hallucination suppression on image captioning datasets; DeCo consistently outperforms alternatives across four MLLMs and three decoding strategies.The comparison uses baseline decoding settings; Table 2 reports CHAIR results, where lower scores indicate fewer hallucinations.Source: Table 2 · p. 7, §4.3 · p. 8
  • DeCo outperforms other approaches across POPE settings and improves LLaVA-1.5’s MME results, extending its reported benefits to attribute-related and complex reasoning tasks.POPE probes object recognition; MME evaluates multifaceted VQA abilities.Source: Table 3 · p. 7, §4.3 · p. 8, Figure 5 · p. 8
  • GPT-4o evaluation finds higher accuracy than greedy decoding across four models, comparable coherence, and detailedness that is marginally higher or sometimes slightly lower.Table 4 reports accuracy, detailedness, and coherence on MSCOCO.Source: §4.3 · p. 8, Table 4 · p. 8

4.4 ANALYSIS

  • DeCo increases latency approximately 1.2 times, versus 1.8 times for VCD and 5.1 times for OPERA, while balancing effectiveness and computational overhead.The comparison covers latency and throughput across decoding baselines.Source: §4.4 · pp. 8–9, Figure 6 · p. 9
  • Layer-selection perturbations degrade performance, supporting the proposed dynamic selection method.The experiment perturbs predetermined layers with random values from -5 to 5 on 200 MSCOCO images.Source: §4.4 · p. 9, Table 5 · p. 9
  • Hallucination suppression is strongest near α = 0.6; excessive α can produce repetitive descriptions, while layers 20–28 mitigate hallucinations more than layers 1–16 or 29–32.The appropriate layer interval should be adjusted empirically for other MLLM families and larger models.Source: §4.4 · p. 9, Figure 7 · p. 9
  • DeCo reduces the accumulation of errors after an initial hallucination and improves response consistency and accuracy.The paper illustrates this snowballing-hallucination pattern with examples.Source: §4.4 · pp. 9–10

5 RELATED WORK

Prior work explains MLLM hallucinations through image-text interactions across layers and language-model prior bias. Mitigation approaches include training, post-processing, and training-free decoding methods, each with distinct requirements or costs.

5.1 MLLM HALLUCINATION MECHANISM

  • Mechanistic studies examine layer-wise image-text interaction and LLM prior bias, including aggregation patterns and text inertia as proposed explanations for hallucination.Grad-CAM analyses report image-text interaction in preceding layers but not deep layers; VCD finds high-confidence answers even with noisy or absent images.Source: §5.1 · p. 10

5.2 HALLUCINATION MITIGATION FOR MLLMS

  • Mitigation work includes finetuning, post-processing, and training-free decoding; finetuning can require costly paired data, while post-processing adds inference costs and task-specific procedures.Training-free approaches include OPERA, VCD, and VDD.Source: §5.2 · p. 10

6 CONCLUSION

The paper concludes that earlier MLLM layers retain awareness of hallucinated objects, while prior-shaped tokens reduce true-token likelihood in final layers. It presents DeCo as a decoding method based on preceding-layer knowledge and reports efficacy with latency and throughput advantages.

6 CONCLUSION

  • Earlier layers show higher confidence for hallucinated objects, while prior-shaped tokens diminish true-token likelihood in final layers; DeCo uses preceding-layer knowledge to mitigate hallucinations.The conclusion reports extensive experiments and advantages in latency and throughput.Source: §6 · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility statement points readers to supplementary code and the paper’s experimental setup details.

REPRODUCIBILITY STATEMENT

  • Code is in the supplementary materials; benchmarks, prompts, and model hyperparameters are in Section 4, with mechanism experiment setup detailed in Appendix C.1 and C.2.Source: REPRODUCIBILITY STATEMENT · p. 11

A LIMITATIONS

The authors note limits in model coverage and detailedness, compare their findings with prior accounts of hallucination, and describe probing and early-exit experiment settings.

A LIMITATIONS

  • Experiments cover only a limited set of MLLMs, leaving additional models and larger parameter sizes unexplored.This limits the scope of the reported evaluation.Source: A LIMITATIONS · p. 16
  • DeCo has a slight negative impact on detailedness, and balancing truthfulness with diversity remains future work.The authors identify this trade-off in their limitation discussion.Source: A LIMITATIONS · p. 16

B COMPARATIVE ANALYSIS, SUMMARY, AND FUTURE DIRECTIONS

  • Prior studies associate deeper-layer aggregation with hallucinations and find stronger visual perception in preceding layers, aligning with the authors’ findings.The paper also notes similarities between its finding and intermediate-layer hallucination detection in LLMs.Source: B COMPARATIVE ANALYSIS, SUMMARY, AND… · p. 16
  • DeCo shares an assumption about suppressive MLLM knowledge priors with OPERA and VCD, is described as simpler, and can be combined with prior approaches.The authors distinguish this from unimodal LLM accounts involving shallow-layer semantics and factual recall.Source: B COMPARATIVE ANALYSIS, SUMMARY, AND… · p. 16
  • The authors speculate that Transformer attention patterns may lose information, with textual modality overshadowing image-related conditions contributing to visual-perception hallucinations.They propose a unified future investigation of hallucination causes in LLMs and MLLMs.Source: B COMPARATIVE ANALYSIS, SUMMARY, AND… · p. 16

C.1 DETAILED SETTINGS FOR FINDINGS 1

  • Probing uses 1,200 balanced MSCOCO object-existence pairs and AMBER evaluation, while a 224px-versus-336px LLaVA comparison supports a visual-resolution scaling pattern.The AMBER set has a different distribution; the comparison also involves models with different language-model weights.Source: §C.1 · p. 17

C.2 DETAILED SETTINGS FOR FINDINGS 2

  • The early-exit experiment re-inputs preceding text from hallucinated-object examples among 500 MSCOCO images to inspect next-token internal states.The prompts are random and listed in Table 6.Source: §C.2 · p. 17, Table 6 · p. 17

D EVALUATION RESULTS IN AMBER

On AMBER, DeCo shows a truthfulness advantage with some diversity cost; a separate ablation examines dynamic soft modulation and its effect on logits.

D EVALUATION RESULTS IN AMBER

  • DeCo shows a significant truthfulness advantage on AMBER, while diversity is somewhat lower but described as acceptable.The evaluation uses LLaVA-1.5-7B and reports CHAIR, Cover, Hal, and Cog; lower CHAIR, Hal, and Cog indicate higher truthfulness, while higher Cover indicates better diversity.Source: Table 7 · p. 18, D EVALUATION RESULTS IN AMBER · p. 18

E ABLATION RESULTS OF DYNAMIC SOFT MODULATION

F RESULTS OF MMVET

The paper evaluates DeCo on MMVet for LLaVA-1.5-7B and Qwen-VL-7B using their default nucleus-sampling settings.

F RESULTS OF MMVET

  • Table 9 reports DeCo results on MMVet for LLaVA-1.5-7B and Qwen-VL-7B under default nucleus sampling.The supplied passages do not state the results’ direction or values.Source: Table 9 · p. 19, F RESULTS OF MMVET · p. 19

G CASE ANALYSIS ACROSS DIVERSE MLLMS

The case analysis presents DeCo across several MLLMs and decoding methods, alongside examples concerning snowballing hallucination and soft modulation.

G CASE ANALYSIS ACROSS DIVERSE MLLMS

  • Figure 8 presents a case of DeCo mitigating snowballing hallucination.Source: Figure 8 · p. 19
  • Figures 10–12 show DeCo’s hallucination-reduction performance across three basic decoding methods for LLaVA-1.5-7B, Qwen-VL-7B, and MiniGPT4-7B.Figure 9 presents the same type of comparison for InstructBlip-7B.Source: Figure 9 · p. 20, Figure 10 · p. 21, Figure 11 · p. 22
  • Figure 13 illustrates soft modulation avoiding hallucination and erroneous semantics caused by abrupt changes in logits.Source: Figure 13 · p. 24
Loading 2410.11779v2…