Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time

Motivation

• MLLMs can generate outputs inconsistent with or unsupported by visual or auditory evidence despite strong multimodal task performance.The paper frames the central issue as multimodal hallucination.Source: §1 · p. 1
• Prior evidence links hallucinations to under-use of visual or auditory inputs, but leaves unclear whether reduced perceptual reliance contributes to hallucination or merely accompanies it.Text tokens may dominate generation, and textual priors can support plausible visual descriptions even without the corresponding inputs.Source: §1 · p. 1

Findings

• Text tokens dominate relevance, while perceptual relevance is diffuse rather than concentrated on query-relevant evidence.Both patterns tend to be most pronounced on samples the model gets wrong, and intervening on the relevance distribution affects whether it hallucinates.Source: §1 · pp. 1–2

Contributions

• Prior training-free approaches use attention weights or output shifts under perturbed inputs, whereas this work uses LRP relevance as a direct guidance signal.The paper identifies this as the first use of LRP, traditionally a post-hoc explanation method, as inference-time guidance in MLLMs.Source: §1 · p. 2
• LIME shifts relevance toward perceptual tokens by optimizing key-value representations during decoding, without modifying model parameters or requiring spatial or temporal supervision.The authors report consistent hallucination reductions and improved localization across multiple MLLMs and tasks in vision and audio.Source: §1 · p. 2
• Figure 2 depicts LIME increasing modality relevance while keeping the base MLLM frozen and optimizing only Δ.The figure also shows the objective remaining close to the original model distribution.Source: Figure 2 · p. 3

2 RELATED WORK

• Existing mitigation methods include training-based parameter updates and training-free interventions using attention, contrastive decoding, or hidden-representation modification.The related work notes that many training-free approaches rely on heuristic rules or indirect signals rather than explicitly quantifying modality contributions.Source: §2 · p. 3
• Unlike CFT, which uses LRP relevance to fine-tune parameters, this method uses relevance to analyze modality utilization and guide inference-time optimization without labeled training data.The paper states that neither model parameters nor training data are required for its approach.Source: §2 · p. 3

3 METHOD

• LIME adds updates to key and value representations at each decoding step while keeping model parameters fixed.The updates span all transformer layers and the full sequence, are computed independently at each step, and are reset afterward.Source: §3 · p. 3, §3.3 · p. 4

3.1 NOTATION AND SETUP

• The MLLM maps image or audio inputs into perceptual tokens in the same embedding space as textual tokens, which are represented together as input sequence X.The setup distinguishes perceptual token indices P from textual token indices T and defines the next-token distribution at each decoding step.Source: §3.1 · p. 4

3.2 RELEVANCE AND PROBLEM FORMULATION

• LRP propagates relevance backward from the generated-token logit to assign token-level scores, which are aggregated as perceptual and textual contributions.The objective increases relevant perceptual-token contribution while keeping the model close to its original behavior.Source: §3.2 · p. 4

3.3 INFERENCE-TIME OPTIMIZATION

• Figure 3 shows visual relevance becoming more concentrated on relevant regions over successive LIME optimization steps.The relevance is initially diffuse, and the figure describes the shift as indicating improved grounding.Source: Figure 3 · p. 5
• At each decoding step, LIME optimizes key-value updates using relevance and a KL-divergence regularizer that constrains changes relative to the pretrained model.Because query-relevant regions are unavailable at inference, the annotation-free objective treats all perceptual tokens as potentially relevant; relevance nevertheless becomes concentrated on query-relevant regions, approaching the supervised formulation's localization.Source: §3.3 · p. 5

4.1 VISION BENCHMARKS

• On POPE, LIME achieves the highest average accuracy and F1; it also reduces CHAIR hallucination rates and improves HallusionBench metrics across evaluated models.On POPE, the largest improvements over the base model occur on the adversarial split; on CHAIR, some higher-recall methods incur more hallucinations.Source: §4.1 · p. 6, Table 2 · p. 7

4.2 AUDIO BENCHMARKS

• On AHQA, LIME improves accuracy and F1 across sampling strategies, and on AIR-Bench it improves performance across domains, with competitive music-task performance.The reported AIR-Bench gains are consistent in speech and sound understanding.Source: Table 3 · p. 7, §4.2 · p. 7

4.3 MODALITY UTILIZATION ANALYSIS

• LIME increases perceptual utilization and relevance localization across vision and audio, selectively concentrating relevance on query-relevant regions without access to those regions.The evaluation uses 100 positive examples per modality; LIME-Sup targets the annotated relevant region and serves as an analysis upper bound, not a deployable configuration. By seven steps, annotation-free LIME attains 81% and 95% of its improvement over vanilla decoding on the two models.Source: §4.3 · p. 8, Table 4 · p. 8

4.4 ABLATION STUDIES

• Joint key–value updates perform best among the tested strategies, while intermediate KL-regularization values provide the best trade-off between modality utilization and preserving original behavior.Key-only and value-only updates also outperform the baseline; smaller λ permits stronger updates, while larger λ constrains them.Source: §4.4 · p. 9, Figure 4 · p. 9

5 CONCLUSION

• LIME uses the model’s relevance signal to optimize key–value representations during decoding without modifying model parameters or requiring training data.It increases perceptual utilization and concentrates relevance on query-relevant regions without spatial or temporal supervision.Source: §5 · p. 9
• Across vision and audio benchmarks, LIME consistently reduces hallucinations while preserving generation quality.The paper also reports increased perceptual utilization and concentration of relevance on query-relevant regions.Source: §5 · p. 9
• Per-step decoding optimization adds computational overhead, and hyperparameter selection may limit LIME in latency-sensitive or resource-constrained settings.The optimization budget and intervention scope can be adjusted to obtain lower-cost variants while retaining meaningful gains.Source: §5 · p. 9

AI USE STATEMENT

• Generative AI tools were used only to improve the manuscript’s grammar, readability, and clarity; the authors say the research and analyses were theirs.They take responsibility for the final content, including any AI-assisted text or artifacts.Source: AI USE STATEMENT · p. 10

A LAYER-WISE RELEVANCE PROPAGATION FOR TRANSFORMER MODELS

• The appendix follows AttnLRP to propagate token-level relevance from predictions back through transformer components to multimodal inputs.LRP decomposes a scalar output into additive contributions, with relevance preserved across layers.Source: A LAYER-WISE RELEVANCE PROPAGATION FOR… · pp. 13, 15
• Its rules stabilize linear propagation, redistribute relevance through softmax and attention-value interactions, and approximate LayerNorm and RMSNorm with identity mappings.The normalization-layer approximation assumes attribution is preserved while activation scale changes.Source: A LAYER-WISE RELEVANCE PROPAGATION FOR… · pp. 14–15

B.1 EVALUATION BENCHMARKS

• Unless otherwise stated, methods use identical prompting and decoding settings for a fair benchmark comparison.Maximum generation lengths and benchmark-specific prompts are specified for the evaluations.Source: §B.1 · p. 15
• POPE (Li et al., 2023b) probes object hallucination with yes/no image questions, reporting accuracy and F1 across random, popular, and adversarial negatives.Evaluation uses 500 sampled images each from MSCOCO and A-OKVQA, plus a yes-or-no instruction.Source: §B.1 · p. 15
• CHAIR (Rohrbach et al., 2018) measures absent-object mentions in captions using per-instance CHAIRI and per-sentence CHAIRS on 500 MSCOCO validation images.Models generate captions from a generic image-description prompt.Source: §B.1 · p. 16
• HallusionBench (Guan et al., 2024) covers image-context reasoning scenarios and reports figure-level, question-level, easy-subset, and hard-subset accuracy.Scenarios include OCR, charts, tables, mathematical figures, maps, and visual illusions.Source: §B.1 · p. 16
• AIR-Bench (Yang et al., 2024) evaluates generative answers across speech, natural-sound, and music tasks, while Audio Hallucination QA (Kuan et al., 2024) tests yes/no sound-object claims.AIR-Bench uses GPT-4-based correctness judgments; Audio Hallucination QA reports accuracy and F1 across three negative-sampling strategies.Source: §B.1 · p. 16

B.2 BASELINES AND IMPLEMENTATION DETAILS

• At each decoding step, the method applies a small number of gradient-based updates to key and value representations using the Section 3.3 formulation.Adam is used for all inference-time optimization steps.Source: §B.2 · p. 16
• To reduce optimization variables and improve stability, updates are shared across attention heads within each layer.The implementation uses Adam for these updates.Source: §B.2 · p. 16
• After each decoding step, perturbations are reset and reinitialized for the next token.The hyperparameters are reported in Table A.1 unless otherwise stated.Source: §B.2 · p. 16

B.3 ADDITIONAL EXPERIMENTS AND RESULTS

• Supplementary tables extend evaluation to Qwen-VL-Chat, Qwen2.5-VL-7B Instruct, A-OKVQA, and Audio Hallucination QA settings.The supplied captions identify POPE results, random-split results, and HallusionBench results, but do not provide their cell values.Source: Table A.2 · p. 17, Table A.3 · p. 17, Table A.4 · p. 18
• In audio examples, LIME concentrates relevance on correct temporal regions and yields grounded, accurate predictions, unlike diffuse relevance under standard decoding.The waveform overlays relevance scores, with dashed lines marking ground-truth event regions.Source: Figure 5 · p. 18
• Qualitative examples illustrate hallucination reduction with LLaVA-1.5-7B, Qwen-VL-Chat, and Qwen2.5-VL-7B-Instruct.Hallucinated predictions are marked in italics and red in the figure captions.Source: Figure 6 · p. 19, Figure 7 · p. 20, Figure 8 · p. 21

C ANALYSIS OF THE RELEVANCE OBJECTIVE

• The relevance objective shifts aggregate relevance toward perceptual tokens without specifying target tokens, using prediction correctness, auxiliary models, or labeled data.This is presented as a trade-off for a training- and external-model-free method applied directly at inference time.Source: C ANALYSIS OF THE RELEVANCE OBJECTIVE · p. 22
• The appendix describes per-sample and controlled-intervention analyses of relevance shifts, plus a mechanistic analysis of why relevance becomes query-focused.The per-sample table groups relevance shifts by prediction outcome; the passage states these analyses address two questions but does not report their findings.Source: C ANALYSIS OF THE RELEVANCE OBJECTIVE · p. 22, Table A.7 · p. 22

C.1 INTERVENTIONAL ANALYSIS OF THE RELEVANCE-HALLUCINATION LINK

• +0.121 versus +0.031 in perceptual utilization and +0.147 versus +0.026 in relevance localization distinguish corrected predictions from those that remain hallucinated.Already-correct samples show intermediate shifts; the correct-to-hallucinated group contains only two samples and is not interpreted.Source: §C.1 · p. 22
• On POPE-MSCOCO, targeting perceptual relevance improves accuracy while reversing it degrades accuracy; matched-magnitude Random stays at baseline (83.35 vs. 83.24).Suppressing relevance on the query-relevant region causes a larger drop (−3.42) than suppressing it across all perceptual tokens (−0.59); LIME-Sup improves on LIME (+1.68).Source: §C.1 · p. 23, Table A.8 · p. 23

C.2 MECHANISM BEHIND RELEVANCE LOCALIZATION

• The relevance objective favors equal shares across perceptual tokens and zero share on text, so by itself it does not target the query-relevant subset.At its stationary point, the fraction of perceptual relevance inside G would equal |G|/|P|.Source: §C.2 · p. 24
• Near zero perturbation, the KL term penalizes changes quadratically in the Fisher metric, assigning higher cost to directions that more strongly change the next-token distribution.LIME reinitializes its key-value perturbations to zero at each decoding step.Source: §C.2 · pp. 24–25
• The frozen model filters the broadly favored relevance direction through its sensitivity structure, concentrating realized increases on query-relevant tokens despite the objective lacking access to that subset.Tokens with similar sensitivity to perturbation directions allowed by the KL constraint tend to undergo similar relevance changes.Source: §C.2 · p. 25

• The analysis varies optimization budget and intervention scope, measures latency and memory overhead, and discusses practical limitations.The computational overhead measurements use the main experiments’ decoding configuration.Source: D EFFICIENCY ANALYSIS AND LIMITATIONS · p. 25
• Figure 9 compares optimization steps when updates are applied to early, middle, late, or all transformer layers.Source: Figure 9 · p. 26
• Table A.9 compares computational overhead for vanilla decoding, representative baselines, and LIME across optimization budgets.Source: Table A.9 · p. 26

D.1 EFFICIENCY ANALYSIS

• Increasing optimization steps improves accuracy across both models and every intervention scope; applying updates to all layers performs best at each budget.Three steps already provide substantial improvements over the base models, and selected layer ranges remain effective.Source: §D.1 · p. 26
• For LLaVA-1.5-7B, slowdown rises from 6.73× at five steps to 9.43× at seven, while peak memory remains essentially unchanged across budgets.Source: §D.1 · p. 26
• At five versus seven steps, LLaVA-1.5-7B accuracy is 88.78% versus 90.27%, and Qwen2-Audio-7B accuracy is 61.5% versus 63.36%.Source: §D.1 · p. 26
• LIME makes the efficiency–performance trade-off controllable through optimization budget and layer scope, unlike baselines with fixed inference configurations.Source: §D.1 · pp. 26–27

D.2 LIMITATIONS

• Inference-time optimization adds non-negligible latency, and reducing it through fewer steps or narrower scope costs some performance; hyperparameter optima may vary by model and task.These constraints may limit use in latency-sensitive or resource-constrained settings.Source: §D.2 · p. 27