Source-linked AI summary
Detecting Object Hallucinations in Large Vision-Language Models via Cross-Modal Attention Drifts and Mask-Based Verification
Xuanbing Wen, Boxu Chen, Le Yang, Jiakai Wang, Zhengyu Zhao, Chenhao Lin, Chao Shen
TL;DR
Object hallucination threatens LVLM reliability, while existing attention-based detectors underexplore how visual grounding evolves across layers. CADMP combines adjacent-layer cross-modal attention drift with targeted masking and probability variation, achieving the best average ACC–AUROC in eight of nine primary settings across three LVLMs and four benchmarks.
Problem
Object hallucination limits LVLM reliability, and changes in full visual-token attention distributions between consecutive layers remain underexplored for detection.
Method
CADMP computes adjacent-layer KL-divergence drift, masks regions identified before the maximum-drift transition, and combines probability changes with drift features in a lightweight detector.
Results
CADMP achieves the best average ACC–AUROC in eight of nine primary settings across three LVLMs and four benchmarks.
Takeaways & Limitations
Attention drift and mask-based probability verification provide complementary signals for detecting object hallucinations across architectures and evaluation settings.
Abstract
from arXiv · showhide
Despite recent advances in large vision-language models (LVLMs), object hallucination remains a major barrier to their reliable deployment. Existing detection methods often characterize visual grounding using attention from individual layers, leaving its evolution across layers underexplored. We propose CADMP, a lightweight object hallucination detection framework that combines adjacent-layer cross-modal attention drift with prediction sensitivity to targeted visual masking. During decoding, CADMP quantifies distributional changes between consecutive cross-modal attention maps to capture abrupt transitions in visual grounding. It then selects the transition with the largest drift, locates the corresponding visually relevant regions, and measures the change in prediction probability after masking these regions. These two signals provide complementary evidence: attention drift characterizes the stability of internal visual grounding, while probability variation verifies whether a prediction truly depends on the identified visual evidence. A lightweight detector integrates both signals to identify hallucinated predictions. Experiments on multiple benchmarks and representative open-source LVLMs demonstrate that CADMP achieves consistently competitive detection performance. Ablation studies further confirm the complementary contributions of adjacent-layer drift modeling and mask-based grounding verification.
Introduction
Object hallucination limits LVLM reliability because models may mention objects absent from or unsupported by images. CADMP addresses an underexplored gap by combining adjacent-layer attention drift with mask-based probability verification.
- Object hallucinations occur when LVLMs mention objects absent from or unsupported by the input image, compromising factual consistency and potentially creating safety risks.
- Existing detectors use external annotations, evaluators, or reasoning pipelines, whereas internal methods rely on uncertainty, hidden representations, or cross-modal attention.
- Cross-modal attention is relevant to visual grounding because it captures interactions between output tokens and visual tokens.
- Prior work leaves changes in the full visual-token attention distribution between consecutive layers underexplored for hallucination detection.
- CADMP computes adjacent-layer KL divergence, masks regions selected before the maximum-drift transition, and measures the same token’s probability change.
- CADMP combines the drift profile, pre- and post-masking probabilities, and their difference in a small MLP while keeping the target LVLM frozen.
- CADMP achieves the best average ACC–AUROC in eight of nine primary settings across three LVLMs and four benchmarks.
Related Work
LVLMs combine vision and language components for multimodal generation, but their outputs can conflict with visual input or factual reality. Hallucinations include object, attribute, and relational errors, motivating detection methods based on output uncertainty and other signals.
- LVLMs typically combine a vision encoder, a language model, and a connector to support multimodal understanding and generation.
- LVLM hallucination refers to generated output that is inconsistent with visual input or factual reality.
- Hallucinations are commonly categorized as object, attribute, and relational errors.
- Object hallucination concerns mentioning nonexistent image objects, while attribute and relational hallucinations concern incorrect properties or object relationships.
- Existing detection methods include uncertainty-based approaches, alongside broader efforts to improve LVLM reliability through hallucination detection.
Method
CADMP models how cross-modal visual grounding changes across decoder layers, then verifies prediction dependence by masking drift-guided regions. A lightweight detector combines these complementary signals without changing the target LVLM.
- Adjacent-Layer Cross-Modal Attention Drifts: CADMP computes KL divergences between consecutive cross-modal attention distributions to capture layer-wise grounding drift.
- Adjacent-Layer Cross-Modal Attention Drifts: The attention sequence averages cross-modal attention from the generated token over heads and represents visual-token distributions across decoder layers.
- Adjacent-Layer Cross-Modal Attention Drifts: Each drift value measures redistribution between adjacent visual-token attention distributions, with larger values indicating stronger change.
- Adjacent-Layer Cross-Modal Attention Drifts: Hallucinated objects show stronger average attention drift in middle layers, particularly around layers 8-16, than grounded objects.
- Drift-guided Adaptive Masking: CADMP selects the layer preceding the maximum drift, uses its attention map to identify relevant regions, and thresholds that map to form a binary mask.
- Mask-based Verification: The masked image is reprocessed to obtain the same token’s probability, enabling comparison with its original probability and calculation of probability variation.
- Hallucination Detector: A lightweight MLP receives the KL profile, probabilities before and after masking, and their difference to output a hallucination score.
- Hallucination Detector: Weighted random sampling addresses class imbalance between hallucinated and grounded samples during detector training.
Experiments
CADMP is evaluated across multiple datasets, LVLMs, metrics, and transfer settings, showing strong and generally consistent object-hallucination detection performance.
- Benchmark Results: CADMP achieves the best result in 16 of 18 model–dataset–metric combinations.The comparison covers ACC and AUROC across three models and three datasets.
- Benchmark Results: 5.6, 8.4, and 4.6 percentage points: CADMP’s AUROC gains over the strongest baseline on COCO-Caption for LLaVA-1.5-7B, Qwen2.5-VL-7B, and InternVL-2.5-4B, respectively.These gains are reported for the three evaluated models in that order.
- Benchmark Results: 97.5% ACC and 94.9% AUROC: CADMP ranks first on Pascal VOC across all three models.On POPE, it remains competitive, with two model–metric results slightly below the best baselines.
- Efficiency and Generalization: CADMP consistently outperforms representative baselines across object, attribute, and relation hallucinations on AMBER using Qwen2.5-VL-7B.The result extends evaluation beyond object hallucination detection.
- Efficiency and Generalization: 88.2% accuracy with faster inference speed: CADMP outperforms the forward-based detection method while maintaining lower practical overhead than GLSim without an additional masked-image forward pass.The efficiency analysis compares latency and ACC on InternVL-2.5-4B using POPE.
- Efficiency and Generalization: CADMP consistently outperforms DHCP across target datasets and hallucination types in source-to-target evaluation without additional adaptation.Reported AUROC scores include 73.2% on AMBER-object, 76.7% on AMBER-Attribute, and 66.4% on AMBER-relation; favorable performance is also maintained on Pascal VOC.
Ablation Studies
Ablation studies identify adjacent-layer KL drift and attention-guided masking as effective design choices, while showing that combining drift and probability signals improves detection.
- Drift Formulation: 85.6% average AUROC: KL divergence outperforms cosine similarity, Euclidean distance, and Pearson correlation for cross-modal attention drift.The comparison is conducted across different models on MSCOCO.
- Masking Choices: The layer immediately preceding the largest drift achieves the best performance among previous-layer, subsequent-layer, fixed-middle-layer, and final-layer selection strategies.This finding is reported on LLaVA-1.5 and Qwen2.5-VL.
- Masking Choices: Top-attention masking achieves the highest AUROC and ACC under a fixed 30% masking ratio and identical replacement method.It outperforms random, least-attention, and center masking, indicating that the removed regions correspond to attended visual evidence.
- Component Contributions: 87.4% AUROC: combining attention drift and probability features improves over 85.7% for attention drift alone and 72.8% for probability alone on LLaVA-1.5-7B.The same improvement trend is observed on Qwen2.5-VL-7B and InternVL-2.5-4B.
- Drift Formulation: Adjacent-layer KL consistently outperforms first-anchor, last-anchor, and first-last formulations on Pascal VOC.The result supports modeling fine-grained grounding transitions rather than deviations from a fixed anchor or one global discrepancy.
- Qualitative Evidence: Hallucinated cases show stronger intermediate-layer drift and smaller post-masking probability drops, whereas grounded cases show milder drift and greater perturbation sensitivity.Qualitative visualizations use these contrasting patterns to support the detector design.
Conclusion
CADMP detects object hallucinations by combining layer-wise attention shifts with mask-based probability verification. Experiments across three representative LVLMs demonstrate effectiveness and generalizability across hallucination detection settings.
- CADMP measures distribution shifts between layers and verifies visual grounding through mask-based probability variation.
Further Ablation Studies
Ablations examine masking, token selection, drift-feature representation, and layer locations. Results favor moderate masking, first-token selection, the full KL sequence, and intermediate-layer drift analysis.
- Masking Ratio: 87.97% AUROC and 82.88% ACC are achieved with a 30% masking ratio on MSCOCO using LLaVA-1.5-7B.Smaller ratios may miss key object regions, while larger ratios can introduce substantial visual perturbations.
- Mask Filling: 87.97% AUROC and 82.88% ACC are achieved when masked regions are filled with gray value 128.Gray filling outperforms black, white, and the ImageNet mean RGB value (123, 116, 103).
- Token Selection: The first-token strategy is most effective for multi-token objects across three LVLMs on MSCOCO.The paper suggests that the first token often captures the object's core semantic meaning.
- KL-Drift Features: 87.97% AUROC and 82.88% ACC are achieved by the full KL sequence, outperforming compressed drift representations.Statistical summaries reduce AUROC by 7.39 percentage points, while maximum KL reduces it by 15.29 points.
- Drift Location: For hallucinated Qwen2.5-VL-7B samples, maximum attention drift most frequently occurs at Layers 15, 10, and 13.These layers are intermediate layers, where abrupt cross-modal attention redistribution tends to emerge.
Experiment Datasets
The evaluation uses COCO-based captioning and question-answering settings, Pascal VOC, and AMBER to test object hallucination detection across diverse datasets and task formats.
- COCO-Caption: COCO-Caption uses 5,000 COCO-val2014 images and independently generated captions from each evaluated LVLM.Captions are generated greedily with up to 128 newly generated tokens, and CHAIR extracts object mentions.
- COCO-Caption: COCO-Caption data are split by image at 70%/10%/20% for training, validation, and testing to prevent information leakage.Image-level splitting avoids placing objects from one captioned image in different subsets.
- POPE: POPE evaluates object-existence hallucinations as binary questions about whether queried objects are present in images.Its subsets are Random, Popular, and Adversarial, constructed from MSCOCO images.
- POPE: The POPE implementation uses 1,500 images with three positive and three negative questions per image, totaling 9,000 image-question pairs.
- Pascal VOC: Pascal VOC provides image-level and object-level annotations for 20 common object categories across varied visual scenes.
- Pascal VOC: The Pascal VOC evaluation set contains 9,000 samples formed from one present-object query and three absent-object queries per image.Images come from the VOC2012 split.
- AMBER: AMBER contains 1,004 images and supports generative and discriminative hallucination evaluation without an external language-model evaluator.The discriminative setting uses targeted questions about visually supported and unsupported claims.
- AMBER: AMBER's discriminative component covers object-existence, attribute, and relation hallucinations.
Baselines
CADMP is compared with uncertainty-, confidence-, visual-grounding-, similarity-, and cross-modal-attention-based detection baselines, using reported configurations for implementation.
- CADMP is compared with six representative object hallucination detection baselines.
- NLL: NLL uses the generation probability of the target object token as the hallucination score, with larger scores indicating higher hallucination likelihood.For multi-token objects, the first token is used.
- Entropy: Entropy measures predictive uncertainty from the vocabulary distribution at the target object token, with higher entropy indicating greater uncertainty.
- IC: IC projects visual-token hidden representations into vocabulary space and uses the maximum target-object probability across decoder layers and visual positions.A higher IC score indicates stronger internal visual support.
- SVAR: SVAR measures visual attention received by a generated object token by averaging attention ratios across heads and summing them over Layers 5–18.Higher SVAR indicates more attention assigned to visual information.
- GLSim: GLSim jointly evaluates global scene-level consistency and local visual grounding to assess semantic compatibility and regional evidence.
- DHCP: DHCP models cross-modal attention assigned by generated tokens to visual tokens across decoder layers as internal evidence for hallucination detection.
- Implementation: For LLaVA-1.5-7B on COCO-Caption, baseline implementations aggregate visual attention over Layers 5–18 for SVAR and use configured image-text layers and top visual tokens for GLSim.
Experiment Settings
CADMP is evaluated across three open-source LVLMs using a cross-modal attention-based procedure and a standardized lightweight detector. Experiments use specified training, validation, testing, and evaluation protocols.
- Models and Hardware: Evaluation covers LLaVA-1.5-7B, Qwen2.5-VL-7B-Instruct, and InternVL2.5-4B, spanning different architectures, alignment strategies, and parameter scales.Experiments run with Python 3.10 and PyTorch 2.6.0 on one NVIDIA A6000 GPU with 48GB memory.
- Attention-Based Localization: CADMP selects the layer preceding the maximum attention-drift transition, reshapes its visual-token attention into an H×W map, and upsamples it to image resolution.Bilinear interpolation assigns a continuous attention weight to each pixel.
- Detector Inputs: The COCO-Caption detector uses layer-wise KL-divergence drift and three probability features: original-image probability, masked-image probability, and their difference.Features are standardized before entering the detector.
- Detector Training: The detector is a two-hidden-layer network with 128 and 64 neurons, Batch Normalization, ReLU, and 0.3 Dropout, trained with AdamW.Training uses learning rate 5×10−4, batch size 128, weight decay 1 × 10−4, up to 150 epochs, and early stopping with patience 20.
- Evaluation Protocol: Data are split 70%/10%/20% into training, validation, and test sets, while threshold selection maximizes validation F1 and results average three random seeds.AUROC treats hallucinated samples as positive; reported results are means and standard deviations across seeds 42, 7777, and 37.
Visualization of CADMP
Figure 7 provides qualitative visualization examples of CADMP's hallucination-detection effectiveness on POPE and COCO-Caption.
- Visualization Examples: Figure 7 presents qualitative CADMP visualization examples from both the POPE and COCO-Caption datasets.The figure is used to examine CADMP's effectiveness qualitatively.