Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training

1 Introduction

• Object hallucination can limit VLP performance and raise safety concerns, including reduced diagnostic accuracy and potential severe consequences in biomedical image captioning.Source: §1 · p. 1
• Recent state-of-the-art VLP models still hallucinate frequently, with about 10% of generated sentences containing at least one hallucinated object.The problem is more severe for out-of-domain images; SCST can increase hallucination even while improving CIDEr.Source: §1 · p. 1
• Patch-based features perform best among region-, grid-, and patch-based encodings, and smaller patch resolution non-trivially reduces object hallucination.Source: §1 · p. 2
• ITC and ITM contribute little to alleviating hallucination, while generative ICLM is the main influential factor and pre-training datasets yield distinctive behaviors.Source: §1 · p. 2
• The study systematically examines modern VLP object hallucination and how image encoding and pre-training objectives affect it.The paper frames these investigations around measuring hallucination and finding ways to mitigate it.Source: §1 · pp. 1–2
• 17.4%: ObjMLM reduces object hallucination without introducing extra training data.The loss enhances alignment and restriction between text tokens and visual objects during generation.Source: §1 · p. 2

2.1 Hallucination in Deep Learning

• Hallucination means output unfaithful to conditional input, and prior mitigation approaches include data refinement, visual grounding, reconstruction, decoding, knowledge augmentation, and post-processing.Source: §2.1 · p. 2

2.2 Vision-Language Pre-training

• VLP developed from BERT-style multimodal understanding objectives to encoder-decoder generation with causal language modeling and dual-stream models aligned by image-text contrastive loss.Source: §2.2 · pp. 2–3
• Prior work showed object hallucination in modern VLP, but how VLP strategies and region-, grid-, or patch-based image encodings affect generated-text faithfulness remained unstudied or unclear.Source: §2.2 · p. 3

3.1 Evaluation Metric

• CHAIRi measures the share of predicted objects that are hallucinated, while CHAIRs measures the share of generated sentences containing at least one hallucinated object.Both variants rely on predefined golden object categories.Source: §3.1 · p. 3

3.2 Evaluation Datasets

• Models are finetuned on COCO Caption training data and evaluated on its test set and the NoCaps validation set for in-domain and out-of-domain performance, respectively.Source: §3.2 · p. 3

3.2.1 COCO Caption

• The COCO Caption Karpathy split contains 82K training, 5K validation, and 5K test images, each annotated with at least five captions.CHAIR calculation maps fine-grained object words to 80 coarse-grained categories using synonyms to avoid false hallucination detections.Source: §3.2.1 · p. 3

3.2.2 NoCaps

• NoCaps evaluates generalization to a wider range of visual concepts, including unseen object categories, using its 4,500-image validation set.The evaluation maps 600 classes to 139 coarse-grained object categories; the test set is not used because ground-truth captions are unavailable.Source: §3.2.2 · pp. 3–4

• The models are evaluated after COCO Caption training on both the COCO test set and the NoCaps validation set.Source: §4 · p. 4
• Higher standard scores do not guarantee greater faithfulness: VinVLLarge scores higher than VinVLBase on CIDEr and BLEU-4 but also has higher CHAIR scores.The authors recommend supplementing conventional metrics with CHAIR to assess faithfulness.Source: §4 · pp. 4–5
• SCST raises VinVLBase CIDEr by 11.1 and BLEU-4 by 2.7, while increasing its COCO Caption CHAIRs score by 0.9.SCST optimizes CIDEr using sampled captions and is used after cross-entropy finetuning.Source: §4 · p. 5
• 10.9% more VinVLBase captions contain at least one hallucinated object after SCST, and the problem is more serious on out-of-domain images.The side effect is alleviated for a very well-pretrained model such as OFA large.Source: §4 · p. 5
• After SCST, captions can add detailed but incorrect objects; the paper's example includes a generated “mirror” absent from the image.The authors note this may further amplify hallucination on unfamiliar images, where model confidence may be lower.Source: §4 · p. 5

5 Probing Image Encoding Methods and VLP Objectives

• The experiments probe region-, grid-, and patch-based image features alongside ITC, ITM, and ICLM objectives and their variants.Source: §5 · p. 5

5.1 Model Architecture

• The flexible architecture pairs a visual encoder with a text decoder, allowing the visual encoder to change while the same BertLM decoder is retained.The explored visual encoders include region-based Faster R-CNN, grid-based CLIP ResNet, and patch-based CLIP ViT variants.Source: §5.1 · pp. 5–6

5.2 Effects of Different Image Features

• Patch-based encoders perform best at avoiding hallucination, and smaller patch resolution further reduces it without increasing model size.Grid encoders hallucinate more than comparable-CIDEr alternatives; the authors conjecture CNN inductive biases may weaken connections among an object's characteristics.Source: §5.2 · p. 6

5.3.2 Image-Text Contrastive (ITC) Loss

• Vanilla ITC has negligible influence on caption faithfulness, whereas token-level ITCLate slightly reduces hallucination.The authors attribute ITCLate's benefit to word-patch alignment.Source: §5.3.2 · pp. 6–7

5.3.3 Image-Text Matching (ITM) Loss

• ITM and ITMHard are not highly correlated with hallucination and only slightly reduce it on out-of-domain images.ITMHard has negligible effect on hallucination in downstream generative tasks.Source: §5.3.3 · p. 7

5.3.4 Image-Conditioned Language Modeling

• VG causal-LM pretraining reduces hallucination compared with the baseline even though it does not improve CIDEr.Source: §5.3.4 · pp. 7–8
• VG captions are shorter and focus on specific image aspects, while COCO captions give broader descriptions; VG pretraining therefore creates a fine-to-coarse training process.After adding VG pretraining, captions contain on average 0.08 fewer objects on COCO and 0.24 fewer on NoCaps, and may omit details when the model is uncertain.Source: Figure 2 · p. 7, Figure 3 · p. 7, §5.3.4 · p. 8

5.3.1 Pre-training Datasets

• CC3M improves all reported metrics and general image-translation ability, but the authors say data augmentation alone may not drastically solve hallucination.They identify controllability of vision-conditioned text generation as a promising direction.Source: §5.3.4 · p. 8

6 Object Masked Language Modeling

• ObjMLM masks object words and phrases that appear in the image to improve object-level image-text alignment.It is described as a variant of MLM that masks objects in the text.Source: §6 · p. 8
• The ObjMLM architecture pairs an image encoder with a text decoder and uses ViT-L/14 for the image encoder.The overview allows region-, grid-, or patch-based encoders.Source: Figure 4 · p. 8
• Whole-object masking uses exact matching against COCO and NoCaps object category lists, replacing each object with one [MASK] token.Source: §6 · p. 8
• 17.4%: plugging ObjMLM into an existing VLP setting reduced CHAIRs, compared with a 1.7% reduction from standard MLM.The authors conjecture ObjMLM constrains generation to visible objects and improves recognition of spatial relationships between objects.Source: §6 · p. 8

7 Conclusion

• Recent large VLP models still hallucinate frequently, and SCST harms faithfulness even when it improves standard metrics.Source: §7 · pp. 8–9
• Patch-based input with smaller resolution helps mitigate object hallucination, while token-level alignment and generation controllability are described as crucial.Source: §7 · p. 9
• ObjMLM reduces object hallucination by 17.4% in an existing VLP setting.Source: §7 · p. 9

Limitations

• The investigation and mitigation focus on object hallucination, leaving other VLP hallucination types for future work.Source: Limitations · p. 9
• The recent-model study includes only models with publicly available pre-trained checkpoints.The authors cite limited GPU resources and private pre-training datasets as reasons they could not pre-train unreleased models.Source: Limitations · p. 9

A Implementation Details

• Experiments used PyTorch on 8 Nvidia V100 GPUs, with official CLIP checkpoints and a 110M-parameter bert-base-uncased initialization for BertLM.Source: A Implementation Details · p. 13
• COCO Caption fine-tuning used batch size 512, AdamW for 10 epochs, learning rate 5 × 10−5, weight decay 1 × 10−2, and 0.85 epochwise decay.Source: A Implementation Details · p. 13
• Text-generation pre-training used a first-epoch learning-rate warmup; ITC and ITM used batch size 1024 for more negative samples.Source: A Implementation Details · p. 13

B Additional Case Studies

• For the kitchen image, the ground truth mentions wooden and blue cabinets, while VinVLbase with SCST adds bottles in the cabinetry.Source: B Additional Case Studies · p. 13
• For the backpack image, the ground truth specifies a green camouflage print, while VinVLbase describes a helmet on top of a bag.Source: B Additional Case Studies · p. 13
• For the cocktail image, the ground truth mentions strawberries, while RN50x64 describes red wine and ViT-L/14 with LM & ObjMLM describes glasses of wine.Source: B Additional Case Studies · p. 13
• Figure 5 marks hallucinated objects in red across additional generated-caption examples from different models.Source: Figure 5 · p. 13