PaperlayerView exact revision on arXiv

Source-linked AI summary

Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models

Andrés Villa, Juan Carlos León Alcázar, Alvaro Soto, Bernard Ghanem

arXiv:2312.02219v3cs.CVcs.CL

TL;DR

IT-LVLMs lack a standardized benchmark for fundamental computer-vision tasks, and their outputs may not be visually grounded. MERLIM evaluates these models with open-ended responses and image edits, finding hidden hallucinations and strong language-query biases that can affect reported performance.

  • Problem

    IT-LVLMs lack a standard test-bed for assessing zero-shot effectiveness on fundamental computer-vision tasks, while their language outputs may not be grounded in visual input.

    Source: §1 · p. 1

  • Method

    MERLIM combines open-ended prediction parsing with image and ground-truth edits to evaluate IT-LVLMs on core vision tasks and identify regular and hidden hallucinations.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Results

    MERLIM finds that hidden hallucinations can unfairly increase IT-LVLM accuracy, alongside weak visual grounding and strong bias toward language tokens in questions and answers.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Seemingly correct predictions may lack visual grounding, so MERLIM distinguishes correctness from grounding through image edits and repeat queries.

    Source: §1 · p. 2, §5 · p. 8

    Relationship-grounding assessment is ambiguous when multiple objects of the same class are present, since the target relationship may also occur between other entities.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Large Vision and Language Models have enabled significant advances in fully supervised and zero-shot visual tasks. These large architectures serve as the baseline to what is currently known as Instruction Tuning Large Vision and Language models (IT-LVLMs). IT-LVLMs are general-purpose multi-modal assistants whose responses are modulated by natural language instructions and visual data. Despite this versatility, IT-LVLM effectiveness in fundamental computer vision problems remains unclear, primarily due to the absence of a standardized evaluation benchmark. This paper introduces a Multi-modal Evaluation Benchmark named MERLIM, a scalable test-bed to assess the capabilities of IT-LVLMs on fundamental computer vision tasks. MERLIM contains over 300K image-question pairs and has a strong focus on detecting cross-modal "hallucination" events in IT-LVLMs. Our results bring important insights on the performance of state-of-the-art IT-LVLMs including limitations at identifying fine-grained visual concepts, object hallucinations across tasks, and biases towards the language query. Our findings also suggest that these models have weak visual grounding, but manage to make adequate guesses from global visual patterns or language biases contained in the LLM component. We name this phenomenon of correct answers with no visual grounding as hidden hallucinations.

1. Introduction

Instruction Tuning Large Vision and Language models (IT-LVLMs) lack a standard test-bed for fundamental computer-vision tasks, and their language outputs may not be grounded in images. MERLIM evaluates these models using open-ended responses and image edits to investigate hallucinations and visual grounding.

1. Introduction

  • The benchmark pairs open-ended natural-language predictions with image and ground-truth edits to distinguish regular hallucinations from seemingly correct predictions lacking visual grounding.The latter are termed hidden hallucinations; a “Tennis Racquet” prediction on an edited image is one example.Source: §1 · pp. 1–2
  • MERLIM compiles image edits, ground truths, and a parsing strategy into an extensible test-bed for object recognition, instance counting, and object-to-object relationships.It evaluates these tasks on established datasets including MS-COCO [35], LVIS [17], and Visual Genome.Source: §1 · p. 2
  • MERLIM’s evaluation identifies hidden hallucinations that can unfairly increase IT-LVLM accuracy and finds strong bias toward language tokens in questions and answers.The benchmark assesses object recognition, counting, and inter-object relationships.Source: §1 · p. 2

2. Related Work

  • Prior VQA benchmarks use predefined questions but do not assess whether responses rely on global image context or local visual patterns; MERLIM tests grounding through image alterations.It also checks the same query on modified visual data to investigate possible LLM hallucinations.Source: §2 · p. 2
  • POPE [32] probes presence or absence of a single instance, while MERLIM modifies visual ground truth; NExT-GQA [51] uses temporal localization, which can remain language-biased [42].Source: §2 · p. 2

3. MERLIM

MERLIM provides a standardized evaluation scheme for IT-LVLMs across three fundamental computer vision tasks, assessing both task performance and visual grounding. It parses open-language responses into task-compatible predictions and uses edited images to test whether answers remain grounded.

3. MERLIM

  • Across 300,664 image-question pairs, MERLIM evaluates recognition, counting, and relationships, parsing open responses into closed-set predictions and testing grounding against original and edited images.Recognition and counting use MS-COCO; relationships use Visual Genome labels on an overlapping subset. Edited images remove one object instance at a time, and MS-COCO ground-truth classes are extended with LVIS labels.Source: §3 · p. 3, Table 1 · p. 4

3.1. Object Recognition

  • For object recognition, models list visible objects in response to five prompts, and noun predictions are matched to ground-truth labels or synonyms to calculate precision, recall, and F1.The evaluation uses MS-COCO ground truth adapted for multi-label classification and WordNet plus a precomputed noun-relation corpus to manage vocabulary variation.Source: §3.1 · p. 4

3.2. Inter-object Relationship Understanding

  • For relationships, yes/no questions are evaluated on original and edited images, with Random and Curated negative sets addressing duplicate-class ambiguity and testing language-based shortcuts.The Curated Set uses more plausible relationships, while the Random Set may include nonsensical ones; both sets are used to analyze language bias and possible language-only shortcuts.Source: §3.2 · pp. 4–5

3.3. Object Counting

  • For counting, models give numeric counts on original and edited images, then answer follow-up yes/no consistency questions; evaluation uses absolute error and follow-up accuracy.The edited image should contain exactly one fewer object, allowing consistency to be examined even when the initial count is not the ground truth.Source: §3.3 · p. 5

4. MERLIM Results

MERLIM evaluates IT-LVLMs across object recognition, counting, and inter-object relationships, comparing responses on original and edited images to probe hallucinations and visual grounding. Across tasks, performance gaps on edited images and sensitivity to prompts reveal substantial grounding weaknesses and language-related shortcuts.

4.1. Object Recognition

  • Only BLIP-2 and Kosmos-2 had consistent performance across all five instructions; BLIP-2’s average F1 was 40.55%, while its instruction-aware successor InstructBLIP was less consistent.The authors suggest InstructBLIP’s instruction-aware Q-former may make it more susceptible to phrasing changes.Source: §4.1 · p. 6
  • The five models with precision near or above 50% on at least one selected original-image prompt had an average precision drop of 8.05% on edited images.These were InstructBLIP, InternLM-XComposer2-VL, BLIP-2, LLaVA-1.5, and xGen-MM; the authors identify them as the methods suffering most from hidden hallucinations.Source: §4.1 · p. 6
  • Some top-performing models retain predictions for objects removed by inpainting, and prior answer tokens have larger average gradient contributions than the image.The authors suggest these models may gain performance by prioritizing global visual context or previously predicted language tokens over effective visual grounding.Source: §4.1 · pp. 6–7

4.2. Inter-object Relationship Understanding

  • No IT-LVLM consistently led across relationship scenarios, while LLM-only models outperformed a random baseline despite having no visual input.The authors interpret the edited-set relationship gap as evidence that models may rely on global visual context and language-model biases.Source: §4.2 · p. 7
  • In the curated relationship set, edited-image accuracy was lower by 11.63%, and the original–edited accuracy gap rose to 16.15%, more than double the random-set gap.No IT-LVLM consistently outperformed the others across evaluated scenarios; Qwen-VL-Chat performed best for affirmative relationships in both image sets.Source: §4.2 · p. 7, Table 2 · p. 7

4.3. Object Counting

  • All tested IT-LVLMs had significantly higher counting error on edited images, which the authors attribute to weak visual grounding and hallucinations.LLaVA showed the highest answer consistency, with over 80% of replies confirming its initial assessment across both sets, whether or not the count was accurate.Source: §4.3 · p. 8, Figure 4 · p. 8

5. Discussion & Conclusions

MERLIM identifies hidden hallucinations: predictions that appear correct but lack visual grounding, potentially inflating model scores. The authors also report prompt sensitivity and note that the full benchmark’s size may limit its usability.

5. Discussion & Conclusions

  • Hidden hallucinations are seemingly correct predictions without meaningful visual grounding, and may partly increase top-performing models’ scores.The authors identify language biases and insufficiently robust, detailed visual grounding as potential causes.Source: §5 · p. 8
  • Semantically equivalent prompts can produce markedly different outputs and performance, indicating that IT-LVLM responses remain strongly biased toward the selected query set.The authors plan to release MERLIM queries and results to support further investigation of these discrepancies.Source: §5 · p. 8
  • MERLIM’s 300,664 image-question pairs may limit usability for evaluating and prototyping new models.The authors recommend the object-recognition subset, which includes 5,608 edited and 3,037 original images.Source: §5 · p. 8
  • The authors conclude that IT-LVLMs perform poorly overall on fundamental computer-vision tasks, with failures associated with weak or absent visual grounding.They present MERLIM as an initial effort toward diagnosing hallucinations and suggest stronger grounding could improve prediction quality.Source: §5 · p. 8

Image-Language Models

The supplementary material summarizes selected task and metric comparisons, then examines inpainting, language-only comparisons, prompt biases, object hallucinations, and input-gradient contributions.

Image-Language Models

  • The supplementary material compares selected tasks and metrics in Figure 9, then analyzes inpainting and IT-LVLM versus LLM-only performance.It also examines nouns generated across five prompts, full precision and recall on original images, and gradient contributions from images, questions, and answers.Source: Image-Language Models · p. 12

6. Analysis of Inpainted Images

MERLIM’s inpainting edits largely preserve the surrounding visual content while removing target instances. Detector comparisons and visual examples support the authors’ conclusion that the observed performance gaps are not attributable to the measured image discrepancies.

6. Analysis of Inpainted Images

  • The inpainting selection required no YOLOv7 detection with IoU > 0.7 to the removed instance and ResNet50 feature cosine similarity above 0.7.These criteria tested object removal and preservation of global image features, respectively.Source: §6 · p. 12
  • Mask R-CNN checked whether edited-image boxes had same-class, spatially overlapping counterparts in the original images.The analysis was performed for box confidence scores above 0.7 and 0.5.Source: §6 · p. 12
  • 96.8% of 762782 boxes in inpainted images matched original-image boxes of the same class at IoU > 0.7.Only 4455 boxes (about 0.59%) could not be mapped to same-class original boxes with at least 0.5 IoU.Source: §6 · p. 12
  • Up to 3290 Mask R-CNN boxes, or 0.7% of the total, lacked high overlap with original boxes across the two confidence thresholds.Table 3 analyzes predictions above 0.7 and above 0.5 confidence.Source: Table 3 · p. 12
  • The authors attribute MERLIM performance gaps to IT-LVLM hallucinations rather than minimal discrepancies measured by ResNet50, YOLOv7, and Mask R-CNN.Figure 5 examples are described as visually indistinguishable from originals, even after large-object removal.Source: §6 · p. 12

7. Object Hallucination

The analysis links object-hallucination patterns to response length and visual grounding. BLIP-2 gives shorter answers with higher precision but lower recall, while larger instruction-tuned models tend toward longer answers, higher recall, and lower precision.

7. Object Hallucination

  • BLIP-2 produces the shortest answers by noun count and consistently has higher precision but lower recall than other methods.The authors suggest its concise-caption training may contribute to lower hallucination propensity.Source: §7 · p. 12
  • Figure 5 presents eight object-removal examples described as visually seamless, even when large objects are removed.The edits are said to be nearly indistinguishable from the original images.Source: Figure 5 · p. 13
  • Larger instruction-trained models tend to produce longer answers, with higher recall but lower precision than BLIP-2.The authors connect longer answers and insufficient visual grounding with increased likelihood of hallucinations.Source: §7 · p. 13

8. LLMs vs IT-LVLMs

The section compares IT-LVLM responses with language-only answers and examines prompt-dependent output and image-edit effects. It concludes that language priors can serve as shortcuts for visual relations, while their usefulness depends on whether they align with visual grounding.

8. LLMs vs IT-LVLMs

  • The language-only comparison prompts LLMs with original visual-relationship questions and permits “do not know” responses.Accuracy is reported both penalizing “I don’t know” and considering only yes/no answers.Source: §8 · p. 13, Table 4 · p. 15
  • On the original image set, BLIP2 predicts a consistent noun count across all formulated prompts, unlike other methods; the biggest models, including InstructBLIP and MiniGPT4 with Vicuna13B and LlaVa, achieve highest recall and lowest precision.Source: Figure 6 · p. 14, Figure 7 · p. 14
  • All methods lose precision on edited images when inpainting removes an entire category, and the figure also reports F1 on original and edited images.The comparison uses the same question for both image sets.Source: Figure 8 · p. 15
  • With yes/no/do-not-know options, FlanT5xl exceeds the 33% random chance baseline and narrows the performance gap between LLMs and IT-LVLMs.Vicuna performs below random overall but improves when only yes/no responses are considered, especially Vicuna-13B.Source: §8 · pp. 14–15
  • Language models can act as strong priors for visual relations without image input, providing textual shortcuts that may align with or conflict with visual grounding.The authors associate these shortcuts with performance differences between Random and Curated sets.Source: §8 · p. 16

9. Gradient Analysis

Gradient analysis finds that autoregressive models give language inputs more influence than visual inputs when predicting answers. Visual gradient contribution is also concentrated in relatively few output tokens.

9. Gradient Analysis

  • The analysis attributes each predicted token’s gradient among image, question, and answer inputs, averaging contributions across answers for autoregressive architectures such as Vicuna.It examines output logits before token selection and uses maximum input activation per output token.Source: §9 · p. 16
  • In Figure 5, visual-input gradients are significantly lower than gradients for the question and answer language inputs.The authors interpret this pattern as language-token prioritization that leads to hallucinations based on language biases.Source: §9 · p. 16
  • For InstructBLIP with Vicuna-7B, 10 of 30 tokens account for nearly 73% of visual gradients.The authors say this concentration makes strong, specific visual information challenging to adequately consider.Source: §9 · p. 16

10. Number of unsuitable outputs

For Object Recognition, some IT-LVLMs give answers without valid object nouns, which are scored as zero for precision and recall. InstructBLIP consistently produces valid object lists, unlike the models with the most unsuitable answers.

10. Number of unsuitable outputs

  • MERLIM reports Object Recognition precision, recall, and F1, plus inter-object relationship accuracy for randomly sampled and curated sets and affirmative and negative instructions.Source: Figure 9 · p. 16
  • Object Recognition replies without valid nouns receive precision and recall scores of 0.Examples include “There are many objects” and “It is a beautiful scene”.Source: §10 · p. 16
  • MiniGPT-4 with Vicuna-7B v0 and BLIP2 with FlanT5xl produce the most unsuitable answers, while InstructBLIP methods consistently return valid object lists.Source: §10 · p. 16

11. Visual Examples of MERLIM’s tasks.

Visual examples show InstructBLIP outputs for original and edited images across Object Recognition, Inter-Object Relationship Understanding, and Object Counting. The model produces visually ungrounded predictions across the illustrated scenarios.

11. Visual Examples of MERLIM’s tasks.

  • InstructBLIP outputs are shown for both original and edited images across three evaluated tasks.Source: §11 · p. 16
  • The examples cover Object Recognition, Inter-Object Relationship Understanding, and Object Counting.Source: §11 · p. 16
  • InstructBLIP produces visually ungrounded predictions across the illustrated scenarios, underscoring the complexity of hallucinations in these models.Source: §11 · p. 16

12. Impact Statements

MERLIM assesses IT-LVLM performance and aims to identify and quantify hallucination events. The paper describes this benchmark as potentially encouraging more robust, factual, and informative responses.

12. Impact Statements

  • MERLIM is designed to assess IT-LVLM performance and identify and quantify hallucination events in their textual responses.Source: §12 · p. 16
  • For unsuitable Object Recognition answers without valid nouns, both precision and recall are set to 0.Examples include generic descriptions such as “There are many objects” or “It is a beautiful scene”.Source: Table 6 · p. 17
  • The benchmark’s potential societal impact is encouraging more robust IT-LVLMs and, consequently, more factual and informative responses.Source: §12 · p. 17
  • Additional visual examples compare InstructBLIP outputs on original and edited images for the benchmark’s evaluated tasks.Source: Figure 10 · p. 18

Related papers

Loading 2312.02219v3…