Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly

1. Introduction

• MMVU pairs positive questions about verifiable image content with negative questions containing hypothetical changes, irrelevant descriptions, or incorrect assumptions.It uses multiple-choice questions with explanatory distractors and manually labels samples across 12 question types.Source: §1 · p. 2
• MLLMs often respond unsatisfactorily to negative questions; the authors attribute this to positive-question bias in training data and models’ lower attention to visual tokens.Question-to-visual attention and answer confidence are also lower for negative than positive questions.Source: §1 · p. 2

2. Related Works

• Prior hallucination benchmarks evaluate response discrepancies with visual content, while MMVU also distinguishes visual misunderstanding from fragility to negative questions.The paper frames this as a gap in how existing benchmarks interpret incorrect answers.Source: §2 · p. 3

3.1. An Overview of MLLMs

• MLLMs tokenize the system prompt and question, encode the image as visual tokens, combine the modalities through a connector, and pass the concatenated tokens to the LLM.The described architecture comprises a visual encoder, connector, and LLM.Source: §3.1 · p. 3

3.2. MMVU Benchmark Design

• MMVU manually constructs paired positive and negative multiple-choice questions at character, attribute, and context levels to test comprehension and resistance to interference.Its 1,786 questions comprise 893 positive and 893 negative samples.Source: §3.2 · p. 4

3.3. Evaluation metrics

• The evaluation metrics classify responses by visual understanding and negative-question correctness, then quantify misleading-question errors and robustness.The four outcome groups are Understanding and Robustness, Understanding but Fragility, Not Understanding and Rigorous, and Not Understanding and Fragility.Source: §3.3 · p. 4

3.4. Analysis Methods

• The analysis measures answer-token attention to system, image, and question tokens, and compares confidence in correct answers under positive and negative question forms.Attention is computed from the final-layer matrix; answer-option logits are converted to probabilities with softmax.Source: §3.4 · p. 6

3.5. MMVU Training Set Construction Pipeline

• The training pipeline extracts detailed image information and uses it to create positive samples and directly contradictory negative counterparts for instruction tuning.It includes text or numbers, objects, attributes, relationships, activities, and overall image context.Source: §3.5 · pp. 6–7

4.1. Analysis of MMVU Benchmark

• GPT-4o led closed-source models and Ovis1.6-Gemma2-9B led open-source models with 66.74% RA, yet both groups remained fragile on negative questions.GPT-4o had 19.53% MR, compared with Ovis1.6-Gemma2-9B’s 18.60% MR.Source: §4.1 · p. 7

4.2. Analysis of attention and logits distribution

• Answer tokens prioritize system tokens, then question tokens, and lastly visual tokens, suggesting potential visual-information under-utilization during decoding.Source: §4.2 · p. 8
• For identical images, question-to-visual attention and confidence in correct answers are lower for negative than positive questions.The analysis also examines correct-option probabilities when models answer positive questions correctly but struggle with negative ones.Source: §4.2 · p. 8

4.3. Analysis of MMVU Training Set

• Fine-tuning with MMVU-Train consistently outperformed baselines across most metrics; LLaVA-OneVision-7B improved character/number recognition from 62.20% to 68.29%.It also improved numerical understanding from 41.98% to 55.56% and activity recognition from 75.56% to 80.00%.Source: §4.3 · p. 8
• MMVU-Train also outperformed models trained with earlier GPT-4V-generated instruction-tuning datasets.Source: §4.3 · p. 8

4.4. Analysis of Understanding-based Prompting

• CGR alone improved positive-question accuracy more than VAR alone, while VAR significantly improved correctness on responses dependent on visual understanding.Combining both improved accuracy after misleading questions but did not reproduce the prior gains in question comprehension; the authors attribute this to their different focus priorities.Source: §4.4 · p. 8

• MMLVs can provide incorrect answers despite accurately understanding visual content, a limitation examined with the MMVU benchmark.Source: §5 · p. 8
• The paper analyzes attention mechanisms and decoding logits, identifying greater attention to system and question tokens than to visual tokens, particularly on negative questions.Source: §5 · p. 8
• The proposed paired positive-negative training data and two inference strategies significantly improve robustness in responding to negative questions.Source: §5 · p. 8

Abstract

• The supplement outlines test-set construction criteria, paired positive and negative examples, and analyses of training-data combinations, prompt versions, and dataset sizes.Source: Abstract · p. 13

6. MMVU Test set Details

• The MMVU test set is manually designed, whereas MMVU-Train is automatically constructed.Source: §6 · p. 14

Character-Level

• The described test categories cover image-grounded character and number recognition, position, presence, color or texture, counting, shape, posture, knowledge, activity, and relationships.Source: Character-Level · p. 14

6.2. Preliminary Evaluation Details.

• Preliminary evaluation found lower accuracy on negative than positive questions, although many models exceeded 70% accuracy on positive-only questions.The authors interpret the positive-question performance as evidence that models can process image content, while negative-question results indicate difficulty with leading questions.Source: §6.2 · p. 14

• Without images, model performance on MMVU falls below random selection, indicating that the benchmark requires multimodal input rather than text-only reasoning.Source: §7 · p. 14
• The reported comparison evaluates models with images as input on the MMVU test set.Source: Table 5 · p. 15
• The benchmark’s reported categories include character/number recognition, presence, color/texture, number, shape, posture, position, knowledge, expertise, activity, and relationships.Source: Table 5 · p. 15

8.1. Notes

• The preparatory test set was a subset rather than the full MMVU test set; its category distribution closely resembled the final set, with category counts scaled.The authors state that observed phenomena were consistent with those on the full test set.Source: §8.1 · pp. 15–16

8.2. Preliminary Testing Data Distribution.

• The preparatory subset covered context, character, and attribute questions, with detailed subcategories across relationships, text or numbers, and visual attributes.It contained 120 context, 40 character, and 140 attribute questions.Source: §8.2 · p. 16

8.3. Training Details

• The training details specify a two-stage strategy with hyperparameters in Table 7 and greedy search for testing.The supplied passage does not give the hyperparameter values.Source: §8.3 · p. 16, Table 7 · p. 17

8.4. Discussion 1: Can We Simply Introduce Negative Samples to Train MLLMs?

• The initial experiment generated negative samples with GPT-4V from 7k and 24k samples drawn from LLaVA data.The motivation was the observation that MLLMs could understand image content yet struggle to answer questions.Source: §8.4 · pp. 16–17

8.4.1. Training and Results of the Version 0.

• Combining sampled data with generated negative samples consistently improved performance across most benchmarks, while concatenating them reduced performance on the generic benchmark.Training on sampled data alone yielded the lowest overall performance; the combined strategy was most effective.Source: §8.4.1 · p. 17
• Using fewer concatenated conversation rounds caused no significant performance degradation, though a minor drop was observed and hypothesized to reflect greater within-sample variability.The ablation focused on the 24k dataset.Source: §8.4.1 · p. 17

8.4.2. Training and Results of the Version 1.

• Generated-only training was effective only when positive and negative pairs were both present; negative-only training restricted the model from producing negative answers.Rigidly specifying question-type counts could also produce nonsensical pairs and degrade performance.Source: §8.4.2 · p. 17

8.5. Discussion 2: What Constitutes an Effective Pair of Positive and Negative Samples for Training?

• Prompt version 3’s paired positive-negative samples yielded the strongest overall performance and became the preferred method for generating training data.The experiments used 24k pairs generated with each prompt version.Source: §8.5 · pp. 18–19

9.1. Framework.

• The model uses SigLIP as its visual encoder, Phi-2 (2.7B) as its language model, and a two-layer MLP connector, based on Bunny.Visual and textual features are concatenated and fed to the language model; training loss is calculated on the response, not the masked instruction.Source: §9.1 · p. 19, §9.2 · p. 19

9.2. Training Scheme.

• Training first aligns the visual and language components by training only the connector, then fine-tunes the connector and language model on MMVU-Train with additional OCR, VQA, and text-only data.The additional data include OCR-VQA, VQA-v2, GQA, OK-VQA, A-OKVQA, Visual Genome, RefCOCO, and WizardLM.Source: §9.2 · pp. 19–20

9.3. Implementation Details.

• The alignment stage trains the connector for one epoch at 1e-5; fine-tuning uses LoRA at 2e-4 on eight Nvidia A100 GPUs.The implementation is in PyTorch.Source: §9.3 · p. 20

9.4. Evaluation Benchmark.

• Evaluation uses MMVU scores, including average RA and MR, alongside general benchmarks spanning perception, cognition, multimodal reasoning, and visual question answering.The listed general benchmarks include MME, MMBench, SEED-Bench-1, MMMU, VQAv2, GQA, and POPE.Source: §9.4 · pp. 20–21

9.5. Expriment Results

• Experiments show that the 3-billion-parameter model trained on this enriched dataset outperforms LLaVA-v1.5-7B on most benchmarks and remarkably exceeds the performance of some 13-billion-parameter models on several benchmarks.Source: §9.5 · p. 21
• Replacing original LLaVA data with MMVU data yielded more substantial enhancements than adding new data, and performance was optimal when the replacement proportion closely matched the original distribution.Adding new data did not consistently improve performance.Source: §9.5 · p. 22

• A positive object-presence question is paired with a negative question that substitutes a visually similar nonexistent object.The prompt labels the questions ques-pos and ques-neg and their correct answers ans-pos and ans-neg.Source: §10 · p. 25
• The five question types form positive-negative pairs by altering object presence, attributes, number or position, knowledge, or activity and relationships.The negative question substitutes a similar but incorrect object detail or proposition.Source: §10 · p. 26
• The supplement shows generated conversation examples for prompt versions 0, 1, 2, and 3, with version 3 split across two figures.Source: Figure 16 · p. 29, Figure 17 · p. 30, Figure 18 · p. 31
• One example asks whether two people are in a kitchen, while a paired misleading question asks whether there are three; its answer says only one person is present.Another example asks whether a bowl of food is on a counter.Source: §10 · pp. 31–33

12.1. Prompt for extraction visual information

• The visual-information extraction prompt is shown in Figure 22.Source: §12.1 · p. 34

12.2. Examples of Content Guided Refinement Strategy Results

• Examples of Content Guided Refinement Strategy results are shown in Figure 23.Source: §12.2 · p. 34

13.1. Detailed Description

• Lower attention between question and visual tokens is associated with a lower probability of correct answers; refinement inverts, filters, and blends an attention map with the image.The blend uses 0.85 for the original image and 0.15 for the attention map.Source: §13.1 · p. 34

13.2. Examples of visual attention refinement strategy

• Figure 24 provides an example of the visual-attention refinement strategy.Source: §13.2 · p. 34

• The authors state that the research will benefit the general development and applications of MLLMs.Source: §14 · p. 34
• 112K instruction-tuning examples are the cost-constrained contribution; the authors say such adversarial datasets can scale with more quota.Source: §14 · p. 34
• Question–visual attention is used to invert, regularize, and filter an attention map, then blend it with the original image to form an inference prompt.The blend uses coefficients of 0.85 for the original image and 0.15 for the attention map.Source: §14 · p. 34
• The image-information extraction prompt structures details about text, objects, people, relationships, activities, and events before answering.Source: §14 · p. 35