PaperlayerView exact revision on arXiv

Source-linked AI summary

MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models

Wenjie Zhu, Yabin Zhang, Wenjun Zeng, Lei Zhang

arXiv:2607.27637v2cs.CV

TL;DR

MLLMs need to refuse truly unanswerable visual questions without losing the ability to answer when context shifts remain answerable. MMOOC benchmarks this balance across diverse shifted-context cases, showing that current models still struggle with it despite stronger general capability or larger size.

  • Problem

    Existing benchmarks largely omit answerable shifted in-context cases and cover limited out-of-context shifts, hindering comprehensive evaluation of MLLM reliability under imperfect visual contexts.

  • Method

    MMOOC benchmarks refusal and robust answering across over 41K image-question pairs spanning three formats, eight shift types, and six visual scenarios.

  • Results

    Current MLLMs still struggle to balance answer preservation and refusal, while larger model size does not necessarily improve OOC robustness.

  • Takeaways & Limitations

    MMOOC provides a benchmark for assessing whether MLLMs can remain answerable under shifted contexts while refusing genuinely out-of-context questions.

  • Takeaways & Limitations

    MMOOC currently focuses on image–text interactions, leaving video, audio, and embodied environments for future work.

Abstract

from arXiv · show

Multimodal Large Language Models (MLLMs) have achieved strong performance on a wide range of vision-language tasks, but often fail under imperfect or shifted contexts. A reliable MLLM should refuse truly out-of-context (OOC) questions with subject-level context shifts while still answering shifted in-context (Shifted IC) questions with non-subject context shifts. Existing benchmarks mainly target OOC or visually unanswerable questions, but overlook answerable Shifted IC cases and cover limited OOC shifts. To fill this gap, we present MMOOC, a large-scale benchmark for evaluating refusal and robust answering abilities of MLLMs. MMOOC contains over 41K image-question pairs, including answerable Shifted IC cases and unanswerable OOC cases, spanning three question formats, eight shift types and six visual scenarios, with data quality ensured through MLLM-based filtering and human verification. We evaluate model responses using Accuracy and Refusal Rate, and further introduce an LLM-as-a-Judge metric to assess the correctness of model reasoning. Experiments on diverse MLLMs show that current models still struggle to balance answer-ability and refusal under shifted contexts. We further analyze key failure patterns and show that post-training can improve robustness. MMOOC will be made publicly available.

1 Introduction

MLLMs must refuse truly out-of-context questions without losing the ability to answer questions that remain answerable under shifted contexts. MMOOC addresses this gap with a comprehensive benchmark spanning diverse question formats, shift types, visual scenarios, and over 41K image-question pairs, while experiments show current models still struggle to balance answerability and refusal.

  • Motivation: Reliable MLLMs should refuse truly OOC questions while preserving answers to queries that remain answerable despite distracting OOC cues.Continuing to answer unsupported queries can produce hallucinated or misleading responses, whereas excessive refusal undermines usability.
  • Motivation: Existing benchmarks incompletely assess this problem by emphasizing unsupported answers or covering limited OOC settings, question formats, and visual scenarios.CLIP-UP evaluates multiple question types but not diverse OOC settings under shifted in-context conditions.
  • Benchmark: MMOOC evaluates both truly unanswerable OOC cases and answerable shifted IC cases across three question formats and six visual scenarios.The benchmark distinguishes five OOC categories from three shifted IC categories involving non-subject context shifts.
  • Benchmark: Over 41K image-question pairs span three question formats, eight shift types, and six visual scenarios for comprehensive evaluation of refusal and robust answering.The formats are yes/no, multiple-choice, and open-ended VQA.
  • Findings: Systematic experiments show current MLLMs still struggle to balance answerability and refusal under shifted contexts, while alignment strategies and prompt engineering can improve OOC robustness.The study also analyzes failure patterns across diverse settings.

2 Related Work

Related work examines trustworthy and reliable behavior in MLLMs through hallucination, faithfulness, honesty, robustness, and refusal. Existing evaluations largely emphasize hallucination avoidance or refusal on unanswerable questions, while alignment methods aim to improve helpfulness, safety, reliability, and reasoning.

  • Trustworthiness and Reliability in MLLMs: Trustworthiness research in LLMs and MLLMs covers hallucination, faithfulness, honesty, robustness, and abstention or refusal.These directions study whether responses are grounded in evidence, honest, robust, or appropriately withheld.
  • Trustworthiness and Reliability in MLLMs: Existing evaluations mainly target hallucination avoidance or refusal on unanswerable questions, overlooking shifted contexts where sufficient evidence remains.This gap motivates evaluating answerable cases with contextual shifts alongside genuinely out-of-context questions.
  • MLLM Alignment: Post-training alignment improves MLLM helpfulness, safety, and reliability through instruction tuning, RLHF, preference optimization, and reasoning-oriented reinforcement learning.Instruction tuning aligns models with human instructions, RLHF uses preference signals, preference optimization provides simpler alternatives, and reasoning-oriented reinforcement learning targets stronger reasoning.

3 Method

MMOOC builds a hierarchical taxonomy separating answerable Shifted IC from unanswerable OOC cases, then generates and verifies diverse samples through automated agreement and human review. It evaluates models with answerability, refusal, and rationality measures using multi-judge assessment.

  • Hierarchical Taxonomy Design: MMOOC first distinguishes Shifted IC samples supported by image–question evidence from OOC samples, then assigns each sample one primary contextual-failure category.The taxonomy contains five OOC categories and three answerable Shifted IC categories.
  • Category Definitions: The taxonomy defines five OOC categories—MA, VFP, USPC, ULS, and MKB—and three Shifted IC categories: MP, PA, and IQM.OOC cases require refusal, whereas Shifted IC cases retain a recoverable visual query despite misleading or partially unsupported context.
  • Data Generation: MMOOC combines ICL generation by Qwen3.5-122B-A10B, GPT-4o, and o1 with manually designed questions and Auto Shuffle derivations from MME, MMStar, and OK-VQA.The pipelines target diverse language, reasoning patterns, and realistic mismatched or insufficient contextual settings.
  • Quality Verification: GPT-4o, o1, and o3 independently assess answerability, retaining samples only when all three agree, after which annotators verify correctness, consistency, clarity, and naturalness.Minor errors are corrected, invalid samples removed, and disagreements resolved by an additional annotator.
  • Evaluation Metrics: MMOOC evaluates IC responses with Accuracy and Answer Rationality, and OOC responses with Refusal Rate and Refusal Rationality.Three independent judges—GPT-5.6, Claude Opus 5, and DeepSeek-V4-Pro—score correctness, relevance, and evidence-grounded reasoning, with rationality scores averaged across judges.

4 Experiments

Experiments on 18 representative MLLMs show that OOC refusal remains low, inconsistent, and sensitive to model family, scale, question format, and OOC type. Post-training improves refusal and reasoning rationality, especially with SFT, but can reduce MMStar performance.

  • Evaluation Setup: 18 representative MLLMs, comprising 13 open-source and 5 closed-source models, are evaluated for OOC capabilities.The open-source set includes Qwen, LLaVA, InternVL, Gemma, Llama, and Ministral models; the closed-source set includes GPT-4o, o1, o3, Gemini-3.1-Pro, and Claude-Opus-4.6.
  • OOC Performance: OOC performance is low and inconsistent across formats, with Qwen3-VL-2B scoring 5.75 on USPC-YesNo and 8.25 on USPC-VQA.These results indicate that robust refusal remains challenging, particularly on USPC and ULS.
  • OOC Performance: Larger models do not consistently achieve better OOC performance: Qwen3-VL-30B outperforms Qwen3-VL-2B, while Qwen3.5-122B-A10B does not consistently surpass smaller Qwen3-VL variants.The Gemma-4 family achieves the strongest open-source performance, although some larger models remain weaker on several OOC categories.
  • OOC Performance: Among proprietary models, o1 achieves the strongest overall OOC performance, ranking first in YesNo and VQA while remaining competitive in MCQ.Gemini-3.1-Pro and Claude-Opus-4.6 nevertheless show weaknesses on several OOC categories.
  • OOC Performance: OOC results vary substantially by format and category: VQA is often higher than YesNo and MCQ, while USPC and ULS are among the most difficult settings.YesNo is especially challenging because it requires a hard binary decision, whereas Visual False Premises is comparatively easier in VQA.
  • Improving OOC via Alignment: SFT consistently increases Rref and Rrat across all three evaluated models, while DPO improves over the vanilla baseline but is generally less effective than SFT.These OOC gains often coincide with a drop on MMStar, indicating a trade-off between refusal alignment and general multimodal capability.

5 Conclusion

MMOOC addresses the reliability challenge of preserving correct answers for answerable shifted-context questions while refusing truly out-of-context queries. Its benchmark and experiments show that current MLLMs struggle to balance answer preservation and refusal, and that general capability or model size does not necessarily improve OOC robustness.

  • Conclusion: Robust MLLMs should refuse truly out-of-context questions while preserving correct answers when distracting OOC cues do not make queries unanswerable.This reliability challenge motivates the MMOOC benchmark.
  • Conclusion: MMOOC covers three question formats, three shifted in-context categories, five out-of-context categories, six visual scenarios, and over 41K image-question pairs.The benchmark was introduced to investigate shifted-context reliability in depth.
  • Conclusion: 18 representative open-source and proprietary MLLMs still struggle to balance answer preservation and refusal under shifted contexts.The experiments systematically evaluated models across these shifted-context conditions.
  • Conclusion: Stronger general capability or larger model size does not necessarily imply better out-of-context robustness.This finding indicates that broad capability and scale alone are insufficient for reliable OOC behavior.

Appendix · A Appendix · A.1 More Related Work

The paper situates MLLM evaluation within a progression from single-task benchmarks toward comprehensive multimodal benchmarks that assess broader perception and reasoning capabilities.

  • A.1 More Related Work: The research community has developed many multimodal benchmarks to probe emerging MLLM capabilities across diverse evaluation axes.These benchmarks reflect expanding efforts to evaluate multimodal systems comprehensively.
  • A.1 More Related Work: Early single-task benchmarks include MS-COCO, VQA, GQA, and OK-VQA.These benchmarks represent task-specific approaches to multimodal evaluation.
  • A.1 More Related Work: MS-COCO, VQA, GQA, and OK-VQA are insufficient for holistically assessing modern MLLMs.The passage identifies a limitation in using early single-task benchmarks to evaluate current systems broadly.
  • A.1 More Related Work: The limitation concerns holistic assessment of multimodal perception and reasoning abilities.The passage specifically identifies perception and reasoning as the abilities not fully captured by early benchmarks.
  • A.1 More Related Work: Comprehensive multimodal benchmarks were proposed to address the limitations of early single-task evaluations.The passage presents comprehensive benchmarks as a response to the insufficient scope of earlier benchmarks.
  • A.1 More Related Work: Examples of comprehensive multimodal benchmarks include MME, MMBench, and MMMU.These benchmarks are cited as representative comprehensive evaluations of MLLMs.

A.2 Benchmark Construction · A.2.1 Shifted In-Context Category Definition:

The benchmark defines shifted in-context questions as visually answerable despite misleading, partial, or mismatched context. It operationalizes these cases across multiple formats with explicit answers and reasoning about context unreliability and visual answerability.

  • A.2.1 Shifted In-Context Category Definition:: Misleading-premise questions embed false, uncertain, unsupported, or irrelevant premises while keeping the core visual question definitively answerable.This category is generated in Yes/No, multi-choice, and open-ended VQA formats.
  • A.2.1 Shifted In-Context Category Definition:: Yes/No misleading-premise generation produces two questions: one whose correct answer is “yes” and one whose correct answer is “no.”Each answer must follow the visual evidence despite the false premise.
  • A.2.1 Shifted In-Context Category Definition:: Reasoning annotations consistently explain why the premise or context is unreliable and why the requested visual answer remains confidently derivable.This requirement applies across misleading-premise, partial-answerability, and image-question mismatch constructions.
  • A.2.1 Shifted In-Context Category Definition:: Each generated item includes a caption, question, answer, reasoning, and an in-context type label identifying the shift category.The labels include “false_premise_ic,” “misleading_premise_ic,” “partially_answerable_ic,” and “image_text_mismatch_ic.”
  • A.2.1 Shifted In-Context Category Definition:: Multi-choice misleading-premise generation requires exactly four inline options—A, B, C, and D—and records the correct option letter.The premise remains false, uncertain, or irrelevant while the visual core ask remains answerable.
  • A.2.1 Shifted In-Context Category Definition:: Partial-answerability questions combine answerable and unanswerable sub-questions, requiring answers only for the portion directly derivable from the image.Reasoning identifies which part cannot be determined and why the remaining part is confident.
  • A.2.1 Shifted In-Context Category Definition:: Image-question mismatch cases use scene-level or contextual descriptions that disagree with the depicted image while preserving answerable specific visual facts.Generation requires describing what is actually visible and explaining both the mismatch and residual answerability.

A.2.2 Out-of-Context Category Definition:

The section defines two out-of-context categories: Multimodal Ambiguity, where visual or textual information is insufficient or unclear, and Visual False Premises, where the question contradicts the image. Each category is divided into two sub-types that target distinct sources of unanswerability.

  • Multimodal Ambiguity: Multimodal Ambiguity arises when image pixels lack sufficient clarity or the textual query is too vague to identify a specific entity.Its sub-types are Insufficient Visual Evidence and Ambiguous Textual Query.
  • Multimodal Ambiguity: Insufficient Visual Evidence covers blurry, occluded, low-resolution, incomplete, or very small visual content that cannot reliably support an answer.The example concerns text on a sign that is too blurry to read.
  • Multimodal Ambiguity: Ambiguous Textual Query covers vague, underspecified, or referentially unclear questions whose target object, attribute, or relation is uncertain.An example asks about an object next to “it” when multiple objects could be intended.
  • Visual False Premises: Visual False Premises arise when a question assumes an object, property, or state that is not actually shown, making its premise invalid.The category includes False Entity Premise and False Attribute Premise.
  • Visual False Premises: False Entity Premise assumes that an object, person, animal, or event exists in the image when it does not.False Attribute Premise instead concerns an existing entity with an unsupported attribute, action, state, or relation.
Loading 2607.27637v2…