PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring Epistemic Humility in Multimodal Large Language Models

Bingkui Tong, Jiaer Xia, Sifeng Shang, Kaiyang Zhou

arXiv:2509.09658v2cs.CV

TL;DR

Existing MLLM hallucination benchmarks emphasize choosing the correct answer, leaving whether models reject all unsupported options less directly assessed. HumbleBench tests that behavior with scene-graph-based multiple-choice questions containing “None of the above”; results show leading models remain imperfect and accuracy can miss rejection failures.

  • Problem

    Recognition-focused evaluations do not directly test whether an MLLM rejects all presented answers when none is supported by the image.

    Source: §1 · p. 2

  • Method

    HumbleBench uses panoptic scene-graph annotations, candidate attribute cues, and generated, manually filtered multiple-choice questions with a “None of the above” option.

    Source: §1 · pp. 2–3, §2.1 · p. 3

  • Results

    Across tested models, leading systems remain imperfect and stress tests show that accuracy alone can hide false-option rejection failures; Gemini-2.5-Pro reaches 75.82% accuracy.

    Source: §1 · p. 2, §5 · p. 11

  • Takeaways & Limitations

    Recognition accuracy alone is insufficient to assess false-option rejection, a specific abstention-oriented behavior relevant to multimodal reasoning.

    Source: §5 · p. 11

    Model size is confounded with architecture, training data, visual encoders, and post-training strategies, so the comparisons do not establish scaling effects.

    Source: §3.1 · pp. 7–8

Abstract

from arXiv · show

Hallucinations in multimodal large language models (MLLMs) -- where the model generates content inconsistent with the input image -- pose significant risks in real-world applications, from misinformation in visual question answering to unsafe errors in decision-making. Existing benchmarks primarily test recognition accuracy, i.e., evaluating whether models can select the correct answer among distractors. This overlooks another important capability for trustworthy AI: recognizing when none of the provided options is supported by the image and abstaining from committing to a false choice, a humility-related behavior. We present HumbleBench, a new hallucination benchmark designed to evaluate false-option rejection in MLLMs under a forced-choice multiple-choice setting with a ``None of the above'' option. Built from a panoptic scene graph dataset, we leverage fine-grained scene graph annotations for objects and relations, use candidate attribute cues, and prompt GPT-4-Turbo to generate multiple-choice questions, followed by a rigorous manual filtering process. Each question includes a ``None of the above'' option, requiring models not only to recognize correct visual information but also to identify when no provided answer is valid. We evaluate a variety of state-of-the-art MLLMs -- including general-purpose, specialized reasoning, and proprietary models -- on HumbleBench and report empirical findings for the community. By incorporating explicit false-option rejection, HumbleBench fills a key gap in current evaluation suites by assessing a narrower but important abstention-oriented behavior that is relevant to trustworthy multimodal reasoning. Our code and dataset are released publicly and can be accessed at \href{https://github.com/maifoundations/HumbleBench}{https://github.com/maifoundations/HumbleBench}.

1 Introduction

HumbleBench evaluates whether multimodal models reject all answer options when none is supported by an image, a specific abstention-oriented behavior related to epistemic humility. Its results show that leading models remain imperfect and that accuracy alone can hide false-option rejection failures.

1 Introduction

  • HumbleBench operationalizes epistemic humility as rejecting every provided option when none is supported by the image, not as a complete measure of humility.The paper distinguishes this observable behavior from calibrated uncertainty, confidence calibration, and principled refusal in their full generality.Source: §1 · p. 2
  • HumbleBench adds a “None of the above” option to forced-choice questions to test false-option rejection, complementing recognition-focused evaluations.The benchmark is built from a panoptic scene graph dataset with fine-grained object and relation annotations.Source: §1 · p. 2
  • HumbleBench contains 22,831 manually filtered questions that require models to reject incorrect options when none is supported by the image.It uses scene-graph object and relation annotations, candidate attribute cues, and GPT-4-Turbo-generated questions and distractors.Source: §1 · pp. 2–3
  • 75.82% accuracy for Gemini-2.5-Pro and 73.52% for the strongest open-source model leave room below saturation, while accuracy alone can hide false-option failures.The experiments also examine option shuffling, cautious prompting, noise-image, forced-grounding, and NOTA-only stress settings.Source: §1 · pp. 2–3

2 HumbleBench

HumbleBench comprises five-choice questions about objects, relations, and attributes, with “None of the above” as an option. Its construction combines panoptic scene-graph annotations, generated candidate cues and questions, and manual verification.

2 HumbleBench

  • The 22,831-question benchmark covers object, relation, and attribute hallucinations, with five options per question and “None of the above” included.Non-NOTA choices and the three question types are fairly balanced, with a slight dominance of attribute questions.Source: §2 · p. 3

2.1 Construction Pipeline

  • The benchmark starts from 4,500 randomly sampled images from the Panoptic Scene Graph dataset, chosen for its pixel-level object and relationship annotations.The dataset contains over 40,000 images, averaging 11.04 objects and 5.65 relations per image.Source: §2.1 · p. 3
  • Because Panoptic Scene Graph lacks explicit attribute labels, InstructBLIP describes cropped objects to generate candidate attribute cues, which annotators verify against the image.Descriptions serve only as intermediate cues; visually unconfirmable or description-dependent answers are corrected or removed.Source: §2.1 · p. 3
  • GPT-4-Turbo generates ten five-option questions per image with plausible distractors, then manual review modifies 10.82% and removes 45.44%, leaving 22,831 questions.Questions are checked for image-based answerability, a single correct answer, and plausible but incorrect distractors; uncertain cases are removed.Source: §2.1 · pp. 3–4

3 Experiments

The experiments evaluate a diverse set of multimodal models on HumbleBench’s original setting and stress tests. Results show that high accuracy can coexist with weak false-option rejection, while performance varies across models and test conditions.

3 Experiments

  • The evaluation spans general-purpose and reasoning open-source MLLMs plus Gemini-2.5-Pro, using six open-source settings that probe option position, prompting, image noise, grounding, and NOTA-only choice.Metrics include overall accuracy and NOTA-aware measures; the NOTA label is identified from the option text rather than a fixed letter.Source: §3 · pp. 5–6, Table 3 · p. 7

3.1 Results on HumbleBench

  • 75.82% overall accuracy and 73.75% humility score make Gemini-2.5-Pro the original-setting leader, while Qwen2.5-VL pairs 78.98% non-NOTA accuracy with a 32.33% NOTA hit rate.These results illustrate that strong overall or non-NOTA accuracy does not necessarily mean reliable false-option rejection.Source: §3.1 · p. 6, Table 4 · p. 7
  • 4B Visionary-R1 outperforms several larger reasoning models and 5B Phi-4 surpasses 12B Pixtral, so these comparisons do not isolate scaling effects.Model size is entangled with architecture, training data, visual encoders, and post-training strategies.Source: §3.1 · pp. 7–8
  • Reasoning fine-tuning has mixed outcomes: R1-Onevision falls below Qwen2.5-VL (67.06% vs. 72.20%), whereas LLaVA-CoT exceeds LLaMA-3.2 (67.70% vs. 65.31%).The authors attribute the contrasting outcomes to dependence on training strategy and data characteristics.Source: §3.1 · p. 8

3.2 Robustness and Stress-Test Experiments

  • Cautious prompting trades average accuracy (64.67% to 60.63%) for higher average humility (60.25% to 63.14%), with GLM-4.1V-Thinking reaching 73.50%.The prompt shifts models toward more conservative NOTA behavior, at a recognition-accuracy cost.Source: §3.2 · p. 8, Table 5 · p. 9
  • 27.58% average accuracy under noise and 22.53% with forced grounding contrast with 26.61% NOTA-only performance; Cambrian leads NOTA-only at 60.67% hit rate.The authors interpret accuracy under noise as residual language-prior matching, not true visual reasoning.Source: §3.2 · p. 8, Table 5 · p. 9, Table 6 · p. 9
  • 64.67% to 62.62% average accuracy follows NOTA shuffling, but DeepSeek-VL2 drops 61.97% to 36.02% while Ovis-2 rises 65.42% to 69.34%.The average change masks substantial model-to-model variation in robustness to option position.Source: §3.2 · p. 8, Table 6 · p. 9

3.3 Error Analysis

  • Qwen2.5-VL hallucinates unshown objects or relations instead of choosing NOTA, miscounts a poster as a television, and fabricates a sky color for a noise image.The illustrated errors include mistaking bread for a sandwich and inferring that a woman is sitting on chairs.Source: §3.3 · pp. 8–10, Figure 4 · p. 10

4 Related Work

Prior MLLM hallucination benchmarks use generative or discriminative formats, while HumbleBench adds a forced-choice reject option to measure false-option rejection as a narrower abstention behavior.

4 Related Work

  • Generative benchmarks such as THRONE and HaloQuest use open-ended questions but typically rely on external LLM judges, raising reproducibility concerns and possible evaluator bias.Discriminative benchmarks are described as more objective and easier to use.Source: §4 · p. 10
  • Discriminative benchmarks include binary judgment tasks such as POPE [43] and multiple-choice tasks such as LongHalQA [48] and Reefknot [42].Earlier efforts covered only one or two hallucination types, while later datasets included more types.Source: §4 · p. 10
  • HumbleBench complements existing visual hallucination benchmarks with forced-choice questions and a reject option that measures false-option rejection.The comparison table characterizes it as large-scale and explicitly including “None of the above.”Source: §4 · p. 11, Table 7 · p. 11
  • MME-RealWorld [49] and Idk study related forms of abstention, while HumbleBench focuses on rejecting plausible but visually unsupported object, attribute, or relation choices.Its controlled setting uses multiple-choice questions to assess visual hallucination behavior.Source: §4 · p. 11

5 Conclusion

The conclusion presents HumbleBench as a complementary benchmark for abstention-oriented behavior and finds substantial weaknesses in false-option rejection, robustness, and visual grounding across evaluated models.

5 Conclusion

  • HumbleBench offers a complementary way to evaluate abstention-oriented behavior in hallucination-prone multimodal reasoning scenarios.The benchmark evaluates visual hallucinations through false-option rejection.Source: §5 · p. 11
  • Many models perform close to random guessing when only the NOTA option remains, while shuffled-NOTA robustness varies substantially across models.The conclusion treats these results as evidence that recognition accuracy alone is insufficient for assessing hallucination robustness.Source: §5 · p. 11
  • Cautious prompting trades higher humility for lower recognition accuracy, and noise-image tests expose sharp drops when visual evidence is destroyed.The conclusion reports that some models with reasonable HumbleBench performance fail sharply under noise-image settings.Source: §5 · p. 11
  • The data and code supporting the study are publicly available through maifoundations/HumbleBench.The availability statement identifies both HumbleBench data and code as public.Source: §5 · p. 11

A.1 Prompt for Automatic Question Generation

The appendix describes GPT-4-Turbo question generation from structured image annotations and candidate attributes, with constraints intended to preserve validity; it also introduces noise images for visual-faithfulness testing.

A.1 Prompt for Automatic Question Generation

  • The question-generation prompt supplies structured image information, including annotated objects, relations, and candidate attributes, and requests machine-readable JSON.The JSON output is designed for direct parsing into the database.Source: §A.1 · pp. 11–12
  • Generation constraints require one hallucination type per question, a mandatory “None of the above” choice, and plausible but incorrect distractors.The prompt also forbids annotation artifacts and requires questions to be unanswerable without the image, followed by manual verification.Source: §A.1 · p. 12
  • The noise-image test substitutes a programmatically generated static noise image for every original benchmark image to test visual faithfulness with non-informative input.The image contains no recognizable visual patterns.Source: §A.1 · p. 12

A.2 Gaussian Noise Image Generation

The appendix specifies a grayscale noise image made from uniformly sampled pixel intensities, intended to contain no recognizable visual patterns.

A.2 Gaussian Noise Image Generation

  • Each generated noise image is 256×256 pixels and grayscale.The procedure is formally described in Algorithm 1.Source: §A.2 · p. 12
  • Each pixel intensity is drawn from a discrete uniform distribution over integers [0, 255].This specifies the image’s pixel-value range and sampling procedure.Source: §A.2 · p. 12
  • The generated image is intended to contain no recognizable visual patterns.It serves as the non-informative image used in the noise-image experiment.Source: §A.2 · p. 12

A.3 Prompts for Cautious and Grounding Settings

Both settings prepend instructions to the original multiple-choice question. The cautious prompt favors “None of the above” when visual evidence is unclear, while forced grounding requires answers to rely only on the image.

A.3 Prompts for Cautious and Grounding Settings

  • The cautious prompt says to prefer “None of the above” over unsupported choices and to avoid guessing when the image is uncertain or ambiguous.It also warns that plausible options are not necessarily correct.Source: §A.3 · p. 12
  • The forced-grounding prompt excludes world knowledge, commonsense expectations, and linguistic priors, requiring choices to be directly supported by the image.Source: §A.3 · p. 12

A.4 Models for Data Generation

The data-generation pipeline separates attribute enrichment from question synthesis. InstructBLIP generates candidate object attributes, and GPT-4-Turbo uses structured prompts to produce questions and distractors.

A.4 Models for Data Generation

  • InstructBLIP’s Vicuna-7B variant generates candidate attribute descriptions for objects as cues for question generation, which are later manually verified.Its Querying Transformer extracts visual features relevant to a textual prompt.Source: §A.4 · p. 12
  • GPT-4-Turbo synthesizes questions and plausible, context-aware distractors from structured prompts, using JSON output for machine-readable results.The paper specifies the gpt-4-turbo-2024-02-15-preview version.Source: §A.4 · p. 12

A.5 Computational and Annotation Cost Indicators

The section reports construction and evaluation workload, a human check of the NOTA-only stress subset, manual quality-control rules, and the filtering interface used to curate the benchmark.

A.5 Computational and Annotation Cost Indicators

  • Construction sampled 4,500 PSG images, targeting 10 candidate questions per request and approximately 49.7k InstructBLIP attribute-generation calls.Source: §A.5 · pp. 12–13
  • 41,843 candidate questions were manually reviewed; annotators modified 4,527 and removed 19,012, leaving 22,831 benchmark questions.Source: §A.5 · p. 13
  • Across 121 model-setting runs, evaluation covered approximately 2.76M image-question inference calls; metric computation requires no logits or additional inference.Source: §A.5 · p. 14

A.7 Manual Filtering Guidelines and Quality Control

  • Filtering checks image-answerability, a single correct option, valid NOTA answers, plausible distractors, and visual verification of attributes and relations; residual annotation noise may remain.Source: §A.7 · pp. 14–15

A.6 Human Sanity Check for the NOTA-only Stress Test

  • 92.00% human accuracy and NOTA hit rate on the stress subset exceeded the 26.61% model-average accuracy in the NOTA-only setting.The authors say this suggests low model scores are not explained solely by question ambiguity.Source: §A.6 · p. 14, Table A1 · p. 15

A.8 Custom Filtering Software

  • The PyQt5 interface presents each image-question pair and its five options, supports Keep, Modify, or Delete decisions, and advances through the candidate set.The workflow supported curation of nearly 42,000 candidates into the final benchmark.Source: §A.8 · p. 15
Loading 2509.09658v2…