PaperlayerView exact revision on arXiv

Source-linked AI summary

Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness

Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, Jae Sung Park, Jack Hessel, Lijuan Wang, Yejin Choi

arXiv:2407.01942v1cs.AIcs.CLcs.CV

TL;DR

Vision-language models are trained to answer despite uncertainty, and existing benchmarks largely emphasize cases with definitive answers. The paper introduces a taxonomy, a 178K-instance contrastive VQA benchmark, and confidence-weighted accuracy. Fine-tuning with the benchmark improves held-out and refusal evaluations, with gains on some hallucination evaluations.

  • Problem

    Vision-language models show limited awareness of uncertainty, while existing benchmarks mostly focus on questions with clear, definitive answers.

    Source: §1 · p. 1

  • Method

    The paper defines a multimodal uncertainty taxonomy, constructs contrastive VQA instances through image inpainting and caption-based question generation, and introduces confidence-weighted accuracy.

    Source: §1 · p. 2, §2 · p. 3, §5 · p. 9

  • Results

    Fine-tuning with CERTAINLYUNCERTAIN improves its held-out and refusal evaluations and shows gains on some hallucination evaluations, while outcomes vary across other benchmarks.

    Source: §3.3 · pp. 8–9, §5 · p. 9

  • Takeaways & Limitations

    The results indicate that uncertainty-focused fine-tuning can improve benchmark and refusal performance, though results differ across evaluation datasets.

    Source: §3.3 · pp. 8–9

    The synthetic generation pipeline may not capture all nuances of real-world uncertain scenarios.

    Source: A Limitations · p. 16

Abstract

from arXiv · show

The ability to acknowledge the inevitable uncertainty in their knowledge and reasoning is a prerequisite for AI systems to be truly truthful and reliable. In this paper, we present a taxonomy of uncertainty specific to vision-language AI systems, distinguishing between epistemic uncertainty (arising from a lack of information) and aleatoric uncertainty (due to inherent unpredictability), and further explore finer categories within. Based on this taxonomy, we synthesize a benchmark dataset, CertainlyUncertain, featuring 178K visual question answering (VQA) samples as contrastive pairs. This is achieved by 1) inpainting images to make previously answerable questions into unanswerable ones; and 2) using image captions to prompt large language models for both answerable and unanswerable questions. Additionally, we introduce a new metric confidence-weighted accuracy, that is well correlated with both accuracy and calibration error, to address the shortcomings of existing metrics.

1 Introduction

The introduction presents a taxonomy and benchmark for uncertainty in vision-language systems, motivated by models’ limited awareness of uncertainty and benchmarks’ focus on definitive answers. It also introduces confidence-weighted accuracy and reports improvements from fine-tuning on the benchmark.

1 Introduction

  • Existing vision-language models often answer under uncertainty, while common benchmarks largely emphasize cases with clear, definitive answers.The introduction links this benchmark gap to training that rewards predictions regardless of confidence.Source: §1 · p. 1
  • CERTAINLYUNCERTAIN contains approximately 178K VQA instances and uses image inpainting and caption-prompted question generation to create answerable and unanswerable examples.The caption-based method prompts GPT-4 to generate both question types about the same image.Source: §1 · pp. 1–2
  • The taxonomy distinguishes epistemic uncertainty from missing information and aleatoric uncertainty from inherent unpredictability, then identifies finer categories including knowledge, complexity, extraneous context, temporal uncertainty, and ambiguity.The taxonomy is presented as the basis for the dataset.Source: §1 · p. 2, Figure 1 · p. 2
  • Confidence-weighted accuracy incorporates prediction confidence into accuracy and is positively correlated with accuracy and negatively correlated with expected calibration error.The authors propose it because accuracy and F1 omit confidence, while ECE does not effectively reflect correctness.Source: §1 · p. 2
  • Fine-tuning with CERTAINLYUNCERTAIN improves performance on its held-out test set and refusal-based benchmarks, reduces hallucinations on some evaluations, and maintains standard VQA performance.Experiments cover supervised fine-tuning, R-tuning, and preference optimization.Source: §1 · p. 2

2 CERTAINLYUNCERTAIN

CERTAINLYUNCERTAIN organizes multimodal uncertainty into epistemic and aleatoric categories and builds contrastive VQA pairs from image perturbations and captions. It also introduces confidence-weighted accuracy to combine prediction accuracy with confidence.

2.1 Taxonomy of Uncertainty Awareness

  • Epistemic uncertainty arises from missing knowledge or information, and its awareness categories include knowledge, complexity, and extraneous uncertainty.These concern information not shown in the image, difficult or multi-part questions, and irrelevant image elements, respectively.Source: §2.1 · p. 3
  • Aleatoric uncertainty is inherent unpredictability; its awareness categories cover temporal reasoning and ambiguity.Temporal questions concern events not inferable from the image alone, while ambiguity allows multiple interpretations.Source: §2.1 · p. 3

2.2 Dataset Creation

  • The dataset pairs answerable and unanswerable VQA instances, using image perturbations for extraneous questions and caption-prompted generation for other uncertainty categories.Caption sourcing contributes around 110K instances; image sourcing includes VQAv2 and GQA samples.Source: §2.2 · pp. 3–4
  • Because its automated pipeline can produce invalid samples when component models fail, the authors manually checked the extraneous test set and retained 4.8K of 6K examples.About 1.2K samples were filtered.Source: §2.2 · p. 5
  • CERTAINLYUNCERTAIN is larger, covers more question types and uncertainty categories, and uses contextually aligned question-image pairs compared with existing benchmarks.Its pipeline also produces more natural-looking images than the masking or copying used in UNK-VQA.Source: §2.2 · p. 5

2.3 Evaluation Metrics

  • Confidence-weighted accuracy weights LAVEidk accuracy by prediction probability and is positively correlated with LAVEidk accuracy and more negatively correlated with ECE than LAVEidk accuracy.The authors obtain prediction probability by asking the model to verify its answer and normalizing the yes-token probability against yes and no.Source: §2.3 · p. 6

3 Experiments

Experiments compare direct evaluation and three training strategies across refusal, hallucination, and standard VQA benchmarks. Fine-tuning with CERTAINLYUNCERTAIN improves several benchmark outcomes while retaining standard-task performance, and existing VLMs show poor performance and confidence alignment on the dataset.

3.1 Experimental Details

  • Experiments evaluate instruction-tuned LLaVA variants, Qwen-VL, and GPT-4V, comparing base models with supervised fine-tuning, R-tuning, preference optimization, and selective prediction.CERTAINLYUNCERTAIN training is also compared with LLaVA and LRV instruction-tuning data; R-tuning re-annotates base-model errors as IDK, while DPO uses contrastive answer pairs.Source: §3.1 · p. 6

3.3 Results and Discussion

  • Existing GPT-4V, LLaVA, and Qwen-VL models perform poorly; LAVEidk accuracy exceeds confidence-weighted accuracy, suggesting miscalibrated confidence.Extraneous is the hardest epistemic category and ambiguous the hardest aleatoric category; temporal-category performance varies little across model sizes, possibly due to limited question diversity.Source: Table 3 · p. 7, §3.3 · pp. 7–8, Figure 5 · p. 7

3.2 Evaluation Benchmarks

  • Evaluation spans refusal, hallucination, and standard VQA benchmarks, including UNK-VQA and TDIUC, MMHal-Bench, POPE and AMBER, and VQAv2 and VizWiz.The experiments use 5K sampled instances from each refusal benchmark and validation splits for the standard datasets.Source: §3.2 · p. 7

3.3 Results and Discussion

  • SFT with CERTAINLYUNCERTAIN learns IDK better than R-tuning or DPO, raises confidence-weighted accuracy, and reduces ECE.The comparison covers training strategies including LoRA-based continued fine-tuning for Qwen-VL-Chat and data augmentation during LLaVA instruction tuning.Source: Table 4 · p. 8, §3.3 · p. 8
  • Fine-tuning improved refusal benchmarks, showed promising hallucination reductions, and preserved standard-task performance; adding the data to LLaVA training improved MM-Hal and VizWiz.For Qwen-VL-Chat on AMBER, SFT with the tested data combinations produced inferior results, especially with CERTAINLYUNCERTAIN alone.Source: Table 5 · p. 8, §3.3 · pp. 8–9

4 Related Work

Prior work addresses abstention, hallucination detection, and answer scoring, but the paper distinguishes its focus on recognizing epistemic and aleatoric uncertainty. It positions CERTAINLYUNCERTAIN as a benchmark for cases where underconfidence or IDK is appropriate.

Abstention

  • Selective prediction and context-gathering approaches reduce incorrect answers despite definitive answers, whereas this work targets epistemic and aleatoric uncertainty.Related approaches include abstention under domain shift or adversarial inputs, evidence accumulation, question rephrasing, and code probing.Source: §4 · p. 9

Hallucinations

  • Hallucination detection uses token- or sentence-level methods, while this work aims to reduce confidence in hallucinatory answers; retrieval-based mitigation can help on extraneous cases.Retrieval approaches condition outputs on factual data from external sources.Source: §4 · p. 9

Evaluation

  • Existing accuracy and generation metrics do not directly capture prediction confidence, motivating confidence-weighted accuracy built on LAVE accuracy and correlated with ECE.LAVE scores semantic overlap, while ECE evaluates whether probability estimates represent true correctness likelihood.Source: §4 · p. 9

Datasets

  • Most multimodal benchmarks emphasize definitive answers or partial hallucinations, while CERTAINLYUNCERTAIN targets cases where underconfidence or IDK is correct.The paper reports that existing datasets it examines are insufficient for benchmarking or improving multimodal epistemic and aleatoric awareness.Source: §4 · p. 9

5 Conclusions and Future Work

The paper presents a taxonomy and benchmark for multimodal uncertainty, reporting that fine-tuning on its data improves performance on several benchmark types while maintaining standard-task performance. It also proposes confidence-weighted accuracy and identifies extensions for future work.

5 Conclusions and Future Work

  • Fine-tuning with CERTAINLYUNCERTAIN improves results on its held-out set, refusal benchmarks, and some hallucination benchmarks while maintaining standard-benchmark performance.The paper concludes that current VLMs lack self-awareness of epistemic and aleatoric uncertainty.Source: §5 · p. 9
  • Confidence-weighted accuracy combines predictive performance with prediction confidence and shows strong correlation with both accuracy and ECE.The authors suggest exploring the metric on unimodal and multimodal datasets and as a possible training reward.Source: §5 · pp. 9–10
  • Future work could add rationales identifying the uncertainty category behind IDK responses.The conclusion also suggests exploring the metric on other datasets and as a model-training reward.Source: §5 · p. 10

A Limitations

The authors note that synthetic uncertainty data may not capture every real-world nuance and that their most effective training approach is more resource-intensive than selective prediction. They describe potential applications while qualifying adaptation on domain-specific inpainting techniques.

A Limitations

  • Synthetic generation may miss nuances of real-world uncertainty; the most effective approach reported, SFT with LoRA, is more resource-intensive than inference-time selective prediction.The authors also leave providing a reasonable or best guess, rather than only answering or abstaining, for future work.Source: A Limitations · p. 16

B Broader Impact

  • Adapting the data creation pipeline to other scenarios depends on having domain-specific inpainting techniques.The authors describe potential use in resource allocation and evaluating performance across unimodal and multimodal domains.Source: B Broader Impact · p. 16

C Samples visualizing CERTAINLYUNCERTAIN benchmark

The samples illustrate how uncertainty categories contrast answerable and unanswerable cases, including image perturbation for extraneous uncertainty and distinct epistemic and aleatoric examples.

Knowledge awareness

Complexity awareness

  • Complexity-awareness examples contrast tedious unanswerable questions with answerable questions that require some effort, such as counting, but are not laborious.The examples are shown in Figure 7.Source: C Samples visualizing… · p. 16, Figure 7 · p. 16

Extraneous awareness

Aleatoric awareness

D Samples visualizing predictions and confidence-weighted metric

The examples show how confidence-weighted accuracy accounts for both answer correctness and prediction probability, including changes after fine-tuning.

Metric behavior

  • Confidence-weighted accuracy combines prediction probability with answer correctness, scoring correct low-confidence answers less and penalizing incorrect high-confidence answers more.The visualization compares this metric with LAVEidk accuracy and predicted probability.Source: D Samples visualizing predictions and… · p. 18
  • For incorrect answers, the metric assigns a negative score but penalizes low-confidence errors less than high-confidence errors.The examples contrast cases (c) and (d).Source: D Samples visualizing predictions and… · p. 18

Fine-tuning examples

  • After fine-tuning, corrected predictions with higher probabilities receive adjusted confidence-weighted scores, unlike LAVEidk accuracy, which does not reflect prediction probabilities.The examples include both corrected high-confidence answers and a distinction between high- and low-confidence errors.Source: D Samples visualizing predictions and… · p. 18

E Additional Results

Additional experiments report gains from training with CERTAINLYUNCERTAIN, compare model scales and metrics, and describe the evaluator selected for LAVE refusal judgments.

LLaVA with LoRA-SFT

Comparing 7B to 13B models

  • With mixed LLaVA instruction data, 13B clearly outperforms 7B on confidence-weighted accuracy and ECE (IDK); with CERTAINLYUNCERTAIN alone, 13B gains are marginal.The study compares 7B and 13B models under both training-data settings.Source: E Additional Results · pp. 18–19, Table 7 · p. 19
  • LAVEidk metrics stay on par across 7B and 13B models using the same training data, while confidence-weighted accuracy still differentiates them.Source: E Additional Results · p. 19

LAVE IDK Judgement Accuracy

  • Mistral-7B was selected for LAVE refusal judgments after high evaluator performance, with latency and inference cost also considered.Table 8 reports evaluator accuracy on 100 random samples from the extraneous split.Source: E Additional Results · p. 19, Table 8 · p. 19

F Implementation Details

The implementation details specify threshold searches, training procedures, and the conversion used for VizWiz evaluation.

Thresholding baselines

Training configurations

  • LoRA SFT follows official Qwen-VL and LLaVA settings; R-tuning trains on instances with zero LAVE accuracy, and DPO follows Silkie implementations.Source: F Implementation Details · p. 19

VizWiz evaluation

  • For VizWiz, predictions judged as IDK by the LAVE refusal prompt are converted to “unanswerable” before standard VQA evaluation.Source: F Implementation Details · p. 19

G.1 More Details on sourcing from image

This section details procedures and prompt designs for creating answerable and unanswerable VQA examples across uncertainty categories. It also describes prompts for judging refusals and rating answer accuracy against expert references.

G.1 More Details on sourcing from image

  • For GQA, the pipeline masks the question-linked object and other objects with the same label, inpaints them, and replaces the answer with a sampled “I don’t know” response.For VQA, GPT-4 identifies salient objects from question-answer pairs; Grounded-SAM masks them before LaMa inpainting. GQA masks use ground-truth bounding boxes.Source: §G.1 · p. 20

G.2 Prompts for Data Creation

  • Knowledge prompts pair similar questions so one requires external knowledge or absent commonsense, while the other is answerable from context despite not being explicit in image pixels.The unanswerable question should not rely on information explicitly absent from the image; questions should be natural and avoid details that remain difficult even with external knowledge.Source: §G.2 · pp. 20–21
  • Complexity prompts make an unanswerable question difficult even for humans to infer about an existing object, while pairing it with a convoluted but directly answerable question.Complexity is to arise only from answer difficulty or framing, not from requiring additional information.Source: §G.2 · p. 21
  • For extraneous-object pairs, the pipeline masks a question-relevant object, inpaints the image, and prompts GPT-4V to generate a question answerable for only one image.Questions should concern object properties rather than mere presence, and should be relevant to each image independently.Source: §G.2 · pp. 22–23
  • Ambiguity prompts make an unanswerable question admit multiple possible answers because the referenced objects or people are indistinguishable, without requiring additional information.The paired answerable question uses convoluted framing but has a direct, accurate answer.Source: §G.2 · pp. 23–24
  • Temporal prompts pair an unanswerable prediction about future events that cannot be inferred from the caption with an answerable question requiring temporal commonsense.They exclude questions about exact time, location, or thoughts, and require the answerable question to have a clear, unambiguous answer.Source: §G.2 · pp. 24–25

G.3 Prompts for Evaluation

  • The refusal prompt assigns 1 to refusals and 0 to non-refusals, and asks evaluators to explain their rating.A response that first says the answer cannot be determined remains a refusal even if it then offers a speculative answer.Source: §G.3 · p. 25
  • Examples treat bare non-answers as refusals, but answers that express uncertainty while still answering as non-refusals.Examples include “I don’t know” and “I don’t see a car” as refusals, versus “I am not sure, but I think the answer is blue” as a non-refusal.Source: §G.3 · pp. 25–26
  • Answer accuracy is rated against expert reference answers on a 1–3 scale: incorrect or irrelevant, ambiguous or incomplete, or correct.The prompt also requires a rationale; an example gives an incorrect answer a rating of 1.Source: §G.3 · p. 26

Related papers

Loading 2407.01942v1…