PaperlayerView exact revision on arXiv

Source-linked AI summary

HalluPrism: When Multimodal Uncertainty Should Diagnose, Not Decide

Aman Prakash, Sourish Dasgupta, Tanmoy Chakraborty

arXiv:2608.29193v1cs.LG

TL;DR

MLLM confidence can be similar across failures caused by weak visual evidence, image-removal persistence, or cross-modal binding instability, limiting confidence as a diagnostic. HalluPrism re-runs predictions under visual degradation, blank-image replacement, and grounding or relation checks to form a joint signature. Across the tested benchmarks and models, this signature improves failure-family diagnosis, but does not automatically improve correctness ranking.

  • Problem

    Scalar confidence ranks likely correctness or abstention risk but does not identify whether a multimodal failure reflects visual weakness, image-removal persistence, or cross-modal binding instability.

  • Method

    HalluPrism forms a behavioral signature from visual-perturbation sensitivity, image-removal confidence retention, and grounding/relation-probe instability, then uses it for diagnosis and intervention routing.

  • Results

    Failure-family AUROC increased from 0.634 to 0.769 on HallusionBench and from 0.707 to 0.817 on VizWiz with (V, L, A), with smaller gains on POPE and VSR.

  • Takeaways & Limitations

    Multimodal uncertainty should characterize failure structure before being used to decide whether to abstain or correct.

  • Takeaways & Limitations

    HalluPrism is a behavioral diagnostic based on specific probes, and its gain varies across datasets; cross-dataset generalization remains open.

Abstract

from arXiv · show

Multimodal Large Language Models (MLLMs) can assign similar confidence to answers that fail for different reasons. We propose HalluPrism, a behavioral diagnostic that re-runs an answer after visual degradation, blank-image replacement, and grounding or relation checks. These targeted probes yield a signature over visual-perturbation sensitivity (V ), image-removal confidence retention (L), and grounding/relation-probe instability (A). Across 58K+ examples from four benchmarks and four MLLMs, image-removal confidence retention is most prevalent, while grounding/relation-probe instability better separates failure families. Only 18 of 48 source-target checks are diagonally aligned, so the coordinates should be interpreted jointly rather than as independent causal sources. With the dataset fixed, the joint signature improves failure-family AUROC from 0.634 to 0.769 on HallusionBench and from 0.707 to 0.817 on VizWiz, with smaller gains on POPE and VSR. In pooled XGBoost analysis, AUROC rises from 0.78 with scalar confidence to 0.95 with (V, L, A) and 0.97 when confidence is added. The same signature does not automatically improve correctness ranking. The three tested direct scalarizations can harm it. These results separate failure diagnosis from abstention scoring: multimodal uncertainty should characterize failure structure before it is used to decide whether to abstain or correct.

1 Introduction

HalluPrism argues that scalar confidence cannot explain heterogeneous multimodal failures and proposes behavioral probes to diagnose them before choosing correction or abstention. Its joint signature improves failure-family diagnosis and supports safer, targeted correction in tested settings.

  • High confidence can mask weak visual evidence, image-independent answer persistence, or cross-modal relation errors.
  • HalluPrism measures visual-perturbation sensitivity (V), image-removal confidence retention (L), and grounding/relation-probe instability (A).The scores are behavioral probes rather than causal decompositions of hidden mechanisms.
  • Only 18/48 source-target checks were diagonally aligned, so the signature should be interpreted jointly rather than as independent causes.
  • With the dataset fixed, AUROC rose from 0.634 to 0.769 on HallusionBench and from 0.707 to 0.817 on VizWiz using (V, L, A).Gains were smaller on POPE and VSR.
  • Matched correction routing maintained 87.6% accuracy and reduced breakage to 2.7% on Qwen3-VL/VSR.It selected visual re-grounding, anti-prior prompting, or relation checking instead of one generic prompt.
  • The contribution is a diagnosis-first view of multimodal uncertainty that distinguishes failure characterization from abstention or correction decisions.

2 Related Work

Prior work evaluates heterogeneous multimodal hallucinations, uncertainty, behavioral failures, and mitigation methods. HalluPrism combines these perspectives by using controlled behavioral probes to diagnose failure structure rather than treating uncertainty only as an abstention or correction score.

  • Multimodal hallucination benchmarks cover object, illusion, attribute, relation, perception, reasoning, and instruction-following failures.
  • Selective prediction and calibration primarily use uncertainty to rank correctness or decide when to abstain.
  • Behavioral testing exposes local failures through controlled input changes, whereas attribution methods analyze regions, attention flow, or image-token attention.
  • Existing mitigation methods commonly apply fixed correction rules, while HalluPrism uses diagnosis-first probing to support targeted intervention routing.

3 Methodology

The methodology treats uncertainty as a behavioral diagnostic: controlled perturbations produce a joint signature that can diagnose failure families and route interventions without assuming clean causal separability. It separately evaluates diagnosis, abstention scoring, and correction-path choice.

  • The study tests separability rather than assuming it, and evaluates failure-family diagnosis, three abstention scalarizations, and matched intervention routing as distinct uses.
  • HalluPrism attaches a bounded signature (V, L, A) to each prediction alongside scalar confidence and correctness or failure-family labels.
  • V measures response sensitivity to degraded visual evidence using blur, crop, reduced brightness, and Gaussian noise.
  • L measures confidence retention after replacing the image with a uniform blank image, while the stricter control additionally requires the original answer to persist.High L is a behavioral marker, not proof that the model ignores the image.
  • A measures sensitivity to grounding and relation pressure, using grounding re-checks and relation swaps when explicit relations are available.Raw A magnitudes are not compared across relation-bearing and non-relation-bearing datasets.
  • The signature is interpreted through raw or within-setting normalized coordinate dominance, with normalization preventing saturated L from dominating routing.
  • The protocol uses one clean pass, four visual-perturbation passes, one blank-image pass, and one or two alignment passes without retraining.

4 Experimental Setup

The study evaluates multimodal uncertainty as a perturbation-response diagnostic and correction-routing signal across multiple benchmarks, models, and failure families. It compares scalar confidence with the signature (V, L, A) under matched perturbation, abstention, routing, and audit protocols.

  • Evaluation Suite: The evaluation covers HallusionBench, POPE, VSR, and VizWiz-VQA, with roles spanning mixed failure diagnosis, object hallucination, relation errors, and answerability.Each model-dataset output receives scalar confidence c and the Perturbation-Response Signature (V, L, A).
  • Evaluation Suite: Four evaluation roles test failure-family diagnosis, object hallucination and abstention, relation-error routing, and answerability diagnosis.The suite is organized around benchmark-specific failure structures rather than an exhaustive taxonomy.
  • Tasks and Protocols: Matched perturbation protocols evaluate failure diagnosis, abstention, and intervention routing using scalar confidence c and s(x) = (V, L, A).The study compares no intervention, a generic reasoning prompt, and matched image-enhancement, anti-prior, or grounding/relation-check interventions.
  • Controls: Dataset-ID and within-dataset controls address partial alignment between failure families and datasets when evaluating source features.Control analyses add source features beyond dataset identity, model identity, and scalar confidence.
  • Human Audit: A blind 200-example audit uses three annotators and adjudicates 36 disagreements, treating the labels as a limited semantic plausibility check rather than ground truth.Annotators assess visual evidence insufficiency, language-prior reliance, alignment error, mixed, unclear, or no failure without seeing model scores or predicted sources.

5 Results

HalluPrism’s joint signature diagnoses heterogeneous failure families more effectively than scalar confidence, but its coordinates are entangled and do not directly determine abstention or correction decisions.

  • Probe entanglement: 18/48 source-target checks pass diagonal dominance, supporting interpretation of (V, L, A) as a joint perturbation-response signature rather than three causal labels.Visual checks pass 11/16, language 2/16, and alignment 5/16.
  • Failure-family diagnosis: 0.634 to 0.769 AUROC on HallusionBench and 0.707 to 0.817 on VizWiz show the strongest fixed-dataset gains, with smaller improvements on POPE and VSR.The corresponding changes are 0.823 to 0.839 on POPE and 0.722 to 0.724 on VSR.
  • Failure-family diagnosis: 0.78 to 0.95 AUROC with (V, L, A), and to 0.97 when confidence is added, demonstrates pooled diagnostic improvement beyond scalar confidence.The alignment coordinate contributes 0.50 of pooled XGBoost importance.
  • Probe robustness: 0.822–0.824 diagnosis AUROC after removing individual visual perturbations indicates robustness to the choice of blur, crop, brightness, or noise.Increasing perturbation strength raises mean V from 0.132 to 0.196 to 0.252.
  • Decision-facing behavior: The three tested source-aware scalarizations underperform in abstention AUROC, so diagnostic usefulness does not imply improved correctness-versus-abstain ranking.For Qwen3-VL, the best source-aware score drops from 0.81 scalar AUROC to 0.73.
  • Decision-facing behavior: Matched routing beats generic prompting in 10/16 settings but is not universally better than no intervention, whose pooled accuracy is 76.6% versus 75.9% for matched routing.On Qwen3-VL/VSR, routing lowers breakage from 15.5% to 2.7% while maintaining 87.6% accuracy.
  • Decision-facing behavior: 3.54 percentage points higher pooled accuracy and 5.12 points lower break rate are achieved by the probe-informed policy relative to generic chain-of-thought prompting.The improvements have 95% confidence intervals of [2.82, 4.25] and [4.12, 6.09], respectively, but VizWiz remains a boundary case.

6 Conclusion

HalluPrism improves failure-family diagnosis across 58K+ examples, but better failure typing does not guarantee better abstention scoring. Its signatures can help assess whether generic correction is safe while preserving this distinction.

  • HalluPrism signatures improve failure-family diagnosis across 58K+ examples.The same evidence distinguishes diagnosis from abstention scoring.

Limitations

HalluPrism’s diagnostic trends remain stable across tested settings, but its scope and interpretation are constrained by probe design, evaluation coverage, computational cost, and limited routing and audit evidence.

  • HalluPrism is behavioral, not causal, and its scores depend on specific probes and perturbation strengths.Broader probe choices remain open for future evaluation.
  • The evaluation covers four public benchmarks and four open-weight MLLM families, excluding several specialized and latency-critical settings.Cross-dataset generalization remains an important direction.
  • Each example requires 7 to 8 forward passes, while routing provides conditional actionability rather than universal mitigation.The human audit is small and checks semantic plausibility rather than causal source recovery.
  • The abstention analysis evaluates three direct hand-designed scalarizations, leaving learned or calibrated scalarizers for future work.

Ethics Statement

The paper frames HalluPrism as a diagnostic and auditing aid rather than a causal explanation or standalone safety guarantee. Its formal results separate failure diagnosis from correctness ranking and emphasize cautious intervention evaluation.

  • HalluPrism should be treated as a behavioral diagnostic layer, not a causal account of hidden MLLM mechanisms.Its signature reflects responses to controlled probes.
  • Scalar confidence cannot generally identify whether an error is an object, spatial, attribute, or answerability failure.The formal result motivates using (V, L, A) for failure-family diagnosis.
  • Failure diagnosis and abstention are distinct tasks: signatures may improve failure-family prediction without improving correctness ranking.Theorem 1 formalizes this role separation.

A.4 Why Naive Source-Aware Scalarization Can Hurt Abstention

Naive source-aware scalarization can reorder confidence scores without improving correctness, especially when a source coordinate is near saturation. Abstention also requires score sufficiency and validation-to-deployment transfer.

  • Multiplicative source penalties can reverse scalar-confidence rankings when penalty variation overwhelms confidence margins.This can reduce abstention quality when penalties are weakly related to correctness.
  • Near-saturated image-removal scores can introduce ranking inversions through small, weakly correctness-related penalty variations.The resulting scalarized score may compress or distort useful rankings.
  • Post-intervention accuracy equals a(1 − Breakπ) + (1 − a)Fixπ, so routing must evaluate repairs and harmful changes together.A generic intervention can repair failures while reducing overall accuracy.
  • If baseline accuracy a > 0.5, a one-point break-rate reduction contributes more to accuracy than a one-point fix-rate increase.Break-rate changes receive coefficient a, while fix-rate changes receive coefficient 1 − a.
  • Score-based abstention requires a scalar sufficient for the grounded safe-answer label and a validation-to-deployment risk-transfer condition.Without these requirements, it has no unconditional inference-time guarantee for open-world inputs.

B Dataset Details

HalluPrism evaluates a heterogeneous benchmark suite to diagnose broad failure families using joint perturbation-response behavior rather than a single pooled leaderboard. Across models and datasets, the signature is informative despite saturated image-removal retention and entangled probe responses.

  • Dataset Suite: The evaluation spans four public benchmarks designed to expose visual fragility, image-removal retention, and grounding or relation instability under different task conditions.The suite includes HallusionBench, POPE, VSR, and VizWiz-VQA, with each benchmark stressing different prediction risks.
  • Dataset Suite: Correct outputs are labeled no failure, while incorrect outputs receive coarse failure-family labels based on benchmark stress factors and available metadata.The mapping tests broad diagnosis rather than exhaustive error-taxonomy coverage.
  • Controls: The signature remains diagnostic when VSR is removed, with AUROC increasing from 0.761 to 0.839 and Macro-F1 from 0.329 to 0.422.This control removes relation-swap examples and tests whether diagnosis depends on VSR-specific alignment probing.
  • Source Profiles: 58,287 of 58,440 samples are L-dominant, but stricter answer matching reduces L dominance to about 51.6–51.7%.Replacing the broader confidence-retention formulation with stricter variants leaves the diagnostic pattern largely stable.
  • Interventional Entanglement: Only 18 of 48 source-target intervention checks pass the diagonal criterion, so V, L, and A should be interpreted as a joint signature rather than independent source labels.Off-target responses occur across model families, including alignment changes under visual or language ruin.
  • Failure-Family Profiles: Object hallucination and answerability cases show near-saturated image-removal confidence retention, with mean L values of 0.996 and 0.997, respectively.These source profiles illustrate why raw prevalence does not determine diagnostic value.

D.8 Dataset-Identity And Within-Dataset Controls

Dataset identity and model identity are important controls because benchmark stress factors and failure families are partly aligned. Within fixed datasets, the joint signature improves diagnosis most on HallusionBench and VizWiz, while transfer across held-out datasets remains weak.

  • Identity Controls: Dataset identity alone reaches AUROC 0.976 and Macro-F1 0.558, while adding V + L + A raises Macro-F1 to 0.783.The AUROC increase is small because the controlled baseline is already strong.
  • Identity Controls: The stricter outcome-family control reaches AUROC 0.853 and Macro-F1 0.348 with the combined representation.Correct predictions are mapped to no failure, requiring separation of correctness and error family.
  • Within-Dataset Diagnosis: On HallusionBench, AUROC rises from 0.634 with scalar confidence to 0.769 with (V, L, A), while the combined representation reaches 0.801.On VizWiz, the corresponding values are 0.707, 0.817, and 0.825.
  • Within-Dataset Diagnosis: On POPE, AUROC rises from 0.823 with scalar confidence to 0.839 with (V, L, A), while VSR changes from 0.722 to 0.724.The gains are smaller than on HallusionBench and VizWiz.
  • Transfer Boundary: Leave-dataset-out transfer is weak, including AUROC values of 0.375 when POPE is held out and 0.492 when HallusionBench is held out with V + L + A.These results are treated as a transfer stress test rather than the main diagnosis evidence.
  • Human Audit: Human auditing finds stronger agreement for source versus non-source attribution than for six-way labels, with Fleiss’ κ of 0.480 versus 0.236.At least two annotators agree on 82.0% of examples, while exact all-annotator agreement is 20.5%.
  • Ranking Boundary: Scalar confidence remains better for correctness ranking, with pooled AUROC 0.783 versus 0.767 for the best source-aware scalarization.At 5% coverage, scalar confidence has risk 0.018 versus 0.023 for weighted scalarization and 0.290 for max-penalty.
  • Routing: Matched routing beats generic intervention in 10 of 16 model-dataset settings and reduces break rate in 10 of 16 settings.The decision-facing benefit is conditional actionability rather than universal mitigation.

E.8 Reporting Metrics

The reporting framework defines behavioral audit quantities, conservative human-label comparisons, and reproducible inference budgets. It distinguishes source-coordinate agreement from broader behavioral diagnosis and reports probe costs explicitly.

  • Audit Metrics: The audit maps clear human labels Visual, Language-prior, and Alignment to V, L, and A, respectively.Mixed, unclear, and no-failure cases are reported separately rather than forced into the three coordinates.
  • Audit Metrics: The human audit treats the labels as diagnostic descriptions of observed examples, not causal claims about model internals.Low agreement would limit human interpretability of a single dominant source without invalidating the behavioral results.
  • Inference Budget: HalluPrism uses 7 forward passes without relation-swap probing and 8 passes when relation-swap probing is available.VSR is the only dataset with relation-swap probing.
  • Inference Budget: The full routing experiment adds two intervention branches per model-output instance beyond the core diagnostic computation.These branches compare generic and matched interventions.

F.2 Training Cost

HalluPrism is an inference-only diagnostic layer whose main resource cost comes from repeated probe passes rather than model training. Its cost is reproducible through fixed forward-pass counts and deterministic decoding, but it is not a zero-overhead confidence replacement.

  • Training and Inference: The diagnostic classifiers use at most four features, [V, L, A, c], and are negligible in cost relative to MLLM inference.The evaluated MLLMs are not fine-tuned; lightweight classifiers include logistic regression, XGBoost, and LightGBM.
  • Training and Inference: Greedy decoding with temperature 0 and disabled sampling reduces stochastic variation and avoids repeated generations as uncertainty samples.This makes the forward-pass budget easier to interpret.
  • Inference Cost: The probe cost scales linearly with the number of enabled probes, so deployments can disable unused branches for narrower diagnostic goals.The full protocol retains all probes to evaluate the joint signature.
  • Resource Reporting: The main reproducibility quantity is the number of forward passes, because runtime depends on backend, batching, model implementation, image resolution, and GPU type.The experiments report fixed pass counts and decoding settings rather than inventing unmeasured energy estimates.
  • Deployment Boundary: HalluPrism is better suited to audit-time diagnostics, high-risk checks, or routing than to running as a cheap scalar abstention score on every query.The full protocol can be reserved for uncertain, high-impact, or audit-selected examples.

G.5 Data Sensitivity

HalluPrism is intended for auditing and guarded correction-path selection, not standalone safety certification or deployment approval. Its responsible use is bounded by evaluated conditions, domain-specific validation, and monitoring.

  • VizWiz-VQA may contain private environments, personal objects, and accessibility-sensitive contexts, so raw images are not redistributed and outputs are treated as evaluation metadata.
  • HalluPrism supports model analysis, audit support, benchmark error analysis, and safer correction-path selection rather than standalone safety certification.
  • Deployment use should specify the task, user population, validation distribution, acceptable risk level, fallback policy, and monitoring plan.
  • The method should not replace human judgment or certify answers in high-stakes domains without domain-specific validation, expert review, and post-deployment monitoring.
  • The signature is bounded by evaluated datasets, models, prompts, perturbations, and routing choices, and may miss risks outside those conditions.
  • Correction routing can improve one model-dataset condition while damaging another, so systems should log original and corrected answers, actions, fix indicators, break indicators, and human-review outcomes.

I.5 Data And Privacy Considerations

The study uses public benchmark data without collecting new user images or personal information from annotators beyond standard logistics. Deployment documentation should define validation, routing, logging, and monitoring requirements before use.

  • The main experiments use public benchmark datasets and do not collect new user images.
  • The human audit protocol uses sampled benchmark outputs and does not require collecting personal information from annotators beyond standard annotation logistics.
  • Annotators do not see source scores or routing decisions, limiting score-induced bias during the audit.
  • Before deployment, system owners should document the target population, validation distribution, risk level, fallback policy, correction or review triggers, logging fields, and monitoring plan.
Loading 2608.29193v1…