PaperlayerView exact revision on arXiv

Source-linked AI summary

GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods

Sujay Belsare, Sudarshan Nikhil, Sushant Kumar, Ponnurangam Kumaraguru, Chirag Agarwal

arXiv:2606.14740v1cs.CV

TL;DR

Existing multimodal explainability evaluations lack ground-truth interactions for distinguishing genuine cross-modal reasoning from shortcuts. GridVQA-X provides a mathematically controlled diagnostic framework, revealing that current explainers fail to distinguish spatial reasoning from shallow cross-modal shortcuts.

  • Problem

    Existing MxAI evaluations lack reliable ground truth for testing whether explanations faithfully capture cross-modal interactions rather than shortcuts.

  • Method

    GridVQA-X uses synthetic, mathematically guaranteed explanations and paired controlled models to evaluate whether explainers distinguish genuine spatial reasoning from cross-modal shortcuts.

  • Results

    Current MxAI methods fail to distinguish models using genuine spatial reasoning from models exploiting shallow cross-modal shortcuts.

  • Takeaways & Limitations

    These findings indicate that current MxAI methods can misrepresent multimodal models’ decision processes and create an illusion of interpretability.

  • Takeaways & Limitations

    GridVQA-X is restricted to a controlled 2D abstraction covering spatial-relational composition and basic attribute binding, not temporal, commonsense, or complex mathematical synergies.

Abstract

from arXiv · show

With the increasing development of Vision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recent Multimodal Explainable AI (MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between true cross-modal reasoning (e.g., spatial composition) and shallow cross-modal shortcuts (e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the first diagnostic framework specifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train paired ground-truth models on identical architectures: $M_{\text{pure}}$, which learns robust spatial-relational reasoning and $M_{\text{spur}}$, which is structurally forced to rely on cross-modal shortcuts. This behavioral divergence creates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuine spatial-relational reasoning and those exploiting cross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.

1. Introduction

The introduction identifies a faithfulness gap in post-hoc multimodal explainability: existing evaluations provide little assurance that explanations reflect true multimodal decisions. GridVQA-X addresses this gap with a controlled diagnostic framework built around synthetic datasets with mathematically guaranteed unique ground-truth features.

  • Motivation: Post-hoc MxAI methods lack guarantees that their explanations faithfully reflect multimodal models’ true decision-making processes.Existing explainability benchmarks largely evaluate unimodal methods, despite targeting multimodal models.
  • Motivation: A gold-standard evaluation of cross-modal explainers requires absolute ground truth for interactions between modalities.Real-world multimodal tasks have noisy feature distributions and lack causally defined per-modality ground-truth attributes.
  • Motivation: Human-annotated multimodal features can be spuriously correlated with non-ground-truth factors, complicating faithfulness evaluation.This limitation further undermines evaluations based on real-world multimodal data.
  • GridVQA-X: GridVQA-X is introduced as a diagnostic framework explicitly designed to objectively evaluate post-hoc cross-modal explainers.The framework is used to benchmark recent state-of-the-art methods.
  • GridVQA-X: GridVQA-X introduces synthetic datasets Dpure and Dspur as a controlled MxAI testbed with mathematically guaranteed unique ground-truth features.The datasets use a spatial-reasoning taxonomy to isolate varying degrees of real multimodal reasoning.

2. Related Work

GridVQA-X is positioned at the intersection of explainability methods and multimodal models. Related work spans programmatically generated multimodal datasets, explainability approaches, and benchmarks that remain insufficient for evaluating cross-modal synergy.

  • Multimodal Datasets: Programmatically generated datasets such as CLEVR evaluate whether visual reasoning systems genuinely reason rather than exploit spurious correlations.These datasets provide generated 3D scenes and questions for rigorous evaluation.
  • Explainability Methods: Existing explainability methods include approaches based on feature representations, error debugging, perturbations, and projections.The passage identifies these as three broad categories of explainability approaches.
  • Comparison with Existing Benchmarks: Existing explainability benchmarks remain structurally insufficient for evaluating cross-modal synergy.The comparison emphasizes limitations in assessing interactions across different semantic spaces.
  • Comparison with Existing Benchmarks: OpenXAI focuses exclusively on unimodal data and provides no mechanism for assessing interactions across different semantic spaces.This limitation prevents OpenXAI from evaluating cross-modal interactions.

3. The GridVQA Dataset

GridVQA provides a controlled, fully observable spatial-reasoning environment with mathematically guaranteed ground-truth explanations. Its paired datasets separate genuine causal reasoning from enumerated shortcut behaviors, especially Bag-of-Words spatial alignment.

  • Dataset design: GridVQA abstracts visual scenes into spatial grids containing objects with independent color, shape, and position features, enabling controlled multimodal evaluation.Samples vary along parameterization axes including depth, question type, form, and density.
  • Ground-truth explanations: The dataset defines target and anchor objects as the unique ground-truth causal explanation, while distractors have zero causal effect on the answer.Any explainer attributing relevance to adversarial distractors is therefore verifiably unfaithful.
  • Shortcut taxonomy: The framework enumerates four shortcut families: Answer Priors, Bag-of-Words Alignment, Visual Feature Dominance, and Logical Decomposition.A spurious correlation is defined when a heuristic using a proper subset of true causal features has mutual information above random-chance threshold ϵ.
  • Paired datasets: Dpure removes the enumerated heuristics to force learning of the true causal graph, whereas Dspur removes unimodal biases while preserving a cross-modal shortcut.The paired construction controls model behavior while retaining identical dataset-level reasoning objectives.
  • Spatial shortcut: In Dspur, restricting attribute sampling makes P(Yans | Target Attrs) = 1.0, so target attributes perfectly predict answers without spatial reasoning.Dspur places no target-matching objects outside the valid spatial region, creating the intended Bag-of-Words trap.
  • Spatial shortcut: Dpure populates invalid regions with adversarial distractors, causing P(Yans | Target Attrs) ≪1 and forcing models to process spatial geometry.The resulting distractor-induced overcounting makes attribute-only prediction unreliable.

4. Model Training and Behavioral Dynamics

The study trains paired MDETR models with identical architectures and losses but different environments to induce distinct reasoning behaviors. Cross-evaluation shows that Mspur performs perfectly in-distribution yet fails on adversarial spatial composition while retaining attribute-only accuracy, confirming a Bag-of-Words shortcut.

  • Controlled Model Training: Identical MDETR architectures and loss functions are trained under Dpure and Dspur to isolate distinct reasoning behaviors.The controlled setup varies training environments while holding the architecture and losses constant.
  • Two-Phase Training: MDETR training uses visual grounding followed by QA prediction over grounded representations.Grounding penalizes unpredicted reference boxes with L1 and generalized IoU losses before QA predicts the logical answer.
  • Behavioral Cross-Evaluation: 1.000 training-distribution accuracy for Mspur falls to 49% on Dpure with adversarial distractors that break its spatial shortcut.The cross-evaluation tests whether the intended behavioral divergence was successfully imprinted.
  • Behavioral Cross-Evaluation: Mspur reaches 8% on Depth-2 Mixed and 14% on Depth-3 Mixed relational queries but retains 100% on Attribute-Only queries.The pattern isolates failure to spatial composition rather than non-relational attribute processing.
  • Behavioral Cross-Evaluation: The observed failure pattern confirms that Mspur relies entirely on a unimodal Bag-of-Words shortcut.Its relational-query failures alongside perfect Attribute-Only performance support the identified Case-1 behavior.

5. The MxAI Evaluation

The MxAI evaluation uses controlled faithful and shortcut-reliant models, multimodal attribution metrics, and pure, spurious, and cross-model scenarios to test diagnostic capacity, grounding, and synergy estimation. Results show that local and global explainers often misrepresent shallow shortcuts, mask volume, or noise as genuine cross-modal reasoning.

  • Evaluation design: The framework evaluates local heatmap methods and global synergy-scalar methods against Mpure and Mspur across controlled behavioral scenarios.Local methods include DIME, MULTISHAP, and MULTIVIZ; global methods include EMAP and InterSHAP.
  • Evaluation design: A faithful explainer should assign near-zero distractor relevance to Mpure, high distractor relevance to Mspur, and increasing synergy with depth only for Mpure.The evaluation uses IoU, Relevance Mass Accuracy, and the additive fallacy check to test these conditions.
  • RQ1: Shortcut diagnosis: MULTISHAP structurally prefers the spurious model because game-theoretic marginals align better with independent 1-to-1 feature detectors than entangled non-linear intersections.This prevents it from reliably distinguishing true spatial reasoning from shallow cross-modal shortcuts.
  • RQ2: Faithfulness grounding: DIME exploits mask volume: its RMA rises in dense grids (d0.7) without improved spatial precision, as IoU remains ≈0.20.The method defaults to shallow feature detection or noise rather than relational cross-modal constraints on Dpure.

6. Conclusion

GridVQA-X provides a diagnostic framework for evaluating multimodal explainability against mathematically verifiable ground-truth reasoning. Its evaluation finds that current explainers fail to capture true cross-modal synergy, instead diffusing relevance or relying on shallow object detection.

  • Framework contribution: GridVQA-X is introduced as the first diagnostic framework designed to evaluate MxAI methods against mathematically verifiable ground-truth reasoning.The framework is presented as a rigorous evaluation setting for multimodal explainability.
  • Main finding: State-of-the-art explainers fundamentally fail to capture true cross-modal synergy.This failure is identified as a critical blind spot in current MxAI research.
  • Failure modes: Local attribution methods either diffuse relevance arbitrarily to exploit mask volume or degrade into shallow object detectors.These behaviors illustrate how explainers can avoid representing genuine multimodal reasoning.

Supplementary Material · A. Detailed Related Work · A.1. Shortcut Learning in Vision-Language Models

Vision-language models can appear successful while relying on spurious correlations rather than intended causal logic. In VQA, documented shortcuts include language-only priors, background statistics, and keyword mappings that can inflate multimodal benchmark performance.

  • A.1. Shortcut Learning in Vision-Language Models: Deep learning models may exploit spurious training correlations instead of the intended causal logic, producing the Clever Hans effect.The term refers to models that appear to perform correctly while responding to subtle cues rather than solving the task.
  • A.1. Shortcut Learning in Vision-Language Models: The Clever Hans effect describes apparent task competence arising from reliance on unintended cues rather than genuine task solving.The phenomenon is named after a horse that seemed to perform arithmetic but responded to subtle cues from human observers.
  • A.1. Shortcut Learning in Vision-Language Models: In VQA, models can predict answers from language-only priors without actually examining the image.This is presented as a multimodal shortcut that bypasses visual evidence.
  • A.1. Shortcut Learning in Vision-Language Models: Shortcut strategies can artificially inflate performance on multimodal benchmarks.The passage introduces linguistic and background statistics and keyword shortcuts as examples.
  • A.1. Shortcut Learning in Vision-Language Models: Models may rely on linguistic and background correlations rather than features of the primary object.These correlations are identified as one category of shortcut in VQA.
  • A.1. Shortcut Learning in Vision-Language Models: Keyword shortcuts can produce a shallow, direct mapping between textual cues and predictions.The passage identifies keyword shortcuts as a distinct shortcut category in VQA.

A.2. Explainability Guided Training · A.3. Detailed Mechanics of Evaluated MxAI Methods

The paper frames explainability-guided training as a way to align vision-language reasoning with human expectations and reduce shortcut reliance. It also organizes evaluated MxAI methods by attribution scope, including local contributions and modality-level cross-modal interactions.

  • A.2. Explainability Guided Training: Explainability-guided training adds auxiliary optimization constraints so model decision-making better aligns with human expectations.In vision-language settings, this commonly uses visual-grounding supervision to penalize attention or predicted bounding boxes that misalign with ground truth.
  • A.2. Explainability Guided Training: Supervising intermediate representations can mitigate shortcut reliance by directly constraining how models form their decisions.GridVQA-X uses this paradigm as a controlled training mechanism rather than solely as an accuracy-improvement strategy.
  • A.2. Explainability Guided Training: GridVQA-X modulates grounding loss and dynamically weights cross-entropy loss to enforce behavioral divergence between M_pure and M_spur.The passage characterizes M_pure as aligning with causal spatial-relational geometry and M_spur as bypassing it.
  • A.3. Detailed Mechanics of Evaluated MxAI Methods: The evaluated explainability methods are broadly classified by attribution scope into local micro-level and global macro-level taxonomies.The local category targets individual image patches, pixels, text tokens, and localized interactions between them.
  • A.3. Detailed Mechanics of Evaluated MxAI Methods: Local attribution methods seek to explain predictions by isolating fine-grained contributions from image regions, text tokens, and localized cross-modal interactions.This framing includes contributions from individual image patches or pixels alongside textual tokens.
  • A.3. Detailed Mechanics of Evaluated MxAI Methods: InterSHAP shifts attribution from local tokens to the overall contribution of entire visual and textual modalities.It is described as a game-theoretic approach that isolates synergistic interactions between the inputs as wholes.
  • A.3. Detailed Mechanics of Evaluated MxAI Methods: EMAP diagnoses whether cross-modal interactions genuinely improve model performance by projecting predictions onto an empirical multimodality-additive function.The method modifies predictions so that cross-modal interactions are eliminated.

B. Dataset Generation Details and Proofs … B.5. Collision-Free Object Placement Requirements

GridVQA-X uses controlled generation and adversarial placement to eliminate answer-prior, spatial, visual, and partial-logic shortcuts. Its Pure split forces full spatial-relational evaluation, while collision-free placement preserves unambiguous relationships.

  • B.1. Case-0: Answer Prior Bias Elimination: The generator makes marginal answer distributions nearly identical across Pure and Spurious, preventing answer-prior bias from explaining behavioral differences.For high-depth sparse counting queries, the most frequent answer can still reach 37.2%, bounding prior-only performance.
  • B.2. Case-1: The Spatial Shortcut (Bag-of-Words) Elimination: 100% success in Dspur versus 36.07% in Dpure shows that adversarial distractors eliminate the bag-of-words spatial shortcut in the Pure split.The Dpure construction forces models to compute cross-modal spatial relations rather than rely on spatial-word matching.
  • B.3. Case-2: Visual Feature Dominance Elimination: Uniform randomization decorrelates color, shape, and direction from answers, removing reliable salience-based visual shortcut signals.The pipeline independently samples semantic attributes from uniform distributions.
  • B.3. Case-2: Visual Feature Dominance Elimination: Adversarial distractors and stochastic target-distractor balancing prevent majority-class prediction from reliably identifying relational targets.In relational Dpure queries, visually matching distractors outside valid regions make majority-class counting overestimate the true relational count.
  • B.4. Case-3: Logical Decomposition (Partial Logic) Elimination: The Pure split eliminates partial-logic heuristics by placing target-attribute confusers in every nonempty region created by dropping a spatial constraint.Because the relaxed intersection contains false positives, dropping any constraint strictly overcounts the valid targets and incurs training loss.
  • B.4. Case-3: Logical Decomposition (Partial Logic) Elimination: 16.40% and 16.87% predictive ratios result from dropping anchor 1 or anchor 2 in dense Depth 2 environments, forcing evaluation of the full intersection.These results are reported for D2 M F0 d0.7.
  • B.5. Collision-Free Object Placement Requirements: A collision-free placement algorithm sequentially places anchors and target objects on a discrete 2D grid while enforcing minimum cell-distance margins for directional relationships.For multi-hop queries, subsequent anchors are placed to maintain geometrically sound spatial relationships and avoid edge-case ambiguities.

C. Dataset Details

Figures 8–11 present representative Dpure and Dspur examples spanning reasoning depth, form, question type, and scene density, with each panel pairing an image and question with its answer.

  • Dataset examples: Figures 8–11 show Dpure and Dspur image–question examples across reasoning depths, forms, question types, and scene densities, each with a corresponding answer.The panels illustrate representative dataset instances.

D. Additional Results … D.3. Global Explainability Algorithms

The additional results compare model accuracies, report bucket-wise performance for three local explainability algorithms, and visualize global attribution and interaction-based explanations for M_pure and M_spur. These results are presented in Tables 3–8 and Figures 12–13.

  • D.1. Model Accuracies: Tables 3 and 4 report accuracies for existing open-source models and the trained models M_pure and M_spur.
  • D.1. Model Accuracies: The model-accuracy comparison includes both existing open-source models and the paper’s trained models.
  • D.2. Local Explainability Algorithms: Tables 5, 6, 7, and 8 provide bucket-wise performance results for three algorithms on the dataset.
  • D.2. Local Explainability Algorithms: The local explainability results are organized across four tables covering the three evaluated algorithms.
  • D.3. Global Explainability Algorithms: Figures 12–13 illustrate attribution mass distributions for both M_pure and M_spur across datasets.
  • D.3. Global Explainability Algorithms: Figures 12–13 also present interaction-based explanations for M_pure and M_spur across datasets.

E. Limitations and Future Work

GridVQA-X provides mathematically guaranteed explanations within a controlled 2D abstraction, but its realism and coverage of cross-modal synergies remain limited. Future work should extend the framework to richer spatial benchmarks and develop explainers that avoid shallow detection and hallucinated interactions.

  • Limitations: GridVQA-X lacks the noisy feature distributions and deeply entangled semantic representations found in real-world multimodal tasks.This limitation follows from its highly controlled 2D abstraction.
  • Limitations: The framework currently covers only spatial-relational composition and basic attribute binding, leaving other complex cross-modal synergies unmeasured.The supplied passage identifies this as a current scope restriction.
  • Future Work: Future work should extend these diagnostic principles to richer spatial reasoning benchmarks with continuous spatial relations and realistic object interactions.Such benchmarks are proposed to provide a more rigorous stress test.
  • Future Work: Future research should develop novel explanation algorithms designed to avoid shallow detectors and hallucinated complex interactions.The passage motivates this direction from the empirical behavior of existing explainers.
Loading 2606.14740v1…