A Review on Explainability in Multimodal Deep Neural Nets

I. INTRODUCTION

• Deep neural nets perform effectively across single- and multimodality tasks, but opaque processing obscures decisions and undermines credibility, transparency, social acceptance, and usability [2]–.Explainability seeks to clarify how and why a model reaches an outcome and is described as an important evaluation metric.Source: §I · p. 1
• In vision-language tasks, models can answer without visual grounding, while attention maps may not identify relevant evidence or guarantee explanatory power,,,.Misaligned image-text pairs show why accuracy alone is insufficient: a model should be right for the right reason.Source: §I · pp. 1–2
• Earlier explainable-AI surveys largely address general or unimodal settings, whereas this review focuses on multimodal explainability, especially vision-language tasks, using disentangled representations, explanations, and counterfactuals.Prior work also explores visual and textual explanations with varied evaluation metrics, and modular interpretability, often tested on synthetic datasets such as CLEVR –.Source: §I · p. 2

A. ORGANIZATION OF THE SURVEY

• The survey covers fusion and XAI taxonomies, multimodal explanation methods, evaluation, stakeholder needs, datasets, bias and fairness, adversarial attacks, challenges, and future research directions.Its organization spans Sections 2–12, including attention-based, counterfactual, interactive, graph-based, and attribute-based explanations.Source: A. ORGANIZATION OF THE SURVEY · p. 2

II. MULTIMODAL MACHINE LEARNING

• Multimodal learning combines heterogeneous sources such as image, text, audio, and video, using early, late, or hybrid fusion and shared representations for downstream tasks,, –,.Modalities differ in scale, format, predictive power, and contribution, and may be analyzed with deep nets, kernel methods, or graphical models,,,.Source: §II · p. 3
• Multiple modalities can improve predictive power and yield richer, task-relevant representations through complementary and cross-modal information, while making explanation of model decisions and bias a key concern,,.The paper notes applications across generic and critical domains, including healthcare, autonomous robots, and finance.Source: §II · p. 3
• Persistent multimodal heterogeneity makes efficient analysis difficult, motivating fusion frameworks, interpretability, and benchmarks to expose flaws in evaluation, dataset bias, robustness, and spurious correlations –.Interpretability, explainability, and contextual cognitive reasoning can help understand multimodal data.Source: §II · p. 4

Typical challenges

• Multimodal AI faces challenges in representing heterogeneous data, translating between modalities, aligning their elements, fusing information, and sharing knowledge across modalities.Fusion must account for modalities’ differing predictive power, importance, noise, and missing data; explainability is important for a comprehensive global view and addressing these challenges.Source: A. TYPICAL CHALLENGES IN MULTIMODAL… · p. 5
• Multimodal fusion can overfit across data sources; dynamic adaptive fusion and gradient blending are proposed to select modality blends based on overfitting behavior.Source: A. TYPICAL CHALLENGES IN MULTIMODAL… · p. 5

Early fusion

• Early fusion combines modality features through concatenation, pooling, or gated units, but differing dimensions and unsynchronized timestamps can cause data loss; similar sampling rates are one proposed remedy,.Training, pooling, and convolution are also discussed for combining sequential and discrete events with continuous data.Source: A. TYPICAL CHALLENGES IN MULTIMODAL… · pp. 5–6

Late fusion

• Late fusion combines modality-specific decisions using rules such as Bayes, max-fusion, average-fusion, or majority vote; it suits uncorrelated sampling rates or dimensions but is not proven superior to early fusion,,,,.Source: A. TYPICAL CHALLENGES IN MULTIMODAL… · p. 6

Hybrid fusion

• Hybrid fusion learns a joint representation at a shared layer and propagates loss back to feature extractors; the review describes it as far superior to early and late fusion,.PCA and autoencoders are used to reduce dimensions, while gradual fusion from higher- to lower-contribution modalities is reported to perform well,,.Source: A. TYPICAL CHALLENGES IN MULTIMODAL… · p. 6

C. RECENT DATA FUSION TECHNIQUES

• Common fusion pools include concatenation, element-wise multiplication, and weighted sums, while tensor fusion models unimodal, bimodal, and trimodal interactions.Low-rank fusion addresses tensor fusion’s computational burden but still creates a complex architecture; bilinear or trilinear pooling can also cause high dimensionality and large outer-product computations,. Single-instance feature fusion may miss local inter-model correlations.Source: C. RECENT DATA FUSION TECHNIQUES · p. 6
• Recent alternatives include gradual fusion from higher- to lower-predictive-power modalities, mutual-information variation maximization, and Tucker decomposition to reduce bilinear-pooling complexity.Tucker fusion uses low-rank matrix decomposition to address bilinear pooling’s drawbacks.Source: C. RECENT DATA FUSION TECHNIQUES · p. 7
• Other methods model modality-specific and cross-modal interactions through memory-based learning, dynamic adaptive fusion, interpretable cross-modal exchange, and parameter-free channel exchange.Channel exchange models the tradeoff between inter- and intra-modal interactions.Source: C. RECENT DATA FUSION TECHNIQUES · p. 7
• Further approaches search for fusion architectures, use global-workspace attention to address uncertainty, or model higher-order common and unique sequences without attention for sentiment analysis.These methods span architecture selection, uncertainty handling, and sequence modeling.Source: C. RECENT DATA FUSION TECHNIQUES · p. 7
• Explainability, interpretability, justification, and reasoning can improve multimodal fusion predictions, while differing representation scales, alignment, and resolutions make multimodal models harder to interpret than single-modality models.Black-box behavior makes outcomes difficult to justify, motivating explainable AI methods for multimodal tasks.Source: C. RECENT DATA FUSION TECHNIQUES · p. 7

• Deep neural networks can be opaque because of hidden-layer processing, and their accuracy often trades off against interpretability; identifying which modalities or features drive predictions is difficult.Source: §III · p. 7
• Interpretability concerns understanding a decision’s cause and consistently predicting model results, while explainability addresses why a prediction occurred and how the system reached it.Source: §III · p. 7
• In high-impact settings, AI explanations should provide understandable evidence, support trust and fairness, and address the GDPR’s “Right to explanation”,,,.Source: §III · p. 7
• XAI can support debugging wrong predictions, improving models, and detecting adversarial attacks or dataset artifacts, and may be introduced before, during, or after modeling.The paper frames this evolving work as part of developing responsible AI.Source: §III · pp. 7–8

• Pre-modeling supports explainability before model development through data understanding, documentation, summarization, unbiased collection, explainable feature engineering, and sound experimental design.Source: A. LEVELS OF EXPLANATION MODELLING · p. 8
• During modeling uses inherently explainable designs, including hybrid models, Contextual Explanation Network, Self-Explaining Neural Network, joint prediction and explanation, and regularization-based architectural adjustments.Source: A. LEVELS OF EXPLANATION MODELLING · p. 8
• Post hoc modeling extracts explanations after model development using perturbations, backpropagation, or proxy models, and can be applied to any model without constraints.Source: A. LEVELS OF EXPLANATION MODELLING · p. 8

B. SCOPE OF EXPLANATION

• Local explanations derive predictions for a particular data point, while global explanations characterize the model as a whole; local explanations are often preferred.Source: B. SCOPE OF EXPLANATION · p. 8
• Gradient-based model-specific methods [148], [149] have been proposed for convolutional neural networks, but do not focus on an entire image region to address a query.Source: Figure 4 · p. 8, B. SCOPE OF EXPLANATION · p. 8
• Model-agnostic methods, are model-independent and modular; PDP, ICE, permutation importance, LIME, and Shapley values [154] explain feature effects or contributions.ICE visualizes feature dependence for individual data points, whereas PDP averages across data points; permutation importance uses the increase in model errors after feature alteration, and LIME perturbs one data point's input.Source: B. SCOPE OF EXPLANATION · p. 8
• Other model-agnostic methods include prototypes and criticisms, influence functions, D-RISE for object detectors, and HINT for human-aligned visual grounding.Prototypes represent the dataset, while criticisms identify data points not well represented by them; influence functions support model understanding, debugging, dataset-error detection, and adversarial-attack creation. HINT attends to image features humans consider important for predictions.Source: B. SCOPE OF EXPLANATION · p. 8

• Feature attribution highlights significant image regions but does not capture semantic reasoning or interactions.Source: C. FEATURE ATTRIBUTION BASED METHOD · p. 8
• Visualization methods identify input features affecting model output and are classified as backpropagation-based or perturbation-based.Source: C. FEATURE ATTRIBUTION BASED METHOD · p. 8
• Gradient methods include pixel-intensity saliency maps, sharper Gradient Input maps, and Grad-CAM class-specific layer gradients for localization.Source: C. FEATURE ATTRIBUTION BASED METHOD · p. 9
• Integrated Gradients computes output gradients to input features without modifying the network; DeepLift uses reference activations, LRP propagates relevance backward, and SHAP averages contributions.SHAP is described as the average contribution of all data points in a prediction.Source: C. FEATURE ATTRIBUTION BASED METHOD · p. 9
• The surveyed attribution methods require access to model parameters and an understanding of the model architecture.Source: C. FEATURE ATTRIBUTION BASED METHOD · p. 9
• Perturbation methods compare model outputs for original and altered inputs to identify sensitive features, with examples including Occlusion Sensitivity, RISE, and LIME.Source: C. FEATURE ATTRIBUTION BASED METHOD · p. 9

D. DISTILLATION METHODS

• Local approximation methods explain predictions for a single data point by building an approximate local model, as in LIME.The methods are presented as a distillation category.Source: D. DISTILLATION METHODS · p. 9
• Model translation builds a surrogate on top of the model being explained, such as an inherently explainable decision tree.The paper distinguishes this approach from local approximation.Source: D. DISTILLATION METHODS · p. 9

E. INTRINSIC METHOD

• Intrinsic methods are inherently explainable; attention mechanisms focus on important visual and textual regions, while joint training combines explanation and prediction tasks.The paper identifies both attention-based self-explanation and joint training as intrinsic approaches.Source: E. INTRINSIC METHOD · pp. 9–10
• LIME models prediction changes from input changes at a local point, SHAP averages data-point contributions, and HINT uses image regions humans attend to [sic].Occlusion sensitivity compares model output for original and altered inputs to identify prediction-sensitive features [163].Source: E. INTRINSIC METHOD · p. 9
• Model-translation examples include tree, graph, and rule-based methods, which build a proxy model such as a self-explanatory decision tree.Source: E. INTRINSIC METHOD · p. 10
• Joint training builds an explainable model by training its explanation task together with its prediction task.Source: E. INTRINSIC METHOD · p. 10

A. SIGNIFICANCE OF EXPLAINABILITY IN MULTIMODAL DATA

• Multimodal explanations can justify decisions and localize supporting evidence better than unimodal approaches by drawing on complementary strengths across modalities.Depending on the scenario, language or vision may provide the more useful insight.Source: A. SIGNIFICANCE OF EXPLAINABILITY IN… · p. 10
• Multimodal explanations can help diagnose model behavior, debug flaws, detect adversarial attacks, and identify fairness issues and bias.These capabilities support troubleshooting and can improve model performance and predictive and explanatory power.Source: A. SIGNIFICANCE OF EXPLAINABILITY IN… · p. 10
• Explanations should answer how and why a model predicts by revealing its driving factors, yet many systems prioritize scientific modeling over user-centric, role-based needs,.For X-ray diagnosis, radiologists may need visual evidence while clinicians may prefer a text-based justification.Source: A. SIGNIFICANCE OF EXPLAINABILITY IN… · p. 10
• As vision-language tasks move from fused-embedding tasks to visual commonsense reasoning, they require semantic understanding, object identification, and relations between scene entities.This growing need for higher-order reasoning has contributed to explainable AI for multimodal settings.Source: A. SIGNIFICANCE OF EXPLAINABILITY IN… · p. 10
• Role-based explanations should meet different stakeholders’ needs and support interactive interfaces where multimodal interactions can be followed by feedback that changes the system.Human visual explanations can indicate where to attend, while textual explanations can describe how image regions were attended [7].Source: A. SIGNIFICANCE OF EXPLAINABILITY IN… · p. 10

B. INTROSPECTION AND JUSTIFICATION SYSTEMS

• Justification-based systems produce explanations people understand but lack causal interpretation,, whereas introspection-based systems examine internal network behavior that people may not understand,.Source: B. INTROSPECTION AND JUSTIFICATION… · p. 10
• Real-world settings need both justification and introspection systems, because lucid explanations can improve human understanding and performance.Source: B. INTROSPECTION AND JUSTIFICATION… · p. 10

V. MULTIMODAL EXPLAINATION METHODS

• The review groups multimodal explanation techniques into attention-based, counterfactual, interactive, and attribute-based approaches.The classification concerns methods for explaining visual and textual modalities.Source: §V · p. 10

A. ATTENTION BASED APPROACHES

• Attention mechanisms weight important inputs and align or fuse modalities; VQA methods visualize image regions and question text, sometimes using human-in-the-loop evaluation [176].Guided backpropagation and occlusion are used to show regions and text a VQA model focuses on; proposed failure and knowledge prediction metrics assess interpretability.Source: A. ATTENTION BASED APPROACHES · pp. 10–11
• PJ-X uses attention and textual explanations with visual regions for VQA and activity recognition, while later frameworks coordinate pointing maps and text or seek faithful, consistent rationales.PJ-X passes attention masks between modules but may provide indirect, inconsistent explanations and requires explanation-annotated data; other work uses supervised human rationales and SQuINT to encourage consistent attention across reasoning and perception questions.Source: A. ATTENTION BASED APPROACHES · p. 11
• Attention-based work also combines visual, language, and spatial inputs, uses image attention and hierarchical features, and applies multiway answers and rationales to VCR and ViQAR [181] [187].Other applications include user-perception evaluation, self-critical image explanation training, self-driving and video explanations, action justification, competing-explanation comparisons, and anticancer compound sensitivity prediction.Source: A. ATTENTION BASED APPROACHES · p. 11
• Attention maps may not identify the right image region or match human focus, and their explanatory faithfulness is questioned because attention weights and gradients may not align.A lack of ground truth for evaluating explanations further limits assessment.Source: A. ATTENTION BASED APPROACHES · p. 11

B. COUNTERFACTUAL EXPLANATIONS

• Counterfactual explanations align with contrastive, causal human thinking by identifying minimal changes that could flip a decision, such as loan rejection, and provide actionable recommendations and recourse.Source: B. COUNTERFACTUAL EXPLANATIONS · p. 11
• In vision-language explanations, visual and textual counterparts assess how removing an input region changes the target classifier’s response toward the negative class.The review also situates this work within interactive, human-centered multimodal causal learning [32].Source: B. COUNTERFACTUAL EXPLANATIONS · p. 12
• Counterfactual studies span annotated phrase explanations, neuron-based visual explanations, video classification, complementary joint explanations, bias analysis, caption resilience, and contrastive pretraining [18, 192, 168, 191, 193, 194, 195].The phrasecritic model improves textual explanation quality but is not accurate or faithful to the underlying model; neuron-based visual explanations are described as accurate and require no additional attribute annotations.Source: B. COUNTERFACTUAL EXPLANATIONS · p. 12

C. INTERACTIVE APPROACHES

• Alipour et al. found explanations improve VQA accuracy when the system is wrong, using active attention and feedback to correct answers.The evaluation also considers user trust, mental models, and usability.Source: C. INTERACTIVE APPROACHES · p. 12
• Interactive explanation work includes virtual agents, XAlgo’s question-answering account of internal state, and conversational systems for multisensory customer relationship management.Explanation-based conversational systems are described as providing better explanations than conventional report-based systems.Source: C. INTERACTIVE APPROACHES · p. 12

D. GRAPH BASED APPROACHES

• Graph approaches use question-specific scene-graph interactions, interpretable context-semantic bottlenecks, object-relation graphs, and VQA explanations from scene graphs and attention without additional explanation data.The scene-graph method uses existing annotations rather than manually generated explanation data.Source: D. GRAPH BASED APPROACHES · p. 12
• Knowledge graphs add semantic information and domain knowledge to improve interpretability in healthcare and education, while neuro-symbolic AI work proposes integrating them into deep learning.One medical approach builds a multimodal knowledge-aware hierarchical attention network.Source: D. GRAPH BASED APPROACHES · pp. 12–13

E. ATTRIBUTE BASED METHODS

• Attribute-based methods explain visual features with attributes and counterexamples, generate textual justifications for novel concepts with minimal supervision, and produce faithful explanations for unseen classes.The section also identifies SIAM and describes attribute maps as linearly combined to reveal learned features and interpretable regions.Source: E. ATTRIBUTE BASED METHODS · p. 13
• Complementary class-pair activations provide discriminative cues for CAM generation, while a joint visual-attribute embedding and multi-channel module produces textual and visual explanations for zero-shot classification.The latter approach is described as improving user trust and revealing model flaws.Source: E. ATTRIBUTE BASED METHODS · p. 13

VI. EXPLAINATION EVALUATION

• Explanation evaluation is task-specific and subjective, spanning functional, human-grounded, and application-grounded methods.Without ground truth, evaluation is difficult; proposed criteria include generalization, fidelity, faithfulness, relevance, persuasibility, usability, satisfaction, causality, novelty, and completeness [53]. Human evaluation can assess simulatability, accuracy, response time, consistency, and satisfaction, including through leakage-adjusted simulatability.Source: §VI · p. 13
• BLEU-4, METEOR, ROUGE, CIDEr, and SPICE automate translation, overlap, and image-description evaluation; uses crowdsourcing, while measures student-model simulation of a teacher.BLEU-4 is described as quick and inexpensive; SPICE is a semantic image-caption metric, and the student-teacher metric targets scalability to unseen data.Source: §VI · pp. 13–14

VII. DIVERSE EXPLAINABILITY REQUIREMENTS

• Explainability needs vary by user and goal: creators may need layer-level responses for debugging, while non-experts need functional explanations of outputs.Examiners may also need to understand how input data informs predictions to assess trustworthiness, bias, and regulatory compliance.Source: §VII · p. 14
• Modular approaches can improve interpretability, and multimodal VQA explanations can justify answers in both textual and visual formats.These different explanation formats can support users with different requirements.Source: §VII · p. 14
• Explainability datasets include RPM visual-reasoning problems, counting questions sampled from VQA 2.0 and Visual Genome, and VQA-X and ACT-X textual justifications.VQA-X became a benchmark for explainable models.Source: §VII · p. 14
• Other resources include VQA-e for VQA predictions and explanations, CUB for attribute-based zero-shot learning, and VQA-CP for multimodal bias evaluation.The review says multimodal XAI datasets were initially scarce and that their availability can make explainability goals more traceable.Source: §VII · p. 14

IX. MULTIMODAL BIAS AND FAIRNESS

• Modality imbalance and biased data can undermine fairness and transparency, while models may amplify biases learned from datasets.Bias can originate in crowdsourcing, natural perceptions, word embeddings, sentence-level representations, and feature selection.Source: §IX · p. 14
• VQA models can rely on statistical irregularities or linguistic priors instead of images, risking wrong evaluation and poor performance outside training distributions.Attention maps can help assess whether models use image evidence rather than keyword dependencies.Source: §IX · p. 14
• A regularization method addresses bias by maximizing functional entropy and balancing modality contributions [233].The paper also notes that automated recruitment systems can retain gender and ethnicity discrimination even after input masking.Source: §IX · p. 14
• Visual cues in image-captioning data can contribute to gender bias; described approaches target gender-neutral captions under occlusion and correct gender classification otherwise.These approaches aim for reasonable predictions based on cause and effect rather than performance alone [193].Source: §IX · pp. 14–15

X. ADVERSARIAL ATTACKS ENHANCE / INTERPRETIBILITY

• Adversarial examples can mislead classifiers despite attention and localization, while adversarial perturbations can also aid network interpretation and robustness analysis.The review describes attention-guided attacks on VQA and visual-grounding attacks [237], underscoring the need for defense and mitigation.Source: INTERPRETIBILITY · p. 15

XI. OBSERVATIONS AND RECOMMENDATIONS

• The review calls for interactive, user-centered explanations that incorporate human feedback, support recourse, and meet the needs of diverse stakeholders.It recommends auditable and accountable systems, human-in-the-loop evaluation, and explanations adapted to lay users, developers, and domain experts.Source: §XI · p. 15, §XII · p. 16
• Multimodal explanations need to be faithful, complete, grounded, robust, and evaluated for bias, fairness, trustworthiness, causality, and visual grounding.The review also identifies inconsistent natural-language explanations, language bias, and vulnerability to small adversarial perturbations as concerns.Source: §XI · p. 15

XII. DISCUSSIONS AND FUTURE DIRECTIONS

• Future work should develop inherently interpretable multimodal models, reasoning-based datasets, and metrics that assess explanation quality and higher-level reasoning.The review also identifies persistent challenges in representation, alignment, translation, fusion, co-learning, common-sense reasoning, contextual adaptation, and labeled-data requirements.Source: §XII · pp. 15–16

OPEN AI DALL- E AND CLIP MODELS

• The review presents DALL·E [239] and CLIP as transformer-based advances connecting text and images, while noting CLIP’s limits, associative bias, and susceptibility to typographic attacks.CLIP uses image-sentence similarity and a single pretraining task for zero-shot generalization; feature visualization can reveal neurons responding to concepts across modalities.Source: OPEN AI DALL- E AND CLIP MODELS · p. 16

XIII. CONCLUDING REMARKS

• The review categorizes multimodal explainability methods and highlights the importance of context, reasoning, semantic attributes, bias, fairness, and adversarial defenses.It emphasizes new benchmarks and diagnostic datasets as key to realizing the benefits of multimodal explainability.Source: §XIII · p. 16
• The review frames its survey as a roadmap for research to improve trust, transparency, and the ability to understand and trace model flaws.It notes the field’s rapid evolution and substantial scope for further development.Source: §XIII · p. 17