PaperlayerView exact revision on arXiv

Source-linked AI summary

Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

Marco Stefani, Edoardo Parpaiola, Matteo Bino, Gianluca Caregnato, Francesco Roncolato, Daniel Fusaro, Loris Nanni, Alberto Pretto

arXiv:2610.12703v1cs.CVcs.HCcs.LG

TL;DR

The paper addresses limited objective evidence on how standard classifiers’ explanations align with human visual attention. It compares post-hoc saliency with held-out gaze across architectures and domains, finding distinct attention patterns and metric dissociations.

  • Problem

    Prior explainability studies often rely on qualitative inspection or use gaze to supervise learning, leaving limited objective evidence about inherent alignment in standard classifiers.

    Source: §1 · p. 2

  • Method

    The authors train classifiers without gaze supervision and compare architecture-specific post-hoc attributions with human fixation maps using a normalized alignment framework.

    Source: §1 · p. 2, §7 · p. 20

  • Results

    CNNs show more spatially distributed attention than ViTs’ localized focus; on CXR, MambaLRP has JSS = 0.8214 and CSS = 0.8742 but PCC = −0.0108.

    Source: §6.4.2 · p. 17, §7 · pp. 20–21

  • Takeaways & Limitations

    Architecture and training choices affect alignment: backbone freezing improves medical-imaging interpretability without improving accuracy, making the objectives a trade-off.

    Source: §7 · p. 21

    Gaze annotations are never observed during learning, so the evaluation measures post-hoc alignment rather than gaze-supervised learning.

    Source: §6.1 · p. 12

Abstract

from arXiv · show

High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the alignment between model attention and human visual perception using eye-tracking ground truth. We formalize this comparison via a standardized normalization pipeline mapping post-hoc attribution maps and gaze heatmaps onto spatial probability distributions. Alignment is evaluated across fine-grained classification, medical imaging, and emotion recognition using architecture-specific XAI methods. We measure fidelity using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation, three complementary estimators capturing support overlap, mass discrepancy, and structural co-variation. Our results reveal distinct attention dynamics: CNNs tend to exhibit a more global focus, with spatially more diffuse saliency maps that correlate well with human perception in general contexts, while ViTs demonstrate fine-grained, patch-level attention, often focused on discriminative microfeatures. The Mamba architecture shows biased behavior: it achieves strong explainability in medical tasks but exhibits divergent intensity distributions in natural images; our analytical characterization of the metrics makes this dissociation explicit, since near-unit support overlap is attained simultaneously with a vanishing linear correlation. We further analyze the effect of backbone freezing and ensemble strategies on model interpretability. The results highlight how architectural and training choices influence the alignment between model saliency and human visual attention.

1. Introduction

The introduction motivates rigorous explainability evaluation for complex vision models and presents a gaze-based comparison of CNNs, ViTs, and Vision Mamba. The study treats gaze as an independent reference and examines alignment across multiple visual domains and training choices.

1. Introduction

  • Complex vision models trade transparency for representational capacity, while high-risk AI systems face transparency and human-oversight requirements.The introduction argues that post-hoc XAI is needed to recover human-understandable insights for complex models.Source: §1 · pp. 1–2
  • The study quantifies alignment between architecture-specific saliency maps and eye-tracking fixation densities for the same visual stimulus.Human gaze serves as an independent benchmark rather than a supervision signal.Source: §1 · pp. 2–3
  • The evaluation spans CNNs, ViTs, and Vision Mamba across natural-image classification, medical imaging, and facial emotion recognition.The authors also examine backbone freezing, ensemble strategies, pairwise model agreement, and generated heatmaps.Source: §1 · pp. 2–3
  • Prior work either relies mainly on qualitative assessment or uses gaze as a training target, unlike this study’s post-hoc evaluation of classifiers trained only for classification.The cited comparisons include medical imaging, facial analysis, and urban-scene attention studies.Source: §1 · p. 2

2. A Concise Taxonomy of XAI Methodologies

The section organizes XAI methods by when and what they explain, their model access, output, and computational mechanism. It formalizes local feature attribution as a common operator and contrasts perturbation-based with propagation-based approaches.

2. A Concise Taxonomy of XAI Methodologies

  • The attribution operator assigns each input location a signed score for its contribution to evidence supporting a target class.Unless stated otherwise, explanations are computed for the model’s predicted class.Source: §2 · p. 3
  • Ante-hoc methods are transparent by design, whereas post-hoc methods explain completed models; global methods summarize dataset behavior, while local methods explain individual predictions.The section distinguishes methods by timing and by whether an explanation describes a model overall or a fixed input.Source: §2 · pp. 3–4
  • Model-specific methods exploit internal gradients, activations, or weights, while explanation outputs include feature attributions, textual explanations, and representative examples.Feature-attribution maps highlight image regions or pixels contributing to a prediction.Source: §2 · p. 4
  • Perturbation methods mask or modify input regions and measure prediction changes, but pixelwise evaluation can require O(|Ω|) model evaluations.Applying masks to patches reduces cost at the price of coarser spatial support; exact Shapley evaluation is exponential in |Ω|.Source: §2 · p. 4
  • Propagation methods use network structure in a backward pass; gradients provide a basic attribution, while Guided Backpropagation retains only positive contributions.Other propagation methods include relevance decompositions such as LRP.Source: §2 · pp. 4–5
  • The paper’s evaluations mainly use post-hoc, local feature-attribution methods derived through propagation-based mechanisms.The section introduces a unified taxonomy spanning ante-hoc and post-hoc paradigms and local and global scopes.Source: §2 · pp. 3, 5

3. Related works

The review contrasts architecture-specific XAI methods and prior explainability studies, noting that existing work rarely validates model saliency quantitatively against human gaze across architectures and domains.

3.1. Explainable AI for CNNs

  • Grad-CAM discards negative evidence and has resolution limited by the final feature grid, while LRP requires a manually designed composite strategy for each model.AblationCAM instead estimates feature-map importance by measuring the target-score drop when a map is switched off, helping avoid gradient instability.Source: §3.1 · pp. 5–6

3.2. Explainable AI for ViTs

  • Vision Transformer explanations include Attention Rollout, LIME, and TransLRP; in one medical-imaging comparison, TransLRP outperformed the other methods in faithfulness and sensitivity.The review notes that naive rollout can highlight irrelevant tokens, while ViTLRP incorporates residual connections in its relevance composition [37].Source: §3.2 · pp. 6–7

3.3. Explainable AI for Vision Mamba

  • Mamba-Attention recasts Mamba as implicit causal attention, while MambaLRP modifies relevance propagation to handle SiLU and multiplicative gating.The review attributes relevance-conservation problems to bilinear gating and SiLU operations.Source: §3.3 · p. 7
  • LaTIM decomposes Mamba token interactions and yields more informative heatmaps than MambaLRP and Mamba-Attention, but its SiLU approximation avoids exact decomposition without model modification and retraining.The cited work links sparse, weak focus on relevant tokens with Mamba’s long-context memory limitations.Source: §3.3 · pp. 7–8
  • A survey describes Mamba as combining RNN and CNN characteristics, with linear-complexity long-range modeling, while contrasting CNN context limits and ViT resource demands [43].The survey also proposes a Mamba model with explainability approaches for medical tasks.Source: §3.3 · p. 8

3.4. Gaze-based Explainable AI

  • Prior gaze-validation studies use similarity metrics for facial analysis, gaze-alignment metrics for urban scenes, or gaze supervision during training, rather than the same evaluation setup [17].Other work extends gaze comparisons to driving [44] and video memorability, while the driving LRP study is limited to VGG16.Source: §3.4 · p. 8
  • This study applies multiple XAI methods across CNN, ViT, and Mamba architectures to assess which methods best fit each architecture and whether an overall superior approach exists.The review frames this as a quantitative alternative to qualitative visual inspection of saliency maps against human attention.Source: §3 · p. 5, §3.4 · p. 8

4. Datasets

The study uses datasets from fine-grained bird classification, chest X-rays, and facial emotion recognition, each paired with eye-tracking gaze annotations for evaluating human-model attention alignment.

4. Datasets

  • Across the three domains, gaze heatmaps recorded for each image provide the human-attention reference used to assess how models replicate perceptual processes.The dataset choice spans fine-grained classification, medical imaging, and facial expressions.Source: §4 · p. 8, §4.1 · p. 9, §4.2 · p. 9

4.1. CUB dataset

  • The CUB dataset contains 11,788 images of 200 bird species, with gaze from 25 non-expert participants observing and classifying similar bird images.The authors note that gaze precision may depend on how similar the compared bird species are.Source: §4.1 · p. 9

4.2. CXR dataset

  • The CXR dataset contains 1,083 MIMIC-CXR images labeled Normal, Pneumonia, or Congestive Heart Failure, with gaze recorded during a radiologist’s blinded cold reading.One board-certified radiologist with more than five years of experience read the images; sessions included calibration and fatigue-reduction procedures.Source: §4.2 · p. 9

4.3. Uniss-FGD dataset

  • KDEF-FGD combines 120 KDEF facial images from 40 individuals with gaze annotations from 20 observers freely viewing each face for three seconds.The authors caution that the predominantly young Western adult cohort may not represent other cultures or age groups.Source: §4.3 · p. 9

5. Models

The study uses canonical ResNet50, ViT-Base, and ViM-Base models to compare convolutional, attention-based, and state-space architectures while supporting established XAI methods.

5. Models

  • The CNN experiments use ImageNet-pretrained ResNet50 with a replaced classification layer and Grad-CAM, Score-CAM, and Ablation-CAM applied to its last convolutional stage.Inputs are resized to 448 × 448 pixels; the new linear classification layer is initialized with Kaiming initialization.Source: §5 · pp. 9–10
  • ViM experiments use ImageNet-1K-pretrained ViM-Base, freezing all parameters except the head and [CLS] token, and apply MambaLRP for explanations.The authors also explored partial SSM-layer unfreezing and LoRA without significant performance improvements.Source: §5 · pp. 10–11
  • The ViT experiments use pretrained ViT-Base with ViTLRP, comparing a frozen-backbone setup with training that leaves all parameters unfrozen.In the frozen setup, only the classification head and [CLS] token are trainable.Source: §5 · p. 11
  • ResNet50, ViT-Base, and ViM-Base were selected as vanilla architectures to isolate their computational paradigms and support reliable application of XAI methods.The authors note that XAI methods relying on gradients and structural assumptions can be difficult to interpret on custom architectures.Source: §5 · p. 11

6. Experimental Analysis

The experimental analysis evaluates gaze-based alignment of machine saliency across model architectures and visual domains, including the effects of freezing and ensembles.

5. Experimental Analysis

  • The evaluation compares CNNs, ViTs, and Vision Mamba across fine-grained natural classification, medical diagnostics, and facial emotion recognition.Architecture-specific XAI methods are compared with empirical eye-tracking ground truth.Source: §6 · p. 11
  • The framework uses eye-tracking ground truth to quantify the fidelity of explanations rather than relying only on qualitative assessment.The study aims to isolate differences associated with network design, backbone freezing, and ensemble strategies.Source: §6 · p. 11
  • The experiments examine how backbone freezing and ensemble strategies affect explanation fidelity across architectures and domains.These factors are analyzed within the study’s cross-architecture, cross-domain evaluation.Source: §6 · p. 11

6.1. Problem Formulation

The formulation treats gaze alignment as an emergent property of classifiers trained on labels alone and evaluates it across architectures, domains, and training interventions. Gaze maps are comparable within each dataset, but not across datasets.

6.1. Problem Formulation

  • Eye-tracker fixation densities are kernel-smoothed from fixation points weighted by dwell time, and the gaze dataset is partitioned into training and test subsets.Source: §6.1 · p. 11
  • Gaze maps are comparable within a dataset but not across datasets because acquisition protocols, sampling frequencies, and observer populations differ.Source: §6.1 · p. 11
  • Gaze annotations are never observed by the learner, so measured alignment is an emergent property rather than a result of gaze supervision.Model parameters are obtained by empirical risk minimization on labels alone.Source: §6.1 · pp. 11–12
  • The study asks how alignment varies by architecture and domain, whether it generalizes from training to test data, and how backbone freezing and ensembling affect it.Source: §6.1 · p. 12
  • Model-to-model agreement separates inter-architecture divergence from human alignment, since models can agree with each other while both align poorly with gaze, or vice versa.Source: §6.1 · p. 12

6.2. Similarity measures between saliency maps

The section maps raw attributions and gaze maps to spatial probability distributions before comparing them. Its three similarity measures capture distinct properties, so their joint use can expose agreement in support or mass alongside weak structural correlation.

6.2. Similarity measures between saliency maps

  • The normalization pipeline maps raw attribution and fixation-density maps onto the probability simplex before similarity is computed.The pipeline resamples maps onto the input pixel lattice and produces distributions for comparison.Source: §6.2 · p. 12
  • For ViM, the comparison retains only positive MambaLRP relevance because fixation densities are non-negative; counter-evidence is discarded for all architectures alike.Source: §6.2 · p. 12
  • JSS compares whole-distribution similarity, CSS emphasizes mass discrepancies including low-activity regions, and PCC measures linear structural co-variation.Source: §6.2 · p. 13
  • JSS and CSS are sensitive to intensity reparameterization, whereas PCC is invariant to positive affine changes and therefore measures structure independently of intensity scale and offset.Source: §6.2 · p. 13
  • Maps with the same support but unrelated intensity profiles can yield JSS and CSS near 1 while PCC approaches 0, making joint reporting necessary to interpret dissociation.The paper reports this pattern for a medical dataset.Source: §6.2 · p. 13

6.3. Model validation

The section first establishes classification performance as context for interpreting model attention, then specifies its metrics and ensemble evaluation. Ensemble outcomes vary by dataset: some combinations improve performance, while others do not.

6.3. Model validation

  • Classification validation establishes that the baseline models achieve solid, competitive performance before their attention is compared with human gaze.Source: §6.3 · pp. 13–14

6.3.1. Classification Metrics

  • Multi-class AUC uses a one-vs-rest strategy and is averaged across classes; reported global metrics use macro-averaging, weighting classes equally.The paper notes this is conservative for imbalanced CXR and immaterial for balanced KDEF classes.Source: §6.3.1 · p. 14

6.3.2. Classification Results

  • Ensembles average members’ softmax outputs before taking the argmax, a sum-rule strategy expected to help when member errors are weakly correlated.Source: §6.3.2 · pp. 14–15
  • On KDEF-FGD, ResNet50 + ViM-base achieved 99.29% accuracy, exceeding ResNet50 (96.32%) and ViM-base (97.24%), while ResNet50 + ViT-base reached 99.08%.In contrast, CNN–ViT ensembles did not improve performance on CUB, while the frozen-plus-unfrozen Transformer ensemble performed better than its individual models.Source: §6.3.2 · p. 15

6.4. XAI Quantitative Results and Discussion

The evaluation finds that model–human gaze alignment varies by domain and architecture, and that overlap and correlation metrics can tell different stories. Backbone freezing and one attribution-operator exception further qualify the results.

6.4.1. Metrics

  • Attribution and gaze maps are normalized to comparable spatial distributions, while metric summaries use dataset subsets and rank agreement can coexist with pointwise dissociations.KDEF-FGD gaze ground truth is available only for its 120-image test subset, and gaze acquisition methodologies differ across datasets.Source: §6.4.1 · pp. 15–16

6.4.2. Analysis and discussion

  • Train–test discrepancies stay at or below 0.027 on CUB and 0.069 on CXR for convolutional and Transformer configurations, except ViM-base with MambaLRP on CXR, which shifts by 0.083, 0.140, and 0.126 across JSS, CSS, and PCC.The authors attribute the exception more plausibly to numerical conditioning of the relevance-preserving rule than to generic classifier overfitting; the study is bounded to three datasets by computational overhead.Source: §6.4.2 · pp. 16–18
  • On CXR, ViT-base-F has JS = 0.6528 versus 0.4822 for ViT-base, but test accuracy is 73.83% versus 81.31%, so alignment and accuracy are not monotonically related.The comparison treats backbone freezing as a controlled intervention because the variants differ in their feasible parameter set.Source: §6.4.2 · p. 17
  • ResNet50 maintains solid scores across natural and medical domains, but on CXR GradCAM has Chi-square σ ≈ 0.36 versus ScoreCAM σ ≈ 0.07; MambaLRP struggles on natural images but reaches JS ≈ 0.82 and Chi-square ≈ 0.87 on CXR.The CNN and Mamba findings contrast broad cross-domain performance with strong but domain-specific localization.Source: §6.4.2 · p. 17
  • On CXR, MambaLRP reaches JSS = 0.8214 and CSS = 0.8742 but PCC = −0.0108, indicating shared support without correlated intensity profiles.ResNet50 with ScoreCAM offers a different trade-off, with PCC ≈ 0.35.Source: §6.4.2 · p. 17
  • Agreement depends on visual domain: models converge on everyday objects and facial markers but diverge on diagnostic features in CXR.The authors identify CUB, KDEF-FGD, and CXR as the domains showing this pattern.Source: §6.4.2 · p. 17
  • On CUB, ResNet50 and ViT-base show PCC ≈ 0.74; on KDEF-FGD, ResNet50 and ScoreCAM reach PCC ≈ 0.71, while fine-tuned ViT-base-F falls to PCC ≈ 0.28.Fine-tuning also shifts attention away from morphological features prioritized by CNNs.Source: §6.4.2 · pp. 17–18
  • ViM-base saliency is weakly aligned with other models across datasets, reaching PCC ≈ −0.11 on facial data; model-to-model agreement is not a proxy for human alignment.On KDEF-FGD, ResNet50 and ViT-base agree at PCC ≈ 0.70 and both align with gaze, whereas on CXR they agree at PCC ≈ 0.22 while both remain weakly aligned with gaze.Source: §6.4.2 · p. 18

6.5. Qualitative Analysis

The qualitative analysis contrasts coarse, diffuse CNN explanations with more localized Transformer and Mamba explanations, while noting that these patterns vary by image and task. It also compares model focus with human gaze across bird, chest X-ray, and facial-expression images.

6.5. Qualitative Analysis

  • For attention-based models such as Transformers, the analysis uses LRP-based alternatives rather than Grad-CAM.The passage describes Grad-CAM maps as coarse and blurred after up-sampling, whereas LRP propagates relevance to input pixels for higher-resolution explanations.Source: §6.5 · p. 18
  • On CUB, ViT and Mamba explanations localize relevance to specific 16 × 16 patches, unlike coarser Grad-CAM maps; this behavior is image-dependent.ViTs can highlight beaks or wing-color patterns, while CNNs often emphasize the bird body or head. ViTLRP has a 28 × 28 token grid versus a 14 × 14 CAM lattice, or four times as many independently addressable cells.Source: §6.5 · p. 18
  • For CXR, CNN heatmaps cover broader anatomy, while Transformer and Mamba explanations are highly localized, a contrast the authors say may be less intuitive for diffuse pathologies.The CHF example shows CNN attention across the cardiac and pulmonary area; the authors describe broader-area highlighting as more natural for holistic clinical review.Source: §6.5 · p. 19
  • On KDEF-FGD, human gaze centers on the nose and periocular area, CNN saliency emphasizes lower-face features, and MambaLRP produces sparse point-like activations.ViTLRP highlights multiple discrete facial landmarks; the authors characterize MambaLRP as failing to capture the contiguous regional focus seen in human gaze and CNN maps.Source: §6.5 · pp. 19–20

7. Conclusion

The paper compares gaze alignment for CNNs, ViTs, and Vision Mamba across three visual domains using a normalized, metric-based framework. It concludes that architecture and training choices shape explanation patterns, and that improved medical-imaging alignment from backbone freezing does not coincide with improved accuracy.

7. Conclusion

  • The study evaluates gaze alignment for CNNs, ViTs, and Vision Mamba on CUB, CXR, and KDEF-FGD using a specified normalization pipeline and three similarity estimators.The conclusion identifies Jensen-Shannon, Chi-square, and Pearson-based estimators and characterizes their invariances.Source: §7 · p. 20
  • CNNs show more spatially distributed attention than ViTs, whose localized, fine-grained focus is bounded by their attribution operators’ 14 × 14 and 28 × 28 supports.The conclusion also reports that backbone freezing improves interpretability in medical imaging, but not accuracy, so the objectives must be traded off explicitly.Source: §7 · pp. 20–21
  • Backbone freezing improves interpretability in medical imaging without improving accuracy, requiring the two objectives to be traded off explicitly.The conclusion links the gain to preservation of pretrained feature representations that produce more semantically meaningful explanations.Source: §7 · p. 21

Author Biographies

The biographies describe researchers affiliated with the University of Padua whose interests span computer vision, explainable AI, robotics, and related applications. They include early-career researchers and senior faculty.

Author Biographies

  • Marco Stefani, Edoardo Parpaiola, Matteo Bino, Gianluca Caregnato, and Francesco Roncolato earned B.S. degrees in computer engineering in 2024 and are pursuing M.S. studies at Padua.Their stated interests span XAI, computer vision, robotics, optimization, and applications including industrial systems, autonomous navigation, and control.Source: Author Biographies · p. 25
  • Daniel Fusaro is a University of Padua research fellow with a Ph.D. in computer engineering and interests including 3D scene understanding, LiDAR, navigation, and self-supervised multimodal perception.Source: Author Biographies · p. 25
  • Loris Nanni is a Padua professor whose work spans AI, pattern recognition, bioinformatics, and biomedical applications; Alberto Pretto is an associate professor leading the Field Robotics Group.Pretto's interests include autonomous robotics, computer vision, robot perception, and AI.Source: Author Biographies · p. 26
Loading 2610.12703v1…