PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations

Shubham Innani, Suhang You, Adam Shephard, Bhakti Baheti, Francesco Ciompi, Joe Yeong, Nasir Rajpoot, Michael Feldman, Solene Florence Kammerer-Jacquet, Dimitrios Makris, Geert Litjens, Anne L. Martel, Jana Lipkova, April Khademi, Spyridon Bakas, for the MICCAI SIG-CompPath

arXiv:2608.28820v1cs.AIcs.CV

TL;DR

CompPath XAI lacks consistent terminology, validation, and evidence needed for safe, accountable clinical deployment. This review formalizes the field with seven definitions, a three-axis taxonomy, five method families, and task-driven recommendations. It argues that clinical utility requires moving from visually plausible explanations toward quantitative verification and coordinated validation.

  • Problem

    CompPath XAI literature is fragmented by inconsistent terminology, overlapping methods, and limited validation, while clinical deployment requires evidence concerning safety, accountability, robustness, and generalizability.

  • Method

    The review defines seven pathology-specific XAI concepts, organizes methods into five families and three orthogonal axes, and develops task-driven recommendations for clinical questions and deployment contexts.

  • Results

    The review identifies the transition from visualization to verification as the defining challenge and recommends integrated pipelines combining localization, attribution, perturbation testing, and concept validation.

  • Takeaways & Limitations

    Clinical utility requires quantitative validation, cross-site testing, concept grounding, and clearer standards involving researchers, regulators, and pathologists.

  • Takeaways & Limitations

    Example-based evidence does not by itself explain why predictions may be unreliable and should be interpreted alongside attribution or concept-based evidence.

Abstract

from arXiv · show

Computational pathology (CompPath) is transforming medicine by leveraging artificial intelligence (AI) algorithms to support diagnosis, prognosis, and treatment prediction from gigapixel whole-slide images. Clinical adoption is progressing, but is constrained by concerns about safety, accountability, and regulatory oversight in high-stakes clinical environments. Explainable AI (XAI) systems hold promise for building trust and enabling verification, yet the literature remains fragmented due to inconsistent terminology, overlapping methodological families, ad hoc validation, and current reviews. This review aims to formalize XAI methods in CompPath through the: i) introduction of a pathology-centric vocabulary comprising seven core terms; ii) development of a taxonomy across methodological families and three orthogonal axes (stage, type, scope); and iii) establishment of a task-driven framework that maps five clinical questions to recommended methods, method evaluation, and deployment context. Five key gaps between current XAI capabilities and clinical deployment are identified, and actionable steps are proposed to advance XAI for CompPath.

1. Introduction

CompPath enables data-driven analysis from digitized pathology and diverse clinical inputs, but opaque models and limited validation constrain trustworthy clinical deployment. This review addresses fragmented XAI terminology and methods through pathology-specific definitions, a three-axis taxonomy, and task-driven recommendations.

  • CompPath uses whole-slide images and other clinical, molecular, and radiological inputs for diagnosis, prognosis, treatment prediction, and biomarker discovery.
  • Regulatory approvals show clinical adoption is possible, but real-world deployment, longitudinal validation, failure-mode characterization, robustness, and generalizability remain limited.
  • Complex deep-learning models are not directly interpretable, making failures difficult to understand and preventing pathologists from consistently tracing individual decisions.
  • Explainability needs depend on clinical application, with high-risk prediction and decision-support tasks requiring stronger evidence than lower-risk or intrinsically inspectable outputs.
  • The review defines seven XAI concepts, positions methods along stage, type, and scope axes, organizes five methodological families, and surveys over 100 CompPath studies.
  • Its recommendations map five clinical questions to methods, evaluation approaches, model architecture, deployment context, and validation requirements.

2. Definitions

The review establishes a pathology-specific vocabulary that separates related XAI concepts by their questions, evaluation criteria, and roles in clinical translation. It distinguishes technical validity from expert judgment and governance functions.

  • The framework defines seven interconnected terms: interpretability, explainability, transparency, causal validity, causability, justifiability, and contestability.
  • Causal validity asks whether perturbing a candidate region changes the prediction, whereas causability asks whether the explanation aligns with established histopathological knowledge.
  • The definitions are grounded in CompPath examples including nuclear pleomorphism, gland density, mitotic count, tumor-infiltrating lymphocyte density, WSIs, TMAs, ROIs, and MIL pipelines.
  • The terms are treated as distinct, complementary concepts with different scopes and evaluation criteria rather than interchangeable synonyms.
  • Interpretability and explainability concern comprehensible model behavior and evidence, transparency concerns system documentation, and justifiability and contestability support defense, auditing, and challenge.

3. Taxonomy

The taxonomy classifies CompPath XAI methods using independent stage, type, and scope axes, alongside five methodological families. Most methods cluster as post hoc, model-specific, and local, but the axes remain independently combinable.

  • Stage: Stage distinguishes intrinsic explanations produced within a model from post hoc explanations applied after training.
  • Type: Type distinguishes model-specific methods using architectural internals from model-agnostic methods based only on input–output behavior.
  • Scope: Scope ranges from local explanations of individual WSIs, patches, or ROIs to regional/slide and global explanations across cohorts or populations.
  • Three-axis framework: The three axes are independent, although CompPath methods most commonly occupy the post hoc, model-specific, local combination.
  • Methodological families: Five methodological families are identified: backpropagation-, perturbation-, feature-, concept-, and example-based methods.
  • Methodological families: Backpropagation methods generate gradient-based heatmaps, perturbation methods modify inputs or fit surrogates, and feature methods attribute predictions to interpretable features.
  • Methodological families: Concept-based methods use human-interpretable pathology concepts, while example-based methods retrieve prototypes, counterfactuals, or neighborhood-based trust scores.

4. Recommendations for Task-Driven Application of XAI in CompPath

XAI methods in CompPath should be selected according to the clinical explanation question, model architecture, and deployment context rather than treated as universally interchangeable. The framework distinguishes associational localization and attribution, causal validation, pathologist-aligned concept and example reasoning, and decision-boundary assessment, with method-specific validation requirements.

  • Method selection depends on the XAI question, model architecture, and deployment context, including research exploration, regulatory submission, and clinical integration.
  • Spatial localization: Spatial localization methods visualize model focus, while expert-annotated ROIs and cross-scanner consistency checks are needed to assess whether highlighted regions are biologically plausible.Recommended approaches include CAM, Grad-CAM, attention overlays, LIME, and RISE; these primarily reveal correlation rather than causation.
  • Feature attribution: Feature-attribution methods provide numerical importance estimates for model-accessible pipelines, but WSI-scale computation and choices such as propagation rules or baselines affect interpretation.Recommended methods include LRP, Integrated Gradients, DeepLIFT, xMIL-LRP, and biologically interpretable-feature uses of SHAP.
  • Causal validation: Localization and attribution identify associated regions or features but do not establish causal influence, so perturbation-based validation should measure prediction changes while controlling for masking artifacts.Gradient-free perturbations provide architecture-agnostic causal validation, although WSI-scale evaluation is computationally intensive.
  • Concept-/example-based reasoning: Concept- and example-based methods translate model behavior into clinical reasoning through semantic concepts, prototypes, and counterfactuals, but require well-defined concepts and expert validation across institutions.Semantic drift can make learned concepts diverge from clinical definitions and produce misleading explanations.
  • Decision-boundary assessment: Decision-boundary methods identify unstable, ambiguous, or out-of-distribution cases for review, but institution- and scanner-specific similarity calibration remains necessary.Example-based evidence should be interpreted alongside attribution or concept-based evidence because it does not itself explain why a prediction may be unreliable.

5. Discussion

The review argues that CompPath XAI must progress from visually plausible explanations toward clinically verifiable evidence, supported by standardized benchmarks, concept validation, cross-site testing, and workflow-aware deployment. It organizes methods and recommendations to address regulatory, computational, conceptual, and integration barriers.

  • Taxonomy and terminology: The review introduces a taxonomy that organizes CompPath XAI by methodological families and orthogonal stage, type, and scope dimensions.It also defines explainability, interpretability, justifiability, causality, and transparency to reduce inconsistent terminology.
  • From visualization to verification: Visualization methods such as heatmaps and attention overlays show where models look but cannot establish whether their reasoning is correct.Verification additionally requires causal attribution, concept-level validation, and cross-site robustness.
  • From visualization to verification: Only a few studies jointly evaluate causal attribution, concept validity, and cross-site robustness, leaving clinical verification uncommon at scale.The review recommends integrated pipelines combining localization, quantitative attribution, perturbation testing, and concept validation.
  • Evaluation gaps: Current XAI evaluation relies largely on qualitative heatmap inspection because CompPath lacks shared datasets, ground-truth explanations, and standardized protocols.The proposed benchmark would span anatomies, institutions, scanners, and staining protocols while measuring faithfulness, stability, and cross-site robustness.
  • Evaluation gaps: Concept-level validation remains limited because pathologists reason about features such as gland architecture and immune infiltration, while annotations are costly and variable between observers.The review calls for hierarchical concept taxonomies, multi-pathologist annotations, automated concept discovery, and concept-drift detection.
  • Computational barriers: High-fidelity methods can require hours per gigapixel slide, encouraging faster but less rigorous alternatives such as Grad-CAM and attention overlays.The review identifies co-optimized engineering and feature-space attribution as potential ways to improve scalability.
  • Regulation and deployment: Regulatory frameworks do not specify sufficient explainability evidence, motivating risk-stratified requirements from qualitative visualization to multi-method validation and cross-site robustness testing.The review also recommends standardized reporting templates and trained regulatory reviewers.
  • Regulation and deployment: XAI outputs are often disconnected from clinical workflows, requiring integration into pathology viewers, multi-slide aggregation, and unified dashboards.Foundation models create additional challenges because larger-scale aggregation does not intrinsically confer interpretability.

Supplementary Material

The supplementary material expands the review’s terminology, taxonomy, method descriptions, and survey of CompPath XAI studies.

  • Supplementary Material: The supplement expands seven pathology-centric XAI term definitions, taxonomy subdimensions, five method-family descriptions, and a survey of over 100 CompPath studies.Its sections organize these materials by definitions, taxonomy, methods, and study survey.

S1. Pathology-Centric Definitions of Seven XAI Terms

The paper distinguishes seven XAI terms by whether they describe model properties, explanation artifacts, clinical usefulness, governance, or stakeholder control. These distinctions clarify what evidence is needed for clinical and regulatory assessment.

  • Interpretability: Interpretability is a non-binary property of a model whose structure, components, or native outputs allow humans to reason about input-output behavior directly.Examples include logistic regression using inspectable histological features and models outputting quantities such as mitotic count or segmented gland maps.
  • Explainability: Explainability is the capacity to provide faithful, context-appropriate evidence for a prediction through an accompanying artifact understandable to its intended user.Artifacts may include heatmaps, attribution scores, segmentation maps, calibrated probabilities, concept scores, or natural-language rationales.
  • Transparency: Transparency is a system-level property concerning accessible, sufficient, and verifiable documentation for scrutinizing, reproducing, monitoring, and governing an AI system across its lifecycle.Technical and regulatory disclosure can diverge: a system may satisfy formal disclosure requirements while remaining technically opaque, or disclose code without meeting regulatory expectations.
  • Causal validity: Causal validity asks whether highlighted features or regions actually influence the model prediction rather than merely correlating with it.Evaluation uses interventions such as occlusion, insertion or deletion curves, and counterfactual modification of candidate histological features.
  • Causability: Causability measures whether an explanation enables a pathologist to form a useful, clinically meaningful causal understanding of the output.It depends on alignment with recognized structures and requires pathologist-centered evaluation, distinguishing clinical usefulness from attribution fidelity alone.
  • Justifiability: Justifiability concerns whether deploying an AI system and using its outputs can be defended under clinical and regulatory standards.It depends on validation, fairness, regulatory compliance, and standard-of-care alignment; a heatmap alone cannot justify deployment or patient-level action.
  • Contestability: Contestability is the ability of pathologists, patients, and other stakeholders to inspect, challenge, override, or appeal AI outputs within the deployment workflow.The paper gives user-facing override mechanisms, with disagreements recorded in diagnostic reports, as an example.

S2. XAI Taxonomy for CompPath: Three Orthogonal XAI Dimensions

The taxonomy organizes CompPath XAI methods along three independent dimensions: stage, architectural dependency, and explanation scope. It distinguishes intrinsic from post hoc methods, model-specific from model-agnostic methods, and local from global explanations.

  • Stage: Stage separates intrinsic methods built into model computation from post hoc methods applied after a model has been trained.Intrinsic approaches include linear models, decision trees, and generalized additive models, while post hoc approaches generate separate explanation artifacts.
  • Stage: Attention-based MIL occupies an intermediate procedural position because attention weights are generated during inference but may not faithfully or causally represent feature importance.Accordingly, attention-based explanations require faithfulness, stability, and clinical validation rather than automatic interpretability claims.
  • Type: Type distinguishes model-specific methods using internal components from model-agnostic methods that infer importance from inputs and outputs, often through perturbations.Model-specific examples include gradients, activations, and attention weights; model-agnostic examples include SHAP, LIME, and occlusion sensitivity.
  • Scope: Scope separates local explanations for individual predictions from global explanations of model behavior across datasets or patient populations.Local methods can identify case-specific drivers such as mitotic activity, whereas global methods can rank influential features or quantify concept-level effects using TCAV.
  • Integrated taxonomy: The three axes are independent, although CompPath methods are mostly clustered as post hoc, model-specific, and local.Table 2 records the combined Stage-Type-Scope categorization using separate symbols for each dimension.

S3. Per-Method Descriptions of XAI Methods in CompPath

This section reviews individual XAI methods in CompPath, organizing their operating principles, strengths, limitations, adoption, and validation needs. Backpropagation and attention methods provide useful localization, but their outputs require careful interpretation and clinical validation.

  • The review uses a uniform narrative structure covering each method’s principle, strengths, limitations, practical considerations, adoption, and validation practices.
  • Attention-based methods: Trainable attention identifies influential patches from slide-level labels and is efficient for MIL, but attention weights may reflect dataset bias or correlation rather than causal importance.Standardized evaluation and regulatory-grade validation remain limited despite broad research use.
  • Attention-based methods: Transformer attention captures cellular and higher-order tissue context, yet diffuse maps and correlation-based weights complicate interpretation and clinical validation.
  • Class activation mapping: CAM produces intuitive heatmaps for quick verification but requires a GAP layer and offers low spatial resolution that can miss subtle structures.
  • Gradient-based localization: Grad-CAM is architecture-agnostic and computationally lightweight, while Grad-CAM++ improves localization of multiple instances through higher-order gradient information.
  • Attribution methods: LRP conserves relevance across layers to produce stable, fine-grained attributions, whereas DeepLIFT can separate contribution signs but depends strongly on baseline and implementation choices.

S3.1.9. Deconvolution 54

This section describes deconvolution, perturbation-based, and feature-based approaches for understanding CompPath models. Their usefulness ranges from exploratory feature inspection to local attribution and interventional sensitivity analysis, with important validity and realism constraints.

  • Deconvolution: Deconvolution reconstructs input patterns that activate CNN filters, revealing learned textures and structures rather than directly explaining individual predictions.
  • Deconvolution: Deconvolution is best suited to exploratory analysis and model debugging, complemented by attribution or perturbation methods for individual predictions or clinical conclusions.
  • Integrated Gradients: Integrated Gradients integrates gradients from a baseline to the input, producing smoother attributions and serving as a benchmark for faithfulness evaluation.
  • LIME: LIME explains local predictions with a sparse surrogate model, but its fidelity and stability depend heavily on segmentation, kernel width, sampling, and regularization.
  • Occlusion Sensitivity: Occlusion Sensitivity provides architecture-independent, interventional importance estimates, but mask size and biologically unrealistic occlusions can produce misleading results.
  • SHAP: SHAP assigns additive feature contributions using Shapley values and supports local and global explanations, but requires meaningful features and can be computationally expensive.

S3.4. Concept-based methods

Concept-based methods explain CompPath predictions through pathologist-recognizable concepts, intermediate concept representations, language, or multimodal alignment. They offer clinically meaningful communication but remain constrained by annotation, dataset, reliability, and validation requirements.

  • Overview: Concept-based methods target high-level, human-interpretable representations rather than individual pixels or low-level features.
  • TCAV: TCAV measures model sensitivity to predefined concepts such as necrosis or glandular pattern through concept activation vectors in latent feature space.
  • TCAV: TCAV supports hypothesis-driven auditing but assumes linear concept separability and is sensitive to the quality, quantity, and diversity of concept examples.
  • Concept Bottleneck Models: Concept Bottleneck Models predict pathologist-defined tissue concepts before the clinical label, enabling quantitative concept validation and enhanced auditability.Pathology applications include Gleason grading, but annotation and generalization challenges remain.
  • Image captioning: Image captioning converts visual patterns into textual descriptions, but requires paired image-text data and can generate inaccurate or visually unsupported descriptions.
  • Vision-language methods: Vision-language alignment supports zero-shot concept matching, semantic search, and clinically intuitive queries, but requires high-quality paired datasets and remains early-stage in CompPath.
  • Example-based methods: Counterfactual explanations test interpretable input changes for causal reasoning, while realistic tissue generation and biological constraints limit their reliability.

S4. Extended Deployment Guidance

Deployment guidance recommends matching XAI methods and validation rigor to the model’s development stage, clinical risk, and computational setting. Expensive methods should establish trustworthiness during validation, while fast, pathologist-aligned explanations support production use.

  • Deployment context: CAM, Grad-CAM, and exported attention overlays provide lightweight qualitative verification for computationally constrained or real-time settings.These methods typically add less than 10% to inference time.
  • Validation strategy: Prospective validation and regulatory submissions should combine intrinsic explanations, post hoc attributions, and gradient-free perturbation methods to obtain qualitative and quantitative evidence.
  • Reporting: Regulatory-grade XAI reporting should document model layers, baselines, propagation rules, faithfulness metrics, and stability metrics to enable independent review.
  • Deployment context: Computationally expensive perturbation and concept-grounding methods belong in development and validation, whereas fast Grad-CAM or attention overlays are preferred for production deployment.
  • Stage-specific guidance: Early validation should test histological plausibility and systematic failure modes, while regulatory evidence supports intended use, limitations, human factors, and risk controls.
  • Clinical integration: Clinical workflows benefit more from concept-based and example-based explanations in pathologist-aligned language than from abstract pixel-level heatmaps.
  • Future directions: Mechanistic interpretability and causal representation learning are identified as emerging directions that could recover recognizable structures or support causal modeling in CompPath.

S5. Literature

Table 8 organizes the studies reviewed in the article by their corresponding XAI methods and specific anatomical contexts.

  • Table 8 lists the studies reviewed in the article.
  • Each study is associated with a corresponding XAI method.
  • The table situates these methods within specific anatomies.
Loading 2608.28820v1…