PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable AI for Bioinformatics: Methods, Tools, and Applications

Md. Rezaul Karim, Tanhim Islam, Oya Beyan, Christoph Lange, Michael Cochez, Dietrich Rebholz-Schuhmann, Stefan Decker

arXiv:2212.13261v3q-bio.QMcs.AIcs.LG

TL;DR

Opaque and increasingly complex AI models create transparency, accountability, and fairness challenges in bioinformatics and healthcare. This paper surveys interpretable ML methods and tools, discusses domain-specific customization, and demonstrates XAI applications across bioimaging, cancer genomics, and biomedical text mining.

  • Problem

    Complex DNN and ML models are difficult to interpret, while bias can arise throughout the ML pipeline in sensitive biomedical applications.

  • Method

    The paper categorizes model-specific and model-agnostic interpretable ML methods, reviews tools, discusses customization for bioinformatics, and provides supporting notebooks and examples.

  • Results

    The paper demonstrates how XAI methods can improve transparency through case studies in bioimaging, cancer genomics, and biomedical text mining.

  • Takeaways & Limitations

    Bioinformatics applications can benefit from customized XAI methods that address diverse data types and support fairer decision-making.

  • Takeaways & Limitations

    Interpreting black-box models may identify statistically important features without establishing their biological significance, limiting trust without domain-knowledge validation.

Abstract

from arXiv · show

Artificial intelligence (AI) systems utilizing deep neural networks (DNNs) and machine learning (ML) algorithms are widely used for solving important problems in bioinformatics, biomedical informatics, and precision medicine. However, complex DNNs or ML models, which are often perceived as opaque and black-box, can make it difficult to understand the reasoning behind their decisions. This lack of transparency can be a challenge for both end-users and decision-makers, as well as AI developers. Additionally, in sensitive areas like healthcare, explainability and accountability are not only desirable but also legally required for AI systems that can have a significant impact on human lives. Fairness is another growing concern, as algorithmic decisions should not show bias or discrimination towards certain groups or individuals based on sensitive attributes. Explainable artificial intelligence (XAI) aims to overcome the opaqueness of black-box models and provide transparency in how AI systems make decisions. Interpretable ML models can explain how they make predictions and the factors that influence their outcomes. However, most state-of-the-art interpretable ML methods are domain-agnostic and evolved from fields like computer vision, automated reasoning, or statistics, making direct application to bioinformatics problems challenging without customization and domain-specific adaptation. In this paper, we discuss the importance of explainability in the context of bioinformatics, provide an overview of model-specific and model-agnostic interpretable ML methods and tools, and outline their potential caveats and drawbacks. Besides, we discuss how to customize existing interpretable ML methods for bioinformatics problems. Nevertheless, we demonstrate how XAI methods can improve transparency through case studies in bioimaging, cancer genomics, and text mining.

Introduction

AI and DNNs support complex bioinformatics and biomedical tasks, but their increasing complexity can make predictions difficult to interpret. XAI addresses this opacity, while bioinformatics applications require adaptation of interpretable ML methods to diverse data and domain needs.

  • Introduction: Biomedical ML analyzes multimodal, high-dimensional data, including genomes, omics, imaging, clinical records, and biomedical texts.These data create challenges involving heterogeneity, dimensionality, and volume.
  • Introduction: Autoencoders learn complex feature interactions through latent representations, but their learned factors are not easily interpretable.Disentangling latent factors can provide insight into captured features and sample attributes.
  • Introduction: Complex DNNs can produce accurate predictions while obscuring how or why they reached particular outcomes, limiting transparency and accountability.Their predictions cannot be traced back easily, creating concerns in healthcare and other sensitive settings.
  • Introduction: XAI interprets black-box decisions by exposing influential factors and supporting human-comprehensible reasoning, transparency, and accountability.Interpretable models can reveal feature effects and interactions.
  • Introduction: Existing interpretable ML methods often require customization for bioinformatics data, and the paper demonstrates applications in bioimaging, cancer genomics, and biomedical text mining.The paper also connects XAI with improved decision fairness.
  • Helps avoid practical consequences: In cancer diagnosis, multimodal and multi-omics models may classify patients accurately while leaving biomarkers and decision logic difficult to explain.This opacity can complicate explanations to patients and raise clinical transparency concerns.

Reduces complexity and improves accuracy

Bioinformatics requires interpretable ML to connect high-dimensional biological features with reliable decisions and to address fairness, transparency, and accountability. The paper organizes existing methods and tools while emphasizing interpretable local and global model behavior.

  • Reduces complexity and improves accuracy: High-dimensional genomics requires selecting biologically significant features to identify cancer biomarkers and study gene interactions and function.Feature selection is challenging because biological processes involve interactions among thousands of genes.
  • Improves decision fairness: Bias can enter any ML stage, from data collection and feature selection through model training, tuning, and interpretation, producing unfair outcomes.Healthcare and biomedicine are especially sensitive because affected groups may receive systematically different advantages or disadvantages.
  • Improves decision fairness: Interpretable models can explain individual decisions while also showing overall model behavior, linking local explanations with global understanding.This supports traceability of model logic in clinical and biological applications.
  • Improves decision fairness: Explainability and accountability are legally required for AI systems affecting human lives, including automated decision-making covered by GDPR safeguards.The paper presents interpretability as a way to make AI-assisted decisions understandable to affected individuals.
  • Techniques and Methods for Interpretable ML: The paper categorizes interpretable ML methods as probing, perturbing, or model surrogation, with local and global interpretability levels.It provides methods, tools, papers, and notebooks covering bioimaging, cancer genomics, text mining, and reasoning.

Terminologies and notations

The paper distinguishes interpretability, explainability, transparency, feature importance, feature impact, and local versus global explanations through formal definitions and notation. These concepts describe how models connect inputs, features, predictions, and human-understandable reasons.

  • Interpretability concerns how well humans can understand a decision’s causes, whereas explainability concerns communicating an AI system’s internal workings.
  • A model is black-box when its internal principles and parameters are not human-traceable, and interpretable when a mathematical interpretation shows how predictions are generated.
  • Algorithmic fairness requires predictions not to favor or discriminate against individuals or groups based on sensitive attributes, while transparency requires an interpretable mapping between inputs and outputs.
  • Feature importance assigns non-negative scores to features, while feature impact assigns values from -1 to 1 that capture each feature’s local or global effect.
  • Global explanations describe model behavior across all instances, whereas local explanations explain an individual prediction using feature importance, decision rules, and counterfactual rules.
  • A W-perturbation changes selected feature values by a minimum delta to produce a different model output, and a surrogate model approximates a black-box model with R^2 approximately equal to 1.

Probing black-box models

Probing methods inspect black-box models using internal signals such as gradients, activations, and relevance propagation. In bioimaging, these methods produce heatmaps that identify image regions associated with predictions, with method-specific localization trade-offs.

  • Probing methods use a model’s internal workings, including gradient information, to generate heatmaps showing the relative importance of input features.
  • Grad-CAM uses globally averaged gradients as class-specific weights to highlight pixels and regions relevant to a target class.
  • Grad-CAM++ replaces globally averaged gradients with weighted pixel-wise gradients to better address multiple occurrences and differing orientations.
  • LRP performs a forward classification pass followed by backward relevance redistribution, assigning greater relevance to nodes contributing to higher layers.
  • LRP localizes class-discriminative regions more accurately than Grad-CAM++, while both methods highlight critical regions in explainable osteoarthritis diagnosis from MRI and X-ray images.
  • Attention methods identify important features or tokens, with SAN targeting high-dimensional feature sets and TabNet selecting semantically meaningful feature subsets at decision steps.

Perturbing black-box model

Perturbation methods explain predictions by changing features and measuring the resulting output changes, while attribution methods distribute prediction effects across features. The section also presents examples of feature, entity, and cancer-prediction explanations.

  • Permutation feature importance randomly changes one validation feature while leaving others intact, treating a significant accuracy drop as evidence of importance.
  • Feature importance can describe a single prediction, many samples, or overall model behavior through an explanation function that returns scores for all features.
  • SHAP uses Shapley values to represent each feature’s average marginal contribution across coalitions and feature orderings.
  • Global SHAP importance is obtained by averaging absolute Shapley values for each feature across instances, while equal contributions should yield equal Shapley values.
  • The examples include explainable biomedical entity recognition, SAN-based cancer-type prediction, and SHAP waterfall plots that trace feature contributions from expected output to model output.
  • Sensitivity analysis perturbs one feature while holding others fixed and measures how strongly the prediction changes, making it suitable for inspecting tabular feature sensitivity.
  • Sensitivity heatmaps may show which pixels resemble a predicted class but do not identify pixels pivotal to a specific prediction or support quantitative global validation.

Tree, textual, and rules-based approaches

Tree-based, rule-based, and textual approaches present model decisions at progressively more human-interpretable levels, from feature paths and impacts to rules and natural-language explanations.

  • Tree, textual, and rules-based approaches: A breast-cancer classifier predicted cancer with 75.3% probability, with gender, age, and marker genes identified as influential features.Combining these feature impacts can remain difficult for patients to understand.
  • Tree, textual, and rules-based approaches: Decision rules express predictions as IF-THEN conditions, optionally combining multiple feature-value statements with AND.The example uses gender, age, genes, and genotype to predict BRCA.
  • Tree, textual, and rules-based approaches: Natural-language explanations translate decision rules and model evidence into wording intended to be intuitive for patients.The example combines patient characteristics, oncogenes, and the predicted probability in prose.
  • Tree, textual, and rules-based approaches: Explanations can also describe how attention localizes malignant and normal image regions underlying a diagnosis.This provides a clinician-facing account of the image evidence used by the model.
  • Tree, textual, and rules-based approaches: LIME locally approximates black-box decisions with interpretable models, while Anchor extends this approach using precision-thresholded rule conditions.Rule explanations are highly interpretable when features are intelligible and conditions are short, but overlapping or contradictory rules are a drawback.
  • Tree, textual, and rules-based approaches: Decision trees represent feature tests at internal nodes, predicted classes at leaves, and classification rules along root-to-leaf paths.Tree ensembles can also support feature contributions and rule extraction from prediction paths.

Model surrogation strategies

Model surrogation explains complex black-box predictions by training an interpretable model to approximate them, often using sampled or top-k feature data.

  • Model surrogation strategies: Model surrogation trains an inherently interpretable model to approximate a black-box model’s predictions, typically as a locally faithful explanation.The surrogate may use the same data as the black box or sampled data, depending on problem complexity.
  • Model surrogation strategies: Top-k feature-space surrogates reduce the input to features identified as most important by the black box.This approach is presented as reasonable because the black box identifies important features with higher confidence.
  • Model surrogation strategies: Any interpretable model, including logistic regression, decision trees, boosted trees, or random forests, can serve as the surrogate.The surrogate is trained on simplified data and corresponding ground truths.

Casual inference and contrastive explanations

The paper distinguishes association-based interpretability from causal and counterfactual approaches that address why questions and alternative outcomes.

  • Casual inference and contrastive explanations: Statistical ML models map features to targets through association rather than establishing that features cause outcomes.A correlation between x and y does not by itself mean that x causes y.
  • Casual inference and contrastive explanations: Causal interpretability addresses “what-if” and “why” questions, whereas statistical interpretability primarily reveals associations.Proposed strategies include oracle models for causal effects and interpretable approximations of those models.
  • Casual inference and contrastive explanations: Causally interpretable models are described as often necessary for ensuring fairness.The paper links causal reasoning with assessing whether features caused a model decision.
  • Casual inference and contrastive explanations: Decision rules let users focus on learned knowledge rather than underlying data representations.Counterfactual explanations additionally ask how a prediction would change if an input were different.
  • Casual inference and contrastive explanations: LORE learns a local interpretable model from a genetic-algorithm-derived neighbourhood and produces rules and counterfactuals.Partial dependence plots depict marginal feature effects by measuring prediction changes after perturbations.

Knowledge-based approaches

Knowledge-based approaches augment ML explanations with structured biomedical knowledge, enabling reasoning over facts and rules for domain-oriented interpretation.

  • Knowledge-based approaches: Interactive human-AI interfaces can help experts evaluate explanation quality and incorporate external knowledge into understanding model results.The paper presents explanation as a bridge for retracing AI results and integrating human expertise.
  • Knowledge-based approaches: Data-trained ML models may predict accurately without incorporating structured domain knowledge or human-like reasoning, abstractions, and connections.Adding expert knowledge or metadata is presented as a way to give black-box models deeper understanding of statistically significant features.
  • Knowledge-based approaches: Black-box feature importance cannot establish whether identified genes or proteins are biologically relevant, limiting trust in purely statistical explanations.The paper notes that important predictive features may not all be biologically meaningful.
  • Knowledge-based approaches: Knowledge graphs integrate biomedical facts through named-entity recognition, entity linking, and relation extraction.These steps connect extracted entities with concepts in a domain-specific knowledge base.
  • Knowledge-based approaches: Domain and human knowledge can support explanations of counterintuitive predictions through machine-readable sources such as ontologies.Knowledge-graph facts can provide biomarker relevance for specific cancer types and support clinician interpretation.
  • Knowledge-based approaches: Inference rules derive implicit knowledge from existing graph facts, such as inferring that TP53 is responsible for cancer from oncogene and cancer relations.The rule structure is expressed as IF-THEN consequences over graph patterns.

Measure of interpretability and explainability

Interpretability is evaluated through both quantifiable model-family criteria and qualitative application-based judgments, but the field lacks consensus on how to benchmark explanations. The paper discusses rationale-based, fairness, surrogate-fidelity, and usability measures.

  • Interpretability evaluation lacks consensus on how to define and benchmark qualitative or quantitative explanations.
  • One evaluation category optimizes within model families that domain users designate as interpretable, such as linear models and rule lists.
  • A second category evaluates explanations qualitatively in application contexts, requiring metrics to assess explanation quality.
  • Surrogate explanation quality depends on predictive fidelity, with R2 near 1 indicating close approximation and R2 near 0 indicating failure to approximate the black-box model.
  • Rationale metrics distinguish whether extracted features are needed for prediction and whether they contain sufficient predictive signal.
  • System causability scale uses ten usability questionnaires and a 5-point scale to assess whether an explanation process suits its intended purpose.

Interpretable ML Tools and Libraries

XAI libraries primarily target general-purpose computer vision, text, and structured-data problems, so bioinformatics applications often require domain-specific customization. The paper illustrates adaptations for sequences, imaging, biomedical text, and model formats.

  • Most XAI libraries address general-purpose problems using methods such as LIME, MUSE, SHAP, partial dependence plots, and individual conditional expectation.
  • Direct application of general-purpose tools to bioinformatics is challenging because they are not specialized for biological data types by default.
  • LIME must be extended for time-series classification, illustrating the need to adapt perturbation strategies to specialized inputs.
  • PoSHAP adapts SHAP for biological sequences by incorporating input positions into model interpretations.
  • Bioimaging requires modality-specific preprocessing, while biomedical language models require domain-specific fine-tuning on resources such as PubMed abstracts.
  • Biomedical text explanations can highlight important tokens using layer-wise relevance propagation and feature attributions.
  • XAI applications may require converting trained models or combining multiple customized tools because one tool cannot generate every explanation type.

Conclusions

The paper presents interpretability as a way to generate insight into model predictions and reviews explanation methods, tools, and applications across bioimaging, genomics, and biomedical text. It also emphasizes that interpretability alone does not guarantee trustworthy or safe AI.

  • The paper surveys interpretable methods, tools, and explanation formats across bioimaging, genomics, and biomedical text.
  • Interpretability can help users understand why and how models make predictions, including through rules, plots, heatmaps, textual explanations, and knowledge-based explanations.
  • Interpretability alone cannot guarantee an AI system’s trustworthiness, so practical benefits still require validation.
  • Decision-makers should select data modalities, explanation types, and local or global interpretability according to intended users and model behavior.
  • Weakness analysis remains difficult for black-box and interpretable models and requires close monitoring of individual data points.
  • Explanations may expose commercially sensitive information and model internals, requiring limits on disclosed detail and data-protection risk assessment.
  • AI systems must address adversarial vulnerability, bias against underrepresented groups, and inadequate individual-data protection while maintaining prediction consistency.
  • Important biological factors should be validated clinically and against domain knowledge, with experts such as oncologists contributing evidence.

Key points

The paper frames XAI design as a set of choices about modeling strategy, explanation scope, interaction, and domain collaboration. It also surveys a broad vocabulary of interpretable ML and XAI methods.

  • Key points: Interpretable models offer simplicity, whereas surrogate explanations of black-box models can be inaccurate without proper evaluation.
  • Key points: A combined pipeline can build a black-box model first, then incorporate interpretable ML logic to explain its decisions.
  • Key points: Clinical scenarios may require different explanation types, including everyday, trace-based, evidence-based, and counterfactual explanations.
  • Key points: Human operators should be able to ask why, how, and what-if questions and receive clear explanations in diverse formats.
  • Key points: The paper’s method vocabulary spans attention, counterfactual, relevance-propagation, rule-based, causal, and deep-learning approaches.
Loading 2212.13261v3…