Explainable AI for clinical and remote health applications: a survey on tabular and time series data

1 Introduction

• XAI supports clinicians, developers, and patients by improving understanding and confidence, but no single approach meets all healthcare users’ needs [2] [3].Clinicians require trust and clinical validation, developers require model reliability, and patients require interpretable, personalised explanations.Source: §1 · p. 2
• Tabular and time series healthcare applications remain less studied than imaging, despite their widespread use in EHRs, remote monitoring, and personalised care [9].EHR tabular data are longitudinal, heterogeneous, and patient-specific, while remote health systems rely heavily on multivariate time series.Source: §1 · pp. 2–3
• Existing XAI methods may overlook feature interactions and mixed predictors in tabular data, or recurrent spatiotemporal patterns, cross-channel correlations, and unintuitive signals in time series [11].Many time series methods are adapted from computer vision and NLP, which can make explanations difficult for even domain experts to interpret.Source: §1 · p. 3
• Effective healthcare explanations require clinical validation, consistency assessment, quantitative quality metrics, and human-centered evaluation that accounts for end-user interaction [15] [16].Functionality-grounded, application-grounded, and human-grounded approaches are complementary, while multidisciplinary collaboration can integrate user needs into explanation design.Source: §1 · p. 3

1.1 Contribution

• The survey covers prominent XAI methods, evaluates their use and explanation assessment in clinical and real-world health data, and discusses methodological limitations and open user-focused challenges.Its structured focus includes clinical validation, cross-model or cross-method consistency, objective quantitative metrics, and user-centered studies.Source: §1.1 · p. 4

2 Background

• XAI methods vary by local versus global scope, ante-hoc versus post-hoc applicability, model dependence, explanation form, algorithmic mechanism, and input data type.Explanation forms include attributions, heatmaps, rules, trees, and dependency plots, while mechanisms include perturbation-, gradient-, and instance-based approaches.Source: §2 · pp. 4–5

2.1 XAI for tabular data

• For tabular data, most XAI techniques are model-agnostic, including ablation and permutation importance, while tree models provide MDI; SHAP aggregates local explanations globally, and LIME uses sparse local surrogates.LIME may become unstable and unfaithful when the underlying model is highly nonlinear locally, while SHAP uses the average prediction over the training/test set as a reference.Source: §2.1 · pp. 5–7
• Tabular explanations also include counterfactual changes, sensitivity analysis, and dependency visualizations, with PDP, ICE, and correlation-aware ALE exposing feature–outcome relationships.Counterfactual methods optimize minimal changes toward a desired prediction, while Morris’ method identifies linear, nonlinear, and interaction effects but can be computationally costly.Source: §2.1 · pp. 7–8

2.2 XAI for time series data

• Time-series XAI primarily targets RNNs and CNNs, using gradient-based attributions, perturbation-based occlusion, CAM, Grad-CAM, DeepLIFT, and LRP.These methods adapt image-derived saliency concepts to highlight influential data points or contiguous sequence regions, while occlusion is architecture-independent and computationally inexpensive.Source: §2.2 · pp. 9–10
• Attention is the primary explainability method for RNNs, while attentive transformers provide feature- and time-level ante-hoc interpretations; SAX and Shapelets offer additional pattern-based alternatives.Transformers can identify globally important variables, persistent temporal patterns, and significant trajectory events, whereas Shapelets extracts class-representative subsequences.Source: §2.2 · pp. 10–11

• The search covered IEEE Xplore, Springer, ACM, Elsevier, and Scopus, combining keywords for explainability, AI, and healthcare; additional relevant Nature Research works were investigated.Source: §3 · p. 12
• The review included papers published from 2017 to 2021, with most studies appearing in the final two years and an exponential increase in healthcare XAI applications.Source: §3 · p. 12, Table 2 · p. 13
• The survey excluded biomedical imaging, NLP, genomics, and molecular biology, retaining tabular data and time series/sequence data such as static EHR, longitudinal EHR, and raw signals.Source: §3 · p. 12
• Eligible methods included post-hoc explanations for black-box models and ante-hoc interpretability embedded during learning, but excluded intrinsically interpretable glass-box models.Source: §3 · pp. 12–13
• The final survey comprised 71 original studies: 46 journal articles (64.8%) and 25 conference papers (35.2%), with distributions also reported by library and journal.Source: §3 · p. 13

4 Results

• Exploratory studies demonstrate XAI integration with complex healthcare models but generally do not evaluate the generated explanations, limiting confidence in clinical decision support.The survey therefore focuses on clinical validation, consistency assessment, and quantitative or qualitative quality assessment as complementary evaluation perspectives.Source: §4 · p. 15

4.1 Clinical validation

• Clinical validation assesses whether tabular explanations reflect established relationships, longitudinal EHR explanations follow clinically plausible trajectories, and physiological-signal explanations highlight meaningful components.Validation uses domain-expert surveys or comparisons with medical literature and may also reveal novel risk-factor relationships for future investigation.Source: §4.1 · pp. 15–16
• SHAP-based studies found clinically consistent predictors for dementia, stroke deterioration, chronic kidney disease, obesity, postoperative complications, liver-transplant AKI, and COVID-19 outcomes.Examples include recent cognitive tests for dementia, a 126 mg/dL fasting-glucose cutoff for stroke deterioration, low-cost CKD feature subsets, and physiologically consistent AKI and COVID-19 risk factors.Source: §4.1 · pp. 16, 17, 18, 21
• For longitudinal EHR, recurrent and attention-based models identified clinically recognized temporal risk patterns, with expert reviews supporting informative features for sepsis, heart-failure survival, AKI, mortality, and heart-failure diagnosis.Local perturbations in DeepAISE caused larger performance drops than global replacement, while TRACER and RetainVis revealed clinically coherent feature rankings and patient trajectories.Source: §4.1 · pp. 19–20
• Multimodal COVID-19 models showed that combining symptom metadata with cough features improved overall performance, while attention and t-SNE analyses supported clinically meaningful symptom and cough-feature relationships.COVID-19 ECG analysis also linked influential HRV features to increased vagal activity, consistent with earlier studies.Source: §4.1 · p. 21

4.2 Explanation consistency assessment

• Explanation consistency compares attribution rankings or decision-rule overlap, mainly for tabular data, whereas time-series explanations commonly rely on saliency and attention methods.Consistency can provide preliminary evidence about stability and robustness, but sensitivity analysis may be needed for stronger conclusions.Source: §4.2 · pp. 23, 25, 26
• Across clinical tabular studies, different models and XAI methods often converged on clinically plausible top features, although local-to-global aggregation and method-specific rankings sometimes disagreed.Examples include shared COVID-19 blood markers [92], diabetes features [96, 97], mortality factors [107], and cancer-survival predictors.Source: §4.2 · pp. 26–29
• Explanation changes can reveal deployment risks: cross-site performance deterioration was associated with feature-importance disparities, while COVID-19-related drift coincided with declining AUC and changing SHAP importance.Periodic explanation monitoring can therefore support transportability assessment and timely model retraining or updating.Source: §4.2 · pp. 29–30

4.3 Explanation quality assessment

• Clinical validation and consistency assessment remain indirect because no globally established metrics or practices enable formal, systematic comparison of explanation quality.The survey recommends objective quantitative evaluation followed by healthcare-professional feedback and domain-expert collaboration in designing clinical explanation tools.Source: §4.3 · pp. 31, 33
• Human-centered studies found that linking explanations to patient context, raw data, and cohort evidence improved clinicians’ interpretation, while physician agreement, trust, and reliance on explanations could be measured through structured evaluation.VBridge interviews, clinician surveys of LIME, and focus groups examining SHAP displays illustrate complementary usability and clinical-relevance assessments [115, 117, 120].Source: §4.3 · pp. 33–34
• Time-series explanation evaluation lacks gold-standard global metrics, motivating annotation-based measures such as Congruence and Annotation Classification and uncertainty-aware measures such as Relevance Accuracy and Relevance Consistency.Guided saliency outperformed alternative methods on expert-annotated PPG data, while an FCN ensemble consistently localized relevant time points across initializations [128].Source: §4.3 · pp. 35–36
• Preliminary attention-quality assessments follow evaluation methods similar to those used for CNN-based saliency maps.Source: §4.3 · p. 36
• LAXCAT combines a CNN encoder with dual attention to learn informative variables and time intervals for multivariate time-series classification and forecasting.It was evaluated on seizure detection and fist movement detection using multichannel EEG signals, with datasets from and Physionet DB, respectively.Source: §4.3 · p. 36
• Approximately 25% was the best global AAM value, although LAXCAT outperformed other attention-based networks in detecting relevant signal time points.AAM measures the average percentage of attention correctly detected across local predictions.Source: §4.3 · p. 37

Static data

• Static-data applications generally favor tree ensembles, whose black-box behavior is explained mainly through post-hoc feature attribution, especially LIME and SHAP.Feature attribution methods are more prevalent than rule-based approaches because they preserve predictive performance while revealing feature importance.Source: §5 · p. 38
• SHAP is preferred to LIME because it offers theoretically grounded, local and global, model-agnostic explanations with feature-interaction analysis.TreeSHAP provides fast and exact explanations for tree-based models, whereas LIME may be unfaithful for highly nonlinear local behavior and unstable when aggregated globally.Source: §5 · p. 38

Time series

• For raw time-series data, RNNs and CNNs perform strongly, with attention typically explaining RNNs and gradient-based methods such as Grad-CAM and LRP explaining CNNs.Attention and saliency methods usually produce local heatmaps over signal components but do not explain how relevant subsequences generate predictions.Source: §5 · pp. 38–39

Evaluation

• Across healthcare applications, evaluations rarely assess clinical relevance, consistency, and explanation quality together, despite quality evaluation needing both method comparison and end-user suitability.Preliminary evaluations are promising, but comprehensive multidimensional assessment remains generally missing, especially for explanation quality.Source: §5 · p. 39

Limitations and challenges

• Consequently, current reference XAI methods may undermine safety and reliability, making their use for understanding models in high-stakes health applications risky.The survey identifies evaluation challenges from user-centered and user-independent perspectives, alongside technical and methodological flaws requiring further research.Source: §5 · p. 39

5.1.1 Clinical Validation

• Clinical validation can detect clinically misaligned explanations, mitigate AI bias, support certification, and reveal emergent biomarkers or risk factors for iterative model improvement.Clinical comparison is also valuable for knowledge discovery and improved data collection, curation, model selection, and training.Source: §5.1.1 · pp. 39–40

5.1.2 Stability and robustness

• XAI must assess stability and robustness because popular feature-importance and counterfactual methods can be non-robust; sensitivity analysis, external validation, and confidence scores can expose vulnerabilities.Consistency comparisons alone are insufficient, while outlier detection can indicate when predictions differ from the validation distribution.Source: §5.1.2 · pp. 40–41

5.1.3 Quality assessment

• Explanation quality lacks standardized evaluation frameworks, requiring objective metrics to be complemented by user-centered studies that assess usefulness, satisfaction, trust, and task performance.Evaluation is complicated by heterogeneous explanations and data, human bias in expert annotations, unintuitive time series, and limited studies of clinicians’ and patients’ needs.Source: §5.1.3 · pp. 41–42

5.1.4 Interactivity

• Effective healthcare XAI needs interactive, human-centered designs aligned with users’ goals, clinical norms, and workflows, including linked data, cohort evidence, context, and WHAT-IF analysis.Only VBridge and RetainVis among reviewed systems included WHAT-IF functionality, while combining multiple explanation formats remains an open problem.Source: §5.1.4 · pp. 42–43

5.1.5 Human-consumable explanations for time series data

• Saliency and attention methods may aid interpretation of clinically familiar signals such as ECGs, but unintuitive time series require data mining or concept-based explanations for human-consumable meaning.Concept methods include T-CAV, ACE, CaCE [152], and ConceptSHAP.Source: §5.1.5 · pp. 43–44

5.1.6 Understanding DNN models

• DNNs learned from time series lack effective global explanations and tools for probing erroneous reasoning, limiting clinicians’ ability to understand latent representations and model decisions.Existing clustering approaches [156] offer preliminary global insights, while Grad-CAM and LRP remain heuristic and locally focused.Source: §5.1.6 · pp. 44–45

5.1.7 Causability

• Because post-hoc explanations often show correlations rather than causes, causability methods such as the System Causability Scale and Causal AI aim to support intervention-oriented decisions.Causal AI could help researchers and clinicians design and simulate interventions using available data, complementing long-term evidence-based medicine.Source: §5.1.7 · p. 45

Trustworthy AI

• Tabular and time series XAI requires clinical relevance, consistency, and quality evaluation to improve human understanding and confidence in healthcare AI.These evaluations are necessary for trustworthy AI and remain frequently missing despite the widespread use of these data types in clinical and remote applications.Source: §6 · pp. 45–46

Ethical implications

• XAI can support ethical and fair healthcare AI by enabling clinical validation, exposing hidden bias, and checking discrimination across patient characteristics such as gender and ethnicity.Explainability can also support accountability, lawfulness, compliance, and the identification of AI decisions, actions, and communications.Source: §6 · p. 46

Patient-centered care

• XAI can strengthen patient-centered care by helping clinicians explain decisions, support shared meaning, resolve human–machine disagreements, and improve patient acceptance, while liability for wrong decisions remains unresolved.Patients are treated as active partners with rights to choose and control their treatments and recommendations.Source: §6 · p. 46

Deployment challenges

• End-to-end XAI systems ready for large-scale deployment with minimal human supervision remain out of reach because technical, social, ethical, methodological, user-centered, and regulatory challenges persist.The survey calls for methodological improvements, user-centered studies, and clear, complete regulations before widespread medical adoption.Source: §6 · p. 46

Limitations and clinical use

• XAI should remain complementary support rather than a replacement for standard medical practice because inherently interpretable models may sacrifice predictive performance, whereas post-hoc explanations approximate model reasoning and are not completely trustworthy.Domain experts must supervise final decisions, and XAI should be carefully coordinated with evidence-based assessment to avoid conflict with established medical practice.Source: §6 · pp. 46–47

Declarations

• The study received partial funding from the European Commission under the H2020-INFRAIA-2019-1 SoBigData-PlusPlus project (Grant No. 871042) and reports no relevant financial or non-financial competing interests.Source: Declarations · p. 47

Appendix A Acronyms

• The appendix defines acronyms spanning healthcare data and conditions, machine-learning models, explainability methods, physiological signals, and evaluation measures.Examples include XAI, EHR, EEG, LSTM, SHAP, LIME, HRV, SpO2, and SCS.Source: Appendix A · pp. 68–69

• Preliminary XAI studies applying to tabular data are summarized in Table B2.Source: Table B2 · p. 71
• The appendix references datasets and resources from NHANES, COVID-19, heart-failure clinical records, and fetal-health classification.Source: Appendix B · p. 71
• Preliminary XAI studies applying to time series data are summarized in Table B3.Source: Table B3 · p. 72