Source-linked AI summary
Software Frameworks for Explainable AI in Time Series Classification: A Systematic Review
Louis Peter, Nils Gumpfer, Jana Fischer, Christin Seifert, Jennifer Hannig
TL;DR
Time-series classification is increasingly used in decision-critical settings, but XAI research for it remains fragmented and insufficiently evaluated through software frameworks. This survey systematically reviews and compares such frameworks across methods, metrics, usability, benchmarking, and reproducibility. It identifies six frameworks and finds limited frequency-domain support, few time-series-specific metrics, and substantial explanation differences across implementations.
Problem
XAI for TSC lacks a systematic framework-level understanding of explanation generation, evaluation, practical limitations, and reproducibility.
Method
The survey searches GitHub and systematically examines six XAI frameworks across methods, metrics, usability, benchmarking, signal representations, and cross-framework reproducibility.
Results
Six frameworks explicitly support time series; only two of 52 evaluation metrics are time-series-specific, while framework implementations can produce substantially different explanations for identical methods.
Takeaways & Limitations
The findings support cautious comparison and reproduction of TSC explanations and motivate unified frameworks for faithful, reproducible, time-series-aware XAI.
Takeaways & Limitations
The review may miss software outside GitHub, and reproducibility analysis covered only five explanation methods and one shared evaluation metric.
Abstract
from arXiv · showhide
Time series arise in a wide range of application domains and are analyzed using machine learning in decision-critical settings. Time series classification (TSC) is one of the most widely studied and relevant tasks. In this context, ensuring the transparency and trustworthiness of TSC models has become an important requirement, motivating the use of explainable artificial intelligence (XAI) methods. Despite growing interest, research on XAI for TSC remains fragmented, and a systematic understanding of the available software frameworks for explanation generation, their evaluation practices, and practical limitations is still lacking. Prior work largely focused on individual explanation methods, while cross-framework consistency, time-series-specific evaluation, and reproducibility have received little attention. In this survey, we analyze existing software frameworks for explanation generation and evaluation in TSC. We compare them along multiple dimensions, including supported XAI methods, evaluation metrics, usability, benchmarking support, and reproducibility, providing the first time-series-specific survey of frameworks with implementation comparisons and an analysis of frequency-domain support. We identify six frameworks that explicitly support time series and reveal common limitations: only one method supports frequency-domain explanations despite their relevance; only two evaluation metrics have been developed specifically for time series; and identical XAI methods can yield substantially different explanations across frameworks. Based on these findings, we discuss open challenges and outline directions for future research, highlighting the need for unified, time-series-specific XAI frameworks that enable faithful, reproducible, and time-series-aware explanations.
1 Introduction
XAI for time series classification requires approaches that account for temporal dynamics, cross-channel dependencies, and alternative signal representations. This survey addresses a framework-level gap by systematically comparing software support for generating, evaluating, and benchmarking explanations.
- Motivation: Deep learning improves time series classification performance but can reduce interpretability and transparency in decision-critical applications.Understanding automated decisions is important for user trust and regulatory human-oversight requirements.
- Challenges: Many time-series explanation methods were developed for images and applied without adaptation or systematic evaluation, leaving faithfulness and usefulness unclear.Time-series explanations can therefore risk misleading interpretations.
- Challenges: Time-series explanations must account for complex temporal dynamics, cross-channel dependencies, and patterns in time, frequency, or time-frequency domains.These properties distinguish TSC explainability from image-based explanations.
- Research gap: Prior surveys emphasized individual XAI methods or cross-domain frameworks, while framework implementation, evaluation, benchmarking, and reproducibility for TSC remained insufficiently studied.Earlier work reported that only one evaluation framework explicitly supported time series as of 2022.
- Contribution: This survey provides the first systematic, framework-centric comparison of XAI software for TSC, including usability, supported methods, evaluation, benchmarking, and practical limitations.Its research questions also examine univariate and multivariate support, alternative signal representations, and implementation-dependent explanations.
2 Methodology
The authors systematically reviewed open-source XAI frameworks for TSC by searching GitHub, filtering repositories, and examining implementations across standardized dimensions. The resulting analysis covered framework usability, methods, metrics, datasets, representations, and dependencies.
- Search and screening: The review searched GitHub with five XAI and time-series queries, producing 750 entries before repository screening.The search was conducted on December 5, 2025.
- Search and screening: After duplicate removal and activity filters, manual screening identified six candidate repositories, while backward dependency searches added no eligible frameworks.Dependencies were excluded when they met at least one exclusion criterion.
- Review scope: The final review included six XAI frameworks and examined source code, publications, documentation, supported datasets, and descriptions of implemented methods and metrics.The analysis covered framework characteristics, XAI methods, evaluation metrics, benchmarking, and cross-framework reproducibility.
- Framework analysis: Framework assessment recorded method and metric counts, machine-learning backends, package availability, repository metadata, and usability across maintenance, community interaction, and documentation.Usability scores followed the three dimensions proposed by Le et al.
- Method analysis: XAI methods were categorized by approach and assessed for time-series specificity, univariate or multivariate applicability, explanation domain, and cross-framework reproducibility.Explanation domains included time, frequency, and time-frequency representations.
- Evaluation and benchmarking: Evaluation metrics were classified by time-series compatibility, time-series specificity, explanation domain, and perturbation-based or ground-truth-based design.Benchmarking analysis examined accessible datasets, ground-truth information, and univariate or multivariate series.
3 Results
The review characterizes six XAI frameworks for time series classification across methods, metrics, datasets, usability, and reproducibility. It finds broad generic support but limited time-series-specific functionality and inconsistent explanations across implementations.
- Framework overview: Six frameworks were analyzed for XAI method support, evaluation metrics, benchmarking capabilities, usability, and cross-framework reproducibility.The frameworks were examined across the review’s five result dimensions.
- Framework overview: Three frameworks support both explanation generation and evaluation, while three focus exclusively on XAI methods.The evaluation-capable frameworks are time_interpret, Quantus, and XTSC-Bench; TSInterpret, tsCaptum, and SIGN-XAI-2 focus on methods.
- XAI methods: The frameworks contain 51 XAI methods, but only 16 explicitly state that they were developed for time series.Most time-series-specific methods target multivariate data, while three are restricted to univariate time series.
- XAI methods: Only one method generates explanations in the frequency and time-frequency domains, while DFT-LRP supports all three domains for univariate data.The AudioMNIST example contrasts scattered time-domain relevance with frequency-domain patterns distinguishing female and male pronunciations.
- Evaluation metrics: The frameworks provide 52 evaluation metrics, but only two were specifically developed for time series and none target frequency or time-frequency explanations.The two time-series-specific metrics, Mask Information and Mask Entropy, evaluate relevance on time-series subsequences against ground-truth masks.
- Benchmarking capabilities: Three frameworks support datasets for comparative explanation evaluation, with synthetic datasets enabling benchmarking through ground-truth information.XTSC-Bench and time_interpret provide synthetic datasets with ground truth, while clinical datasets lack ground-truth information.
- Reproducibility: Cross-framework comparisons showed that identical XAI methods can produce different explanations, whereas the shared ROC-AUC metric produced consistent values.The comparison used five methods on ECG records and found implementation differences for Integrated Gradients, while other investigated methods matched when parameters matched.
4 Open Challenges & Further Research
Although more frameworks now claim time-series support, the survey finds persistent gaps in time-series-specific methods, evaluation, signal representations, benchmarking, dependency explanations, and reproducibility.
- Five additional frameworks now claim time-series support compared with 2022, when only one framework did, but explicit support for time-series properties remains limited.
- Method Applicability: Only a small fraction of supported XAI methods was explicitly developed for time series, while many frameworks reuse generic methods without explicit validation of temporal or cross-channel properties.
- Cross-Channel Dependencies: Gradient-based methods are efficient for high-dimensional inputs but cannot explain cross-channel dependencies, whereas perturbation-based methods can at increased computational cost.
- Frequency and Time-Frequency Domains: Only SIGN-XAI-2 among the analyzed frameworks implements a frequency-aware method, while DFT-LRP generates explanations across time, frequency, and time-frequency domains.
- Evaluation Metrics: Only two of 52 analyzed evaluation metrics were developed specifically for time series, and no metric considers frequency-domain representations.
- Benchmarking: Only half of the frameworks support benchmarking datasets, and only two include ground-truth information, all based on synthetic or predefined distributions.
- Reproducibility: Explanations can differ substantially across frameworks, but the reproducibility analysis covered only five shared explanation methods and one shared evaluation metric.
- Transferability: The identified limitations may also affect forecasting and anomaly detection, which remain less studied than time series classification.
5 Conclusion
The survey concludes that current XAI frameworks for TSC remain limited in time-series awareness, benchmarking, and reproducibility. It advocates shared infrastructure to support faithful, reproducible, and comparable explanations.
- The survey identifies limited time-series-specific methods and metrics, minimal frequency and time-frequency support, insufficient real-world ground truth, and missing cross-channel explanations.
- Different framework implementations can yield substantially different results, so researchers should exercise caution when applying or comparing existing TSC XAI methods and metrics.
- A standardized, community-driven initiative could support faithful, reproducible, time-series-aware explanations and systematic comparison and evaluation of XAI methods.
Appendix A
Appendix A defines how the framework comparison tables encode XAI-method compatibility, dimensionality, frequency-domain support, implementation status, and evaluation-metric compatibility.
- XAI methods: Table A.1 records whether each XAI method is explicitly time-series-specific and whether it supports univariate, multivariate, or both types of time series.The dimensionality codes are U, M, and B, respectively.
- XAI methods: Table A.1 marks frequency-domain support when a method generates explanations for frequency or time-frequency representations.
- Framework coverage: Table A.1 distinguishes direct framework implementations from wrapped or reused implementations and unsupported methods.The symbols are ✓, (✓), and ✗, respectively.
- Evaluation metrics: Table A.2 organizes evaluation metrics by time-series compatibility and records whether Quantus, time interpret, and XTSC-Bench implement, reuse, or omit each metric.Compatibility is coded as ✓ for time-series-specific metrics, C for time-series-compatible metrics, and ✗ otherwise.
- Evaluation metrics: The appendix notes that two metrics share the name Sufficiency and are distinguished by their original publications.