Source-linked AI summary
Towards a Rigorous Evaluation of XAI Methods on Time Series
Udo Schlegel, Hiba Arnout, Mennatallah El-Assady, Daniela Oelke, Daniel A. Keim
TL;DR
XAI methods designed for images and text need evaluation approaches that account for temporal structure in time series. This paper applies those methods to time series, introduces sequence-aware verification techniques, and conducts preliminary experiments across benchmark datasets. The experiments find SHAP robust across models, while DeepLIFT, LRP, and Saliency Maps work better with particular architectures.
Problem
Existing XAI methods often target images or text, while evaluating whether their explanations correctly capture temporal dependencies in time series remains difficult.
Method
The paper applies selected XAI methods to time series and evaluates their relevance explanations using perturbation and two sequence-dependent verification methods.
Results
SHAP works robustly across all tested models, whereas DeepLIFT, LRP, and Saliency Maps work better with specific architectures.
Takeaways & Limitations
The findings support using XAI methods on time series while motivating more suitable methods for better human understanding.
Takeaways & Limitations
Heatmap explanations are hard to interpret and difficult to scale to large datasets or long time series.
Abstract
from arXiv · showhide
Explainable Artificial Intelligence (XAI) methods are typically deployed to explain and debug black-box machine learning models. However, most proposed XAI methods are black-boxes themselves and designed for images. Thus, they rely on visual interpretability to evaluate and prove explanations. In this work, we apply XAI methods previously used in the image and text-domain on time series. We present a methodology to test and evaluate various XAI methods on time series by introducing new verification techniques to incorporate the temporal dimension. We further conduct preliminary experiments to assess the quality of selected XAI method explanations with various verification methods on a range of datasets and inspecting quality metrics on it. We demonstrate that in our initial experiments, SHAP works robust for all models, but others like DeepLIFT, LRP, and Saliency Maps work better with specific architectures.
1. Introduction
XAI methods need evaluation strategies suited to time series because many existing approaches target images or text and rely on visual inspection. The paper introduces a methodology and verification techniques that incorporate temporal structure.
- XAI explainers are methods applied on top of trained complex models to produce explanations of their decision making.The introduction distinguishes the explainer from the underlying model.
- Many prominent XAI methods are tailored to images or text, while temporal dimensions in videos, audio, and sequence data receive limited consideration.Image-oriented explanations often use input heatmaps, but this representation does not directly address temporal dependencies.
- Correct explanations are needed to debug and optimize time series prediction models, but raw time series and explanations are difficult to inspect reliably.Large, difficult-to-interpret data and limited connectable domain knowledge motivate quantifiable verification.
- Existing evaluation methods such as setting relevant pixels to zero omit temporal dependencies by assuming feature independence or only short-term dependence.The paper argues that adapted or novel verification methods are needed for time-oriented data.
- The paper applies XAI methods to time series, evaluates selected methods on real-world benchmark datasets, and introduces two sequence verification methods.The proposed methodology automatically evaluates explanations using verification techniques and preliminary experiments.
2. Time Series Explanations
The paper represents time series explanations as relevance assigned to individual time points and visualizes those relevances as heatmaps. Because heatmaps are difficult to interpret and scale, automated verification is needed.
- Time series classification samples contain ordered time points, which XAI methods treat as features to produce local feature importance.For a sample t, the explanation assigns relevance to each time point.
- A model is trained on labeled data to predict unseen time-series samples, after which an XAI method explains the model’s decisions.The XAI method forms an additional layer on top of the trained model.
- An XAI explanation for a time series is represented as a relevance vector r = (r0, r1, r2, ..., rm) over its m time points.Each time point ti receives a corresponding relevance ri.
- Relevance values can be visualized as heatmaps over line plots, allowing qualitative inspection together with domain knowledge.The paper illustrates this representation with time-series relevance heatmaps.
- Heatmaps are hard to interpret and difficult to scale to large datasets or long time series, motivating automated verification.Visual inspection alone is therefore insufficient for broad evaluation.
3. Evaluating Time Series Explanations
The paper evaluates time-series explanations by perturbing relevant points and by applying sequence-dependent transformations that preserve the temporal setting. These methods compare model quality on original, randomly changed, and relevance-based changed data.
- 3. Evaluating Time Series Explanations: Time-series explanation evaluation requires specialized heuristics because replacing values with zero can itself indicate an anomaly.The paper therefore introduces two verification methods explicitly designed to account for sequence properties.
- 3.1. Perturbation on time series: Perturbation analysis changes time points whose relevance exceeds a threshold and compares the resulting quality metric with random-relevance changes.Changed values may be set to zero or transformed by the inverse max-based operation.
- 3.1. Perturbation on time series: The perturbation baseline assumes that changing relevant time points should substantially reduce an accurate model’s quality, whereas random changes should stagnate or reduce it less.The same number of points is changed for the relevance-based and random procedures.
- 3.2. Sequence Evaluation: Sequence-dependent evaluation tests whether explanations capture features such as slopes or minima, which pointwise perturbations may fail to evaluate.The paper motivates this extension when a model can generalize despite a value replacement that disrupts a local pattern.
- 3.2. Sequence Evaluation: The proposed sequence methods begin from highly relevant points and modify the associated subsequences rather than treating points independently.One method swaps time points, while another replaces a subsequence with its mean.
- 3.3. Methodology: The evaluation pipeline trains and evaluates a model, creates explanations for test samples, modifies the test data, and compares quality measures on the newly created sets.The methodology links explanation generation to subsequent model-quality comparisons.
4. Discussion
The preliminary experiment evaluates five XAI methods across ten real-world time-series datasets using CNN, RNN, and, when available, dataset-specific architectures. Results vary by architecture, with SHAP strongest in ResNet settings and LIME weakest overall.
- 4.1. Datasets & Models: Ten datasets cover binary and multi-class time-series classification, including nine UCR datasets and one ECG heartbeat dataset.
- 4.1. Datasets & Models: CNN and RNN architectures serve as baseline models, with dataset-provided architectures added when available.
- 4.2. XAI Methods: Five prominent XAI methods are evaluated: LIME, LRP, DeepLIFT, Saliency Maps, and SHAP.
- 4.3. Results: DeepLIFT and LRP show the largest overall quality-metric decreases for CNNs in perturbation and sequence analyses.
- 4.3. Results: Saliency Maps and SHAP outperform the other methods in RNNs, while SHAP produces the best results in more advanced ResNet architectures.
- 4.3. Results: LIME performs poorly across cases, most likely because its surrogate linear classifier handles the time-series dimensionality poorly.
5. Conclusion and Future Work
The methodology shows that image- and text-derived XAI methods can assign relevance to time points in time-series data, but their effectiveness depends on model architecture. SHAP is robust across models, while the study calls for methods and visual representations better suited to temporal data.
- Image- and text-derived XAI methods work on time series by specifying relevance for individual time points.
- SHAP works robustly across all evaluated models, whereas DeepLIFT, LRP, and Saliency Maps work better with specific architectures.
- LIME performs worst, most likely because converting time points into features creates large dimensionality for its approach.
- The study identifies a need for more suitable time-series XAI methods and more abstract visual representations than difficult-to-interpret saliency heatmaps.