PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey

Thomas Rojat, Raphaël Puget, David Filliat, Javier Del Ser, Rodolphe Gelin, Natalia Díaz-Rodríguez

arXiv:2104.00950v1cs.LGcs.AI

TL;DR

State-of-the-art time-series models are often deep and difficult to interpret, while explainability research for time series remains less developed than in vision and NLP. This survey reviews existing XAI methods, their evaluation, and their intended roles in confidence and trust. It concludes that explanations can provide model insight, but additional metrics and practices are needed to address robustness, stability, and confidence.

  • Problem

    Most state-of-the-art time-series methods are highly complex and difficult to interpret, despite time-series applications such as medicine and autonomous driving being critical.

  • Method

    The paper surveys XAI methods for time-series models by their methodology, scope, targets, and qualitative and quantitative evaluation approaches.

  • Results

    The survey finds that many methods identify input regions receiving model attention, but do not themselves provide confidence or mitigate model vulnerabilities.

  • Takeaways & Limitations

    XAI insights can increase epistemic confidence and may support new metrics and training practices for confidence and robustness.

Abstract

from arXiv · show

Most of state of the art methods applied on time series consist of deep learning methods that are too complex to be interpreted. This lack of interpretability is a major drawback, as several applications in the real world are critical tasks, such as the medical field or the autonomous driving field. The explainability of models applied on time series has not gather much attention compared to the computer vision or the natural language processing fields. In this paper, we present an overview of existing explainable AI (XAI) methods applied on time series and illustrate the type of explanations they produce. We also provide a reflection on the impact of these explanation methods to provide confidence and trust in the AI systems.

I. INTRODUCTION

Time-series deep learning improves modeling accuracy but remains difficult to interpret, motivating a survey of XAI methods, evaluation approaches, and their relationship to trust, confidence, stability, and robustness.

  • Motivation: Deep learning methods are state of the art for time-series tasks, but their high complexity limits interpretability.The survey highlights classification, forecasting, and clustering as major tasks.
  • Motivation: Time-series explainability remains less developed than explainability in computer vision and natural language processing.The paper associates this gap partly with the unintuitive nature of temporal signals for human observers.
  • Survey scope: The survey critically examines the state of the art in explainability for models learned from time-series data.It organizes the discussion around methods, scopes, targets, evaluation, limitations, and future research.
  • Contributions: The survey reviews XAI methods by methodology, scope, and targets, including approaches intended to increase model confidence, stability, and robustness.It also covers qualitative and quantitative evaluation of explanations.
  • XAI concepts: Explainability methods are framed as tools for making abstract models understandable in critical applications where uninterpretable systems cannot be trusted.The paper distinguishes explainability, interpretability, trustworthiness, interactivity, stability, robustness, reproducibility, and confidence.
  • Open challenges: The survey identifies unresolved concerns about whether explanations provide stable, robust, and confident behavior under perturbations.It notes that many methods offer model insights while neglecting final-user interaction and cannot by themselves assert these properties.

A. Stability, robustness and confidence of systems

Stability, robustness, and confidence are important for evaluating AI systems in critical applications because accuracy alone does not ensure reliable behavior under perturbations or noise.

  • Automated systems can cause severe harm when incorrect decisions occur in critical settings such as autonomous driving.
  • Accuracy measures correct classifications but does not establish that outputs remain unchanged under perturbations or added noise.
  • High accuracy therefore does not ensure good system behavior when real-world disturbances may occur.
  • Stability, robustness, and confidence provide additional ways to assess AI-system behavior beyond accuracy.

1) Stability:

The paper distinguishes ordinary perturbations, adversarial attacks, and counterfactual changes, while emphasizing that no training procedure can guarantee stability and robustness against every disturbance.

  • 1) Stability:: A stable model should correctly classify an unusual perturbed input or indicate uncertainty when the situation is unfamiliar.
  • 2) Robustness:: Robustness concerns whether a model can withstand adversarial attacks that intentionally modify inputs to change its output.
  • 2) Robustness:: Counterfactuals are plausible input changes designed to produce a predefined prediction, unlike imperceptible adversarial perturbations.
  • No model can be guaranteed to remain stable and robust against every possible perturbation or noise.
  • A confidence score and threshold can flag inputs far from the validation distribution when the model should abstain from deciding.

3) Confidence:

The paper frames confidence and trust as related but distinct concerns: technical uncertainty can be assessed, while user trust also depends on understandable feedback and human-centered interaction.

  • 3) Confidence:: Epistemic uncertainty arises from limited data or knowledge, especially when inputs lie far from the training distribution.
  • 3) Confidence:: The paper focuses on epistemic uncertainty and proposes that XAI can reduce it to provide confidence in model outcomes.
  • 3) Confidence:: Explanations can support trustworthiness by helping users understand model behavior and predictions.
  • 3) Confidence:: User interaction and feedback are important when people disagree with an AI prediction or need to handle uncertainty and risk.
  • 3) Confidence:: Most time-series explanations target developers and overlook human factors such as the psychology of drivers.

III. XAI TECHNIQUES FOR TIME SERIES

The survey organizes XAI techniques for time series by model and explanation strategy, covering post-hoc CNN methods and ante-hoc RNN methods. CNN explanations include backpropagation and perturbation approaches, with Gradient*Input attributing predictions to input subsequences.

  • III. XAI TECHNIQUES FOR TIME SERIES: Post-hoc methods explain predictions by approximating model behavior through relationships between feature values and outputs.
  • III. XAI TECHNIQUES FOR TIME SERIES: The survey divides CNN-oriented post-hoc methods into backpropagation-based and perturbation-based approaches, while also discussing ante-hoc methods for RNNs.
  • 1) Backpropagation-based methods:: Backpropagation methods use a single forward and backward pass, and many image-based methods transfer to time-series deep learning models.
  • 1) Backpropagation-based methods:: CAM highlights time-series subsequences that are most representative of a class, but requires a global average pooling layer after convolutional layers.
  • 1) Backpropagation-based methods:: Gradient*Input multiplies a layer's input gradient by the input to identify subsequences processed by highly activated filters as influential for a prediction.
  • 1) Backpropagation-based methods:: Gradient*Input applies to classification and regression because it requires only neuron activations, and extracted subsequences can be clustered by similar patterns.

2) Perturbation-based methods:

Perturbation-based methods explain time-series predictions by altering input subsequences and measuring the resulting output change. The section also contrasts these methods with attention mechanisms for recurrent models.

  • Perturbation-based methods: Perturbation-based methods remove, mask, or alter input features, then compare the new model output with the original to estimate contribution.Larger output differences indicate greater contribution from the altered subsequence.
  • Perturbation-based methods: ConvTimeNet applies occlusion sensitivity by masking time-series regions and measuring the probability difference for the predicted class.Its experiments cover 85 UCR datasets across seven categories.
  • Perturbation-based methods: Tonekaboni et al. define observation importance as the model-output change caused by replacing an observation with a generated value.They apply this approach to mortality prediction using the MIMIC ICU time-series dataset.
  • Attention mechanisms: Attention mechanisms assign importance values to time-series parts and support classification or forecasting with recurrent models.They can be global or local, and temporal or spatio-temporal.
  • Attention mechanisms: Attention mechanisms are ante-hoc explanations embedded in recurrent networks, unlike post-hoc explanations developed separately for convolutional networks.Attention explanations are available directly after learning.

C. Data mining based XAI models

Data-mining approaches provide time-series-specific explainability through symbolic representations, fuzzy reasoning, and representative examples. These methods emphasize interpretable features or subsequences, while shapelets involve efficiency and computational trade-offs.

  • Symbolic Aggregate Approximation: SAX transforms time series into strings by first computing piece-wise aggregate approximations over equal-sized segments.The representation supports interpretable high-level features for classification.
  • Symbolic Aggregate Approximation: SAX-based classifiers select high-level features for each representation, offering interpretability and variable-length input handling compared with deep learning approaches.Experiments use datasets from the UCR Time Series Classification Archive.
  • Fuzzy logic: Fuzzy logic expresses outputs with degrees between 0 and 1, producing approximate reasoning through linguistic terms closer to natural language.It is used for forecasting, hidden temporal-pattern detection, and combinations with neural networks.
  • Fuzzy logic: Fuzzy-based systems combine data-driven representation with interpretable rules or component interactions for tasks such as diabetes prediction.Examples include fuzzy rule-based systems and fuzzy cognitive maps.
  • Shapelets: Shapelets are subsequences maximally representative of a class and can explain predictions through representative examples.They are identified with thresholds that maximize information gain when splitting subsequences into classes.
  • Shapelets: Shapelets are described as more interpretable, faster, and more accurate than k-nearest neighbours, but training efficiency and interpretability can conflict.Regularization and piece-wise aggregate approximation are used to improve interpretability or reduce computational time.

IV. EXPLANATIONS SCALE

Explanation scope distinguishes local explanations for individual samples from global explanations for datasets or groups. The scope depends on how models process instances and retain shared information.

  • Explanation scope: Local explanations apply to a specific sample, whereas global explanations apply to a set of samples or an entire dataset.This distinction defines the scale at which an explanation is valid.
  • Local explanations: Convolutional-network methods naturally produce local explanations because predictions use sample-specific activations and do not share knowledge across predictions.CAM uses last-layer channel activations that change for each prediction.
  • Local explanations: Perturbation methods produce local relevance by altering an input subsequence and comparing its prediction with the original.ConvTimeNet is given as an example.
  • Recurrent networks: Recurrent-network explanation scope depends on whether internal states are reset after each prediction or retain information across several instances.Resetting states yields local explanations; states spanning several instances yield global explanations.

B. Global explanations

Global explanations summarize model behavior across samples or classes and can be produced by aggregating local explanations or clustering learned representations. Their value depends on the intended audience and purpose.

  • Global explanation methods: Shapelet and SAX methods can search across an entire dataset rather than processing samples individually, so their explanation scope is set by the input time-series size.Their candidate discovery is not limited to one prediction.
  • Global explanation methods: Tsviz combines local saliency with filter clustering to produce global insights from activation patterns.The method highlights important input regions and clusters filters by their activation pattern.
  • Global explanation methods: Global CAM averages explanations across training samples within a class to visualize its main discriminative features.Oviedo et al. generalize CAM to all training samples in a class.
  • Purpose: Global explanations can support trustworthiness and confidence, while local explanations can clarify the reasons for a specific prediction.The survey links explanation scope to different purposes rather than treating one scope as universally preferable.
  • Audience: Machine-learning experts and failure-responsible decision makers tend to seek global explanations, whereas model consumers tend to seek local explanations.The distinction follows the different information needs of these audiences.
  • Audience: The survey presents XAI applications intended to provide trustworthiness for developers, decision makers, and users.Applications include medical uses and model-debugging insights.
  • Audience: Most reviewed methods target developers because their technical insights are difficult for non-experts to interpret.This audience pattern follows from the algorithm-focused nature of many methods.

1) Explanations for developers:

Developers generally seek global technical explanations of model behavior, while users and decision makers need more local, situation-specific evidence to judge whether outputs are reasonable. Surveyed methods use feature contributions, prototypes, perturbations, and counterfactuals to study robustness, confidence, and model sensitivity.

  • Developers generally prefer global explanations that reveal how the whole model works, whereas users typically seek local explanations for predictions affecting them.
  • In critical medical applications, explanations can help clinicians check whether the model uses relevant patterns when detecting myocardial infarction.
  • Feature contributions and sensitivity analyses use salient inputs to examine adversarial attacks, noise sensitivity, and features that may cause misclassification.
  • Prototype diversity can focus learning on difficult class-separation regions and improve stability for samples far from the input distribution.
  • Perturbation methods test attribution stability by changing less salient regions or sensitivity by perturbing salient regions, using opposite perturbation strategies.
  • The survey identifies early attempts to use XAI insights to provide epistemic confidence in model outcomes.

VI. EVALUATING EXPLANATIONS

Explanation evaluation in time series combines quantitative approaches with expert assessment, but no globally recognized metric exists. Qualitative judgment is especially constrained because temporal explanations can be unintuitive even for domain experts.

  • No metric is globally recognized for assessing explanation quality, partly because explanations and input data types differ.
  • Quantitative approaches can objectively assess explanations, while domain experts can qualitatively judge their relevance.
  • Methods without built-in evaluation can be assessed by experts, including global attribution maps formed by averaging class activation maps.
  • Tsviz evaluates explanation explicitness by clustering hidden representations and showing their influence on the output.
  • Qualitative evaluation may have limited potential in time series because their unintuitive nature makes explanation quality difficult for domain experts to assess.

B. Quantitative evaluations

Quantitative evaluation tests whether highlighted time-series regions matter, while newer sequence-level methods account for temporal dependencies. The survey concludes that evaluation choice depends on the explanation’s purpose, but current XAI methods remain limited in robustness, time-series specificity, and user interaction.

  • Perturbation evaluation tests whether changing regions identified as relevant causes a large performance decrease in an accurate model.
  • Figure 9 compares Gaussian, inverse, and zero perturbations using perturbed-region ratios and activation changes in channel 55.
  • Swap Time Points and Mean Time Points address perturbation methods’ limited treatment of time-series trends and patterns by accounting for point inter-dependency.
  • Qualitative assessments may suit user- or decision-maker-focused explanations, whereas quantitative assessments may better support discovery of predictive knowledge.
  • Many surveyed methods were adapted from computer vision, and the survey identifies a lack of CNN explainability methods specifically designed for time-series tasks.
  • Most methods identify attended input regions but do not provide confidence or mitigate vulnerabilities, and interpreting their outputs may require expert knowledge.
  • XAI insights may support new metrics and training practices intended to improve confidence and robustness in complex models.
  • End-to-end XAI remains distant because current methods focus on technical components and lack objective robustness tools and user-feedback interactions.
Loading 2104.00950v1…