Counterfactual Explanations for Machine Learning on Multivariate Time Series Data

• Black-box ML frameworks can be difficult to understand and debug, degrading user trust and system usefulness.Source: §I · p. 1
• Sample- and feature-based explanations can leave users to compare hundreds of metrics or interpret unfamiliar feature meanings; the paper demonstrates this inadequacy for HPC time series.The illustrated Voltrino sample covers a 45-second window from one node.Source: §I · p. 2, Figure 1 · p. 2
• Counterfactual explanations show which time series to replace in an individual sample to obtain a different classification.The method aims to use only a small number of substitutions so users can inspect the changed metrics.Source: §I · p. 2
• The paper formalizes multivariate time-series explainability, proves the problem NP-hard, and designs a heuristic algorithm.Source: §I · p. 2
• The proposed method is demonstrated on several ML frameworks and four data sets.Source: §I · p. 2
• The method performs better than baselines in faithfulness and robustness across the evaluated time-series data sets.The evaluation includes HPC telemetry and motion classification data.Source: §I · p. 2

II. RELATED WORK

• Prior work spans interpretable models, sample- and feature-based explanations, and time-series methods, but leaves high-dimensional multivariate counterfactual explanations unaddressed.Source: §II · pp. 2–3

III. COUNTERFACTUAL TIME SERIES EXPLANATION PROBLEM

• For a test sample and target class, the method substitutes a small number of whole time series from a training-set distractor until the black-box classifier predicts that class.The classifier is assumed to expose no internal weights or gradients.Source: §III · p. 3

A. Problem Statement

• The optimization seeks a modified sample that maximizes target-class probability while minimizing substitutions; the problem is NP-hard, motivating practical heuristics.The substitution mask selects which metrics are replaced.Source: A. Problem Statement · p. 3

B. Rationale for Chosen Explanation

• The method pairs a realistic counterfactual sample with the small set of time series that must change, avoiding the need to interpret an entire high-dimensional sample.It also gives users the test and distractor values for those metrics as comparison points.Source: B. Rationale for Chosen Explanation · p. 3
• Synthetic time-series modifications can be infeasible because HPC telemetry values and rates of change are bounded by physical constraints.Source: B. Rationale for Chosen Explanation · p. 4
• Selecting the distractor from the training set yields a feasible, realistic sample from the same system and lets administrators inspect its associated logs and other information.Source: B. Rationale for Chosen Explanation · p. 4

IV. OUR METHOD: COUNTERFACTUAL EXPLANATIONS

• The method seeks the fewest substitutions from a distractor that maximize the target-class probability for the test sample.Source: §IV · p. 4
• The loss uses a target probability of τ = 0.95 and a desired explanation size of δ = 3; ReLU avoids penalizing explanations shorter than δ.Source: §IV · p. 4

A. Choosing Distractors

• For each distractor, the method finds a substitution matrix and returns the solution with the lowest loss, using nearest correctly classified training samples from the target class as candidates.Euclidean distance and a per-class KD-tree support neighbor queries; users set how many distractors to try.Source: A. Choosing Distractors · p. 4

B. Sequential Greedy Approach

• Sequential greedy search repeatedly substitutes the metric that most increases target-class probability, stopping when that probability reaches the predefined threshold τ.Source: B. Sequential Greedy Approach · p. 4

C. Random-Restart Hill Climbing

• Random-restart hill climbing minimizes the loss by iteratively accepting better neighboring substitution matrices, avoiding a search tree but potentially settling in local minima.When it finds no viable matrix, the method prunes non-impactful metrics and uses greedy search if none remain.Source: C. Random-Restart Hill Climbing · p. 5

D. How to Measure Good Explanations

• Faithfulness means an explanation reflects the classifier’s actual reasoning; the method tests it on a simple model with known reasoning using precision and recall.Source: D. How to Measure Good Explanations · p. 5
• Comprehensibility is evaluated by comparing how many time series different methods include, since operators may struggle to interpret many metrics.The text notes that people prefer one or two causes over a full list.Source: D. How to Measure Good Explanations · p. 5
• Robustness concerns whether similar samples receive similar explanations, while allowing explanations to reflect drastic changes in model behavior.The local Lipschitz constant measures explanation change relative to sample change; the paper uses nearest neighbors and a maximum to assess worst-case robustness.Source: D. How to Measure Good Explanations · p. 5
• Generalizability is tested by applying the same metric substitutions from a distractor to other samples and checking whether they also flip the prediction.Source: D. How to Measure Good Explanations · p. 6

V. EXPERIMENTAL SETUP

• The evaluation covers three HPC telemetry datasets and the NATOPS motion-classification dataset.Source: A. Data Sets · p. 6

A. Data Sets

• HPAS supplies synthetic performance-anomaly telemetry from HPC applications; its dataset has 839 time series per sample and 617 samples split into 350 training and 267 test samples.The samples are single-node application runs segmented into 45-second windows with 30-second overlaps.Source: A. Data Sets · p. 6
• Cori provides application-classification telemetry from a 12,076-node supercomputer, with 9,216 single-node samples and 819 time series per sample.The dataset is split into 7,373 training and 1,843 test samples.Source: A. Data Sets · p. 6
• Taxonomist contains 4,728 samples from 11 applications, each with 563 time series, split into 3,776 training and 952 test samples.Source: A. Data Sets · p. 6
• NATOPS contains 24 Kinect-derived time series for six repeated hand-and-arm motions, with 180 training and 180 test samples.The samples use 50-second time windows.Source: A. Data Sets · p. 6

B. Machine Learning Techniques

• The evaluation covers three pipelines representing different analytics frameworks: feature extraction plus random forest, autoencoder, and feature extraction plus logistic regression.The random-forest pipeline represents uses such as failure prediction and performance-variability diagnosis,,,,.Source: B. Machine Learning Techniques · pp. 6–7
• The random-forest pipeline trains on 11 statistical features extracted from each time series, while its HPAS model contains 100 trees and over 50k nodes.Operators would need to inspect decision paths across trees to understand predictions without explainability tools.Source: B. Machine Learning Techniques · pp. 6–7
• The autoencoder learns a compressed representation from healthy telemetry and classifies runtime data as anomalous when its reconstruction error is high.The implementation converts error to class probabilities by subtracting a chosen threshold and applying the sigmoid function; the architecture follows Borghesi et al.,.Source: B. Machine Learning Techniques · p. 7
• Logistic regression is inherently interpretable and serves as a sanity-check pipeline when experiments require ground-truth explanations.Because the classifier learns a weight vector, features with zero weights have no classification effect, and absolute weight values rank feature impact.Source: B. Machine Learning Techniques · p. 7

C. Baseline Methods

• The evaluation compares the proposed method with LIME, SHAP, and a baseline that selects a random subset of metrics.The evaluation examines minimality, faithfulness, robustness to small input perturbations, generalizability, and usefulness.Source: C. Baseline Methods · p. 7, §VI · pp. 7–8

1) LIME:

• LIME fits a local interpretable linear model to classifier predictions on randomly generated samples weighted by distance to the test sample.For time series, the evaluation applies LIME to feature-extraction frameworks; generating representative samples that respect temporal and metric constraints is challenging, and users must choose the number of explanation features.Source: 1) LIME · p. 7

C. Baseline Methods

• The proposed method’s Fig. 2 explanation shows two metrics for changing a correctly classified “memleak” time window to “healthy.”The second metric, successful huge-page allocations, may indicate that the dataset lacks runs with high memory fragmentation.Source: Figure 2 · p. 7

2) SHAP:

• SHAP uses additive feature-importance values whose sum yields the classification; the evaluation uses KernelSHAP because model parameters are unavailable.Like LIME, it is applied to feature-extraction frameworks rather than directly to time series, but it does not require the user to specify the number of features.Source: 2) SHAP · p. 7

VI. EVALUATION

• The evaluation compares explanations using qualitative comparisons and metrics, asking whether they are minimal, faithful, robust, generalizable, and useful.The robustness question concerns whether small input perturbations produce different explanations.Source: §VI · pp. 7–8

A. Qualitative Evaluation

• The counterfactual explanation uses page-allocation rate and successful huge-page allocations to change a “memleak” prediction to “healthy.”The first metric shows many periodic allocations in leak runs; the second may reflect short-lived benchmarks and a training set biased toward systems with less fragmented memory.Source: A. Qualitative Evaluation · p. 8
• SHAP’s Fig. 3 explanation contains 187 features with very similar values, making it difficult to decide how many suffice; its class-independent explanation may emphasize CPU-anomaly differences, not memory-leak characteristics.The 75th percentile of pgalloc_normal was 5,127, but this does not indicate normal values for the metric or whether it was too high or too low.Source: A. Qualitative Evaluation · p. 8
• In the LIME explanation, green features favor memory leak and red features oppose it, while users must interpret percentile, standard-deviation, and threshold values; the class implied by red features is unclear.For example, a 95th percentile of page allocations over 83 is said to make the run likely to be a memory leak.Source: A. Qualitative Evaluation · p. 8
• LIME and SHAP generate random data while assuming feature independence, which can yield synthetic runs that are impossible in practice.The proposed method instead uses the whole time series and does not generate synthetic data.Source: A. Qualitative Evaluation · p. 8

B. Comprehensibility

• The proposed method usually returns fewer than three time series, while challenging cases can reach 10; SHAP returns 187 features in the example and typically hundreds for HPC data.LIME requires a user-selected feature count but gives no guidance for choosing it.Source: B. Comprehensibility · pp. 8–9
• For a classifier with known feature importances, OptimizedSearch, Greedy, and SHAP have perfect precision, while LIME has lower precision on Cori and Taxonomist.Low recall indicates that not every feature is used in every local decision.Source: Figure 5 · p. 9

C. Faithfulness Experiments

• Both proposed methods and SHAP achieve perfect precision, while recall remains below 1 because not every classifier feature affects every decision.LIME has low precision on Cori and Taxonomist, possibly due to randomness in its data sampling.Source: C. Faithfulness Experiments · p. 9

D. Robustness Experiments

• The proposed methods are the most robust overall; on NATOPS, Greedy is more robust than OptimizedSearch because it inspects every metric.Fig. 6 compares normalized Lipschitz constants, with lower values indicating better robustness.Source: Figure 6 · p. 9, D. Robustness Experiments · p. 9

E. Generalizability Experiments

• Explanations for one misprediction apply to over 40% of similarly mispredicted samples on average.For cpuoccupy runs misclassified as cachecopy, every explanation applies to every other sample with the same misclassification.Source: E. Generalizability Experiments · p. 9, Figure 7 · p. 9

F. Investigating Misclassifications

• In a network-anomaly case, all six metrics had to change together to flip the prediction, suggesting the autoencoder learned their parallel behavior.The example is a network anomaly misclassified as healthy; the figure displays two of the six metrics.Source: F. Investigating Misclassifications · p. 10, Figure 8 · p. 10

Contributions

• The paper investigates explainability for multivariate time-series ML, defines the counterfactual problem, and presents a heuristic that generates feasible explanations.It demonstrates the method on several frameworks and compares it with other explainability methods.Source: §VII · p. 10

Minimizing probabilities

• The objective maximizes the target-class probability; for multiclass classification this can differ from minimizing the probabilities of other classes.The authors report that maximizing one class's probability explained both false and true positives in practice.Source: §VII · p. 10

Approximation algorithms

• No bounds on the heuristic algorithm's optimality are provided; approximation algorithms with provable bounds remain an open problem.Such algorithms may yield better explanations in less time.Source: §VII · p. 10

Explainable models

• In a preliminary HPAS experiment, CORELS took over 24 hours to train and classified every time window as healthy, motivating further work on accurate interpretable models for multivariate time series.The experiment varied configuration options.Source: §VII · p. 10

Production systems

• The authors hope to deploy the method on production HPC systems and study how administrators use the frameworks and explanations.They identify user studies as critical for evaluating explainability methods.Source: §VII · p. 10

SERIES EXPLAINABILITY PROBLEM

• The proof first establishes hardness for a random forest with length-one time series and binary values by encoding sets as trees.Each tree predicts class 1 when any member of its corresponding subset has value 1.Source: SERIES EXPLAINABILITY PROBLEM · p. 11
• A minimum set of substitutions in this construction yields a minimum hitting set, linking the restricted explanation problem to the NP-hard hitting set problem.The proof then extends hardness to real-valued time series and more complex classifiers.Source: SERIES EXPLAINABILITY PROBLEM · p. 11
• The general problem of choosing a distractor and substitutions to maximize the target-class probability is NP-hard.The theorem applies given a classifier, class of interest, test sample, and training set.Source: SERIES EXPLAINABILITY PROBLEM · p. 11