PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal Explanations of Process Monitor Predictions

Tom Yaacov, Nathan Blake, Hana Chockler

arXiv:2608.24672v1cs.SE

TL;DR

Predictive process-monitoring models are often opaque, while common local explanation methods do not represent the temporal and causal structure of event logs. The paper introduces AC4PM, which uses an actual-causality model to estimate event responsibility, and finds generally more stable and concise explanations than competing approaches while remaining computationally practical.

  • Problem

    PPM models can be opaque, and commonly used SHAP and LIME explanations do not account for temporal and causal structures in event logs.

  • Method

    AC4PM models trace events and monitor output within a process-specific causal model, then approximates each event's causal responsibility in a model-agnostic algorithm.

  • Results

    AC4PM produced generally more stable and concise explanations than LIME and SHAP while remaining computationally practical across 22 benchmark datasets.

  • Takeaways & Limitations

    Actual causality provides a process-tailored basis for local explanations that captures temporal dependencies and causal influence on monitor predictions.

Abstract

from arXiv · show

Process mining is widely used to diagnose processes and identify performance and compliance issues. Specifically, Predictive Process Monitoring (PPM) techniques use AI models to predict outcomes of ongoing process instances. While these models can achieve high predictive performance, their black-box nature makes it difficult to understand the underlying reasons behind their output predictions. In this paper, we propose a novel approach for generating local (case-level) explanations of process monitor predictions based on the framework of actual causality. We define a causal model tailored to processes that captures temporal dependencies between events in a trace, thus allowing us to reason about causal influence of events on the predicted outcome. Our method uses this model implicitly to compute causes and quantify the importance of different events with respect to the predicted outcome. We present a practical, model-agnostic algorithm that approximates event responsibility given the process structure reflected in the causal model. We evaluate our approach on a range of datasets derived from real-life event logs from a standard PPM benchmark. Each dataset contains up to 130,000 traces, with trace lengths of up to 1,800 events and up to 400 distinct event types. We compare our approach with state-of-the-art local explanation methods. The results demonstrate that our approach produces more stable and concise explanations while maintaining competitive efficiency.

I. INTRODUCTION

Predictive process-monitoring models can predict future process events effectively, but their opacity and generic explanations make causal, case-level interpretation difficult. The paper addresses this gap with actual causality and evaluates AC4PM against established local explanation methods.

  • Motivation: PPM applies machine learning to predict future process events, while deep-learning models typically provide little insight into how predictions were made.Post-hoc methods such as SHAP and LIME are widely used, but were developed for imaging and tabular data.
  • Motivation: Generic XAI methods do not account for temporal and causal structures inherent in event logs, especially in highly correlated domains such as healthcare.The paper links faithful explanations to trust and to practical or medico-ethical needs in business processes and medicine.
  • Contribution and evaluation: The paper introduces a process causal model capturing temporal dependencies, quantifies event importance through causal responsibility, and implements a model-agnostic approximation in AC4PM.The approach is compared with LIME and SHAP on datasets from a widely used predictive process-monitoring benchmark.
  • Limits of counterfactual explanations: Counterfactual explanations may miss over-determination, preemption, non-minimal changes, and interventions that violate the underlying causal structure.These behaviors can make an altered prediction fail to identify the actual causal path or distinguish sufficient independent causes.
  • Actual causality: Actual causality supports backward-looking reasoning about what caused a particular outcome while restricting alternatives to plausible contingencies in the causal model.This framework is designed to capture preemption and over-determination and to avoid inconsistent or unrealistic scenarios.

II. PRELIMINARIES

Process mining analyzes event logs to understand and improve operational processes. Predictive Process Monitoring uses machine learning to predict future aspects of ongoing process executions, including categorical outcomes.

  • Process mining analyzes event data to discover, monitor, and improve operational processes.
  • A process instance is represented as a trace: an ordered sequence of activities from a finite activity set.Events may include attributes, but this work records activity names and uses timestamps only for ordering.
  • An event log is a multiset of traces representing multiple executions of the same process.
  • Predictive Process Monitoring applies machine learning to predict future aspects of an ongoing business process execution.Targets include process-instance outcomes, completion, and future activities.
  • This paper focuses on categorical prediction targets and models the process monitor as a classifier C.

B. Actual Causality

Actual causality explains whether variables genuinely caused an observed outcome by testing counterfactual dependence under permissible contingencies. The framework also supports minimal causes and responsibility scores that rank causal influence.

  • A causal model separates exogenous variables U from endogenous variables V determined through causal relationships.A model is formally represented as M = (S, F), with domains and structural equations.
  • Intervening on variables replaces their structural equations with assignments, enabling counterfactual evaluation of outcomes.
  • An actual cause must satisfy AC1, AC2, and AC3: occurrence and outcome, a contingency that changes the outcome, and minimality.
  • Actual causality asks whether an observed outcome counterfactually depends on a proposed cause under some contingency on other variables.This structured reasoning can capture preemption and over-determination while restricting contingencies to the causal model.
  • Responsibility assigns variable X a score of 1/(|X| + |W|) for a smallest actual cause and contingency containing X, or 0 when none exists.In black-box explanations, this score orders features by importance to the predicted value.

III. CAUSAL MODEL FOR PROCESSES

The proposed process causal model represents trace events and the monitor output as endogenous variables, with exogenous factors determining the trace. It preserves temporal ordering and restricts interventions to reachable process behavior.

  • The model MP,C contains endogenous variables for trace events and an output variable O representing the predictive monitor’s result.Exogenous variables capture external factors and randomness influencing process execution.
  • Assigning the exogenous context determines the resulting trace and therefore the values of its event variables.
  • Temporal aspect: Each event is modeled as causally dependent on preceding events so interventions can affect subsequent process events.The model permits dependence on all earlier events as a general representation of temporal dependencies.
  • Natural ordering: Causal dependence follows the natural temporal ordering of events, although particular events may depend on only some predecessors.
  • Reachability: The model considers only interventions producing traces consistent with the underlying process behavior.This reachability restriction reflects the fact that the available causal information comes from observed traces.
  • Simplicity: Despite scaling with trace length, the model can approximate causes and responsibilities without explicitly constructing the full causal model.Its event variables also avoid auxiliary state variables because each event depends on previous events.

IV. COMPUTING RESPONSIBILITIES FOR PROCESS OUTCOMES

The responsibility algorithm samples reachable traces, identifies prediction-changing candidate causes, checks their minimality, and assigns approximate responsibility scores to events. It thereby produces an importance ranking for explaining process-monitor predictions.

  • The algorithm approximates event responsibilities and converts them into an importance ranking over trace events.The ranking can be used to compute local explanations.
  • Algorithm inputs and sampling: It takes an input trace, predictive monitor, process simulator, and trace dataset, then samples additional traces similar to the original.
  • Candidate causes: For a changed prediction, the candidate-cause procedure finds the smallest sampled prefix whose prediction is guaranteed by the dataset and returns its changed events.
  • Prediction changes: The algorithm evaluates monitor predictions for the dataset and retains sampled traces whose predictions differ from the original prediction.
  • Responsibility approximation: Candidate causes are checked for minimality, and each event receives the maximum score 1/(1 + |S|) across accepted causes.
  • Computational considerations: The stated overall complexity is O(n^3m^2), with O(n) calls to the predictive monitor.Here n is the number of traces including samples, and m is the maximal trace length.

B. Correctness analysis

The correctness analysis characterizes when the algorithm identifies actual causes and explains how sampling and intervention assumptions limit exact guarantees. With complete causal-model access, correctness and minimality are exact; practical use approximates causes from sampled traces.

  • Correctness guarantees: With access to the causal model, Algorithm 1 computes an actual cause satisfying AC1, AC2, and AC3.The proof establishes validity, counterfactual dependence, and minimality by evaluating interventions and minimizing candidate causes.
  • Correctness guarantees: With access to all possible process traces, Algorithm 1 satisfies AC1 and AC2, while AC3 is only approximated.The algorithm can identify an intervention that changes the prediction, but minimality is not guaranteed and is instead approximated through subset search.
  • Practical approximation: In practice, the simulator generates only a subset of possible traces, so the algorithm approximates actual causes and its quality depends on sample exhaustiveness.Experiments report that the resulting causes are close to optimal and more accurate than those from other methods.
  • Practical approximation: Assuming W = ∅ ensures reachable interventions but may overapproximate responsibility values.Despite this restriction, the lemmas preserve actual-cause capture and experiments show a highly effective responsibility ranking.

V. EXPERIMENTAL EVALUATION

The evaluation measures explanation quality using an insertion test that derives a minimal sufficient feature set from the algorithm’s responsibility ranking.

  • Explanation quality: The insertion test greedily adds features in responsibility order until the selected subset alone reproduces the explained instance’s prediction.This evaluates how efficiently an explanation ranking identifies a minimal sufficient set.

A. Case Studies

The case studies use benchmark-derived datasets and compare AC4PM with established local explanation methods under neural and tree-based predictive models.

  • Datasets: The case-study datasets differ in traces, variants, trace lengths, activity classes, and label distributions, including datasets with different labels from the same process.Table I summarizes these dataset statistics.
  • Experimental setup: AC4PM uses an LSTM process simulator pretrained separately for each case study and samples 100 additional traces during evaluation.The simulator predicts the next event from a trace prefix, and the train set supplies the algorithm’s input dataset.
  • Baselines: AC4PM is compared with LIME and SHAP, which assign feature-importance scores aligned with AC4PM’s causal responsibility ranking.The comparison targets local explainability methods for predictive process monitoring in black-box classifier settings.
  • Predictive models: Explanations are derived for XGBoost and MLP predictive classifiers using standard static one-hot encoding.For each classifier, AC4PM is compared against LIME and SHAP.

E. Evaluation Metrics

The evaluation measures explanation stability, succinctness, and computational efficiency across MLP and XGBoost classifiers. AC4PM generally produces stable, concise explanations, although runtime increases with process complexity and SHAP can run out of memory.

  • SHAP runs out of memory in some settings, whereas AC4PM and LIME complete across all evaluated datasets.The comparison reports VSI, explanation-size fractions, and average execution time.
  • AC4PM achieves the highest VSI stability in most datasets across both classifiers, with LIME slightly higher for XGBoost.
  • AC4PM’s stability is more consistent across classifier types than LIME’s, with Spearman correlations of ρ ≈0.88 and ρ ≈0.43, respectively.
  • AC4PM explanations are consistently smaller than LIME and SHAP explanations, indicating greater succinctness.
  • AC4PM explanation size decreases with activity count and mean trace length, with correlations of MLP ρ ≈−0.71 and ρ ≈−0.53.For XGBoost, the corresponding correlations are ρ ≈−0.68 and ρ ≈−0.52.
  • Runtime increases with traces, trace length, and activity count, especially for AC4PM as traces lengthen and datasets contain more traces.The algorithms generally produce explanations within a few minutes.

VI. RELATED WORK

Related work applies LIME, SHAP, counterfactual reasoning, and causal analysis to process predictions and mining. This paper focuses on local, case-level explanations for black-box PPM models while accounting for temporal and structural dependencies.

  • LIME and SHAP typically provide feature-based explanations while treating process traces as tabular data and ignoring temporal and structural dependencies.
  • Counterfactual explanations seek minimal trace changes that alter predictions, but may miss nuanced causal behaviors such as over-determination and preemption.In over-determination, multiple independent causes can each suffice; in preemption, an earlier event prevents another causal pathway from occurring.
  • Existing causal-analysis work in process mining primarily studies global process-level causal discovery rather than local explanations for black-box PPM predictions.
  • Actual causality supports backward-looking explanations of events that already occurred and has been applied to AI explanations in several domains.
  • Unlike prior depth-2 models that assume causal independence between inputs and the black-box model, this work represents dependencies with deeper causal models.

VII. CONCLUSION

The paper presents AC4PM, an actual-causality approach for local PPM explanations that models temporal dependencies and approximates event responsibility. Experiments find explanations generally more stable and concise while remaining computationally practical, but the current representation is simplified.

  • AC4PM uses a process-specific causal model to represent temporal dependencies between trace events and their influence on monitor predictions.
  • The algorithm approximates each event’s causal responsibility to estimate its importance to the predicted outcome.
  • Experiments on benchmark-derived datasets show explanations that are generally more stable and concise while remaining computationally practical.
  • Larger sample sets yield more precise explanations, and the approach remains stable with respect to trace complexity.
  • The current trace representation records only activity names, using timestamps solely to order events, while richer attributes and contextual data remain future-work targets.
  • Future algorithmic work will address contingency-set size to obtain more accurate responsibility estimates and more faithful rankings.
Loading 2608.24672v1…