Source-linked AI summary
Predictive Business Process Monitoring with LSTM Neural Networks
Niek Tax, Ilya Verenich, Marcello La Rosa, Marlon Dumas
TL;DR
Existing predictive process monitoring methods are task-specific and sensitive to datasets and prediction points, motivating more general models. The paper develops LSTM architectures for next-event, continuation, and remaining-time prediction, and reports that they outperform existing approaches across these tasks. It also identifies repeated-activity traces as a limitation and leaves case-outcome prediction unaddressed.
Problem
Existing methods are tailor-made for specific prediction tasks, and their accuracy varies across datasets and prediction points, requiring combinations or tuning for consistency.
Method
The paper develops LSTM architectures for predicting next activities and timestamps, full case continuations, and remaining cycle time.
Results
LSTM-based methods outperform existing baselines for next-event and timestamp prediction and generalized approaches outperform tailor-made methods for continuation and remaining-time prediction.
Takeaways & Limitations
A single LSTM-based approach can address multiple predictive process monitoring tasks and achieve higher accuracy than specialized alternatives in the evaluated settings.
Takeaways & Limitations
LSTM models can predict overly long sequences when traces contain multiple occurrences of the same activity, and case-outcome prediction is not addressed.
Abstract
from arXiv · showhide
Predictive business process monitoring methods exploit logs of completed cases of a process in order to make predictions about running cases thereof. Existing methods in this space are tailor-made for specific prediction tasks. Moreover, their relative accuracy is highly sensitive to the dataset at hand, thus requiring users to engage in trial-and-error and tuning when applying them in a specific setting. This paper investigates Long Short-Term Memory (LSTM) neural networks as an approach to build consistently accurate models for a wide range of predictive process monitoring tasks. First, we show that LSTMs outperform existing techniques to predict the next event of a running case and its timestamp. Next, we show how to use models for predicting the next task in order to predict the full continuation of a running case. Finally, we apply the same approach to predict the remaining time, and show that this approach outperforms existing tailor-made methods.
1 Introduction
Predictive process monitoring methods vary in accuracy across tasks, logs, and prediction points, motivating a more general approach. This paper investigates LSTMs for next-event, continuation, and remaining-time prediction across four real-life logs.
- Existing approaches are tailor-made for specific prediction tasks and are not readily generalizable.
- Their relative accuracy varies with the input dataset and prediction point, sometimes requiring combinations of techniques or substantial tuning.
- LSTMs have shown consistently high accuracy in sequence-modeling domains including natural language processing and speech recognition.
- The paper investigates whether LSTMs generalize across predictive process monitoring tasks, event logs, and prediction points.
- The proposed architectures predict next activities and timestamps, complete case continuations, and remaining cycle time.
2 Related Work
Related work addresses time-related properties, case outcomes, and future events using specialized statistical, process-mining, classifier, and sequence-based methods. This paper positions LSTMs as a generalized alternative for several of these tasks, while excluding case-outcome prediction.
- Time-related predictions: Prior work predicts delays, deadline violations, late-show events, and remaining cycle time using specialized methods.
- Time-related predictions: Remaining-time prediction is treated as a special case of process-continuation prediction, with better accuracy than the methods in and.
- Case outcomes: Case-outcome approaches classify whether a case will reach a normal or deviant state using activity sequences and data attributes.
- Case outcomes: The paper does not address case-outcome prediction, although its proposed architectures could be extended in that direction.
- Future events: Earlier next-activity methods use probabilistic finite automata or LSTMs, while this paper proposes an LSTM architecture with higher reported accuracy and broader generalization.
- Future events: For full-case continuation, prior methods use Markov chains, shortest paths, or annotated transition systems; the paper uses the latter as a baseline.
3 Background
The paper represents process behavior as timestamped event sequences and introduces recurrent neural networks and LSTMs for sequence modeling. LSTMs replace the simple recurrent state with a gated memory cell designed for long-term dependencies.
- Event logs, traces and sequences: Prefixes and suffixes divide a sequence at a chosen position, while lifted functions apply event-level properties across entire sequences.
- Event logs, traces and sequences: Events have properties including timestamps and activities, with activity values drawn from a finite set of process activities.
- Event logs, traces and sequences: A trace is a finite non-empty sequence of unique events ordered by non-decreasing timestamps, and an event log is a set of such traces.
- Neural Networks & Recurrent Neural Networks: Figure 1 depicts an unfolded simple recurrent neural network, where each unfolding step represents a time step.
- Neural Networks & Recurrent Neural Networks: A recurrent neural network processes one sequence element per time step and updates a hidden state containing information from prior inputs.
- Long Short-Term Memory: An LSTM replaces the recurrent state with a gated memory cell that can be accessed, written, and cleared through output, input, and forget gates.
- Long Short-Term Memory: LSTM weights and biases are learned during training.
4 Next Activity and Timestamp Prediction
The section develops LSTM architectures to predict a running case’s next activity and timestamp from event-prefix features, then evaluates them against tailor-made baselines on two logs.
- Approach: LSTMs predict both the next activity and the time until the next event from feature vectors representing each event in a case prefix.Inputs combine one-hot activity encodings with three time-based features: elapsed time, time of day, and time of week.
- Approach: The time target is the difference between the next and current event, which permits calculating the next timestamp from the current event’s timestamp.Training minimizes mean absolute error for time prediction alongside cross entropy for next-event classification.
- Architectures: Three LSTM designs are evaluated: separate models, a fully shared multitask model, and a hybrid with shared and task-specialized layers.The shared-layer count distinguishes the architectures shown in Figure 2.
- Experimental setup: Evaluation uses chronologically separated training and test traces, measuring next-activity accuracy and timestamp MAE across prefixes from two process logs.The datasets are the Helpdesk log and the BPI’12 W subprocess log; a transition-system method provides the timestamp baseline.
- Results: All LSTM architectures outperform the timestamp baseline on both datasets, while the best results usually come from fully shared or hybrid architectures.Differences among the three LSTM architectures are small; one-layer models with 75 or 150 neurons performed worse, consistent with underfitting or overfitting.
- Results: 71% accuracy is the best next-activity result on Helpdesk, while BPI’12 W reaches 76%, exceeding reported accuracies of 71.9% and 62.3%.The LSTM results are reported as consistently higher than both comparison approaches on BPI’12 W.
5 Suffix Prediction
The paper predicts complete case continuations by repeatedly applying next-activity and time-until-next-event predictions until the end of the case. It evaluates these suffixes with a parallelism-aware similarity measure and finds that LSTM outperforms the baseline across logs, despite difficulties with repeated activities.
- 5.1 Approach: LSTMs generate a full case continuation by iteratively predicting the next activity and time until the next event until the end-of-case marker.The same next-event functions are repeatedly applied to extend predictions beyond one time step.
- 5.2 Experimental Setup: Damerau-Levenshtein Similarity measures suffix quality while treating swapped parallel activities as a smaller error than ordinary Levenshtein distance.The metric normalizes Damerau-Levenshtein distance by the longer suffix and subtracts it from 1.
- 5.2 Experimental Setup: The evaluation compares a two-layer LSTM with one shared layer and 100 neurons per layer against a prior method for full-case continuation.The architecture was selected based on good performance in earlier next-activity and next-event-time experiments.
- 5.2 Experimental Setup: The environmental permit log contains 937 cases, 38,944 events, and 381 event types, with almost every case following a unique path.This makes suffix prediction more challenging on that dataset.
- 5.3 Results: LSTM outperforms the baseline on all evaluated logs, but performance remains low on BPI’12 W because repeated activities produce overly long predicted suffixes.Removing repeated occurrences yields only a mild improvement on the modified BPI’12 W log.
6 Remaining Cycle Time Prediction
The paper derives remaining cycle-time predictions from predicted timestamps for all future events, using the predicted case-end timestamp to estimate time remaining. LSTM generally outperforms baselines, but repeated events in BPI’12 W can cause overestimation.
- 6.1 Approach: The predicted timestamp of the final future event provides the case end time needed to estimate remaining cycle time.The estimate subtracts the timestamp of the last observed event from the predicted end timestamp.
- 6.2 Experimental Setup: Remaining-time prediction uses the same LSTM architecture as suffix prediction and evaluates every prefix after the second event.Two transition-system and regression-based methods serve as baseline methods.
- 6.2 Results: LSTM consistently outperforms baselines on Helpdesk across prefix sizes, while BPI’12 W is an exception for short prefixes.The BPI’12 W issue is linked to repeated events that make predicted suffixes too long and remaining times too large.
- 6.2 Results: On BPI’12 W without duplicate events, LSTM outperforms the baseline for remaining-time prediction.The last event is retained when removing repetitions so the ground-truth remaining cycle time is unchanged.
- 6.2 Results: Figure 3 reports mean absolute error by prefix size for Helpdesk, BPI’12 W, BPI’12 W without duplicates, and Environmental Permit.The figure compares remaining-time prediction across four logs and varying prefix lengths.
7 Conclusion & Future Work
The paper’s LSTM technique outperforms existing approaches for next-activity and timestamp prediction, and its generalized form addresses continuation and remaining-time prediction. The authors identify overly long repeated-event predictions as a limitation and outline extensions to other tasks and attributes.
- Contributions: LSTM neural networks predict a running case’s next activity and timestamp, with a single multi-task model achieving higher accuracy than separate models.The technique outperforms existing baselines on real-life datasets.
- Contributions: The technique generalizes to predicting an entire case continuation and its remaining cycle time, outperforming tailor-made approaches for both problems.
- Limitation: Traces containing multiple occurrences of the same activity can cause LSTM models to predict overly long sequences of that event.Addressing this limitation is identified as future work.
- Future Work: Future work includes aggregate performance indicators, case outcomes, additional case and event attributes, and broader multi-task prediction.Case outcomes can be approached as classification, with output neurons predicting corresponding outcome probabilities.
- Reproducibility: The source code and supplementary material required to reproduce the reported experiments are available online.