PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Predictive Business Process Monitoring: Review and Benchmark

Efrén Rama-Maneiro, Juan C. Vidal, Manuel Lama

arXiv:2009.13251v4cs.LGcs.AI

TL;DR

Disparate process logs and experimental setups make fair comparisons of deep-learning predictive-monitoring approaches difficult. The paper reviews and classifies these approaches, then benchmarks available methods across process logs. Results identify different leading approaches across prediction tasks, while the benchmark excludes approaches without available code.

  • Problem

    Deep-learning predictive-monitoring approaches are evaluated on disparate logs and experimental setups, complicating fair comparison and selection of a suitable approach.

    Source: §1 · p. 1

  • Method

    The paper systematically reviews and classifies deep-learning approaches, then benchmarks available implementations in a fair setting across process logs.

    Source: §1 · p. 2, §6 · p. 13

  • Results

    Tax et al. ranks best for next-activity prediction, Camargo et al. with random sampling for suffix prediction, and Navarin et al. significantly outperforms others for direct remaining-time prediction.

    Source: §5.2 · pp. 12–13

  • Takeaways & Limitations

    The authors made the source code, trained models, and experimental results public as a starting point for new predictive-monitoring researchers.

    Source: §1 · p. 2

    The benchmark excludes seven approaches whose code was unavailable or could not be shared.

    Source: §5.1.4 · p. 11

Abstract

from arXiv · show

Predictive monitoring of business processes is concerned with the prediction of ongoing cases on a business process. Lately, the popularity of deep learning techniques has propitiated an ever-growing set of approaches focused on predictive monitoring based on these techniques. However, the high disparity of process logs and experimental setups used to evaluate these approaches makes it especially difficult to make a fair comparison. Furthermore, it also difficults the selection of the most suitable approach to solve a specific problem. In this paper, we provide both a systematic literature review of approaches that use deep learning to tackle the predictive monitoring tasks. In addition, we performed an exhaustive experimental evaluation of 10 different approaches over 12 publicly available process logs.

1 INTRODUCTION

Predictive monitoring forecasts relevant information for ongoing business-process cases, but deep-learning approaches vary in architectures, encodings, tasks, datasets, and experimental setups. This paper reviews and classifies those approaches and benchmarks available implementations fairly across process logs.

1 INTRODUCTION

  • Predictive monitoring trains classifiers on running-case prefixes to forecast next activities, remaining time, or outcomes; event logs record case IDs, activities, timestamps, and optional attributes.The event log example also includes the resource that executed each activity.Source: §1 · p. 1
  • Different architectures, encodings, tasks, datasets, and experimental setups make it difficult to compare approaches fairly or select a suitable one.The paper also notes the computational resources associated with deep learning.Source: §1 · p. 1
  • The paper identifies and classifies deep-learning approaches, then fairly compares available implementations across process logs and uses results to highlight their differences.The authors also made source code, trained models, and experimental results public as a starting point for new researchers.Source: §1 · pp. 1–2

2 DEFINITIONS

Predictive monitoring forecasts how aspects of the next event or sequence of events unfold from a running-case prefix through the end of the case.

2 DEFINITIONS

  • From an event prefix, the defined prediction tasks include next activity, next attribute, next timestamp, and the outcome of the prefix.The passage introduces functions for these predictions, with attribute suffix and remaining-time functions defined analogously.Source: §2 · p. 2
  • Activity suffixes can be predicted recursively, feeding predicted activities forward until [EOC], or directly as a suffix.The definitions also describe recursive and direct prediction of attribute suffixes.Source: §2 · p. 2
  • Remaining time can be predicted directly or calculated from predicted times through the last event.The text identifies the predicted timestamp for the last event in the sequence.Source: §2 · p. 2

3 RELATED WORK

Prior work reviewed predictive-monitoring methods and studied particular prediction tasks, but the paper distinguishes its review by benchmarking original deep-learning implementations under controlled conditions.

3 RELATED WORK

  • Earlier reviews classified methods by process-model use and prediction type, or organized approaches across prediction, domain, algorithm, and input dimensions [17].A third review examined three deep-learning approaches across encodings and prediction targets.Source: §3 · p. 2
  • A benchmark compared 20 supervised classifiers across six logs for next-event prediction and reported the credal decision tree as most accurate..Source: §3 · p. 2
  • Prior task-focused studies compared remaining-time methods and reviewed outcome prediction with different machine-learning sets and encodings,.The remaining-time study concluded that LSTMs outperformed other approaches.Source: §3 · p. 2
  • Only and have similar review objectives; covers three deep-learning approaches without benchmarking, and also lacks benchmarking.The results reported in are taken from the original papers.Source: §3 · p. 3
  • Other comparisons included LSTM and GRU alongside machine-learning algorithms [15],,, while experiments on CNN/LSTM variants and key-value-predict attention were limited in approaches or datasets,.The paper argues that no other review benchmarks original state-of-the-art deep-learning implementations under controlled conditions.Source: §3 · p. 3

4 ANALYSIS AND CLASSIFICATION

The paper classifies deep-learning predictive-monitoring studies along five dimensions: input data, prediction targets, neural-network type, sequence encoding, and event encoding.

4 ANALYSIS AND CLASSIFICATION

  • The classification covers input data from business-process logs and the elements of events that models are trained to predict.These dimensions describe the model's data inputs and prediction targets.Source: §4 · p. 3
  • It distinguishes neural-network types, including feedforward, autoencoder, convolutional, recurrent, and transformer networks, and how event prefixes become learnable tensors..Source: §4 · p. 3
  • It also classifies how individual categorical and continuous event variables are encoded in feature vectors..Source: §4 · p. 3

4.1 Input data

Input design determines which event-log information a predictive model can use, while noisy logs and irrelevant or variable features complicate that choice.

4.1 Input data

  • Most prediction problems use case identifiers, activities, and timestamps; attribute or outcome prediction requires the relevant case attribute.The needed information depends on the attribute being predicted.Source: §4.1 · p. 3
  • Real-life process logs can contain missing, duplicated, inconsistent, or otherwise incorrect data, and predictive-monitoring approaches often disregard noise or filter traces.Missing values range from absent tasks to absent event-level attributes; imputation can be applied to the latter.Source: §4.1 · p. 3
  • Timestamp features require preprocessing, commonly using elapsed time since the previous event or case start, with day-of-week and time-until-midnight features also used.One approach applies a decay function to elapsed time since the last activation of a Petri-net place.Source: §4.1 · p. 3
  • Some log attributes may add little predictive information, while noisy resources and highly variable inter-event times can complicate model training.Approaches select or cluster attributes, and groups resources by activity-execution profiles; it is the only approach noted as distinguishing resources from other attributes.Source: §4.1 · p. 3

4.2 Predictions

The section surveys prediction targets and choices involved in producing them, from forecasting a next event to generating an activity suffix. It also describes multitask learning and its loss-balancing challenge.

4.2 Predictions

  • Activity prediction forecasts the next activity, while activity-suffix prediction produces the sequence of activities following a running case.Next-activity prediction given a partial prefix is the most common prediction problem described.Source: §4.2 · pp. 3–4
  • The collected-study classification and morphological box of prediction possibilities are presented in Tables 2 and 3.Source: Table 2 · p. 4, Table 3 · p. 4
  • Next-timestamp prediction estimates the interval to the next event, whereas remaining-time prediction estimates the interval from the current event to the trace’s last event.Source: §4.2 · p. 4
  • Approaches that predict an activity suffix while using other log attributes as inputs must also predict the attribute suffix for subsequent events.Source: §4.2 · p. 4
  • Outcome prediction is usually supervised on fully labeled logs; instead uses a partially labeled log in a two-stage semisupervised LSTM procedure.The first LSTM predicts next activities on the full log; its learned parameters initialize an LSTM trained to predict outcomes on labeled cases.Source: §4.2 · p. 4
  • Suffix generation samples subsequent events from the network’s output distribution; highest-probability selection can repeat activities and fail to predict trace termination.Beam search retains the b traces with highest composed probability; stops at a complete candidate conformant with mined LTL rules, while samples randomly.Source: §4.2 · pp. 4–5
  • Multitask learning predicts multiple outputs together; shared layers receive gradients from each task, and combined learning may improve generalization, particularly with scarce logs.For suffix prediction, input features such as attributes or time features must also be predicted in subsequent steps.Source: §4.2 · p. 5
  • Task losses are commonly combined by summation, but differences in their magnitudes can let one task dominate and lead researchers to avoid multitask learning for next-activity-only prediction.Task weights may instead be set manually or learned automatically.Source: §4.2 · p. 5

4.3 Neural network types

The section surveys feedforward, autoencoder, recurrent, and convolutional architectures used in predictive monitoring, emphasizing how they represent process-event inputs and the constraints associated with them.

4.3.1 Feedforward networks

  • Feedforward networks suit tabular data but are uncommon in predictive monitoring because they do not exploit dependencies between activities.They are also used with autoencoders or after features are extracted from a process model.Source: §4.3.1 · p. 5

4.3.2 Autoencoders

  • Stacked undercomplete autoencoders compress hashed event-prefix n-grams before an encoder-fed softmax classifier, but disregard longer trace dependencies.Sparsity penalties and noise-corrupted inputs are suggested improvements that had not yet been explored in predictive monitoring.Source: §4.3.2 · pp. 5–6

4.3.3 Recurrent Neural Network

  • Vanilla RNN gradients can vanish or explode when learning long dependencies, motivating LSTM, GRU, and memory-augmented alternatives.Source: §4.3.3 · p. 6
  • LSTMs and GRUs use memory cells and gates to support deeper gradient flow through sequences than vanilla RNNs.LSTM gates regulate information forgotten, updated, or exposed; GRUs use update and reset gates but omit an output gate.Source: §4.3.3 · pp. 6–7
  • A bidirectional LSTM with attention predicts running-case outcomes by combining forward and backward hidden states and weighting timesteps.Source: §4.3.3 · p. 7
  • Memory-augmented networks add external memory for longer-term dependencies, though they can be expensive to train and sensitive to hyperparameters.The NTM uses attention-based memory addressing; the DNC improves memory management, and Khan et al.’s variation prevents its decoder from writing to memory.Source: §4.3.3 · p. 7

4.3.4 Convolutional neural network

  • The surveyed CNNs use either 1D convolutions with categorical embeddings or 2D convolutions with frequency-based encodings; 2D inputs can complicate adding attributes.For 2D convolution, event attributes require additional channels, which can be problematic when attribute and activity counts differ.Source: §4.3.4 · pp. 7–8
  • CNNs may train and infer faster than RNNs on longer traces, while RNN hidden states can capture dependencies beyond a CNN kernel’s k most recent events.The CNN dependency window is limited by kernel size k.Source: §4.3.4 · pp. 8–9

4.4 Sequence encoding

Sequence encoding converts variable-length traces into fixed-size tensors, creating choices that affect both the input representation and training targets. The surveyed formats differ in which event dependencies they retain and which neural architectures they suit.

4.4 Sequence encoding

  • Because trace lengths vary, fitting traces into the fixed-size tensors required by deep-learning architectures is an important encoding challenge.Encoding also conditions how training targets are supplied to the neural network.Source: §4.4 · p. 9
  • Continuous encoding uses event windows that may cross traces, with incomplete windows discarded or zero-padded and targets formed by shifting each window left.It may train faster than prefix-padded encoding, at the expense of capturing shorter dependencies.Source: §4.4 · p. 9
  • Prefix-padded encoding considers every event prefix, either retaining the W most recent events or all events, pads shorter prefixes with zeroes, and targets the next event.When all events are retained, vector length is often set to the log’s longest trace length.Source: §4.4 · p. 9
  • N-gram encoding represents traces by subsequences up to length k, then hashing maps features to a fixed-length vector and uses a sign update to counter collisions.For the predictive-monitoring proposals described, inputs and targets are equal, and this encoding has been used for autoencoders.Source: §4.4 · p. 9
  • Single-event encoding uses one event and its attributes while disregarding trace event order; its target is the next event.The review notes it is used less because it disregards dependencies between events.Source: §4.4 · p. 9
  • Timed-state encoding represents a Petri net’s state after replaying a partial trace, but is not directly compatible with more expressive RNN or CNN models.It combines decay values, token counts and positions, and other trace attributes; it may capture dependencies not apparent from event order alone.Source: §4.4 · p. 9

4.5 Event encoding

Event attributes require different preparation depending on whether they are continuous or categorical. The reviewed categorical encodings range from one-hot and learned representations to frequency-based and pretrained embeddings.

4.5 Event encoding

  • Continuous event attributes must be normalized, while categorical attributes must be represented as fixed feature vectors.The review lists lognormalization, min-max normalization, z-score normalization, and tanh-estimators for continuous variables.Source: §4.5 · p. 9
  • One-hot encoding represents each category with a binary vector, avoiding the false ordering implied by assigning categories integers.It is computationally and implementationally easy and adds no parameters, though large vectors can increase memory use.Source: §4.5 · pp. 9–10
  • Learned embeddings map categories to feature vectors whose parameters can be randomly initialized or learned for the prediction task.The review distinguishes pretrained embeddings, which are learned on another task and may provide richer information that eases neural-network convergence.Source: §4.5 · p. 10
  • Frequency-based encoding records how many times an activity has occurred up to the current event, adding temporal information to activity encoding.For the cited timed-state approach, frequency instead counts how often a token has passed through a Petri-net place.Source: §4.5 · p. 10
  • The reviewed pretrained embeddings learn representations by distinguishing observed from unobserved role-activity pairs in a log.The authors of that approach claim the embeddings discriminate events better; the review notes pretrained embeddings may provide richer information than task-trained embeddings.Source: §4.5 · p. 10

5 BENCHMARK

The benchmark evaluates deep-learning predictive-monitoring approaches across varied event logs using task-specific metrics and Bayesian comparisons. Results vary by task: Tax et al. and Hinkka et al. lead next-activity comparisons, Camargo leads suffix prediction with random sampling, and Navarin leads remaining-time prediction.

5.1.1 Datasets

  • The benchmark used 12 real-life logs from varied domains; most showed high event-time and trace-length variability.The NASA log had zero time-related measures because its time variability was low.Source: §5.1.1 · p. 10

5.1.2 Data split

  • Evaluation used five-fold cross-validation, with each four-fold remainder split 80/20 into training and validation sets.The authors limited folds, repetitions, and nested cross-validation partly because some approaches were costly to train or test; they also appended an end-of-case activity to every trace.Source: §5.1.2 · p. 10

5.1.3 Metrics

  • Accuracy measured next-activity classification, normalized Damerau–Levenshtein similarity measured suffix predictions, and MAE measured next-timestamp and remaining-time predictions.For comparison, the paper used Plackett–Luce rankings followed by hierarchical Bayesian pairwise tests of the top three approaches.Source: §5.1.3 · pp. 10–11

5.1.4 Approaches and experimental setup

  • Of 21 surveyed approaches, 10 were benchmarked; outcome-specific approaches and seven without publicly available code were excluded.Hyperparameter optimization was used only when available in the approach’s code; statistical comparisons excluded NASA and Sepsis because required resource attributes were unavailable for one approach.Source: §5.1.4 · pp. 11–12

5.2 Results and discussion

  • Camargo with random sampling ranked best for activity-suffix prediction, and the random procedure worked reliably across a wide range of datasets.The paper notes that argmax can repeat activities in long traces and loops, while random sampling can escape these repetitions.Source: §5.2 · p. 12
  • Navarin ranked best for remaining-time prediction, using direct prediction from a prefix rather than forecasting timestamps for a predicted suffix.The authors report that direct prediction avoids compounding errors and that argmax and random sampling made almost no difference in this task.Source: §5.2 · pp. 12–13
  • In 5-fold cross-validation for next-activity prediction, Hinkka et al. led 5 process logs, Tax et al. led 4, Theis et al. 2, and Pasquadibisceglie et al. 1.Plackett–Luce ranked Tax et al. first, followed by Hinkka et al.; simplex plots gave Hinkka a 56.5% probability of outperforming Tax and a 33.9% probability of equality.Source: §5.2 · p. 12

6 CONCLUSIONS

The paper classifies deep-learning predictive-monitoring approaches and benchmarks them across 12 process logs; its conclusions distinguish task-dependent performance patterns from efficiency trade-offs.

6 CONCLUSIONS

  • The review classifies deep-learning approaches across input data, predictions, neural-network type, sequence encoding, and event encoding, alongside an evaluation of 10 approaches on 12 logs.Source: §6 · p. 13
  • Attributes may not help except in some cases; random sampling does better for very long traces, argmax for shorter ones, direct remaining-time prediction beats accumulation, and RNNs seem to outperform CNNs overall.The authors attribute accumulated-prediction underperformance to compounding prediction errors.Source: §6 · p. 13
  • CNN pooling can add hyperparameters and risk suboptimal architectures, while CNNs run faster and may suit applications where speed is a priority.RNNs are often stacked sequentially; the approach of Mauro et al. addresses the pooling issue but unsatisfactorily in some logs.Source: §6 · p. 14
  • Encoding choices showed no performance differences in the experiments, but differed in efficiency: one-hot event encoding uses more memory at high activity counts, while continuous sequences train faster than padded prefixes.Source: §6 · p. 14
  • The experiments are intended as a baseline for future predictive-monitoring research; proposed extensions include non-deep-learning approaches and a graphical interface for benchmarking and visualization.Source: §6 · pp. 14–15

8 BIOGRAPHIES

The biographies describe the three authors' academic backgrounds, current roles, and research areas.

8 BIOGRAPHIES

  • Efrén Rama-Maneiro earned engineering and Big Data master's degrees and is a researcher pursuing a PhD, with interests in process mining and deep learning.Source: §8 · p. 18
  • Juan C. Vidal holds a computer-science engineering degree and an AI PhD; he is an associate researcher whose interests include process mining, fuzzy logic, machine learning, and linguistic summarization.Source: §8 · p. 18
  • Manuel Lama is an associate professor of artificial intelligence whose work spans process discovery, predictive monitoring, and dynamic-process management.He has participated in over 30 funded projects and published over 150 reviewed scientific articles.Source: §8 · p. 18
Loading 2009.13251v4…