PaperlayerView exact revision on arXiv

Source-linked AI summary

A Node Embedding Framework for Model-Centric Conformance Checking in Process Mining

Mostafa Abbasi, Homayoun Najjaran

arXiv:2610.17702v1cs.AI

TL;DR

Noisy, incomplete, or inconsistent event logs can make traditional conformance metrics unreliable for evaluating discovered process models. This paper compares aligned node embeddings of Petri-net graphs using a unified distance, finding robust performance under log noise and sensitivity to model degradation.

  • Problem

    Traditional fitness and precision assessments are highly dependent on event-log quality, so noise, deviations, and incomplete traces can distort process-model evaluation.

    Source: §1 · p. 1, §1.3 · p. 2

  • Method

    The framework converts Petri nets into graphs, learns process-aware node embeddings, aligns embeddings across models with Procrustes, and aggregates node distances into a bounded measure.

    Source: §1.6 · p. 3, §5 · pp. 18–19

  • Results

    Across real-life datasets and controlled synthetic noise, the framework provides reliable performance and remains robust to log noise while reflecting progressive process-model degradation.

    Source: §4.4 · p. 18, §5 · p. 19

  • Takeaways & Limitations

    Embedding-based structural distances offer an event-log–agnostic complement to traditional conformance metrics for process-model evaluation.

    Source: §5 · p. 19

    Full node coverage is not always achieved for some discovery algorithms, which can inflate distances in complex models.

    Source: §4.2 · p. 15

Abstract

from arXiv · show

Evaluating the quality of process models discovered from event logs is a fundamental challenge in process mining. Traditional evaluation relies on conformance metrics such as fitness and precision, which are highly sensitive to noise and deviations in event logs and therefore often fail to give a consistent assessment of discovery algorithms. We introduce a model-centric evaluation framework that combines node embeddings with Procrustes alignment to enable cross-graph comparability, since embeddings derived from different graphs reside in distinct vector spaces. The proposed PetriNetGraph method transforms Petri net process models into graph representations compatible with embedding algorithms. Comparing the node embeddings of ground-truth and discovered models through a parameterized cosine distance yields a unified and bounded metric for evaluating process discovery across datasets and methods. We evaluate the framework under static and context-aware scenarios to examine how prior knowledge of activity existence influences the assessment of structural similarity. The framework also integrates a compliance checking mechanism that localizes deviations at the activity level and an entropy-regularized node distance aggregation function that quantifies global deviations across the model. Experiments on nine publicly available event logs show that the approach provides a reliable embedding-based conformance indicator that reflects the behavior of fitness and precision. Experiments on synthetic logs with controlled noise levels confirm that it is substantially more robust to log deviations than traditional conformance metrics, PetriNet2Vec, and its dynamic-window and Procrustes-aligned variants, while remaining sensitive to process model degradation. All implementation details and datasets are publicly available.

1 Introduction and Related Work

Noisy or incomplete event logs can undermine traditional conformance assessments, while existing model-similarity approaches often do not compare discovered models against event-log traces. The paper proposes aligned graph embeddings and a unified distance to evaluate process models across graphs.

1.1 Event log repair

  • Event-log repair approaches include synthetic traces, Bayesian or stochastic repair, and human-in-the-loop methods, but some ground-truth evaluations rely on expert models and manual noise injection.Those dependencies can limit scalability.Source: §1.1 · p. 1

1.2 Process model discovery advances

  • Process discovery research spans supervised and unsupervised methods, including GNN-based approaches that use structural learning and commonly evaluate models with fitness, precision, or F-score.Balancing precision with acceptable fitness remains challenging.Source: §1.2 · p. 2

1.3 Conformance metrics evaluation

  • Traditional fitness and precision measures depend on event-log quality, and noise, deviations, or incomplete traces can distort process-model assessments.Balancing fitness and precision is also challenging because improving one can come at the expense of the other.Source: §1.3 · p. 2

1.4 Business process similarity evaluation

  • Prior process-model similarity techniques compare structures or labels but do not establish trace-level correspondence between discovered models and event logs.This limits their direct applicability to conformance checking.Source: §1.4 · p. 2

1.5 Embedding-based approaches

  • Petri-net embedding methods provide graph representations, but embeddings learned from separate models occupy different vector spaces and cannot be directly compared node by node.Many node-embedding frameworks are also designed for learning from a single, large graph.Source: §1.5 · p. 3

1.6 Research gaps and contributions

  • Separate embedding spaces and the omission of long-range dependencies limit prior approaches to comparing process models across graphs.The framework addresses these gaps with alignment and random walks that capture higher-order activity dependencies.Source: §1.6 · p. 3
  • The study evaluates both context-aware comparisons, which filter activities absent from each log trace, and static comparisons using the same model across comparisons.It also tests sensitivity to process-model deviations while assessing robustness to event-log noise.Source: §1.6 · pp. 3–4
  • PetriNetGraph conversion, adaptive walks, activity-level deviation checks, and entropy-regularized aggregation produce a bounded cross-graph distance for process-model evaluation.The framework represents Petri nets as graphs and aggregates node distances into a unified measure.Source: §1.6 · p. 4

2 Preliminaries and Methodology

The framework converts Petri nets into trace-aware graphs, embeds their nodes, aligns embeddings across models, and aggregates node distances to compare process structure.

2.1 PetriNetGraph construction

  • A Petri net becomes a directed graph, then trace-specific subgraphs by removing transitions absent from each trace and structurally disconnected nodes.Static and context-aware filtering provide complementary representations for comparison.Source: §2.1 · pp. 4–5, §2 · p. 4

2.2 Random walk mechanism

  • Directed random walks respect process-flow direction and Petri-net structure while capturing constructs such as parallelism and loops.Walks explore places and silent transitions, with selected starting nodes and transitions constrained by execution logic.Source: §2.2 · pp. 5–6

2.3 Skip-gram

  • Skip-gram learns node embeddings by predicting neighboring nodes in random-walk sequences, so co-occurring activities receive higher similarity.Embedding proximity is used as a data-driven indicator of behavioral alignment.Source: §2.3 · pp. 6–7

2.4 Distance function

  • Cosine distance compares corresponding node embeddings as a proxy for conformance, but independently trained graph embeddings are not directly comparable.Even structurally equivalent graphs can yield embeddings in different vector spaces.Source: §2.4 · p. 7

2.5 Procrustes alignment

  • Procrustes alignment learns an orthogonal transformation from shared node correspondences, placing the two graphs’ embeddings in a common space.The alignment is most directly supported on the matched anchor nodes; unmatched nodes are handled separately.Source: §2.5 · p. 7

2.6 Entropy-regularized node distance aggregation

  • Missing expected labels receive a penalty of α times the maximum matched-node distance, scaling the charge to the comparison.The default penalty does not infer activity criticality; domain-specific importance can instead be represented through node-level weights.Source: §2.6 · p. 8
  • Entropy-regularized weights trade off emphasizing well-aligned nodes against considering all expected labels; large λ yields uniform weights and average distance.The resulting Dweighted lies in [0,1], with lower values indicating greater structural similarity; it is a structural proxy, not an arithmetic combination of fitness and precision.Source: §2.6 · pp. 8–9
  • The workflow embeds static or context-aware graph representations, aligns them to the ground-truth space, then computes their structural distances.In the context-aware scenario, each case’s activities are used to filter the model before embedding.Source: §2.6 · p. 9

3 Datasets and Experimental Setup

The framework is evaluated on nine existing event-log benchmarks and on synthetic logs with controlled deviations. The setup spans several process domains, discovery methods, and increasing noise levels.

3 Datasets and Experimental Setup

  • Nine publicly available event logs and corresponding benchmark Petri nets cover healthcare, construction, permitting, and finance, with fitness and precision computed using alignment-based methods.The evaluated logs include BPIC_2012, BPIC_2017, BPIC_2020 variants, Road Traffic Fines, and Sepsis; the benchmark models were released by Sommers et al..Source: §3 · pp. 9–10
  • The framework was evaluated with Inductive Miner, ILP Miner, Heuristics Miner, Split Miner, and GNN Miner.Source: §3 · p. 10
  • Embeddings use a half-walk context window, one negative sample, and 128 dimensions; sensitivity tests found modest residual variation and no material change in comparative conclusions.The study describes 128 dimensions as a compromise between representational capacity and alignment stability.Source: §3 · p. 10

3.1 Synthetic log

  • Synthetic traces combine predefined behavioral rules with randomly sampled non-conflicting activities, then controlled noise levels from 0% to 90% generate logs and discovered models for comparison with the ground truth.The rules include precedence, co-occurrence, and mutual exclusion; deviations are assessed in incremental and decremental scenarios.Source: §3.1 · p. 10

4 Results and Discussions

The results compare embedding-based distances with traditional conformance metrics across discovery methods and noise conditions to assess how well the distances reflect behavioral deviations and model quality.

4.1 Static model and context-aware model

  • In the static setting, lower fitness and precision generally accompany higher cosine distance; methods with acceptable conformance typically have distances of 0.1–0.3.Split Miner has the lowest distance (0.1) in Sepsis; Split Miner and Heuristics Miner commonly fall in the 0.1–0.2 range, while Inductive Miner reaches 0.3–0.6 in cases where high fitness does not offset lower precision.Source: §4.1 · p. 11, Figure 3 · p. 12
  • Activity-level embedding visualizations identify nodes that diverge from expected behavior, helping locate deviations and potentially filter outdated or irrelevant activities.PCA reduces the trained 128-dimensional vectors to three dimensions for visualization.Source: §4.1 · p. 11, Figure 4 · p. 13
  • In context-aware comparisons, ILP Miner can have higher distance despite higher fitness and precision than Inductive Miner, reflecting its broader scenario coverage and lower simplicity.Filtering activities can retain connected edges and include many behavioral variants, sometimes producing an overgeneralized flower model; the static setting instead fixes the activity set and constrains random walks.Source: §4.1 · pp. 11, 13, Figure 5 · p. 14
  • Simplicity adds a structural quality dimension: on BPIC_2012, Heuristics Miner and Split Miner simplicity metrics differ by approximately 20%.Higher simplicity values indicate more readable models.Source: §4.1 · p. 13, Table 1 · p. 13

4.2 Unmatched nodes

  • In complex models, incomplete random-walk coverage can leave unmatched nodes and inflate distance; the ILP Miner visualization shows more than half of nodes missing or unmatched.The distance method assigns unmatched nodes the maximum distance among matched nodes rather than zero or the maximum cosine distance.Source: §4.2 · p. 15, Figure 7 · p. 16

4.3 Robustness of model-centric evaluation

  • Across increasing and decreasing log-noise tests, the method remains more stable than PetriNet2Vec+DW by 22.83% and 18.78%, respectively.The authors attribute robustness in part to preserving local activity neighborhoods while capturing long-term dependencies.Source: §4.3 · pp. 16–17, Figure 8 · p. 17, Table 2 · p. 17

4.4 Assessment of deviated process models

  • With a clean event log, embedding distance rises monotonically as model noise increases, and its CV is 29.19% higher than the best PetriNet2Vec variant with a dynamic window.Fitness remains relatively unchanged in this setting, while precision, PetriNet2Vec, and its variants respond less consistently and sensitively.Source: Table 2 · p. 17, Figure 9 · p. 18, §4.4 · p. 18

5 Conclusions, Future Directions, and Limitations

The framework compares process models through aligned node embeddings and an entropy-regularized distance, and experiments report robust performance under noisy logs and sensitivity to model corruption. The paper also identifies activity-level compliance checking and broader graph-comparison directions as uses and opportunities for the approach.

Framework

  • The four-stage framework constructs PetriNetGraphs, uses process-aware directed random walks, aligns embeddings with Procrustes, and aggregates distances with entropy regularization.The resulting measure is described as unified and bounded.Source: §5 · pp. 18–19

Results

  • 18.78% and 22.83% CV improvements under decreasing and increasing event-log noise, respectively, and 29.19% higher CV in process-model noise experiments support robustness and sensitivity.The framework was benchmarked against fitness, precision, PetriNet2Vec, and two intermediate variants; the authors report reliable performance across real-life datasets and controlled synthetic-log experiments.Source: §5 · p. 19

Compliance checking

  • The framework identifies deviations and outliers across activities in both discovered models and ground truth, extending its use to compliance checking.The paper links this capability to performance analysis and process enhancement.Source: §5 · p. 19

Future directions

  • The authors propose exploring embedding-based distances as model-training loss functions and adapting Procrustes alignment and entropy-regularized aggregation to other graph domains.Suggested domains include supply-chain and social networks; generalizability and scalability across graph topologies remain future research directions.Source: §5 · p. 19

Appendix

The appendix reports distance sensitivity to λ in context-aware and static settings and describes how the tables are organized. Its accompanying prose states that distance rises with λ in the context-aware results.

Context-aware results

  • In the context-aware results, distance increases as λ rises because the weighting scheme places less emphasis on smaller distances.Table 3 groups distances by dataset and method and reports matched and unmatched node counts, Procrustes residuals, and per-anchor residuals.Source: A Extended Results of λ Variations… · p. 21, Table 3 · p. 21
  • The reported total and per-anchor Procrustes residuals quantify alignment fit over shared anchors and distinguish alignment error from penalties for absent nodes.The per-anchor residual is the total residual divided by the number of shared anchors.Source: A Extended Results of λ Variations… · p. 21

Static results

B Context-Aware and Static Fitness and Precision vs Node Embedding Distance

Figures 10 and 11 relate fitness and precision to normalized embedding distance for context-aware and static models. The prose reports a general upward association between fitness and precision and larger point sizes, with a few exceptions.

Figure interpretation

Reported relationship

Figure scope

C Parameter Sensitivity Analysis

The appendix examines embedding-dimension, unmatched-node-penalty, aggregation, and min_count sensitivity. It reports broadly comparable alignment-residual patterns across dimensions, higher distances with larger α and λ, and stable embedding-pair counts across tested min_count values.

Embedding dimension

  • Across tested embedding dimensions, total Procrustes residuals vary but retain a comparable overall pattern in the challenging BPIC_2020_Permit_log ILP Miner case.This case has one of the smallest shared anchor sets among evaluated settings.Source: C Parameter Sensitivity Analysis · p. 32, Table 5 · p. 32

Penalty and aggregation parameters

min_count

  • Across min_count values from 20 to 200, embedding-pair counts vary by less than 1%, suggesting min_count has no material effect under the tested walk configuration.The authors describe min_count as primarily a safety guard; with substantially fewer walks or rarer node occurrences it could become more influential.Source: C Parameter Sensitivity Analysis · pp. 32–33, Table 8 · p. 33

Walk corpus

  • The authors recommend generating a sufficiently large walk corpus so unmatched nodes reflect the discovery algorithm rather than an embedding-level artifact.They state that random-walk generation is computationally negligible and attribute small pair-count fluctuations to random-walk stochasticity.Source: C Parameter Sensitivity Analysis · p. 33

D Runtime Analysis

Runtime varies substantially across the nine datasets and evaluation scenarios, with static evaluation often slower despite avoiding repeated graph construction. Word2Vec training dominates computation, while Procrustes alignment and distance computation take only a small fraction.

D Runtime Analysis

  • The runtime analysis decomposes the pipeline into graph construction, random walk generation, Word2Vec training, and Procrustes alignment plus distance computation.Measurements cover all nine benchmark datasets in static and context-aware scenarios.Source: D Runtime Analysis · p. 33
  • Word2Vec training is the dominant cost in both scenarios, while Procrustes alignment and final distance computation contribute only a very small fraction.In context-aware evaluation, graph construction is the second main contributor; in static evaluation, it is negligible because it is performed only once.Source: D Runtime Analysis · p. 33, Figure 12 · p. 34
  • Static evaluation can be slower overall despite lower graph-construction cost because it generates more Word2Vec training pairs, substantially increasing embedding time.Fig. 13 compares total runtime between the scenarios.Source: D Runtime Analysis · p. 33, Figure 13 · p. 34
  • Static runtime ranges from 165 s to 5,775 s, versus 45 s to 1,794 s for context-aware evaluation across the nine datasets.The runtime breakdown is shown in Fig. 12.Source: D Runtime Analysis · p. 33, Figure 12 · p. 34
  • Word2Vec training time is plotted against generated training pairs in both scenarios, while context-aware graph-construction time increases with the number of cases.For comparability, Fig. 13(a) proportionally rescales static training-pair counts and times to the corresponding context-aware dataset basis.Source: D Runtime Analysis · p. 33, Figure 13 · p. 34

Related papers

Loading 2610.17702v1…