Source-linked AI summary
Process Mining for Python (PM4Py): Bridging the Gap Between Process- and Data Science
Alessandro Berti, Sebastiaan J. van Zelst, Wil van der Aalst
TL;DR
Process-mining tools offered limited algorithmic customization, extensibility, and support for large-scale experimentation. The paper introduces PM4Py, a Python framework integrated with data-science libraries, and reports adoption through educational use, academic projects, and public distribution.
Problem
Existing process-mining tools and RapidProM provide limited support for customizable algorithms, custom integration, and large-scale experimentation.
Method
PM4Py uses Python and its ecosystem to provide an extendable, customizable process-mining framework with modular architecture and standardized factory methods.
Results
PM4Py 1.0 was used by 200 students, supported two academic projects, and was distributed through PIP with documentation and a collaborative repository.
Takeaways & Limitations
PM4Py provides a growing library of process-mining techniques with installation, documentation, examples, and collaborative code contributions available to users.
Abstract
from arXiv · showhide
Process mining, i.e., a sub-field of data science focusing on the analysis of event data generated during the execution of (business) processes, has seen a tremendous change over the past two decades. Starting off in the early 2000's, with limited to no tool support, nowadays, several software tools, i.e., both open-source, e.g., ProM and Apromore, and commercial, e.g., Disco, Celonis, ProcessGold, etc., exist. The commercial process mining tools provide limited support for implementing custom algorithms. Moreover, both commercial and open-source process mining tools are often only accessible through a graphical user interface, which hampers their usage in large-scale experimental settings. Initiatives such as RapidProM provide process mining support in the scientific workflow-based data science suite RapidMiner. However, these offer limited to no support for algorithmic customization. In the light of the aforementioned, in this paper, we present a novel process mining library, i.e. Process Mining for Python (PM4Py) that aims to bridge this gap, providing integration with state-of-the-art data science libraries, e.g., pandas, numpy, scipy and scikit-learn. We provide a global overview of the architecture and functionality of PM4Py, accompanied by some representative examples of its usage.
I. INTRODUCTION
PM4Py is proposed to address process-mining tools’ limited support for extensibility, algorithmic customization, and large-scale experimentation. It uses Python’s data-science ecosystem to support a more collaborative and rigorously tested development environment.
- I. INTRODUCTION: Existing process-mining tools emphasize graphical usability, but this hampers large-scale scientific experimentation.RapidProM supports repeated experiments but lacks easy algorithmic customization and integration of custom algorithms.
- I. INTRODUCTION: PM4Py is proposed as an easily extendable framework for algorithmic customization and large-scale process-mining experiments.The framework is implemented in Python, which is widely used across classic data science and machine-learning research.
- I. INTRODUCTION: PM4Py lowers the barrier for algorithmic development and customization compared with existing academic tools.The stated comparison includes ProM, RapidProM, and Apromore.
- I. INTRODUCTION: The framework integrates process-mining algorithms with algorithms from state-of-the-art Python data-science packages.This integration is presented as a central focal point of PM4Py.
- I. INTRODUCTION: PM4Py aims to support collaboration, documentation, and algorithmic stability through shared code, rich user support, and rigorous testing.These are listed as focal points of the library.
II. ARCHITECTURE AND FEATURES
PM4Py’s architecture separates process objects, algorithms, and visualizations to improve code reuse and large-scale experimentation. Factory methods standardize algorithm access while supporting variants and backward-compatible extension.
- II. ARCHITECTURE AND FEATURES: PM4Py adopts architectural guidelines intended to maximize code understanding, reuse, and execution of large-scale experiments.These guidelines organize the library’s design around modularity and standardized access.
- II. ARCHITECTURE AND FEATURES: Objects, algorithms, and visualizations are placed in separate packages.The object package stores and converts process-mining objects, while the algorithm package contains discovery, conformance, and enhancement methods.
- II. ARCHITECTURE AND FEATURES: Factory methods provide a single standardized access point for algorithms using input objects such as event data and parameter objects.The Alpha Miner factory exposes classic and plus variants.
- II. ARCHITECTURE AND FEATURES: Factory methods support algorithm extension while maintaining backward compatibility.They typically accept an algorithm variant name and shared parameters.
A. Object Management
PM4Py represents event data through logs, streams, and data frames, with conversion utilities connecting formats. It also supports several additional process-mining object types.
- A. Object Management: An event log is a collection of events describing activities performed for different process instances.PM4Py treats event data as a primary source for process mining.
- A. Object Management: Event logs represent lists of traces, with each trace containing events stored as key-value maps.This structure organizes events by process case.
- A. Object Management: Event streams represent a single list of key-value-map events that has not yet been organized into cases.Streams differ from event logs in their lack of case organization.
- A. Object Management: Conversion utilities transform event data between formats, and pandas data frames support efficient processing of larger event data.PM4Py also supports heuristic nets, accepting Petri nets, process trees, and transition systems.
B. Algorithms
PM4Py provides mainstream process-mining algorithms and analysis functions, including discovery, conformance checking, model evaluation, filtering, case statistics, graphing, and social-network analysis.
- B. Algorithms: PM4Py supports process discovery through Alpha(+) Miner and Inductive Miner.These algorithms are listed among the library’s mainstream process-mining techniques.
- B. Algorithms: Conformance checking is supported through token-based replay and alignments.The library also provides functions for measuring model fitness, precision, generalization, and simplicity.
- B. Algorithms: PM4Py supports filtering by time frame, case performance, trace endpoints, variants, attributes, and paths.These filters operate across several event-log and trace characteristics.
- B. Algorithms: Case management provides statistics on variants and cases, while graphs summarize case duration, events over time, and numeric-attribute distributions.These functions provide complementary descriptive analyses of event data.
- B. Algorithms: Social Network Analysis includes handover-of-work, working-together, and subcontracting networks, with Pyvis enabling dynamic web-based visualization.The Handover of Work metric is illustrated using Pyvis visualization.
C. Visualizations
PM4Py uses separate Python packages for visualization libraries and supports multiple static, web-based, and graph-oriented representations.
- The project uses Python visualization libraries including GraphViz, NetworkX, and Pyvis.
- GraphViz represents directly-follows graphs, Petri nets, transition systems, and process trees.
- NetworkX provides static representations of social networks.
- Pyvis provides web-based, dynamic representations of social networks.
III. EXAMPLES
The paper introduces representative PM4Py examples, including process discovery with the Alpha Miner.
- The examples section presents uses of PM4Py.
- Figure 5 shows PM4Py performing process discovery with the Alpha Miner.
A. Process Discovery
The process-discovery example imports an XES log, applies the Alpha Miner, lays out the resulting graph, and displays the process model; related figures document PM4Py usage statistics.
- The example loads factory methods for the XES importer, Alpha Miner, and Petri net visualization.
- An XES log is imported before the Alpha Miner is applied to the log object.
- The resulting visualization is laid out and shown in a window.
- Figure 5 presents the result of the process-discovery example.
- Figures 6 and 7 report February 2019 website-access and PyPI-download statistics for PM4Py.
B. Conformance Checking
The conformance-checking example applies alignments between a log object and a process model and displays each aligned trace's result.
- The example loads the alignments factory method.
- Alignments are obtained between a log object and a process model.
- Each aligned trace's alignment result is displayed on the screen.
- The alignment of a trace is reported in the lower part of Figure 3.
IV. MATURITY OF THE TOOL
PM4Py showed early maturity through educational use, publicly available academic projects, integrations, usage tracking, and certification.
- PM4Py 1.0 was used by 200 students in an Introduction to Data Science course at RWTH Aachen University.
- Two academic projects using PM4Py were publicly available, covering probabilistic automata for compliance checking and prefix alignments for streaming event data.
- PM4Py was integrated into bupaR for alignments and Inductive Miner models, while a Vue.JS data analytics web interface used the library.
- Google Analytics and PIP statistics were reported to track access to the PM4Py website and downloads of the library.
- The library received maximum-score XES certification, and issues were managed through GitHub.
V. CONCLUSION
PM4Py provides a growing Python library for process mining, with installation, documentation, and collaboration support for its users.
- PM4Py supports a rapidly growing set of process mining techniques, including discovery, conformance checking, and enhancement.
- The library can be installed with the command pip install pm4py.
- Extensive documentation and a collaborative GitHub ecosystem support user guidance, problem reporting, and code contributions.