Source-linked AI summary
Data-Related Challenges and Requirements for Event Log Generation in Process Mining: A Systematic Literature Review
Ghita El Alaoui Talibi, Oleksandr Kosenkov, Anastasija Nikiforova
TL;DR
Data-related challenges in event log generation remain insufficiently mapped to the requirements needed to address them. This paper conducts a systematic literature review and finds 29 challenges that are addressed by five main requirement groups, while existing solutions already use some data and domain knowledge engineering techniques.
Problem
Research lacks a systematic mapping between data-related event-log generation challenges and the requirements needed to address them.
Method
The paper conducts a systematic literature review of event-log generation challenges, requirements, and solutions, including their links to data and requirements engineering.
Results
The review identifies 29 challenges and five main requirement groups addressing practically all of them, while existing solutions apply some data and domain knowledge engineering techniques.
Takeaways & Limitations
The review provides a structured basis for understanding event-log generation challenges and requirements and for examining engineering techniques used in available solutions.
Takeaways & Limitations
The mapping between challenges and requirements relied on the authors’ interpretation because primary studies lacked consistent and explicit descriptions of their interconnection.
Abstract
from arXiv · showhide
Process mining is becoming an essential technology that not only supports the improvement of business processes but also enables the software technologies underlying the future digital economy. Among the multiple challenges discussed by the process mining community, data quality is emerging as the most pressing one. However, insufficient attention has been paid to event log generation as the very first phase of process mining and to its relationship with data engineering and requirements engineering. To address this gap, we conducted a systematic literature review of data-related challenges, requirements, and solutions in event log generation. Our results show that a wide range of 29 challenges can be addressed with 5 basic categories of requirements. Existing solutions already embed some form of data engineering and domain knowledge engineering techniques for event log generation. This review can support researchers and practitioners in understanding current trends in event log generation for process mining and in leveraging data and requirements engineering techniques to improve log generation and, ultimately, process mining outcomes.
1 Introduction
Process mining depends on high-quality event data, yet data-related challenges in event log generation remain unresolved. This review addresses the lack of an overview linking those challenges to requirements and solutions.
- Event-data discovery, integration, and cleaning remain core unresolved challenges in process mining.
- No existing overview systematically connects data-related challenges with the requirements needed for event log generation.
- This missing mapping makes it harder to identify research gaps, prioritize improvements, and design effective event log generation solutions.
- The study conducts a systematic literature review of challenges, requirements, and corresponding solutions in event log generation.
- The review emphasizes addressing data-quality challenges during log generation rather than only refining logs afterward.
2 Related work
Prior studies address individual event-log challenges, techniques, or domains but generally lack a systematic, cross-domain mapping between challenges and requirements. This review targets that unaddressed connection.
- Prior work discusses data quality but does not systematically map specific challenges to requirements for quality-focused logging.
- Existing technique overviews emphasize extraction, correlation, and abstraction methods rather than the challenges those methods address.
- A practical survey examined only a subset of event-log generation challenges and lacked systematic categorization.
- Earlier logging guidelines were not empirically derived, were not linked to specific challenges, and are partly outdated.
- A healthcare study categorized challenges by log-generation steps but offered no higher-level cross-domain synthesis.
- The review addresses the broader gap by establishing a structured mapping between data-related challenges and requirements.
3 Background
Event log generation transforms raw system data into structured business event logs suitable for process mining. The study distinguishes relevant business logs from system logs and situates the work within data and requirements engineering.
- Event log generation converts raw event data and system logs into structured business event logs for process mining.
- System event logs record system behavior, whereas business event logs capture information relevant to business-process execution.
- Raw system data often lacks the structure and context required for direct process-mining analysis.
- Data engineering manages data across its lifecycle, including ingestion, integration, modeling, preprocessing, processing, and querying.
- Requirements engineering systematically elicits, models, and analyzes stakeholder needs, constraints, and relevant domain information.
- Here, data-related challenges concern data processing during event log generation, while requirements specify what must be implemented in that process.
4 Methodology
The study uses a Kitchenham-guided systematic literature review to synthesize challenges, requirements, and solutions for event log generation. It searches recent literature, applies explicit selection criteria, and extracts findings through a structured two-round analysis.
- Study design: The review is a meta-synthesis organized around three questions covering challenges, requirements, and applications of engineering techniques.
- Study design: The search was limited to studies from the last five years to represent recent process-mining practices and challenges.
- Search strategy: Scopus and IEEE Xplore were selected as the primary databases, with search development validated against benchmark studies.
- Search strategy: The search query combined event-log and event-data terms with generation, extraction, or logging and process mining.
- Selection: The review applied explicit inclusion and exclusion criteria covering event-data logging, quality, generation, and related engineering techniques.
- Selection: 561 studies were initially retrieved, 479 remained after duplicate removal, and 59 were selected for data extraction after screening.
- Data extraction: Each selected paper was read in full and manually recorded using a transparent, replicable Excel-based extraction protocol.
- Data extraction: The extraction categorized bibliographic information, challenges, requirements, solutions, and sparse data-quality information across the review questions.
5 Results
The review identifies 29 data-related challenges in event log generation and organizes requirements around data availability, quality, access and structure, clear definitions and mapping, and domain knowledge. Frequently reported problems include data quality, integration, non-process-centric data, heterogeneity, case identification, granularity, effort, and event correlation.
- Data-related challenges: 29 data-related challenges were identified in event log generation, with data quality issues, data integration, and non-process-centric data the most frequently cited.These appeared in 39%, 27%, and 24% of primary studies, respectively.
- Data-related challenges: Data quality issues affected 23 primary studies (39%) and included missing, imprecise, inconsistent, incorrect, and irrelevant data.The literature emphasizes that these issues can significantly affect process mining results.
- Data-related challenges: Data integration from multiple sources was reported in 16 studies (27%), while heterogeneity across sources and structures appeared in 13 studies (22%).Event logs often require combining data scattered across systems with different formats and structures.
- Data-related challenges: Non-process-centric data, case notion, and granularity each create difficulties in extracting meaningful events and defining coherent process instances.Non-process-centric data appeared in 14 studies (24%), while case notion and granularity each appeared in 13 studies (22%).
- Data-related challenges: Event log generation was described as effort- and time-consuming in 12 studies (20%), often because extraction and transformation require substantial manual work.The literature links this burden to a lack of structured methodologies.
- Requirements: The requirements review groups reported needs into five broad categories and includes approach-, tool-, and standard-specific requirements such as XES and OCEL.The categories cover data elements, data quality attributes, data access and structure, clear definitions and mapping, and domain knowledge.
- Requirements: Data elements must include precise timestamps, unique case identifiers, meaningful activities and labels, and relationships linking events, activities, and cases.Timestamps establish event order, while case identifiers organize events into coherent process executions.
- Requirements: Requirements also call for accurate, complete, and non-duplicated data, accessible source data, understood database structures, and clear definitions and mappings.These requirements support reliable extraction, transformation, event grouping, and interpretation.
6 Discussion
The review maps 29 event-log-generation challenges to five requirement groups, while showing that data engineering, domain knowledge, and requirements engineering are closely interconnected but not yet applied systematically.
- Challenge–requirement mapping: 29 challenges are mapped to five main requirement groups, with only the effort- and time-consuming nature of event log generation lacking a specific requirement.The authors therefore characterize nearly all identified challenges as known and their corresponding requirements as established.
- Challenge–requirement mapping: Classical data-quality and missing-data challenges are mainly addressed through data-element availability and data-quality-attribute requirements.The literature discusses these requirements extensively, but offers limited guidance for contextualizing them across domains or projects.
- Challenge–requirement mapping: Data-processing challenges such as integration and locating data require data access and structure, sometimes supplemented by clear definitions and mappings.These requirements support data-engineering activities during event log generation.
- Challenge–requirement mapping: Domain-knowledge challenges, including non-process-centric data and the case notion, are addressed mainly through clear definitions and mappings, with domain experts needed in some cases.The mapping also identifies in-depth domain knowledge as an additional requirement for some challenges.
- Software logging and requirement interdependence: Some challenges arise from software logging practices and can be addressed mainly through data-quality requirements.Examples include missing data, non-process-centric data, granularity, logging variability, lack of standardization, and formatting errors.
- Software logging and requirement interdependence: Process scoping can require data access and structure, clear definitions and mappings, and domain knowledge simultaneously.This illustrates the close interconnection among requirement groups in event log generation.
- Existing solutions and adjacent disciplines: Existing solutions already reuse data-engineering techniques and address domain-knowledge challenges, but these techniques are not applied systematically.Data federation is identified as a potentially useful technique for data-integration challenges.
- Existing solutions and adjacent disciplines: Requirements engineering can support logging practices, process scoping, and domain-knowledge capture, while only one reviewed primary study focused on LLM application.The review suggests requirements engineering can help synthesize logging practices, identify relevant source-code fragments, and model process-mining requirements.
7 Threats to validity
The review addressed validity threats through repeated author discussion, iterative analysis, and independent mapping reviews, while acknowledging interpretive subjectivity in linking challenges to requirements.
- Review procedures: The authors followed systematic-literature-review guidelines, tested search strategies against gold-standard papers, and repeatedly discussed extraction and intermediate results.The analysis was conducted in two rounds by the two authors.
- Interpretive subjectivity: Challenge–requirement mapping involved author interpretation because primary studies lacked consistent, explicit descriptions of their interconnection.The authors independently reviewed the mappings in two iterative rounds and resolved discrepancies through discussion.
- Transparency and reproducibility: An open dataset documenting the reported challenge–requirement mapping was provided to improve transparency and reproducibility.
8 Conclusion
The conclusion presents event log generation as a fundamental process-mining phase and identifies five requirement groups as addressing its broad challenge landscape. It also points to data, domain-knowledge, and requirements engineering as complementary areas for improving available solutions and practice.
- Conclusion: Event log generation transforms raw event data and system logs into business event logs suitable for process mining.The study positions this transformation as an initial and fundamental process-mining phase.
- Conclusion: Five basic requirement groups can address the wide variety of event log generation challenges reported in the literature.
- Conclusion: Existing solutions already apply data-engineering and domain-knowledge engineering techniques, but not systematically.Data engineering can support extraction and integration, while requirements engineering can support scoping, mapping, log analysis, and logging implementation.
- Conclusion: The review recommends closer exploration of intersections among requirements engineering, software engineering, and data engineering in process-mining research and practice.These areas respectively address domain and system-operation knowledge, logging implementation and log quality, and data processing and integration.