Source-linked AI summary
Process mining for self-regulated learning assessment in e-learning
R. Cerezo, A. Bogarin, M. Esteban, C. Romero
TL;DR
Assessing self-regulated learning in e-learning remains challenging because students’ online interactions must be translated into meaningful learning-process models. This study applies Inductive Miner to Moodle interaction data and finds that passing students followed more self-regulated learning patterns, including forum-supported collaboration, than failing students.
Problem
Assessing self-regulated learning in e-learning requires converting large volumes of online interaction data into meaningful, theory-informed process models.
Method
The study applies Inductive Miner to Moodle interaction logs, coding student activities according to self-regulated learning phases and forum-supported collaborative learning.
Results
Passing students showed comprehension, learning, execution, review, and forum-supported collaborative-learning actions that did not appear in the failing group’s model.
Takeaways & Limitations
Process-mining models can support assessment and understanding of self-regulated learning skills in e-learning and inform teaching-learning decisions.
Takeaways & Limitations
The models may differ for students at other study levels or in different degree programs.
Abstract
from arXiv · showhide
Content assessment has broadly improved in e-learning scenarios in recent decades. However, the eLearning process can give rise to a spatial and temporal gap that poses interesting challenges for assessment of not only content, but also students' acquisition of core skills such as self-regulated learning. Our objective was to discover students' self-regulated learning processes during an eLearning course by using Process Mining Techniques. We applied a new algorithm in the educational domain called Inductive Miner over the interaction traces from 101 university students in a course given over one semester on the Moodle 2.0 platform. Data was extracted from the platform's event logs with 21629 traces in order to discover students' self-regulation models that contribute to improving the instructional process. The Inductive Miner algorithm discovered optimal models in terms of fitness for both Pass and Fail students in this dataset, as well as models at a certain level of granularity that can be interpreted in educational terms, which are the most important achievement in model discovery. We can conclude that although students who passed did not follow the instructors' suggestions exactly, they did follow the logic of a successful self-regulated learning process as opposed to their failing classmates. The Process Mining models also allow us to examine which specific actions the students performed, and it was particularly interesting to see a high presence of actions related to forum-supported collaborative learning in the Pass group and an absence of those in the Fail group.
Sample
The sample comprised 101 undergraduate students from a university in northern Spain who completed an online Moodle 2.0 course, yielding 21,629 events after preprocessing.
- Sample: 101 undergraduate students (83% women; mean age=20.23, SD=1.01) from a university in northern Spain completed an online Moodle 2.0 course.Preprocessing produced 21,629 events.
Pre-processing
The Moodle event log was cleaned, anonymized, filtered, and recoded into self-regulated learning categories aligned with Zimmerman’s model. The processed data were then partitioned by Pass/Fail status and by the course’s 11 instructional units for process discovery.
- Log cleaning and filtering: The log was reduced to Time, anonymous student IDs, Action, and Information, with duplicate, staff, administrator, test-user, and irrelevant records removed.Student names were converted into IDs to preserve anonymity and support ethical and professional conduct.
- Outcome-based partitioning: The processed log was split into Pass and Fail files using final marks of 5–10 and 0–4.9, respectively.This transformed numerical marks on a 10-point scale into categorical outcomes using traditional Spanish academic grading.
- Unit-level partitioning: Event files were additionally divided by unit to analyze behavior at finer granularity across the course’s 11 units.The Information attribute was used to identify each unit, which shared the same instructional design despite differing content.
Procedure
The study followed 101 students through an 11-lesson, one-semester eTraining course on study strategies, using weekly Moodle activities that combined theoretical content, practical tasks, and discussion forums. Students consented to data collection and could choose their learning path, although weekly practical tasks and forum participation were compulsory.
- Study setting: The course comprised 11 lessons delivered weekly during a one-semester assignment in a compulsory third-year subject completed outside teaching hours.Students were informed about data collection and gave informed consent.
- Instructional activities: Weekly activities addressed declarative knowledge through theoretical content, procedural knowledge through practical tasks, and conditional knowledge through contextual discussion forums.The activities focused on study strategies related to the subject topic.
- Instructional sequence: The instructor recommended a sequence of understanding theory, practicing through the corresponding task, and sharing weekly experiences in the forum.This sequence was supported by the instructional design and self-regulated learning theory.
- Participation requirements: Students could follow their own learning path, but each unit required completion of the weekly practical task and at least one forum comment.Each unit was estimated to require 2.5-3 hours of work per week, including reading, practical tasks, and forums.
Data analysis
The analysis transformed Moodle logs into XES files, applied Inductive Miner, and evaluated discovered models with fitness. Grouping students by Pass and Fail and analyzing individual units produced more informative workflow models than analyzing the whole course together.
- Analysis procedure: The analysis comprised log preprocessing into XES, Inductive Miner process discovery, and interpretation of the resulting models.XES is required for process mining with the ProM framework.
- Model evaluation: Fitness measured how accurately each discovered model reproduced the cases recorded in the event log.Precision and generalization were considered secondary because they are meaningful only after fitness is established.
- Model evaluation: Fitness improved when logs were grouped by Pass and Fail students rather than analyzed as one course-wide dataset.The Inductive Miner results also showed higher fitness by units than for all course units together.
- Whole-course models: Whole-course Pass and Fail models had lower fitness and extensive parallelism, obscuring students’ routes as collections of actions rather than clear processes.The Fail model showed only two isolated forum actions, while the Pass model retained some initial forum peer learning before becoming highly parallel.
- Unit-level models: Analyzing individual units produced longer, more developed networks with more structured workflows than the flattened whole-course models.The study increased granularity by creating sub-files for each unit to examine student behavior more thoroughly.
- Unit-level models: In Unit 4, Pass students began by viewing forum discussions before branching into URL, forum-posting, or quiz-related routes.The observed quiz route included quiz attempt, quiz view summary, and quiz continue attempt actions.
CONCLUSIONS
Inductive Miner produced useful models of students’ Moodle interactions and distinguished meaningful self-regulated learning processes between passing and failing students. The findings also support visual analytics, early-risk prediction, and personalization, while requiring validation across student populations and learning platforms.
- Main findings: Inductive Miner produced models with good fitness for reproducing students’ Moodle interactions, with better results after dividing the data by course units.The study analyzed 21,629 events from an online course and applied preprocessing, process discovery, and algorithm interpretation.
- Main findings: Failing students did not follow the instructor-supported learning path, whereas passing students followed a meaningful self-regulated process without exactly matching instructor suggestions.Passing students began with comprehension and learning actions and ended with execution and review actions.
- Main findings: Passing students’ process models showed forum-supported collaborative-learning actions that were absent from the failing group.These actions were identified through the discovered process models.
- Applications: Visual analytics of learning models could support real-time teaching decisions, while Inductive Miner models could contribute to early-warning systems and personalized e-learning environments.The proposed applications include predicting at-risk students during a course and supporting self-regulation development.
- Limitations and future work: The findings may vary for novice students, students in different degree programs, and other learning platforms such as Personal Learning Environments or MOOCs.The study used data from third-year graduate students, and the authors call for verification across different virtual learning environments.