PaperlayerView exact revision on arXiv

Source-linked AI summary

Early Prediction of AI-Assisted Cheating Risk in Online Exams Through Learning Analytics

Gökhan Akçapınar

arXiv:2609.37280v1cs.CYcs.HC

TL;DR

This study tests whether LMS traces from the first eight weeks can predict students’ behaviorally defined AI-assisted cheating risk in a final exam. It uses exam interactions to define risk groups and early-semester traces to predict group membership; Logistic Regression achieved an AUC of 0.763 and Balanced Accuracy of 0.718. The authors frame these predictions as support for timely guidance, not as evidence of misconduct.

  • Problem

    The study asks whether first-eight-week LMS behavior can meaningfully predict students’ AI-assisted cheating risk in the final exam.

    Source: §I · p. 2

  • Method

    The study uses early-semester LMS traces to predict risk groups defined by repeated suspicious exam events, including copying, focus loss, and right-clicking.

    Source: A. Target Variable Construction · p. 2, §I · p. 2

  • Results

    Logistic Regression achieved an AUC of 0.763 and a Balanced Accuracy of 0.718.

    Source: §IV · p. 5

  • Takeaways & Limitations

    LMS interactions may provide an early risk signal to support timely guidance, but predictions are not evidence of misconduct.

    Source: §IV · p. 5

    The sample is small and drawn from one programming course at one public university, and risk labeling and classification thresholds require local validation before use elsewhere.

    Source: §IV · p. 5

Abstract

from arXiv · show

AI-assisted cheating has become an important threat to the security of online exams. This study examines whether the risk of AI-assisted cheating in the final exam can be predicted using students' digital traces in the learning management system (LMS) during the first eight weeks of the semester. The sample comprised 52 first-year undergraduates enrolled in a bachelor's program in Computer Education and Instructional Technology and taking an Introduction to Programming course at a public university in Turkiye. Students were labeled as low- or high-risk based on suspicious behaviors recorded in the final-exam logs, including copy, focus-loss, and right-click events. Of the 52 students, 23 (44.2%) were labeled as high-risk in a proctored, face-to-face exam. Group membership was then predicted using five features selected from 27 candidates extracted from students' digital traces. Logistic Regression, Naive Bayes, Random Forest, and Gradient Boosting algorithms were used to build the prediction models. Model performance was evaluated using leave-one-out cross-validation (LOOCV) with fold-specific preprocessing and feature selection. Logistic Regression achieved the best performance (Accuracy = 73.1%). The results indicate that LMS interaction data can provide an early signal of AI-assisted cheating risk. Course-module views, assignment submissions, and the number of days on which course videos were accessed were the most consistently selected features across the LOOCV folds. These predictions are intended to support timely academic guidance, not to establish misconduct or initiate disciplinary action.

I. INTRODUCTION

The study asks whether LMS activity from the first eight weeks can predict individual students’ risk of AI-assisted cheating before the final exam. It extends learning analytics from academic-outcome prediction to an academic-integrity outcome, while treating prediction as a prompt for support rather than a finding of misconduct.

I. INTRODUCTION

  • Earlier exam-log analysis distinguished suspicious behavior, but identifying risk during or after an assessment leaves limited opportunity for preventive intervention.The study instead draws on learning-analytics research linking LMS traces with engagement and academic risk.Source: §I · p. 1
  • LMS traces are widely used to characterize behavioral engagement and academic risk, but primarily capture observable activity rather than cognitive or emotional engagement.Recorded interactions include assignment submissions, quiz attempts, and resource viewing.Source: §I · pp. 1–2
  • Early-semester LMS traces are examined as predictors of individual students’ AI-assisted cheating risk before the final exam.The study asks whether behavior during the first eight weeks can meaningfully predict risk derived from final-exam behavior.Source: §I · p. 2
  • The study extends early behavioral-engagement analytics from later academic outcomes to a behaviorally derived AI-assisted cheating-risk outcome.The authors frame this as an early-warning approach to academic integrity, with potential for supportive action before the final exam.Source: §I · p. 2

II. METHOD

The study uses a cohort of 52 first-year undergraduates in a hybrid programming course, with exam-log behavior defining risk labels and first-eight-week LMS and video-player records supplying candidate predictors. Preprocessing, feature selection, and model fitting are performed within each LOOCV training fold before prediction for the held-out student.

II. METHOD

  • The sample comprised 52 first-year undergraduates taking a hybrid Introduction to Programming course at a public university in Türkiye.Course materials were delivered through Moodle; the face-to-face final exam used a web-based system with 25 multiple-choice questions.Source: §II · p. 2
  • The target is a behavioral indicator of AI-assisted cheating risk, not a measure of confirmed misconduct.Students were informed that only the exam application could remain open and that interactions would be recorded; records were anonymized before analysis.Source: §II · p. 2
  • Within each LOOCV fold, preprocessing and Mann-Whitney U feature selection used only the 51-student training set before predicting the held-out student.The 52 out-of-sample predictions per classifier were aggregated for evaluation, preventing the test student's data from influencing preprocessing, feature selection, or model fitting.Source: §II · p. 2

A. Target Variable Construction

Risk labels are constructed from repeated prohibited exam events, counted question by question, with a five-question threshold separating high- and low-risk groups. The threshold lies above a gap in the observed distribution, and sensitivity checks preserve the overall group pattern.

A. Target Variable Construction

  • A question was flagged when copying, focus loss, or right-clicking occurred while it was displayed; text selection alone was excluded.Each question was counted at most once, and suspicious responses were tallied separately across all 25 questions; these labels are a behavioral proxy, not verified AI use.Source: A. Target Variable Construction · p. 2
  • The 20% threshold classified 23 students as high-risk and 29 as low-risk, above an empty region between two groups in suspicious-question frequency.Twenty-five students had no suspicious behavior, four had it in only one or two questions, and most of the remaining students had it in more than half the exam.Source: A. Target Variable Construction · p. 3
  • Sensitivity checks at 10%, 20%, and 30% preserved the overall group pattern, and cut-offs from 9% to 20% produced the same partition.Thresholds up to 48% affected only the four observations nearest the boundary.Source: A. Target Variable Construction · p. 3
  • The score-versus-frequency figure shows that several high-risk students scored highly despite suspicious behavior across multiple questions.The figure plots each student's suspicious-question frequency against final-exam score; final-exam scores were excluded from label construction and model training.Source: A. Target Variable Construction · p. 3, Figure 2 · p. 3

B. Learning Management System Data

The predictor set comprises 27 features derived from Moodle logs and a custom video player during the first eight weeks, before the midterm exam. It covers video interactions, viewing purposes, and Moodle activity, while excluding exam, assignment, and quiz grades.

B. Learning Management System Data

  • The study derived 27 features from first-eight-week video-player and Moodle activity, excluding exam, assignment, and quiz grades.Features covered video interactions (9), viewing purposes (5), and Moodle activity (13), including course-module views, assignment submissions, resource views, quiz attempts, and URL clicks.Source: B. Learning Management System Data · p. 3

C. Data Pre-Processing and Feature Selection

Preprocessing and feature selection were tailored to model requirements and small-sample conditions. Feature selection was performed within each training fold, without using the held-out student's data.

C. Data Pre-Processing and Feature Selection

  • MinMax scaling was applied to Logistic Regression within each training fold because it is sensitive to feature scale; tree-based models were not normalized.The dataset's features had important scale differences.Source: C. Data Pre-Processing and Feature… · p. 3
  • Within each training fold, the five candidate features with the smallest Mann-Whitney U p-values were selected from 27, excluding the held-out student's data.The number of selected features was fixed to constrain model complexity under small-sample conditions and support interpretability.Source: C. Data Pre-Processing and Feature… · p. 3

D. Classification Models and Evaluation

The study compared four binary classifiers and evaluated them with leave-one-out cross-validation. Fold-specific preprocessing and feature selection used only training data, and Balanced Accuracy and AUC were primary metrics because class sizes were unequal.

D. Classification Models and Evaluation

  • Four binary classifiers were compared: Logistic Regression, Gaussian Naive Bayes, Random Forest, and Gradient Boosting.Random Forest used 200 trees, max_depth = 4, and class weighting; Gradient Boosting used 100 trees and max_depth = 2.Source: D. Classification Models and Evaluation · p. 3
  • With N = 52, leave-one-out cross-validation held out one student per fold and confined scaling and Mann-Whitney U feature selection to the other 51.Models were evaluated using Accuracy, Balanced Accuracy, Macro-F1, and AUC; Balanced Accuracy and AUC were primary because class sizes were unequal.Source: D. Classification Models and Evaluation · p. 3

A. Model Performance Comparison

Logistic Regression achieved the highest AUC and Balanced Accuracy among the four models, and every model exceeded majority-class accuracy. Balanced Accuracy remained close to raw Accuracy for Logistic Regression and Naive Bayes.

A. Model Performance Comparison

  • Balanced Accuracy and raw Accuracy were 0.718 versus 0.731 for Logistic Regression and 0.701 versus 0.712 for Naive Bayes.The authors report that mild class imbalance did not dominate the model ranking.Source: A. Model Performance Comparison · pp. 3–4
  • AUC = 0.763 and Balanced Accuracy = 0.718 for Logistic Regression, the highest values on both metrics among the four models.All models exceeded the majority-class accuracy baseline of 0.558 (29/52); chance level for AUC was 0.500.Source: A. Model Performance Comparison · p. 3, Table I · p. 4

B. Detailed Analysis of the Best Model

The detailed analysis examines Logistic Regression's held-out predictions and how its operating threshold changes classification outcomes. At the 0.50 threshold, its AUC was 0.763; lower thresholds increased sensitivity while also increasing false positives.

B. Detailed Analysis of the Best Model

  • AUC = 0.763; at the 0.50 threshold, Logistic Regression identified 14 of 23 high-risk students and 24 of 29 low-risk students.These corresponded to Sensitivity = 60.9% and Specificity = 82.8%; 14 of the 19 predicted high-risk students were high-risk (Precision = 73.7%).Source: Figure 3 · p. 4, B. Detailed Analysis of the Best Model · p. 4
  • At 0.30, Sensitivity = 91.3%, Specificity = 51.7%, and Precision = 60.0%; the model identified 21 of 23 high-risk students.The lower threshold increased sensitivity while also increasing false positives.Source: B. Detailed Analysis of the Best Model · p. 4
  • At 0.60, Sensitivity = 52.2%, Specificity = 89.7%, and Precision = 80.0%; the model identified 12 high-risk students.The authors note that threshold choice depends on the intended use of the prediction.Source: B. Detailed Analysis of the Best Model · p. 4

C. Feature Selection Stability and Interpretation

The selected feature set was stable across folds, and course-module views, assignment submissions, and video-access days were the most consistently selected features. Low-risk students had higher group means for all seven selected variables.

C. Feature Selection Stability and Interpretation

  • Three features were selected in every fold, one in 51 folds, and another in 45; only seven of 27 candidates were selected in any fold.The selected feature set varied little across the 52 training folds.Source: C. Feature Selection Stability and… · p. 4
  • Course-module views, assignment submissions, and video-access days were most consistently selected, and each was lower in the high-risk group.These features were selected across the LOOCV folds; selection frequency does not quantify model-specific feature importance.Source: C. Feature Selection Stability and… · p. 4
  • Low-risk students had higher group means for all seven selected variables, with Cliff's delta values from 0.50 to 0.60.Group means and Cliff's delta describe between-group differences, not model-specific feature importance.Source: C. Feature Selection Stability and… · p. 4

IV. DISCUSSION AND CONCLUSION

First-eight-week LMS traces distinguished students labeled high- or low-risk in the final exam, with lower recorded engagement associated with higher behavioral risk. The findings support low-stakes, human-reviewed guidance, not misconduct determinations, and require validation beyond this single-course study.

Discussion and conclusion

  • Logistic Regression achieved an AUC of 0.763 and Balanced Accuracy of 0.718, while course-module views, assignment submissions, and video-access measures were lower in the high-risk group.The engagement differences indicate an association, not that lower engagement caused risk.Source: §IV · p. 5
  • 13 of 23 high-risk students scored 80 or above, compared with none of the 29 low-risk students; exam scores were excluded from labeling and training and do not independently validate the labels.Among 12 students with suspicious events in at least 20 of 25 questions, 11 scored 80 or above.Source: §IV · p. 5
  • Course-module views, video-access days, and assignment submissions may reflect self-regulation, but these trace-based proxies were not validated against direct measures.The authors recommend combining LMS traces with validated self-regulated-learning measures in future studies.Source: §IV · p. 5
  • Logistic Regression and Naive Bayes outperformed the tree-based models in this dataset, but this small-sample result does not establish that simple models generalize better elsewhere.The authors limit this comparison to the present dataset and caution against generalizing it to other courses or larger samples.Source: §IV · p. 5
  • At a 0.50 threshold, specificity exceeded sensitivity; lowering the threshold to 0.30 raised sensitivity to 91.3% but increased false positives.The authors suggest lower thresholds may suit low-stakes outreach, not disciplinary decisions.Source: §IV · p. 5
  • The proposed use is supportive outreach, with human review, transparent communication, and a way for students to question how their data were interpreted.Suggested support includes reminders about course materials, assignment follow-up, and brief instructor check-ins.Source: §IV · p. 5
  • The 52-student sample came from one programming course at one public university, and both the risk-labeling rule and decision threshold require local validation before use elsewhere.The authors call for larger, multi-course and multi-institution studies and stronger validation of risk labels before operational use.Source: §IV · p. 5
  • The study shifts risk identification from during or after an exam to before it, but the authors describe the approach as an early signal rather than an accusation.Replication with larger, more diverse samples and stronger label validation is required before operational use.Source: §IV · p. 5
Loading 2609.37280v1…