Source-linked AI summary
Predicting University Students' Academic Success and Major using Random Forests
Cédric Beaulac, Jeffrey S. Rosenthal
TL;DR
Universities need to identify students at risk of not completing their programs so they can intervene. The article builds two classifiers from students’ early courses and grades, reports higher accuracy than linear classifiers, and finds a strong but non-causal relationship between grades in low-grading departments and undergraduate success.
Problem
Predicting whether students are at risk of not completing their programs matters because universities may want to intervene and help them progress.
Method
The article builds two classifiers using students’ first year of courses and grades: one predicts undergraduate completion and the other predicts major.
Results
Classifier accuracy exceeds that of linear classifiers, while grades in low-grading departments show a strong relationship with undergraduate-program success.
Takeaways & Limitations
Early course grades provide useful information for predicting undergraduate completion and major, and their associations with success should not be interpreted as causal.
Takeaways & Limitations
The data’s formatting is a weakness, and grades are not necessarily causal indicators of undergraduate success.
Abstract
from arXiv · showhide
In this article, a large data set containing every course taken by every undergraduate student in a major university in Canada over 10 years is analysed. Modern machine learning algorithms can use large data sets to build useful tools for the data provider, in this case, the university. In this article, two classifiers are constructed using random forests. To begin, the first two semesters of courses completed by a student are used to predict if they will obtain an undergraduate degree. Secondly, for the students that completed a program, their major is predicted using once again the first few courses they have registered to. A classification tree is an intuitive and powerful classifier and building a random forest of trees improves this classifier. Random forests also allow for reliable variable importance measurements. These measures explain what variables are useful to the classifiers and can be used to better understand what is statistically related to the students' situation. The results are two accurate classifiers and a variable importance analysis that provides useful information to university administrations.
1 Introduction
The article uses student records and random forests to predict undergraduate completion and major, while analyzing which early-course variables are most informative for university decisions.
- 1 Introduction: Using existing student records could help universities predict success and major without collecting additional data.The motivation is to support intervention, understand program demand, and allocate resources.
- 1 Introduction: Random forests improve classification trees by combining multiple trees and provide reliable variable-importance assessments.Classification trees can represent high-degree interactions across predictors.
- 1 Introduction: The classifiers exceed linear-classifier accuracy, making them useful for predicting where universities should allocate resources.The stated application is supporting decisions about intervention and resource allocation.
- 1 Introduction: Variable-importance analysis highlights grades in low-grading departments as highly important predictors and suggests possible grade inflation.The article presents this as a statistical symptom rather than a demonstrated causal effect.
2 Literature review
Prior work used socioeconomic, personality, and academic measures to predict success, while this study emphasizes academic records and random forests for prediction and variable interpretation.
- 2 Literature review: Earlier studies predicted academic success using personality traits, socioeconomic measures, high-impact practices, admission tests, and academic records.Reported predictors include freshman seminars, internships, and a trial-studying test.
- 2 Literature review: Aulck et al. reported similar performance for logistic regression, k-nearest neighbours, and random forests when predicting dropouts.The present analysis differs by using strictly academic information available in student records.
- 2 Literature review: The study’s proposed use of existing academic data could reduce reliance on entry tests or outside firms for predicting success and major.The intended administrative applications include preventing dropout and allocating departmental resources.
- 2 Literature review: Random-forest variable importance is presented as a tool for identifying useful predictors and examining the predictive role of grades across departments.The analysis is also motivated by questions about grade inflation and course-choice incentives.
- 2 Literature review: Differences in grading policies and course difficulty motivate examining whether grades are equally informative across departments.Prior work describes high- and low-grading departments and adjusted GPAs that account for course difficulty.
3 Methodology
The methodology uses first-year course records to predict program completion and completed major with classification models. It defines student-level predictors and response labels, partitions the data for training and evaluation, and uses random forests for classification and variable-importance analysis.
- The dataset contains 1 656 977 course-level observations, with each observation representing one student’s grade in one course.
- The study asks whether first-year course information can predict program completion and, for completers, the major they will complete.
- Predictors summarize attempted credits and average grades across 71 departments during the first five attempted credits, producing a 142-dimensional vector.
- Program completion is labelled using an 18-credit threshold, while students with at least 5 attempted credits, fewer than 18 successful credits, and three inactive semesters are classified as non-completers.
- Classification trees recursively choose conditions that reduce label impurity, while random forests combine tree-based classifiers and support variable-importance assessment.
4 Results
The random-forest classifiers achieved useful accuracy for predicting program completion and majors, while variable-importance analyses identified informative courses and grades.
- Predicting program completion: 91.19% accuracy was achieved for students who completed their program, while accuracy was 52.95% for 418 students who did not.
- Predicting program completion: 78.84% accuracy over the complete test set exceeded the 68.08% majority-class accuracy and the 74.21% logistic-regression benchmark.
- Predicting program completion: About 75% of students identified as dropouts were true dropouts, supporting targeted assistance for students at higher risk.
- Variable importance: Across the three forests, Mathematics, Finance, and Economics grades were consistently among the most important grade variables.
- Variable importance: ASSEM, a first-year seminar course, was important across the three plots, and its predictive relevance may help administrations assess new courses and programs.
- Predicting the major: The major classifier achieved 47.41% accuracy across 71 programs, exceeding the 4.75% weighted-probability baseline and 42.63% multinomial-logistic benchmark.
- Variable importance: Major-prediction grade importance was reduced by the data format, because credits in the highest-credit department effectively define the completed major.
- Predicting the major: First-year English and Computer Science course counts were especially informative because many students taking them completed programs in those departments.
5 Conclusion
Using first-year courses and grades, random forests predict program completion and completed major, while variable-importance analysis identifies statistically useful academic predictors. The study also highlights scope limitations and possible improvements, especially for multi-major students and missing predictor values.
- First-year courses and grades support two classifiers: one predicts undergraduate program completion and the other predicts the major completed.
- Random forests outperform linear logistic models in prediction accuracy and provide predictor-importance measurements for these tasks.
- Grades in low-grading departments, including Mathematics, Economics, and Finance, are consistently important for predicting program completion.
- Grades are less important for major prediction, although Mathematics, Finance, Economics, and Psychology remain frequently significant predictors.
- Major prediction may have low accuracy because some students complete multiple majors or specializations, making the task multi-label rather than single-label.
- Potential improvements include combining similar departments, handling missing predictor values, and developing trees that adapt split variables to previous partitioning.
A Appendix
The appendix defines classification trees and random forests, then describes their use alongside logistic regression for predicting program completion and completed major.
- Statistical learning setup: Classification problems use predictor vectors and categorical response variables, with k-class classification covering responses that take k possible values.The appendix frames supervised learning as estimating a relationship between a response variable and an m-dimensional predictor vector.
- Classification trees: A classification tree recursively partitions feature space and assigns each new observation the majority class in its terminal region.Splits are selected by minimizing the total impurity of the resulting regions, with the Gini index used here as an impurity measure.
- Classification trees: Tree construction stops when a region contains fewer than β observations, with β treated as a tuning parameter that can be selected by cross-validation.For continuous predictors, candidate splits use X_j ≤ s and X_j > s; categorical predictors allow partitions of their possible values.
- Random forests: Random forests aggregate predictions from bootstrap-trained trees by majority vote, while random feature selection makes trees less correlated and can reduce variance.Each bootstrap sample forms a learning set and an unpruned tree; the aggregated classifier selects the most frequently predicted class.
- Model comparison: The study compares random-forest classifiers with simple logistic regression for program completion and multinomial logistic regression for completed major.The binary model represents completion as Y_i = 1, while the multinomial model represents the completed program as one of k classes.