PaperlayerView exact revision on arXiv

Source-linked AI summary

Educational data mining and learning analytics: An updated survey

C. Romero, S. Ventura

arXiv:2402.07956v1cs.HCcs.AI

TL;DR

Educational data have expanded across sources and formats, creating a challenge of turning them into useful insights. This updated survey reviews EDM and LA research and reports a field whose literature, terminology, environments, tools, datasets, and application topics have broadened since 2013.

  • Problem

    The growth and diversity of educational data create a challenge in transforming it into insights, while six years of new publications made an updated overview of EDM and LA necessary.

    Source: INTRODUCTION · pp. 1–3

  • Method

    The paper provides an updated, general survey of EDM and LA publications, milestones, knowledge discovery, educational environments, tools, datasets, methods, objectives, and future trends.

    Source: INTRODUCTION · pp. 3–4

  • Results

    Since the previous survey, the literature and related terminology have grown, alongside interest in new educational environments, tools, free datasets, application topics, and future trends.

    Source: INTRODUCTION · pp. 3–4

  • Takeaways & Limitations

    The survey presents a broad overview of the current EDM and LA landscape across research, educational settings, resources, methods, objectives, and trends.

    Source: INTRODUCTION · pp. 3–4

    The taxonomy does not cover all possible EDM/LA tasks, whose specific objectives can vary by stakeholder viewpoint.

    Source: METHODS AND APPLICATIONS · p. 20

Abstract

from arXiv · show

This survey is an updated and improved version of the previous one published in 2013 in this journal with the title data mining in education. It reviews in a comprehensible and very general way how Educational Data Mining and Learning Analytics have been applied over educational data. In the last decade, this research area has evolved enormously and a wide range of related terms are now used in the bibliography such as Academic Analytics, Institutional Analytics, Teaching Analytics, Data-Driven Education, Data-Driven Decision-Making in Education, Big Data in Education, and Educational Data Science. This paper provides the current state of the art by reviewing the main publications, the key milestones, the knowledge discovery cycle, the main educational environments, the specific tools, the free available datasets, the most used methods, the main objectives, and the future trends in this research area.

INTRODUCTION

Educational data have expanded across systems and learning environments, making automated analysis necessary to turn them into insights that benefit students, teachers, and administrators. This survey updates the 2013 overview of EDM and LA in response to the field’s growth and changing scope.

INTRODUCTION

  • Expanding educational repositories span diverse sources, formats, and learning environments, creating data too extensive for manual analysis and motivating automated tools to reveal insights about learning.These data come from traditional information systems, web-based systems, and environments such as blended, virtual, mobile, and game-based learning.Source: INTRODUCTION · pp. 1–2
  • EDM develops methods and applies data mining to educational data, while LA measures and analyzes learner-context data to understand and optimize learning (Bakhshinategh et al. 2018; Romero and Ventura, 2013; Lang et al., 2017).LA’s definition emphasizes data, analysis, and action. The communities differ mainly in focus, research questions, and use of models, while sharing methods and goals of enhancing educational practice.Source: INTRODUCTION · p. 2
  • EDM and LA combine computer science, education, and statistics, alongside related areas including data mining and machine learning, educational statistics, and computer-based education.Their interdisciplinary scope also includes information retrieval, recommender systems, visual data analytics, social network analysis, psychopedagogy, cognitive psychology, and psychometrics.Source: INTRODUCTION · p. 2, Figure 1 · p. 3
  • Related terms describe institutional analytics for political and economic insight (Long and Siemens, 2014), teaching analytics from instructors’ perspectives, data-guided decisions , educational big-data techniques, and educational data science (Romero & Ventura, 2017).These terms include Academic and Institutional Analytics, Teaching Analytics, Data-Driven Education and Data-Driven Decision Making in Education, Big Data in Education, and Educational Data Science; the latter unifies statistics, data analysis, machine learning, and related methods.Source: INTRODUCTION · p. 3
  • The survey updates its 2013 predecessor to cover exponential publication growth, new learning environments, more tools and datasets, broader topics, and emerging trends.The authors report that publication growth has been greater in LA than EDM and note increased interest in MOOCs, virtual and augmented reality, serious games, and blended learning.Source: INTRODUCTION · pp. 3–4

BACKGROUND

The background traces EDM and LA through their conferences, books, journals, and influential reviews, while describing the field’s growing visibility and changing terminology.

BACKGROUND

  • EDM and LA began with separate communities: the first EDM conference was held in 2008, and the first Learning Analytics and Knowledge conference in 2011.The first EDM conference was organized by the IEDM society; the first Learning Analytics and Knowledge conference was organized by SOLAR.Source: BACKGROUND · p. 4
  • The first EDM/LA book appeared in 2006; book titles shifted from Education Data Mining terminology toward Learning Analytics, Data Science, and Big Data, with Learning Analytics most used in recent years.The survey identifies the Handbook of Educational Data Mining (Romero et al., 2010b) and the Handbook of Learning Analytics as the two most important books, and notes an online MOOT with course resources.Source: BACKGROUND · pp. 4–5
  • The Journal of Educational Data Mining, launched in 2009, and the Journal of Learning Analytics, launched in 2014, are the two most specific journals for the fields.The survey also notes that iJAI and Computer-Based Learning in Context launched in 2019.Source: BACKGROUND · p. 7
  • Search results for both fields grew exponentially from 2000 to 2018; EDM had more references until 2011, after which LA surpassed it.The survey notes that a bibliometric study describes the fields together as Learning Analytics, with Educational Data Mining as a prominent subset.Source: BACKGROUND · p. 8, Figure 2 · p. 9
  • Four in ten of the most cited EDM and LA papers are reviews or surveys, including influential reviews by Romero and Ventura (2007, 2010a, 2013) and Siemens and Baker (2012).Other cited contributions include the U.S. Office of Educational Technology report and LA reviews by Ferguson (2012) and Dawson et al. (2014).Source: BACKGROUND · p. 9, Table 4 · p. 10

EDM/LA KNOWLEDGE DISCOVERY CYCLE

EDM/LA applies the general Knowledge Discovery and Data Mining cycle, with educationally specific characteristics at each step. The process spans data collection and preprocessing through mining and the use of findings to support interventions and decisions.

EDM/LA KNOWLEDGE DISCOVERY CYCLE

Educational Environment

  • The educational environment and information system shape which data can be collected to address different educational problems (Romero et al., 2011).Examples include traditional classrooms, computer-based or blended learning, LMSs, ITSs, and MOOCs.Source: Educational Environment · p. 11

Educational Data

  • Raw educational data span multiple sources and levels of granularity, and gathering and integrating them for mining requires preprocessing.Sources include learning interactions, administrative and demographic records, and student affectivity; granularity ranges from keystrokes to schools.Source: Educational Data · p. 11

Preprocessing

  • Preprocessing can take more than half of data-mining time and includes aligning and transforming data for each problem, engineering features, and protecting student confidentiality.The paper also advises considering ethics, privacy, and informed consent.Source: Preprocessing · pp. 11–12

Methods and techniques

  • Visualization, classification, clustering, and association analysis have been applied in education, while hierarchical and longitudinal data require specific mining and preprocessing treatment.The survey notes a wide range of EDM and LA methods for different educational problems.Source: Methods and techniques · p. 12

Interpretation and application of the new knowledge

  • Follow-up actions by instructors and academic authorities are the ultimate goal of learning analytics and determine whether findings improve student learning performance.The findings are intended to inform interventions and decisions.Source: Interpretation and application of the… · p. 12
  • Decision-useful models need to be comprehensible; visualization aids interpretation, and recommender systems provide explanations and recommendations to students and non-expert users such as instructors.The paper discusses white-box decision trees and black-box neural networks as model examples.Source: Interpretation and application of the… · p. 12

EDUCATIONAL ENVIRONMENTS AND DATA

Educational environments include traditional, computer-based, and blended systems, each providing different data sources. Their data range from attendance and marks to information gathered across face-to-face and computer-mediated instruction.

EDUCATIONAL ENVIRONMENTS AND DATA

Traditional face-to-face education

  • Traditional education, the most widely used system, gathers attendance, marks, curriculum goals, class, and schedule information through face-to-face teaching.It may also use computer-based systems to complement face-to-face sessions.Source: Traditional face-to-face education · p. 13

Computer-based educational systems

  • Computer-based education has evolved from standalone applications to web-based intelligent systems, including AIHS, ITS, LMS, and MOOCs.These systems use computers to direct, deliver, or manage instruction.Source: Computer-based educational systems · p. 14

Blended Learning Systems

  • Blended learning combines face-to-face and computer-mediated instruction, gathering data from both types of systems.Also called hybrid learning or mixed-mode instruction, it moves some face-to-face sessions online and offers increased access, convenience, and flexibility.Source: Blended Learning Systems · p. 15

TOOLS AND DATASETS

EDM/LA research can use general-purpose analytics tools, but these demand method and parameter expertise, while specialized tools are limited to particular data and educational problems. Public datasets offer an alternative to costly data collection, yet remain scarce and unevenly representative of educational environments.

TOOLS AND DATASETS

  • General-purpose tools such as RapidMiner, Weka, SPSS, Knime, Orange, and Spark Lib require educators to select methods and parameters, while specialized tools address specific data and problems.Educators need expertise to find appropriate settings for good results or models (Romero and Ventura, 2013).Source: TOOLS AND DATASETS · p. 15
  • Using free public datasets can avoid the hard, time-consuming work of gathering and preprocessing educational data.Researchers commonly use their own data to address specific educational problems (Romero et al., 2014).Source: TOOLS AND DATASETS · p. 16
  • DataShop was one of the first and largest datasets for ITS research, but public datasets remain few and mostly come from e-learning systems.The survey highlights DataShop for its research tools as well as its dataset.Source: TOOLS AND DATASETS · p. 17
  • The survey recommends a dedicated EDM/LA dataset repository with portable data that follows ethics, privacy, protection, and consent principles, using standards such as xAPI or IMS Caliper.The proposed repository would be similar to the UCI Machine Learning Repository and help prevent incompatible data silos.Source: TOOLS AND DATASETS · pp. 17–18

METHODS AND APPLICATIONS

EDM/LA applies a mix of broadly used data-mining methods and techniques with particular prominence in education. Its possible objectives extend across many stakeholders and educational problems, with the survey highlighting varied current applications and topics.

METHODS AND APPLICATIONS

  • EDM/LA uses broadly applicable methods including visualization, prediction, clustering, outlier detection, relationship and causal mining, social network analysis, process mining, and text mining.The survey also notes education-focused techniques such as distillation of data for human judgment, discovery with models, knowledge tracing, and non-negative matrix factorization.Source: METHODS AND APPLICATIONS · p. 18
  • Knowledge tracing estimates skill mastery from a cognitive map of problem-solving items and students’ correct or incorrect answers, monitoring knowledge over time.Non-negative matrix factorization decomposes positive student test-outcome data into item and student skill-mastery matrices.Source: METHODS AND APPLICATIONS · pp. 18–19
  • The taxonomy does not cover all EDM/LA objectives, which vary by final user and involve more stakeholders than learners and instructors.Table 11 presents examples of current applications and topics of interest.Source: METHODS AND APPLICATIONS · p. 20, Table 10 · p. 20, Table 11 · p. 22
  • Highlighted application topics include curriculum analysis, visual learning analytics, deep learning, causal discovery, and early prediction of student performance or risk for intervention.Curriculum analytics can use program structure, course grades, and administrative curricular data to support curriculum development and program quality.Source: METHODS AND APPLICATIONS · p. 21
  • Other application topics include studying emotion in learning, evaluating interventions, automated feature engineering, and analyzing player interactions in serious games.Source: METHODS AND APPLICATIONS · p. 21
  • Further topics include interpretable learner models, foreign-language learning, and measuring students’ self-regulated learning features and behaviors.Source: METHODS AND APPLICATIONS · p. 21
  • Emerging applications use machine learning and affordable sensors to support learning analytics, alongside personalized feedback and analysis of navigation paths.Source: METHODS AND APPLICATIONS · pp. 21–22

CONCLUSIONS AND FUTURE TRENDS

EDM/LA has expanded rapidly toward broader institutional and market use, but its impact on practice and theory remains limited. Future progress depends on scalable, portable analytics, responsible implementation, and research that extends across populations, technologies, and data types.

CONCLUSIONS AND FUTURE TRENDS

  • EDM/LA has grown into a broad research community moving toward wider use, but its impact on practice and theory remains limited without stronger organizational support.The area has dedicated conferences and journals, and the authors expect use by researchers, instructors, administrators, and businesses worldwide.Source: CONCLUSIONS AND FUTURE TRENDS · p. 22
  • Although many specific-purpose EDM/LA tools exist, general-purpose tools that support several educational problems through one interface and greater portability of their models are still needed.Source: CONCLUSIONS AND FUTURE TRENDS · p. 23
  • Higher-education LA initiatives need transparent goals, suitable learner feedback and visualizations, and clear policies on data storage, security, evaluation, and use (Leitner et al. 2019).Policies should align with organizational principles and clarify who uses the data and what will or will not happen with the information.Source: CONCLUSIONS AND FUTURE TRENDS · p. 23
  • LA implementations must plan infrastructure and operational scalability, determine automation levels, protect privacy throughout personal-data lifetimes, and meet ethical standards for all stakeholders (Leitner et al. 2019).The authors note that the general lifetime of personal data requires further discussion.Source: CONCLUSIONS AND FUTURE TRENDS · p. 23
  • The Baker Learning Analytics Prizes identify transferability, intervention effectiveness, and interpretability as challenges, including testing model transfer, longer-term outcomes, and users’ explanations.The effectiveness challenge compares analytics-guided intervention with analytics-no-intervention and control conditions; interpretability tests whether users understand model decisions.Source: CONCLUSIONS AND FUTURE TRENDS · pp. 23–24
  • The challenges specify applicability beyond screens by modeling at least four students and four skills each to predict immediate future performance, plus tests of generalization across systems and populations.One test requires an affect detector to work in an entirely new learning system with different interactions at AUC ROC >= 0.65. Another requires quality degradation under 0.1 and performance remaining better than chance on a distinct new population.Source: CONCLUSIONS AND FUTURE TRENDS · p. 24
  • The authors envision integrating lifelong personal and educational data to personalize learning, and applying EDM/LA to emerging environments that need real-time portable analytics.They describe mobile, ubiquitous, virtual- and augmented-reality environments, holograms, and possible machine instructors using robots or conversational agents (Cerezo et al., 2019; Newton and Newton, 2019; Kloos et al., 2019).Source: CONCLUSIONS AND FUTURE TRENDS · pp. 24–25
  • The authors foresee real-time mining of brain and other multimodal data, enabled by neuroscience and pervasive neurotechnology, to deepen understanding of learning.They point to small, high-quality EEG devices and brain-computer interfaces as potential sources of neural and psychological-state data.Source: CONCLUSIONS AND FUTURE TRENDS · p. 25

Related papers

Loading 2402.07956v1…