PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Knowledge Tracing: Models, Variants, and Applications

Shuanghong Shen, Qi Liu, Zhenya Huang, Yonghe Zheng, Minghao Yin, Minjuan Wang, Enhong Chen

arXiv:2105.15106v4cs.CYcs.LG

TL;DR

Knowledge tracing seeks to monitor students’ evolving knowledge states from behavioral data, but the field lacks a comprehensive account of its many variants and applications. This survey organizes fundamental KT models and their variants, reviews educational applications, and provides open-source resources for datasets and implementations. It concludes that KT research still faces coverage and interpretability challenges as methods become more complex.

  • Problem

    The field lacks a comprehensive survey covering KT models together with their numerous variants and applications.

  • Method

    The survey reviews fundamental KT models, variants under more stringent learning assumptions, applications, datasets, and open-source algorithm libraries.

  • Results

    The survey presents a technical taxonomy, synthesizes KT research across learning scenarios, and releases EduData and EduKTM for datasets and unified model implementations.

  • Takeaways & Limitations

    Researchers and practitioners can use the survey and its libraries to select KT models and access related datasets and baselines for specific requirements.

  • Takeaways & Limitations

    The survey may overlook representative emerging works, while growing model complexity can compromise applicability and user confidence in reliability.

Abstract

from arXiv · show

Modern online education has the capacity to provide intelligent educational services by automatically analyzing substantial amounts of student behavioral data. Knowledge Tracing (KT) is one of the fundamental tasks for student behavioral data analysis, aiming to monitor students' evolving knowledge state during their problem-solving process. In recent years, a substantial number of studies have concentrated on this rapidly growing field, significantly contributing to its advancements. In this survey, we will conduct a thorough investigation of these progressions. Firstly, we present three types of fundamental KT models with distinct technical routes. Subsequently, we review extensive variants of the fundamental KT models that consider more stringent learning assumptions. Moreover, the development of KT cannot be separated from its applications, thereby we present typical KT applications in various scenarios. To facilitate the work of researchers and practitioners in this field, we have developed two open-source algorithm libraries: EduData that enables the download and preprocessing of KT-related datasets, and EduKTM that provides an extensible and unified implementation of existing mainstream KT models. Finally, we discuss potential directions for future research in this rapidly growing field. We hope that the current survey will assist both researchers and practitioners in fostering the development of KT, thereby benefiting a broader range of students.

I. INTRODUCTION

Knowledge tracing analyzes students’ learning interactions to monitor evolving knowledge states and support individualized learning. This survey reviews the field’s models, variants, applications, datasets, implementations, and future directions.

  • Motivation: Online learning systems record exercises, answers, and supplementary interaction data to provide more intelligent educational services.Recorded side information includes response time, opportunity count, and tutor intervention.
  • Models: Existing KT research includes Bayesian, logistic, and deep learning models, with deep learning further divided by neural-network architecture.The deep-learning subcategories include recurrent, memory, self-attention, and graph neural networks.
  • Variants: The survey extends beyond fundamental models by reviewing variants that represent additional phases and factors in the learning process.These include individualization before learning, engagement during learning, forgetting after learning, and side information across learning.
  • Resources: The authors provide EduData and EduKTM as open-source resources for KT datasets, preprocessing, and unified model implementations.The libraries are intended to facilitate researchers’ and practitioners’ work.
  • Problem Definition: Knowledge tracing monitors evolving knowledge states during learning and predicts performance on future exercises.The traced states can support individualized learning schemes.

B. Categorization

The survey categorizes KT models by technical route into Bayesian, logistic, and deep learning approaches, then organizes variants by learning-process considerations. Bayesian Knowledge Tracing models latent knowledge states from observed answers under probabilistic assumptions.

  • Categorization: The taxonomy divides KT methods into Bayesian models, logistic models, and deep learning models implemented through probability models, logistic functions, and neural networks.Deep learning models are subdivided into recurrent, memory, self-attention, and graph neural-network approaches.
  • Categorization: The survey also groups variants by individualization before learning, engagement during learning, forgetting after learning, and side information across learning.These variants address distinct phases and factors in the learning process.
  • Bayesian Models: Bayesian models assume a Markov learning process and estimate latent knowledge states from observed student performance.The survey discusses Bayesian Knowledge Tracing and Dynamic Bayesian Knowledge Tracing.
  • Bayesian Knowledge Tracing: BKT is a two-state Hidden Markov Model that uses learning, forgetting, guessing, slipping, and initial-mastery probabilities.Its knowledge state is modeled as learned or unlearned, with no forgetting after mastery in the described framework.
  • Bayesian Knowledge Tracing: BKT updates mastery probability from an observed answer and uses it to estimate the probability of a correct answer on the next interaction.The update combines posterior mastery after the answer with the transition probability P(T).

2) Dynamic Bayesian Knowledge Tracing:

Dynamic Bayesian Knowledge Tracing extends Bayesian knowledge tracing by modeling dependencies among knowledge concepts rather than treating each concept independently. It represents mastery as latent variables and estimates model parameters from observed and unobserved learning-process variables.

  • Dynamic Bayesian Knowledge Tracing: DBKT represents dependencies and hierarchies among knowledge concepts within a dynamic Bayesian model.For example, mastery of KC3 can depend on prerequisite mastery of KC1 and KC2.
  • Dynamic Bayesian Knowledge Tracing: Student knowledge mastery is represented by binary latent variables estimated from learning interactions.
  • Dynamic Bayesian Knowledge Tracing: DBKT estimates parameters by maximizing the joint probability of observed answers and unobserved mastery-related variables.
  • Dynamic Bayesian Knowledge Tracing: Its log-linear formulation maps observed and latent spaces to feature vectors, with a normalizing constant and learned weights.

2) Performance Factor Analysis:

Performance Factor Analysis extends logistic knowledge tracing by modeling student performance through prior successes and failures, alongside knowledge-concept easiness. Logistic models estimate mastery probabilities from student and knowledge-concept parameters derived from learning interactions.

  • Performance Factor Analysis: PFA extends LFA by incorporating students’ prior successes and failures for each knowledge concept.
  • Performance Factor Analysis: PFA retains knowledge-concept easiness as a factor, matching the corresponding factor in LFA.
  • Performance Factor Analysis: The model uses separate coefficients for successes and failures to represent their learning rates.
  • Performance Factor Analysis: Factorization-machine extensions can encode exercise, student, knowledge-concept, and other side information through factorized feature interactions.

1) Deep Knowledge Tracing:

Deep Knowledge Tracing uses recurrent neural networks to process learning-interaction sequences and infer evolving knowledge states, while memory-aware models add explicit memory mechanisms. These approaches improve representation capacity but introduce interpretability and temporal-consistency concerns.

  • Deep Knowledge Tracing: DKT processes learning-interaction sequences with recurrent neural networks whose hidden state summarizes prior interactions and current input.
  • Deep Knowledge Tracing: For many unique knowledge concepts, DKT replaces sparse one-hot inputs with random-vector embeddings before recurrent processing.
  • Deep Knowledge Tracing: A linear mapping and activation function transform recurrent hidden states into students’ knowledge states.
  • Deep Knowledge Tracing: DKT’s shortcomings include limited interpretability, failure to reconstruct observed input, and inconsistent predicted knowledge states across time steps.
  • Memory-aware Knowledge Tracing: DKVMN uses key-value memory to correlate exercises with latent knowledge concepts, read mastery, predict performance, and update memory after responses.
  • Memory-aware Knowledge Tracing: SKVMN combines recurrent modeling with memory capacity through Hop-LSTM cells that follow latent-knowledge-concept relevance to capture long-term dependencies.

3) Attentive Knowledge Tracing:

Attentive and graph-based KT models extend sequence modeling by capturing long-term dependencies, psychometric characteristics, and relationships among knowledge concepts. These approaches update contextual or graph-structured knowledge states before predicting student performance.

  • Attentive Knowledge Tracing: Transformers capture global sequence dependencies, but SAINT adapts self-attention to KT with separate exercise and answer encoding and decoding.SAINT uses an encoder-decoder structure to improve self-attentive computation for KT.
  • Attentive Knowledge Tracing: AKT combines self-attention with psychometric Rasch-based embeddings to model exercise difficulty and context-aware exercise representations.Its monotonic attention makes each exercise representation depend on itself and previous exercises, while preserving exercise-specific characteristics.
  • Attentive Knowledge Tracing: Attentive KT models can benefit substantially from repeated interactions on the same exercises, while DTransformer targets stable knowledge-state tracing beyond next-performance prediction.Removing repeated interactions reduced AKT performance toward DKVMN, and prior work argues attentive models may primarily trace activity patterns.
  • Graph-based Knowledge Tracing: Graph-based KT uses the natural graph structure among knowledge concepts to aggregate neighboring states, update temporal knowledge states, and predict future performance.GKT’s update combines multilayer perceptrons, DKVMN-style erase-followed-by-add operations, GRU gating, and neighbor information propagation.
  • Graph-based Knowledge Tracing: Structure-based KT models use relationships among knowledge concepts to represent influence propagation and transfer of knowledge across related concepts.These models treat educational relations and concept-map topology as information or constraints for KT construction.

D. Summarization

Deep learning KT models often achieve strong performance but remain difficult to interpret because their data-driven, end-to-end knowledge-state calculations lack theoretical guidance. Researchers have therefore explored post-hoc explanation methods such as layer-wise relevance propagation.

  • D. Summarization: Deep learning KT models’ superior performance over Bayesian and logistic models is accompanied by limited interpretability and explainability.Their data-driven calculations make the modeled knowledge state difficult to understand, which can undermine perceived reliability when errors occur.
  • D. Summarization: Layer-wise relevance propagation offers a post-hoc approach for interpreting DKT by measuring relevance between its outputs and inputs.Preliminary experiments identify this approach as promising for explaining DKT.

IV. VARIANTS OF KNOWLEDGE TRACING MODELS

Variants of fundamental KT models introduce more realistic assumptions about individual differences, engagement, and other learning-process factors. The survey organizes these variants around richer representations of how students learn in real-world settings.

  • IV. VARIANTS OF KNOWLEDGE TRACING MODELS: Fundamental KT models mainly use exercises and responses, whereas variants incorporate additional factors to represent real-world learning processes more comprehensively.The survey groups variants into individualization, engagement, forgetting, and side-information categories.
  • Modeling Individualization before Learning: Individualization variants model differences in students’ learning rates or prior knowledge using personalized parameters, clustering, or student-specific representations.Examples include personalized BKT parameters, clustered KT models, DKT-DSC, FGKT, and CKT.
  • Modeling Individualization before Learning: Clustering-based KT trains different models for student groups and combines their predictions through uniform or weighted averaging.DKT-DSC assigns students with similar learning ability to groups whose knowledge states are traced by different DKT models.
  • Incorporating Engagement during Learning: Student engagement is treated as a learning-process factor because greater engagement is associated with greater knowledge gains.Engagement-aware variants use sensor-derived mental states, latent knowledge-and-engagement models, or behavioral features such as video playback actions.

C. Considering Forgetting after Learning

Forgetting-aware KT variants model knowledge decline over time, while side-information variants incorporate temporal, behavioral, textual, and intervention features. These extensions address limitations of models based mainly on exercises and responses.

  • Considering Forgetting after Learning: Forgetting-aware KT models represent declining proficiency by incorporating elapsed time, repeated interactions, intervention trials, or continuous decay functions.The survey describes day-scale BKT-Forget, exponential decay, and DKT-forget variants using multiple time-gap and trial-count features.
  • Considering Forgetting after Learning: BKT overestimates answer accuracy after at least a day because it assumes performance remains unchanged as time passes.BKT-Forget adds time-related parameters, while continuous-time extensions model exponential knowledge decay.
  • Utilizing Side Information across Learning: Side-information variants use response time, opportunity count, first actions, text content, tutor intervention, and other learning-process signals alongside exercises and responses.EKT derives semantic exercise representations with bidirectional LSTM processing, while other models represent intervention or response-time effects.
  • Utilizing Side Information across Learning: FAST replaces BKT conditional probability tables with logistic-regression parameters so complexity grows linearly rather than exponentially with the number of features.FAST uses Expectation Maximization with Features for parameter learning and focuses on emission features.

V. APPLICATIONS

KT applications span learning-resource recommendation and broader adaptive support, using inferred knowledge states to select materials and guide learning. Recommendation methods increasingly target difficulty, review, and engagement rather than simple non-mastery remediation.

  • Applications: KT applications include learning-resource recommendation, adaptive learning, and broader scenarios beyond student learning.The surveyed educational recommendation scenarios include exercises, videos, and figures.
  • Learning Resource Recommendation: Resource recommendation methods use inferred knowledge states to select exercises that advance students’ knowledge acquisition.BKT extensions incorporate exercise difficulty and multiple-attempt behavior in BKT-sequence recommendation.
  • Learning Resource Recommendation: Video-resource models incorporate video observations and learning rates, estimating whether a video imparts knowledge to support evaluation and recommendation.The model conditions transitions on whether the resource is a video or exercise.
  • Learning Resource Recommendation: Recommendation objectives can include review and exploration, smooth difficulty progression, and student engagement.These objectives extend beyond assigning non-mastered exercises.

B. Adaptive Learning

Adaptive learning uses students’ evolving knowledge states and learning rules to design materials, strategies, and pathways. Applications extend from individualized tutoring to MOOCs, cognitive-structure-aware navigation, games, crowdsourcing, and teacher exercise selection.

  • B. Adaptive Learning: Adaptive learning designs learning materials, strategies, and dynamic pathways around learning rules and students’ evolving knowledge states.It goes beyond merely providing resources.
  • B. Adaptive Learning: The ACT Programming Tutor used BKT to estimate evolving knowledge and present individualized exercises until students mastered each rule.This was among the first applications of KT to adaptive learning.
  • B. Adaptive Learning: In MOOCs, BKT was adapted by treating problems as knowledge concepts and questions as exercises, while addressing varying difficulty and multiple learning pathways.Resource influences were framed as a credit/blame inference problem.
  • B. Adaptive Learning: CSEAL combines DKT-based state tracing, knowledge-structure navigation, and actor-critic decisions to produce logical adaptive-learning paths.The navigation algorithm also reduces the search space in the decision process.
  • Broader Applications: KT also supports player modeling, volunteer training, annotator-performance prediction, and reinforcement-learning-based exercise collection.KT4Crowd surpassed traditional rating systems, while BKT monitored volunteer knowledge states.
  • Broader Applications: DragonBox users enjoyed the experience more, whereas Lynnette users performed significantly better on the test.The comparison illustrates a distinction between enjoyment and measured test performance.

VI. DATASETS AND BASELINES

The survey presents datasets spanning tutoring, mathematics, engineering, and exercise hierarchies, alongside EduData for dataset preparation and EduKTM for unified KT implementations. Dataset scope and metadata vary substantially, affecting how they support KT evaluation.

  • Datasets and Baselines: EduData downloads and preprocesses KT datasets, while EduKTM provides extensible, unified implementations of popular KT models.The libraries are intended to support researchers and practitioners conducting KT work.
  • Datasets: The datasets differ substantially in scale, subject, and learning scenario, reflecting complex practical applications for KT models.The survey summarizes their basic information and statistics in Table II.
  • ASSISTments Datasets: ASSISTments provides assessment and tutoring assistance, while its 2009–2010 dataset records students working on exercises associated with similar knowledge concepts.The latest ASSISTments2009 version fixes duplicated records, scaffolding mismatches, and repeated response sequences with different KCs.
  • ASSISTments Datasets: ASSISTments2015 contains learning records for 100 KCs but no exercise information or other side information.This restricts the available contextual information for analyses using that dataset.
  • ASSISTments Datasets: ASSISTments2017 tracks students across more than a decade, with average learning records exceeding 1,000 and rich side information.Examples include average BKT knowledge, exercise time, and first-response help requests.
  • Other Datasets: Junyi includes expert-labeled exercise hierarchies and prerequisite relationships, making it useful for research on knowledge structure in KT.It contains fewer exercises and KCs than ASSISTments datasets.
  • Other Datasets: Eedi2020 contains multiple-choice mathematics responses, while Statics2011 represents complex engineering problems by treating each sub-problem as an exercise.These datasets extend KT evaluation beyond a single educational setting.

5) EdNet Dataset:

EdNet is a large English-learning dataset family with increasingly detailed records of student actions and learning activities. The survey also describes EduKTM’s reproducibility purpose and cautions against universal baseline comparisons under fixed settings.

  • 5) EdNet Dataset:: EdNet contains 131,441,538 learning records from 784,309 students and is described as the largest public KT dataset.It records English-learning data from the Santa AI tutoring system for e TOEIC preparation.
  • 5) EdNet Dataset:: EdNet-KT1 contains basic exercise-answering logs for 784,309 students, 13,169 exercises, 188 KCs, and 95,293,926 learning records.Exercises are organized into bundles sharing a passage, picture, or listening material.
  • 5) EdNet Dataset:: EdNet-KT2 records action sequences such as entering bundles, selecting answers, and submitting responses, capturing finer-grained learning behavior.Alternating answer choices can reflect a student’s uncertainty before submission.
  • 5) EdNet Dataset:: EdNet-KT3 adds activities such as reading explanations and watching lectures, while EdNet-KT4 adds detailed actions including media controls, payments, refunds, and coupon enrollment.These activities may affect students’ knowledge states and can be analyzed as behavioral information.
  • Baselines: EduKTM addresses nonstandardized languages, frameworks, environments, and dependencies that make existing KT implementations difficult to reproduce.The library was created as a baseline implementation library for KT methods.
  • Baselines: The survey does not provide baseline performance evaluations because model settings target different learning scenarios and fixed comparisons would be unfair.It also notes that student-performance prediction does not directly reflect effectiveness in practical applications.
  • Future Directions: KT evaluation commonly relies on future-response prediction, but the survey identifies interpretability as important because students may ask why a learning decision was made.The paper presents interpretability as a future research direction.

B. Knowledge Tracing with Sparse Learning Interactions

The survey identifies sparse interactions, open-ended responses, limited feedback use, and model complexity as challenges for knowledge tracing, while reviewing broad applications and future directions. It also frames KT as a tool for modeling users’ knowledge and supporting individualized learning.

  • Sparse learning interactions: Sparse learning interactions create a data sparsity problem that researchers address with graph convolution, improved attention, and contrastive learning.The survey notes that these approaches alleviate sparsity in different ways, but comprehensive improvement remains necessary.
  • Open-ended responses: Most KT research assumes binary responses, leaving open-ended natural-language answers in subjective exercises relatively underexplored.One reviewed approach uses language models to predict precise solutions to programming questions.
  • Student feedback: Students’ feedback can provide direct indicators of their learning situation, yet few KT models incorporate feedback-related training data.The survey describes feedback as potentially useful for correcting KT results.
  • General user modeling: KT diagnoses users’ proficiency on specific skills or concepts and can therefore support user modeling beyond education, including games, sports, and recruitment.The survey describes user modeling as characterizing behaviors, personal information, and latent features to provide targeted services.
  • Survey scope and resources: The survey reviews fundamental and variant KT models, applications, datasets, algorithm libraries, and future research directions as a guide for researchers and practitioners.EduData supports KT-related datasets, while EduKTM provides extensible and unified implementations of existing KT models.
  • Limitations and future directions: The survey acknowledges that emerging KT methods may be overlooked and that increasing model complexity can compromise applicability and reliability.It also states that substantial work remains before deep-learning KT models become completely transparent.
Loading 2105.15106v4…