PaperlayerView exact revision on arXiv

Source-linked AI summary

Knowledge Tracing: A Survey

Ghodai Abdelrahman, Qing Wang, Bernardo Pereira Nunes

arXiv:2201.06953v1cs.CYcs.AIcs.LG

TL;DR

Knowledge Tracing asks how to track a student’s evolving knowledge so teaching can be adapted to the learner. This paper surveys traditional and deep-learning KT methods, datasets, applications, and research directions, finding that deep-learning models achieve state-of-the-art results on most benchmark datasets. It also identifies dataset diversity and annotation limitations that constrain the literature.

  • Problem

    Online education requires methods that can track students’ learning progress and knowledge states from their interactions.

  • Method

    The paper provides a comprehensive survey that classifies and analyzes traditional and deep-learning KT methods, benchmark datasets, applications, and future research directions.

  • Results

    Deep-learning KT models achieve state-of-the-art results on the majority of benchmark datasets.

  • Takeaways & Limitations

    The survey organizes KT methods by theoretical and algorithmic characteristics and clarifies how deep-learning approaches extend earlier models across knowledge representation, forgetting, and architecture.

  • Takeaways & Limitations

    Public KT datasets are often concentrated in particular domains and regions, and many lack demographic information or knowledge-component annotations.

Abstract

from arXiv · show

Humans ability to transfer knowledge through teaching is one of the essential aspects for human intelligence. A human teacher can track the knowledge of students to customize the teaching on students needs. With the rise of online education platforms, there is a similar need for machines to track the knowledge of students and tailor their learning experience. This is known as the Knowledge Tracing (KT) problem in the literature. Effectively solving the KT problem would unlock the potential of computer-aided education applications such as intelligent tutoring systems, curriculum learning, and learning materials' recommendation. Moreover, from a more general viewpoint, a student may represent any kind of intelligent agents including both human and artificial agents. Thus, the potential of KT can be extended to any machine teaching application scenarios which seek for customizing the learning experience for a student agent (i.e., a machine learning model). In this paper, we provide a comprehensive and systematic review for the KT literature. We cover a broad range of methods starting from the early attempts to the recent state-of-the-art methods using deep learning, while highlighting the theoretical aspects of models and the characteristics of benchmark datasets. Besides these, we shed light on key modelling differences between closely related methods and summarize them in an easy-to-understand format. Finally, we discuss current research gaps in the KT literature and possible future research and application directions.

1 INTRODUCTION

Knowledge Tracing addresses the challenge of observing and representing students’ evolving knowledge during online learning. This survey organizes traditional and deep-learning approaches, benchmark datasets, applications, and research directions while highlighting modelling differences and limitations.

  • Knowledge Tracing aims to observe, represent, and quantify a student’s knowledge state, including mastery of skills underlying teaching materials.
  • Tracking knowledge is difficult because questions can require multiple skills, while skills may also have prerequisite dependencies.For example, one question may require multiple skills, and mastery estimates may need to include prerequisites not directly attached to the question.
  • Forgetting can cause knowledge to decay differently across skills, making recency and forgetting behavior relevant to knowledge-state modelling.The survey describes deep-learning models that incorporate forgetting features to account for this behavior.
  • Deep-learning KT models incorporate memory structures, attention, graph representations, textual features, and forgetting features to model knowledge states and question relationships.These extensions target finer-grained skill mastery, relationships among questions and skills, question-text information, and forgetting behavior.
  • Deep-learning KT models have achieved state-of-the-art results on the majority of benchmark datasets.
  • The survey reviews traditional and deep-learning KT methods, compares their architectures and assumptions, summarizes datasets and applications, and identifies future research opportunities.Its stated contributions include tracing the evolution of KT categories and consolidating results from relevant literature for dataset-level comparison.

2 CATEGORIZATION OF KNOWLEDGE TRACING MODELS

The survey categorizes knowledge tracing models into two broad groups: traditional models and deep-learning models.

  • Knowledge tracing models are broadly divided into Traditional Knowledge Tracing Models and Deep Learning Knowledge Tracing Models.

2.1 Traditional Knowledge Tracing Models

Traditional knowledge tracing models use Bayesian and factor-analysis approaches to represent and predict students’ changing knowledge states. BKT models differ in their assumptions about skills, students, dependencies, forgetting, and inference complexity.

  • Traditional KT research follows Bayesian Knowledge Tracing and Factor Analysis Models.Figure 3 summarizes major traditional models in the literature.
  • Bayesian Knowledge Tracing: Standard BKT represents each skill with binary learned and unlearned states, updating mastery from question-answer observations.Its parameters model learning, slipping, guessing, and the initial mastery probability.
  • Bayesian Knowledge Tracing: Standard BKT assumes students share the same prior knowledge and learning rates, which can underestimate above-average students and overestimate below-average students.Individualized BKT introduces student-specific parameters to address this limitation.
  • Bayesian Knowledge Tracing: Individualized BKT models provide better correlation between actual and expected student accuracy and can improve performance prediction.Their skill- and student-specific parameters are summarized in Table 2.
  • Bayesian Knowledge Tracing: Dynamic BKT extends earlier models by jointly representing multiple skills and dependencies between prerequisite skills.It adds a forgetting probability and learns dependency weights within a Dynamic Bayesian Network.
  • Factor Analysis Models: Factor analysis models use Item Response Theory to estimate student performance from factors associated with students and problems.Their logistic formulations provide the theoretical basis for models such as IRT, AFM, and PFA.
  • Discussion: BKT inference commonly assumes a first-order Markov chain, while dynamic variants can become computationally intractable and trade prediction accuracy for efficiency.The first-order assumption uses only the most recent observation.

2.2 Deep Learning Knowledge Tracing Models

Deep learning KT models formulate knowledge tracing as sequence prediction and progressively expand the representation of student knowledge, question relationships, and contextual features. The survey highlights recurrent, memory-augmented, text-aware, and dependency-aware designs alongside persistent modelling limitations.

  • Sequence Modeling for Knowledge Tracing: Deep learning KT models apply sequence modelling to predict a student’s probability of answering a new question correctly from prior interactions.KT observes question-answer pairs and estimates the student’s knowledge state over time.
  • Sequence Modeling for Knowledge Tracing: DKT uses recurrent hidden states to encode previous interactions, but assumes a single hidden knowledge component and cannot model relationships among multiple components.It also assumes all questions are equally related within the interaction sequence.
  • Memory-Augmented Knowledge Tracing Models: Memory-augmented models use key-value structures to represent complex knowledge components, with DKVMN keeping keys static while dynamically updating values after each response.DKVMN computes question–latent-component correlations, retrieves the relevant knowledge state, predicts the response, and updates memory after the answer.
  • Memory-Augmented Knowledge Tracing Models: SKVMN adds Hop-LSTM to capture sequential dependencies among relevant questions while skipping LSTM cells associated with irrelevant questions.This design targets long-term dependencies among questions requiring similar knowledge components.
  • Feature Augmentation: EERNN extracts question representations from text with a bidirectional LSTM and combines them with previous-question representations to trace knowledge states.
  • Discussion: Deep KT models differ in whether they represent one or multiple knowledge components, how they learn component dependencies, and whether they incorporate forgetting or other features.Attention and graph models address dependencies, while temporal and textual augmentation can improve prediction but requires corresponding database features.

3 KNOWLEDGE TRACING DATASETS

The survey reviews benchmark datasets by documenting their interactions, questions, students, knowledge components, and availability. It also identifies comparability, versioning, annotation, and diversity problems that complicate KT evaluation.

  • Dataset Overview: The dataset review reports general characteristics such as student interactions, numbers of questions, and data availability across publicly available KT benchmarks.The authors downloaded, inspected, and summarized the datasets used in the literature.
  • ASSISTments: ASSISTments datasets are the most popular KT benchmarks and collectively contain the largest number of questions among the reviewed datasets.They comprise longitudinal tutoring data and multiple versions collected during different periods.
  • ASSISTments: ASSISTments2012 contains 6,123,270 interactions from 46,674 students and 179,999 distinct questions, but 126,908 questions lack associated knowledge components.
  • Considerations: Benchmark datasets often use inconsistent knowledge-component counts, question–component associations, preprocessing, and task settings, hindering correct interpretation and direct comparison.The survey notes that different subsets or preprocessing procedures can produce non-comparable reported performance.
  • Considerations: Public KT datasets lack diversity because they are concentrated in mathematics and a small number of regions, while demographic information is often unavailable.This limits analyses such as gender-based or other similar predictions.
  • Considerations: Dataset updates are not always readily identifiable, so the survey recommends version control to support consistent and comparable results.

4 KNOWLEDGE TRACING APPLICATIONS

KT applications use estimated knowledge states to personalize educational experiences, organize curricula, support interactive learning, and guide teaching of machine-learning agents. The survey also describes knowledge tracking as a component of learning-to-teach systems.

  • Application Areas: The survey groups KT applications into recommender systems, learning provision and quality assurance, interactive learning, and learning to teach.
  • Recommender Systems: Online education systems can use traced knowledge states to tailor learning materials, exercises, and student grouping to learners’ capabilities and skills.One described approach constructs a dynamic knowledge graph from student knowledge vectors and mastery similarities around a selected component.
  • Learning Provision and Quality Assurance: KT models can support curriculum provision by simulating learning outcomes from historical exercise records to identify ordered materials intended to maximize students’ knowledge gain.
  • Interactive Learning: Educational games can use KT to assess player progress and adjust challenge difficulty, unlock game parts, or change computer-opponent competency.The cited applications connect knowledge assessment with adapting the gaming experience.
  • Learning to Teach: Learning-to-teach systems can treat machine-learning models as students and use their tracked knowledge states to customize training data, architectures, or loss functions.The KT output can form the state for a teacher model optimizing the student model’s training procedure.
  • Learning to Teach: KADT dynamically tracks latent knowledge components and uses attention pooling to select significant training samples, outperforming other state-of-the-art machine teaching methods across four tasks.The evaluated tasks were knowledge tracing, sentiment analysis, movie recommendation, and image classification.

5 KNOWLEDGE TRACING FUTURE RESEARCH DIRECTIONS

The survey identifies limitations in current KT models and datasets, then outlines directions involving richer representations, weaker supervision, interactive questioning, and explainability.

  • Multimodal and informative representation learning & datasets: Current KT models and datasets often overlook informative multimodal and domain-specific data, motivating richer representations and new contextual benchmark datasets.Images, mathematical equations, and code snippets remain largely unexplored, while datasets should span varied domains, cultures, and educational levels.
  • Self-supervised learning in knowledge tracing: Self-supervised learning could support informative KT representations and address limited, cold-start, or skewed student-activity data.The survey proposes pre-trained models and examines whether SSL can mitigate cases where few students contribute to most activities.
  • Interactive knowledge tracing: Interactive KT remains underexplored because most models passively observe response histories rather than actively probing students’ knowledge states.The survey proposes optimized question-sampling policies, particularly for cold-start scenarios, with reinforcement learning as a possible approach.
  • Explainable Artificial Intelligence: Explainable KT is proposed to improve transparency for educational stakeholders and clarify how model predictions and algorithmic decisions affect learning processes.Suggested directions include explaining prediction processes and studying impacts on course design, instructor performance, learning materials, and student engagement.

6 CONCLUSION

The conclusion presents the paper as a comprehensive KT synthesis that categorizes methods, reviews deep learning and datasets, consolidates performance results, and identifies applications and future directions.

  • 6 CONCLUSION: The paper organizes the KT literature chronologically and categorizes methods according to shared theoretical and algorithmic aspects.It covers early KT attempts and uses the historical landscape to structure the review.
  • 6 CONCLUSION: The survey deeply reviews deep-learning KT approaches across representation learning, forgetting behavior, and architecture design, while tracing how later methods extend earlier ones.The chronological flow is used to clarify limitations and research directions.
  • 6 CONCLUSION: The paper reviews KT datasets, describes their characteristics and limitations, and curates reported performance results for key methods across covered datasets.This combines dataset analysis with a consolidated view of results from the literature.
  • 6 CONCLUSION: The survey discusses KT applications in human and machine teaching and highlights future directions using multimodal data, weak or no supervision, and interactive reinforcement learning.These directions are presented as ways to address cold-start challenges and extend current KT methods.
Loading 2201.06953v1…