Source-linked AI summary
GIKT: A Graph-based Interaction Model for Knowledge Tracing
Yang Yang, Jian Shen, Yanru Qu, Yunfei Liu, Kerong Wang, Yaoming Zhu, Weinan Zhang, Yong Yu
TL;DR
Knowledge tracing must predict students’ performance while handling sparse question data, high-order question-skill relations, long-term history, and interactions between student mastery, questions, and skills. GIKT combines GCN-based representations with history recap and generalized interaction modules, and achieves the highest performance across three datasets with at least 1% higher results than other baselines.
Problem
Existing KT methods often omit question information and high-order question-skill correlations, struggle with long-term dependencies, and inconsistently model student-question and student-skill interactions.
Method
GIKT uses GCN embedding propagation, a recap module for relevant historical exercises, generalized question-skill interactions, and attention for prediction.
Results
GIKT achieves the highest performance over three datasets and at least 1% higher results than other baselines.
Takeaways & Limitations
GIKT exploits high-order question-skill relations and models mastery of new questions and related skills in a consistent way for knowledge tracing.
Takeaways & Limitations
The authors revised the realization of soft selection after finding the original implementation somewhat unreasonable, causing differences from the proceedings results.
Abstract
from arXiv · showhide
With the rapid development in online education, knowledge tracing (KT) has become a fundamental problem which traces students' knowledge status and predicts their performance on new questions. Questions are often numerous in online education systems, and are always associated with much fewer skills. However, the previous literature fails to involve question information together with high-order question-skill correlations, which is mostly limited by data sparsity and multi-skill problems. From the model perspective, previous models can hardly capture the long-term dependency of student exercise history, and cannot model the interactions between student-questions, and student-skills in a consistent way. In this paper, we propose a Graph-based Interaction model for Knowledge Tracing (GIKT) to tackle the above probems. More specifically, GIKT utilizes graph convolutional network (GCN) to substantially incorporate question-skill correlations via embedding propagation. Besides, considering that relevant questions are usually scattered throughout the exercise history, and that question and skill are just different instantiations of knowledge, GIKT generalizes the degree of students' master of the question to the interactions between the student's current state, the student's history related exercises, the target question, and related skills. Experiments on three datasets demonstrate that GIKT achieves the new state-of-the-art performance, with at least 1% absolute AUC improvement.
1 Introduction
Knowledge tracing predicts students’ answers from learning history, but existing methods often overlook question characteristics, high-order question-skill relations, long-term dependencies, and consistent student-question/skill interactions. GIKT addresses these gaps with graph convolution, history recap, and generalized interaction modules.
- Knowledge tracing predicts whether students answer new questions correctly from their previous learning history.
- Skill-based methods neglect question characteristics, despite questions sharing skills while differing in difficulty.
- Question information introduces sparsity because many questions are attempted by few students, while multi-skill relations complicate simple augmentation.
- GIKT uses a GCN to learn question and skill embeddings from high-order question-skill relations and feeds question embeddings with answer embeddings into KT models.
- A recap module selects relevant historical exercises, while generalized interactions model mastery of the target question and related skills using attention-weighted predictions.
- GIKT outperforms state-of-the-art baselines substantially across three benchmark datasets.
2 Related Work
Related KT work includes traditional probabilistic and factor-analysis models, deep sequential architectures, question-aware methods, and graph-based approaches. The paper positions GIKT as using GCNs to capture high-order question-skill, inter-question, and inter-skill relations.
- Traditional KT methods include Bayesian Knowledge Tracing and factor-analysis models such as IRT, PFA, and KTM.
- Deep KT methods use neural networks to trace knowledge states, but skill-only inputs can lose question information.
- Question-aware methods incorporate difficulty, question content, or question-skill relations, but face unavailable content, sparsity, or incomplete inter-question and inter-skill modeling.
- GCNs update graph-node representations from neighboring nodes, allowing representations to contain neighbor attributes.
- GIKT uses GCNs to extract high-order skill-question relations into representations and is presented as the first method to model these relations through a graph neural network.
3 Preliminarilies
Knowledge tracing models students’ sequential exercise performance, while the question-skill relation graph represents which skills are associated with which questions. The graph is bipartite, connecting question and skill sets through binary relations.
- An exercise is x_i = (q_i, a_i), where q_i is a question ID and a_i indicates whether the answer is correct.
- Given prior exercises X and a new question q_t, KT predicts p(a_t = 1|X, q_t), the probability of a correct answer.
- Each question corresponds to one or more skills, and each skill is usually related to many questions.
- The question-skill relation is represented as a bipartite graph G over question set Q and skill set S, with r_qs = 1 indicating an association.
4 The Proposed Method GIKT
GIKT combines graph-based question-skill representation learning with sequential state modeling, history recap, and generalized interaction for knowledge tracing. Its design addresses sparse question information and long-term dependencies by selecting relevant history exercises and modeling interactions among student states, questions, and skills.
- Optimization: GIKT trains its question, skill, and answer embeddings end-to-end and optimizes prediction with cross-entropy loss.The framework uses the predicted probability of a correct answer against the student’s true answer label.
- Embedding Propagation: GIKT uses GCN embedding propagation over the question-skill graph to encode high-order information into question and skill representations.The graph is bipartite, so multi-hop propagation connects questions with related skills and other questions sharing those skills.
- Student State Evolution: Question and answer embeddings are transformed into exercise representations, while an LSTM models sequential changes in the student’s knowledge state.The resulting hidden state represents the student’s coarse-grained skill mastery.
- History Recap Module: The recap module selects history exercises relevant to the target question because related exercises may be scattered across long, incoherent sequences.GIKT supports hard selection by shared skills and soft selection through attention-based relevance scoring and top-k selection.
- Generalized Interaction Module: The generalized interaction module models pairwise interactions among the current state, selected history exercises, the target question, and related skills.These interactions extend question mastery estimates beyond the conventional student-state–question pair and are weighted by attention before prediction.
5 Experiments
Experiments on three public KT datasets compare GIKT with established baselines and evaluate its graph, interaction, and recap components. GIKT achieves the highest performance, while ablations support high-order relations, interaction modeling, and direct history-exercise selection.
- Overall Performance: GIKT achieves the highest AUC across three datasets and exceeds other baselines by at least 1%.The comparison includes traditional machine-learning and deep-learning KT models.
- Overall Performance: Question-level DKT-Q can outperform skill-level DKT, but its performance declines on ASSIST09, where question attempts are sparse.DKT-Q is comparable or better on ASSIST12 and EdNet, yet worse on ASSIST09.
- Overall Performance: Combining question and skill information improves DKT-QS over DKT-Q and DKT except on single-skill ASSIST12.The result supports using both information types when questions can involve multiple skills.
- Embedding Propagation: Increasing GCN aggregation layers improves GIKT, supporting the usefulness of high-order question-skill relations.Performance changes only slightly from zero to one layer but improves with additional layers.
- Interaction Module: The full interaction module performs best, while removing relevant history or skill interactions reduces performance.The ablation findings indicate that relevant history states and simultaneous question-skill mastery modeling contribute to prediction.
- Recap Module: Selecting history exercises outperforms selecting hidden states, whereas attention provides broader selection coverage than hard selection.Direct exercise selection can reduce irrelevant information contained in general student hidden states.
6 Conclusion
GIKT combines high-order question-skill graph representations with history-aware interaction modeling for knowledge tracing. Its framework uses recap, generalized interactions, and attention to predict mastery of new questions and related skills.
- GIKT incorporates high-order question-skill relation graphs into question and skill representations.The framework uses graph-based representations as inputs for knowledge tracing.
- A recap module selects relevant history states to represent the student’s ability.This targets relevant exercises in the student’s history when modeling mastery.
- A generalized interaction module represents mastery of new questions and related skills consistently.The module connects the student’s current state, relevant history, target question, and related skills.
- An attention mechanism distinguishes relevant interactions for prediction.
- The experimental results show that GIKT achieves better performance.
Addendum Version
The authors revised the realization of soft selection after finding the original implementation somewhat unreasonable. The revised implementation changes some reported results from the proceedings version.
- The authors found their Section 4.4 realization of soft selection somewhat unreasonable after the camera-ready deadline.
- They adopted another suitable realization for the soft-selection strategy.
- The revised realization causes differences between this version’s results and the proceedings version’s results.