Source-linked AI summary
Incorporating Cognitive Load and Knowledge Transfer for Multi-Domain Knowledge Tracing
Haotian Zhang, Shucun Wang, Jinze Wu, Liang Ding, Shuochen Liu, Zhenya Huang, Jing Sha, Shijin Wang, Qi Liu
TL;DR
Multi-domain learning complicates knowledge tracing because domain switching and multi-domain questions introduce cognitive load, while knowledge states can transfer within and across domains. LT-MKT models these effects with an LLM-built hierarchical graph, cross-domain features, and knowledge propagation, and consistently outperforms representative baselines on real-world datasets. The paper constrains prerequisite relations to satisfy a directed acyclic graph property.
Problem
Multi-domain learning introduces cognitive load from temporal domain switching and multi-domain knowledge requirements, while existing methods largely focus on single-domain settings.
Method
LT-MKT constructs an LLM-assisted Multi-domain Hierarchical Graph and jointly models cross-domain cognitive-load features with within- and across-domain knowledge transfer.
Results
LT-MKT consistently achieves state-of-the-art performance across four real-world datasets and evaluation metrics, outperforming representative knowledge tracing baselines.
Takeaways & Limitations
Explicitly modeling cognitive load and knowledge transfer supports knowledge-state assessment in multi-domain learning scenarios.
Takeaways & Limitations
Prerequisite relations in the Multi-domain Hierarchical Graph are constrained to satisfy the directed acyclic graph property.
Abstract
from arXiv · showhide
Knowledge Tracing (KT) aims to assess students' dynamic knowledge states from their learning histories. While most existing KT methods focus on single-domain learning with notable success, real-world learning scenarios often involve multiple domains simultaneously, introducing two critical factors: 1) Cognitive load, arising from managing learning across domains in both temporal and knowledge dimensions. 2) Knowledge transfer, where knowledge states in one domain influence related states both within and across domains. In this paper, we focus on exploring these factors to improve students' knowledge state assessment in multi-domain learning scenarios and propose a novel method incorporating cognitive Load and knowledge Transfer for Multi-domain Knowledge Tracing (LT-MKT). Specifically, to bridge isolated domains, LT-MKT first integrates textual information from questions and their associated concepts to construct a Multi-domain Hierarchical Graph, leveraging the advanced representational capabilities of large language models (LLMs). Then, cross-domain features in both the temporal and knowledge dimensions are explicitly modeled to capture the effects of cognitive load. Additionally, a knowledge transfer module is designed to model the propagation of knowledge states within and across domains. By jointly modeling these factors, LT-MKT enables more accurate prediction of students' future performance. Finally, extensive experiments on real-world datasets demonstrate that our method achieves state-of-the-art performance.
1 Introduction
Existing knowledge tracing methods largely target single-domain learning, whereas multi-domain settings add cognitive load and knowledge-transfer challenges. LT-MKT addresses these challenges with an LLM-assisted hierarchical graph and explicit cross-domain modeling, achieving state-of-the-art performance in real-world experiments.
- Motivation: Most existing knowledge tracing methods focus on single-domain learning, despite real-world scenarios requiring simultaneous engagement across multiple domains.Multi-domain questions may require knowledge from multiple domains at once.
- Motivation: Multi-domain learning introduces cognitive load through domain switching over time and multi-domain knowledge requirements within individual questions.These effects influence students’ knowledge acquisition and evolving knowledge states.
- Method: LT-MKT incorporates cognitive load and knowledge transfer to improve knowledge-state assessment in multi-domain learning scenarios.The method explicitly captures cross-domain interactions rather than modeling knowledge states independently within each domain.
- Method: LT-MKT uses question and concept text with LLMs to construct a Multi-domain Hierarchical Graph linking heterogeneous concepts across domains.The graph establishes semantic relationships that support subsequent cross-domain modeling.
- Results: Experiments on four real-world datasets show that LT-MKT achieves state-of-the-art performance against 11 baselines.The evaluation is presented as validating explicit modeling of cognitive load and knowledge transfer.
2 Related Work
Knowledge tracing has progressed from statistical models to deep, memory-enhanced, graph-based, attention-based, and LLM-assisted approaches. Recent LLM-based KT methods use educational text to improve concept identification and question-difficulty estimation, particularly in data-sparse settings.
- Knowledge Tracing: Early knowledge tracing methods used hidden Markov models and logistic functions to estimate students’ knowledge states.Deep Knowledge Tracing later introduced neural networks into KT.
- Knowledge Tracing: Deep Knowledge Tracing initiated neural-network-based KT, followed by memory-augmented, graph-based, attention-based, and specialized architectural variants.These variants model past interactions, concept relations, key interactions, or auxiliary information.
- LLMs in Education: LLMs have shown strong performance across educational tasks and near student-level proficiency on standardized mathematics and physics examinations.The passages identify tutoring, writing assistance, and reading comprehension as potential educational applications.
- LLMs for KT: LLM-SBCL analyzes question text within student-question interaction networks to identify underlying concepts, especially in cold-start scenarios with limited data.This use targets semantic representation and concept identification.
- LLMs for KT: DCL4KT+LLM estimates question difficulty from question stems and associated concepts, addressing missing difficulty annotations for unseen questions.The method uses textual educational resources as auxiliary information for KT.
3 Problem Definition
Multi-domain knowledge tracing predicts a student’s next-question correctness from a history of question responses across domains. Unlike single-domain KT, it must account for domain switching and questions requiring concepts from multiple domains.
- Setting: An intelligent tutoring system is modeled with students, questions, and knowledge concepts distributed across multiple domains.These sets are denoted S, Q, C, and D, respectively.
- Learning History: A student’s learning history is represented as a sequence of domain-specific question–response interactions.The sequence records each answered question and its response correctness over time.
- Learning History: Each interaction identifies a question’s domain and records correctness as r_t=1 for correct or r_t=0 otherwise.The domain label specifies which domain contains the answered question.
- Prediction Objective: The prediction objective is the probability that a student correctly answers the next question from a target domain given the historical interaction sequence.This formalizes multi-domain KT as next-response prediction.
- Problem Complexity: Compared with single-domain KT, multi-domain KT includes domain switching and questions requiring knowledge from multiple domains, complicating knowledge-state modeling.These cross-domain behaviors introduce additional factors into the tracing problem.
4 The LT-MKT Model
LT-MKT builds a unified graph for multi-domain concepts, models cognitive load from question and cross-domain learning factors, and transfers knowledge within and across domains before predicting performance.
- Hierarchical Graph Construction: LT-MKT constructs a Multi-domain Hierarchical Graph linking questions to concepts, intra-domain prerequisites, and inter-domain correlations.LLM-guided reasoning identifies associated, prerequisite, and cross-domain correlated concepts from question and domain information.
- Prediction and Objective Function: The resulting representations are fused to predict the student’s performance on the next interaction.The overall framework combines graph construction, cognitive-load modeling, state evolution, and domain-aware transfer before prediction.
- Hierarchical Graph Construction: The graph can be incrementally extended for unseen concepts while constraining prerequisite relations to a directed acyclic graph.The DAG constraint avoids circular learning dependencies, and the same prompt-and-aggregation procedure supports extension.
- Cognitive Load Driven State Evolution: Question difficulty is estimated from the proportion of students answering incorrectly, with larger incorrect proportions indicating greater difficulty.The method discretizes difficulty using a predefined granularity level and represents it with an embedding matrix.
- Cognitive Load Driven State Evolution: LT-MKT models cognitive load through question difficulty, recent domain transitions, and the number of domains covered by each question.These factors are embedded into interactions before GRU-based knowledge-state evolution.
- Domain Aware Knowledge Transfer: Knowledge transfer proceeds through intraGAT prerequisite propagation followed by interGAT cross-domain semantic interaction modeling.Only the primary concept and its associated intra-domain concepts are updated to limit unnecessary propagation noise.
5 Experiments
The experiments evaluate LT-MKT on four real-world multi-domain learning datasets using representative knowledge-tracing baselines. Dataset statistics indicate that students frequently engage in cross-domain learning.
- Experiments use four real-world multi-domain learning datasets, including Java&Python and C&DS records derived from PTADisc.
- Students frequently engage in cross-domain learning in real-world multi-domain educational settings.
- Eleven representative knowledge-tracing models were selected as baselines for evaluating LT-MKT.
5.3 Experimental Setup
The experiments compare LT-MKT with baselines under specified training and implementation settings, then assess overall performance and component contributions through ablation studies. LT-MKT achieves the best reported performance, while removing modeled components reduces performance.
- Overall Performance: LT-MKT is compared with eleven representative baselines on the student performance prediction task.
- Overall Performance: The complete LT-MKT model achieves the best performance across all datasets and evaluation metrics.
- Ablation Study: The ablation study removes cognitive load, transfer graph, or state fusion components from LT-MKT.
- Ablation Study: Removing any component causes a performance decline, with cognitive load and knowledge transfer producing larger effects than state fusion.
5.6 Effect of the Graph Construction
The paper evaluates the quality of the Multi-domain Hierarchical Graph generated by different LLM backbones using expert judgments of educational and structural properties. Larger and more capable models receive stronger evaluations, with GPT-5 performing best across metrics.
- Twenty experts evaluate graph quality from educational and structural perspectives using seven representative LLM backbones.
- Expert evaluation uses Educational Relevance and Structural Consistency scores on a five-point Likert scale, plus Fleiss’ 𝜅 for agreement.
- GPT-5 achieves the best performance across all graph-quality metrics, including the highest inter-annotator agreement.
- Larger and more capable LLMs consistently achieve higher educational-relevance and structural-consistency scores.
5.7 Analysis of Cognitive Load Representation
The paper analyzes cognitive-load representations and evaluates knowledge transfer in cold-start settings. LT-MKT organizes representations along a clearer load gradient and outperforms other methods when target concepts are unseen during training.
- Analysis of Cognitive Load Representation: Cognitive load is indexed from question difficulty, domain transition, and domain coverage, then divided into low-, medium-, and high-load groups.
- Analysis of Cognitive Load Representation: LT-MKT produces a clearer continuous low-to-high load gradient than LT-MKT w/o CL, whose representations are highly mixed.
- Performance under Cold-Start Scenarios: The cold-start test set contains 16% of interaction records, with previously unseen concepts used to evaluate generalization.
- Performance under Cold-Start Scenarios: In the JuniorH cold-start scenario, LT-MKT consistently outperforms all other knowledge-tracing methods on AUC and ACC.
5.9 Parameter Sensitivity Analysis
LT-MKT performs best with a moderate λ_P, while the qualitative case study illustrates how knowledge transfer and cognitive load affect inferred knowledge states.
- Parameter Sensitivity Analysis: LT-MKT achieves its best overall performance when λ_P is set to a moderate value.The analysis varies λ_P over {10, 30, 50, 70} while fixing w_s = 20.
- Parameter Sensitivity Analysis: Very small λ_P values compress empirical difficulty into coarse categories, limiting fine-grained cognitive-load distinctions.
- Case Study: Strong mastery of correlated concept c_2 increases the inferred state of c_3 despite limited direct practice.The case study attributes this increase to knowledge transfer through the concept graph.
- Case Study: Frequent domain switching can weaken immediate learning gains even for previously well-trained concepts.The representative student fails on q_11, which involves c_2, after frequent domain switching.
6 Conclusion
The conclusion presents LT-MKT as a framework that jointly models cognitive load and knowledge transfer for multi-domain knowledge tracing. Experiments on real-world datasets show consistent improvements over representative KT baselines, while future work targets richer cognitive signals and broader applications.
- LT-MKT jointly models cognitive load and knowledge transfer in multi-domain knowledge tracing.It constructs a multi-domain hierarchical graph to represent cross-domain learning relations.
- The framework captures load effects from cross-domain learning behaviors and propagates knowledge states through intra- and inter-domain relations.
- Experiments on real-world datasets show that LT-MKT consistently outperforms representative knowledge-tracing baselines.
- Future work will incorporate richer cognitive signals and investigate broader personalized-learning and intelligent-tutoring applications.
GenAI Usage Disclosure
The authors disclose limited use of generative AI tools during manuscript preparation for grammar correction and sentence refinement.
- Generative AI tools including ChatGPT and Grammarly were used strictly for grammar correction and sentence refinement.
- The paper states that no text was generated entirely by large language models.
- The authors report responsibility for the original ideas, experimental designs, data analyses, and final edited content.