Source-linked AI summary
Neural Cognitive Diagnosis for Intelligent Education Systems
Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, Shijin Wang
TL;DR
Cognitive diagnosis must infer students’ proficiency on knowledge concepts, but existing manually designed linear interactions may miss complex student–exercise relationships. NeuralCD uses neural networks with monotonicity constraints, including Q-matrix-based NeuralCDM and text-enhanced NeuralCDM+, and experiments report accurate and interpretable results.
Problem
Existing cognitive-diagnosis methods rely on handcrafted linear interaction functions that may be insufficient for modeling complex student–exercise relationships.
Method
NeuralCD projects students and exercises into factor vectors, models their interactions with neural layers, and applies monotonicity to preserve interpretability.
Results
Experiments on real-world datasets show that NeuralCD models provide accurate and interpretable cognitive-diagnosis results.
Takeaways & Limitations
NeuralCD provides a general cognitive-diagnosis framework with Q-matrix-based NeuralCDM and text-enhanced NeuralCDM+ implementations.
Takeaways & Limitations
Positive restrictions on neural-network weights may limit approximation ability, and static models may be unsuitable when student proficiency changes during continued practice.
Abstract
from arXiv · showhide
Cognitive diagnosis is a fundamental issue in intelligent education, which aims to discover the proficiency level of students on specific knowledge concepts. Existing approaches usually mine linear interactions of student exercising process by manual-designed function (e.g., logistic function), which is not sufficient for capturing complex relations between students and exercises. In this paper, we propose a general Neural Cognitive Diagnosis (NeuralCD) framework, which incorporates neural networks to learn the complex exercising interactions, for getting both accurate and interpretable diagnosis results. Specifically, we project students and exercises to factor vectors and leverage multi neural layers for modeling their interactions, where the monotonicity assumption is applied to ensure the interpretability of both factors. Furthermore, we propose two implementations of NeuralCD by specializing the required concepts of each exercise, i.e., the NeuralCDM with traditional Q-matrix and the improved NeuralCDM+ exploring the rich text content. Extensive experimental results on real-world datasets show the effectiveness of NeuralCD framework with both accuracy and interpretability.
1 Introduction
Cognitive diagnosis infers students’ proficiency on knowledge concepts from exercise responses, supporting services such as exercise recommendation and targeted training. NeuralCD replaces handcrafted linear interactions with neural modeling while preserving interpretability through monotonicity, Q-matrix-based NeuralCDM, and text-enhanced NeuralCDM+.
- Motivation: Cognitive diagnosis infers students’ knowledge states from their exercise choices and response logs.Its reports support exercise recommendation and targeted training.
- Limitations of Existing Methods: Existing methods use handcrafted linear interaction functions, which may not capture complex student–exercise relationships.Examples include logistic functions and inner products.
- NeuralCD Framework: NeuralCD uses neural networks to model complex nonlinear student–exercise interactions while preserving explainability.Students and exercises are projected into factor vectors and modeled through multiple neural layers.
- Interpretability: Monotonicity assumptions and positive fully connected layers support interpretable factors in the framework.The assumption is derived from an educational property.
- Implementations: NeuralCDM uses traditional Q-matrices, whereas NeuralCDM+ incorporates exercise text to refine knowledge relevancy vectors.The framework also covers traditional models such as MF, IRT, and MIRT.
2 Related Work
Related cognitive-diagnosis work includes DINA, IRT, MIRT, and matrix factorization. These approaches generally rely on manually designed, often logistic-like interactions between student and exercise traits, limiting flexibility.
- Cognitive Diagnosis: DINA and IRT model correct exercise responses as interactions between student trait features and exercise trait features.DINA uses binary traits, while IRT uses continuous latent traits.
- Cognitive Diagnosis: In IRT, student ability and exercise difficulty interact through a logistic function with an exercise discrimination parameter.A simple form is sigmoid(a(θ − β)).
- Cognitive Diagnosis: MIRT extends latent traits to multiple dimensions, but its item-response functions remain largely logistic-like.
- Matrix Factorization: Matrix factorization treats students and exercises as users and items, respectively, and factors their score matrix into latent representations.
- Limitations: Traditional cognitive-diagnosis models depend on manually designed functions that are labor-intensive and restrict application scope.
3 Neural Cognitive Diagnosis
NeuralCD models student–exercise interactions with multilayer neural networks while preserving interpretable proficiency factors through monotonicity. The framework includes Q-matrix-based NeuralCDM, text-enhanced NeuralCDM+, and connections to traditional diagnostic models.
- 3.2 Neural Cognitive Diagnosis Framework: NeuralCD represents students and exercises with factor vectors and uses multilayer neural networks to learn their interaction function.Student and exercise one-hot inputs produce diagnostic factors, whose interactions are learned by interactive layers.
- 3.3 Neural Cognitive Diagnosis Model: NeuralCDM uses continuous student knowledge-proficiency vectors, Q-matrix knowledge relevancy vectors, and optional difficulty and discrimination factors.The student proficiency vector is the diagnosis result after training; Q-matrix entries indicate exercise–concept relevance, while additional exercise factors refine diagnosis.
- 3.2 Neural Cognitive Diagnosis Framework: Monotonicity constrains the interaction so that increasing any student knowledge proficiency cannot decrease the probability of a correct response.The implementation restricts the interactive-layer weights to positive values, ensuring the relevant partial derivatives remain positive.
- 3.4 NeuralCD Extension with Text Information: NeuralCDM+ refines manually labeled Q-matrix information by learning from exercise text through neural networks.The text-derived representation replaces the original knowledge relevancy vectors, while Q-matrix labels receive higher confidence through the model’s training objective.
- 3.5 Generality of NeuralCD: NeuralCD is presented as a general framework that can cover many traditional cognitive diagnostic models.The framework requires student proficiency and exercise knowledge relevancy factors, while additional factors such as exercise discrimination can be integrated when reasonable.
4 Experiments
Experiments on Math and ASSIST evaluate NeuralCD models for prediction accuracy and diagnosis interpretability. NeuralCD models generally outperform baselines while producing interpretable proficiency and difficulty estimates, though positive weight restrictions and static modeling remain boundaries for future work.
- Dataset Description: The experiments use Math and ASSIST, with response logs and knowledge-concept information available for evaluating cognitive diagnosis models.Math contains examination exercises and logs, while ASSIST is an open tutoring-system dataset.
- Dataset Description: ASSIST has higher AVG#log than Math, but their STD#log>1 values are close, supporting the stability assumption for static models.The authors therefore consider static NeuralCD models and baselines applicable to both datasets.
- Student Performance Prediction: NeuralCD models outperform almost all baselines on both datasets in student performance prediction.The comparison includes DINA, IRT, MIRT, and PMF, with evaluation using classification and regression metrics.
- Student Performance Prediction: NeuralCDM+ performs better than NeuralCDM, supporting Q-matrix refinement and more finely estimated knowledge-relevancy vectors.NeuralCDM+ incorporates exercise text content through a neural network.
- Model Interpretation: NeuralCDM and NeuralCDM+ achieve significantly higher DOA than baselines, while removing the Q-matrix or monotonicity assumption lowers DOA.These results support the interpretability of diagnosed knowledge proficiency vectors.
- Model Interpretation: A NeuralCDM case study shows exercise difficulty and student proficiency values corresponding to whether the student answers an exercise correctly.For example, proficiency 0.60 exceeds difficulty 0.35 for Ordering Fraction, and the response is correct.
5 Conclusion
NeuralCD provides an interpretable neural framework for student cognitive diagnosis by modeling complex student–exercise interactions under a monotonicity assumption. NeuralCDM uses Q-matrices, while NeuralCDM+ incorporates exercise text, and experiments support effectiveness in accuracy and interpretability.
- NeuralCD models complex student–exercise interactions with neural networks while preserving interpretability through a monotonicity assumption.
- NeuralCDM implements the framework using traditional Q-matrices, while NeuralCDM+ extends it by incorporating exercise text to refine the Q-matrix.
- Experiments on real-world datasets showed that the models were effective in both accuracy and interpretability.