Source-linked AI summary
AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling
Cristina Conati, Kaska Porayska-Pomsta, Manolis Mavrikis
TL;DR
AI in education must model learners to personalize support, but machine-learning approaches raise questions about interpretability and explainability. The paper draws on OLM research and examples to discuss those design considerations; initial evaluations offer indications about user preferences, trust, and model accuracy, and the authors propose this work as a starting point for broader high-stakes AI.
Problem
Method
The paper reviews OLM approaches and initial evaluation findings to identify considerations for revealing AI model information and calibrating user access and control.
Source: §4 · p. 5Results
Initial OLM evaluations offer indications about user preferences and implications for trust and model accuracy, while the authors propose OLM research as a conceptual and practical framework for ML-based models.
Source: §4 · p. 5Takeaways & Limitations
The authors propose OLM research as a starting point for developing interpretable AI in education and considering AI models in other high-stakes contexts involving human behaviour and decision-making.
Source: §4 · p. 5Further research is needed for a universal framework for interpretable AI and to understand how ML-based OLMs can support learning and development at scale.
Source: §4 · p. 5
Abstract
from arXiv · showhide
Interpretability of the underlying AI representations is a key raison d'\^{e}tre for Open Learner Modelling (OLM) -- a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research in ITS (also known as AI in Education) produced important work, which informs about how AI can be used in Education to best effects and, through the OLM research, what are the necessary considerations to make it interpretable and explainable for the benefit of learning. We argue that this work can provide a valuable starting point for a framework of interpretable AI, and as such is of relevance to the application of both knowledge-based and machine learning systems in other high-stakes contexts, beyond education.
1. Introduction of the ITS field
ITS research develops personalized educational technologies by modelling learners and tailoring support to their needs. Its student models support individualized instruction, while the potential long-term effects of interventions make education a high-stakes AI context.
1. Introduction of the ITS field
- ITS architecture includes domain, pedagogical, and student models; the student model is a defining characteristic of an ITS.These components capture the instructional domain, relevant pedagogical knowledge, and the student.Source: §1 · p. 1
- Pedagogical interventions can have potentially long-lasting effects on learning, development, and lifelong functioning, making education a high-stakes context for AI.The passage contrasts ITS modelling with educational technologies that match responses to pre-programmed solutions.Source: §1 · p. 1
- ITS systems provide personalized problem-solving support across domains, and studies report learning and grade outcomes comparable with human tutoring.Some systems are used in real-world settings, reaching several thousand U.S. schools and influencing traditional curricula.Source: §1 · p. 1
- Since the early 1970s, ITS research has used AI to personalize educational technologies to individual learners, aiming to approximate benefits of one-to-one instruction.Intelligent pedagogical agents model, predict, and monitor learner behaviours, abilities, and mental states to provide personalized help and feedback.Source: §1 · p. 1
2. New developments in ITS and the need for machine learning
New educational activities create modelling challenges that traditional knowledge elicitation may not readily address, prompting interest in machine learning. The paper argues that interpretability and explainability matter for pedagogical effectiveness and user trust, and presents OLM research as a conceptual starting point for broader high-stakes AI.
2. New developments in ITS and the need for machine learning
- ITS has traditionally relied on knowledge-based techniques because inferential transparency is important for pedagogically effective instruction.The field’s symbolic, rule-based expert-systems tradition also contributed to this reliance.Source: §2 · p. 2
- New learning activities require modelling tasks and learner abilities that may be poorly defined or difficult to capture, motivating machine-learning approaches.Examples include learning by exploration or games and learning by teaching; exploration support may require modelling effective exploration and abilities such as planning and self-assessment.Source: §2 · p. 2
- Machine learning can learn models from data when expert knowledge is difficult to obtain and predict learner states in high-dimensional, ill-defined behaviour spaces.The paper cites applications including modelling self-efficacy and emotions, predicting scientific-inquiry ability, and automatically generating hints.Source: §2 · p. 2
- Explanations can support pedagogical effectiveness by helping students understand system judgments, and can nurture student and teacher trust in AI decisions.The paper notes that systematic investigation is still needed into who benefits from interpretability, why, when, and to what extent.Source: §2 · p. 2
- The authors present OLM research as an emergent framework for interpretable and explainable educational AI, with potential relevance to other high-stakes contexts requiring user control.They point to both knowledge-based and machine-learning systems beyond education.Source: §2 · p. 2
3. Open Learner Modelling and Interpretability
Open Learner Models expose student-model content to users with different levels of access and control. Examples illustrate how interpretable representations can support learner reflection, model negotiation, and human teaching, while outcomes and trust depend on the design and support provided.
3. Open Learner Modelling and Interpretability
- A scrutable OLM lets students view an assessment, while cooperative or negotiable models support joint assessment and editable models let users change it.OLMs traditionally aim to encourage learning skills such as self-assessment and reflection, and to improve model accuracy by enabling users to adjust predictions or representations.Source: §3 · pp. 2–3
- Greater user interaction with an OLM can require more interpretable and explainable representations; a negotiation system can support disagreement with evidence and targeted questions.In the example described, the system may show prior responses indicating a misconception and ask questions to let the user try to change its assessment.Source: §3 · p. 3
- TARDIS uses machine learning to infer interview-behaviour quality from signals such as gaze, gestures, posture, and voice, and presents learners with assessments and signal timelines.The visualizations give learners a basis for reflecting on how to improve subsequent interviews.Source: §3 · p. 3, Figure 1 · p. 4
- A controlled study of 28 high-risk students reported significant improvements in key behaviours for TARDIS’s OLM condition, despite its relatively shallow information.The model’s interpretations also support practitioners’ nuanced discussions of learner behaviour in follow-up briefings.Source: §3 · p. 3
- Long and Aleven’s scrutable OLM uses Bayesian Knowledge Tracing to assess knowledge components and displays probabilities as skill bars that update with problem-solving.Students self-assess before seeing the system’s assessment; a formal user study reported significantly improved learning outcomes for students using the OLM.Source: §3 · p. 3, Figure 2 · p. 4
- An early evaluation found decreased trust in fully editable OLMs, especially among novice learners lacking confidence in their own judgments.More recent examples describe effective, engaging, and trusted learning tools when detailed system support accompanies the models.Source: §3 · pp. 3–4
- Basu et al. (2017) use a hierarchical task-and-strategy representation to compare users’ action sequences with desired and suboptimal variants and offer targeted support.The representation enables fine-grained conceptual support and step-by-step model-building interfaces; the passage reports increased confidence, buy-in to system advice, and learning outcomes.Source: §3 · p. 4
4. Discussion and Future Work
Initial OLM evaluations highlight how disclosure and user control relate to trust and learning support. The authors frame four design questions as a starting point for interpretable AI, while noting that further research is needed.
4. Discussion and Future Work
- As educational technology scales, ML-based student models raise demands for interpretability, explainability, and trustworthiness in supporting learning and teaching.Source: §4 · p. 4
- The authors present three decades of knowledge-based OLM research as a conceptual and practical starting point for ML-based education models and potentially other high-stakes settings.Source: §4 · p. 5
- In Mabbott & Bull (2006), maximum-control editable OLMs were least favored, while users preferred negotiable or co-operative models, underscoring the balance between user control and system support.Learners did not trust their own judgments and expected targeted support; without it, trust and motivation tended to dwindle.Source: §4 · p. 5
- The four design questions concern the OLM’s purpose, which model information users see, how they access or manipulate it, and who can access it.The information dimension includes model scope, representation fidelity, uncertainty, temporal views, input sources, and explanations of personalization.Source: §4 · p. 5
- The four dimensions may help OLM architects calibrate designs, but a universal interpretable-AI framework and viable ML-based OLMs still require substantial research.Source: §4 · p. 5