Source-linked AI summary
Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues
Alexander Scarlatos, Naiming Liu, Jaewook Lee, Richard Baraniuk, Andrew Lan
TL;DR
Existing AI tutors are adapted to follow pedagogical principles but are not trained to maximize student learning throughout a dialogue. The paper scores candidate tutor utterances by predicted student correctness and pedagogical quality, then trains Llama 3.1 8B with DPO. The resulting model improves predicted student outcomes while matching GPT-4o on pedagogical aspects, though real-student evaluation remains absent.
Problem
Current AI tutors often follow pedagogical principles without directly optimizing student learning outcomes, while many rely on costly, proprietary models.
Method
The method scores candidate tutor utterances with an LLM-based student simulator and a GPT-4o pedagogical rubric, then fine-tunes Llama 3.1 8B using DPO.
Results
DPO significantly outperforms other methods on predicted student correctness and matches GPT-4o on pedagogical aspects.
Takeaways & Limitations
A small open-source LLM can be trained to increase predicted student response correctness while maintaining high pedagogical quality.
Takeaways & Limitations
The study does not experiment with real students and instead relies on a simulated student model.
Abstract
from arXiv · showhide
Generative artificial intelligence (AI) has the potential to scale up personalized tutoring through large language models (LLMs). Recent AI tutors are adapted for the tutoring task by training or prompting LLMs to follow effective pedagogical principles, though they are not trained to maximize student learning throughout the course of a dialogue. Therefore, they may engage with students in a suboptimal way. We address this limitation by introducing an approach to train LLMs to generate tutor utterances that maximize the likelihood of student correctness, while still encouraging the model to follow good pedagogical practice. Specifically, we generate a set of candidate tutor utterances and score them using (1) an LLM-based student model to predict the chance of correct student responses and (2) a pedagogical rubric evaluated by GPT-4o. We then use the resulting data to train an open-source LLM, Llama 3.1 8B, using direct preference optimization. We show that tutor utterances generated by our model lead to significantly higher chances of correct student responses while maintaining the pedagogical quality of GPT-4o. We also conduct qualitative analyses and a human evaluation to demonstrate that our model generates high quality tutor utterances.
1 Introduction
Prior AI tutoring work adapts LLMs to follow pedagogical principles, but this paper targets the additional goal of maximizing student learning outcomes. It proposes training a small open-source tutor by combining predicted student correctness with pedagogical quality.
- Prior approaches improve LLM tutors’ adherence to effective tutoring principles through expert decisions, move annotations, synthetic data, and tutor uptake.
- Existing approaches do not directly establish whether tutor behavior maximizes student learning outcomes, motivating student outcomes as a reward signal.
- The method gathers candidate tutor utterances from human tutors and LLMs, scores correctness likelihood and pedagogical quality, then fine-tunes Llama 3.1 8B with DPO.
- The approach seeks a small, open-source tutor that directly optimizes student learning outcomes while following effective pedagogical principles.
- Real-student experimentation remains necessary because the study uses a simulated student model instead.
2 Related Work
Prior dialogue tutors range from engineered cognitive systems to prompt-based and fine-tuned LLM tutors. This work connects dialogue tutoring with student-outcome modeling and rubric-based pedagogical evaluation.
- AI Tutors in Dialogues: Earlier AI tutors used cognitive models, targeted questions, semantic analysis, or proof construction, whereas LLM tutors adapt more flexibly through prompting and fine-tuning.
- Student Outcome Modeling: Student outcome modeling commonly predicts next-item correctness from a student’s interaction history using knowledge tracing models.
- Student Outcome Modeling: This paper uses LLMKT to predict whether a student will answer a tutor-posed task correctly at the next dialogue turn.
- Evaluating Pedagogical Quality of LLMs: Pedagogical quality is commonly evaluated with rubrics applied by human experts or language models, and this work uses both evaluation modes.
3 Methodology
The methodology trains a small LLM to generate tutor utterances that balance predicted student correctness with pedagogical quality. It constructs preference pairs from diverse candidates, then applies distillation followed by DPO.
- Candidate generation and evaluation: The framework generates diverse tutor-utterance candidates and prefers those likely to elicit correct student responses while following pedagogical principles.Candidates come from human tutors and LLMs with different sizes and styles; LLMKT predicts correctness and a rubric evaluates pedagogy.
- Student outcome prediction: LLMKT predicts next-turn student correctness from dialogue history, prior student turns, and knowledge components in the tutor-posed task.The model serves as a student simulator for evaluating generated tutor utterances.
- Pedagogical evaluation: GPT-4o evaluates tutor utterances with rubric items covering accuracy, progress, and guidance, plus an overall 1-10 score.The rubric is intended to discourage oversimplified utterances and encourage meaningful guidance.
- Preference pair construction: The combined candidate score weights predicted correctness and normalized rubric quality, with λ = 0.5 by default to balance the objectives.The method varies λ to study the tradeoff between eliciting correct responses and following pedagogical practice.
- Preference pair construction: Preference pairs are formed only when one candidate’s score exceeds another by a threshold, using ϵ = 0.1 to reduce noisy comparisons.Pairs within the threshold are excluded while enough data are retained for DPO training.
- Model training: Training first distills GPT-4o-generated tutoring behavior into Llama 3.1 8B, then applies DPO to steer the model using preference pairs.The distilled model initializes both the trainable policy and the reference model; β = 0.1 allows greater divergence to increase predicted correctness.
4 Experimental Settings
The experiments use MathDial mathematics tutoring dialogues, compare DPO with prompted, supervised, distilled, proprietary, and human-tutor baselines, and evaluate correctness prediction alongside pedagogical quality.
- Dataset: MathDial contains tutoring dialogues about GSM8K mathematics problems in which tutors address misconceptions in incorrect student solutions.Crowd workers role-play tutors and GPT-3.5 simulates students.
- Data splits: The test set contains 588 dialogues and 3,101 tutor turns, while the train and validation sets contain 1,809 and 453 dialogues, respectively.The overgenerated training data use subsets of the train and validation splits to reduce labeling costs.
- Baselines: DPO is compared with base Llama 3.1 8B, prompted Llama, supervised fine-tuning, GPT-4o distillation, GPT-4o, and human tutor utterances.The comparison includes both open-source model variants and human or proprietary-model references.
- Results presentation: Table 2 reports test-set tutor-utterance evaluations, highlighting the best and second-best result for each metric.Its caption frames the comparison around student correctness improvement and pedagogical-rubric performance.
- Metrics: Evaluation measures predicted next-turn student correctness and pedagogical scores for accuracy, progress, error identification, strategic hinting, withholding, encouragement, and overall quality.Student correctness is estimated with LLMKT and averaged across turns.
- Implementation: The experiments use Llama 3.1 8B with LoRA adaptation, AdamW optimization, and learning rates of 1 × 10^-4 for distillation and 3 × 10^-5 for DPO.Training uses a 10% linear warmup and an effective batch size of 64.
5 Experimental Results
DPO improves predicted student correctness while preserving high pedagogical quality, and qualitative and human evaluations indicate that it generates manageable, actionable tasks. However, performance remains limited when students provide partially correct responses, and real-student learning outcomes remain untested.
- Quantitative Results: DPO improves student correctness prediction over GPT-4o by 33% while achieving similar pedagogical-rubric scores.On the MathDial test set, DPO also improves over distillation on almost all metrics.
- Quantitative Results: Human-written utterances are usually accurate but score relatively low on most other rubric items because they often provide little guidance.The SFT model trained on human utterances consequently scores lower on the rubric than distilled and base Llama models.
- Student Outcomes vs. Pedagogical Principles: Increasing λ generally raises student correctness prediction while lowering rubric scores, with λ around 0.5–0.75 balancing both objectives.Increasing λ from 0.75 to 1.0 does not improve student-outcome performance, while reducing λ from 0.25 to 0.0 lowers pedagogical performance.
- Qualitative Analysis: DPO-generated utterances pose concrete, nontrivial tasks that are more feasible for students than GPT-4o’s tasks.The model more often asks questions and can decompose difficult work into simpler actionable steps.
- Qualitative Analysis: LLM tutors may mishandle partially correct responses by overlooking student progress and focusing on irrelevant errors.The paper notes that additional techniques may be needed to address this behavior.
- Human Evaluation: DPO outperforms human tutors and GPT-4o on both human-rated student-correctness ranks and pedagogical-rubric scores, with all differences statistically significant.The human evaluation covered 50 instances from 10 dialogues, using blinded comparisons among human, GPT-4o, and DPO utterances.
- Human Evaluation: The evaluation provides preliminary evidence, but whether the trained tutors improve learning for real students remains untested.The authors identify deployment with real students as an important next step.
6 Conclusions and Future Work
The paper trains LLM tutors to improve predicted student outcomes while maintaining pedagogical quality. Its model outperforms other methods on predicted student outcomes and matches GPT-4o on pedagogical aspects, but real-student evaluation remains future work.
- The paper introduces a methodology for training LLM tutors to maximize student outcomes in dialogues while maintaining high pedagogical quality.
- Student simulation predicts the likelihood of a correct response, while GPT-4o evaluates tutor utterances with a pedagogical rubric.
- The resulting model significantly outperforms other methods for increasing predicted student outcomes and matches GPT-4o on pedagogical aspects.
- Future work should evaluate the method with real students, optimize longer-term learning outcomes, model student affect and engagement, and extend beyond mathematics.