Source-linked AI summary
EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education
Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He, Xipeng Qiu
TL;DR
Educational LLMs face gaps in domain knowledge, current information, and truthful generation, motivating EduChat’s domain-specific design. EduChat combines educational pre-training, instruction fine-tuning, theory-informed skills, and retrieval, and reports strong C-Eval performance while supporting open QA, essay assessment, Socratic teaching, and emotional support.
Problem
Educational LLMs lack sufficient domain knowledge and current information and may generate untruthful responses.
Method
EduChat combines educational-corpus pre-training, customized instruction fine-tuning, theory-informed tool use, and retrieval-augmented answering.
Results
EduChat reports strong C-Eval performance and improves education-specific functions while maintaining comparable foundational capabilities to similarly sized large models.
Takeaways & Limitations
EduChat provides an open-source system for personalized educational support spanning open QA, essay assessment, Socratic teaching, and emotional support.
Abstract
from arXiv · showhide
EduChat (https://www.educhat.top/) is a large-scale language model (LLM)-based chatbot system in the education domain. Its goal is to support personalized, fair, and compassionate intelligent education, serving teachers, students, and parents. Guided by theories from psychology and education, it further strengthens educational functions such as open question answering, essay assessment, Socratic teaching, and emotional support based on the existing basic LLMs. Particularly, we learn domain-specific knowledge by pre-training on the educational corpus and stimulate various skills with tool use by fine-tuning on designed system prompts and instructions. Currently, EduChat is available online as an open-source project, with its code, data, and model parameters available on platforms (e.g., GitHub https://github.com/icalk-nlp/EduChat, Hugging Face https://huggingface.co/ecnu-icalk ). We also prepare a demonstration of its capabilities online (https://vimeo.com/851004454). This initiative aims to promote research and applications of LLMs for intelligent education.
1 Introduction
EduChat addresses educational LLM challenges through domain-specific training, psychology- and education-informed skills, retrieval, and tool-controlled interaction. It is released as an open-source system with functions intended for realistic educational scenarios.
- EduChat targets educational LLM gaps in domain knowledge, knowledge freshness, and truthful generation.
- It pre-trains on educational books and four million cleaned instructions, then fine-tunes on 500,000 customized instructions aligned with expert and teacher feedback.
- EduChat provides functions including essay assessment, Socratic teaching, emotional support, and retrieval-augmented open question answering.
- Psychology and education theories inform domain adaptation, while system prompts and instructions control tool use and stimulate education-specific skills.
- The project releases its system, data, and model resources to accelerate research and applications in intelligent education.
2 Related Work
Related work shows that general LLMs achieve strong language capabilities but lack the specialized expertise required for practical domain assistance. This motivates domain-specific models for fields such as healthcare, finance, law, and education.
- General LLMs demonstrate strong language generation and understanding through large-scale pre-training and instruction tuning.
- Their lack of subject-matter expertise limits application in specialized domains where practical users require comprehensive domain knowledge.
- Existing specialized models target healthcare, finance, and legal applications, establishing a precedent for domain-focused language systems.
3 Core Functions of EduChat
EduChat combines retrieval-augmented question answering with specialized functions for essay assessment, Socratic teaching, and psychology-based emotional support. These functions emphasize current information, detailed feedback, independent thinking, and personalized assistance.
- Retrieval-Augmented Open Question Answering: Retrieval-augmented open QA uses updated internet corpora and self-assessed relevance to reduce fabrications and maintain current knowledge.
- Fine-grained Essay Assessment: Essay assessment provides overall and aspect-level ratings, detailed comments, and identified standout sentences for personalized writing guidance.
- Socratic Teaching: Socratic teaching uses multi-step dialogue, questions, challenges, and analyses rather than direct answers to encourage independent thinking.
- Psychology-based Emotional Support: The emotional-support framework draws on REBT and ABC theory to simulate counseling with personalized diagnoses and support.
4 Data Construction
EduChat’s data construction combines educational, foundational instruction, retrieval, emotional-support, Socratic-teaching, and essay-assessment data. The datasets are expanded, curated, manually evaluated, and deduplicated to support the system’s educational functions.
- 4.1 Pre-training Data: Educational pre-training data includes textbooks, Chinese exam question banks, over 70,000 poems, and selected psychology books.
- 4.1 Pre-training Data: Foundational instruction data combines bilingual task data and multi-turn dialogues from several open-source repositories.
- 4.2 Fine-tuning Data: Educational instruction data covers retrieval-augmented QA, emotional support, Socratic teaching, and essay assessment.
- 4.2 Fine-tuning Data: Retrieval QA data uses ChatGPT scoring and self-checking to filter irrelevant retrieved content before answer generation.
- 4.2 Fine-tuning Data: Emotional-support data combines translated and expanded ESConv dialogues with manually reviewed and real-life Chinese counseling data.
- 4.2 Fine-tuning Data: Socratic dialogues are manually evaluated for accuracy, fluency, and progression, while essay data receives pedagogical-expert curation of model-generated assessments.
- 4.2 Fine-tuning Data: Semantic deduplication removes one item from embedding pairs whose cosine similarity exceeds 0.7, with GPU acceleration used for computation.
5 EduChat
EduChat combines educational pre-training, task-specific fine-tuning, online retrieval, and system prompts to support domain knowledge and pedagogical skills. Its demo exposes scene-specific functions for interactive assistance to students, teachers, and parents.
- 5 EduChat: EduChat first pre-trains on large-scale educational data, then fine-tunes on curated task-specific datasets to acquire foundational knowledge and educational skills.The two-stage training covers educational books, Q&A pairs, instruction tuning, open-domain dialogue, retrieval-augmented QA, emotional support, Socratic teaching, and essay assessment.
- 5 EduChat: Online retrieval improves the accuracy and timeliness of EduChat’s responses, while self-checking filters retrieval results for usefulness before generation.The model asks whether retrieved information helps answer the question and appends useful snippets to the dialogue history.
- 5 EduChat: System prompts combine personal profile, tool availability, and skill selection to activate scene-specific abilities such as Socratic teaching and emotional support.Tool settings can enable retrieval or disable calculators, while function names select the communication skill required by the scenario.
- 5 EduChat: The demo lets users choose functions such as Open QA and Emotional Support, each using a scene-specific prompt for interactive conversations.The system is intended to assist students, teachers, and parents and is described as adapting through user interactions.
6 Experimental Results
EduChat is evaluated on C-Eval and illustrated through retrieval-augmented QA, Socratic teaching, and emotional-support cases. It outperforms a comparable Chinese Alpaca-13B model by over seven points and demonstrates several education-specific interaction modes.
- 6.1 Results of C-Eval: C-Eval contains 13,948 multiple-choice questions across 52 disciplines and four difficulty levels, providing a broad evaluation suite for foundation models.The supplied passage introduces Table 1 as the source of the reported benchmark results.
- 6.1 Results of C-Eval: Over seven points: EduChat outperforms Chinese Alpaca-13B on C-Eval, despite both models using the LLaMA-13B base model.The comparison concerns models with similar parameter scales, including Chinese Alpaca-13B and WastlackLM.
- 6.2 Case Studies: Retrieval-augmented open QA provides answers with retrieved relevant information, while Socratic teaching guides students through problems step by step.These capabilities are illustrated in Figure 4 cases.
- 6.2 Case Studies: In emotional-support cases, EduChat interacts like a psychological counselor rather than offering general advice.Additional psychology-based support and fine-grained essay-assessment cases are provided in the appendix.
7 Conclusion
The paper presents EduChat as an education-focused LLM chatbot designed to provide personalized, fair, and compassionate support. It combines psychology- and education-informed functions with educational pre-training and task-specific fine-tuning, and reports strong C-Eval performance.
- 7 Conclusion: EduChat is an LLM-based chatbot system intended to provide personalized, fair, and compassionate support to teachers, students, and parents.Its educational functions include open QA, essay assessment, Socratic teaching, and emotional support.
- 7 Conclusion: Educational-corpus pre-training and task-specific instruction fine-tuning support EduChat’s education-oriented functions and reported C-Eval performance.The conclusion identifies psychology and education theories as part of the system’s design.