PaperlayerView exact revision on arXiv

Source-linked AI summary

EduWild: How Teacher-Configured Chatbots Fail in Real-World Use and How to Fix Them

Ziqi Liu, Ruiwei Xiao, Yuxuan Li, John Stamper

arXiv:2610.23530v1cs.CL

TL;DR

Evidence about failures of teacher-created chatbots across real-world tasks is limited, despite the need for real-world conversations to identify them. The paper introduces EDUWILD, applies a 29-code taxonomy with a human-calibrated LLM judge, and evaluates a prompt-level harness informed by observed failures. A harness targeting these failures reduced the mean number of target failure codes per conversation by 65.2% in a study with 200 participants.

  • Problem

    Evidence about failures of teacher-created chatbots across real-world tasks is limited, despite the need for real-world conversations to identify them.

    Source: §1 · p. 1

  • Method

    The paper introduces EDUWILD, applies a 29-code taxonomy with a human-calibrated LLM judge, and evaluates a prompt-level harness informed by observed failures.

    Source: §1 · pp. 1–2, §4.2 · p. 6

  • Results

    A harness targeting these failures reduced the mean number of target failure codes per conversation by 65.2% in a study with 200 participants.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Real-world failure analysis can guide practical improvements to educational chatbot behavior.

    Source: §6 · p. 9

    The dataset comes from adult learners using one platform, while the study covers ten frequently used bots and participants role-playing intended users rather than pursuing their own coursework.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Teachers increasingly configure educational chatbots through natural-language instructions, but how these bots fail in real-world use remains poorly understood. We introduce EduWild, a dataset of 1,726 conversations with 217 teacher-configured chatbots, each paired with the instructions and configuration behind it, including 900 conversations from the intended end users of 35 deployed chatbots. We develop a 29-code taxonomy covering factual, workflow, diagnostic, and pedagogical failures and apply it using a human-calibrated LLM judge. The most frequent failures are lapses in following teacher intent --- bots skip required workflow steps (53.8% of end-user conversations), omit expectation setup (33.4%), and overload learners with information or questions (37.1%). Guided by these findings, we design a prompt-based harness targeting the six most frequent failure modes. In a study with 200 college students across ten bots, the harness reduces the mean number of target failure codes per conversation from 1.406 to 0.489, a 65.2% relative reduction with bots weighted equally. These findings show how real-world failure analysis can guide practical improvements to educational chatbots.

1 Introduction

The paper addresses limited evidence about how teacher-created educational chatbots fail across real-world tasks. It introduces EDUWILD and a failure taxonomy, then uses observed failures to guide and evaluate a prompt-level harness.

1 Introduction

  • Real-world conversation data are needed to identify failures in teacher-created chatbots across tasks and improve their behavior.Prior datasets and controlled benchmarks offer limited insight into these bots’ everyday failures.Source: §1 · p. 1
  • EDUWILD pairs 1,726 conversations from 217 teacher-configured chatbots with their instructions and configuration, enabling assessment against teacher intent.A 29-code taxonomy covers factual, workflow, diagnostic, and pedagogical failures.Source: §1 · pp. 1–2
  • The prompt-level harness targets task requirements, expectation setting, and manageable responses and questions.It is informed by frequent failures found in the conversation analysis.Source: §1 · pp. 1–2
  • The harness reduced distinct target failure modes per conversation by 65.2% in a 200-participant study across ten chatbots.The study compared the original chatbots with versions using the harness.Source: §1 · p. 2

2 Related Work

Prior work spans pedagogical dialogue systems and educational conversation datasets, but the paper focuses on deployed chatbots executing diverse teacher-authored roles and where that execution fails.

2.1 Educational Dialogue and Pedagogical Control

  • Educational dialogue systems range from mixed-initiative tutoring to LLMs following pedagogical instructions and tools supporting human tutors.Classroom evidence that tutoring guardrails affect subsequent learning motivates evaluating instructional behavior alongside answer quality.Source: §2.1 · p. 2

2.2 Educational Conversation Data and Evaluation

  • Existing resources include simulated-student and real tutoring conversations, pedagogical benchmarks, and simulated-student evaluations, while EDUWILD adds deployed teacher-configured bots.It supports conversation-level failure analysis against teacher instructions and mitigation evaluation with human participants.Source: §2.2 · p. 2

3 EDUWILD Dataset

EDUWILD combines teacher-authored chatbot configurations with authentic conversations, retaining data from deployed bots and their intended users. Its comparison with prior datasets emphasizes real-world deployment and paired authoring data.

3.1 Data Collection

  • EDUWILD was collected through a nine-week online professional-development program where educators built and could share Playlab chatbots.The exported authoring data include prompts and configurations; conversation exports include transcripts, speaker roles, timestamps, and formatting.Source: §3.1 · p. 2
  • After excluding research-team tests, a chatbot without a clear educational purpose, and empty conversations, the dataset contains 217 chatbots by 85 educators and 1,726 conversations.The retained data include 35 deployed chatbots and 900 end-user conversations; failure analysis focuses on those end-user conversations.Source: §3.1 · pp. 2–3

3.2 Dataset Statistics

  • The dataset pairs authoring data for 217 chatbots with 1,726 transcripts, and analyses focus on 35 deployed bots and their 900 end-user conversations.End users are primarily students; the dataset also contains author and peer-educator conversations.Source: §3.2 · p. 3
  • Deployed chatbots had longer system prompts than the overall set, and deployed bots more often attached reference files.The reported median prompt lengths were 296 words overall and 500 for deployed bots; file attachments were 70% overall and 91% among deployed bots.Source: §3.2 · p. 3
  • The five most active deployed chatbots account for 56.4% of end-user conversations, motivating equal chatbot weighting in later analyses.End-user use also follows the academic calendar, and 12.4% of conversations resume after a gap exceeding 30 minutes.Source: §3.2 · p. 4

3.3 Comparative Analysis

  • Compared with prior datasets, EDUWILD captures real-world deployment with intended users and pairs conversations with chatbot authoring data.This pairing permits evaluation against educator intent and tracing educators’ chatbot-design iterations, which no existing dataset supports.Source: §3.3 · p. 4, Table 2 · p. 4
  • Figure 1 groups and sorts the prevalence of all 29 failure codes by taxonomy category, using the 900 end-user conversations as its denominator.Saturated bars identify the six codes above 10% that the harness targets.Source: Figure 1 · p. 5

4 Failure Modes

EDUWILD pairs real-world chatbot conversations with teacher instructions to identify failures against intended behavior. Its taxonomy and analysis show that these failures extend beyond incorrect answers to workflow, interaction-load, and instructional-support problems.

4 Failure Modes

  • EDUWILD’s failure analysis covers 900 end-user conversations, and the taxonomy table presents its failure-mode categories.The dataset pairs conversations with teacher prompts so chatbot behavior can be judged against intended behavior.Source: §4 · p. 5, Table 3 · p. 6

4.1 Theory-backed Failure Mode Taxonomy

  • The 29-code taxonomy groups factual, workflow, diagnostic, and pedagogical failures, drawing on prior frameworks and refining categories against observed conversations.Coding uses observable conversational evidence and the task-specific requirements in each teacher’s prompt; multiple codes can apply to a turn.Source: §4.1 · p. 5

4.3 Analysis

  • 53.8% of end-user conversations skip required workflow steps, compared with 10.1% containing incorrect content; absent expectation setup affects 33.4%.Workflow omissions and expectation setup co-occur in 25.8% of conversations; guardrail breaches affect 16.6%.Source: §4.3 · p. 6
  • 37.1% of conversations contain cognitive or multi-question overload, which captures distinct demands on information processing and question pacing.The two labels overlap in 100 conversations, supporting separate attention to response volume and question pacing.Source: §4.3 · p. 6
  • Early answer dumping (8.6%) and over-scaffolding (5.7%) reflect opposing mismatches in the amount and timing of help.Early answers often co-occur with missing prior diagnosis, while over-scaffolding often accompanies repetition loops; the patterns vary across bots.Source: §4.3 · pp. 6–7

5 Mitigation

The authors wrap each teacher’s original chatbot in a harness that targets frequent failure modes, then evaluate it against the original chatbot. In a 200-participant study, the harness reduces target failures, while early answer dumping increases.

5.1 Harness

  • The harness wraps the teacher’s unchanged prompt and targets W7, W9, P3, W6, F5, and F1 with failure-specific prompts, an opening self-introduction, and persistent teacher-intent memory.The six target codes are those occurring in more than 10% of conversations.Source: §5.1 · p. 7

5.2 Evaluation

  • The evaluation randomly assigned 200 participants to control or harness chatbots, using the ten bots with the most end-user conversations in EDUWILD.The study included 100 participants per condition.Source: §5.2 · pp. 7–8
  • The mean number of distinct target failure codes falls from 1.406 in control to 0.489 with the harness, a 65.2% relative reduction.Across all 29 codes, the mean decreases from 2.968 to 0.991; the proportion with at least one target failure falls from 84.7% to 55.7%.Source: §5.2 · p. 8, Table 4 · p. 8
  • Cognitive overload decreases from 39.7% to 7.6%, and multi-question overload from 27.7% to 1.1%; missing workflow steps remain the most prevalent target failure.All six target codes have lower observed prevalence with the harness, but missing workflow steps fall only from 43.9% to 32.2%.Source: §5.2 · p. 8
  • Early answer dumping increases from 6.0% to 9.8%, alongside decreases in over-scaffolding and repetition loops.The authors describe this as a trade-off and conclude that the amount of help must adapt to the learner’s current state.Source: §5.2 · p. 8

6 Conclusion

The conclusion presents EDUWILD and its failure taxonomy as a basis for analyzing teacher-configured educational chatbots. The authors report that a targeted harness reduced failures in a participant study.

6 Conclusion

  • EDUWILD contains 1,726 real-world conversations with 217 teacher-configured educational chatbots, paired with the instructions behind them, alongside a 29-code failure taxonomy.Source: §6 · p. 8
  • In 900 end-user conversations, failures to follow teacher-defined workflows and boundaries outnumber incorrect content, alongside difficulties managing interaction load and timing support.Source: §6 · p. 9
  • A harness targeting observed failures reduced the mean number of target failure codes per conversation by 65.2% in a 200-participant study.The largest absolute reductions were in cognitive and multi-question overload.Source: §6 · p. 9

Limitations

The dataset and participant study cover adult learners, one educational platform, and role-played scenarios; the annotation and harness evaluation also leave questions for further study.

Limitations

  • The dataset covers adult learners on one platform, and the study sampled the ten most-used bots with college students role-playing intended users rather than pursuing their own coursework.The authors note that motivation and behavior may differ from those of real students; broader age groups, platforms, and classroom settings would clarify generalizability.Source: Limitations · p. 9
  • The human-calibrated LLM judge assesses some codes involving task demands and learner needs, while conversation-level labels record failure presence rather than frequency or severity.Further human review and evaluation on a separately annotated sample would strengthen the assessment.Source: Limitations · p. 9
  • The harness was evaluated through observed conversation failures, not learning outcomes or learner experience, and its components were not tested separately.Bots with the harness still miss required workflow steps, and early answer dumping is more frequent in that condition.Source: Limitations · p. 9

Ethical Considerations

The study reports adult-participant consent and privacy protections, while noting risks from educational context and limits on redistributing third-party materials. It calls for educator oversight and further evaluation before broader deployment.

Ethical Considerations

  • The corpus and participant study received institutional IRB approval, and educators and end users consented to research use of chatbot and conversation data.All participants were adults; Prolific participants were U.S.-based college students informed that their interactions would be studied.Source: Ethical Considerations · p. 9
  • Transcripts and prompts replace personal information, but educational conversations may still reveal sensitive information or enable indirect identification.The de-identified dataset is to be shared under a data-use agreement, while third-party teaching materials require attention to licenses and permissions.Source: Ethical Considerations · p. 9
  • Reduced failure labels do not establish improved learning or safe autonomous deployment, so consequential guidance warrants educator oversight, verification, and access to human help.The authors also caution against using codebook or judge labels to grade learners or rank educators, and call for evaluation across language, accessibility, and prior-knowledge differences.Source: Ethical Considerations · p. 9

A Data Schema and Platform Interface

EDUWILD links chatbot-level configuration records with conversation-level interaction records through chatbot_id. The appendix also provides the failure-code definitions and examples used by annotators.

A Data Schema and Platform Interface

  • The schema has two linked record types, illustrated with one chatbot and its Playlab export interface.The cited table caption identifies interaction data by user group and notes that anonymous end-user conversations are counted individually.Source: A Data Schema and Platform Interface · p. 11, Table 5 · p. 11
  • Each chatbot record stores the educator-authored system prompt and builder configuration, while each conversation record stores user group, timestamps, redacted input values, and ordered messages with roles, content, timestamps, and attachments.Chatbot records also retain model and variability settings, references, optional start inputs, platform settings, publication status, and pseudonymous author and timestamp data.Source: A Data Schema and Platform Interface · p. 11

B Failure Mode Codebook

  • The codebook gives annotator definitions, prior-work grounding, and examples for 29 secondary failure codes; codes marked inductive emerged from the data.Source: B Failure Mode Codebook · p. 11

B.1 Factual

The factual codebook distinguishes incorrect information from omissions, contradictions, and failures to use supplied context. It also defines instruction breaches, off-task drift, and over-refusal as separate failure modes.

B.1 Factual

  • F1 covers false, fabricated, or miscalculated content and misgrading, while F2 covers content that omits a required element or is too vague for the prompt’s goal.The F1 example is an incorrect derivative confirmed again after the student repeats it; the F2 example omits light as a target concept in an explanation of photosynthesis.Source: §B.1 · pp. 11–12
  • Figure 3 pairs an example chatbot record with its Playlab interface, including the start screen, system-prompt editor, and attached references.The displayed chatbot name is a purpose-reflecting placeholder; the original is withheld for anonymity.Source: Figure 3 · p. 12
  • F3 is contradiction within a conversation, whereas F4 is ignoring or misreading course material or context supplied by the prompt or uploaded references.The F4 example grades against generic criteria despite an attached rubric.Source: §B.1 · p. 12
  • F5 captures actions explicitly forbidden by the prompt, F6 captures unredirected departure from the bot’s purpose, and F7 captures refusal of an in-scope request through misapplied guardrails.The examples include drafting an essay despite a prohibition, extended unrelated game recommendations, and refusing to explain a p-value.Source: §B.1 · p. 12

B.2 Workflow

The appendix defines workflow, diagnostic, and pedagogical failure codes, then documents a prompt harness that preserves teacher instructions while guiding chatbot behavior. Its implementation extracts learner responsibilities, initializes a welcome, applies six response controls, and limits source access to filenames in this experiment.

B.2 Workflow

  • Workflow codes cover abandonment, unanswered questions, broken or repetitive output, overload, skipped prompt steps, language or format mismatch, and missing expectation setup.W1 marks a student stopping before the goal is reached; W6 covers several questions in one turn, while W7 and W9 cover skipped required steps and absent opening expectations.Source: §B.2 · pp. 12–13

B.3 Diagnose

  • Diagnostic codes capture missing pre-instruction diagnosis, misjudged level, missed or falsely identified errors, ignored learner signals, and misread requests.D1 concerns procedural questions answered before eliciting prior knowledge or an attempt; D5 concerns proceeding unchanged after explicit confusion, frustration, or time constraints.Source: §B.3 · pp. 13–14

B.4 Pedagogical

  • Pedagogical codes cover premature solutions, over-scaffolding, cognitive overload, strategy-task mismatch, insufficient engagement, poor feedback, and failure to recover.The taxonomy distinguishes answer dumping when guided help is called for from over-scaffolding when guidance withholds useful information on factual or logistical questions.Source: §B.4 · p. 14

C.1 Prompt Composition

  • The harness uses the original teacher prompt as authority while adding preparation, opening, and response-generation stages; each learner turn receives one tutor call without a reviewer or repair call.Preparation caches quoted learner responsibilities; a separate call generates the welcome, and response generation combines the original prompt, source context, six controls, turn context, and cached requirements.Source: §C.1 · p. 15

C.2 Teacher-Intent Extraction and Memory

  • Teacher-intent extraction returns learner responsibilities with exact supporting prompt quotations, while validation checks field structure and quotation provenance, not semantic fidelity.An empty learner-requirements array is valid; semantic fidelity remains dependent on the extraction model.Source: §C.2 · p. 15

C.3 Opening Initialization

  • The initialization call produces a brief student-visible welcome that establishes role and scope, then includes the teacher’s required opening question when applicable.A separate runtime trigger starts the educational-bot session.Source: §C.3 · p. 16

C.4 Six Failure-Specific Response Controls

  • Six response controls target evidence-grounded claims, teacher task boundaries, applicable workflow, one learner action at a time, task-fit responses, and brief expectation-setting.The controls are F1, F5, W7, W6, P3, and W9; workflow guidance says to answer direct in-domain questions before adapting any bypassed required check.Source: §C.4 · p. 16

C.5 Source-Access Context

  • In this experiment, the tutor receives attachment filenames and counts but cannot access attachment contents or retrieval, so file-specific claims must remain unverified.The source-access instructions direct the tutor to use filenames only as clues and to answer the supported part of a request when exact information depends on unavailable file contents.Source: §C.5 · p. 17

D Prolific Study Details

The Prolific study recruited 200 U.S.-based college students to use one of ten student-facing educational chatbots in assigned scenarios. Both chatbot conditions used the same scenario cards and interface, while harness and control differed in response generation.

Recruitment and procedure

  • 200 U.S.-based college students were recruited from 9,242 eligible Prolific participants, and the study estimated a five-minute duration with $3 base payment.Eligibility required current college enrollment; the task required at least three minutes of chatbot interaction.Source: D Prolific Study Details · p. 17

Task interface

  • Participants first read a scenario card and passed a comprehension check, then chatted for at least three minutes while meeting time, message-count, and scenario-fit criteria.The interface displayed the same scenario cards and chat window for control and harness conditions; only response generation differed.Source: D Prolific Study Details · p. 17, Figure 5 · p. 18

Ten selected chatbots

  • The ten student-facing bots included four for syllabus or logistics questions, three for writing or analysis, and three for tutoring or coaching.Their described uses ranged from course-policy questions to physics tutoring, nursing instruction, ESL goal-setting, art description, and writing tasks.Source: D Prolific Study Details · p. 18

Control rates versus corpus prevalence

  • Expectation setup occurred in 33.4% of corpus conversations versus 6.9% in control, while cognitive overload occurred in 27.1% versus 39.7%.Corpus rates pool conversations across deployed bots, whereas study rates weight the ten most active bots equally; the authors offer role-playing and required duration as possible explanations.Source: D Prolific Study Details · p. 19
Loading 2610.23530v1…