PaperlayerView exact revision on arXiv

Source-linked AI summary

TutorCaseBench: A Case-Grounded Benchmark for Evaluating LLM Tutors in Supporting Students with Challenging Behaviors

Yeon Su Park, Chanyoung Park, Tae Soo Kim, Seungju Kim, Hyunwoo Kim, Daho Jung, Jinsung Yoon, Juho Kim

arXiv:2610.30828v1cs.CLcs.HC

TL;DR

Existing AI-tutor benchmarks provide limited evidence about case-sensitive support for students with challenging behaviors. TutorCaseBench converts qualitative case studies into dialogues and intervention-grounded rubrics; across 11 LLMs, models select appropriate strategies better than they generate tailored responses, with the strongest model satisfying a documented intervention in 55.8% of open-ended cases.

  • Problem

    Existing AI-tutor benchmarks focus on cognitive aspects and idealized student behavior, offering limited evidence about case-sensitive support for challenging behaviors.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    TutorCaseBench uses an automated pipeline to convert qualitative case studies into evidence-grounded scenarios, dialogues, and rubrics for open-ended and multiple-choice evaluation.

    Source: §1 · pp. 1–2

  • Results

    Models select provided strategies better than they generate tailored responses; the strongest model satisfies a documented intervention in 55.8% of open-ended cases.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results show that LLM tutors remain limited in generating support aligned with documented interventions, even when they often recognize appropriate strategies in multiple-choice settings.

    Source: §6 · p. 8

    Text-based dialogues do not fully capture the complexity of real interactions, where tutors must recognize and respond to challenging behaviors.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLMs are increasingly used as AI tutors, yet existing benchmarks overlook the challenging behaviors common among students with learning and behavioral difficulties. We introduce TutorCaseBench, a benchmark that grounds AI-tutor evaluation in qualitative case studies from pedagogy, educational psychology, and special education. Our pipeline converts documented student categories into case-specific scenarios, tutor-student dialogues, and rubrics grounded in source evidence. The benchmark contains 923 dialogues across five challenge types, evaluated through open-ended response generation (1,560 items) and multiple-choice selection (2,537 items). Across 11 state-of-the-art LLMs, even the strongest model satisfies a rubric criterion in only 55.8% of open-ended cases, while models are better at selecting appropriate strategies than generating situated responses. These results reveal a gap between intervention recognition and context-specific adaptation, especially for open-weight models. All data are publicly available at https://github.com/kixlab/TutorCaseBench.

1 Introduction

TutorCaseBench addresses limited evidence on whether AI tutors can support students whose challenging behaviors make tutoring difficult to sustain. It grounds scenarios, dialogues, and evaluation criteria in qualitative case studies, then finds that models are better at selecting documented strategies than generating tailored responses.

1 Introduction

  • Existing AI-tutor benchmarks offer limited evidence on case-sensitive support because they focus on cognitive tasks, idealized behavior, and common pedagogical criteria.Students with learning and behavioral difficulties may show refusal, withdrawal, distress, or social conflict.Source: §1 · p. 1
  • Qualitative case studies document student backgrounds, precipitating events, and practitioner responses, supporting realistic scenarios with documented interventions.The cases come from pedagogy, educational psychology, and special education.Source: §1 · p. 1
  • The pipeline extracts learner and instructional evidence, then builds scenarios, tutor–student dialogues, and rubrics grounded in documented interventions.Evaluation uses open-ended responses and multiple-choice selection in immediate and long-term settings.Source: §1 · pp. 1–2
  • TutorCaseBench contains 923 dialogues across five challenge types, with 1,560 open-ended and 2,537 multiple-choice evaluation items.The five types are emotional distress, task resistance, social-interpersonal difficulty, attention/engagement failure, and academic struggle.Source: §1 · p. 2
  • Models select provided strategies better than they generate tailored responses, and even the strongest model satisfies a documented intervention in only 55.8% of open-ended cases.The benchmark provides systematic evidence of difficulty supporting students with challenging behaviors in controlled settings.Source: §1 · p. 2

2 Related Work

Prior work simulates learner knowledge and traits, but simulated students often remain cooperative and idealized. AI-tutor evaluation has moved toward pedagogy-grounded criteria, yet fixed and fragmented criteria leave limited coverage of context-dependent support for challenging behaviors.

2.1 Literature-Grounded Student Simulations

  • Prior student simulations model knowledge, reasoning, personas, or motivational profiles, but often omit frustration and disengagement and portray learners as attentive and receptive.Even fine-tuned models show limited alignment with real student behavior.Source: §2.1 · p. 2
  • Grounding dialogue synthesis in educational case literature can support realistic challenging behaviors beyond cooperative learner settings.The paper draws on pedagogy, educational psychology, and special education literature.Source: §2.1 · p. 2

2.2 Evaluation of AI Tutors

  • Pedagogy-grounded evaluation has replaced some reference-matching metrics, but proposed criteria remain fragmented and lack consensus.Prior frameworks include teacher-likeness, student understanding, helpfulness, coherence, correctness, equitable tutoring, usefulness, care, and human-likeness.Source: §2.2 · p. 2
  • Because challenging-behavior support is context-dependent, the paper derives evaluation criteria from interventions documented in pedagogical case studies rather than relying on fixed criteria.Existing benchmarks largely use fixed criteria.Source: §2.2 · p. 3

3 Data Construction

TutorCaseBench turns case-study papers into English tutor–student dialogues and evaluation tasks using an automated LLM pipeline; its pipeline consistently grounds outputs in source evidence, according to a human validation study.

3.1 Paper Collection

  • The paper collects ERIC case-study candidates published from 2010–2026 and filters them for concrete descriptions of student behavior, learning context, and teacher actions.Abstract screening supplements descriptor-based collection.Source: §3.1 · p. 3

3.2 Open-Ended Response Task

  • The open-ended task tests whether a tutor’s next response aligns with interventions documented for a case-grounded dialogue.The benchmark also includes multiple-choice strategy selection.Source: §3.2 · p. 3, §3 · p. 3

3.2.1 Tutor–Student Dialogue Construction

  • For each student category, the pipeline extracts verbatim learner and instructional evidence, including behaviors, affect, triggers, strategy enactment, outcomes, and temporal scope.It composes scenarios from distinct evidence spans rather than collapsing different tasks or contexts into a generic scenario.Source: §3.2.1 · p. 3
  • Scenarios pair visible task, context, and triggering behavior with hidden effective or ineffective strategies; synthesized six-turn dialogues end with the challenging behavior and keep student turns evidence-grounded.The category-level persona captures recurring patterns, triggers, and what tends to work or fail; six-turn variants were tested with consistent results.Source: §3.2.1 · pp. 3–4

3.2.2 Open-Ended Response Evaluation

  • Category- and response-specific rubrics use documented effective interventions as checklist criteria, with ineffective strategies as negative anchors.A response succeeds if it satisfies at least one independently judged criterion, since the criteria represent alternative approaches.Source: §3.2.2 · pp. 4–5

3.3 MCQ Selection Task

  • Each four-option MCQ pairs one target-scenario strategy with three strategies from other scenarios as distractors, testing selection for fit to the dialogue.Distractors are retrieved by embedding similarity, selected with an LLM judge, and rewritten into a uniform surface form.Source: §3.3 · p. 5

3.4 Data Statistics

  • The benchmark has 923 scenarios from 300 papers and 518 student categories, with 1,560 open-ended items and 2,537 MCQs.Challenge types range from task resistance (28.7%) to emotional distress (13.7%).Source: §3.4 · p. 5

3.5 Data Validation

  • In a validation study of 30 papers rated by two reviewers each, stage ratings ranged from 2.42 for evidence to 2.69 for rubrics on a three-point scale.Overall, the authors report that pipeline outputs align with human judgments of source-supported content.Source: §3.5 · p. 5

4 Experiments

The experiments evaluate 11 LLMs on open-ended response generation, multiple-choice selection, and reference-based response generation. The supplied passages describe the task setups and evaluation procedures, but do not report their comparative outcomes.

4 Experiments

  • The study evaluates 11 LLMs across open-ended generation, MCQ selection, and reference-based response generation.The supplied experiment passages do not give results for these tasks.Source: §4 · p. 6
  • Open-ended prompts ask for an immediate next turn or a long-term plan using minimal context and the dialogue, and a response succeeds if it satisfies at least one rubric criterion.Gemini 3.1 Pro Preview judges each criterion as satisfied or not against the rubric and full scenario context.Source: §4 · p. 6
  • MCQ prompts provide four options, while the reference-based variant adds strategies from the most similar scenarios for a paired comparison with dialogue-only responses.MCQ correctness is based on the option grounded in the scenario’s effective strategy.Source: §4 · p. 6

5 Evaluation Results

Across the three evaluation settings, models struggle to generate case-specific support, though they do better at selecting strategies and often benefit from references for immediate responses.

5.1 Open-Ended Response

  • 55.8% open-ended success for Claude Opus 4.7 led the models, followed by GPT-5.5 at 54.7%; DeepSeek V4 Pro led open-weight models at 47.0%.Llama 4 Maverick was lowest at 31.7%.Source: Table 1 · p. 6, §5.1 · p. 6
  • 67.5% vs. 35.2% for Gemini 3.1 Pro marks the largest immediate/long-term gap; social-interpersonal difficulty ranked last for 6/11 models immediately, while academic struggle ranked last for all 11 long-term.All models performed better on immediate responses than long-term interventions.Source: §5.1 · pp. 6–7, Figure 6 · p. 7
  • ρ = 0.800 (p < 0.01) linked 4-gram diversity with immediate success, and case-recovery rate also correlated with immediate success (ρ = 0.691, p < 0.05), but neither relationship held for long-term interventions.Maverick’s most frequent 4-gram appeared in 50.4% of its immediate responses, versus 6–9% for Claude Opus 4.7, GPT-5.5, and Gemini 3.1 Pro.Source: §5.1 · p. 7, Figure 7 · p. 7
  • Models reassured a student who believed her correct work was wrong and corrected a language-related math error without addressing the underlying comprehension difficulty.The case-prescribed responses were to verify the student’s work and use simpler terms such as “double” or “triple.”Source: §5.1 · p. 7, Table 2 · p. 8
  • Models over-guided a nervous student instead of letting him lead, and all models calmed a refusing student rather than redesigning the rejected reading task.The case called for the teacher to intervene only for content issues in the first scenario and replace the reading task in the second.Source: §5.1 · pp. 7–8, Table 2 · p. 8

5.2 MCQ Selection

  • 63.0% accuracy made Claude Opus 4.7 the top MCQ model; GPT-OSS 120B scored 49.2%, still above the 25% chance level.MCQ scores were higher and less variable than open-ended scores, suggesting models can recognize an option they struggle to formulate.Source: Table 3 · p. 8, §5.2 · p. 8
  • Social-interpersonal difficulty was the weakest immediate MCQ category for all 11 models, while academic struggle performed strongly in MCQ despite being weakest in long-term generation..Source: §5.2 · p. 8

5.3 Reference-Based Response

  • 10 of 11 models improved on immediate responses with references, gaining 9.9 points on average; Llama 4 Maverick rose from 41.0% to 64.8%, but Gemini 3.1 Pro did not improve.The reference effect did not extend to long-term plans.Source: Figure 8 · p. 8, §5.3 · p. 8

6 Conclusion

TutorCaseBench evaluates LLM tutors using case-grounded dialogues and case-specific criteria. Across 923 dialogues and 11 models, generation remains limited even when models often recognize suitable strategies in multiple-choice settings.

6 Conclusion

  • TutorCaseBench converts qualitative case studies into tutor–student dialogues and case-specific evaluation criteria, rather than relying on idealized learner simulations and fixed pedagogical rubrics.The benchmark targets support for students with challenging behaviors.Source: §6 · p. 8
  • Across 923 dialogues and 11 state-of-the-art LLMs, models remained limited in generating support aligned with documented interventions, even when they often recognized appropriate strategies in multiple-choice settings.Source: §6 · p. 8
  • The results highlight the need for AI tutors to move beyond plausible responses toward context-sensitive support grounded in individual student needs.Source: §6 · p. 8

Limitations

The benchmark’s construction and scope impose four stated limitations: Gemini-family dependence, uneven population and context coverage, text-only interactions, and no cross-paper grouping of student categories.

Limitations

  • TutorCaseBench’s construction primarily relies on Gemini, so model-specific interpretations may enter its scenarios and rubrics.Human validation and robustness analyses with alternative construction models and judges support the consistency of the findings.Source: Limitations · p. 9
  • Coverage is concentrated in particular regions and age groups, limiting representation of some student populations and educational contexts.The cases come from accessible, contemporary ERIC studies with detailed evidence about student behavior, learning context, and teacher actions.Source: Limitations · p. 9
  • Text-based dialogues enable controlled evaluation independent of behavior detection, but may not fully capture the complexity of real interactions.The dialogues describe actions and nonverbal cues; the authors suggest extending the approach to interactive, multimodal settings.Source: Limitations · p. 9
  • Constructing student categories separately within each paper preserves case-specific intervention evidence but limits analysis of recurring patterns across cases.The authors suggest clustering related categories to examine recurring challenges and support strategies.Source: Limitations · p. 9

Ethical Considerations

The paper documents its human-validation safeguards, data provenance and framing, pipeline fallback cases, paper-screening procedures, and dialogue-oriented challenge taxonomy.

Ethical Considerations

  • Human validation received institutional review board approval, required informed consent, and allowed participants to withdraw at any time.Participants reviewed each assigned paper for about 90 minutes and received 120,000 KRW (≈ 89.68 USD).Source: Ethical Considerations · p. 9
  • Released materials contain synthesized dialogues, rubrics, and MCQs, not source texts or intermediate extraction artifacts; source-paper metadata supports provenance.Source documents were publicly accessible ERIC materials used for non-commercial academic research, with copyright retained by original authors and publishers.Source: Ethical Considerations · p. 9
  • Challenging behaviors are included only when grounded in source literature and framed as context-dependent student needs, not fixed traits.Source: Ethical Considerations · p. 9
  • The authors used ChatGPT and Claude for writing and figure icons, and Codex and Claude Code for pipeline implementation.Source: Ethical Considerations · p. 9

A Model Fallbacks in the Pipeline

  • Gemini was prioritized for pipeline consistency, with persistent failures rerouted to OpenAI models; fallback use was expected to have limited overall impact.Seven papers used gpt-5.4-mini for extraction and one student category used gpt-5.5 for rubric construction.Source: A Model Fallbacks in the Pipeline · p. 13

B ERIC Search Descriptors

C Abstract-Based Paper Filtering Procedure

  • Title-and-abstract screening uses a cost-efficient, high-recall first pass, retaining papers only when they meet all four criteria for grounded learner evidence and practical teacher guidance.The criteria exclude papers without direct student observation or interview, quantitative studies, policy or measurement-tool papers, and papers without teacher guidance.Source: C Abstract-Based Paper Filtering… · p. 13
  • The pipeline also excludes papers at least 30 pages long or unavailable for access, partly to avoid mistaking literature-review content for learner evidence.Source: C Abstract-Based Paper Filtering… · p. 13

E Impact of Dialogue Turns on Evaluation Results

Changing dialogue length from four to eight turns barely altered open-ended evaluation results. The appendix also describes how plausible distractors and binary effective–ineffective choices are constructed for MCQ evaluation.

E Impact of Dialogue Turns on Evaluation Results

  • 45.8%, 46.6%, and 46.8% success rates for four-, six-, and eight-turn dialogues show little effect from dialogue length.Model rankings remained highly consistent: ρ = 0.916 (p < 0.001) for four versus six turns and ρ = 0.911 (p < 0.001) for six versus eight turns.Source: E Impact of Dialogue Turns on… · p. 14

F.1 4-Option MCQs

  • Four-option MCQs use three distractors selected from the top 10 candidates, favoring plausible actions compatible with the target setting.Candidates are scored for context fit while penalizing behavior similarity and similarity to effective target interventions; all options are rewritten to match dialogue vocabulary, length, and tone while preserving their mechanisms.Source: §F.1 · p. 14

F.2 Binary MCQs

  • The binary MCQ variant contrasts one effective and one ineffective strategy from the same target scenario, testing rejection of an inappropriate response.In contrast, the four-option task asks models to select the best fit among effective strategies.Source: §F.2 · p. 14

F.2.1 Construction

  • Binary items are created for immediate or long-term responses when a scenario has at least one matching effective and one ineffective strategy.When multiple options are available, a judge selects a scope-matched, concrete pair with comparable granularity; options are then rewritten to match in style while preserving meaning.Source: §F.2.1 · p. 15

F.2.2 Results

Models distinguish documented effective interventions from ineffective ones with high accuracy on binary MCQs. The appendix also reports demographic coverage, performance patterns across groups, and human-validation interface procedures.

F.2.2 Results

  • 85.8%–93.1% accuracy across 11 models and 751 binary MCQs is far above the 50% chance baseline.The result supports interpreting lower four-option performance as difficulty choosing among plausible effective strategies, rather than basic inability to reject ineffective responses.Source: §F.2.2 · p. 15, Figure 9 · p. 15

G.1 Demographic Distribution

  • Student categories were drawn from 300 source papers; country was reported for 99.8% of categories, education level for all, gender for 74.9%, and age for 60.4%.North America accounts for 61.4% of categories, and the United States for 58.9%.Source: §G.1 · p. 15

G.2 Performance by Demographic Groups

  • Across demographic groups, immediate responses generally outperform long-term interventions, but uneven group sizes warrant caution in comparisons.Table 5 presents open-ended success rates by geographic region, education level, and gender, averaged across 11 LLMs.Source: §G.2 · p. 15, Table 5 · p. 16

H Human Validation Interface

  • The human-validation interface showed participants each pipeline output alongside its criterion, definition, and a validation question, then collected a rating and written rationale.Participants first read the source paper and wrote a summary of at least 30 words; each rationale had to be at least 50 characters.Source: H Human Validation Interface · p. 16

I Further Details on Human Validation

Human reviewers generally judged the pipeline’s outputs grounded in source papers, while identifying interpretation and adaptation issues across stages. The evaluation used a subset of papers and student categories, with two independent reviewers per paper.

I Further Details on Human Validation

  • Across 882 three-point judgments, 96.6% were Yes or Partly and only 3.4% were No; 76.3% of 114 binary judgments confirmed assigned challenge types.Exact inter-rater agreement was 58.2% overall, highest for challenge types (84.2%) and lowest for instructional strategies (43.4%).Source: I Further Details on Human Validation · p. 16, Table 6 · p. 16
  • Reviewers rated 65.0% of student-category judgments Yes, with concerns mainly about omissions, overlapping groupings, and imprecise descriptions.Relatively few concerns identified substantive grounding issues.Source: I Further Details on Human Validation · p. 16
  • Evidence excerpts received 50.9% Partly ratings, often because excerpts were broad, insufficiently specific, or drawn from other students’ cases or prior literature.The remaining evidence in the evaluated sets was considered appropriate.Source: I Further Details on Human Validation · pp. 16–17
  • Most challenge-type judgments (76.3%) were confirmed, though reviewers noted that students often exhibited overlapping challenges beyond the assigned type.Source: I Further Details on Human Validation · p. 17
  • Scenarios were usually judged grounded (61.1% Yes), but reviewers flagged combined moments, unsupported contextual details, and occasional behavior misinterpretations.Source: I Further Details on Human Validation · p. 17
  • Instructional strategies received 50.5% Yes and 47.5% Partly ratings, with concerns about added context or outcomes and general recommendations framed as student-specific interventions.Source: I Further Details on Human Validation · p. 17
  • Most dialogue judgments (65.7%) were Yes; partial ratings chiefly concerned conversational naturalness despite the dialogues remaining grounded in their scenarios.Source: I Further Details on Human Validation · pp. 17–18
  • Rubrics received 69.3% Yes ratings and no No ratings, though some criteria or rationales exceeded source evidence when context-specific findings became broader rules.Reviewers said the rubrics captured core instructional guidance, with interpretation involved in translating strategies into scenario-specific criteria.Source: I Further Details on Human Validation · p. 18

K Average Criterion Satisfaction Rate

Average criterion satisfaction rates preserved the overall model-performance pattern: the leading models remained the same, and all models did better on immediate responses than long-term plans. Challenge-type patterns differed across settings.

K Average Criterion Satisfaction Rate

  • 28.8% for Claude Opus 4.7, 28.0% for GPT-5.5, and 26.4% for Gemini 3.1 Pro were the highest average criterion satisfaction rates.All 11 models performed better on immediate responses than on long-term plans.Source: Table 7 · p. 18, K Average Criterion Satisfaction Rate · p. 18
  • For immediate responses, task resistance and social-interpersonal difficulty generally had lower satisfaction rates, while emotional distress performed relatively well.For long-term interventions, academic struggle and social-interpersonal difficulty tended to have lower rates; social-interpersonal difficulty remained challenging across both settings.Source: K Average Criterion Satisfaction Rate · p. 18

L Impact of Construction Model Choice on Evaluation Results

Model rankings were highly consistent when the benchmark construction pipeline used GPT or Claude instead of Gemini on a stratified sample of source papers.

L Impact of Construction Model Choice on Evaluation Results

  • ρ = 0.943 for Gemini- versus Claude-built rankings and ρ = 0.925 for Gemini- versus GPT-built rankings (both p < 0.001), indicating high ranking consistency across construction models.The pipeline was rerun on a 25% (N = 75) stratified sample of source papers, with prompts and evaluated models held fixed.Source: L Impact of Construction Model Choice… · p. 19

M Impact of Judge Model Choice on Evaluation Results

The appendix reports that relative model performance remained consistent across judge choices, then describes reference-based evaluation and further analyses of lexical diversity, case specificity, and MCQ challenge-type results.

M Impact of Judge Model Choice on Evaluation Results

  • Model rankings were highly consistent across judge pairs, with Spearman correlations of 0.96–0.97, all significant at p < 0.001.Responses from all 11 models were reevaluated with two judges while the rubrics and evaluation prompt stayed fixed.Source: M Impact of Judge Model Choice on… · p. 19, Table 8 · p. 19

N Reference-Based Response Details

  • Reference responses used three close scenarios from other papers, filtered for an effective strategy matching the target response type, as comparison material alongside dialogue-only evaluation.Source: N Reference-Based Response Details · p. 19

O Lexical Diversity

  • For immediate responses, lexical diversity correlated with success at ρ = 0.855 for 3-grams and ρ = 0.791 for 5-grams; only the long-term 3-gram correlation was significant.Both immediate-response correlations had p < 0.01; the long-term 3-gram correlation was ρ = 0.627 (p < 0.05).Source: O Lexical Diversity · p. 19, Figure 12 · p. 20, Figure 13 · p. 20

P Case Specificity

  • Case-recovery rate was positively associated with immediate-response success across models (ρ = 0.691, p < 0.05).The authors interpret this association as suggesting stronger models produce responses more specific to the current dialogue.Source: P Case Specificity · pp. 19–20, Figure 14 · p. 20

Q MCQ Selection by Challenge Type

  • Academic struggle was weakest for all 11 models in long-term open-ended responses but performed relatively well in MCQ selection, while social-interpersonal difficulty remained challenging in both settings.In immediate MCQs, social-interpersonal difficulty was weakest for all 11 models and academic struggle strongest for 10 of 11; social-interpersonal difficulty remained weakest for 7 of 11 in long-term MCQs.Source: Q MCQ Selection by Challenge Type · p. 20, Figure 15 · p. 20

R MCQ Failure Analysis

This section describes the evidence-grounded procedures and prompts for constructing TutorCaseBench scenarios, dialogues, rubrics, and multiple-choice items, and for eliciting and grading immediate or long-term tutor responses. It also covers MCQ answer patterns, shared errors, and how references affect long-term intervention-planning performance.

R MCQ Failure Analysis

  • All 11 models answer 26.8% of MCQ questions correctly and miss 17.5%; on questions all models miss, they select the same distractor in 40.3% of cases.Social-interpersonal cases have the lowest all-correct share, highest all-wrong share, and highest rate of shared wrong answers.Source: R MCQ Failure Analysis · p. 20, Figure 16 · p. 21, Figure 17 · p. 21

S Reference-Based Response

  • Only 3 of 11 models improve long-term intervention-planning performance with references, while 7 decline and one is unchanged, for an average drop of 5.2 percentage points.Unlike immediate responses, adding reference cases often reduces performance on long-term plans.Source: Figure 18 · p. 21, S Reference-Based Response · p. 21

T.1 Abstract-Based Paper Filtering

  • Abstract screening filters papers lacking student observation or interviews, qualitative behavioral focus, or practical classroom guidance, and excludes quantitative intervention-efficacy studies.Papers are kept only when all four filter answers are N.Source: §T.1 · pp. 21–22

T.2 Paper Screening

  • Paper screening scores student behavior, learning context, and teacher actions on 1–5 scales, passing papers only when all three scores are at least 3.The dimensions assess observable actions, task and setting specificity, and instructional strategies or outcomes.Source: §T.2 · p. 22

T.3 Student Category Extraction

  • Category extraction uses only explicitly stated information and verbatim excerpts, separating student types when their characteristics or needed approaches differ.Each category is assessed for behavior, learning context, and teacher actions.Source: §T.3 · pp. 22–23

T.4 Learner Evidence Extraction

  • Learner-evidence extraction seeks extensive verbatim coverage of student behavior, cognition, affect, difficulty triggers, and quotes, with source locations and applicable category labels.The procedure prioritizes student-centered evidence and excludes purely teacher-centered intervention descriptions.Source: §T.4 · p. 24
  • Learner evidence extraction targets at least 10 excerpts per category when source content permits and requires word-by-word verification of every excerpt and student quote.Extracted evidence includes task context and source location, with non-verbatim items marked and explained.Source: §T.4 · pp. 24–25

T.5 Instructional Evidence Extraction

  • Instructional evidence extraction collects verbatim teacher actions and outcomes, classifies their context and temporal scope, and links each excerpt to learner behaviors with an explicit rationale and strength.The prompt distinguishes effective from ineffective actions and calls for checking coverage, orphaned excerpts, and verbatim accuracy.Source: §T.5 · pp. 25–26

T.6 Scenario Construction

  • Scenario construction requires evidence-grounded, student-specific scenarios with documented context, observable trigger-linked behaviors, and effective or ineffective strategies kept hidden from the visible problem.Scenarios should be meaningfully distinct and aligned with the task and context; strategies involving tools or conditions must be documented in the context field.Source: §T.6 · pp. 27–28
  • Scenario construction extracts documented behaviors, triggers, student-specific intervention effects, and persona traits, then assigns a primary challenge and an optional distinct secondary challenge.Ineffective strategies are included only when the source explicitly describes failure.Source: §T.6 · p. 28

T.7 Dialogue Synthesis

  • Dialogue synthesis creates one evidence-grounded dialogue per scenario, using neutral tutor turns and ending with a student turn that displays the documented challenging behavior.The dialogue is planned around the task, context, student persona, difficulty trigger, and available actions.Source: §T.7 · p. 30

T.8 Rubric Construction

  • Rubrics derive distinct, student-specific checklist criteria from intervention evidence, separating immediate and long-term principles without padding or relying on generic dimensions.Each criterion has a rationale and a check condition, and diverse tutor approaches can satisfy the principle-level check.Source: §T.8 · pp. 31–32
  • Rubrics derive student-specific principles from dialogue examples and intervention evidence, with separate immediate and long-term criteria when specified and no fixed criterion count.Criteria must accommodate diverse valid responses and be supported by evidence.Source: §T.8 · pp. 32–33

T.9 MCQ Distractors Selection

  • Distractor selection returns exactly three plausible interventions, rejecting candidates only when their mechanism is structurally impossible or too close to an effective answer.If fewer than three clean candidates remain, the least-bad rejected choices fill the remaining slots and are listed as forced.Source: §T.9 · p. 33

T.10 MCQ Options Adaptation

  • Option adaptation aligns surface vocabulary, length, and tone while preserving each intervention’s mechanism, the correct option’s appropriateness, and distractors’ wrongness.Only surface changes and neutral length-matching filler are allowed; neither may alter substance or add claims about effects.Source: §T.10 · p. 34

T.11 Tutor Response Generation (Immediate / Long-Term)

  • Tutor-response prompts distinguish a direct, 2–3-sentence response to the student’s behavior in the moment from a separate long-term intervention prompt.The immediate prompt explicitly excludes a multi-step plan or later actions.Source: §T.11 · p. 35
  • Long-term response generation asks for a narrative plan of sustained interventions and support structures over weeks or months after challenging behavior.Source: §T.11 · p. 35

T.12 Rubric Grading

  • Rubric grading uses a variable-length checklist, scores each criterion independently, and requires substantive evidence rather than surface mentions.The prompt also says responses need not satisfy every criterion to be acceptable.Source: §T.12 · pp. 35–36

T.13 MCQ Response

  • The MCQ response prompt asks the model to choose the most appropriate intervention for the dialogue’s specific student and return an option letter with brief reasoning.Source: §T.13 · p. 36

T.14 MCQ Pair Selection

  • MCQ pair selection matches both options to the response type, favors concrete tutor actions plausible in the dialogue, and seeks comparable granularity.When candidate pairs otherwise meet the criteria, the prompt favors the effective option most directly responsive to the moment.Source: §T.14 · p. 37

T.15 MCQ Pair Adaptation

  • MCQ pair adaptation addresses style cues by making options parallel in form and length with minimal edits while preserving which is effective and the ineffective option’s flaw.The prompt prefers verbatim options when already parallel, then surface tweaks or light rewrites; fuller rewrites are a last resort.Source: §T.15 · pp. 37–38

T.17 Dialogue Synthesis Prompt with Variable Turns

  • Dialogue synthesis requires documented evidence for behaviors and messages, with each scenario mapped to a dialogue ending in a challenging student turn.The prompt specifies planning each dialogue around the scenario’s task and context.Source: §T.17 · p. 39
  • Each dialogue must contain exactly the specified number of alternating tutor–student messages, beginning with the tutor and ending with the student.The prompt defines the total as twice the specified number of exchanges and requires turn IDs to run from 1 through the total.Source: §T.17 · p. 39
  • Dialogues begin with a concrete task, use neutral tutor responses without effective strategies midstream, and end with student behavior matching the documented challenge trigger.The final student turn must show observable struggling behavior grounded in documented behavior patterns.Source: §T.17 · p. 39
  • Student responses should follow the documented persona and evidence, while minimal context gives the task and setting without student characteristics, struggles, or solutions.Physical actions are allowed only when supported by evidence and context.Source: §T.17 · p. 39
Loading 2610.30828v1…