PaperlayerView exact revision on arXiv

Source-linked AI summary

From Instructional Intent to Classroom-Ready Lessons: Investigating Teacher-AI Collaboration in Lesson Authoring for Secondary Computing Education

Tahreem Yasir, Wenbo Li, Deniz Ozturk, Yasitha Rajapaksha, Dmitri Droujkov, Xiaoyi Tian, Tiffany Barnes

arXiv:2610.12834v1cs.HC

TL;DR

Secondary computing teachers’ use and direction of AI during lesson authoring remain underexplored, particularly in relation to subject-specific classroom needs. Across focus groups and LessonBuilder think-aloud sessions, the paper finds that teachers selectively delegate work while retaining control over materials and classroom-readiness judgments. It synthesizes these findings into a conceptual model and design recommendations for teacher-orchestrated authoring.

  • Problem

    Evidence is limited on how secondary computing teachers use and direct AI when creating, adapting, and judging materials for students.

    Source: §1 · p. 2

  • Method

    Two qualitative studies with eight secondary computing teachers combined focus groups about existing practices with LessonBuilder think-aloud sessions.

    Source: §5.1 · p. 13

  • Results

    Teachers retain control by setting requirements, selecting AI use, inspecting intermediate artifacts, and judging classroom fit.

    Source: §7 · p. 22, §9 · p. 25

  • Takeaways & Limitations

    The model and recommendations address selective revision, evolving requirements, judgment of AI feedback, and inspection of computing materials.

    Source: §1 · p. 3, §9 · p. 25

    Participants came from one professional development cohort, limiting the range of educational contexts represented.

    Source: §8 · p. 24

Abstract

from arXiv · show

Generative AI is increasingly studied as a tool for supporting lesson authoring; the creation and adaptation of classroom materials, yet little is known about how it should fit into secondary computing teachers' subject-specific practices. We conducted two complementary qualitative studies with eight secondary computing teachers: focus groups examining existing authoring practices and a think-aloud study using LessonBuilder, a structured AI-supported lesson authoring system, as a research probe. We found that teachers' instructional intentions developed through creating, inspecting, and adapting materials, while teachers selectively delegated tasks to AI and retained responsibility for judging classroom readiness. The think-aloud sessions further showed how examining AI-generated materials prompted teachers to clarify and refine their intentions, making authoring an ongoing negotiation between what teachers intended and what the AI produced. In the context of computing lessons, classroom readiness also involved considering how programming materials and supporting guidance matched students' understanding and intended learning goals. We synthesize these findings into a conceptual model explaining how teachers coordinate instructional intent, AI assistance, and judgments of classroom readiness during lesson authoring. We derive design recommendations for supporting this coordination while preserving teacher control and addressing computing-specific authoring needs.

1 Introduction

The paper examines how secondary computing teachers direct AI during lesson authoring and judge whether materials are ready for their classrooms. Across two qualitative studies, it develops a model and design recommendations for coordinating instructional intent, AI assistance, and teacher control.

1 Introduction

  • Existing lesson-authoring systems provide limited insight into how secondary computing teachers direct AI when creating, adapting, and judging materials for students.Computing-specific decisions include matching programming materials to students’ prior knowledge and anticipating errors and debugging difficulties.Source: §1 · p. 2
  • Two qualitative studies with eight secondary computing teachers paired focus groups about existing practices with LessonBuilder think-aloud sessions.The studies examined teachers’ authoring, judgments of classroom suitability, and decisions while using structured AI support.Source: §1 · p. 2
  • The paper’s central contribution is a conceptual model of teacher-orchestrated AI-supported lesson authoring, grounded in both studies.It describes how teachers decide what to delegate and where to retain control while refining intent, adapting materials, and judging classroom readiness.Source: §1 · p. 2
  • Four design recommendations address selective revision, explicit and revisable lesson requirements, evaluation of AI feedback, and inspection of computing materials.The recommendations cover both technical and instructional aspects of those materials.Source: §1 · p. 3

2 Related Work

The related work frames lesson authoring as adapting resources to classroom needs, while computing education adds subject-specific demands. It also positions AI assistance alongside teachers’ continuing review and control of generated materials.

2.1 Teachers’ Use and Adaptation of Curriculum Materials

  • Teachers adapt and combine curriculum resources to meet learning objectives, student needs, and classroom conditions.They assess curricular expectations, engagement, challenge, and practicality before revising materials for instructional use.Source: §2.1 · p. 3
  • Computing teachers must match programming activities and guidance to students’ prior knowledge, while anticipating code behavior, errors, and debugging difficulties.Differences in programming experience can require activity adaptations and different levels of guidance.Source: §2.1 · p. 3

2.2 Generative AI for Teacher Work and Lesson Authoring

  • AI can help produce or reshape lesson materials, but generated content may still need teacher review for accuracy, pedagogical suitability, goals, and student needs.Review can include checking information, comparing alternatives, and revising selected content before incorporation.Source: §2.2 · pp. 3–4
  • The paper examines how computing teachers connect instructional intent, AI assistance, and classroom-readiness judgments across reported practices and LessonBuilder authoring.Its focus is how teachers refine intentions, direct assistance, and judge resulting materials.Source: §2.2 · p. 4

2.3 Teacher–AI Collaboration, Agency, and Control

  • Prior human–AI interaction work distinguishes teacher agency, transparency through inspectable outputs, and human authority over whether AI contributions are used.These perspectives connect directing assistance, inspecting outputs, and deciding what to retain, revise, or reject.Source: §2.3 · p. 4

3 LessonBuilder: A Teacher-Directed AI System for Multi-Phase Lesson Authoring

LessonBuilder structures lesson creation and adaptation around teacher-directed operations and inspectable intermediate outputs. Its pathways use specialized AI operations while leaving teachers to review materials and choose whether to proceed or revise.

3 LessonBuilder: A Teacher-Directed AI System for Multi-Phase Lesson Authoring

  • LessonBuilder supports creating or adapting computing lessons through intermediate artifacts and teacher-directed transitions rather than one-step lesson generation.Teachers review outputs and choose whether to request revisions or continue.Source: §3 · p. 4

3.1 Design Rationale

  • The design rationale combines creation and import pathways, inspectable and revisable materials, and teacher control over consequential authoring decisions.Teachers decide whether to generate materials from a plan, apply quality-review feedback, and publish.Source: §3.1 · pp. 4–5

3.2 Role-Specialized LLM Architecture

  • Separate Planner, Generator, and Quality Reviewer calls handle distinct tasks; a deterministic orchestration layer invokes them in response to teacher actions.The agents do not select subsequent operations or communicate through open-ended conversation.Source: §3.2 · p. 5

3.2.1 Generation Pathway: Role-Specialized Agents.

  • The generation pathway turns teacher goals, classroom context, and desired materials into a structured plan, then generates lesson items if the teacher chooses to proceed.Optional review returns criterion-specific observations and suggested revisions, but is advisory rather than independent verification of correctness or readiness.Source: §3.2.1 · p. 5

3.2.2 Import Pathway: Importing Items from Existing Materials.

  • The import pathway extracts source text, proposes topics and learning, practice, and quiz items, and uses retrieved passages to generate editable content for teacher review.Teachers can revise the preview before confirming; the current import pipeline does not generate practice-item test cases.Source: §3.2.2 · p. 6

3.3 Teacher-Facing Authoring Workflows

LessonBuilder presents generation and import as workflows with review points before teachers save, publish, or revise materials. Teachers can inspect computing-specific content and retain manual or AI-assisted revision options.

3.3 Teacher-Facing Authoring Workflows

  • The interface supports inspection and revision across generation and import pathways, with distinct decision points in each workflow.The import pathway lets teachers upload materials, inspect proposed topics and items, then edit, add, or remove content before confirming.Source: §3.3 · p. 6, Figure 2 · p. 7

3.3.1 Generating a New Lesson.

  • For new lessons, teachers specify goals, classroom context, and assessments before reviewing a plan and generated explanations, programming examples, activities, and assessments.Optional quality-review findings remain separate from the lesson; teachers may continue without applying them or request a collective revision.Source: §3.3.1 · p. 7

3.3.2 Importing and Adapting Existing Materials.

  • For imported lessons, teachers inspect and revise inferred topics and items before confirming, including programming explanations, examples, instructions, and starter code.Imported practice items lack generated test cases, which teachers can add later in the lesson editor.Source: §3.3.2 · p. 7

3.3.3 Teacher Control Over Lesson Revision.

  • Teachers can save drafts or publish lessons, manually edit items, and approve or discard updates proposed by the teacher-facing chatbot.Programming edits can include instructions, starter code, hints, model solutions, and optional test cases.Source: §3.3.3 · pp. 7–8

3.4 Implementation Details

  • The implementation uses a React interface, Node.js and Express backend, MongoDB storage, and role-specialized LLM operations for authoring tasks.The programming environment supports Python and JavaScript, with browser execution and output or runtime-error capture.Source: §3.4 · p. 8

4 Methods

Two qualitative studies with eight secondary computing teachers combined accounts of existing lesson preparation with observation of authoring in LessonBuilder. The methods examined how teachers specify intent, direct AI, inspect materials, and retain instructional control.

4 Methods

  • The IRB-approved studies took place during the final week of a four-week professional development workshop at a southeastern U.S. public university.Source: §4 · p. 8

4.1 Participants

  • Eight teachers—seven high-school and one middle-school—brought varied computing courses, platforms, and prior AI use.All had used at least one computing curriculum or platform; Code.org was most common (n=8), followed by CodeHS and Scratch (n=5 each).Source: §4.1 · pp. 8, 10

4.2 Study 1: Semi-Structured Focus Groups

  • Four two-teacher focus groups examined authoring practices, AI use, instructional control, and classroom-readiness criteria.Sessions lasted approximately 50–60 minutes and followed a shared protocol with four modules.Source: §4.2 · pp. 10–11

4.2.1 Focus Group Data Analysis.

  • Researchers used hybrid deductive-inductive thematic analysis, refined a codebook collaboratively, and organized results into five code families.Both researchers coded the full corpus independently and resolved disagreements through discussion.Source: §4.2.1 · p. 11

4.3 Study 2: Think-Aloud Tool Interaction

  • Study 2 used LessonBuilder as a research probe to observe decisions through staged workflows and inspectable intermediate artifacts.All eight teachers completed individual think-aloud sessions on generating a conditionals lesson and adapting existing loops materials.Source: §4.3 · p. 11

4.3.1 Activity 1: Generating and Revising a New Lesson.

  • In the generation activity, teachers specified a conditionals lesson, revised its plan, inspected generated materials, requested quality feedback, and published it.They could revisit earlier stages to refine the plan and materials.Source: §4.3.1 · p. 12

4.3.2 Activity 2: Importing and Adapting Existing Materials.

  • In the adaptation activity, teachers imported and edited loops materials, then used AI assistance to revise them after publication.The lesson had been reviewed for suitability by an experienced computing teacher outside the cohort; moderators did not direct instructional choices.Source: §4.3.2 · p. 12

4.3.3 Think-Aloud and Post-Study Reflection Analysis.

  • Researchers inductively coded Study 2 transcripts and reflections, then independently applied categories covering support, unmet needs, constraints, and anticipated use.Screen recordings helped clarify actions when verbal comments were ambiguous; disagreements were resolved through discussion.Source: §4.3.3 · pp. 12–13

5.1 Study 1: Reported Authoring and Adaptation Practices

Teachers began with existing curricular and personal materials, adapted them to their students and teaching conditions, and selectively delegated preparation tasks to AI. They inspected and repaired AI outputs, while retaining responsibility for deciding whether lessons were ready for their classrooms.

5.1.1 Authoring Starting Points.

  • All eight teachers began lesson preparation from existing sources, including standards, curriculum guides, course resources, and materials they had previously used or created.Six explicitly described consulting standards, objectives, or curriculum guides when identifying goals and content.Source: §5.1.1 · pp. 13–14

5.1.2 Adapting for Classroom Fit.

  • Teachers adapted materials for coverage, pacing, prior knowledge, reading level, class composition, engagement, and classroom conditions.They differentiated directions and tasks, changed presentation formats, and continued revising materials during and after instruction; the extent varied with course and institutional requirements.Source: §5.1.2 · p. 14

5.1.3 Selective AI Delegation.

  • Teachers delegated specific tasks to AI, such as brainstorming, creating supplementary materials, and adapting content, while specifying intended changes and learner context.Four teachers explicitly described time savings, but teachers also limited AI’s role and evaluated or revised its outputs.Source: §5.1.3 · pp. 14–15

5.1.4 AI Output Breakdowns and Repair.

  • Teachers reported AI outputs that mismatched learners, changed content they wanted preserved, or contained unreliable information, prompting inspection, clarification, and repair.Checking and editing could offset anticipated time savings, including when teachers had to spend longer revising than creating materials themselves.Source: §5.1.4 · p. 15

5.1.5 Judging Classroom Readiness.

  • Teachers retained responsibility for readiness, judging alignment with instructional goals, content sufficiency, student understanding, and appropriate cognitive demand.They also considered whether activities were feasible with actual students, available time, and classroom conditions.Source: §5.1.5 · pp. 15–16

5.2 Study 2: Negotiating Instructional Intent and Control Through Staged Authoring

In LessonBuilder sessions, teachers clarified and refined instructional intent as they inspected plans, materials, and AI feedback. Across generation and adaptation, they maintained control through targeted revision and teacher-led judgments of classroom fit.

5.2 Study 2: Negotiating Instructional Intent and Control Through Staged Authoring

  • Teachers continued to clarify requirements and assess outputs throughout authoring rather than exercising control only at the initial request or final approval.The findings describe recurring patterns, not fixed authoring steps.Source: §5.2 · p. 16

5.2.1 Teachers Assessed Classroom Fit Across Planning and Review Phases.

  • Teachers combined structured selections and prompts to express requirements, but a single-choice instructional-style field could not represent some combinations.Reviewing generated materials also exposed fit issues, such as assumed prior knowledge, and prompted refinements to examples, activities, and objectives.Source: §5.2.1 · pp. 16–17

5.2.2 Plan Review Surfaced Additional Instructional Decisions.

  • Reviewing evolving plans surfaced instructional decisions teachers had not specified initially, including conceptual comparisons, prerequisites, activity formats, and learning environments.Planner questions sometimes prompted teachers to develop new activity structures before material generation.Source: §5.2.2 · pp. 17–18

5.2.3 Quality Review Informed Rather Than Determined Teacher Judgment.

  • Teachers treated Quality Review feedback as input rather than a verdict, accepting, revising, or contesting it using classroom and curricular knowledge.They questioned duration estimates and curricular claims while recognizing issues such as a mismatch between generated and taught programming languages.Source: §5.2.3 · p. 18

5.2.4 Post-Study Reflections.

  • Teachers viewed generation and adaptation as complementary starting points: adaptation fit commonly available materials, while generation could address new or differentiated needs.They wanted precise, targeted revision controls and quality review for adapted materials, as well as capabilities for standards, assessment, programming-language coverage, and teacher-facing guidance.Source: §5.2.4 · pp. 18–19
  • The conceptual model organizes authoring around framing requirements, creating or adapting materials, and judging readiness, with teacher-selected AI assistance and control throughout.Arrows represent possible transitions and revision loops rather than a fixed sequence.Source: §5.2.4 · p. 19, Figure 5 · p. 19
  • Across activities and reflections, teachers used pedagogical and curricular knowledge to inspect system interpretations and decide whether outputs and recommendations suited their classrooms.Source: §5.2.4 · p. 19

6 Cross-Study Synthesis: Conceptual Model and Design Recommendations

The model treats lesson authoring as an iterative process in which teachers refine instructional needs, develop materials, and judge classroom readiness while selectively delegating work to AI and retaining instructional control. It informs recommendations for selective revision, evolving requirements, teacher evaluation of AI feedback, and computing-specific support.

6.1 Teacher-Orchestrated AI-Supported Lesson Authoring

  • Teachers coordinate intent, material development, and readiness judgments through recurring authoring activities, with AI involvement but teacher control.The model describes recurring activities and decisions, not a fixed sequence.Source: §6 · pp. 19–20, §6.1 · p. 20
  • Teachers can begin with existing materials, refine instructional needs, create or adapt content, and inspect readiness, revisiting earlier work when needed.Readiness judgments concern fit to the intended class and context; they do not guarantee pedagogical effectiveness or technical accuracy.Source: §6.1 · p. 20
  • Teachers delegate bounded tasks—such as generating, supplementing, or restructuring materials—rather than handing AI the whole authoring process.AI outputs return to the broader process for inspection and possible refinement.Source: §6.1 · p. 21
  • Teachers retain authority to set constraints, choose what may change, and accept, revise, or reject AI materials, though mismatches can require repair work.Teachers may restate requests, revise materials manually, or return to an earlier activity when outputs fail to meet requirements.Source: §6.1 · p. 21

6.2 Design Recommendations for AI-Supported Lesson Authoring

  • Systems should represent requested changes and protected lesson elements, enabling local edits or rollback without regenerating unaffected materials.This recommendation responds to AI changing content or assumptions teachers intended to preserve.Source: §6.2 · p. 21
  • Systems should track revisable instructional intent across authoring so teachers can compare intermediate materials against both initial and emerging requirements.Reviewing intermediate plans can surface intents not fully expressed in an initial request.Source: §6.2 · p. 22
  • AI review should connect concerns to relevant materials and explain its criteria, while letting teachers question assumptions and act on suggestions individually.Teachers use curricular and classroom knowledge to decide whether feedback applies.Source: §6.2 · p. 22
  • Computing-specific systems should treat code, examples, tests, and technical environments as authoring components and support execution and validation.These capabilities help teachers inspect whether computing components can be executed, debugged, and used under classroom technical conditions.Source: §6.2 · p. 22

7 Discussion

Across the studies, teachers adapted materials for particular learners and conditions, clarified instructional intent while inspecting AI-generated plans and materials, and retained evaluative authority. The model characterizes AI assistance as optional and bounded within recurring authoring work.

7 Discussion

  • Teachers exercised control by setting requirements, choosing what to change and when to use AI, inspecting intermediate artifacts, and judging classroom fit.AI adoption extended beyond prompting or approving a completed lesson.Source: §7 · p. 22

7.1 RQ1: Creating and Adapting Materials for Classroom Fit

  • Teachers typically combined or extended existing curricular and other resources rather than building complete lessons from scratch.They created additional content when available materials did not provide sufficient coverage.Source: §7.1 · p. 22
  • Readiness depended on alignment with students and classroom conditions, so the same material could suit one class but need substantial change for another.Teachers considered prior knowledge, reading level, engagement, time, and feasibility; computing added concerns such as technical accuracy and technology access.Source: §7.1 · p. 23
  • Teachers’ curricular responsibilities shaped how much they authored, from localizing prescribed materials to combining resources or developing lessons.Systems should support both extensive construction and smaller adaptations.Source: §7.1 · p. 23

7.2 RQ2: Instructional Intent and Control with Structured AI Support

  • Teachers clarified initially incomplete intent while answering planning questions and reviewing proposed plans, which surfaced decisions before materials were generated.The plan gave teachers a representation against which to examine the system’s interpretation and refine requirements.Source: §7.2 · p. 23
  • Teachers repeatedly compared generated materials with prior decisions and classroom knowledge; useful transparency required being able to identify changes, revise components, or preserve accepted material.Teachers also accepted some AI review observations and contested others; AI review did not transfer evaluative authority from teachers.Source: §7.2 · p. 23

7.3 RQ3: Teacher-Orchestrated AI-Supported Lesson Authoring

  • The model makes AI delegation conditional: teachers choose the needed transformation and whether to perform it manually, delegate it, or combine approaches.Teacher authority also requires choosing targets, inspecting changes, preserving unaffected material, and deciding whether to incorporate results.Source: §7.3 · p. 24
  • The model describes work and transitions for systems to accommodate, not a universal sequence or a prescribed workflow for every teacher.Teachers may start from different points or return to earlier work when inspection reveals a missing requirement or unsuitable component.Source: §7.3 · p. 24

8 Limitations

The findings are bounded by the participant cohort, study activities, and brief tool exposure; classroom readiness was assessed during authoring rather than during subsequent teaching.

8 Limitations

  • Eight teachers from one professional development cohort provide a limited range of educational contexts, leaving transfer across curricula, institutions, and AI familiarity open.Further research across these settings could examine which parts of the model transfer or need adaptation.Source: §8 · p. 24
  • The study interface and assigned operations shaped available actions, so observed session sequences should not be read as teachers’ preferred everyday workflow.Focus groups captured accounts of usual preparation, while the tool study showed decisions made with LessonBuilder.Source: §8 · p. 24
  • Single-session tool use captures initial interactions, and readiness judgments were not checked through subsequent classroom use.Longitudinal observation could examine how delegation and judgments change with familiarity and during instruction.Source: §8 · pp. 24–25

9 Conclusion

The studies connect teachers’ evolving instructional intent with material development and classroom-readiness judgments. Their model and recommendations position AI as optional assistance while keeping teachers responsible for instructional decisions.

9 Conclusion

  • Teachers’ authoring involved iteratively refining intent, developing materials, and judging readiness while inspecting and revising AI contributions.Teachers retained responsibility for deciding whether to accept AI-generated plans, materials, and review feedback.Source: §9 · p. 25
  • The model separates deciding what authoring work is needed from deciding whether to delegate it to AI, which remains optional across recurring activities.Teachers retain control over what becomes part of a classroom lesson.Source: §9 · p. 25
  • Recommendations address selective revision, evolving requirements, teacher judgment of AI feedback, and technical and instructional inspection of computing materials.Together, they offer a basis for designing systems that help teachers carry instructional decisions through AI-supported authoring.Source: §9 · p. 25

Ethics and Privacy Statement

The study received IRB approval, and participation was voluntary, with teachers able to decline or withdraw. The research team protected confidentiality through de-identification and restricted access to study records.

Ethics and Privacy Statement

Related papers

Loading 2610.12834v1…