PaperlayerView exact revision on arXiv

Source-linked AI summary

AI Can Do Your Homework. Now What? Report from an Online Workshop on Computing Assessment in the Age of Generative AI

Muhammad Sajjad Akbar, Geoffrey Challen, Fraida Fund, Casey Hopkins, Oscar Karnalim, Kevin Lin, James McGuffee, Shubbhi Taneja, Ranysha Ware

arXiv:2609.27842v1cs.CY

TL;DR

Computing educators must decide what assignments to set as generative AI reshapes assessment. This report records approaches and questions from an online workshop of 73 educators on adapting or preserving computing assessment. Participants reported that AI’s learning benefits accrue fastest to students who already understand the material, while weaker students need training to benefit.

  • Problem

    The working group sought direct input from instructors deciding what assignments to give as generative AI’s effects are felt in computing assessment.

    Source: Why the workshop was designed this way · p. 3

  • Method

    The report synthesizes educator discussion across strategy rooms on assessment approaches and question rooms on cross-cutting concerns.

    Source: The strategy · p. 15, Discussion · p. 32

  • Results

    Participants reported that AI benefits accrue fastest to students who already understand the material, while weaker students need training to benefit.

    Source: Discussion · p. 17

  • Takeaways & Limitations

    Participants proposed assessing students’ contributions through deliverables, AI interaction logs, and reflection rather than judging output polish alone.

    Source: Discussion · p. 19

    The workshop evidence is self-reported, mostly lacks outcome data, and often concerns one offering of one course.

    Source: Limitations · p. 38

Abstract

from arXiv · show

On 28 July 2026, the SIGCSE Virtual 2026 Working Group on computing assessment and generative AI held an open online workshop attended by 73 computing educators. The first hour ran seven strategy rooms, one per approach to adapting (or deliberately preserving) computing assessment in the AI era: open-ended and authentic task design; ambitious, AI-leveraged projects; evaluating and fixing AI-produced work; controlled and AI-free assessment; process evidence and effort signals; rubric and grading redesign; and oral and interactive assessment. The second hour ran question rooms seeded from those strategies, plus two cross-cutting rooms on fairness and trust and student motivation. This report records what was discussed: the approaches participants have tried, the results they reported, and the questions every room left open. It is the first public artifact of the working group, whose taxonomy of computing assessments that accommodate generative AI use will follow.

Introduction

This report documents discussion at a two-hour online workshop on how computing educators are adapting assignments to generative AI; it is an account of practice, not measured or controlled research.

Why the workshop was designed this way

  • The workshop used discussion instead of talks, organized rooms around concrete adaptation strategies, and followed with a second round of cross-cutting questions.The strategy-first format was intended to give educators an accessible starting point, while the second round brought together participants who had made different choices.Source: Why the workshop was designed this way · p. 3

Who came, and what they wanted to discuss

  • Of 118 registrants, 73 attended, and the seven strategy rooms each drew 12–21 registrants, with no strategy attracting even a fifth of the total.Room-interest counts came from registrants’ stated preferences, not the people who attended.Source: Who came, and what they wanted to… · pp. 3–4
  • Task redesign drew 37 registrants and verification or valuation redesign drew 40; Transparent AI Use and Literacy drew one and did not run separately.Its concerns surfaced in other rooms, particularly those on rubrics and process evidence.Source: Who came, and what they wanted to… · p. 4

How the two hours ran

  • The two-hour session included fifty minutes in strategy rooms and fifty in question rooms; participants could self-select and move between rooms, but movement was not tracked.Each room opened with registrants’ submitted questions; the trustworthy-process-evidence room did not run.Source: How the two hours ran · pp. 4–5

How this report was made

  • Room accounts drew on shared-document notes and transcripts, were edited to preserve reported specifics, and were anonymized by default.The session was not recorded; the live transcript supported note-taking, and identifying institutional details were removed or generalized.Source: How this report was made · p. 5

How to read this report

  • The report groups strategy rooms and their paired question rooms in Part I, cross-cutting rooms in Part II, and recurring arguments and patterns in Part III.Seeded questions from registrants are reproduced because the editors treat them as evidence of community concerns.Source: How to read this report · p. 5

Part I: Strategies · 1. Open-Ended & Authentic Task Design

The room’s central design move was to assess what students can demonstrate by leaving choices about what to build to them; discussion tested this through AI-era projects and questions about resistance, learning, and coordination.

The strategy

  • The strategy defines assignments by what students must demonstrate, leaving them to choose what to build and how to show the target skill.The facilitator compared this to a photography assignment such as “motion,” rather than a prescribed picture subject.Source: The strategy · p. 6

Seeded questions

  • Participants left open how to teach fundamentals through AI-assisted projects, keep tasks specific yet open, scaffold across levels, assess technical quality at scale, and evaluate human contribution.They also asked how to balance specified and open-ended work and what makes a task resist having AI do it.Source: Seeded questions · p. 6

Discussion

  • In a 700-student systems course, roughly half produced easier-than-typical work and half met or exceeded expectations, but in-person test failures rose and inspected code was poor.Students submitted a written explanation of how their work met technical specifications, plus a demo video; videos were mostly genuine, while many scripts were AI-written. The instructor reported substantial grading time and difficulty training TAs to apply the unfamiliar rubric.Source: Discussion · pp. 6–7
  • A deliberately oversized commercial-system project required students to decompose work before delegating to AI, with class time spent negotiating cuts while preserving a working system.The instructor said the course had ultimately taught problem decomposition; assignments also required AI-use disclosure and citation.Source: Discussion · p. 7
  • Participants argued that tasks should be worthwhile even as AI capability changes; resistance may last only a semester, so difficulty should survive AI completing part of the work.Source: Discussion · p. 7
  • Participants said assessment change requires curricular coordination, but inconsistent faculty policies and slow discussions leave instructors making local decisions whose costs are local.Questions about whether students still need fundamentals arose independently in four rooms and were deferred to the cross-cutting chapter.Source: Discussion · pp. 7–8
  • Open-ended corpus work makes students choose comparisons and tests, while comparative-pattern tasks ask them to inspect model implementations and diagram design differences.The corpus assignment asks students to find structure and choose a story; participants described students as customers and AI as the programmer, with the student responsible for deciding what to build.Source: Discussion · p. 8

2. Ambitious, AI-Leveraged Projects

Participants described projects that make AI collaboration integral to work beyond students’ unaided reach, while identifying unresolved challenges in access, evaluation, and balancing independent effort with AI assistance. The discussion emphasized assessing students’ reasoning and AI-collaboration skills, not only the resulting artifacts.

The strategy

  • AI-leveraged projects deliberately exceed what students could complete unaided, treating AI collaboration as a premise and raising the project ceiling rather than shrinking tasks.The strategy assumes students and AI agents can undertake work beyond either’s solo capacity.Source: The strategy · p. 9

Seeded questions

  • Open questions include project structure, access and cost, designing work AI cannot fully solve, grading higher-order work, preventing handoff of scaffolded tasks, and teaching responsible agent use.Participants also asked how to build showcase-ready semester projects aligned with portfolios and industry.Source: Seeded questions · p. 9

Discussion

  • Participants described AI-supported work with large codebases, software specifications, open-source projects, and research, though how much students learned remained open.Networking students felt they contributed more than they could have unaided; a machine learning systems course embedded an ML feature in an existing open-source project, and a requirements engineering course paired specification-writing with coding agents.Source: Discussion · pp. 9–10
  • Because AI use is integral, participants considered equitable access and costs, including provider-flexible harnesses, institutionally supported or free models, and research infrastructure.Students can continue using provider-flexible harnesses with free models after the course.Source: Discussion · p. 10
  • For students with ethical, sustainability, or privacy concerns, participants suggested small self-hosted models, while emphasizing evaluation of problem framing, prompting, and critical guidance over output quality.Less capable models can expose tradeoffs between ethical considerations and performance.Source: Discussion · p. 10
  • Participants combined deliverable assessment with presentations, oral exams, demonstrations, peer feedback, and reports to examine students’ decisions and understanding.One vertically integrated program used weekly presentations to TA and mentor sub-teams to correct course throughout the semester.Source: Discussion · p. 10
  • Evaluation remains difficult: students may submit rubrics to AI, oral checks are hard to scale, and weak explanations may reflect communication or understanding; client satisfaction alone cannot establish learning.Participants therefore called for examining reasoning, decision-making, critique, explanation, and adaptation alongside final output quality.Source: Discussion · p. 10
  • Participants worried that overreliance may weaken core problem-solving, while effective AI collaboration requires effort to articulate goals, delegate, evaluate results, and integrate contributions.They identified unfamiliar-code evaluation, task decomposition, useful feedback, and agent management as important skills, but educators’ pre-AI learning paths leave the right balance unresolved.Source: Discussion · pp. 10–11

3. Evaluating & Fixing AI-Produced Work

Participants argued that assessment should develop students’ judgment of AI-produced work rather than try to prevent AI use, while verifying individual understanding through settings where AI is absent or irrelevant. They also discussed how to teach critique, motivate foundational learning, and scale assessment as AI use spreads across computing courses.

The strategy

  • The strategy makes AI-produced code the object of study: students read, test, critique, debug, and repair work they did not write.Its premise is that judgment about code remains assessable even when students do not produce code from scratch.Source: The strategy · p. 12

Discussion

  • Participants said students already use AI across courses, so assessments should focus on developing critique and use in settings where AI is absent or irrelevant.They reported that students sometimes rely on AI without reading its output, and that text-based evaluation tasks can themselves become prompts for AI-generated evaluations.Source: Discussion · pp. 12–13
  • Educators said students may question why they should learn coding, so they should explain professional expectations to evaluate, debug, test, and justify solutions and clearly communicate why some work is AI-free.Participants noted uncertainty in industry makes professional norms harder to invoke, while job interviews remain a concrete motivating anchor.Source: Discussion · p. 13
  • Participants debated whether effective critique requires prior programming fundamentals, with one proposal using a difficult early critique task to motivate learning those fundamentals.Some argued that reading code may matter more than writing it, while others compared critique to music criticism, which presupposes theory.Source: Discussion · p. 13
  • Participants noted that introductory AI code can be too good to critique, suggesting deliberately flawed outputs while recognizing that students may still use AI to complete evaluations.The seeded questions also left open how to scale critique to 400+ students, ensure students rather than AI perform it, decide when critique belongs in the curriculum, and teach skills beyond basic coding.Source: Seeded questions · p. 12, Discussion · p. 13
  • Participants generally supported combining closed-book exams for foundational knowledge with open, AI-supported assessments that reflect professional software development.They also discussed peer review, class tests, debates, and output or AI-generated unit-test analysis to scale assessment; peer assessment could count for only part of the grade, and accessibility remained a concern.Source: Discussion · p. 13
  • Participants proposed introducing code critique and software-engineering practices progressively, while acknowledging disagreement about whether code reading will remain a durable skill.They suggested beginning with correctness and style, then advancing to efficiency and design, and introducing testing and code review earlier in the curriculum; another room questioned whether models may increasingly review code for one another.Source: Discussion · pp. 13–14

Approaches described in the room

  • Participants recommended scaffolding AI use, assessing analysis of generated code and tests, using peer review, and pairing AI-supported work with in-class checks and a final proctored fundamentals assessment.They also proposed opening with a critique task students find difficult and assessing the product of AI-assisted co-development rather than the codebase.Source: Approaches described in the room · p. 14

4. Controlled & AI-Free Assessment

Participants treated AI-free assessment as a continuing way to verify learning where AI compromises outcomes, especially foundational work, while tailoring its place and format to course needs. The report reflects instructors’ practice and beliefs, not measured learning effects.

The strategy

  • The strategy preserves rather than adapts assessment, using proctoring, in-class work, or tasks designed to resist AI completion where students must demonstrate unaided learning.Participants framed the decision as determining where in the curriculum unassisted work still needs to be demonstrated.Source: The strategy · p. 15

Seeded questions

  • The seeded questions asked where AI-free assessment is needed, how to test understanding fairly and at scale, what fundamentals students should learn unaided, and how to sustain homework and motivation.They also asked where AI supports learning rather than short-circuiting it.Source: Seeded questions · p. 15

Discussion

  • Participants increasingly used proctored AI-free assessment, shifting its weight and placement toward outcomes compromised by AI, especially foundational learning early in a course.They also used it to verify students’ knowledge when submitted work made critical thinking hard to evaluate.Source: Discussion · pp. 15–16
  • Participants deployed larger projects, interviews, non-text responses, and originality-focused tasks to assess work they believed models would struggle to complete.They cautioned that model capabilities keep improving and that difficulty can arise from students’ inability to construct a prompt, not from a question itself.Source: Discussion · p. 16
  • For 200-student classes, participants said AI-free assessment was possible but difficult to verify and time-consuming to grade fairly.They reported false positives and indistinguishable AI-generated work with detectors, and used tutor-shared verification, automated marking, or fewer costly assessments; digital proctored entry was one machine-gradable alternative to paper exams.Source: Discussion · p. 16
  • Participants encouraged freshmen to learn fundamentals without AI, while AI-free assessment in senior courses depended on whether the course remained foundational or was advanced and applied.For homework, reported approaches included assignments too large for AI alone, flipped-classroom supervision, beginning without AI, and micro-milestones with version-control history. Flipped classrooms were suggested for large assignments; micro-milestones and version control for distance students were suggestions, not reported experience.Source: Discussion · pp. 16–17
  • Participants defined fundamentals broadly, from foundational concepts and problem solving to AI literacy, platform competence, and code comprehension, and proposed less building from scratch and more verification and adaptation.They suggested memorization may no longer belong; unaided-work checks included controlled assessments, scaffolding, interviews, presentations, oral exams, and preliminary self-reflections. Simplified scales or detailed rubrics could support consistent qualitative grading, but communication barriers in oral formats remained unresolved; the facilitator’s inferred position was that introductory assessments should be AI-free or controlled before vibe coding.Source: Discussion · p. 17
  • Participants said AI’s personalized tutorials benefit students fastest when they already understand the material, have strong computational thinking, or can direct the tool effectively.They noted weaker students need training before they can benefit.Source: Discussion · p. 17

5. Process Evidence & Effort Signals

Participants argued that assessment should look beyond the submitted artifact to the process by which students produce it, while acknowledging that process evidence can itself be AI-generated. They discussed ways to assess that process and motivate students to value it, but left unresolved how to do so reliably, fairly, and without excessive burden.

The strategy

  • Process evidence can include AI interaction logs, drafts and commit history, prompt portfolios, time on task, and reflections, shifting assessment from the artifact to how the work was done.Participants questioned whether any of these records can reliably prove that learning occurred if the documentation itself can be AI-generated.Source: The strategy · p. 18

Seeded questions

  • Participants asked how to verify process and reflection, grade documentation without busywork, teach prompt-writing, and help students value genuine learning over the finished answer.They also questioned how to make reflection meaningful rather than a superficial checkbox.Source: Seeded questions · p. 18

Discussion

  • Participants saw AI as an opportunity to assign more realistic professional work, but struggled to identify prerequisite skills for novices and align process-focused assessment with learning objectives.They also raised agency and responsibility, student grade-focused mindsets, and the challenge of adapting within a rapidly changing field as concerns in helping students value learning beyond answers.Source: Discussion · pp. 18–19
  • Oral assessment can expose process, but is time-consuming and may reward students who study AI-generated work without being able to produce it themselves.Participants identified no reliable way to measure unaided capability short of a lengthy invigilated session.Source: Discussion · p. 19
  • A proposed submission combines a deliverable, AI interaction logs showing the student’s contribution, and a video reflection, but participants noted AI could script the video too.Assessing the account of contribution rather than output polish may help address inequities in access to AI tools; the broader concern is that AI may follow each proposed shift in evidence.Source: Discussion · p. 19
  • Participants framed student acceptance of process-weighted grading as persuasion: instructors should explain the long-term value of skills and position themselves as allies in students’ learning.They described this as a way to counter the adversarial relationship intensified by AI, while noting that AI can also help instructors create more interesting assignments.Source: Discussion · p. 19
  • The Build Your Own Feature assignment embeds process in an engineering design lifecycle, but fine-grained skills may still need controlled assessment and repeating the pattern across courses risks sameness.The facilitator described the assignment as integrating smaller learning objectives into a larger lifecycle rather than directly assessing fundamental objectives.Source: Discussion · pp. 19–20
  • Participants proposed assessing how students coach AI through a process, including structured decomposition; the AI-Enhanced Developer profile in Kam et al. (2025) was cited as one account of that competence.They distinguished evidence of going through a process from simply producing the final artifact.Source: Discussion · p. 20

6. Rubric & Grading Redesign

Participants discussed shifting grading toward process while retaining product quality, and explored ways to verify learning in AI-assisted work. They left open how to distinguish genuine understanding from polished products or accounts of process, and how much foundational practice students still need.

The strategy

  • The strategy shifts what grades measure: rubrics can name AI use, weight process and product, permit AI with disclosure, and assess AI literacy explicitly.This frames AI as something students are assessed about as well as something they use.Source: The strategy · p. 21

Seeded questions

  • Seeded questions asked how to balance product and process, grade AI-assisted homework relative to exams, serve AI users and avoiders fairly, and preserve productive learning friction.Participants also questioned equity for students without paid access, readiness for process grading, and application to team projects and systems or networking courses.Source: Seeded questions · p. 21

Discussion

  • Participants favored greater emphasis on process while retaining product, citing standards-, mastery-, and specs-based grading as ways to reward demonstrated learning over one deliverable.Clear specifications may also improve product quality, though participants noted that students can push back on process-focused rubrics and that AI-completed deliverables alone are weak evidence of learning.Source: Discussion · p. 21
  • Participants urged early clarity about course goals, cheating, and when process matters, while questioning whether AI-generated process accounts reveal learning or tool access and prompting skill.They also raised whether process-based grading is valid when AI can reconstruct process, and what assessment reliably demonstrates genuine learning.Source: Discussion · pp. 21–23
  • For stronger verification, participants proposed oral explanations, mastery or fluency checks, whiteboard exams, in-class debugging, and multi-stage peer projects.They recommended relying primarily on directly verifiable work for high-stakes credit and using take-home assignments for practice; oral assessments may require accommodations and language options so grades reflect communication and process rather than English fluency.Source: Discussion · p. 22
  • When AI is permitted, rubrics can assess the thinking behind a product through prompts, source citations, tool comparisons, reflections, design reasoning, or non-code artifacts.Participants urged rewarding understanding and substantive decisions over polish, and targeting higher-order analysis, evaluation, and design rather than tasks AI can readily complete.Source: Discussion · pp. 22–23
  • Tiered grading can offer a baseline grade ceiling and optional assessments for higher grades, preserving choice for students who avoid AI; AI literacy, if disciplinary, should be an explicit learning goal.Participants cited ethical objections to AI use and noted that institutional constraints and mixed faculty messages can complicate student choice.Source: Discussion · pp. 22–23

7. Oral & Interactive Assessment

The room favored live oral and interactive assessments because students’ answers make them robust to how submitted work was produced, while identifying scale as the most consistent objection. Participants described practices for scoring students’ explanations, distributing or sampling assessment work, and distinguishing poor performance from misconduct, alongside unresolved questions about rubric design, scale, peer assessment, and comprehension.

The strategy

  • Live oral exams, checkpoints, walkthroughs, presentations, and demos were considered robust to how submitted work was produced, with scale the most consistent objection.The room drew the workshop’s most detailed practitioner experience.Source: The strategy · p. 24

Seeded questions

  • Participants posed questions about oral-assessment rubrics, scaling, peer or UTA assessors, grading consistency, and probing comprehension of AI-generated code.The Hour 2 question room also asked how oral, interactive, or authentic assessments scale to hundreds of students and what must be given up at scale.Source: Seeded questions · p. 24

Discussion

  • Oral rubrics focused on the quality of a student’s account rather than the artifact, including explanation without help and precision.Examples included scripted questions with attached rubrics and a CS2 five-point scale; code questions could be generated from the student’s own implementation.Source: Discussion · p. 25
  • Participants addressed scale by distributing or sampling assessment: peer presentations counted for up to 40% of an assignment, while code reviews for 200 students were distributed across 10–15 calibrated TAs.Other approaches included calling only a subset of students for individual oral assessments each week and using a 20-minute algorithm-design-and-proof question.Source: Discussion · p. 25
  • For grading consistency, participants suggested consent-based recordings, withholding grades until all students had completed the assessment, and staff role-play, while accepting more inconsistency on semester-end projects than weekly work.Role-play could help TAs become comfortable conducting oral assessments even when it did not calibrate grading.Source: Discussion · p. 26
  • Whether AI use was permitted determined whether successful explanation of AI-generated code merited credit; when misconduct was unestablished, participants proposed downweighting the artifact and increasing oral-assessment weight.They recommended requiring students to extend or modify code in real time, since explaining a submitted artifact alone could be rehearsed.Source: Discussion · p. 26
  • Distance, short courses, testing centers, remote-proctoring review burdens, recording restrictions, and national rules constrained scale; participants disagreed about AI’s role in assessment.Some described local models generating per-student assessments or artifact-based questions, while another rejected AI grading on principle; participants also noted a student’s oral score at the 70th percentile versus homework at the 90th.Source: Discussion · pp. 26–27

Part II: Cross-cutting Question Rooms · 8. Fairness & Trust

Participants grappled with how to assess learning fairly when AI can produce much of the work instructors once evaluated directly, while preserving credible grades and trust. They discussed process evidence and controlled assessments, but left open how to distinguish learning from offloading without surveillance or adversarial interactions.

The question

  • Integrity, unequal access to AI, and instructor-student trust surfaced across every strategy, prompting a cross-cutting room on fair assessment.Source: The question · p. 28

Seeded questions

  • The room asked how to distinguish performance problems from integrity cases, handle unequal AI access, and rebuild trust without turning assessment into surveillance.Source: Seeded questions · p. 28

Discussion

  • As AI changes technical work, participants questioned whether learning outcomes and grades still measure intended learning, and emphasized problem decomposition, iteration, testing, judgment, and responsibility alongside code correctness.They also stressed that assessment must remain reliable and defensible while distinguishing genuine learning from work substantially offloaded to AI.Source: Discussion · p. 28
  • Process logs, code histories, testing records, and reflections can reveal problem-solving and encourage responsibility, but participants viewed them as metacognitive aids rather than reliable grading evidence.Students may omit AI use, have unequal access to paid tools, interpret AI use inconsistently, or avoid disclosures they perceive as risky; finding a non-antagonistic way to ask remained unresolved.Source: Discussion · p. 28
  • Some instructors reported strong project outputs alongside declining written-assessment performance on foundational material, while curricula lag changing AI tools and industry expectations.Because grades remain signals to graduate programs and employers, participants felt pressure to make them credible and comparable.Source: Discussion · p. 29
  • Controlled exams, quizzes, computer-based tests, orals, demonstrations, and individual meetings can provide stronger evidence of individual understanding, but consume time, resist scaling, and may increase anxiety.Open-book exams, retakes, application-focused questions, and controlled testing environments were discussed as ways to reduce stress and visible cheating without making students fear that innocent behavior will be misread.Source: Discussion · p. 29
  • Conventional exams cannot adequately assess extended design, collaboration, or project work, while laboratory-style supervision raises questions about isolation and oversight.One participant also resisted using class time for quizzes.Source: Discussion · p. 29

8. Fairness & Trust

  • Participants said misconduct by a few students can undermine instructor confidence and breed resentment, while surveillance, in-class work, and repeated warnings can make honest students feel distrusted.They emphasized clear rules, starting from trust, responding transparently to breaches, and accountability without adversarial assessment; high education costs and competition add stakes neither students nor instructors chose.Source: Discussion · pp. 29–30

9. Motivation

Participants described student motivation as varied and hard to reduce to one answer, and argued that foundational learning remains necessary for students to understand and evaluate AI-produced work. The discussion also emphasized explaining the purpose of learning and creating opportunities for student ownership and peer interaction.

The question

  • The room’s cross-cutting question was how to keep students motivated to learn, treating motivation as an object of discussion rather than only an assessment constraint.Source: The question · p. 31

Seeded questions

  • The seeded questions asked how to motivate effort when AI can produce answers, align process or authentic assessment with intrinsic motivation, and sustain interest in difficult material.Source: Seeded questions · p. 31

Discussion

  • Participants named career prospects, gaming, and problem-solving for its own sake as student motivators, while several concluded motivation resists a single answer.When asked about their own motivation, participants named the fun and joy of learning, which they thought was not what most of their students were working from.Source: Discussion · p. 31
  • Two participants estimated motivation groups at 10/20/50/20 and 10/60/20/10; both estimates put fully self-motivated students at roughly one in ten.The estimates disagreed substantially about the middle groups but suggested most students rely on career prospects, structure, and external nudges.Source: Discussion · p. 31
  • Participants proposed building ownership by explaining the purpose of course content, holding brief private reflection conversations, and using small-group discussions about learning’s value.The room also acknowledged that the profession had not done a good job explaining why it teaches what it teaches, even before AI made the question urgent.Source: Discussion · pp. 31–32
  • Participants questioned whether motivating students is faculty’s job, but argued that peer interaction—including for high achievers—is a value of university learning that faculty can foster.They also worried that grades were becoming the only motivational tool available, while recognizing that grades are necessary but insufficient.Source: Discussion · p. 32
  • Participants argued that foundational problem-solving remains valuable, since AI can misrepresent students’ understanding and students need fluency to catch errors, debug systems, and evaluate its output.They compared AI to a calculator, and noted that students also need to read and understand their AI conversations.Source: Discussion · p. 32
  • A participant pasted an approximately 900-word language-model reply to the seeded questions into the shared document; this account sets that reply aside and summarizes the human discussion.The report records the episode as a datum about the workshop’s moment.Source: Discussion · p. 32

Part III: Themes Across the Workshop

Workshop participants left unresolved which computing fundamentals and code skills students need as AI capabilities expand, while describing assessment constraints and largely instructor-by-instructor responses.

Do students still need the fundamentals, and what are they now?

  • Participants disagreed whether AI changes the need for fundamentals: some defended deep coding knowledge, while others argued models already handle tasks and curricula could make room for useful computational work.Proposed enduring skills ranged from foundational concepts, code comprehension, correctness, design, and systems knowledge to evaluating unfamiliar code and managing agents; educators also noted that AI benefits students with stronger prior understanding most.Source: Do students still need the… · pp. 33–34

Does code remain something humans work with?

  • Participants questioned whether code-reading remains durable when models review and revise code, leaving unresolved whether code is a human-facing artifact or the right unit of instruction.Counterarguments included resilience and cost-based reasons to retain coding ability, and the claim that code’s precision is necessary to verify what models infer from ambiguous natural language.Source: Does code remain something humans work… · pp. 34–35

Scale is the binding constraint

  • Nearly every assessment remedy became harder at scale, and participants said evidence-rich approaches were least available to institutions teaching the most students.Implemented responses distributed or sampled work through peer review, calibrated TA pools, rotating student checks, testing centers, and automated grading; participants also debated AI-generated questions and human grading.Source: Scale is the binding constraint · pp. 35–36

Process evidence is valued but not trusted

  • Instructors collected process evidence but did not grade it, because disclosures are inconsistent and AI can generate plausible accounts of processes that never occurred.A proposed submission combining deliverables, interaction logs, and video reflection did not resolve the concern: students had used AI to script video reflections, and no process evidence was identified as resistant to generation.Source: Process evidence is valued but not… · p. 36

Unequal access is a validity problem, not only a fairness one

  • Unequal access to paid AI tools threatened not only fairness but assessment validity, because process evidence could measure purchasing power and leave instructors unsure what grades represent.Participants discussed institutional access, provider choice, free or self-hosted models, and support for students declining AI use, though less capable alternatives could produce worse results.Source: Unequal access is a validity problem,… · p. 37

Difficulty is now a moving target

  • Assignment difficulty became a moving target as models improved, while resistance strategies could expire within semesters and curricular revision took years.Participants also noted that assessment difficulty partly depends on students’ prompting, making “hard for AI” a feature of the interaction rather than the question alone.Source: Difficulty is now a moving target · p. 37

Almost everything here was done by someone acting alone

  • Most changes were local instructor improvisations, amid departmental disagreement and slow curricular discussions that left current students facing mixed messages.Participants described incompatible course policies, limited institutional freedom, and no coordinated disciplinary response.Source: Almost everything here was done by… · p. 38

Limitations · About This Report

This report records a self-selected workshop conversation, not measured outcomes, and its discussion reflects uneven room participation and a late-July 2026 snapshot. The working group drafted sections from facilitator notes and transcripts, then reviewed them with room facilitators.

Limitations

  • Workshop participation was self-selected twice, over-representing educators who believed assessment must change and those with latitude to redesign courses.Educators prevented from attending by constraints are invisible in the report.Source: Limitations · p. 38
  • Nothing was measured: reports of success were mostly self-reports without outcome data, often from one course offering.The sole numeric report—a five-point semester-to-semester exam-score drop—was accompanied by a hypothesis, not a finding.Source: Limitations · p. 38
  • Room sizes and discussion depth varied, although the report’s sections were made comparable in length editorially.Some rooms had twenty participants and skipped introductions; others had four and worked through every question.Source: Limitations · p. 38
  • This late-July 2026 snapshot may quickly become outdated, as participants said positions about model capabilities had shifted within six months.The report notes that positions recorded here may also not survive six months.Source: Limitations · p. 39

About This Report

  • Facilitators took notes in shared editable documents; the session was not recorded, and the working group drafted sections from notes and transcripts for facilitator review.Zoom’s live transcript supported note-taking, and some facilitators kept fuller transcripts of their rooms.Source: About This Report · p. 40
  • The report is anonymous by default, removes or generalizes identifying details, and attributes no ideas to named individuals.Contributor acknowledgments are opt-in, collected after the draft circulated, and remain open; two shared teaching resources are linked with permission.Source: About This Report · p. 40
  • The nine working group members are the report’s authors, while workshop participants who opt in are listed separately as contributors.Both lists are alphabetical by surname; all nine authors facilitated or supported rooms and contributed to the report.Source: About This Report · p. 40
Loading 2609.27842v1…