The Robots are Here: Navigating the Generative AI Revolution in Computing Education

Motivation

• Since ChatGPT’s November 2022 release, students have actively used freely available generative AI across computing tasks rather than merely encountering AI through educational tools.LLMs now support source-code generation and interpretation through natural-language interfaces [72], and early reports document assignment use.Source: §1 · p. 2

Challenges

• LLMs may undermine introductory programming practices because they readily solve simple problems [164], while producing wrong, biased, plagiarised, or poorly understood code [101].Students may also become over-reliant on models, and AI detectors are currently ineffective [134].Source: §1 · p. 2

Opportunities

• With appropriate context, scaffolding, and support, LLMs can partner in pair programming and explain code or error messages [101] [115] [148].They can also help instructors rapidly create personalised teaching materials and assignments [73] [163], including new Prompt Problems [70].Source: §1 · p. 2

Report contribution

• LLMs are expected to change how, what, and whom educators teach across computing and education as the technology matures.The report aims to summarize early movements in computing education, set a research agenda, and collect effective educator practices.Source: §1 · p. 2

Reviewing Literature

• The authors review LLM literature in computing education and provide a guide to the field’s opportunities and challenges.Source: §2 · p. 2

Evaluating Current Attitudes

• An international survey of students and instructors provides a snapshot of current attitudes toward LLMs and their uses.Source: §2 · p. 2

Identifying New Instructional Approaches

• Interviews with instructors reveal advantages and disadvantages of teaching about and using LLMs in computing education.Source: §2 · p. 2

Exploring Ethical Implications of LLMs

• The report analyzes university AI policies through the ACM Code of Ethics and addresses academic integrity, offering resources on permissible LLM use.Source: §2 · p. 2

Encouraging Replication

• Replicating prior work with new LLMs highlights challenges from rapid model improvement and inconsistent research methods, while identifying datasets and dataset-quality concerns.Source: §2 · p. 3

3.1.2 Snowballing (phase 1).

• 71 papers were included after two forward-snowballing phases, reflecting the rapid growth of research on LLMs in computing education.The review began with 10 seed papers and ultimately included 10 reference papers, 28 papers from the first phase, and 33 from the second.Source: §3.1.2 · p. 4, §3.1.3 · p. 5

3.3 Classification of literature

• The literature clustered into five roles, with assessing LLM performance, capabilities, and limitations by far the largest category.Other categories covered teaching-material generation, student-work analysis, programmer–LLM interactions, and position papers or surveys/interviews.Source: §3.3 · p. 5

3.3.1 Assessing the performance, capabilities and limitations of LLMs (35):

• LLMs generally matched or exceeded average students on code generation, but performed less well on multiple-choice, Parsons, computational-thinking, semantic-understanding, and code-reasoning tasks.Question- and hint-generation findings were inconclusive, while LLM-generated code explanations often outperformed student explanations; detection tools showed 96-98% accuracy but text detectors produced many false positives.Source: §3.3.1 · p. 5

3.3.2 Position papers and surveys/interviews (17):

• The reviewed literature broadly agreed that LLMs will substantially affect computing education, but interviews found no consensus about their higher-education benefits or risks and widespread concern about academic integrity.Professional developers saw many use cases but still needed to understand generated code because quality varied and over-reliance was a risk.Source: §3.3.2 · p. 7

3.3.3 Studying the interactions between programmers and LLMs (9):

• AI-code-generator use improved novice outcomes without harming learning, yielding 1.15x progress, 0.59x errors, 1.8x higher correctness, and 0.57x time spent.Students used exploration, acceleration, shepherding, and drifting interaction modes; generated explanations were generally useful, while perceived utility and engagement varied by explanation type.Source: §3.3.3 · p. 7

3.3.4 Using LLMs to analyse student work (5):

• LLM-assisted student-work analysis surpassed prior automated-repair results, reaching up to 96.5% repair and 89.4% repair, while error-message enhancement averaged about 54% and remained unsuitable for direct student use.A feedback method reached 72.4% precision with 64.2% coverage on one dataset and 76% precision with 31.2% coverage on another.Source: §3.3.4 · p. 7
• LLMs could generate novel, sensible, themed programming exercises and reasonable learning objectives, potentially saving instructor time, but quality was not consistently sufficient for direct student deployment.LLM-generated content was reported as comparable to student-generated content in one study, while another proposed tool lacked an evaluation.Source: §3.3.5 · p. 8

3.4 Educational opportunities and risks

• Opportunities included reducing instructor workload, personalising student help, supporting pair programming and problem solving, and generating resources, but erroneous outputs could confuse novices and model quality changed rapidly.The reviewed literature also identified pedagogical uses for natural-language solution expression, scaffolding, hints, code repair, and clearer error messages.Source: §3.2 · p. 5, §3.5 · p. 8, §3.4 · p. 8

3.5 Opportunities

• LLMs can reduce instructor workload, enhance learning experiences, support problem-solving, and enable assessment focused on code understanding, review, and grading.Their strong performance on typical CS1 and CS2 problems makes assessment redesign essential.Source: §3.5 · p. 9

3.6 Risks

• AI-generated learning resources may contain errors, incomplete tests, unsuitable content, poor style, and limited example variety, potentially teaching students incorrect or undesirable practices.Authoritative presentation can discourage students from questioning incorrect material.Source: §3.6.1 · p. 9

3.6.1 Risks for students.

• Students may waste time prompt-engineering or pursuing hallucinated problems, while line-by-line explanations and over-reliance can displace higher-level understanding and obscure actual mastery.Novices may also become demoralised, confused, or frustrated when AI outperforms them or produces overwhelming code.Source: §3.6.1 · p. 9

3.6.2 Risks for teachers.

• Teachers face academic-integrity concerns because AI performs well on introductory assessments and generated solutions are difficult to identify as plagiarism, requiring new strategies and resources.Educators must balance early over-reliance against late, inauthentic introduction, teach effective tool use, and manage flawed AI-generated exercises and reduced scaffolding.Source: §3.6.2 · pp. 9–10

3.6.3 Risks for the community.

• More knowledgeable students and professionals benefit more from code generation, potentially widening achievement gaps, while generated code may introduce bias, bugs, and security vulnerabilities.Tool design should seek more direct benefits for novice programmers.Source: §3.6.3 · p. 10

3.7 Limitations and threats to validity

• The review covered only literature from 2021–2023, used one snowballing step, and included arXiv and grey literature, so recent, less visible, or insufficiently peer-reviewed work may be missing.The authors retained included papers despite some reviewer concerns about quality.Source: §3.7 · p. 10

4.1 Prior explorations of student and instructor perceptions.

• Prior studies likewise found that students were generally more optimistic than educators, while both groups raised concerns about dependence, ethics, trustworthiness, plagiarism, and the need to adapt computing education.Interviewed educators variously favored invigilated assessment, resisting AI, or curricular integration through personalized help, code critique, open-ended design, and accessibility support.Source: §4.1 · pp. 10–11

4.2 Methods for data collection and analysis

• The study conducted two corresponding online surveys with 57 instructors and 171 students, including 27 identical or lightly reworded questions and thematic analysis of 228 open responses.Source: §4.2.1 · p. 12, §4.2.2 · p. 12, §4.2.3 · p. 12

4.3.2 Comparison of student and instructor perceptions.

• Students and instructors similarly used GenAI mainly for code and text, but students had slightly more coding experience; both groups supported restrictions while differing over policy clarity.Instructors largely viewed university policies as unclear, whereas students expressed mixed views, although both groups more strongly agreed that course policies were defined clearly.Source: §4.3.2 · p. 14
• Students and instructors largely agreed that GenAI cannot replace human instructors, while expecting its role in teaching, learning, and careers to increase.Both groups considered human teachers more effective sources of guidance, suggesting instructors may increasingly emphasize curating learning environments and achieving learning objectives.Source: §4.3.2 · p. 14

4.3.3 Help-seeking behaviors.

• Students primarily sought help through web searches, but GenAI surpassed discussion forums and was more favored by upper-level students, whereas first-year students preferred peers.Reliance varied by academic stage, potentially reflecting differences in help sought or willingness to adopt emerging technologies.Source: §4.3.3 · p. 14
• Instructors unanimously viewed auto-generating complete assignment solutions without understanding the code as unethical, while opinions split when students understood the code or translated between languages.Instructors generally accepted partial solution generation, debugging, concept explanation, and style improvement when learning outcomes were preserved; 50.8% believed many or almost all students used GenAI unethically.Source: §4.3.4 · p. 14

4.4.1 Instructor use of GenAI.

• Most instructors were not using GenAI for text or code generation, but users employed it to create assessment questions, course materials, examples, exercises, explanations, and other educational resources.Reported limitations included variable output quality, insufficient course context, and cases where writing code themselves was faster, although some valued generated data and critique activities.Source: §4.4.1 · pp. 14–15

4.4.2 Instructor observations of student use of GenAI.

• Instructors observed student GenAI use less often than they reported using it themselves, with 20% of responses describing academic misconduct involving copied complete solutions and over-reliance.Reported problems included submitting generated work as students’ own and failing to check or understand incorrect or inappropriate outputs.Source: §4.4.2 · p. 15
• Instructors observed GenAI supporting debugging, test-data generation, code explanation, conceptual learning, and English communication, but also generating complete assignment solutions and enabling misconduct.Some instructors also reported student frustration with AI-generated code and uncertainty when misconduct was suspected without direct proof.Source: §4.4.2 · pp. 15–16

4.4.3 Student use of GenAI.

• Around 40% of students had not used GenAI for either text or code generation, while common uses among adopters included paraphrasing, summarising, debugging, and code generation.Text paraphrasing or summarising was reported by 20% of students, new-text generation by fewer than 15%, and code debugging by 25%.Source: §4.4.3 · p. 16
• Some students rejected GenAI because they viewed struggling, debugging, originality, and the discovery process as essential to learning.Students also argued that relying on generated solutions can undermine understanding and defeat the purpose of coursework and examinations.Source: §4.4.3 · p. 16, §4.4.4 · pp. 17–18

4.4.4 Student perceptions of the effects of GenAI tools on employ-

• Students expressed mixed employment expectations: 33 feared reduced opportunities, while 28 were unconcerned; others believed GenAI would improve productivity and create new careers.Concerns included threats to entry-level software work, AI-assisted résumé screening, and difficulty distinguishing genuine skills from AI-enhanced portfolios.Source: §4.4.4 · pp. 16–17
• Both instructors and students generally favored conditional GenAI use based on course level, assessment type, task goals and size, and whether use involved understanding rather than blind copying.Instructors tended to restrict GenAI during foundational learning, while students commonly supported upper-level or coursework use but opposed use in exams, major assessments, or concept-learning tasks.Source: §4.4.4 · pp. 17–18

5 CURRICULUM AND ASSESSMENT

• LLMs’ ubiquity is driving possible redesign of computing curricula, pedagogy, assessment, and delivery, despite education systems’ slow institutional change.The discussion applies constructive alignment, linking learning objectives with exercises, assignments, and assessment methods.Source: §5 · p. 18

5.1 Methodology for expert interviews

• The interview study purposively recruited educators planning changes to course structures, assessments, or classroom practices and analyzed interviews thematically.The sample comprised 22 computing educators from nine countries and five continents; interviews were semi-structured, transcribed, and reviewed by multiple authors.Source: §5.1 · p. 19

5.2 Learning objectives

• Only 3% of 234 reviewed syllabi listed tracing program execution, while reading, explaining, and identifying foreign-code output appeared on fewer than 10% of German university syllabi.Writing programs was among the most common objectives, motivating renewed emphasis on code reading as students increasingly use LLMs to generate programs.Source: §5.2 · pp. 19–20

5.2.1 How instructors are changing their learning objectives.

• Instructors are adding LLM-related objectives, critical evaluation, prompt engineering, problem decomposition, code comprehension, and cybersecurity topics, while official curricula often remain unchanged.Some educators require citations for LLM-generated code, teach students that LLMs are fallible, and anticipate further changes as technology evolves.Source: §5.2.1 · p. 20

5.2.2 Preserving core learning outcomes.

• Many educators preserve foundational programming objectives while using LLMs as complements, balancing coding practice with algorithmic thinking, program comprehension, debugging, and future industry expectations.Some intend to introduce LLMs after students master traditional concepts, while others maintain hands-on coding to build knowledge beyond what tools can perform.Source: §5.2.2 · pp. 20–21

5.2.3 Towards conversational computing.

• For non-majors, LLM tools may support computational-thinking goals or make programming-specific courses unnecessary, potentially broadening participation and improving educational quality.This perspective treats LLMs as quantitatively intensifying longstanding educational issues rather than introducing wholly new ones.Source: §5.2.3 · p. 21

5.3 Course activities

• LLMs can generate assignment variations, code explanations, iterative solution support, and tutoring, but constrained tools such as CodeHelp aim to prevent complete-solution dependence.These uses may reduce teaching-assistant burdens and provide students with personalized, around-the-clock assistance, while raising academic-integrity concerns.Source: §5.3 · p. 21

5.3.1 In-class Activities.

• In-class practices include prompt-engineering exercises, live analysis of incorrect outputs, and demonstrations of LLM limitations, while some instructors exclude or defer LLM use to protect fundamentals and equity.Educators variously frame LLMs as hints or help rather than oracles, note novices’ difficulty filtering buggy solutions, and worry about biased outputs affecting marginalized students.Source: §5.3.1 · p. 22

5.3.2 Unsupervised Activities.

• LLMs can support unsupervised learning and formative feedback, but misleading outputs and over-reliance require guidance, critical reflection, and ensuring students understand their own work.ChatGPT performs reasonably well on some introductory programming tasks, while a classroom comparison found ChatGPT users solved problems faster but appeared to understand and critically assess outputs less than e-book users.Source: §5.3.2 · pp. 22–23

5.4.1 Exams.

• Assessment is shifting toward invigilated exams, oral components, and course-specific responses because written submissions and unsupervised programming work are no longer reliable evidence of intellectual achievement.Some instructors reduced programming-assignment weighting, introduced coding interviews, planned 20–30 minute oral exams, or required answers to use course-specific terminology.Source: §5.4.1 · p. 23, §5.4.2 · p. 23

5.4.2 Homework.

• Homework is increasingly permitted or redesigned around LLM use, with educators assessing staged submissions, reflections, critiques, interviews, portfolios, journals, and presentations rather than only final products.One course changed a scientific-paper assignment to allow LLM production, while another replaced exercises with diary-based reflection assignments documenting learning processes.Source: §5.4.2 · p. 24, §5.4.3 · p. 24

5.4.3 Process over product.

• Educators are emphasising process, explanation, code reading, debugging, and critical thinking over writing code from scratch, including assignments that require students to diagnose why LLM-generated code is wrong.Some interviewees proposed a ‘use, modify, create’ progression, while others cautioned that basic competencies remain necessary because LLMs can amplify expertise but cannot replace it.Source: §5.3.2 · p. 23, §5.4.3 · p. 24

5.5 Institutional initiatives, policy, and context

• LLM policies range from bans to deliberate integration, but institutional responses remain unsettled and can be constrained by testing resources, timetabling, and accreditation cycles.Interviewees described prohibited in-class use, planned quiz integration, gradual introduction after basic competencies, and learning-objective changes delayed by five-year accreditation cycles.Source: §5.5 · p. 24

5.6 Other challenges and opportunities

• Interviewees identified challenges involving blind spots, cheating, intellectual laziness, over-reliance, and learning avoidance, alongside opportunities to rethink teaching, individualise support, and scale feedback.Proposed opportunities include personalised progress and feedback, broader support for educators and students, and automated concept maps explaining mistakes across large numbers of exam scripts.Source: §5.6.1 · pp. 24–25, §5.6.2 · p. 25

5.7 Discussion

• The discussion calls for constructive alignment between explicit and hidden program outcomes, learning activities, and assessments, while recognising that novice developers may execute code they cannot evaluate.The authors note that LLMs may expose implicit goals such as code reading, debugging, self-direction, teamwork, and persistence, and report that LLMs solved 97% of programming problems in one bioinformatics course [142].Source: §5.7 · pp. 25–26

5.8 Limitations and threats to validity

• The interview findings are limited by geographically skewed and gender-imbalanced sampling, semi-structured interviewer bias, and unverified participant claims about their classes, departments, and institutions.The interview pool included 22 educators across five continents but no African participants, only one each from Asia, Oceania, and South America, and three women.Source: §5.8 · p. 26

6 ETHICS

• LLM ethics concerns societal effects, opaque training, unclear authorship responsibility, academic integrity, and possible influence on users’ opinions.The paper emphasizes provenance and quality of training data, usage, and associated costs rather than integrating ethics into the system.Source: §6 · p. 26

6.1 Ethics in LLM literature

• Reviewed computing-education papers emphasized performance, generalisation, efficiency, novelty, scalability, and practical uses such as solving coursework or producing teaching materials.Unlike ML research, performance was often used to signal research timeliness rather than propose performance improvements; “free” access can hide data costs and unequal model access.Source: §6.1 · pp. 26–27

6.2 Code of ethics implications

• The paper uses the ACM Code of Ethics’ general principles to frame ethical issues raised by LLMs in computing education.The ACM code covers fundamental principles, professional responsibilities, ethical leadership, and compliance.Source: §6.2 · p. 27

ACM General Ethical Principles.

• University policies were often unavailable or still developing, and most sampled policies did not explicitly address coding, limiting clarity about permitted GenAI use.The review purposefully sampled top-ranked universities in Canada, the USA, the UK, and Australia rather than conducting a systematic survey.Source: ACM General Ethical Principles · p. 27
• Only MIT linked LLM adoption to student academic well-being, while sampled policies generally omitted broader human well-being and stakeholder responsibilities.The paper characterizes this as an oversight because institutions can guide students toward concerns beyond their immediate circumstances.Source: ACM General Ethical Principles · p. 27
• Policies identified risks that LLM use can undermine learning, critical thinking, and future competence while producing incorrect, fabricated, outdated, biased, or discriminatory outputs.They also raised concerns about unequal access, originality, privacy, confidentiality, intellectual property, and disclosure or reuse of user data.Source: ACM General Ethical Principles · pp. 27–28

7 ACADEMIC INTEGRITY IMPLICATIONS

• The report identifies plagiarism, collusion, contract cheating, falsification, and unauthorised resources as academic-integrity concerns involving generative AI.Plagiarism involves unattributed use, collusion involves unauthorised collaboration, contract cheating involves outsourcing assessment work, falsification involves invented content, and unauthorised resources are prohibited tools or materials.Source: §7 · p. 28

7.1 Plagiarism

• The report treats the student using generative AI as the author, so AI-generated content is generally not plagiarism, except when a tool reproduces someone else’s work exactly.AI-generated work should not be cited as independently authored, while AI-detection tools are unreliable and produce false positives, especially for code [134].Source: §7.1 · p. 29

7.2 Collusion

• Because generative AI lacks agency, the report does not classify submitting generated content as collusion.Collusion presupposes willing cooperation between parties with agency, whereas generative AI is treated as a tool rather than a collaborating person.Source: §7.2 · p. 29

7.3 Contract cheating

• The report rejects classifying generative AI use as contract cheating, despite policies that explicitly include AI tools, because the student remains the author.It compares generative AI with calculators and other software tools, while acknowledging that AI can produce more extensive content.Source: §7.3 · p. 29

7.4 Falsification

• Students remain responsible for verifying AI-assisted content, so hallucinated facts, nonexistent citations, and other inaccuracies make falsification a relevant integrity issue.The report warns that these errors can harm students, publishers, and universities.Source: §7.4 · pp. 29–30

7.5 Use of unauthorised resources

• Generative AI should be treated as an unauthorised resource when assessment rules prohibit it, with permitted uses varying by course level and learning outcome.The report describes entire generated answers as generally wrong, while survey responses support permitting components, style assistance, or bug fixing in some upper-division contexts; restrictions must be explicit.Source: §7.5 · p. 30

7.6 Advice for students

• The report recommends student guidance that reflects professional standards, explains permitted AI use, and requires disclosure of AI use in assessments where allowed.Educators should teach ethical use throughout the curriculum, state restrictions and consequences in syllabi, and provide adaptable guidance such as the sample Appendix D handout.Source: §7.5 · p. 30, §7.6 · p. 30

8 BENCHMARKING LARGE LANGUAGE MODELS FOR COMPUTING EDUCATION

• The benchmark methodology reviews datasets, replicates an earlier study with GPT-4, GPT-3.5-turbo, and GitHub Copilot, and releases problems and test cases for future replication.The primary task is generating solutions to programming problems, a major student activity and focus of existing literature.Source: §8 · p. 31

8.1 Review of empirical datasets

• Available datasets primarily contain small introductory programming exercises, especially in Python, with little data for advanced courses, other languages, chat logs, or code repair.Published evaluations are also often weak: limited tests can accept incorrect solutions, and some datasets require manual intervention.Source: §8.1 · p. 31

8.2 Problem context

• Dataset analysis suggests that benchmark results largely reflect early CS1 programming rather than common CS2 or advanced-course problems.HumanEval and FalconCode were dominated by introductory material, while APPS included harder contest-style problems that may not represent educational contexts.Source: §8.2 · pp. 31, 33

8.3 Replication: The robots are coming

• GPT-4 outperformed GPT-3.5 and GitHub Copilot in the replication, with all models generally performing well enough to pass the exams.GPT-4 solved every problem except T2-Q12, while Copilot solved 20 of 23 exam problems and four of seven Rainfall variants.Source: §8.3 · pp. 33–34, Table 9 · p. 34

8.4.1 APPS.

• 51.5% versus 39.2% average test-case scores, and 36.1% versus 21.0% passing all tests, show GPT-4 outperforming GPT-3.5 on APPS.GPT-4 averaged 72.2% on introductory problems but 28.7% on competition-level problems.Source: §8.4.1 · p. 35, Table 10 · p. 36
• APPS provides extensive tests but may be unsuitable for novice education, and its public coding-site problems may have appeared in recent models’ training data.The authors sampled 100 interview-level problems, evaluated up to 25 test cases, and manually assessed failures.Source: §8.4.1 · pp. 35–36
• APPS benchmarking required prompts to specify input statements and prohibit method definitions, because omitting this formatting instruction caused many tests to fail.The APPS analysis used one attempt per dataset problem, making exact output-format alignment an important evaluation condition.Source: §8.4.1 · p. 36

8.4.2 FalconCode.

• FalconCode evaluation used 661 introductory Python problems, de-duplicated to 385, with extracted prompts and unit tests used to generate and score GPT-3.5/4 solutions.The workflow extracted problem statements and tests, generated solutions, ran the supplied tests, and analyzed the results; scores averaged available unit-test results for each problem.Source: §8.4.2 · pp. 36–37
• GPT-4 failures often reflected unavailable PDFs, starter code, data files, incorrect or missing tests, unexpected libraries or structures, and genuinely incorrect solutions.All nine Project tasks lacked the external PDFs containing their actual instructions; missing resources also substantially affected Skill and Lab assignments.Source: §8.4.2 · p. 37
• 74.1% success on 236 sufficiently specified FalconCode problems increased to 86.8% after excluding unexpected solution structures and libraries, while genuinely incorrect solutions occurred in fewer than 5% of cases.The raw success rate was 45.4% across 385 problems; GPT-4 performed markedly better than GPT-3.5, and the clean subset excluded insufficient-information problems and invalid test cases.Source: §8.4.2 · pp. 37–38, Table 11 · p. 37

8.5 Discussion

• The replication and FalconCode results suggest newer LLMs are more capable than current literature reports, although APPS success rates were lower because its problems were more challenging.The comparison is explicitly attributed to the replication of Finnie-Ansley et al. and the FalconCode experiment.Source: §8.5 · p. 38

8.5.1 Limitations.

• Reliable evaluation requires complete problem descriptions, runnable and comprehensive tests, needed resources, updateable datasets, and clearly reported prompts and LLM parameters.Automated testing does not capture code quality or suitability for learners’ prior concepts, while non-code-generation exercises require additional datasets and potentially qualitative frameworks; broader languages, task types, and levels also need new datasets.Source: §8.5 · p. 38, §8.5.1 · p. 38

• The report urges researchers and practitioners to explore LLMs, adapt teaching and integrity policies, develop best practices, and share them across the computing education community.It presents the work as both an early snapshot of the AI revolution and a call for collective action.Source: §9 · pp. 38–39
• A review of 71 primary articles found that current models generally perform at least as well as most students on introductory programming tasks.The review also identified over-reliance on LLMs for generating and debugging code as the most common concern.Source: §9 · pp. 38–39
• Survey responses from 171 students and 57 instructors across 20 countries showed broadly similar perceptions, but instructors judged course policies less clear than students.Both groups generally supported restrictions on coursework use, while many respondents had limited experience with generative AI tools.Source: §9 · p. 39
• Interviews with 22 instructors across five continents showed emerging shifts toward assessing process over product and increasing the weighting of invigilated assessments.These adaptations reduced reliance on unsupervised homework and are expected to expand across learning objectives, content, and assessment.Source: §9 · p. 39
• University policies addressed many ACM ethics principles, but the report found uncertainty about how students are taught to use generative AI ethically.It recommends explicit course restrictions, usage statements for graded work, clear misconduct penalties, and resources such as a syllabus handout.Source: §9 · p. 39
• A replication using a state-of-the-art model found an extraordinary performance improvement over the original programming-task evaluation conducted two years earlier.The report releases problem descriptions and test cases to facilitate future replication despite inconsistent parameters, prompts, and evaluation methods.Source: §9 · p. 39

Study and publication characteristics

• The form captured bibliographic information, article type, author affiliation, participant countries and education levels, participant counts, and participant descriptions.Article types included discussion papers, supervised and unsupervised studies, tool and evaluation papers, and other categories.Source: A PAPER EXTRACTION FORM · p. 46

Research design and evaluation

• It recorded authors’ motivations, LLMs or tools used, explicit research questions, goals or hypotheses, and programming languages involved in the study.The tool list included GPT-3, GPT-4, Codex, Copilot, Unclear, N/A, and Other.Source: A PAPER EXTRACTION FORM · p. 46

Quality and findings

• A quality-assessment grid examined whether research questions were defined, processes described, results detailed, and threats to validity or limitations explicitly addressed.The form also requested a short summary of each article’s contribution and key results.Source: A PAPER EXTRACTION FORM · p. 46

Educational implications

• The form solicited potential curriculum changes, opportunities, threats, and additional notes relevant to computing education, including reviewer commentary where appropriate.These fields could capture implications prompted by the work even when they were not explicitly stated by the paper’s authors.Source: A PAPER EXTRACTION FORM · p. 46

Demographics and background

• The survey collected students’ gender, country, study level, degree specialization, and number of completed programming courses.Study-level options ranged from first through fifth year, and specializations included computer science, software engineering, information technology, computer engineering, bioinformatics, and other fields.Source: B STUDENT SURVEY QUESTIONS · p. 47

GenAI use and verification

• Students rated their regular GenAI use for text, code, and images and reported how carefully they verify generated code.Verification options ranged from immediate use or brief skimming to skeptical reading and writing tests, alongside a non-applicable option for students who had not generated code with GenAI.Source: B STUDENT SURVEY QUESTIONS · p. 47

Learning and help-seeking

• The survey examined expectations that GenAI use will increase, its effects on programming learning, its relationship to human teaching, and whether students should be taught to use it well.It also asked students to rank GenAI, instructors, peers, discussion forums, online searches, and other resources when seeking help with course material or problems.Source: B STUDENT SURVEY QUESTIONS · p. 47

Policies and ethics

• Students reported views on university and course policies, permitted uses in programming assignments, unauthorized use, and which GenAI-assisted practices are unethical.Questions covered always allowing, conditionally allowing, or always disallowing GenAI, as well as cases including generating assignments, explaining solutions, debugging, code improvement, and translation.Source: B STUDENT SURVEY QUESTIONS · p. 47

Risks and future impacts

• The survey assessed concerns about GenAI’s effects on jobs, transferable skills, reliance, trust in generated code, instructor detection, monitoring, and students’ current course use.Open responses addressed text and code generation, future employment prospects, and allowed GenAI use in coursework and exams.Source: B STUDENT SURVEY QUESTIONS · p. 48

• The survey recorded instructors’ gender, country, teaching experience, class sizes, and department or school affiliation.Source: C INSTRUCTOR SURVEY QUESTIONS · p. 49
• It asked about regular GenAI use for text, code, and images, and how instructors review generated code, from immediate use to testing.Source: C INSTRUCTOR SURVEY QUESTIONS · p. 49
• Likert questions examined expected future use, effects on programming and transferable skills, reliance and trust, teacher replacement, career prospects, and detection of unauthorized use.Source: C INSTRUCTOR SURVEY QUESTIONS · pp. 49–50
• The survey assessed planned changes to teaching, assessment, and curriculum, along with acceptable programming-assignment use, ethical student practices, and clarity of institutional and course policies.Source: C INSTRUCTOR SURVEY QUESTIONS · p. 49
• Open-ended and categorical questions covered instructors’ current GenAI applications, teaching and assessment changes, new curriculum content, student use, academic-integrity and syllabus policies, implementation success, and interview participation.Source: C INSTRUCTOR SURVEY QUESTIONS · p. 50

D STUDENT GUIDE

• Generative AI generates information from prompts using training data and predictive models, simulating understanding rather than genuinely understanding the prompts.Examples include ChatGPT and Bard for text, and Midjourney and DALL-E for images.Source: D STUDENT GUIDE · p. 51
• AI tools may increase productivity and support learning, but unproductive use can provide answers without improving learning.Source: D STUDENT GUIDE · p. 51
• Students may use AI tools during lab exercises and assignments, but not during secure assessments such as tests and exams.Source: D STUDENT GUIDE · p. 51

• Students should learn to use generative AI appropriately because these tools are used in industry and are likely to feature regularly in future work.Source: Examples of productive use · p. 51
• Students can ask generative AI to explain topics, programming constructs, programs line by line, or the meaning of error messages.Source: Examples of productive use · p. 51
• Generative AI can produce examples resembling assignment questions and generate code for tasks students have already mastered.Source: Examples of productive use · p. 51

Examples of inappropriate use

• Inappropriate uses include using AI on forbidden official assessments, completing laboratory questions or assignments, debugging erroneous code, or translating a self-written solution into the assignment’s required language.These practices may leave students unable to complete tests and exams or continue courses that require mastery of foundational programming content.Source: Examples of inappropriate use · p. 51

• AI-generated solutions may be correct yet fail required algorithms, so students remain responsible for checking the accuracy and correctness of submitted work.Double-check generated content, especially for assessed work and tasks requiring specific algorithms.Source: Risks of generative AI · p. 51
• Generated content may be generic or poor quality, and code can contain bugs or security flaws that users must understand and evaluate.Evaluation is especially important when users lack familiarity with the generated topic.Source: Risks of generative AI · p. 51
• Relying on AI to complete learning tasks denies students opportunities to learn and receive accurate feedback on their progress.Teacher-assigned tasks are intended to support learning rather than merely assignment completion.Source: Risks of generative AI · p. 51
• Consistent over-reliance may help complete assignments short term but leave students unprepared for examinations, later coursework, or future jobs.Source: Risks of generative AI · p. 51
• Students may lose motivation when AI completes tasks, but mastering simple tasks remains necessary before tackling more complex problems.Generative AI can complete simple tasks but cannot complete complex problems.Source: Risks of generative AI · p. 51

• Other students may be harmed when AI use prevents learning or introduces academically dishonest content into group assignments.Source: Impact on others · p. 51
• Enforcing academic-integrity standards around AI use consumes faculty and administrators’ time and energy and can be emotionally draining.Source: Impact on others · p. 51
• Universities may face reputational loss or lawsuits when students include AI-generated code they do not understand.Source: Impact on others · p. 51

• Prohibited uses of generative AI are treated as academic misconduct with serious consequences.Source: Academic misconduct · p. 51