Source-linked AI summary
On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?
Kamil Malinka, Martin Perešíni, Anton Firc, Ondřej Hujňák, Filip Januš
TL;DR
This paper examines how ChatGPT affects university education, especially computer-security-oriented study, by evaluating its use across exams, programming assignments, and term papers. It finds that ChatGPT can be easily misused for cheating while also serving as an assistant that may accelerate learning and support students and educators.
Problem
ChatGPT’s ability to simplify writing and programming creates challenges for university education, including cheating, plagiarism, and possible effects on learning and academic integrity.
Method
The paper evaluates ChatGPT across exams, programming assignments, and term papers, comparing different assistance levels with average student results and examining text-detection tools.
Results
The evaluation indicates that ChatGPT can be easily misused and may pass courses required for a university degree, while also providing useful assistance that can accelerate learning and save teachers’ time.
Takeaways & Limitations
Higher education should discuss appropriate adoption and adapt its educational model to address misuse while using AI assistance to support learning.
Takeaways & Limitations
The paper expects further related work to experimentally confirm its statements about the quality and usability of AI tools for cheating.
Abstract
from arXiv · showhide
In late 2022, OpenAI released a new version of ChatGPT, a sophisticated natural language processing system capable of holding natural conversations while preserving and responding to the context of the discussion. ChatGPT has exceeded expectations in its abilities, leading to extensive considerations of its potential applications and misuse. In this work, we evaluate the influence of ChatGPT on university education, with a primary focus on computer security-oriented specialization. We gather data regarding the effectiveness and usability of this tool for completing exams, programming assignments, and term papers. We evaluate multiple levels of tool misuse, ranging from utilizing it as a consultant to simply copying its outputs. While we demonstrate how easily ChatGPT can be used to cheat, we also discuss the potentially significant benefits to the educational system. For instance, it might be used as an aid (assistant) to discuss problems encountered while solving an assignment or to speed up the learning process. Ultimately, we discuss how computer science higher education should adapt to tools like ChatGPT.
1 INTRODUCTION
The paper examines ChatGPT’s disruptive educational potential in computer security, including both misuse and assistance. It evaluates student-like tasks and argues that higher education should adapt rather than rely on prohibition alone.
- Motivation: ChatGPT’s public release generated strong interest, while its quality, multilingual responses, and contextual querying exceeded many expectations.The paper links this reaction to broad public availability and rising search popularity.
- Educational challenge: The authors identify cheating, plagiarism, reduced learning, and programming assistance as risks requiring educational responses.They note that students can quickly adapt AI outputs and that cheating cases prompted bans and detection tools.
- Research question: Initial experiments found that ChatGPT completed Czech-language tasks without difficulty, prompting the question whether it could satisfy standard university requirements.The paper combines this observation with documented student misuse.
- Study focus: The study performs in-depth testing of exams, term papers, programming tasks, and Capture The Flag exercises in information security courses.It compares multiple AI-assistance levels, from copying outputs to using ChatGPT for supplementary advice, against prior student results.
- Contributions: The paper discusses both harms and benefits, including accelerated learning through AI assistance, and proposes revising educational processes for adaptation, detection, and prevention.Its stated goal is to open discussion about appropriate adoption in higher education.
- Position: The authors argue that higher education should prepare students for effective tool use where appropriate instead of pursuing mere prohibitions.They present conversational assistance as one possible use that can partially substitute for absent peers or teachers.
2 RELATED WORK
Prior work explores AI-assisted teaching, academic writing, assessment, and ethical use, but reports limitations in helpfulness, quality, common-sense reasoning, and consistency. The paper positions its study as a broader empirical evaluation of ChatGPT in university computer-security tasks.
- AI in education: Research on AI in education has focused on improving learning and simplifying repetitive processes for teachers.Examples include personalized teaching and automated support for educational activities.
- Personalized teaching: Studies of GPT-3 and conversational agents report useful capabilities but continuing problems with common-sense knowledge, inconsistent outputs, and limited assistance quality.One study extracted correct equations in 80% of tested math tasks, while another found conversational agents lacking when helping students.
- Assessment: GPT-3 assessment applications have not always achieved sufficient quality, including classification of student-created questions into Bloom’s revised taxonomy.The cited evaluation did not reach the quality needed for the task.
- Academic writing: GPT models can generate plausible academic papers, but reported examples include weak claims, unsuitable references, and text that passes existing plagiarism checks.These findings motivate concern about academic writing and integrity.
- Examination performance: GPT-3.5 performed better than expected on a U.S. law-examination multiple-choice component, although it was reported as 17% worse than human takers overall.It exceeded the passing range in some subcategories without passing the examination overall.
- Ethics and study gap: Related ethical guidance cautions against bans and recommends teaching AI capabilities, ethical usage, citation, and critical evaluation of responses.The present paper extends this discussion through evaluation in computer-security education.
3 EXPERIMENT DESIGN
The experiment tests ChatGPT on diverse university assessment tasks in four information-security courses, using three assistance modes and student-comparison procedures. Tasks span written exams, essays, code, projects, and a CTF environment.
- Evaluation procedure: The authors assess AI outputs and compare their points with real students, while acknowledging that the testing sample is a baseline rather than an exhaustive coverage of variants.Experiments used Czech tests and assignments, which may differ from English-language materials.
- Assessment categories: The evaluation includes written fulltext exams, multiple-choice tests, 4–6-page term essays, and programming assignments.Written exams combine general knowledge with practical questions, while essays cover security topics and research-style work.
- AI usage modes: AI usage is evaluated in Copy&Paste, Interpretation, and Assistant modes corresponding to low, basic, and good user expertise.The modes range from directly copying answers to using ChatGPT for discussion and supplementary help.
- Scope: The study uses examination methods from four courses covering information-systems security, cryptography, secure coding, and secure hardware.The methods are grouped into written exams, term essays, and programming assignments.
- Programming assignments: Programming tasks comprise code completion, a small RSA project in C++, and a Smart City information-system project implemented as a PHP web application.The selected projects vary in size, language, requirements, and collaboration structure.
- Interactive project: The interactive CTF assignment places students in a virtual network to solve six cybersecurity secrets involving challenges such as encryption cracking, reverse engineering, and web exploitation.Only one AI-based solution was considered relevant for the CTF evaluation.
4 EVALUATION
Across computer-security evaluation tasks, ChatGPT produced mixed but often substantial performance: it improved with interpretation, generated useful code, and could achieve passing course-level results. Its limitations remained clearest in context-dependent questions, complex interactive assignments, and educational depth.
- Fulltext exam: ChatGPT gave correct but broader and less specific fulltext answers, with context-dependent application remaining problematic.Its answer variability was similar to that of ordinary students.
- Tests: Interpretation substantially improved test response quality, whereas Copy&Paste answers were not always correct under harsh negative scoring.An incorrect response caused a -1 score for the question.
- Term essays: ChatGPT-generated essays performed worse than student essays, although assistance required fact-checking and saved time for threat-modeling descriptions.A roughly four-page Copy&Pasted essay was completed in less than an hour.
- Completing Predefined Code: At least 30% of points came from a naive predefined-code implementation completed in no more than 10 minutes.ChatGPT also reduced the need to look up formulas or algorithm steps.
- Small Projects: Small projects completed by copying ChatGPT outputs generally received at least 40% of points, while database-project code and structure reached average-student quality.The RSA project involved key generation, encryption, decryption, and message breaking without special prime-number libraries.
- Interactive Projects: ChatGPT solved some Capture The Flag subtasks but could not independently solve the full assignment, mainly guiding students through approaches.It successfully handled hidden-file and obfuscated-JavaScript tasks but produced incorrect cipher-decoding results.
- Final Assessment: The final assessment found ChatGPT could pass all evaluated courses, except for a minor Subject A loss when Copy&Paste was used on the test.Passing required at least 50% of total semester points.
5 DISCUSSION
The discussion finds that ChatGPT can substantially facilitate cheating and reduce learning, while also serving as an assistant that accelerates learning when used responsibly. These findings motivate immediate changes to assessment, detection, prevention, and student guidance.
- ChatGPT enables easy access to correct exam answers, term papers, and functional code, potentially reducing the learning experience.
- Incorrect answers and fabricated events, links, and references create risks of misinformation and undetected errors in submitted work.
- Czech-language experiments indicate that the cheating threat is not limited by the user’s native language and may allow students to pass without understanding course content.
- ChatGPT can act as a teaching assistant by helping students discuss assignment problems and ideas, including those who lack peer support.
- Well-prepared queries can accelerate learning about unfamiliar technologies by eliciting explanations and sample code alongside official documentation.
- The paper recommends discussing academic integrity, using controlled milestones, verifying submitted work, and adapting assignments and exams to limit misuse.
6 CONCLUSIONS
The paper concludes that ChatGPT can be misused easily enough to pass university courses, threatening academic integrity without educational changes. It also identifies responsible assistance as a potential benefit and argues that higher education must adapt while preserving core student capabilities.
- ChatGPT might pass the courses required for a university degree, while plagiarism and cheating could produce low-quality graduates without educational-model changes.
- Restrictions alone are unlikely to be sufficient; education should prepare students to use the technology effectively while fostering critical thinking, independent work, and creativity.
- An AI assistant can reduce performance differences associated with students’ social groups, boost student performance, and save teachers’ time.
- ChatGPT can accelerate experienced users’ learning of unfamiliar technologies by helping them understand documentation and specifications.
- The authors expect further experimental work to confirm their claims about AI tools’ quality and usability for cheating.