Source-linked AI summary
Testing of Detection Tools for AI-Generated Text
Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomáš Foltýnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut, Lorna Waddington
TL;DR
The paper asks whether existing tools can reliably distinguish human-written from ChatGPT-generated text in academic contexts, including after translation or obfuscation. It tests a broad set of tools using controlled documents and evaluates accuracy and error types. The tools are neither accurate nor reliable, often favour human-written classifications, and perform worse after obfuscation, leading the authors to reject their use as evidence of academic misconduct.
Problem
The paper addresses limited evidence about whether AI-text detectors reliably distinguish human-written and ChatGPT-generated academic text and withstand translation or obfuscation.
Method
The study tests detection tools on controlled human-written, translated, ChatGPT-generated, manually edited, and paraphrased texts, evaluating accuracy and educationally consequential error types.
Results
The tested tools are neither accurate nor reliable, generally bias classifications toward human-written text, and worsen substantially when AI-generated text is obfuscated.
Takeaways & Limitations
The authors conclude that tested detection systems should not be used in academic settings as evidence of misconduct and recommend prevention-focused pedagogical strategies.
Takeaways & Limitations
The authors identify a conceptual limitation in asking an LLM to verify authorship because LLM outputs are stochastic and can hallucinate.
Abstract
from arXiv · showhide
Recent advances in generative pre-trained transformer large language models have emphasised the potential risks of unfair use of artificial intelligence (AI) generated content in an academic environment and intensified efforts in searching for solutions to detect such content. The paper examines the general functionality of detection tools for artificial intelligence generated text and evaluates them based on accuracy and error type analysis. Specifically, the study seeks to answer research questions about whether existing detection tools can reliably differentiate between human-written text and ChatGPT-generated text, and whether machine translation and content obfuscation techniques affect the detection of AI-generated text. The research covers 12 publicly available tools and two commercial systems (Turnitin and PlagiarismCheck) that are widely used in the academic setting. The researchers conclude that the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text. Furthermore, content obfuscation techniques significantly worsen the performance of tools. The study makes several significant contributions. First, it summarises up-to-date similar scientific and non-scientific efforts in the field. Second, it presents the result of one of the most comprehensive tests conducted so far, based on a rigorous research methodology, an original document set, and a broad coverage of tools. Third, it discusses the implications and drawbacks of using detection tools for AI-generated text in academic settings.
1. Introduction
The paper examines whether AI-text detection tools can reliably distinguish human-written from ChatGPT-generated text and withstand translation or obfuscation. This concern arises because unauthorised content generation threatens academic integrity, while institutions also need ethically informed approaches to AI use.
- Higher-education institutions must uphold the integrity of assessments and diplomas because they educate professionals and develop students’ ethical values.
- Unauthorised or undeclared human or technological assistance in academic work is presented as a threat to academic integrity.
- AI-generated content can be difficult to distinguish from human-written text and can therefore be used in higher-education assessment tasks.
- AI use is not automatically unethical, so institutions should teach its benefits, limitations, and ethical, transparent use rather than treating all use alike.
- The paper evaluates detection tools’ functionality, accuracy, obfuscation resistance, and sensitivity to machine translation of human-written text.
- The study asks whether tools reliably classify human-written and ChatGPT-generated texts, how translation and editing affect results, and how consistent tools are.
2. Large Language Models
Large language models generate contextually plausible text through statistical language modelling, attention, and transformer architectures. Their accessibility and human-like output have increased concern about distinguishing authorised from unauthorised AI-assisted academic content.
- LLMs predict the likelihood of a character, word, or token in context, and their statistical language models have roots in earlier language technologies.
- ChatGPT is an OpenAI natural-language-processing system based on transformer architecture and successive GPT model generations.
- ChatGPT-generated text can be hard to distinguish from human-written text, while GPT-3 also supports multilingual performance.
- The accessibility and quantity of AI tools have led educators to seek technical solutions for distinguishing human-written from AI-generated texts.
- Text-matching software can miss plagiarism and falsely flag non-plagiarised content, motivating examination of whether AI detectors distinguish content clearly.
3. Related work
Prior research on AI-text detection reports highly variable performance, methodological weaknesses, and susceptibility to bias or paraphrasing. The paper addresses these shortcomings through broader tool coverage, original data, and error-focused evaluation.
- The related work spans scientific publications, preprints, and non-scientific comparisons of existing detection tools.
- The authors criticise a prior study because text matching is poorly suited to stochastic LLM output and an LLM cannot reliably verify authorship of a text.
- Reported findings range from strong distinctions on selected abstracts to success rates below 50% and frequent classification of texts as human-written.
- One prior test reported 27.9% overall accuracy, a best-tool maximum of 50%, and almost 83% accuracy for human-written content.
- Existing studies often use few tools or documents, incompletely describe methods, omit translation effects, and rely on limited performance metrics.
4. Methodology
The methodology tests detection tools on controlled human-written, translated, ChatGPT-generated, manually edited, and machine-paraphrased English texts. Results are collected across a broad tool set and evaluated for accuracy, error type, and consistency.
- Test corpus: The test categories include human-written text, machine-translated human text, ChatGPT-generated text, manually edited AI text, and machine-paraphrased AI text.
- Test corpus: Human-written and translated documents were approximately 10,000 characters, with translated texts originating in seven non-English languages.
- Test corpus: ChatGPT generated the AI texts, while Quillbot produced the automatically paraphrased cases and researchers manually altered another set.
- Coverage: Nine researchers prepared 54 test cases with known ground truth, and all cases were presented to each tool for 756 total tests.
- Tool selection: The study tested 14 detection tools, including free online tools and the commercial systems Turnitin and PlagiarismCheck.
- Data quality: Repeated tests sometimes produced different results, including Check for AI changing from low to high AI risk after five days.
5. Results
The study evaluates detection tools using accuracy and error-type analysis across human-written, AI-generated, translated, edited, and paraphrased documents. Results show substantial weaknesses: performance varies by document type, tools tend to favor human classifications, and obfuscation increases undetected AI-generated text.
- Accuracy and error analysis: The researchers evaluated classification outcomes using accuracy measures and error analysis because false accusations and undetected AI use have different educational consequences.The evaluation included binary, semi-binary, and logarithmic accuracy approaches.
- Overall accuracy: Turnitin ranked highest across all accuracy approaches, followed by Compilatio and GPT-2 Output Detector; Crossplag and Go Winston also reached at least 70% accuracy.GPT-2 Output Detector was not trained to detect GPT-3.5 output.
- Variations in accuracy: 96% accuracy for original human-written documents fell by 20% after machine translation into English.The results suggest that machine translation leaves traces interpreted as AI-related, despite the source text being human-written.
- Variations in accuracy: 42% accuracy for machine-generated text after human editing contrasted with 74% for unmodified machine-generated text.The edited cases represent texts altered through synonym substitution or sentence reordering.
- Variations in accuracy: 26% accuracy for machine-generated text after machine paraphrasing meant that most AI-generated texts remained undetected.The classifiers generally treated AI-transformed AI text as human-written.
- Classification bias and errors: Except for GPT Zero, the tested tools were more accurate on human-written text than on text generated or modified by AI or machine tools, indicating a bias toward human classification.The authors link this pattern to the higher risk of undetected AI-generated work than false accusation.
- Classification bias and errors: Six of fourteen tools produced false positives, with false-accusation risk increasing dramatically for machine-translated human-written texts.For AI-generated documents, 13 of 14 tools produced false or partially false negatives for unmodified AI classes, and no tool correctly classified all edited or paraphrased AI documents.
- Undetected cases: Undetected AI-generated text increased to approximately 50% after manual editing and rose still higher after machine paraphrasing.These obfuscation strategies therefore increased the risk of students gaining an unfair academic advantage.
6. Discussion
The tested detection tools were neither accurate nor reliable, and their performance worsened with obfuscation and machine translation. Their outputs were also difficult to interpret and unsuitable as standalone evidence for misconduct decisions.
- All tools scored below 80% accuracy, with only five exceeding 70%, and they showed both false positives and false negatives.The tools were biased toward classifying text as human-written; approximately 20% of AI-generated texts would likely be misattributed to humans.
- Approximately 50% of obfuscated AI-generated texts would likely be misattributed to humans.Manual editing and machine paraphrasing worsened detection performance, while machine translation also challenged the tools.
- Detection outputs often provide unverifiable classifications or highly precise statistics that do not help average users interpret them.Some tools highlight supposedly machine-generated text, while others report values such as perplexity or burstiness scores without supporting evidence.
- Some tools, including Writer, suggest editing text to reduce its detectable AI content.This creates a practical route for users to conceal AI-written text from detection systems.
- Detection reports should not be the sole basis for accusing students because they provide no evidence and cannot support a meaningful defence.The authors recommend further dialogue with students rather than relying on classifier reports alone.
- A prevention-focused approach should prioritize ethical AI education and assessment strategies over unreliable detection.The paper also emphasizes training students, faculty, and staff on ethical and unethical AI use.
7. Conclusion and Future Work
The paper concludes that current AI-generated-text detectors do not provide a dependable or evidential basis for academic misconduct decisions. It recommends prevention-oriented educational practices and further research on obfuscation, translation, and broader impacts.
- 7. Conclusion: The tested detection systems are unsuitable as evidence of academic misconduct because they produce frequent false positives and false negatives without verifiable evidence.The authors contrast them with text-matching software, which can at least provide evidence of potential misconduct.
- 7. Conclusion: The findings suggest that an easy solution for detecting AI-generated text does not exist.The authors recommend rethinking academic assessment strategies around the process of developing student skills rather than only the final product.
- Future work: Future research should test multiple levels and combinations of obfuscation, including machine paraphrasing, translation, and patch writing.The paper also proposes studying detection at cohort level through effects on assessment scores and education systems.
A. Case studies 05-ManEd
The 05-ManEd case studies compare ChatGPT-generated texts with human-obfuscated versions. The visualizations show that some texts were heavily rewritten while others changed only slightly.
- A. Case studies 05-ManEd: The case studies display generated texts on the left and human-obfuscated versions on the right.Identical text is shown in matching colors, while edits appear in white.
B. Case studies 06-Para
The 06-Para case studies use ChatGPT-generated texts that were automatically rewritten with Quillbot’s default settings. The accompanying figures identify the tested documents.
- B. Case studies 06-Para: The 06-Para test cases were first generated with ChatGPT and then automatically rewritten using Quillbot’s default settings.The original appears on the left and the rewritten version on the right.