GenAI Detection Tools, Adversarial Techniques and Implications for Inclusivity in Higher Education

Introduction

• GenAI adoption in academia is growing, but unequal access and structurally biased outputs may disadvantage some students and researchers.Barriers include internet access, paid-tool costs, and disability-related access issues.Source: Introduction · p. 2

Efficacy of AI Text Detection Tools

• Detectors claim to identify AI-generated text and are premised on supporting fair, secure assessment, but their reliability claims are disputed.Prior studies report varied accuracy, while paid tools have been found only slightly more accurate than free versions.Source: Introduction · p. 2, Efficacy of AI Text Detection Tools · p. 4

Objectives

• The study evaluates detector efficacy and sensitivity to linguistic changes and adversarial prompting to assess implications for inclusive education.The authors define inclusivity as equal opportunity without disadvantage from lack of opportunity or unfair assessment.Source: Objectives · p. 3, Introduction · p. 3

Efficacy of AI Text Detection Tools

• Research links detector linguistic measures such as perplexity and burstiness to potential bias against non-native English speakers.Prior work also reports false positives for NNES writing, though GPTZero and Turnitin research disputes whether such disparities occur.Source: Introduction · p. 2, Efficacy of AI Text Detection Tools · p. 4

Adversarial techniques as a method of evading GenAI detection

• Adversarial approaches include deliberate errors, style and complexity changes, prompt engineering, recursive paraphrasing, and spoofing.Prior work reports that paraphrasing and translation can reduce detection, and that watermarking does not eliminate vulnerability to manipulation.Source: Adversarial techniques as a method of… · p. 5

Research gap

• Existing studies have not comprehensively compared detectors against different GenAI tools and evasion techniques while examining inclusion and equity consequences.The authors identify this gap as limiting evidence-based recommendations for educators.Source: Research gap · p. 5

Overview

• 114 samples—15 original AI texts, 89 manipulated versions, and 10 human controls—were tested against seven AI text detectors (n=805).Sample creation and testing took place in September and October 2023.Source: Overview · p. 6
• Three GenAI tools were used to generate the original short text samples, which were then altered using six adversarial techniques.The human-written samples served as controls.Source: Overview · p. 6
• The prompts and developed samples are publicly available through Mendeley Data.The passage cites Perkins et al. (2024).Source: Overview · p. 6

GenAI Sample Generation

• Fifteen initial samples were generated using GPT4, Claude 2, and Bard, with five samples from each tool.All tools received the same five prompts.Source: GenAI Sample Generation · p. 6
• The prompts covered a university essay, professional blog post, internship cover letter, middle-school comparative analysis, and magazine article.The tasks were intended to span different higher-education and broader writing contexts.Source: GenAI Sample Generation · p. 6
• The prompts varied topics, linguistic styles, and complexity levels to test detector consistency across text forms.Ten human-written controls came from proficient English-speaking academic staff and Vietnamese undergraduate NNES students at an English-medium UK-oriented international university.Source: GenAI Sample Generation · p. 6

Adversarial sample generation

• All 15 original AI samples were modified using six adversarial techniques, generally with the same foundation model used to generate each sample.Paraphrasing instead used Quillbot’s free standard mode.Source: Adversarial sample generation · pp. 6, 8
• Researchers followed standardized prompts, used delimiters and up to five prompt-chaining requests, and made no manual edits.Samples without a suitable output after five iterations were excluded.Source: Adversarial sample generation · p. 7
• The technique prompts targeted spelling errors, varied sentence length, increased text complexity, simplified text, and NNES-style writing.The table is identified as presenting the prompt formats for the techniques.Source: Adversarial sample generation · pp. 7–8
• The six techniques yielded 89 manipulated AI-text samples because suitable outputs could not be obtained in one case.The authors avoided few-shot prompting to limit external-text influence and let each tool interpret the technique prompts.Source: Adversarial sample generation · p. 8
• The spelling-error technique often produced at least 20 errors, and the authors accepted these outputs despite attempts to reduce the error count.The errors were intended to mimic insufficient proofreading.Source: Adversarial sample generation · p. 8
• Varying sentence length sometimes produced overly short sentences, while added complexity often reduced readability and stylistic consistency.Simplification instead sought to reduce linguistic complexity while preserving the sample’s core message.Source: Adversarial sample generation · p. 8

• Samples for testing were approximately 500 words, with final lengths ranging from 350 to 625 words.Source: Summary of sample generation · p. 9
• The stated length restriction arose from limitations in some AI text detectors.Source: Summary of sample generation · p. 9
• Table 3 lists the samples used for testing.Source: Table 3 · p. 9

Testing protocol

• Seven detectors were selected based on prior literature, and 114 samples were tested with each of six tools, yielding 805 tests and 797 valid tests.Source: Testing protocol · pp. 9–10
• Outcomes were classified as positive or negative according to whether each tool correctly identified human-written or GenAI text.Source: Testing protocol · p. 10

Interpretation protocol

• Because detectors report results in quantitative or qualitative forms, the researchers used interpretation protocols to standardize comparisons.Source: Interpretation protocol · p. 10

GPTKit

• Table 5 presents an interpretation protocol for AI detectors.Source: Table 5 · p. 11

• Accuracy was assessed using binary, semi-binary, and logarithmic methods, with the mean of the combined methods also reported.Source: Accuracy and error analysis · pp. 11–12
• Table 6 presents calculations of accuracy rates.Source: Table 6 · p. 12
• False-accusation likelihood was calculated using a 60% threshold for AI-generated content detected by the tools.Source: Accuracy and error analysis · p. 12
• The analysis also considered undetected cases, where detectors fail to identify AI-produced text that may receive credit like honest work.Source: Accuracy and error analysis · p. 12

Baseline testing

• 39.5% was the mean baseline accuracy on unmanipulated AI-generated content, while human-written controls were correctly classified in 67% of tests.Copyleaks had the highest sensitivity at 64,8%, followed by Turnitin at 61%; Bard outputs were most detectable, at 76.9%.Source: Baseline testing · p. 13, Table 7 · p. 13

Accuracy of AI detectors following the application of adversarial techniques

• 17.4% was the mean accuracy reduction after adversarial techniques; manipulated-text accuracy ranged from Copyleaks at 58.7% to GPT Kit at 4.5%.Turnitin had the highest accuracy drop at 42.1%, falling to fifth place among seven detectors.Source: Table 8 · p. 14, Accuracy of AI detectors following the… · p. 14
• Spelling errors (12.9%) and higher burstiness (15.9%) were almost undetectable, while increasing text complexity reduced detectability by only 2%.The spelling-error outputs often seemed unlikely to be submitted by students and could receive poor marks; the reported overall accuracy drop was 17.5%.Source: Table 9 · p. 14, Accuracy of AI detectors following the… · p. 14

Error analysis

• Four of seven detectors made no false accusations on human samples, but Copyleaks mislabeled 50%; six of seven missed over half of manipulated texts.Even Copyleaks failed to identify 39% of AI-generated cases.Source: Table 10 · p. 15, Error analysis · p. 15

Comparative Performance of Generative AI tools

• Bard had the largest mean reduction in detectability at 38.8%, versus 8% for Claude 2 and 7.6% for GPT-4.Bard's reduction was especially large given detectors' high baseline accuracy on its output; some GPT-4 attacks made text easier to detect.Source: Table 11 · p. 15, Comparative Performance of Generative… · p. 16

Accuracy and vulnerabilities of AI text detectors as a barrier to inclusivity

• Detector effectiveness varies across tools, and adversarial techniques further reduce accuracy, making institutional outcomes dependent on which tool is used.The authors caution against using detectors alone to determine academic-integrity violations because of accuracy limitations and false-accusation risks.Source: Accuracy and vulnerabilities of AI… · p. 16, The use of GenAI text detectors in… · pp. 17–18

Effectiveness of adversarial techniques

• Spelling errors and increased burstiness effectively evade detection, while increasing complexity has minimal impact.The contrast suggests that dense, complex text alone is insufficient to evade detection.Source: Effectiveness of adversarial techniques · pp. 16–17

Implications of adversarial techniques for inclusive education in the age of AI

• Different detectors may produce different efficacy rates even within one institution, while existing threats to equity remain.The discussion cautions that focusing too heavily on GenAI may divert attention from contract cheating and traditional plagiarism.Source: Implications of adversarial techniques… · p. 17
• 15% false accusations raise equity concerns, while detectors with no false accusations may instead miss more AI-generated content.The authors also warn that access to paid tools and technical knowledge may give some students an unfair advantage.Source: Implications of adversarial techniques… · p. 17

The use of GenAI text detectors in education

• The authors do not recommend detectors as the sole basis for academic-integrity decisions, but suggest non-punitive use to support student learning.They note that detectors may help educators discuss how GenAI can be integrated with students’ own writing.Source: The use of GenAI text detectors in… · pp. 17–18

GenAI and the future of inclusive education

• Inclusive assessment should account for GenAI’s ubiquity, clearly state expected AI use, and avoid neglecting longstanding threats such as contract cheating and plagiarism.The authors recommend reconsidering traditional assessment and integrating GenAI into learning and assessment.Source: GenAI and the future of inclusive… · p. 18

Scope and generalisability

• The evaluation covered a relatively small number of samples, three GenAI tools, and seven detectors, so it does not encompass the full range of tools, writing styles, or manipulations.The authors characterize the findings as indicative rather than exhaustive and recommend broader evaluation, including newer foundation models.Source: Limitations · pp. 18–19
• Because GenAI tools and detectors are rapidly updated, the findings represent only a specific snapshot in time.The authors note that subsequent evaluation should include currently available detection software and newer models.Source: Limitations · pp. 18–19

Experimental design

• The experiment did not reproduce iterative writing and refinement, so real-world hybrid text may be even harder for detectors to identify.Students may combine several evasion techniques with repeated software-aided and manual editing.Source: Limitations · p. 19

Real-world sample quality

• Some manipulated samples may not reflect student submissions because poor quality, unusual phrasing, and excessive errors could prompt human markers’ suspicion.The authors identify some Claude 2 and Bard samples with added spelling errors as examples.Source: Limitations · p. 19

Conclusion

• 39.5% average accuracy on non-manipulated AI text fell to 22.14% after adversarial techniques; human-written controls had 67% accuracy.Adding spelling errors and increasing burstiness were among the techniques that effectively evaded detection.Source: Conclusion · p. 19
• The authors warn that detectors may enable students with resources and technical skill to disguise AI content while unfairly penalising some NNES and lower-proficiency English writers.They conclude that detection software may create barriers to inclusive assessment rather than reduce them.Source: Conclusion · p. 19

Declaration of Generative AI and AI-assisted technologies in the writing process

• The authors used Claude 3 and GPT-4 to draft text, revise wording, and adjust the manuscript, then reviewed and edited the content.They state that they take full responsibility for the publication.Source: Declaration of Generative AI and… · p. 19