Source-linked AI summary
GPT detectors are biased against non-native English writers
Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, James Zou
TL;DR
GPT detectors’ fairness and reliability remain uncertain, especially for non-native English writing. This study evaluates widely used detectors across native and non-native samples and finds consistent overclassification of non-native writing as AI-generated.
Problem
GPT detectors have received limited evaluation of their effectiveness and reliability, despite the risks of misidentifying human writing, particularly in education.
Method
The study evaluates seven widely used GPT detectors on human-authored TOEFL essays and US eighth-grade essays representing non-native and native English writing.
Results
Non-native TOEFL essays had a 61.22% average false-positive rate, while detectors achieved near-perfect accuracy on US eighth-grade essays.
Takeaways & Limitations
The findings caution against using current GPT detectors in evaluative or educational settings, particularly when assessing non-native English speakers.
Takeaways & Limitations
The pilot study uses relatively small datasets, so larger and more diverse samples may be needed to validate and generalize its findings.
Abstract
from arXiv · showhide
The rapid adoption of generative language models has brought about substantial advancements in digital communication, while simultaneously raising concerns regarding the potential misuse of AI-generated content. Although numerous detection methods have been proposed to differentiate between AI and human-generated content, the fairness and robustness of these detectors remain underexplored. In this study, we evaluate the performance of several widely-used GPT detectors using writing samples from native and non-native English writers. Our findings reveal that these detectors consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified. Furthermore, we demonstrate that simple prompting strategies can not only mitigate this bias but also effectively bypass GPT detectors, suggesting that GPT detectors may unintentionally penalize writers with constrained linguistic expressions. Our results call for a broader conversation about the ethical implications of deploying ChatGPT content detectors and caution against their use in evaluative or educational settings, particularly when they may inadvertently penalize or exclude non-native English speakers from the global discourse. The published version of this study can be accessed at: www.cell.com/patterns/fulltext/S2666-3899(23)00130-7
Introduction
The introduction frames GPT detectors as necessary but insufficiently evaluated tools whose errors may harm human writers. This study systematically examines detector bias across native and non-native English writing and highlights fairness concerns in evaluative and educational settings.
- Motivation: GPT-based generative language models offer productivity and creativity benefits but create risks when AI-generated content is presented as human-written.These risks may produce potential harms, motivating the development of reliable detection methods.
- Problem: Humans struggle to detect AI-generated content, while publicly available GPT detectors remain uncertain in effectiveness and reliability because evaluation is limited.Misidentifying human-written content may have damaging consequences.
- Contribution: The study evaluates several publicly available GPT detectors on writing samples from native and non-native English writers.The evaluation addresses the need for trustworthy and accurate detection methods amid concerns about generative-model misuse.
- Contribution: GPT detectors consistently misclassify non-native English writing as AI-generated while not making the same mistakes for native writing.The finding identifies a systematic bias in detector performance across writer groups.
- Implications: The findings call for greater attention to detector fairness and robustness because overlooked biases may marginalize non-native speakers in evaluative or educational settings.The paper is presented as among the first systematic examinations of bias in ChatGPT detectors and advocates further research to address it.
Results
Seven GPT detectors accurately identified US eighth-grade essays but misclassified most TOEFL essays by non-native English writers as AI-generated. Increasing linguistic diversity reduced this bias and could also help GPT-generated text evade detection.
- Detector performance: 61.22% average false positive rate: detectors misclassified over half of the 91 TOEFL essays as AI-generated, despite near-perfect accuracy on US eighth-grade essays.All seven detectors showed this pattern, according to the reported evaluation.
- Bias mitigation: A prompt to enhance TOEFL essays’ word choices substantially reduced their misclassification as AI-generated.The intervention aimed to emulate native-speaker vocabulary usage, but the supplied passage truncates the resulting numerical value.
- Bias mechanism: Reduced linguistic variability and constrained word choices in non-native writing contributed to misclassification as AI-generated text.The findings specifically warn that low perplexity can be a biased indicator of AI-generated text.
- Detector evasion: Enhancing linguistic diversity can mitigate bias against non-native English writers and enable GPT-generated content to bypass detectors.The study demonstrated this possibility using ChatGPT-3.5 essays and a second-round self-edit prompt, although the supplied passage is truncated.
Discussion
The discussion identifies substantial bias and manipulability in GPT detectors, especially against non-native English writers, while emphasizing study limitations and cautioning against evaluative use. It attributes misclassification partly to constrained linguistic variation and calls for larger datasets and more equitable detection methods.
- Detector bias: GPT detectors showed high misclassification of non-native-authored TOEFL essays, contrasting with near-zero misclassification of college essays presumably authored by native speakers.
- Detector bias: Limited linguistic variability and word choices among non-native writers may produce lower-perplexity text that detectors misclassify as AI-generated.Prior work associates non-native writing with reduced lexical richness, lexical diversity, syntactic complexity, and grammatical complexity.
- Detector reliability: A straightforward second-round self-edit prompt drastically reduced detection rates for college essays and scientific abstracts, exposing the susceptibility of perplexity-based approaches to manipulation.
- Limitations: The pilot study’s relatively small samples limit validation and generalization, motivating larger and more diverse datasets across broader contexts and populations.
- Recommendations: The authors strongly caution against using GPT detectors in evaluative or educational settings, particularly when assessing non-native English speakers, because of high false-positive rates.
Correspondence · Materials and Methods · Data availability
The study evaluated GPT detectors on human-written and ChatGPT-generated texts spanning TOEFL essays, college admissions essays, and scientific abstracts. Materials, prompts, statistical procedures, and openly available data, results, and code are documented.
- Data availability: The study’s data, results, and code are available through GitHub and Zenodo.
- Evaluation of off-the-shelf GPT detectors: The evaluation covered seven widely-used off-the-shelf GPT detectors, accessed on March 15, 2023.
- TOEFL Essays: The dataset included 91 human-written TOEFL essays collected from a Chinese educational forum.TOEFL measures English proficiency among non-native speakers.
- US College Admission Essays: The study assembled 70 authentic US college admission essays from two publicly available essay-example websites.Sixty essays came from PrepScholar and 10 from College Essay Guy.
- Scientific Abstracts: The scientific-abstract corpus contained 145 authentic course project titles and abstracts from Stanford’s CS224n Winter 2021 quarter.The selected course connects education, scientific research, AI, deep learning, and NLP.
- Statistical test: Perplexity differences were tested with a paired t-test using a one-sided alternative and SciPy, with GPT-2 XL as the language-model backbone.ICLR 2023 analyses controlled for rating effects by calculating residuals from linear regression.