PaperlayerView exact revision on arXiv

Source-linked AI summary

People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text

Jenna Russell, Marzena Karpinska, Mohit Iyyer

arXiv:2501.15654v2cs.CLcs.AI

TL;DR

Automatic detectors often have low detection rates, weak robustness to evasion, and limited explainability. This paper compares human annotators with automated detectors on 300 articles and finds that frequent LLM-writing users detect AI text near perfectly without additional training.

  • Problem

    Automatic detectors have low detection rates, poor robustness to evasion attempts, and limited explainability for users.

  • Method

    The study collects 1,790 annotations and explanations from annotators judging 300 non-fiction English articles as human-written or AI-generated.

  • Results

    1 of 300 articles is misclassified by the majority vote of five frequent LLM-writing users, outperforming almost every tested automatic detector.

  • Takeaways & Limitations

    Frequent LLM-writing users are highly accurate, robust, and explainable detectors of AI-generated text without additional training.

  • Takeaways & Limitations

    The study is limited to professionally written, proofread articles in American English and does not investigate factual accuracy.

Abstract

from arXiv · show

In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1). We hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such "expert" annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. Qualitative analysis of the experts' free-form explanations shows that while they rely heavily on specific lexical clues ('AI vocabulary'), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity) that are challenging to assess for automatic detectors. We release our annotated dataset and code to spur future research into both human and automated detection of AI-generated text.

1 Introduction

This paper evaluates human detection of modern LLM-generated text under evasion attempts and finds that frequent LLM users detect it exceptionally well. Experts outperform nearly all tested automatic detectors, relying on lexical, structural, and originality cues that evasion does not fully remove.

  • Method: The study collects 1,790 annotations on 300 non-fiction English articles generated by GPT-4O, Claude-3.5-Sonnet, and O1-PRO, including paraphrasing and humanization.Annotators labeled each article as human- or AI-written and supplied paragraph-length explanations; the study cost $4.9K USD.
  • Results: 1 of 300 articles was misclassified by five frequent LLM users’ majority vote, without training or feedback.These high-performing annotators used LLMs frequently for editing, copywriting, or creative writing.
  • Results: Frequent LLM users detect AI text well, whereas rare or non-users perform poorly and overestimate their ability through high-confidence decisions.The expert subset was identified from annotators who frequently use LLMs for writing tasks.
  • Results: 100% true positive rate was achieved for detecting O1-PRO articles after humanization, outperforming almost every tested commercial and open-source detector.Only Pangram matched the experts’ near-perfect detection accuracy across the 300 articles.
  • Analysis: Experts most often cite “AI vocabulary,” followed by formulaic structures and originality, while paraphrasing and humanization do not eliminate these signatures.The paper notes that these evasion tactics and corresponding defenses remain underexplored.
  • Analysis: An LLM prompted with a guidebook derived from expert explanations competes with existing detectors on easier cases but struggles with humanized articles.The approach attempts to replicate the experts’ decision-making process automatically.

2 How good are humans at detecting AI-generated text?

Human detection ability depends sharply on LLM writing experience: frequent users accurately identify AI-generated nonfiction, while limited users perform near chance despite high confidence. Expert performance remains strong across Claude, paraphrased, O1-PRO, and humanized outputs, supported by lexical and broader stylistic cues.

  • Experiment 1: The four nonexperts performed near chance, averaging 56.7% TPR and 52.5% FPR despite reporting high confidence.Their average confidence was 4.03, suggesting overestimation of their detection ability.
  • Experiment 1: The five expert annotators achieved 92.7% TPR and 3.3% FPR, with majority vote correctly determining authorship for all 60 articles.Experts rarely mislabeled human-written articles as AI-generated.
  • Experiment 2: Experts generalized beyond GPT-4o: their Claude majority vote was again 100% accurate, although one annotator’s TPR fell to 80%.The results test whether experts rely on GPT-4o-specific artifacts and show mostly consistent performance on Claude-generated articles.
  • Experiment 3: Paraphrasing did not materially change expert performance, and the five-expert majority again correctly classified all 60 articles.Overall TPR and FPR remained almost unchanged from the earlier experiments.
  • Experiment 4: For O1-PRO, the expert majority achieved 96.7% TPR and 0% FPR, with one AI-generated article misclassified and lower confidence indicating greater difficulty.Four of five experts remained largely consistent with prior experiments, while explanations more often examined human-sounding characteristics.
  • Experiment 5: Humanization likewise left the expert majority perfect on all 60 articles, although confidence dropped and AI clues such as unusually frequent honorifics remained detectable.Fifteen-point-one percent of annotations received the lowest confidence rating, while humanization did not eliminate the texts’ detectable AI signature.

3 Analysis of expert performance

Expert annotators outperform most automatic detectors, while their explanations reveal both common lexical clues and harder-to-automate properties such as originality, formality, and clarity. Different annotators attend to complementary textual cues, helping explain the ensemble’s strong performance and exposing opportunities for training.

  • Human experts vs. automatic detectors: Pangram Humanizers matched the human-expert majority vote, averaging 99.3% TPR with 2.7% FPR, and outperformed each individual expert except slightly on humanized O1-PRO articles.GPTZero struggled significantly on O1-PRO articles, with and without humanization.
  • Coding expert explanations: Vocabulary appeared in 53.1% of explanations, followed by sentence structure at 35.9%, grammar at 24.8%, and originality at 23.7%.Originality, factuality, and tone are harder to assess automatically than vocabulary, potentially giving humans an advantage.
  • Coding expert explanations: Correct explanations shifted across generation settings: vocabulary appeared in 57.1% of O1-PRO explanations, 42.3% of humanized O1-PRO explanations, and 88% of paraphrased-article explanations.Quotations appeared in 33.8% of paraphrased-article explanations, often because experts noticed consistent formatting and style.
  • Annotators don’t always focus on the same clues: Annotators relied on complementary cues: some emphasized grammaticality or grouped examples, while others examined article flow, detail specificity, or the naturalness of quotations.Annotator 1 uniquely noticed AI-associated names, and this diversity helps explain the ensemble’s strong performance while suggesting opportunities for training.
  • When experts are incorrect: Experts were sometimes misled by stylistic variation: O1-PRO’s contractions and colloquialisms fooled one annotator, while false-positive explanations frequently cited vocabulary.The annotator mentioned formality in none of their Experiment 2 and 3 explanations, compared with 66.7% for O1-PRO and 83.3% for humanized O1-PRO explanations.

4 Can LLMs be prompted to mimic human expert detectors?

The prompt-based detector shows promise and outperforms Binoculars and RADAR, but it remains behind human experts and closed-source detectors such as Pangram. GPT-4O performs best with chain-of-thought and the guidebook, whereas O1 favors chain-of-thought without the guidebook and is more conservative.

  • Implementing a prompt-based detector: The detector prompts GPT-4O and O1 to classify texts as human-written or AI-generated and explain decisions using criteria from the guidebook.The prompt template is provided in Table 22.
  • Implementing a prompt-based detector: Because the detector outputs labels rather than scalar scores, its false-positive-rate threshold cannot be adjusted, despite controllable prompting options such as chain-of-thought.This limitation complicates threshold-based evaluation and deployment.
  • Prompt-based detection shows promise but struggles with humanization and high FPRs: GPT-4O reaches 78% average TPR with chain-of-thought and the guidebook, while O1 reaches 54% with chain-of-thought but without the guidebook.The best configuration differs by model: GPT-4O benefits from the guidebook, whereas O1 does not.
  • Prompt-based detection shows promise but struggles with humanization and high FPRs: O1 is more conservative than GPT-4O, achieving lower average FPRs at the cost of lower TPR.This reflects a precision–recall-style tradeoff between false-positive control and true-positive detection.
  • Prompt-based detection shows promise but struggles with humanization and high FPRs: GPT-4O-2024-11-20 performs comparably to Binoculars and Fast-DetectGPT, but at much higher cost, while still lagging human experts and Pangram.The authors hypothesize that fine-tuning could reduce the gap between prompt-based detection and humans.

5 Related work

Prior work finds that human ability to detect AI-generated text varies across annotators, while automatic detectors rely mainly on perplexity or trained classifiers and can be vulnerable to paraphrasing, style changes, and humanization. Other studies characterize differences between human- and AI-written text and analyze human detection explanations.

  • Human detection of AI-generated text: Human detection is unreliable for naïve annotators, although some annotators perform very well, and studies also examine detection across languages.Prior work includes studies of detection in languages other than English.
  • Automatic detection: Automatic detectors primarily rely on perplexity or trained classifiers, with prior work comparing their performance across domains and regimes.These approaches represent the main methodological families in automatic detection research.
  • Automatic detection: Detectors can be bypassed through paraphrasing, style changes, and humanization attacks, while other studies seek to improve detector robustness.The cited work covers both attacks designed to evade detection and methods intended to strengthen detectors.
  • Analyzing differences between human-written and AI-generated text: Research on human–AI text differences identifies gaps in depth and content quality, repetitiveness, syntactic patterns, and categories of human nonexpert detection explanations.These studies provide frameworks for analyzing AI-generated text and human judgments about it.

6 Conclusion

The paper concludes that frequent LLM users are highly accurate and robust detectors of AI-generated text without additional training. Five-expert majority voting performs near perfectly on 300 articles and outperforms all automatic detectors except Pangram, which it matches.

  • 6 Conclusion: Five-expert majority voting performs near perfectly on 300 articles, outperforming every automatic detector except Pangram, which it matches.The experts achieved this performance without additional training.
  • 6 Conclusion: Frequent LLM users can serve as highly accurate and robust AI-text detectors without specialized training.
  • 6 Conclusion: Analysis of expert explanations shows that these annotators identify textual signals underlying their detection decisions.The supplied passage begins describing this analysis but is truncated before specifying all identified signals.

Limitations

The study is limited to professionally written, proofread articles in American English with consistent formatting. It did not assess factual accuracy and may include AI-generated edits beyond the detection scope.

  • Limitations: The dataset is restricted to professionally written and proofread articles in American English selected for consistent formatting.This sampling choice limits the study’s scope across languages, writing quality, and formatting conditions.
  • Limitations: Factual accuracy was not investigated because it did not appear to be a significant cue for annotators across the broad range of topics.
  • Limitations: Some articles from reputable sources may have contained AI-generated edits beyond the study’s detection scope.

Ethical Considerations

The study received exempt IRB review, obtained informed consent from annotators, compensated them fairly, and acknowledged risks associated with misinformation and hallucinated AI content.

  • Ethical Considerations: The study was reviewed by the UMass Institutional Review Board (IRB #5927) and deemed exempt; annotators were briefed and provided informed consent.Participants who wished to be acknowledged by name explicitly agreed in their consent forms.
  • Ethical Considerations: The researchers provided fair compensation and acknowledged potential risks from misinformation and hallucinated AI content.

A Human Evaluation · A.1 Finding Expert Annotators

The study identifies expert human detectors by screening annotators on a difficult sample task, focusing on people who frequently use LLMs for writing. The annotation protocol uses trained, within-subject judgments with randomized article order and recorded decisions, confidence, and explanations.

  • A Human Evaluation: The initial annotation pool comprised five native English speakers from the US or South Africa with university degrees, varied professions, and varied AI-assistant experience.Among them, one had never used AI, two had little experience, and two used AI daily.
  • A Human Evaluation: All annotators read guidelines and signed consent forms before labeling, with authors estimating a throughput of 8–12 articles per hour.The collection process required about 20 hours of author communication.
  • A Human Evaluation: Each annotation recorded a human-versus-AI decision, confidence score, and explanation after annotators read and highlighted the article.Batches contained 60 articles, typically required 6–8 hours, and could be completed within a week with extensions when needed.
  • A Human Evaluation: Annotators judged both human-written and AI-generated articles in a within-subjects design, with randomized order and concealed pairing to reduce variability and bias.The design also required fewer annotators than a between-subjects arrangement.
  • A.1 Finding Expert Annotators: Five annotators were designated experts after passing performance requirements designed to identify reliable human detectors of AI-generated text.The screening followed a pilot in which only the daily LLM user achieved high accuracy, motivating an investigation of the upper bound of human performance.
  • A.1 Finding Expert Annotators: 50% of recruited candidates passed the five-question screen, but only 4 of those 5 passed the subsequent 60-question set.The initial requirement for the longer set was at least 90% correct, or 54/60.
  • A.1 Finding Expert Annotators: The selection targets frequent LLM users in writing-related professions because their experience may help them recognize systematic patterns in AI-generated text.The authors specifically mention English teachers, editors, and publishers, while acknowledging that some unfamiliar users may also be reliable detectors.

B Dataset · B.1 Article Corpus · B.2 AI Text Generation

The appendix documents the article corpus and the procedures used to generate, paraphrase, and humanize AI articles. The corpus spans American-English publications, while generation used standardized prompts and progressively refined evasion strategies.

  • B Dataset: The dataset appendix details the article corpus and AI-article generation procedures used in the study.It organizes this material into corpus details (B.1) and AI text generation (B.2).
  • B.1 Article Corpus: The corpus comprises articles from American-English publications, with publication lists and source-level length statistics reported in Tables 5 and 7.Table 5 identifies the included publications and their website sections, while Table 7 reports token and word counts by source.
  • B.2 AI Text Generation: AI articles were generated from titles, subtitles, approximate lengths, publications, and sections, with stories prompted using Reddit thread titles and closed-source models accessed through provider APIs.The procedure also included rounding instructions so articles in the same topic would have similar lengths.
  • B.2 AI Text Generation: The generation prompt requested an approximately 750-word, concise, understandable article that included relevant expert quotations for a New York Times science article.The example used the title “The Science That Makes Baseball Mud ‘Magical’” and its subtitle as inputs.
  • B.2 AI Text Generation: Sentence-level paraphrasing passed the already paraphrased text into later calls to improve the article’s overall flow.The first sentence was paraphrased alone, while subsequent sentences incorporated the preceding paraphrased portion.
  • B.2 AI Text Generation: Humanization experiments progressed from generation-then-humanize and step-by-step editing to a two-step method that first identified AI-like traits and then applied suggested edits.The step-by-step method targeted creativity, tone, grammar, and typical AI vocabulary but was abandoned because repeated calls added AI characteristics; the two-step method performed best among these efforts.
  • B.2 AI Text Generation: The final evader instead generated articles as humanized text in one step using reference articles, detection-guide instructions, examples, and the initial article prompt.The appendix also notes commercial humanization services such as Undetectable AI, MyEssayWriter.ai, and Stealth Writer.

C Pilot Study … D.2 What do expert annotators see that nonexperts don’t?

The supplementary sections describe a pilot study comparing expert and nonexpert detection on articles and informal Reddit stories, then detail how expert explanations were categorized. The story results show higher nonexpert detection performance than on articles, while the comment analysis combines human annotation with GPT-4O classification.

  • C Pilot Study: The pilot tested five annotators with varying LLM familiarity across two rounds covering the article corpus and r/WritingPrompts stories.Both rounds followed the annotation process described in §2.
  • C Pilot Study: The story corpus contained 30 r/WritingPrompts stories paired with AI-generated stories produced using the prompt in Table 15.The corpus was designed to support the second pilot round.
  • C Pilot Study: Nonexperts achieved a 62.5% TPR on stories, rising to 69.3% when the expert annotator was included.The story results are reported in Table 16.
  • C Pilot Study: Reddit stories produced more superficial detection differences because their informal style, including acronyms, was not reflected in the generation prompt.The passage contrasts Reddit’s informality with the prompt used to generate stories.
  • D.2 What do expert annotators see that nonexperts don’t?: The analysis examines what expert annotators identify in their explanations and provides additional explanation-level analysis.This section’s stated purpose is to analyze expert explanations.
  • D Comment Analysis: The comment-analysis framework categorizes expert explanations using categories defined in Table 3 and refined through independent annotation and author agreement.The authors labeled a stratified sample of 25 explanations, with one comment per expert for each of five experiments.
  • D.1 Categorization of Comments: GPT-4O-2024-11-20 classified all 1,500 explanations, allowing multiple categories per explanation at a total cost of $6.02 USD.The model was prompted only after its classifications aligned with the authors’ labels on most sampled explanations.

D.3 Individual Experts Commentary … E.2 Explainable Detection

Individual experts used varied textual clues, remained effective against paraphrased articles, but one expert was notably vulnerable to o1-pro content. The paper also benchmarks automatic detectors and tests prompt-based detectors designed to mimic human reasoning, including explanations, guides, and majority voting.

  • D.3 Individual Experts Commentary: Experts relied on distinct clues: names, linguistic conventions, example groupings, article flow and specificity, motivation, and natural-sounding quotations.Annotator 1 uniquely noticed frequent AI-associated names, while Annotators 2–5 emphasized linguistic, structural, content, and quotation features.
  • D.3 Individual Experts Commentary: 88% of paraphrased-article explanations mentioned AI vocabulary, compared with 69.8% for non-paraphrased GPT-4o articles.Experts also continued identifying formulaic sentence structures and cheerful summary conclusions after paraphrasing.
  • E.1 Automatic Detectors Benchmarked: The study benchmarks closed-source Pangram and GPTZero alongside open-source Binoculars, Fast-DetectGPT, and RADAR detectors.The benchmark uses vendor labels where available and 5% false-positive-rate thresholds for methods requiring calibration.
  • E.1 Automatic Detectors Benchmarked: Automatic-detector thresholds without author recommendations were calibrated on 40 held-out human-written articles to achieve a 5% false-positive rate.The reported thresholds were 0.6051510572 for RADAR, 0.96 for Fast-DetectGPT, and 0.8963184953 for e5-lora.
  • E.2 Explainable Detection: Prompt-based detection was motivated by the cost and slowness of human annotation, which averaged $2.82 per article with a week for each experiment.The prompt-based detector was designed to mimic human detection rather than current automatic detectors.
  • E.2 Explainable Detection: The prompt-based experiments compared zero-shot, chain-of-thought, guide, and combined chain-of-thought-plus-guide configurations, with explanations intended to mimic human reasoning.The combined setup asks for both a classification and an explanation of the clues supporting the judgment.
  • E.2 Explainable Detection: Expert annotators performed better with majority voting, suggesting that prompt-based detectors could use voting to improve accuracy and avoid false positives.The supplied passage reports the strategy-level observation but does not provide its quantitative result.

E.3 Results · CATEGORY FREQ DEFINITION EXAMPLE EXPLANATIONS

Detector performance varied substantially across models, with GPT-4O-2024-11-20 outperforming GPT-4O-2024-08-06 on paraphrased and O1 articles. Expert explanations most often cited vocabulary and sentence structure, while also identifying originality, formality, clarity, and other broader stylistic cues.

  • E.3 Results: At a 5% false-positive rate, e5-lora correctly classified none of the AI-generated articles.The model was one of the top-performing systems on the Raid benchmark, but failed in this evaluation.
  • CATEGORY FREQ DEFINITION EXAMPLE EXPLANATIONS: Experts most frequently identified vocabulary (53.1%) and predictable sentence patterns (35.9%) as evidence of AI-generated writing.They associated LLMs with repetitive or unnatural wording and constructions such as “not only ... but also ...” or lists of three items.
  • CATEGORY FREQ DEFINITION EXAMPLE EXPLANATIONS: Additional cues included grammatical perfection (24.8%), excessive explanation or irrelevant detail (19.5%), consistent formatting (15.0%), and formal tone (12.3%).Annotators contrasted these features with human errors, varied formatting, concise flow, filler words, contractions, slang, and abbreviations.
  • CATEGORY FREQ DEFINITION EXAMPLE EXPLANATIONS: Experts also used originality (23.7%), quote style (22.3%), and conclusion patterns (13.1%) to distinguish AI-generated from human writing.AI writing was described as safe and unsurprising, formally homogeneous in quotations, and repetitive or overly optimistic in conclusions.
  • CATEGORY FREQ DEFINITION EXAMPLE EXPLANATIONS: Annotator examples show that human writing was associated with varied grammar, creative phrasing, authentic short quotes, distinctive names, and concrete or emotionally detailed descriptions.AI examples instead emphasized formulaic comparisons, homogeneous quotations, standardized formatting, generic names, formal diction, and optimistic abstraction.
  • CATEGORY FREQ DEFINITION EXAMPLE EXPLANATIONS: The zero-shot-plus-chain-of-thought-plus-guide detector was prompted to describe candidate-text features before giving a HUMAN-WRITTEN or AI-GENERATED judgment.This setup explicitly required using the detection guide and producing both a description and final answer.
Loading 2501.15654v2…