PaperlayerView exact revision on arXiv

Source-linked AI summary

MAGE: Machine-generated Text Detection in the Wild

Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, Yue Zhang

arXiv:2305.13242v3cs.CL

TL;DR

Machine-generated text detection must handle diverse domains and LLMs without known sources, but existing evidence is limited in such settings. MAGE builds a broad testbed and finds substantial out-of-distribution difficulty, while the best detector reaches 86.54% AvgRec on GPT4-generated texts.

  • Problem

    Existing detection research is limited by evaluations on specific domains or particular LLMs, despite practical detectors facing unknown sources.

  • Method

    MAGE builds a testbed from human texts across 7 writing tasks and machine texts generated by 27 LLMs, organized into 8 increasingly wild testbeds.

  • Results

    86.54% AvgRec is achieved by the Longformer detector on out-of-domain texts generated by the new GPT4 model.

  • Takeaways & Limitations

    Detection remains difficult as domain and model diversity increases, but 0.1% in-domain data can produce over 80% recall for texts from unseen LLMs.

  • Takeaways & Limitations

    The benchmark-based text sources may not represent new online texts, and newly emerging LLMs may be absent from the dataset.

Abstract

from arXiv · show

Large language models (LLMs) have achieved human-level text generation, emphasizing the need for effective AI-generated text detection to mitigate risks like the spread of fake news and plagiarism. Existing research has been constrained by evaluating detection methods on specific domains or particular language models. In practical scenarios, however, the detector faces texts from various domains or LLMs without knowing their sources. To this end, we build a comprehensive testbed by gathering texts from diverse human writings and texts generated by different LLMs. Empirical results show challenges in distinguishing machine-generated texts from human-authored ones across various scenarios, especially out-of-distribution. These challenges are due to the decreasing linguistic distinctions between the two sources. Despite challenges, the top-performing detector can identify 86.54% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios. We release our resources at https://github.com/yafuly/MAGE.

1 Introduction

MAGE studies machine-generated text detection when texts span diverse human writings, domains, and LLMs whose sources are unknown. Results show that detection becomes harder as diversity increases, although limited in-domain data and perplexity-based distinctions support practical detection.

  • LLM-generated text can support fake news, public-opinion manipulation, and plagiarism, motivating detection of machine-written versus human-written text.
  • Existing studies often target specific domains, particular models, or settings where the source LLM is accessible.
  • MAGE collects human texts from 7 writing tasks and machine-generated texts from 27 LLMs using 3 prompt types across 8 progressively wilder testbeds.
  • 61.95% of human-written texts from unseen domains are misclassified by the best detector, while 0.1% in-domain data raises recall above 80% for unseen-LLM text.
  • As domain and model diversity increases, linguistic differences between human and machine texts converge, leaving detectors fewer surface patterns to exploit.
  • 86.54% AvgRec is achieved on out-of-domain texts generated by GPT4, and perplexity remains a fundamental classification feature across domains and generation LLMs.

2 Related Work

Prior detection research uses linguistic statistics, neural detectors, and watermarks, but robustness and transfer to unseen domains or models remain uncertain. MAGE evaluates detection under combined distribution shifts and paraphrasing attacks.

  • Machine-writing detection has used n-gram frequencies, entropy, perplexity, and negative-curvature regions of model log probabilities.
  • Statistics-based methods can require model prediction distributions, limiting use with models accessed only through APIs.
  • Existing detectors face robustness concerns from paraphrasing exploits, while transferability to unseen domains or models remains uncertain.
  • MAGE combines texts from varied domains and language models and extends evaluation to out-of-distribution settings with paraphrasing attacks.

3 Dataset Construction

MAGE constructs a broad detection testbed from diverse human writing tasks and machine text generated by many LLMs under multiple prompt conditions. The resulting data are organized to increase distributional variance and detection complexity.

  • Human-written data cover diverse tasks including opinion statements, news writing, question answering, story generation, and commonsense reasoning.
  • The dataset uses a wide spectrum of 27 representative LLMs, including GPT, LLaMA, GLM-130B, FLAN-T5, OPT, and BigScience models.
  • Continuation prompts use the previous 30 words of a human text, whereas topical prompts generate text from a topic.
  • Specified prompts add source information such as BBC News or Reddit Post to topical prompts and are designed for OpenAI models.
  • For each human-written text, MAGE generates machine-written counterparts with 27 LLMs and 3 prompt types.

4 Detection Methods

The study compares detectors that classify text as human or machine generated, or output a probability distribution, using naive human and ChatGPT baselines alongside a supervised PLM-based classifier.

  • Detection systems either assign human-versus-machine labels or output a probability distribution over the two sources.
  • Naive baselines ask humans and ChatGPT to identify each text’s source.
  • The supervised methods include a PLM-based classifier commonly used for text detection.

5 Experimental Setup

The experiments evaluate detection across eight increasingly challenging settings, varying whether domains and language models are seen or unseen and including paraphrasing attacks. Performance is measured with AUROC and AvgRec, using separate classifiers and weighted averages across cross-validation testbeds where applicable.

  • Experimental settings: The benchmark spans eight settings, from within-domain white-box detection to paraphrasing attacks, with progressively increasing detection difficulty.The settings vary the sources of training and evaluation data, including seen or unseen domains and model sets.
  • Experimental settings: MAGE evaluates fixed or arbitrary domains crossed with model-specific or arbitrary model sets, then adds unseen-model, unseen-domain, joint unseen-domain/model, and paraphrasing settings.The first four testbeds cover in-distribution combinations; later testbeds assess generalization and robustness to paraphrased texts.
  • Out-of-distribution evaluation: Unseen-model and unseen-domain evaluations exclude the relevant models or domains from training, train separate classifiers for cross-validation, and report weighted-average performance.The unseen-model setting creates 7 testbeds, while the unseen-domain setting creates 10 testbeds.
  • Out-of-distribution evaluation: The joint unseen-domain/unseen-model test uses four datasets and GPT-4 to evaluate detection of machine-generated news, dialogues, scientific answers, and movie reviews.The datasets are CNN/DailyMail, DialogSum, PubMedQA, and IMDb, with 200 sampled instances from each.
  • Robustness evaluation: The paraphrasing-attack test paraphrases each sentence with gpt-3.5-turbo and labels paraphrases from both human and machine sources as machine-generated.This test set is constructed from the unseen-domain/unseen-model test set.
  • Evaluation metrics: Performance is reported with AUROC and AvgRec, where AvgRec averages recall on human-written and machine-generated texts for realistic detection assessment.AUROC ranges from random guessing at 0.5 to perfect classification at 1.0; AvgRec is also useful when AUROC cannot be evaluated.

6 Results

Detection is effective in narrow, source-matched settings but degrades as domains and language models diversify, especially out of distribution. Longformer is the strongest detector and remains feasible for unseen domains and models, although paraphrasing reduces its performance.

  • In-domain Detection: All methods perform solidly when texts come from one domain and one known LLM, particularly in the white-box setting.DetectGPT performs well when its scoring model matches the generation LLM.
  • In-domain Detection: GLTR’s AUROC drops from 0.94 to 0.80 and DetectGPT’s from 0.92 to 0.57 on texts from multiple models.The DetectGPT decline is attributed to reliance on access to the generation LLM.
  • In-domain Detection: Longformer consistently outperforms the other detectors on AUROC and AvgRec as domains and models diversify.Its margin is especially large in the arbitrary-domains-and-models setting.
  • Out-of-domain Detection: 61.95% of out-of-distribution texts are misclassified by the best-performing detector, while novel domains remain difficult for every method.Except for Longformer, all detectors perform poorly on texts generated by unseen models, and none effectively classifies texts from novel domains.
  • Out-of-domain Detection: Longformer reaches AUROC 0.95 and AvgRec 86.61% on texts generated by unseen LLMs, although specific prompts are harder to distinguish than continuation or topical prompts.The detector has the most difficulty with OpenAI and FLAN-T5 outputs; specific prompts are more similar to human writing.
  • Out-of-domain Detection: On unseen domains, Longformer’s AvgRec falls from 90.53% to 68.40%, mainly because unfamiliar human texts are labeled machine-generated.ROC, XSum, and TLDR are most often misclassified, potentially because of their low average perplexity scores.
  • Out-of-domain Detection: After boundary refinement, Longformer achieves 0.94 AUROC and 86.54% AvgRec on texts from new datasets generated by GPT-4.The refined boundary balances detection of human and machine sources in the unseen-domains-and-model setting.
  • Robustness: Paraphrased texts reduce Longformer’s AUROC from 0.94 to 0.75.The paper attributes this vulnerability to a shifted perplexity distribution in paraphrased texts.

7 Analysis

The analysis examines linguistic differences and perplexity as potential bases for distinguishing human-written from machine-generated text. As domains and language models diversify, linguistic differences diminish, while perplexity remains useful but introduces systematic errors and paraphrase vulnerability.

  • Linguistic differences: Diverse domains and LLMs reduce linguistic dissimilarity between human-written and machine-generated texts, eventually eliminating significant statistical differences.This reduction aligns with the increasing difficulty of detection in broader in-distribution settings.
  • Perplexity: Perplexity can serve as a domain- and model-independent feature for clustering human-written and machine-generated texts.An untuned Longformer assigns machine-generated texts notably lower average perplexity than human writings, producing distinct peaks.
  • Perplexity bias: Perplexity bias causes detectors to classify low-perplexity human texts as machine-generated and high-perplexity machine texts as human-generated.False human-text predictions concentrate in lower-perplexity regions, whereas false machine-text predictions spread across higher-perplexity regions.
  • Paraphrasing: Paraphrasing shifts perplexity distributions and causes substantial confusion for the Longformer detector.Paraphrased human-written texts can occupy a position between machine-generated and paraphrased machine-generated peaks.

8 Conclusion

The paper introduces MAGE, a comprehensive testbed for evaluating machine-generated text detection across diverse writing tasks and LLMs. Detection becomes more difficult out of distribution, although Longformer achieves 86.54% AvgRec on out-of-domain texts generated by GPT4, and perplexity remains a general classification feature.

  • Contribution: MAGE gathers human-written texts from diverse writing tasks and machine-generated texts from different LLMs to evaluate detection in varied settings.The testbed is designed to represent increasing distributional variance and detection complexity.
  • Findings: Out-of-distribution settings pose a greater challenge for machine-generated text detectors.The conclusion identifies OOD detection as a central obstacle to deployment in application scenarios.
  • Findings: 86.54% AvgRec was achieved by the Longformer detector on out-of-domain texts generated by the new GPT4 LLM.The result is reported on the authors’ testbeds.
  • Findings: Perplexity can serve as a fundamental classification feature regardless of text domain or generation LLM.The paper presents this as a general distinction between human and machine compositions.

Limitations

The paper identifies two limitations: the testbed may omit newly emerging LLMs, and benchmark texts may overlap with LLM pretraining data, potentially affecting detection on newer online texts.

  • Scope: The testbed may not include newly emerging LLMs such as Alpaca and Vicuna.The authors intend the diverse testbed to support selecting detectors that can handle newly developed models in the future.
  • Scope: Using benchmark datasets as text sources may affect how detection performs on online texts absent from LLM pretraining data.The authors plan to collect previously unseen online texts to study this variation.

Ethics Statement

The paper reports ethical safeguards and documents dataset construction, prompting, preprocessing, annotation, model configurations, robustness checks, and challenging evaluation settings. These details specify how MAGE and its experiments were assembled and tested.

  • Ethics Statement: The work uses no private data or non-public information and recruits university linguistics students as paid annotators.Annotators receive 60 CNY per hour through public advertisement.
  • Dataset construction: MAGE pairs human-written texts with generations from 27 LLMs across continuation, topical, and specified prompts.The latter two prompt types are used for the OpenAI GPT model set.
  • Dataset construction: The dataset is preprocessed, filtered for length, and split into train, validation, and test sets using an 80%/10%/10% partition.Preprocessing includes punctuation normalization and line-break removal.
  • Evaluation: Human annotation uses three expert annotators who independently label a randomly paired test subset and whose performance is averaged.The subset is created by pairing machine-generated and human-written texts through random sampling.
  • Methods: Longformer is fine-tuned for five epochs on eight V100 GPUs, while FastText, GLTR, and DetectGPT use their specified n-gram, ranking, or perturbation-based configurations.DetectGPT uses T5-3B for infilling and GPT-J-6B for scoring.
  • Robustness and comparisons: The testbeds are robust to randomness, while Longformer performs best among evaluated PLM backbones and masked language models outperform GPT2.Balancing machine-generated and human-written training data has little effect on AvgRec and AUROC, although human-text false positives persist.
  • Challenging settings: All methods suffer severe AUROC degradation on unseen-domain/model and paraphrase-attack test sets.DetectGPT is excluded from these evaluations because it relies on a white-box detection setting.

H Text Characteristics

Across diverse domains and language models, human-written and machine-generated texts show few reliable linguistic or stylistic differences. Machine-generated texts are only marginally longer on average, while sentiment and grammatical formality vary by domain without a consistent source distinction.

  • Length Statistics: Machine-generated texts are marginally longer on average than human-written texts.The comparison did not impose explicit length control during generation.
  • Linguistic Pattern: Human-written and machine-generated texts share similar word-frequency, part-of-speech, named-entity, and constituent distributions.These distributions provide no significant linguistic difference to help classifiers distinguish the two sources.
  • Sentiment Polarity: Large-scale sentiment analysis finds no clear polarity distinction between human-written and machine-generated texts across domains and LLMs.Although ChatGPT has been reported to express more neutral sentiments than humans, that distinction does not persist in the broader setting.
  • Grammatical Formality: Machine-generated texts are equally or more grammatical than human-written texts in less formal domains.The study quantifies grammar formality using the average number of edits from a grammar error correction model.
  • Grammatical Formality: Grammar-formality differences depend on domain, with formal settings such as news and Wikipedia requiring separate comparison.The cited formal-domain discussion continues beyond the supplied passage.
Loading 2305.13242v3…