Source-linked AI summary
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
TL;DR
The paper addresses the need for robust detection of fluent machine-generated text amid concerns about misuse. It introduces and evaluates M4, a multi-generator, multi-domain, multilingual benchmark, finding that detectors struggle on unseen domains, generators, or languages. The results indicate that machine-generated text detection remains unsolved and requires further improvement.
Problem
Fluent LLM-generated text creates misuse concerns, while existing detection must support varied generators, domains, and languages.
Method
The paper constructs M4, a multi-generator, multi-domain, multilingual corpus, and evaluates detectors across domains, generators, languages, and time periods.
Results
Detectors struggle to distinguish machine-generated from human-written text when the domain, generator, or language is unseen during training.
Takeaways & Limitations
M4 is released to support research toward more robust approaches to detecting malicious machine-generated text.
Takeaways & Limitations
Detection performance is constrained by evolving LLMs and datasets that can become outdated relatively quickly.
Abstract
from arXiv · showhide
Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. However, this has also raised concerns about the potential misuse of such texts in journalism, education, and academia. In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse. We first introduce a large-scale benchmark \textbf{M4}, which is a multi-generator, multi-domain, and multi-lingual corpus for machine-generated text detection. Through an extensive empirical study of this dataset, we show that it is challenging for detectors to generalize well on instances from unseen domains or LLMs. In such cases, detectors tend to misclassify machine-generated text as human-written. These results show that the problem is far from solved and that there is a lot of room for improvement. We believe that our dataset will enable future research towards more robust approaches to this pressing societal problem. The dataset is available at https://github.com/mbzuai-nlp/M4.
1 Introduction
The paper addresses the difficulty of detecting increasingly fluent machine-generated text by introducing a broad benchmark designed for more general detection. It evaluates detectors across generators, domains, languages, and time periods to identify where generalization fails.
- LLMs produce fluent content across journalism, education, academia, and other channels, raising concerns about misuse such as disinformation and educational disruption.
- Human classification of machine-generated versus human-written text performs only slightly better than chance, motivating automatic detection research.
- M4 is a large-scale corpus spanning multiple generators, domains, and languages for black-box machine-generated text detection.
- The study evaluates detectors across domains, generators, multilingual settings, and generation time periods.
- M4 combines released data and code with plans to add generators, domains, and languages over time.
2 Related Work
Prior work developed diverse machine-generated-text corpora and black-box detection methods, but typically focused on limited languages, domains, or generators. This work emphasizes broader evaluation and notes that black-box effectiveness depends heavily on training-corpus quality and diversity.
- Detection approaches: Detection strategies are categorized as black-box or white-box according to access to the suspected generating LLM.White-box methods include zero-shot detection, watermarking, and expected per-token log probabilities.
- Research scope: Black-box techniques target a more practical and general use case, but their effectiveness heavily depends on training-corpus quality and diversity.This dependence is presented as an important consideration for black-box detection.
- Related corpora: Existing corpora include TuringBench, HC3, TOEFL essays, and RuATD, covering different combinations of models, domains, languages, and generation settings.TuringBench contains 200K texts from 19 generative models, while HC3 contains nearly 40K English and Chinese question-answer pairs.
- Research scope: Previous studies generally concentrated on one or two languages, a specific LLM, or a single domain, whereas M4 broadens coverage across languages, widely used LLMs, and domains.The paper also selects diverse state-of-the-art black-box methods and includes an off-the-shelf detector.
- Detection approaches: Black-box detectors commonly use statistical distributions, linguistic patterns, or fact-verification features with neural or traditional classifiers.Examples include GLTR-like word rankings, vocabulary and stylistic features, and RoBERTa-based classification.
3 The M4 Dataset
M4 is a multilingual, multidomain benchmark pairing human-written and machine-generated texts from diverse sources and generators. The dataset construction includes prompt diversity, artifact cleaning, model-level quality checks, and analyses of vocabulary richness and human detection performance.
- Dataset composition: Human-written texts come from diverse sources spanning multiple domains and languages, including Wikipedia, WikiHow, Reddit, arXiv, PeerRead, Baike/Web QA, RuATD, and news.The sources vary by language and domain, with academic, encyclopedic, question-answering, social, and news content represented.
- Dataset composition: Machine-generated texts are produced by GPT-4, ChatGPT, GPT-3.5, Cohere, Dolly-v2, and BLOOMz 176B for tasks such as article writing, abstract generation, reviewing, summarization, and question answering.The generators are prompted for domain-specific outputs including Wikipedia articles, arXiv abstracts, PeerRead reviews, news briefs, and answers.
- Data construction: Multiple prompts per generator were designed in varied styles to produce diverse outputs aligned with divergent real-world generations.The number of prompts ranges from 2 to 8 depending on the generator and domain.
- Data construction: The authors minimally cleaned artifacts such as repeated newlines, bullet points, references, URLs, short paragraphs, and PDF-conversion line breaks.The cleaning aimed to avoid simple formatting differences that could help detectors learn non-generalized signals.
- Data construction: Quality control sampled 10–20 cases per domain and generator for plausibility, while model-level checks excluded fundamentally failed outputs and example-level checks required at least 1,000 characters.The authors explicitly state that examples were not selected individually for quality beyond the length check.
- Dataset composition: M4 contains ∼147k human–machine parallel examples and over 10M non-parallel human-written texts across English and five other languages.The parallel data include 102k English examples and 45k examples in Chinese, Russian, Bulgarian, Urdu, Indonesian, and Arabic.
- Evaluation design: The dataset uses 500 human and 500 machine-generated examples per generator for development and testing, with the remainder typically used for training.This split is applied across languages and domains.
- Dataset analysis: Human-written texts have richer unigram and bigram vocabularies than each individual generator, while the combined generators have vocabulary comparable to humans.Dolly-v2 has the largest generator vocabulary, followed by davinci, ChatGPT, BLOOMz, and Cohere.
4 Detectors
The study evaluates seven detectors spanning pretrained multilingual and monolingual classifiers, feature-based models, and an off-the-shelf system. The detectors use learned representations, token-ranking information, stylistic properties, news-landscape features, or a combination of these approaches.
- Detector suite: Seven detectors are evaluated using specified hyper-parameter settings.The evaluated systems include pretrained classifiers, feature-based models, and GPTZero.
- Neural detectors: RoBERTa is fine-tuned to detect machine-generated texts, while ELECTRA is fine-tuned using a pretraining objective aligned with token replacement detection.ELECTRA was pretrained to predict whether a corrupted token was replaced by a plausible alternative from a small generator network.
- Neural detectors: XLM-RoBERTa is fine-tuned as a multilingual detector for machine-generated text.XLM-R is described as a multilingual variant of RoBERTa.
- Feature-based detectors: A logistic-regression detector uses 14 GLTR features based on word-ranking information from high-probability token distributions.The features include counts of tokens appearing in top-10, top-100, and top-1000 probability groups.
- Feature-based detectors: SVM detectors use stylistic features covering character, syntactic, structural, and word-based properties, or NELA features covering style, complexity, bias, affect, moral, and event aspects.These feature families capture text form, writing complexity, sentiment, subjectivity, and temporal or location-related information.
- Cross-domain evaluation: Figure 1 reports cross-domain accuracy when training on one domain and testing across domains for ChatGPT and davinci generations across five detectors.The figure directs readers to Tables 12 and 13 for more detail.
- Off-the-shelf detector: GPTZero is used without adaptation and was trained on a diverse corpus of human-written and AI-generated texts, with a focus on English.It analyzes inputs ranging from individual sentences to entire documents.
5 Experiments and Results
M4 evaluates black-box machine-generated text detectors across domains, generators, languages, and time. Detection is strong in matched settings but generalization to unseen domains, generators, and languages remains difficult.
- Experimental settings: The experiments cover same-generator cross-domain, same-domain cross-generator, cross-lingual cross-generator, zero-shot GPTZero, and temporal evaluations.The study also examines feature behavior and detector performance across multiple experimental perspectives.
- Same-generator, cross-domain: In-domain detection often approaches 100% accuracy, especially for RoBERTa, while LR-GLTR reaches only 79.6% on WikiHow for davinci-003.RoBERTa reaches 100% on ChatGPT-generated arXiv text and 99.7% on both Wikipedia and WikiHow in the reported examples.
- Same-generator, cross-domain: Out-of-domain detection is difficult, but ELECTRA often performs best because its replaced-token pre-training objective supports generalization.For davinci-003 trained on Wikipedia and tested on Reddit, RoBERTa is near random guessing while ELECTRA achieves 87.9% accuracy; LR-GLTR wins in some scenarios.
- Same-generator, cross-domain: Reddit generally provides the strongest training-domain generalization, whereas arXiv and PeerRead generalize worst and WikiHow is the hardest target domain.Academic writing style is described as especially specific, while PeerRead is reported as the easiest target domain.
- Same-domain, cross-generator: RoBERTa leads cross-generator accuracy on arXiv at 95.9% and Wikipedia at 99.4%, but performance drops substantially when training and testing use different LLMs.NELA features are especially weak, with many scores around or below 50.0%, particularly on arXiv.
- Same-domain, cross-generator: BLOOMz is the most divergent generator: cross-generator accuracy is often ≤50.0% and machine-text recall is below 0.5, producing many false negatives.ChatGPT and Cohere show comparable cross-detection accuracy, suggesting shared generative patterns.
- Zero-shot evaluation: GPTZero: GPTZero performs best on Wikipedia and worst on arXiv, where every generator has F1 below 50%; its BLOOMz recall is near 0% across domains.Zero-shot detection on unseen domains and generators remains a major challenge for GPTZero.
- Multilingual evaluation: Multilingual XLM-R performs best when language and generator match, struggles on unseen Russian, Urdu, and Indonesian, but remains effective for Arabic after English-only training.For cross-generator testing, the All-language training row usually performs best, except for Bulgarian.
6 Conclusion and Future Work
The paper presents M4 as a broad benchmark for machine-generated text detection and evaluates detectors across generators, domains, languages, and settings. These experiments show that generalization remains difficult, motivating continued dataset expansion and more robust detection research.
- M4 is a large-scale dataset spanning multiple generators, domains, and languages for machine-generated text detection.
- The authors evaluate detectors through cross-domain, cross-generator, cross-lingual, and zero-shot experiments using seven detectors.
- Detectors struggle when test texts come from an unseen domain, generator, or language.
- M4 has already been extended for SemEval-2024 Task 8 with additional languages, domains, and three new task reformulations.
- The authors plan to expand M4 with new generators, domains, languages, tasks, and prompts as LLMs evolve.This expansion is intended to keep the dataset up-to-date and relevant.
Ethics and Broader Impact
The authors address ethics and broader impact through documentation, licensing guidance, privacy-conscious data collection, and discussion of dataset biases. They emphasize responsible use because biases may originate in both source corpora and the LLMs used for generation.
- The dataset uses preexisting publicly released corpora approved for research, rather than scraping raw website data.
- Potential biases may arise from both human-written corpora and LLMs, and external parties may create biased subsets beyond the authors’ control.
- The authors provide documentation covering human-text sources, generation processes, prompts, cleansing measures, and potential biases.They argue that this transparency helps researchers understand data origins and make informed usage decisions.
- M4 is intended to support detection systems that mitigate malicious LLM-generated content, while requiring adherence to source-corpus and LLM licensing terms.
Limitations
M4 is useful for studying detection across generators, domains, and languages, but the authors caution that it cannot guarantee broad generalization or remain current as LLMs evolve. They also identify possible dataset biases and prompt sensitivity as limitations.
- Generalization: Prompt sensitivity limits how broadly machine-generated outputs in M4 can represent possible generations.The dataset uses diverse prompts, but the authors note that outputs are highly sensitive to prompting.
- Generalization: M4 may not suffice to train detectors that generalize across all possible domains and generators or to serve as a standard benchmark.The authors describe these as limitations of M4 as a general resource.
- Up-to-Date: Detection remains challenging when the generator and domain are unknown, with human- and machine-written text indistinguishable in some situations, including BLOOMz outputs.The authors report this as an up-to-date challenge for detection.
- Up-to-Date: Because LLMs evolve rapidly, datasets for machine-generated-text detection can become outdated relatively quickly.The authors address this by continually extending M4, including with newer GPT-4 responses.
- Bias: Biases may affect detection outcomes in both human-written and machine-generated texts, including biases introduced during human data collection.The authors leave bias analysis to future work.
A.4 N-gram Analysis
The appendix reports n-gram statistics for human-written and machine-generated English texts and documents detector settings and computational or API costs.
- N-gram Statistics: Tables 9 and 10 report unique uni-gram and bi-gram statistics, including per-document counts, for English human-written and machine-generated texts.Table 9 gives overall counts, while Table 10 gives per-document counts.
- Detector Settings: Table 11 lists the non-default hyper-parameters used for the five trained detectors.LR-GLTR uses 1,000 maximum training iterations; Linear-SVM changes its penalty parameter C and maximum iterations from defaults.
- Resources: The study spent $600 on OpenAI API calls for ChatGPT and davinci-003 generations and about 2,500 GPU hours on Dolly-v2 and BLOOMz generation.The reported costs cover generation and detector-related API usage described in the passages.
C Results: Same-Generator, Cross-Domain
The same-generator, cross-domain experiments train detectors on one domain and test them across domains for ChatGPT and davinci-003, using accuracy, precision, recall, and F1.
- Experimental Setup: Tables 12 and 13 evaluate cross-domain generalization when training on a single domain and testing across domains.Table 12 uses ChatGPT versus human text, while Table 13 uses davinci-003 versus human text.
- Evaluation: The reported metrics are accuracy, precision, recall, and F1 score with respect to machine-generated text.Results are reported in percentage units across four detectors.
D Results: Same-Domain, Cross-Generator
The same-domain, cross-generator experiments train and test detectors within a single domain while changing the machine-text generator, using arXiv and Wikipedia settings.
- arXiv: Table 14 evaluates cross-generator detection on arXiv when training and testing use a single machine-text generator versus human text.The evaluation reports accuracy, precision, recall, and F1 score for four detectors.
- Wikipedia: Table 15 provides the corresponding cross-generator evaluation on Wikipedia.The table reports accuracy, precision, recall, and F1 score with respect to machine generations.
E Results: Multilingual Evaluation
The multilingual experiments evaluate cross-language detection with XLM-R on ChatGPT- and davinci-003-generated test sets. Tables 16 and 17 report accuracy and machine-generated-class F1 across languages.
- Table 16 reports XLM-R accuracy and machine-generated-class F1 across languages for ChatGPT-generated test sets.Performance is averaged across five runs, with standard deviations shown in parentheses.
- Table 17 reports the corresponding XLM-R cross-language results for davinci-003-generated test sets.The table uses the same accuracy and machine-generated-class F1 metrics, averaged across five runs.
- The experiments compare cross-language detection for two generators, ChatGPT and davinci-003, using separate result tables.
F Results: Impact of Text Length
Detection performance declines as generated texts become shorter. Across arXiv and Reddit texts from ChatGPT, davinci, and Cohere, shorter inputs reduce machine-generated-text F1.
- As character length decreases from 1000 to 125, machine-generated-text F1-score decreases across all evaluated subsets.The result is reported for arXiv and Reddit texts generated by ChatGPT, davinci, and Cohere.
- Figure 3 studies text-length effects on detection accuracy for arXiv and Reddit texts generated by ChatGPT, davinci, and Cohere.
- Short text has a negative impact on detection performance in the reported experiments.
G Feature Analysis with LIME
Figure 4 visualizes LIME-extracted features for Reddit texts generated by ChatGPT. The examples include false-positive, true-negative, and true-positive cases.
- The visualization includes one false-positive, one true-negative, and one true-positive example.
- The supplied figure description presents the visualization as an example-based analysis of detector decisions.
- Figure 4 visualizes features extracted by LIME for Reddit as the domain and ChatGPT as the generator.