Source-linked AI summary
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn
TL;DR
DetectGPT addresses zero-shot detection of machine-generated text, where human classification performs only slightly better than chance and detectors operate without labeled samples. It uses negative curvature in a source model’s log-probability function, estimated through perturbations, and outperforms existing zero-shot methods while remaining competitive with bespoke detectors.
Problem
Zero-shot detection asks whether a candidate passage was sampled from a source model without using human-written or generated samples, addressing the difficulty of reliably identifying fluent but potentially inaccurate machine-generated text.
Method
DetectGPT estimates a source model’s log-probability curvature by comparing the original passage with meaning-preserving perturbations, using a tractable Hessian-trace approximation as the detection signal.
Results
DetectGPT’s curvature signal is more discriminative than existing zero-shot detection methods and competitive with bespoke detectors trained on millions of model samples.
Takeaways & Limitations
Model-generated text tends to occupy negative-curvature regions, suggesting that LLMs can implicitly expose a bias toward their sampled phrasing through lower-probability rephrasings.
Takeaways & Limitations
DetectGPT assumes white-box access to model log probabilities and a perturbation function whose rephrasings meaningfully represent the domain.
Abstract
from arXiv · showhide
The increasing fluency and widespread usage of large language models (LLMs) highlight the desirability of corresponding tools aiding detection of LLM-generated text. In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection. Specifically, we demonstrate that text sampled from an LLM tends to occupy negative curvature regions of the model's log probability function. Leveraging this observation, we then define a new curvature-based criterion for judging if a passage is generated from a given LLM. This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g., T5). We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. See https://ericmitchell.ai/detectgpt for code, data, and other project information.
1. Introduction
LLMs produce fluent text that can be difficult to distinguish from human writing, while existing detection approaches face practical limitations. DetectGPT addresses this problem by using negative curvature in model log probabilities as a zero-shot detection signal.
- Human performance in distinguishing machine-generated from human-written text is only slightly better than chance.
- Training a separate detector can overfit to topics and requires a new model for each released source model.
- Model-generated text tends to occupy negative-curvature regions where minor rewrites have lower log probability than the original.
- DetectGPT compares a candidate’s source-model log probability with the average log probability of perturbations generated by a generic language model such as T5.
- The paper identifies and empirically validates negative log-probability curvature at model samples and develops an algorithm approximating the Hessian trace for detection.
2. Related Work
Prior work includes supervised detectors, simple probability-based zero-shot criteria, and watermarking methods. These approaches differ in their reliance on training data, model modifications, or learned representations.
- Earlier work fine-tuned GROVER to detect its own generations after human evaluators found its propaganda at least as trustworthy as human-written propaganda.
- Supervised detectors use neural representations, bag-of-words features, or handcrafted statistical features.
- A simple zero-shot baseline thresholds a passage using its average log probability under the generative model.
- Watermarking methods modify model generations to make them easier to detect.
- Related analysis connects detection performance to the total variation distance between model and human text distributions.
3. The Zero-Shot Machine-Generated Text Detection Problem
The paper formulates zero-shot detection as deciding whether a candidate passage came from a particular source model without using human-written or generated detection samples. The white-box setting permits log-probability evaluation, while DetectGPT also uses generic perturbation models.
- Zero-shot detection asks whether candidate passage x is a sample from source model pθ without access to human-written or generated samples.
- Algorithm 1 samples k perturbations, estimates their mean source-model log probability, and computes the perturbation discrepancy.
- The normalized discrepancy is compared with a threshold to classify a passage as probably generated by the source model.
- The white-box setting allows scoring candidate text with model log probabilities without requiring access to the model’s architecture or parameters.
- DetectGPT uses off-the-shelf generic pre-trained mask-filling models to generate nearby candidate passages.
4. DetectGPT: Zero-shot Machine-Generated Text Detection with Random Perturbations
DetectGPT measures how source-model log probability changes under meaning-preserving perturbations, interpreting larger drops as evidence of negative curvature around model samples. Experiments show this discrepancy separates generated from human text across models and datasets.
- 4. DetectGPT: Zero-shot Machine-Generated Text Detection with Random Perturbations: DetectGPT defines perturbation discrepancy as the source-model log probability of the original minus the average log probability of nearby perturbations.
- 4. DetectGPT: Zero-shot Machine-Generated Text Detection with Random Perturbations: The gap hypothesis predicts large positive discrepancy for source-model samples and values tending toward zero for human-written text.
- 4. DetectGPT: Zero-shot Machine-Generated Text Detection with Random Perturbations: Using 100 T5-3B perturbations on 500 XSum articles, model samples had significantly larger perturbation discrepancies than human-written articles.
- 4. DetectGPT: Zero-shot Machine-Generated Text Detection with Random Perturbations: Normalizing perturbation discrepancy by the observed standard deviation typically increased AUROC by around 0.020.
- Interpretation of perturbation discrepancy as curvature: The perturbation discrepancy approximates the negative trace of the Hessian of log probability near a candidate passage.
- 5.1. Main Results: DetectGPT consistently provided the most accurate detections from 1.5B-parameter GPT-2 through 20B-parameter GPT-NeoX across the Table 1 model-dataset combinations.
5. Experiments
The experiments evaluate DetectGPT against zero-shot and supervised detectors across domains, models, decoding strategies, revisions, and implementation choices. DetectGPT generally provides the strongest zero-shot detection, while distribution shift, surrogate-model mismatch, passage length, and perturbation count affect performance.
- Zero-shot machine-generated text detection: DetectGPT improves average AUROC by 0.06 and is most accurate for 14 of 15 dataset-model combinations.Its largest average gains are 0.1 AUROC on XSum stories and 0.05 AUROC on SQuAD Wikipedia contexts.
- Comparison with supervised detectors: Supervised detectors match or exceed DetectGPT on in-distribution text but perform significantly worse under domain or language shift.They fail on German writing, whereas zero-shot methods generalize relatively easily and DetectGPT is mostly unaffected by the English-to-German change.
- Detecting paraphrased machine-generated text: DetectGPT maintains AUROC above 0.8 when nearly a quarter of generated text is replaced with T5-3B output, although revision generally degrades detection.DetectGPT shows the strongest detection performance at every revision level in the XSum experiment.
- Decoding strategy: Top-k and nucleus sampling make detection easier on average, with DetectGPT providing the clearest zero-shot signal across domains.The evaluated settings use k = 40 and p = 0.96.
- Source and mask-filling model scale: Detection performance decreases when the surrogate mask-filling model differs from the source model, making DetectGPT best suited to the white-box setting.Larger T5 mask-filling models improve discrimination, supporting estimation of curvature in a latent semantic space.
- Other factors impacting performance of DetectGPT: Detection accuracy improves until 100 perturbations, where it converges, while very long passages may reduce performance because T5 can fail to track many mask tokens simultaneously.Applying perturbations in multiple sequences is identified as a possible way to address the long-sequence issue.
6. Discussion
DetectGPT uses log-probability curvature as a zero-shot signal for identifying model-generated text, motivated by the need to verify content as LLM use expands. Its applicability is bounded by model access, perturbation quality, computational cost, and passage-length effects.
- Discussion: The method uses a tractable approximation to the log-probability Hessian trace as a discriminative signal for model samples.Experiments find this signal more discriminative than existing zero-shot methods and competitive with bespoke detectors trained on millions of samples.
- Discussion: DetectGPT can be viewed as detecting an implicit watermark: model-generated text favors specific phrasing whose semantically similar rephrasings receive lower probability.The paper suggests manually adding watermarking biases may further improve DetectGPT’s effectiveness.
- Limitations: Probability-based detection assumes access to model log probabilities, which may be unavailable or costly for models exposed through APIs.DetectGPT also depends on perturbation models that represent meaningful rephrases well; poor perturbations can reduce curvature-estimation quality.
- Limitations: Detection quality versus passage length depends on the dataset and model or tokenizer, with longer passages potentially challenging T5’s ability to fill many masks simultaneously.Sequentially applying mask-fills in smaller batches may mitigate this issue.
- Future Work: Future work includes combining watermarking with detection, using model ensembles in black-box settings, studying prompting effects, and testing curvature beyond text.The proposed extensions target robustness, black-box detection, evasion, and other generative media domains.
A. Complete Results for Top-p and Top-k Decoding
Top-p and top-k sampling generally make detection easier, and DetectGPT usually achieves the highest AUROC across the evaluated methods. However, its advantage narrows for top-k generations, while LogRank matches it for WritingPrompts.
- Sampling effects: Top-p and top-k sampling appear to make detection easier because they increase the average log likelihood of model generations by truncating low-probability tokens.The passage reports this pattern across the complete results for XSum, SQuAD, and WritingPrompts.
- Top-p sampling: Top-p sampling generally makes detection easier for all methods, while DetectGPT still provides the highest average AUROC.For WritingPrompts, the LogRank baseline performs as well as DetectGPT.
- Top-k sampling: Top-k sampling generally gives DetectGPT the highest AUROC, although the performance gap narrows relative to direct sampling.The narrower gap is attributed to top-k generations being more generic.