Source-linked AI summary
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein
TL;DR
Detecting machine-generated text is difficult because prompts can make LLM responses appear either unsurprising or unexpectedly surprising to a detector. Binoculars contrasts two language models through perplexity and cross-perplexity, enabling zero-shot detection across multiple LLMs. It detects GPT-3 and GPT-4 samples with 92% and 89.57% accuracy, respectively, on Open Orca.
Problem
Prompt-dependent responses can produce misleading perplexity signals, while model-specific detectors often fail to transfer to new LLMs.
Method
Binoculars contrasts an observer model’s text perplexity with the observer-assessed perplexity of next-token predictions from a performer model.
Results
92% of GPT-3 samples and 89.57% of GPT-4 samples were detected using a global threshold on the Open Orca dataset.
Takeaways & Limitations
A single zero-shot detector can spot multiple LLMs with high accuracy without model-specific training data, supporting applications such as platform moderation.
Takeaways & Limitations
The study focuses on machine-generated text in normal use, does not test explicit efforts to bypass detection, and does not investigate source code or other non-conversational domains.
Abstract
from arXiv · showhide
Detecting text generated by modern large language models is thought to be hard, as both LLMs and humans can exhibit a wide range of complex behaviors. However, we find that a score based on contrasting two closely related language models is highly accurate at separating human-generated and machine-generated text. Based on this mechanism, we propose a novel LLM detector that only requires simple calculations using a pair of pre-trained LLMs. The method, called Binoculars, achieves state-of-the-art accuracy without any training data. It is capable of spotting machine text from a range of modern LLMs without any model-specific modifications. We comprehensively evaluate Binoculars on a number of text sources and in varied situations. Over a wide range of document types, Binoculars detects over 90% of generated samples from ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being trained on any ChatGPT data.
1. Introduction
Binoculars detects LLM-generated text without source-model training examples, using two language models to contrast perplexity and cross-perplexity. The method performs strongly across models and is motivated by the need for model-agnostic detection.
- Binoculars detects LLM-generated text in a zero-shot setting without training examples from the generation model.
- The detector can identify multiple different LLMs with high accuracy, whereas existing solutions fail to transfer across models.
- Model-agnostic zero-shot detection is important for moderating bots, fake reviews, and misinformation produced through diverse LLM APIs.
- Binoculars computes text perplexity with an observer model and cross-perplexity for performer-model next-token predictions, using their ratio as a detection signal.
2. The LLM Detection Landscape
Modern transformer-based language models have made primitive machine-text detectors ineffective, motivating post-hoc approaches that do not require cooperation from model owners.
- Transformer language models have made primitive mechanisms for detecting machine-generated text ineffective.
TPR @ 0.01% FPR
Detection evaluation emphasizes performance at an extremely low false positive rate, because false alarms across diverse human-written text are consequential. Existing approaches include trained classifiers and statistical signatures, while theoretical work identifies limits on general-purpose language models.
- TPR @ 0.01% FPR: 0.01% false positive rate is the operating point used to assess detection performance in high-stakes settings.The supplied passages motivate low false positive rates but do not provide a complete result table for this section.
- Detection approaches: Trained detection models fine-tune pretrained language-model backbones for binary classification, including adversarial-training and abstention techniques.
- Detection approaches: Statistical-signature detectors use features such as perplexity, perplexity curvature, log rank, intrinsic dimensionality, and n-gram analysis.
- Detection limits: Theoretical analyses generally hold that fully general-purpose language models would be impossible to detect, although near-optimal models may remain detectable with enough samples.
- Evaluation: Balanced-set accuracy and AUC can be poorly suited to detection, which requires low false positive rates across broad distributions of human-written text.
- Evaluation: The paper compares Binoculars with Ghostbuster, GPTZero, DetectGPT, Fast-DetectGPT, and DNA-GPT in post-hoc, out-of-domain, black-box settings.
3. Binoculars: How it works
Binoculars detects machine-generated text by contrasting perplexity and cross-perplexity from two language models, addressing cases where raw perplexity fails because prompts are unknown.
- Core mechanism: Binoculars contrasts two language models by comparing a text’s perplexity with cross-perplexity.The observer measures the text’s log perplexity, while the performer’s next-token predictions are evaluated according to the observer.
- Background and notation: A string is tokenized into token indices, and log-perplexity is computed as the average negative log-likelihood across its tokens.The token sequence length is L, and each token corresponds to an entry in the language model vocabulary.
- Detection challenge: Raw perplexity alone can misclassify prompted machine text because unusual content may appear surprising without its generating prompt.The paper calls this failure mode the “capybara problem.”
- Assumption: The two-model formulation requires the observer and performer to share a tokenizer.
- Illustrative example: 0.73 correctly classifies prompted GPT-4 text as machine-generated despite Falcon assigning it perplexity of 2.20.The Binoculars score is below the global threshold of 0.901, whereas DetectGPT misclassifies the sample as human.
- Detection score: The Binoculars score uses perplexity divided by cross-perplexity to normalize how surprising text and model predictions are to the observer.This compares how much a human-authored string diverges from the observer with how much the performer diverges from it.
4. Accurate Zero-Shot Detection
The experiments evaluate Binoculars as a zero-shot detector across multiple domains and language-model sources, emphasizing performance at a low false-positive rate. Results compare it with existing detectors and examine performance as document size varies.
- Experimental setting: Binoculars is evaluated as a zero-shot detector on machine-generated text from modern LLMs across multiple domains.The experiments focus on common-use generation without considering the detection mechanism.
- Evaluation criteria: TPR at 0.01% FPR is prioritized because false positives incorrectly labeling human text are especially concerning.The paper argues that AUC and F1 alone are inadequate for evaluating low-FPR detection performance.
- Benchmark comparisons: Binoculars outperforms Ghostbuster in the out-of-domain setting on three balanced datasets containing human and ChatGPT samples.The comparison uses the Writing Prompts, News, and Student Essay datasets and evaluates TPR at 0.01% FPR.
- Document size: Figure 2 shows that Binoculars and Ghostbuster improve with more information, while Binoculars’ advantage is clearer in the few-token regime.Document size is varied by prefixing sample documents, with TPR at 0.01% FPR plotted against observed token count.
- Open-source language models: Binoculars detects outputs from LLaMA and Falcon, whereas Ghostbuster reliably detects ChatGPT but fails to reliably detect LLaMA-generated text.Figure 3 compares the methods across thresholds using detailed ROC plots.
5. Reliability in the Wild
Binoculars remains effective across domains, languages, memorization cases, and prompting variations, but low-resource multilingual detection trades recall for precision and memorized human text can be classified as machine-generated.
- Other languages: Binoculars generalizes across domains and languages, including Urdu, Russian, Bulgarian, and Arabic, using ChatGPT-generated samples from varied sources.The M4 datasets include Arxiv, Reddit, Wikihow, and Wikipedia samples.
- Other languages: Low-resource languages retain low false-positive rates but often produce poor recall because machine text is classified as human.The authors hypothesize that stronger multilingual models could improve detection in these languages.
- Memorization: Memorized human-written texts can receive machine-like scores, with 3 of 11 famous texts classified on the machine side of the threshold.The US Constitution received a Binoculars score of 0.76, the lowest among the 11 texts examined.
- Detection comparisons: For LLaMA-2-13B generations, Binoculars achieves higher true-positive rates at low false-positive rates than other methods.The comparison is shown across thresholds with false-positive rate on a logarithmic scale.
- Other generative models: Binoculars detects 92% of GPT-3 samples and 89.57% of GPT-4 samples on the Open Orca dataset using the global threshold.The dataset covers a wide range of tasks and contains machine-generated completions from both models.
- Modified Prompting Strategies: Stylistic system-prompt changes generally do not significantly affect accuracy, while pirate-style prompting decreases sensitivity by 1%.The tested prompts requested Carl Sagan’s voice, non-robotic wording, or pirate-like language.
6. Discussion and Limitations
Binoculars detects LLM output in a zero-shot setting and can transfer across multiple LLMs. The study is limited to smaller open-source models, naturally occurring machine text, and selected domains.
- Binoculars detects LLM output without data from the generation model and can spot multiple different LLMs with one detector.
- The evaluation does not broadly study larger open-source models of 30B parameters or more because of limited GPU memory.
- The study focuses on machine-generated text in normal use and does not consider explicit efforts to bypass detection.
- Source code and other non-conversational text domains are outside the study’s scope.
Impact Statement
The paper frames LLM detection as potentially useful for reducing harm across platforms, training data, and chat applications. It also emphasizes that reliability analysis is only an initial deployment step and that detectors may not be worthwhile in every scenario.
- LLM detection may support monitoring machine-generated text, filtering training data, and identifying chat-application responses.
- The paper cautions that detection mechanisms should reduce harm rather than proliferate or increase it.
- Reliability analysis, including consideration of text written by non-native speakers, is presented as only a first step toward deployment.
A.1.1. DATASET GENERATION
The experiments generate machine-text datasets from human prompts across several domains, compare Binoculars with established detectors, and examine reliability, scoring components, model choices, and prompting conditions.
- Dataset generation: Machine-text datasets use CC News, CNN, and PubMed samples, with LLaMA-2-13B and Falcon-7B generating up to 512 tokens after 50-token prompts are removed.
- Evaluation setup: Evaluation uses ultra-low TRP@ 0.01%FPR alongside threshold-tuned F1, Recall, Precision, and False Negative Rate.
- Evaluation setup: Thresholds are globally optimized for accuracy using reference datasets, while out-of-domain Ghostbuster evaluations tune thresholds only on CC News, CNN, and PubMed generations.
- Baselines: Comparisons cover Ghostbuster, GPTZero, DetectGPT, Fast-DetectGPT, and DNA-GPT using out-of-domain baselines where applicable.
- Score components: Perplexity and cross-perplexity alone are poor detectors, whereas Binoculars performs well at low false-positive rates.
- Performer model ablation: Falcon-7B-Instruct as the performer model achieves the best detection performance, with gains increasing nearly monotonically with instruction fine-tuning.