Source-linked AI summary
GLTR: Statistical Detection and Visualization of Generated Text
Sebastian Gehrmann, Hendrik Strobelt, Alexander M. Rush
TL;DR
The paper addresses the need for accessible methods to detect increasingly convincing generated text and presents GLTR, a visual tool based on statistical generation artifacts. In a human-subjects study, the interface improved fake-text detection from 54% to 72% without prior training, while its scope depends on biased sampling assumptions and conditioning effects.
Problem
Increasingly capable language models can generate human-indistinguishable text, motivating accurate detection methods that non-experts can understand and use.
Method
GLTR combines word probability, rank, and entropy tests with visual token overlays to assist per-instance detection of generated text.
Results
54% without GLTR versus 72% with GLTR: the interface improved human fake-text detection without prior training.
Takeaways & Limitations
Simple statistical properties can be presented visually to assist human readers and improve their ability to detect fake text.
Takeaways & Limitations
GLTR assumes biased sampling, while hidden seed conditioning can change the conditional distribution and requires further evaluation.
Abstract
from arXiv · showhide
The rapid improvement of language models has raised the specter of abuse of text generation systems. This progress motivates the development of simple methods for detecting generated text that can be used by and explained to non-experts. We develop GLTR, a tool to support humans in detecting whether a text was generated by a model. GLTR applies a suite of baseline statistical methods that can detect generation artifacts across common sampling schemes. In a human-subjects study, we show that the annotation scheme provided by GLTR improves the human detection-rate of fake text from 54% to 72% without any prior training. GLTR is open-source and publicly deployed, and has already been widely used to detect generated outputs
1 Introduction
Large language models can produce human-indistinguishable text, lowering barriers to abuse and creating a need for accurate, accessible detection. GLTR combines simple statistical detection methods with visual annotations to help non-experts identify generated text.
- Large language models can generate text that appears human-written to non-experts, increasing the potential for abuse.
- Accurate, easy-to-explain forensic techniques are needed to detect automatically generated comments, articles, and reviews.
- GLTR applies statistical metrics in a visual tool that highlights text passages for generation detection.
- GLTR is publicly deployed and open-source, with its tool and code made available online.
- 54% without the tool versus over 72% with GLTR: participants detected fake text more accurately with the tool and no prior training.
2 Method
The method detects generated text through distributional properties rather than supervision, assuming generation systems favor high-confidence regions of a language-model distribution. It uses word probability, rank, and prediction entropy as complementary tests.
- The detection task classifies a word sequence as human-written or model-generated without task-specific supervision.
- The method assumes natural-looking generation samples disproportionately from the head of the language distribution.
- Test 1 measures the probability assigned to each observed word by a detection model.
- Test 2 measures each observed word’s absolute rank in the predicted distribution.
- Test 3 measures prediction entropy to assess whether the preceding context produces an overly certain next-word prediction.
3 GLTR: Visualizing Outliers
GLTR visualizes probability, rank, and entropy signals through token overlays, graphs, and tooltips, enabling per-instance inspection of generated text. Case studies and cross-validated experiments show that generated and human text exhibit different distributional patterns.
- GLTR: Visualizing Outliers: GLTR applies three statistical tests to any textual input and presents their evidence through visual analysis.
- GLTR: Visualizing Outliers: The backend supports BERT and GPT-2 117M detection models, using left-to-right or bidirectional context to estimate token probabilities.
- GLTR: Visualizing Outliers: The overlay colors tokens by rank: top 10 green, top 100 yellow, top 1,000 red, and remaining tokens purple.
- GLTR: Visualizing Outliers: The interface combines token heatmaps with three global graphs and hover tooltips showing predictions, probabilities, rank, and entropy.
- GLTR: Visualizing Outliers: Generated case-study text is dominated by green and yellow tokens, while human NYT and scientific text contains more red and purple tokens.
- GLTR: Visualizing Outliers: Distributional information yields more informative fake-text discrimination than bag-of-words features in logistic regression.
4 Empirical Validation
Across generated and human-written sources, GLTR’s simple rank- and entropy-based features separate real from generated text. Human text more often uses tail words, including in low-entropy contexts.
- GLTR features separate real and generated text better than word features, both with and without access to the true generating model.
- Real text more frequently uses words from the tail of GPT-2’s predicted distribution than generated text.
- 2.41 times as frequently under GPT-2, real texts use words outside the top 100 predictions compared with generated text; the corresponding BERT ratio is 1.67.
- Human-written text is more likely than GPT-2 text to use high-rank words even when contextual entropy is low.
5 Human-Subjects Study
In a study with 35 students, GLTR’s overlay substantially improved detection of generated text over unaided judgments. Participants also reported that the tool helped accuracy and speed and drew attention to generation artifacts.
- The study presented students with generated and human texts in two rounds, first without and then with the overlay.
- 72.3% accuracy with the interface improved on 54.2% without it, an 18.1% average treatment effect with p < 0.001.
- Qualitative Findings: Students noticed limited synonym and referring-expression variation in model text and high sentence-structure parallelism in Heliograf samples.
6 Related Work
GLTR builds on earlier statistical approaches to generated-text detection, but argues that increased language-model power has changed the relevant empirical patterns.
- Earlier work used prediction entropy as an indicator of fake text, whereas GLTR reports low entropy for generated text.
- The paper situates GLTR among prior methods using language-model perplexity and rare-bigram frequencies to distinguish texts.
7 Discussion and Conclusion
The paper applies simple statistical properties of language-model predictions to build GLTR, a tool that assists readers in detecting automatically generated text. It reports public educational impact while identifying biased sampling and hidden seed conditioning as important boundaries.
- GLTR uses simple statistical properties from detection models to assist human readers in identifying automatically generated text.
- Impact: Within its first month, the public GLTR demo received 30,000 page views and its blog received 21,000.
- Future Work: GLTR assumes that generation systems use biased sampling, while adversarial tail sampling could fool the overlay but reduce text coherence.
- Future Work: Hidden seed conditioning changes the conditional distribution, motivating more detailed evaluation of how conditions affect GLTR.