Real or Fake? Learning to Discriminate Machine from Human Generated Text

Anton Bakhtin; Sam Gross; Myle Ott; Yuntian Deng; Marc'Aurelio Ranzato; Arthur Szlam

Source-linked AI summary

Real or Fake? Learning to Discriminate Machine from Human Generated Text

Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc'Aurelio Ranzato, Arthur Szlam

arXiv:1906.03351v2Jun 7, 2019cs.LGcs.CLstat.ML

TL;DR

Applying EBMs to text is difficult because discrete sequences make negative-sample mining challenging. The paper uses pre-trained auto-regressive language models to generate negatives and trains residual EBMs to discriminate human from machine-generated text. These EBMs generalize robustly across generator architectures but are more sensitive to the generators’ training datasets.

Problem
EBMs had not been successfully applied to text because mining informative negatives is difficult for high-dimensional discrete sequences.
Method
The paper generates negative text with pre-trained auto-regressive language models and trains a residual EBM to distinguish it from human text.
Results
Residual EBMs are remarkably robust to changes in generator architecture but sensitive to the training dataset of the test generator.
Takeaways & Limitations
Residual EBMs provide a studied approach to real-versus-machine text discrimination and support investigation of EBM generalization for text.
Takeaways & Limitations
The energy function is not robust to truly out-of-domain samples, including randomly generated text that it may score lower than real text.

Abstract

from arXiv · show

Energy-based models (EBMs), a.k.a. un-normalized models, have had recent successes in continuous spaces. However, they have not been successfully applied to model text sequences. While decreasing the energy at training samples is straightforward, mining (negative) samples where the energy should be increased is difficult. In part, this is because standard gradient-based methods are not readily applicable when the input is high-dimensional and discrete. Here, we side-step this issue by generating negatives using pre-trained auto-regressive language models. The EBM then works in the residual of the language model; and is trained to discriminate real text from text generated by the auto-regressive models. We investigate the generalization ability of residual EBMs, a pre-requisite for using them in other applications. We extensively analyze generalization for the task of classifying whether an input is machine or human generated, a natural task given the training loss and how we mine negatives. Overall, we observe that EBMs can generalize remarkably well to changes in the architecture of the generators producing negatives. However, EBMs exhibit more sensitivity to the training set used by such generators.

1 INTRODUCTION

This work addresses the difficulty of applying EBMs to discrete text by using pre-trained language models to generate negative samples. It studies whether the resulting residual EBMs generalize when detecting human versus machine-generated text.

Text EBMs are difficult to train because discrete inputs prevent gradient-based negative mining, while alternative methods may be inefficient at scale.
The proposed EBM learns in the residual space of a pre-trained auto-regressive language model, which generates negative text samples.The EBM can use binary cross-entropy or ranking loss to assign lower energy to human text than to generated text.
The central question is whether residual EBMs generalize well, which is required for using them to model text and corresponds to discriminating real from machine-generated text.
The study finds that residual EBMs are remarkably robust to generator-architecture changes but sensitive to the training dataset of the test generator.Performance also improves with larger generators and longer generations.

2 RELATED WORK

The paper distinguishes its pre-trained-generator discriminator from prior reranking, adversarial, and machine-text-detection work. Its main related-work contribution is a broader, quantitative study of generalization across generator architectures and corpora.

Unlike adversarial models, this approach trains the generator beforehand and learns the discriminator afterward from the generator’s residual errors.
Earlier reranking methods used separately trained scoring functions to improve weak generators with hand-designed features, whereas this work uses language-model-generated negatives.
Prior machine-text detectors generally assumed knowledge of the generator and did not study differing generator architectures and corpora between training and testing.
The paper provides a rigorous experimental protocol and quantitative results for the cross-generator and cross-corpus question raised by earlier detection work.
Compared with released GPT-2-generation detection data, this work evaluates several discriminator architectures across multiple generators and training corpora.

3 ENERGY-BASED MODELS FOR TEXT

The method trains a scalar energy function to score human text below generated text, using language models to mine negatives. It considers binary cross-entropy and ranking losses and generates context-conditioned prefixes or suffixes.

3.1 LEARNING: The energy function assigns a real-valued compatibility score to a token sequence conditioned on context and model parameters.
3.1 LEARNING: Training aims to give human-generated sequences lower energy than other sequences.
3.1 LEARNING: The study considers binary cross-entropy and ranking losses, whose findings are reported as similar.
3.1 LEARNING: Negative samples share the positive example’s context and contain at least some machine-generated tokens.
3.1 LEARNING: The loss uses the most offending negative, defined as the candidate with the lowest energy under the model.
3.2 GENERATING NEGATIVES: Pre-trained auto-regressive language models generate negatives instead of using continuous-space methods such as MCMC or Langevin dynamics.
3.2 GENERATING NEGATIVES: Left-to-right models generate suffixes from prefixes, while right-to-left models generate prefixes from suffixes using top-k sampling with k=10.

4 EXPERIMENTAL SETUP

The experiments use three heterogeneous corpora, multiple generator and energy-function architectures, and four generalization settings. Evaluation reports balanced average classification accuracy for distinguishing positive from negative text.

Datasets: The corpora span fiction books, factual Wikipedia text, and large-scale CommonCrawl news, enabling tests across data size, style, and domain relatedness.
Preprocessing: Positive sequences are extracted from 160-token windows, with corpus-dependent strides and a 60,000-sample test limit.
Architectures: Negative generators include small, big, and huge transformers, a convolutional model, and GPT-2 models.
Architectures: Unless otherwise specified, examples contain 20 or 40 generated tokens conditioned on 140 or 120 context tokens, respectively.
Architectures: Energy functions include Linear, BiLSTM, bidirectional Transformer, and causal unidirectional Transformer architectures.
Evaluation settings: The study evaluates in-domain, cross-architecture, cross-corpus, and unseen generalization settings.
Evaluation settings: These settings vary the training and testing corpora and generator architectures, with different training seeds producing distinct generators even when architectures match.
Evaluation settings: Performance is measured using average classification accuracy balanced across positive and negative classes.

5 RESULTS

Residual EBMs discriminate human from machine-generated text and generalize across generator architectures, but performance is more sensitive to training corpora and out-of-domain text. Larger models, more diverse training data, longer generations, and harder negatives generally improve discrimination.

In-domain generalization: More than 90% accuracy was reached on Books, while accuracy exceeded 88% on the more challenging CCNews dataset.These in-domain results used TransfBig generators and energy models with comparable representational power.
Cross-architecture generalization: 87.9% accuracy on TransfSmall negatives versus 92.9% on Conv negatives shows architecture changes can affect cross-generator discrimination.Training with TransfSmall negatives was more robust to Conv generations, with only a 1.4% accuracy drop in the reverse comparison.
Cross-corpus generalization: 59.1% and 65.5% accuracy on Wikitext followed training on Books and CCNews, respectively, demonstrating weaker cross-corpus generalization.Training on the union of two corpora substantially improved testing on the third, while training on all three produced near in-domain robustness.
Cross-corpus generalization: More than 5% higher CC-News accuracy was obtained by the 355M-parameter BiTransf when trained on all corpora.This supports the reported pattern that larger energy models trained on more data achieve better discrimination.
Generalization in the wild: In-domain BiTransf reached almost 100% across datasets, but wild-setting performance lagged behind the in-domain TF-IDF baseline.Cross-architecture performance exceeded TF-IDF only when the test generator was less than three times larger than the training generator.
Ablation study: Zero-prefix unconditional discrimination was easiest, whereas prefixes of 120 and 140 tokens were hardest; increasing prefix length made discrimination harder.Using more negatives and harder negatives also improved accuracy, while truly out-of-domain random text could receive lower energy than real text.

6 FINAL REMARKS

The paper proposes using pre-trained language-model generations as negative samples for text EBMs and studies their generalization. EBMs generalize well across generator architectures, less well across generator corpora, and may improve with larger, more diverse training data.

Pre-trained language-model generations provide a practical source of negative samples for training text EBMs.The EBM learns in the residual space of the language model rather than generating negatives with its own energy function.
The study finds good generalization when EBMs are tested on negatives from generators with different architectures.
Generalization is weaker for generators trained on different corpora but improves after training on larger composite datasets.
Larger EBM architectures and more diverse, larger training datasets are proposed as future improvement directions, although scaling poses engineering challenges.
Real/fake text discrimination is a natural application, and the EBM outperforms the original language-model log-likelihood for this task.

A CORPORA SIZES

This appendix reports the number of BPE tokens in millions for each dataset.

The dataset-size table records the number of BPE tokens in millions for each dataset.

B MODEL SIZES

This appendix documents parameter counts for the generator language models and scoring functions, along with related implementation notes.

The generator-model table reports parameter counts in millions.
Computational cost is tied directly to parameters outside the input embedding layer for both generators and scoring functions.
The appendix notes that HuggingFace model sizes are reported as used for data generation, while the OpenAI GPT-2 repository uses different size labels.
The scoring-function table also reports parameter counts in millions.

C RANKING LOSS

The ranking loss compares the energies of paired positive and negative sequences, making the score context-dependent rather than globally calibrated.

The per-sample ranking loss operates on the energy difference between a positive sequence and a negative sequence.
Ranking loss produces local energy values because each comparison uses positive and negative sequences sharing the same context.
Unlike ranking loss, binary cross-entropy encourages negative energy for positives and positive energy for negatives independently of context.
The authors report similar empirical findings for ranking loss and binary cross-entropy.
Ranking-loss evaluation uses precision at 1, the fraction of test sequences for which the ground truth has the lowest energy among its negatives.

D HYPER-PARAMETER SETTING

The scoring functions use PyTorch and Adam; the largest UniTransf and BiTransf models are trained synchronously across multiple GPU-equipped machines.

All models are implemented in PyTorch and optimized with Adam.
The largest UniTransf and BiTransf models use 8 machines with 8 GPUs each in synchronous data-parallel training.
Training combines large batches, float16 reduced precision, and cosine learning-rate scheduling without restarts.
Table 11 lists the hyper-parameter values used in the scoring functions.

E SCORE DISTRIBUTIONS

Figure 3 compares score differences between ground-truth and generated completions for two Wikitext generation directions, with 40 generated tokens in each case.

Figure 3 shows distributions of negative-energy score differences between ground-truth completions and generated completions.
The figure reports separate results for left-to-right and right-to-left generations from Wikitext.
Each generation produces 40 tokens, and examples right of the red margin = 0.1 line have zero ranking loss.

F PERTURBING THE ENERGY FUNCTION

The section demonstrates gradient-based token perturbations of the energy function: editing a few words can reverse whether examples are scored as real or negative, especially outside the generator’s observed distribution.

Method: The worked example uses a 160-BPE-token block, with 120 context tokens and a 40-token ground-truth completion.
Method: A language model generates 10 negative completions from the context for comparison with the ground-truth completion.
Method: UniTransf separates real from fake examples, and token replacements are selected using the energy gradient and a first-order Taylor approximation.
Results: The same procedure can change a positive sample into a negative one by replacing a few words.
Results: The discriminator can be fooled by editing a few words, but the edited sentences have low generator probability and were absent from training negatives.