Source-linked AI summary
Truncation Sampling as Language Model Desmoothing
John Hewitt, Christopher D. Manning, Percy Liang
TL;DR
Long neural-language-model samples can be poor because smoothing assigns probability beyond the true support, while truncation criteria may remove plausible words. The paper frames truncation as desmoothing, proposes entropy-dependent η-sampling, and reports more plausible long English documents and more reasonable behavior than prior methods.
Problem
Long neural language-model samples can be nonsensical, and existing truncation methods may remove high-probability words while deciding which support to retain.
Method
The paper models neural language-model distributions as mixtures of the true distribution and smoothing, then derives η-sampling with absolute and entropy-dependent probability thresholds.
Results
η-sampling generates more plausible long English documents, breaks out of repetition more effectively, and behaves more reasonably on test distributions than previous algorithms.
Takeaways & Limitations
Truncation can be understood as desmoothing: estimating which words belong to the true distribution’s support rather than merely retaining a fixed probability mass.
Takeaways & Limitations
The evaluations test only English language models, while languages with rich morphology may have different conditional-distribution properties, especially with subword models.
Abstract
from arXiv · showhide
Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top-$p$ or top-$k$ -- address this by setting some words' probabilities to zero at each step. This work provides framing for the aim of truncation, and an improved algorithm for that aim. We propose thinking of a neural language model as a mixture of a true distribution and a smoothing distribution that avoids infinite perplexity. In this light, truncation algorithms aim to perform desmoothing, estimating a subset of the support of the true distribution. Finding a good subset is crucial: we show that top-$p$ unnecessarily truncates high-probability words, for example causing it to truncate all words but Trump for a document that starts with Donald. We introduce $η$-sampling, which truncates words below an entropy-dependent probability threshold. Compared to previous algorithms, $η$-sampling generates more plausible long English documents according to humans, is better at breaking out of repetition, and behaves more reasonably on a battery of test distributions.
1 Introduction
Neural language models can produce nonsensical long samples, motivating truncation methods that remove low-probability words. This paper frames truncation as desmoothing and introduces η-sampling to better approximate the true support.
- Neural language models can generate nonsensical output despite progress in coherence and length, especially for long-form generation.
- Truncation sampling improves sample quality by setting selected word probabilities to zero, but its truncation criterion remains the central design question.
- The paper models a neural language model as a mixture of the true distribution and uniform-like smoothing, viewing truncation as approximate recovery of the true support.
- Top-p can truncate high-probability words: with p = 0.95, a next-most-likely word with probability 4% may be excluded after a few words consume 95%.
- η-sampling truncates words below both an absolute and entropy-dependent threshold, allowing multiple continuations where top-p may retain only Trump after Donald.
- Across evaluations, η-sampling truncates more reasonably on a battery of test distributions than top-p and typical decoding.
2 Background
Language models factorize sequence probabilities into next-token conditional distributions and are trained to approximate the true distribution. Truncation sampling reallocates probability toward likely words by restricting each conditional distribution to an allowed set.
- An autoregressive language model represents a sequence distribution as a product of conditional next-token distributions given prior context.
- Language models are trained to minimize KL-divergence to an empirical approximation of the true distribution.
- Generating high-variety, high-quality long samples requires reallocating probability from the tails of conditional distributions toward their heads.
- At each step, truncation sampling retains an allowed set of words, renormalizes their probabilities, and assigns zero probability to all others.
- Top-k keeps the k most likely words, whereas top-p keeps the smallest set containing a specified probability mass; other methods adapt truncation differently.
3 Truncation as Desmoothing
The paper models neural language models as smoothed versions of a true distribution and frames truncation as recovering the true support. This framing yields absolute and relative probability principles for avoiding both implausible continuations and unnecessary loss of diversity.
- 3.1 Motivation: Truncation aims to approximate the true distribution’s support by removing probability mass introduced through smoothing.The framework treats smoothing as necessary for finite KL divergence and perplexity, but undesirable during generation when it assigns mass to unsupported words.
- 3.2 A neural LM as a smoothed distribution: The framework assumes smoothing combines the true conditional distribution with a near-uniform distribution whose strength depends on the context.The paper notes that the assumption linking high entropy to better estimation is its most tenuous assumption.
- 3.3 A local measure of truncation quality: The support-weighted objective balances losing true-distribution mass against retaining probability on unsupported words.The two terms are weighted by βvar and βsup, respectively.
- 3.4 Principles for truncation: The absolute probability principle says that sufficiently high-probability words should not be truncated because unsupported words have low probability bounds.This principle protects plausible words from removal based solely on other words’ cumulative probability.
- 3.4 Principles for truncation: The relative probability principle says that truncation should target words whose probabilities are low relative to the rest of a high-entropy distribution.The entropy-dependent bound reflects lower smoothing in high-entropy distributions.
- 3.1 Motivation: Uniform smoothing can make n-gram generation degenerate because an unsupported sampled word creates an unseen history with poorly estimated future probabilities.This provides a concrete illustration of why desmoothing matters for generation quality.
4 Methods
The methods section analyzes existing truncation rules through the absolute and relative probability principles, then introduces η-sampling to satisfy both. η-sampling combines an absolute threshold with an entropy-dependent threshold.
- 4 Methods: The methods section positions η-sampling alongside analyses of top-p, typical decoding, and ϵ-sampling as an approach to desmoothing.The proposed method is introduced after identifying failures of existing criteria relative to the two probability principles.
- 4.2 Typical decoding: Typical decoding can truncate the most probable words while also breaking the absolute probability principle.It orders words by distance from the distribution’s entropy-based informativeness level before covering p percent of probability.
- 4.3 ϵ-sampling (ours): ϵ-sampling preserves words above an absolute probability threshold but fails when many continuations are plausible.For GPT-2 on “The,” over ten thousand words are allowed by the model, requiring an impractically small ϵ to retain them all.
- 4.4 η-sampling (ours): η-sampling retains words only when their probabilities exceed both an absolute threshold and an entropy-dependent threshold.The entropy-dependent threshold is α exp(−hθ,x<i), and the paper sets α = √ϵ empirically.
5 Experiments & Results
Experiments compare truncation methods using MAUVE, human judgments, entropy-based tests, and repetition stress tests. η-sampling generally behaves more reasonably, though ε-sampling achieves the lowest repetition rate in the reported adversarial setting.
- η-sampling behaves better across long-document generation and defensible truncation of low-entropy distributions.
- Overall MAUVE performance was similar across methods, with typical decoding slightly worse than top-p and the authors’ methods.
- Human evaluators preferred η-sampling over top-p 53% to 40% in Study 1, and the preference was significant in Study 2.
- Top-p heavily truncates low-entropy distributions, ε-sampling heavily truncates high-entropy distributions, and η-sampling balances both tendencies.
- In adversarial repetition tests, ε-sampling had a 23% repetition rate, η-sampling 26%, and top-p 47% for GPT-2 large.
- On CheckList-style prompts, top-p sometimes permits only one continuation, whereas ε- and η-sampling allow more alternatives on low- and high-entropy distributions.
6 Related Work
Related work situates truncation sampling among stochastic decoding, distribution-support and smoothing approaches, and methods for evaluating high-entropy language generation.
- Prior stochastic decoding methods include ancestral sampling and approaches that seek high-likelihood sequences, such as stochastic beam search.
- Related language-modeling work studies sparse probabilities, loss truncation, and structural zeros to avoid assigning mass uniformly or penalizing rare events.
- Evaluating high-entropy generation requires considering both sample quality and diversity because quality is difficult to measure and may not track model probability.
7 Conclusion
The paper frames truncation sampling as desmoothing: recovering words supported by the true distribution from a smoothed language-model distribution. This framing yields principles, η-sampling, and evaluations aimed at improving neural text generation.
- 7 Conclusion: Truncation algorithms are framed as desmoothing methods that approximate the training distribution by recovering words supported by the true distribution.The authors present this framing as the basis for principles governing truncation, a new algorithm, and evaluations of existing methods.
- 7 Conclusion: The resulting insights are intended to support further research into understanding and improving generation from neural language models.
8 Limitations
The authors cannot fully characterize the sequence-level effects of truncation sampling, including which sequences, language types, or sources may be excluded. Their experiments also cover only English language models, leaving languages with richer morphology less examined.
- 8 Limitations: The authors find it difficult to understand all sequence-level effects of truncation, including which sequences or sources of language algorithms may disallow.
- 8 Limitations: The algorithms were tested only on English language models, while morphologically rich languages may have different conditional-distribution properties.The authors particularly flag potential differences when subword models are used.
9 Ethics Statement
The paper highlights ethical risks in text generation, including hate speech, misinformation, and the possible removal of rare topics or names. It also argues that decoding choices can affect measured fairness and that training distributions require careful selection.
- 9 Ethics Statement: Truncation methods may remove models’ ability to discuss topics or names that are already rare, creating a particular risk of harm.
- 9 Ethics Statement: The choice of stochastic decoding algorithm affects measured fairness metrics in open-ended generation.
- 9 Ethics Statement: Text-generation systems raise ethical concerns involving negative uses such as hate speech and misinformation.
- 9 Ethics Statement: Recovering a distribution close to the training distribution requires the training distribution itself to be chosen with care.
A.2 Analysis of η-sampling
Under the smoothing model, η-sampling approximates truncation that avoids words outside the true support while minimizing unnecessary truncation. The analysis shows why threshold misspecification can either allow unsupported words or remove supported ones, and notes practical approximations used by η-sampling.
- A.2 Analysis of η-sampling: η-sampling approximates an algorithm that avoids sampling outside the true support while minimally truncating the distribution.
- A.2 Analysis of η-sampling: Setting the threshold below the model-derived value can incorrectly allow words outside the true support.
- A.2 Analysis of η-sampling: Setting the threshold above the model-derived value can unnecessarily truncate words that belong to the true support.
- A.2 Analysis of η-sampling: The model-derived threshold guarantees support-consistent sampling without excessive truncation under the smoothing assumptions.
- A.2 Analysis of η-sampling: In practice, η-sampling searches task-specific hyperparameters and substitutes observed language-model entropy for the unknown true-distribution entropy.
B More Experimental Details
The appendix documents experimental implementation and human-study procedures, including hyperparameter selection, a smoothed n-gram demonstration, randomized presentation, worker participation, and prompt filtering.
- MAUVE-maximizing hyperparameters for each truncation algorithm and model are reported in Table 5.
- A small demonstration trains a 5-gram model on 10,000 The Pile documents and smooths it with the uniform distribution.
- Human-study interfaces are provided for transparency, with method generations presented in randomized order as “Option 1” and “Option 2”.
- Among 59 unique workers, 44 participated in Study 1 and 36 participated in Study 2.
- Manual prompt filtering rejected 36 of 146 prompts for Study 1 and 100 of 402 for Study 2 because of quality concerns.The filtering followed prior work and addressed noisy WebText prompts that were not clearly natural language.