Source-linked AI summary
Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time
Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv
TL;DR
Verbatim looping remains difficult to suppress without harming formatting or fluency because standard controls target token recurrence. DRY selectively penalizes suffix continuations that would reproduce earlier spans, and across scales it reduces looping while preserving or improving quality measures.
Problem
Autoregressive language models can enter prolonged verbatim loops, while token-level repetition controls may penalize structurally necessary reuse and degrade formatting or fluency.
Method
DRY is a sampling-time logit adjustment that penalizes candidate tokens only when they extend the current suffix into an earlier context span, using selective matching and sequence breakers.
Results
Across models from 1.5B to 120B parameters, DRY reduces SER@4 by 47%, improves distinct-4 from 0.958 to 0.975, and preserves benchmark performance within run-to-run variance at frontier scale.
Takeaways & Limitations
The placebo result identifies suffix matching rather than generic logit perturbation as the operative mechanism, while DRY provides a practical loop-control method adopted by open-source inference frameworks.
Takeaways & Limitations
Contrastive decoding is degenerate on Qwen 2.5-1.5B because its expert and amateur are the same model, and no-repeat n-gram has SER@8 = 0 by construction.
Abstract
from arXiv · showhide
Large Language Models generate text autoregressively, but open-ended generation is prone to verbatim looping, in which models repeat spans already present in context. Standard defenses such as repetition, presence, and frequency penalties and n-gram blocking act on token recurrence rather than the sequential structure of a loop, and often suppress looping only at strengths that also degrade formatting or fluency. We propose Don't Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context. Sequence breakers protect chat templates and formatting tokens. Across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study, DRY reduces suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no comparable reduction, identifying suffix matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8K performance, whereas standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its practical impact on text generation.
1 Introduction
Verbatim looping is a persistent autoregressive failure mode, while token-level repetition controls can trade loop suppression for degraded formatting and fluency. DRY instead targets suffix continuation and is evaluated against existing controls across models, prompts, and human judgments.
- Verbatim loops repeat spans already present in context and can continue for dozens or hundreds of tokens once initiated.
- Standard repetition, presence, frequency, and n-gram controls act on prior token occurrence rather than the sequential structure of loops.
- Increasing penalty strength suppresses loops but degrades formatting and fluency, whereas weaker settings preserve fluency but leave loops uncontrolled.
- DRY penalizes a candidate only when it would extend the current suffix into a sequence previously seen in context.The penalty grows exponentially with matching-span length, while sequence breakers protect structural boundaries.
- DRY has been adopted by open-source inference frameworks including text-generation-webui, llama.cpp, and ExLlamaV2.
- The paper evaluates DRY against six baselines and a placebo across models from 1.5B to 120B parameters, nine prompt families, and three seeds.The evaluation also includes a blind 600-pair human study.
2 Related Work
Prior work addresses degeneration through distribution-shaping, token-level, training-time, and sequence-level methods. DRY differs by operating entirely at sampling time while targeting suffix continuation rather than generic token reuse.
- Decoding strategies such as top-k, nucleus, typical, Mirostat, truncation, min-p, and contrastive sampling reshape token distributions but do not directly target verbatim-loop structure.
- Token-level repetition penalties and hard n-gram blocking cannot distinguish benign structural reuse from the start of a verbatim loop.They may suppress newlines, chat-template tokens, and other structurally necessary tokens.
- Training-time alternatives include unlikelihood training, preference fine-tuning, and controlled generation methods that require training access.
- DRY operates entirely at sampling time on the target token sequence, avoiding the training-access requirement of those alternatives.
3 DRY Sampling
DRY selectively adjusts logits when a candidate would continue a previously seen suffix, with penalties that strengthen for longer matches and breakers that protect structural boundaries.
- DRY identifies the longest current-context suffix that matches an earlier span and would be extended by candidate token v.
- DRY subtracts λβ^(n−L) from the candidate logit when the match exceeds threshold L and the token is not a breaker.Its parameters are threshold L, multiplier λ, base β, and breaker set B.
- Tokens that do not continue a previously seen suffix remain unchanged, so DRY modifies only a small fraction of the distribution.
- The penalty grows exponentially beyond L, giving short matches mild nudges and long matches strong suppression without hard blocking.
- Configurable sequence breakers interrupt matching at structural boundaries such as newlines, quotation marks, and formatting markers.
4 Experimental Design
The experiments compare DRY with common repetition controls across instruction-tuned models, prompt families, decoding settings, and loop- and quality-oriented metrics. The design includes multiple seeds, negative controls, and explicit scope caveats.
- Models and conditions: The primary benchmark uses Qwen 2.5-1.5B, Llama 3.2-3B, and Qwen 2.5-7B, with an additional Qwen 2.5-14B evaluation.The models cover Qwen and Llama tokenizer families and an order-of-magnitude scale range.
- Models and conditions: DRY is compared with repetition, presence, and frequency penalties, no-repeat n-gram blocking, contrastive decoding, and a suffix-unaware placebo.
- Models and conditions: Nine prompt families test loop suppression, false positives, structural formatting, necessary repetition, exact copying, and low-loop behavior.
- Metrics and protocol: SER@L measures the fraction of decoding steps where the sampled token extends a previously seen suffix of length at least L.Additional metrics include maximum matched suffix length, repeated n-gram rates, and loop-free generation length.
- Metrics and protocol: Quality is measured with distinct-4, MAUVE, and compression ratio, with results macro-averaged across models and prompt families over three seeds.
- Scope caveats: The contrastive-decoding configuration on Qwen 2.5-1.5B is degenerate because its expert and amateur are the same model.Excluding 1.5B yields a similar SER@4 of 0.140.
- Scope caveats: No-repeat n-gram has SER@8 = 0 by construction because repeated n-grams are made impossible.
- Main evaluation: Across 16,566 regime-A generations, DRY achieves a 47% relative SER@4 reduction over no intervention and outperforms all soft baselines.The table macro-averages three models and three seeds and reports distinct-4, SER@8, and loop-free length.
5 Results
Across model scales and evaluation settings, DRY suppresses verbatim loops while preserving or improving diversity, formatting, human preference, and capability scores. Its selective sequence-aware control outperforms soft baselines and avoids the collateral effects of hard blocking.
- Main Loop Suppression: 47% relative reduction in SER@4 over the uncontrolled baseline across 16,566 generations, exceeding every evaluated soft method.The placebo is statistically indistinguishable from no intervention, while no-repeat n-gram achieves slightly lower raw SER@4 through hard blocking.
- Quality Preservation: 0.095 macro-averaged MAUVE-vs-human for DRY, higher than the uncontrolled baseline at 0.077 and all listed alternatives.DRY also sits on the SER@4 versus distinct-4 Pareto frontier alongside no-repeat n-gram.
- Structure Preservation and False-Positive Control: 56% SER@4 reduction on the structured-formatting family, while sequence breakers keep structural tokens unpenalized.Disabling breakers lowers SER@4 marginally but increases false-positive penalties on structural tokens by more than 23×.
- Structure Preservation and False-Positive Control: DRY suppresses roughly half the loop rate on necessary-repetition prompts while preserving legitimate repeated content better than no-repeat n-gram.On low-loop-control prompts, DRY has the smallest deviation from baseline SER@4 among active methods and slightly higher distinct-4 than baseline.
- Frontier Scale and Capability Preservation: Roughly half the loop rate on both Llama-3-70B-Instruct and GPT-OSS-120B, with improved distinct-4 and baseline-referenced MAUVE relative to every soft baseline.No-repeat n-gram attains slightly lower raw SER@4 through hard blocking, while the repetition penalty provides roughly half DRY’s loop suppression.
- Capability and Human Evaluation: DRY tracks the uncontrolled baseline on MT-Bench, MMLU, and GSM8K, while repetition penalty and no-repeat n-gram incur measurable capability losses.In a 600-pair human study, annotators preferred DRY for loop avoidance and formatting preservation over the repetition penalty.
6 Discussion
DRY combines sequence-aware loop suppression with preserved output quality across decoding settings and model scales. Human judgments and capability evaluations support its practical effectiveness, while its low latency and composability aid deployment.
- DRY outperforms token-level penalties on loop suppression while maintaining or improving output diversity.An intervention-matched placebo attributes the reduction to suffix matching rather than generic logit perturbation.
- DRY’s deviation from the baseline on benign low-loop prompts is the smallest among active methods.
- Across a 600-pair MTurk study, DRY matches the uncontrolled baseline on fluency and formatting while outperforming it on loop avoidance.Frequency penalty and no-repeat n-gram score lower on the same axes.
- DRY does not degrade MT-Bench, MMLU, or GSM8K accuracy on Llama-3-70B, whereas standard alternatives do.
- DRY remains effective from 1.5B to 120B parameters and adds under 3% latency overhead at 128K context.Results are reported as stable across decoding regime, random seed, tokenizer family, and model scale.
- No-repeat n-gram achieves marginally lower raw SER@4 than DRY only by blocking all repeated n-grams, including structurally necessary ones.This makes hard blocking unsuitable for chat, structured output, and evidence-grounded generation requiring verbatim reproduction.
7 Conclusion
The paper defines DRY as a selective, sequence-aware logit adjustment that penalizes candidates extending previously seen suffixes while preserving structural repetition. Evaluation shows broad loop reduction, diversity gains, stable performance, and practical deployment benefits.
- 7 Conclusion: DRY finds the longest prior suffix match that a candidate token would extend, then selectively adjusts that candidate’s logit.The match search halts at context boundaries, mismatches, or sequence breakers.
- 7 Conclusion: DRY leaves logits unchanged when the candidate is a breaker or the match length is below the allowed repetition threshold.Beyond the threshold, the penalty grows exponentially with match length.
- 7 Conclusion: 55% is DRY’s largest relative SER@4 reduction on Qwen 2.5-1.5B, while Llama 3.2-3B drops from 0.167 to 0.086.Distinct-4 improves under DRY on all three models in the per-model regime-A results.
- 7 Conclusion: Results are reported across models, decoding regimes, seeds, and prompt families, with macro-averaging and per-model and per-family breakdowns.The main evaluation uses tuned hyperparameters selected on a development split.
- 7 Conclusion: DRY achieves similar relative SER@4 reductions across two decoding regimes, indicating effectiveness does not depend on baseline decoding temperature.On Qwen 2.5-14B under regime B, SER@4 falls from 0.138 to 0.076 (45%).
- 7 Conclusion: The evaluation compares DRY with token-level penalties, hard n-gram blocking, contrastive decoding, and a suffix-unaware placebo across balanced prompt families.The benchmark covers loop stress, structure and copy prompts, low-loop controls, and paraphrase variants.
I Ablation Design
The ablation design tests DRY’s hyperparameters, breakers, and intervention mechanism across balanced prompt families. Sequence breakers trade a small amount of loop suppression for substantially better preservation of structurally required repetition.
- I Ablation Design: 34,944 generations sweep DRY’s multiplier, base, threshold, breaker set, and range limit.The main evaluation selects one tuned configuration per model on the development split.
- I Ablation Design: The benchmark balances loop-stress, structure-and-copy, low-loop negative-control, and paraphrase prompt families across 240 prompts.The compared conditions include primary penalties, no-repeat n-gram blocking, contrastive decoding, and a matched-strength placebo.
- I.1 The Necessity of Sequence Breakers: 28.4% is the false-positive penalty rate without breakers, versus 1.2% with breakers, while structural MAUVE falls from 0.94 to 0.61.Removing breakers lowers SER@4 from 0.053 to 0.041 but damages Markdown tables, dialogue boundaries, and bullet structure.
K 14B Scale Experiment
The 14B experiment extends DRY beyond the primary model scale using the same benchmark configuration. DRY preserves the established pattern of lower looping and higher lexical diversity.
- K 14B Scale Experiment: 39%: DRY reduces SER@4 on Qwen 2.5-14B from 0.130 to 0.079 while distinct-4 rises from 0.948 to 0.962.The paired reduction is 0.051 with 95% CI [0.048, 0.054].
L Robustness
DRY remains effective across decoding regimes, seeds, tokenizer families, model scales, stacking configurations, and extreme context lengths, while limiting collateral damage and computational overhead.
- Decoding and scale: DRY’s relative SER@4 reduction under regime B remains within one percentage point of regime A.This indicates stable effectiveness across decoding configurations.
- Decoding and scale: A cross-seed SER@4 standard deviation of 0.001 is more than an order of magnitude smaller than DRY’s absolute baseline reduction.The result is not attributable to one random sequence.
- Decoding and scale: DRY consistently reduces loops across Qwen and Llama tokenizers, including Qwen 2.5-14B, with the largest relative reduction on Qwen 2.5-1.5B.Its largest absolute reduction occurs on Llama 3.2-3B, which has the highest baseline loop rate.
- Efficiency and composition: DRY’s reverse-search implementation keeps per-token overhead under 3% through 128K-token contexts and below multiplicative repetition-penalty overhead.The benchmark uses Llama 3.2-3B on an NVIDIA RTX 4090.
- Collateral effects: Across 2,988 paired comparisons, DRY reduces SER@4 on 38.2% of instances, while diversity degradation occurs on only 2.6%.Only 4.0% of instances show an SER@4 increase exceeding 0.01, and no prompt family is systematically harmed.
T Limitations
DRY’s evaluation and quality claims have defined scope boundaries, and its MAUVE comparisons require careful interpretation because the reference genre does not match the prompt suite.
- Method scope: DRY targets exact surface-form continuation loops, not semantic repetition, discourse-level looping, or hallucination.The intervention operates in logit space over surface lexical features.
- Evaluation scope: The primary evaluation covers open-weights models from 1.5B to 120B parameters, so results may not transfer directly to closed proprietary systems.Human evaluation covers Qwen 2.5-7B outputs and broader studies remain valuable for subjective-quality claims.
- Metric interpretation: MAUVE values should rank methods relative to the uncontrolled baseline rather than establish absolute human-likeness.The WikiText-103 reference is encyclopedic and mismatched with the creative, dialogue, and structured-formatting prompt genres.
- Generalization: DRY improves every tested stacked decoding configuration without degrading diversity, but this composability evidence is macro-averaged over two smaller models.The reported sweep covers Qwen 2.5-1.5B and Llama 3.2-3B.