PaperlayerView exact revision on arXiv

Source-linked AI summary

The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, Chloé Clavel

arXiv:2311.09807v2cs.CL

TL;DR

As synthetic text increasingly enters training data, this study asks how recursive training affects linguistic diversity, an aspect often overlooked in evaluation. It measures diversity across lexical, semantic, and syntactic dimensions and finds declining diversity across iterations, particularly on creative tasks.

  • Problem

    Although language-model evaluation often emphasizes task performance, relatively little attention has been paid to linguistic diversity, despite concerns about recursively training on predecessor-generated text.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    The study develops lexical, semantic, and syntactic diversity metrics and applies them in recursive finetuning experiments across natural-language-generation tasks.

    Source: §1 · p. 2

  • Results

    Recursive finetuning is associated with declining output diversity, with the fastest decline and largest human–model diversity gap in story generation.

    Source: §1 · p. 2, §4.2 · p. 7

  • Takeaways & Limitations

    The findings point to a risk of diminishing linguistic richness and variety in outputs as models increasingly train on predecessor-generated text.

    Source: §5 · p. 9

    The study does not evaluate multilingual diversity, and its authors say realistic web-setting experiments are not feasible with the available data and detection methods.

    Source: Limitations · p. 9

Abstract

from arXiv · show

This study investigates the consequences of training language models on synthetic data generated by their predecessors, an increasingly prevalent practice given the prominence of powerful generative models. Diverging from the usual emphasis on performance metrics, we focus on the impact of this training methodology on linguistic diversity, especially when conducted recursively over time. To assess this, we adapt and develop a set of novel metrics targeting lexical, syntactic, and semantic diversity, applying them in recursive finetuning experiments across various natural language generation tasks in English. Our findings reveal a consistent decrease in the diversity of the model outputs through successive iterations, especially remarkable for tasks demanding high levels of creativity. This trend underscores the potential risks of training language models on synthetic text, particularly concerning the preservation of linguistic richness. Our study highlights the need for careful consideration of the long-term effects of such training approaches on the linguistic capabilities of language models.

1 Introduction

High-quality text may become scarce as synthetic content grows in training data, raising questions about recursively training on model-generated samples. The study measures linguistic diversity across three dimensions and finds that it declines over recursive finetuning iterations.

1 Introduction

  • The discussion is motivated by the possibility that usable high-quality text may approach exhaustion within an order of magnitude.The cited estimate places the world’s high-quality text supply at up to 17 trillion tokens, with 4–5% annual growth; the passage compares this with Llama 2’s roughly 2 trillion training tokens.Source: §1 · p. 1
  • As model-generated content enters web pretraining and finetuning data, future models are increasingly likely to train on blended human- and model-produced text.Synthetic examples are also used for instruction tuning and task-specific data augmentation.Source: §1 · p. 1
  • Recursive training on predecessor-generated text raises concerns about lost human-distribution tails and increasingly uniform model behavior.The passages describe these risks as possible consequences of recursive sampling, not as results established by this study.Source: §1 · pp. 1–2
  • The study asks how linguistic diversity can be quantified and whether recursive synthetic-text training reduces it, rather than centering task-solving performance.The authors frame linguistic diversity as important but overlooked.Source: §1 · p. 2
  • Successive recursive finetuning iterations produce a notable decrease in output diversity across the studied generation tasks.The study develops metrics for lexical, semantic, and syntactic diversity and applies them to news summarization, scientific abstract generation, and story generation.Source: §1 · p. 2

2 Related Work

Related work reviews approaches to linguistic-diversity evaluation and training with synthetic text. The paper positions its contribution as evaluating recursive training’s effects on diversity rather than task performance.

2 Related Work

  • The related-work section covers existing diversity-evaluation approaches and recent research on language-model-generated training text.Source: §2 · p. 2

2.1 Evaluating Linguistic Diversity

  • Prior evaluation research largely emphasizes task-solving performance, while linguistic diversity receives comparatively little attention.Existing diversity work often focuses on lexical measures, leaving other dimensions less comprehensively evaluated.Source: §2.1 · p. 2
  • Lexical measures such as distinct-n, entropy, and Self-BLEU capture word or n-gram variation, while proposed semantic and syntactic approaches remain varied.The paper describes its evaluation across lexical, semantic, and syntactic diversity with novel automatic metrics as the first comprehensive treatment of all three aspects.Source: §2.1 · pp. 2–3

2.2 Training with Synthetic Text

  • Recent synthetic-text training work includes iterative instruction-data generation, self-improvement, and model-generated chat corpora.The related-work section frames large-scale use of synthetic text to train generative models as a recent trend.Source: §2.2 · p. 3
  • Building on concerns about recursive training and model collapse, this paper investigates linguistic diversity rather than performance degeneration.The authors state that, to their knowledge, theirs is the first work to address this diversity question.Source: §2.2 · p. 3

3 Methodology

The methodology simulates recursive finetuning from human-generated data through successive model-generated datasets and evaluates outputs with lexical, semantic, and syntactic diversity metrics. It also uses perplexity to check whether generated texts remain suitable for diversity analysis.

3.1 Recursive Training Simulation

  • Each generation is finetuned on data generated by its predecessor, beginning with human-generated task data and repeating the process across generations.The bases share an architecture but are independently initialized, and the simplified setup trains each successor only on the immediately preceding generation’s data.Source: §3.1 · p. 3, Figure 1 · p. 4
  • The experiments cover news summarization, scientific abstract generation, and story generation, ordered from most constrained to most creative.The finetuned models are evaluated on each task’s corresponding test set.Source: §3.1 · p. 3

3.2 Perplexity

  • Perplexity serves as a check that finetuned models remain aligned with training data and produce texts of sufficient quality for diversity evaluation, not as a default superiority measure.The authors caution that creative or unusual language can have high perplexity despite being grammatical and coherent.Source: §3.2 · pp. 3–4

3.3 Linguistic Diversity Metrics

  • The paper evaluates linguistic diversity from lexical, semantic, and syntactic perspectives.The methodology introduces metrics for each of these three aspects.Source: §3 · p. 3, §3.3 · p. 4

3.3.1 Lexical Diversity

  • The diversity framework measures lexical variety at word, n-gram, and sentence levels, including TTR, distinct-n, and 1 − Self-BLEU.TTR and distinct-n are length-sensitive, so texts are truncated to fixed lengths for TTR and distinct-n is reported for n = 2 and n = 3.Source: §3.3.1 · p. 4

3.3.2 Semantic Diversity

  • Semantic diversity is the average pairwise cosine distance among Sentence-BERT sentence embeddings.The embeddings represent sentences in a semantic space, and their dispersion is reported as Div_sem.Source: §3.3.2 · p. 5

3.3.3 Syntactic Diversity

  • Syntactic diversity is measured by average pairwise distance between vectorized dependency-tree graphs using the Weisfeiler-Lehman graph kernel.A neural parser constructs the dependency trees under the universal dependencies formalism; the resulting measure is Div_syn.Source: §3.3.3 · p. 5

4 Experiments and Results

Across six recursive finetuning iterations on three generation tasks, the experiments track how linguistic diversity changes as models train on predecessor-generated text. Diversity generally declines, most sharply in story generation, while filtering and fresh human data produce different effects on that decline.

4.1 Experimental Setup

  • Each task uses six recursive iterations, with OPT-350M models finetuned on human data first and predecessor-generated data thereafter.Each iteration starts from a new base-model instance, and synthetic datasets contain as many samples as the original training set.Source: §4.1 · p. 5

Task1: Abstractive News Summarization

  • News summarization uses XL-SUM and is treated as low-entropy because lengthy source articles constrain the summaries.The training set has 306,522 samples and the test set 11,535.Source: Task1: Abstractive News Summarization · p. 5

Task2: Scientific Abstract Generation

  • Scientific abstract generation prompts models with a paper title and its first abstract sentence, providing context while leaving room for continuation.The dataset contains 40,512 training samples and 2,132 test samples; Table 1 illustrates repeated URL generation after several synthetic-training iterations.Source: Task2: Scientific Abstract Generation · pp. 5–6, Table 1 · p. 6

Task3: Story Generation

  • Story generation on WritingPrompts is a high-entropy task with limited context: prompts average 30 tokens and generally lack plot structure, while resulting stories average 389 tokens.WritingPrompts has 272,600 training samples and 15,138 test samples.Source: Task3: Story Generation · p. 6

4.2 Results

  • Diversity generally declines across iterations, with the fastest and most pronounced loss in high-entropy story generation; lower-entropy tasks also eventually decline.For scientific abstracts, syntactic diversity initially rises above human-written abstracts before falling below them in later iterations.Source: §4.2 · pp. 7–8
  • Syntactic diversity declines especially on creative tasks, while semantic diversity is more stable and sentence coherence drops.The authors report that syntactic diversity declines comparably to lexical diversity and more than semantic diversity; generated meanings remain diverse across iterations despite reduced coherence between sentences.Source: §4.2 · p. 8

4.3 Playing with Recursion Settings

  • On story generation, fresh human data only minimally mitigates the decline, while filtering synthetic samples significantly amplifies it.The study filters out the 20% of synthetic samples with the lowest linguistic acceptability scores; Figure 2 reports Distinct-3 for lexical diversity and D_syn_c for syntactic diversity.Source: §4.3 · pp. 8–9, Figure 2 · p. 8

5 Conclusion

Recursive training on predecessor-generated synthetic text is associated with declining lexical and syntactic diversity, risking reduced linguistic richness in model outputs.

5 Conclusion

  • Successive rounds of recursive synthetic-text training reduce lexical and syntactic diversity in language model outputs.The study emphasizes preserving linguistic diversity alongside technical performance.Source: §5 · p. 9
  • The observed reductions create a tangible risk of diminishing linguistic richness and variety as models increasingly rely on predecessor-generated text.Source: §5 · p. 9
  • The findings support a more nuanced approach to model development that values linguistic diversity alongside technical performance.Source: §5 · p. 9

Limitations

The experiments are English-only, use one moderately large model, and do not reproduce a realistic web-training setting; the authors also identify a limited mixed-data condition.

Language diversity

  • The experiments are exclusively in English, and the metrics may not perform optimally in other languages, especially those with limited resources.They depend on language-specific tokenization or segmentation, dependency parsing, and sentence embeddings.Source: Limitations · p. 9

Resource constraint

  • Resource limits restricted the study to OPT-350M, so its cross-model comparisons do not establish whether the decline recurs across models.The authors focus on tasks and settings rather than models and anticipate recurrence in other models.Source: Limitations · p. 9

Realistic Web Setting

  • A realistic web setting was infeasible because a suitable synthetic-text dataset could not be constructed and the amount of synthetic online content could not be estimated.The proposed ShareGPT data route raised copyright issues, and no reliable algorithm could detect model-generated text.Source: Limitations · p. 9
  • Mixing a fixed percentage of human data only marginally reduced the diversity decline; varying the proportions remains future work.Source: Limitations · p. 9

Ethical Considerations

The study reports that its datasets and model are used for their designated research purposes, while cautioning against overgeneralizing its findings.

Ethical Considerations

  • The study uses datasets under CC BY-NC-SA 4.0 or MIT licenses and the OPT model under the OPT-175B License Agreement.The authors state that the resources contain no information linkable to private individuals in a harmful way and that their use aligns with designated research purposes.Source: Ethical Considerations · p. 10
  • The authors warn that public interpretations may exaggerate or overgeneralize findings validated only on specific datasets and in a particular context.Source: Ethical Considerations · p. 10

A Visualization of Diversity Metrics

The visualizations illustrate successive-iteration declines in lexical, syntactic, and semantic diversity, with the semantic change less pronounced than the syntactic change.

A Visualization of Diversity Metrics

  • Low-frequency, unique words become less prevalent across story-generation iterations than in human-generated text, indicating declining lexical diversity.The word-frequency axis is truncated at 100 for visual clarity, although the distribution extends further.Source: Figure 3 · p. 14
  • Dependency-tree embedding distributions become more compact across iterations, indicating declining syntactic diversity.Source: Figure 4 · p. 14
  • Sentence-embedding dispersion decreases across iterations, indicating lower semantic diversity, with a less pronounced change than for syntax.Source: Figure 5 · p. 15

B Implementation of Diversity Metrics

Generated texts are preprocessed before diversity metrics are computed. The steps vary by task and metric, with punctuation removed only for lexical-diversity calculations.

  • Prompts are removed from generated texts for all three tasks.Source: §B.1 · p. 15
  • Story-generation texts have newline tokens removed, while scientific-abstract texts have URL links replaced with WEBSITE.Source: §B.1 · p. 15
  • Punctuation is removed for lexical-diversity calculations but retained for semantic- and syntactic-diversity calculations.Source: §B.1 · p. 15

B.2 Lexical Diversity Metrics

The lexical-diversity setup controls for text length when computing distinct-n and uses the mean of Self-BLEU-2 and Self-BLEU-3. Table 3 reports average post-truncation word counts.

B.2 Lexical Diversity Metrics

  • Table 3 reports average post-truncation generation lengths measured in words.Source: Table 3 · p. 15, §B.2 · p. 15
  • 20 words for news summarization, 50 for scientific abstracts, and 150 for stories are the truncation thresholds used to compare distinct-n at fixed lengths.Post-truncation average lengths fall within a narrow range, supporting fair comparisons across iterations.Source: §B.2 · p. 15
  • Self-BLEU is the mean of Self-BLEU-2 and Self-BLEU-3.Source: §B.2 · p. 15

B.3 Semantic Diversity Metrics

The semantic and syntactic diversity procedures use pairwise cosine distances over sentence embeddings and dependency-graph representations, respectively.

B.3 Semantic Diversity Metrics

  • Sentence splitting uses the NLTK tokenizer, and Sentence-BERT uses the all-mpnetbase-v2 model.Source: §B.3 · p. 15
  • Pairwise cosine-distance calculations draw 2,000 sentences and report the mean across five randomizations.Source: §B.3 · pp. 15–16

B.4 Syntactic Diversity Metrics

  • Dependency graphs are built with Stanza, then compared using the Weisfeiler-Lehman graph kernel with two iterations and pairwise cosine distances.The pairwise-distance procedure matches the one used for semantic diversity.Source: §B.4 · p. 16
Loading 2311.09807v2…