How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN

• Generated text can be grammatical and coherent, but text quality alone cannot distinguish model construction from training-set copying (See et al., 2019; Brown et al., 2020, section 3.9.4).The paper therefore argues that novelty should be evaluated alongside standard quality measures (Gatt and Krahmer, 2018; Celikyilmaz et al., 2020).Source: §1 · p. 1
• Novelty matters because language involves combining familiar parts productively, learning should generalize beyond memorization, and copied text can leak sensitive information or repeat hate speech (Carlini et al., 2021).Source: §1 · p. 1
• RAVEN evaluates novelty in sequential n-gram and syntactic structure across four language-model families, 12 generation methods, four prompt lengths, and human test-set baselines.The models include an LSTM, Transformer, Transformer-XL, and all four GPT-2 sizes with accessible training data.Source: §1 · p. 1
• 74% of Transformer-XL-generated sentences have syntactic structures absent from training, while GPT-2 generates novel words but sometimes duplicates passages exceeding 1,000 words.Models generate novel n-grams, morphological combinations, and syntactic structures, yet human baselines contain 1.4–3.3 times as many novel bigrams as model-generated text; for n-grams larger than 5, models are more novel than the baseline.Source: §1 · p. 2

Memorization and copying

• The study broadens prior copying work—Brown et al. (2020), Lee et al. (2021), Chen et al. (2021), and Ziegler (2021)—across n-gram sizes, architectures, generation methods, and linguistic structures.It examines copying beyond n-grams, including dependency arcs, to study linguistic generalization rather than primarily data privacy.Source: §2 · p. 2

NLG evaluation

• Open-ended NLG evaluation should measure novelty alongside quality because copied training text can appear coherent without demonstrating the underlying ability.This concern responds to the ELIZA effect and the difficulty of evaluating open-ended outputs with a single quality score or property-specific metrics.Source: §2 · p. 2
• Novelty concerns differences from the training set, whereas diversity concerns differences among generated outputs; either property can occur without the other.A model may copy a diverse set of training sentences or repeatedly generate the same novel sentence.Source: §2 · p. 2

Scope

• The main analyses assess novelty in surface form, such as syntax, rather than whether models understand meaning.This scope contrasts with broader debates about language understanding, including Bender and Koller (2020) and Marcus (2020).Source: §2 · p. 2

Motivation

• RAVEN treats novelty as essential for assessing linguistic generalization and developing language models that handle unfamiliar situations.Only novel text can demonstrate generalization, although novelty may not matter for every practical use case.Source: §3 · p. 3

Approach

• The analysis generates many language-model samples and labels n-grams or syntactic structures as duplicated when they occur in training data or preceding context.Structures absent from both sources are classified as novel.Source: §3 · p. 3
• RAVEN compares model-generated text with human text from the model’s in-distribution test set to contextualize expected duplication within the training domain.If generated text is at least as novel as this baseline, the model is not considered to be copying excessively.Source: §3 · p. 3

Assumption

• Copying is not inherently undesirable because some duplicated long n-grams, such as book titles, can be reasonable.The comparison therefore evaluates excessive copying rather than treating all duplication as a defect.Source: §3 · p. 3

Models

• LSTM, Transformer, and Transformer-XL are compared on identical 103-million-word Wikitext-103 training data, spanning recurrence, self-attention, and both mechanisms.The architectures are associated with Hochreiter and Schmidhuber (1997), Vaswani et al. (2017), and Dai et al. (2019), respectively.Source: §4 · p. 3
• GPT-2 is evaluated separately as the largest accessible Transformer model, using 7.7-billion-word WebText and subword rather than word-level tokenization.WebText consists of webpages linked from Reddit; the other models use word-level tokenization.Source: §4 · p. 3

Prompts

• Each model receives test-set prompts from its training distribution and generates a continuation, with the corresponding corpus continuation serving as the human-generated baseline.Wikitext-103 uses 1,000 512-word prompts followed by 1,000 generated words; WebText uses 1,000 564-subword-token prompts followed by 1,100 generated tokens.Source: §4 · p. 3

Decoding method: top-40 sampling

• All models use top-40 sampling, truncating predictions to the 40 highest-ranked words before renormalization and sampling.The method prioritizes high-quality text because novelty without quality can reflect random nonsense; it follows choices associated with Radford et al. (2019) and Dai et al. (2019).Source: §4 · p. 3
• The same decoding scheme is applied to LSTM and Transformer because those models lack an established alternative, while other decoding methods are examined in Section 5.3.Top-40 sampling was selected for GPT-2 and Transformer-XL by their creators.Source: §4 · pp. 3–4

5.4 Other n-gram analyses

• The LSTM is least novel for small n-grams, the Transformer most novel, and Transformer-XL intermediate, consistent with recurrence creating a recency bias toward memorizing local sequences.Model size and prompt length show no clear effect on novelty; positional effects are small for some models, and training-set-only duplication produces similar results.Source: §5.1 · p. 4, §5.4 · p. 6

5.1 How often are generated n-grams novel for various values of n?

• For n < 6, generated n-grams are less novel than test-set baselines, whereas for n > 6 they are more novel; n-grams of size 5 or larger are mostly novel.On Wikitext-103, bigram novelty is 2%–3% for models versus 6% for the baseline, and trigram novelty is 17%–22% versus 31%.Source: Figure 1 · p. 4, §5.1 · p. 4

5.2 Do models ever duplicate large n-grams?

• All models occasionally duplicate passages of at least 100 words, including GPT-2 passages exceeding 1,000 words; the paper calls duplication of n-grams of size 100 or larger supercopying.Supercopied 100-grams appear more often in the training set than randomly selected 100-grams, consistent with repeated training text contributing to supercopying.Source: §5.2 · pp. 4–5

5.3 How is novelty related to the decoding scheme and the generated text’s quality?

• Increasing top-k, top-p, or temperature increases novelty, but every modification that increases novelty also decreases quality.The pointwise duplication score averages each token’s smallest novel n-gram size, truncated at 5 to reduce the influence of supercopying.Source: §5.3 · p. 5
• The novelty–quality tradeoff is clear: Wikitext models underperform their baseline, whereas GPT-2 matches it with top-p p = 0.95, reaching perplexity 93.7 versus 89.4 and duplication score 4.41 versus 4.47.Transformer architectures manage the tradeoff better than LSTMs, and GPT-2’s advantage may reflect WebText’s larger or lower-quality training corpus rather than architecture or model size.Source: §5.3 · pp. 5–6, Figure 2 · p. 6

Findings

• Most generated sentences have an overall syntactic structure absent from training, while models are less novel than baselines for local dependency structure.This global–local pattern parallels the reported n-gram findings: models are less novel for small-scale structure but more novel for larger-scale structure.Source: §6 · p. 6

Method

• The analysis parsed generated text and training data with constituency and dependency parsers, evaluating novelty across seven syntactic aspects.The parsers were based on Kitaev and Klein, 2018 and Zhang et al., 2020b.Source: §6 · p. 6

Limitations

• Manual analysis found four of seven attributes accurate enough for numerical reporting because parser errors particularly affected novel examples.The numerical results are reported in Figure 3, while parser output for the remaining attributes was not considered sufficiently reliable.Source: §6 · p. 6

Findings

• Across models and baselines, most sentences have novel POS sequences and parse structures, with little difference between models and baselines.For dependency arcs and dependency relations, baselines are far more novel than models.Source: §6 · p. 6

7.1 When GPT-2 generates novel words, are they morphologically well-formed?

• GPT-2 chooses English plural and possessive forms correctly in 72 of 74 novel plurals and 135 of 136 novel possessives.However, only 28% of novel acronyms with expansions are suitable, compared with 81% in the baseline.Source: §7.1 · p. 7

7.2 When GPT-2 generates novel words, do they fit within their syntactic context?

• GPT-2 generally handles syntactic consequences of novel words, including agreement across intervening distractors, plural possessives, coreferential pronouns, and number-sensitive determiners.It also increments ordered numbers, variable-name endings, and alphabetical company lists while generating novel words.Source: §7.2 · pp. 7–8
• In human-generated text, novel words are more likely to appear inside quotation marks, and quoted words are more likely to be novel; this association also appears in GPT-2’s generated text.Figure 5 statistics are computed over all word-level, not subword-level, unigrams.Source: §7.2 · p. 8, Figure 5 · p. 8

7.3 When GPT-2 generates novel words, do they result in reasonable meanings?

• GPT-2’s novel words are mostly morphologically well-formed (96%) and grammatically appropriate (94%), but semantic errors are substantially more common.The morphological rate trails the 99% baseline, while syntactic usage remains broadly correct; numerical mistakes include incorrect conversions, physical impossibilities, and inconsistent exchange rates.Source: §7.1 · p. 7, Figure 4 · p. 7, §7.2 · p. 7
• GPT-2’s novel words sometimes receive clear, accurate definitions or contextually appropriate uses despite frequent semantic failures.Positive cases include “re-nitrification,” “co-causation,” and “bondbreaking enchantment.”Source: §7.3 · p. 9

7.4 What does GPT-2 generalize from?

• GPT-2 appears to use both compositional and analogy-based generalization, although individual novel words do not always reveal which process produced them.Unseen stems modified with contextual affixes support composition, whereas variants of torero support analogy because English has no process adding -ro to verbs.Source: §7.4 · p. 9, LHAWs · p. 9

Overall findings

• Models generate novel n-grams, syntactic structures, and morphological combinations, yet are less novel locally and sometimes duplicate training passages exceeding 1,000 words.These results indicate both broad novelty and substantial copying across scales.Source: §8 · p. 9

Compositionality

• All models show compositional generalization, but below the human baseline; GPT-2 also overgeneralizes, producing forms such as “752th” and requiring rules alongside exception handling.Most baseline test sentences had sentence-level syntactic structures absent from training, showing compositionality matters even in natural in-distribution data.Source: §8 · pp. 9–10

Evaluating novelty

• Generated text should be at least as novel as an in-distribution test-set baseline, which helps rule out excessive copying as an explanation for apparent generative ability.The authors argue novelty is necessary for evaluating capabilities beyond copying, such as coherence.Source: §8 · p. 10
• Because large training sets can undermine assumptions about novelty, explicit checking is necessary: the incorrect regular form of all 92 basic English irregular verbs appears in GPT-2’s training set.Examples include “becomed” and “teached”; nonsense words such as “wug” are another category often assumed to be novel.Source: §8 · p. 10

Improving novelty

• Novelty may increase through sampling penalties or training-set deduplication, since repeated training data is associated with supercopying and deduplication can reduce copied 50-grams.These approaches target copying through post-hoc sampling changes or more nuanced training-data preprocessing.Source: §8 · p. 10
• Deeper improvements may come from compositional architectures and explicit semantic planning, which could reduce models’ reliance on repeating text when generation lacks a meaning-driven plan.The discussion contrasts human symbolic, compositional generation with neural text generation that primarily manipulates text alone.Source: §8 · p. 10

9 Conclusion

• RAVEN shows models are rarely novel in local structure but often novel at larger scales, while occasionally copying very long passages from training data.The authors argue that novelty must be considered when evaluating generated text because copied training material is not withheld.Source: §9 · p. 10

A Model details

• The experiments use LSTM, Transformer, Transformer-XL, and GPT-2 models, with GPT-2-xl serving as the principal GPT-2 model.The LSTM achieved test-set perplexity 45.7, while the Transformer and Transformer-XL achieved 18.65 and 18.30, respectively.Source: A Model details · p. 16

B Tokenization and other text preprocessing

• N-gram analyses use Moses-tokenized text, whereas syntactic analyses sentence-tokenize and parse appropriately detokenized or untokenized text.Wikitext and GPT-2 outputs receive different preprocessing before n-gram and syntactic novelty calculations.Source: B Tokenization and other text… · p. 17

C Novel bigrams

• Figure 7 provides examples of novel bigrams generated by each model.The figure illustrates the sequential-structure novelty examined by RAVEN.Source: C Novel bigrams · p. 17

D Supercopying examples and statistics

• Figures 8 and 9 document supercopying as training-set duplication of passages at least 100 words long and quantify occurrences of copied 100-grams.These analyses directly examine the long-passage copying highlighted in the conclusion.Source: D Supercopying examples and statistics · p. 17

E Untruncated pointwise duplication scores

• Figure 10 reports both truncated and untruncated pointwise duplication scores, extending the truncated-score presentation in Figure 2.The supplementary comparison preserves both versions of the duplication metric.Source: E Untruncated pointwise duplication… · p. 17

F Plots showing the effects of the decoding scheme

• Figure 11 shows how pointwise duplication scores change when three decoding parameters are varied.This analysis assesses the sensitivity of duplication to decoding choices; generation otherwise used beam size 1 and usually top-40 sampling.Source: A Model details · p. 16, F Plots showing the effects of the… · p. 17

G.1 Evaluating overlap between Wikitext-103 and WebText

• The cross-evaluation assumes no training-set overlap because Wikitext-103 comes from Wikipedia and Wikipedia articles were excluded from WebText, though copied material could still enter indirectly.The overlap experiment tested this assumption using normalized 20-grams and Moses tokenization.Source: §G.1 · pp. 17–18
• 12 of 1,000 Wikitext 20-grams appeared in WebText, versus 0 of 1,000 WebText 20-grams in Wikitext, yielding only 0.012 overlap overall.These results support using TXL for WebText-trained models and GPT-2 for Wikitext-trained models despite a small amount of Wikipedia-derived text in WebText.Source: §G.1 · p. 18

G.2 Details of perplexity evaluation

• Perplexity was evaluated in 1,024-token segments with a 512-token stride and 512 preceding context tokens, giving every evaluated token at least 512 prior tokens.The final token of each segment had 1,535 tokens of available context.Source: §G.2 · p. 18

H How does model size affect novelty?

• GPT-2 XL was most novel across the various n-gram sizes, but GPT-2 Medium exceeded GPT-2 Large, so model size had no consistent effect.The comparison used four GPT-2 sizes with top-40 sampling.Source: H How does model size affect novelty? · p. 18

I How does prompt length affect novelty?

• Prompt length generally had little effect on training-set duplication: Transformer and GPT-2 became slightly more novel with longer prompts, whereas Transformer-XL became slightly less novel.The analysis excluded copying from the context itself, which would trivially increase with prompt length; LSTM and the Wikitext-103 test baseline showed no discernible effect.Source: I How does prompt length affect novelty? · p. 20
• Decoding manipulations that produced higher-quality text, measured by lower perplexity, also decreased novelty by increasing duplication.Each plotted point represented a different decoding scheme.Source: Figure 10 · p. 20
• The WebText baseline showed no consistent prompt-length generalization, with length 0 more novel than length 18 but less novel than length 141.Thus, the observed prompt-length differences were not monotonic for the WebText baseline.Source: I How does prompt length affect novelty? · p. 20

Method

• The analysis measures training-set duplication, excludes context copying, truncates pointwise scores to 1–10, and averages scores within 100-word position bins after discarding the first bin.Later positions have more context to copy from, and the first 10 positions have a different score range.Source: J Position in generated text · p. 20

Results

• GPT-2 and Transformer-XL show greater duplication at later positions, while baselines, the LSTM, and Transformer show little positional effect.The positional effect is consistent for GPT-2 and Transformer-XL but small.Source: J Position in generated text · p. 20
• The pointwise duplication score increases by only about 0.2 from the start to the end of generation for GPT-2 and Transformer-XL.Source: J Position in generated text · p. 20

K Duplication from the training set vs. the context

• Models showed much less duplication from the context than from the training set, and almost never copied context content that was not also in the training set.The training sets are much larger than the contexts, so more text pieces count as duplicated from the training set.Source: K Duplication from the training set… · p. 21

L Vetting syntax

• Manual analyses covered seven syntactic aspects to assess whether parse-based novelty estimates were approximately accurate.The authors used these checks to determine which numerical results could be trusted.Source: L Vetting syntax · p. 21
• POS-tag novelty was unreliable because most apparently novel tags reflected mislabeled training instances, while every inspected novel CFG rule resulted from a parser error.Consequently, the authors did not trust numerical results for either POS tags or CFG rules.Source: L Vetting syntax · p. 21
• POS-sequence estimates were approximately correct, and parse-structure estimates required no separate vetting because novel POS sequences guarantee novel parse structures and the measures are close.Manual checks of 100 generated sentences generally found accurate POS sequences with high novelty scores.Source: L Vetting syntax · pp. 21, 23
• Manual checks found dependency-arc and dependency-role novelty estimates reliable enough to report, but dependency argument structure received no numerical results because it was too labor-intensive to validate.For arcs and roles, the authors checked 100 cases per model and examined plausible training sentences for true novelty.Source: L Vetting syntax · p. 23

• Language models were less novel than the baseline for local structures but more novel for larger-scale n-grams and overall sentence structure.Source: M Example of a mismatch between local… · p. 23
• Two generations can have identical trigram and global syntactic novelty while differing in bigram and individual dependency-arc novelty.In the example, both sentences have novel trigrams and novel overall parses, but only one has novel bigrams and noun-verb dependency arcs.Source: M Example of a mismatch between local… · pp. 23, 25

N Examples of syntactic novelty

• The authors could not provide reliable numerical results for several syntactic phenomena because manual checking was infeasible and automatic methods had high error rates.They therefore present manually verified examples as existence proofs that models sometimes perform the relevant syntactic generalizations.Source: N Examples of syntactic novelty · p. 25
• The syntactic examples were manually vetted for perfect precision, but recall was uncontrolled because candidates were limited to cases requiring inspection of at most 500 training sentences.The procedure supplied relevant training examples for Wikitext-based models but not WebText, whose training data was not publicly released.Source: N Examples of syntactic novelty · p. 25
• Across models, generated text contains novel determiners, dependency roles, and some verb argument structures, including transitive-to-intransitive use, but no confirmed novel dative alternations.The evaluation found no confirmed transitive usages for verbs seen only intransitively, and no novel dative alternations; these patterns were also absent from the human baseline.Source: N Examples of syntactic novelty · pp. 26–28

O Morphology categorization

• The broader unigram analysis pooled GPT-2 samples across four prompt lengths and manually annotated tokens absent from training, using the same procedure for WebText baseline text.Manual annotation was chosen because automatic methods were considered unreliable for rare, long-tail novel-unigram phenomena.Source: O Morphology categorization · pp. 28, 30
• GPT-2’s morphological novelty is defined over exact token forms, with manual normalization checks finding examples that were absent, spaced, capitalized differently, or embedded within larger training words.The stricter search lowercased words and removed spaces, punctuation, and non-Roman characters before manually inspecting candidate matches.Source: O Morphology categorization · p. 29

O.1 Unanalyzed

• Among 838 examples lacking discernible internal structure, 658 were strings of random characters, while another 180 were pronounceable words without apparent meaning or internal structure.Source: §O.1 · p. 30

O.2 Typo

• GPT-2 generated 754 typographical errors, including 706 spacing or punctuation errors and 48 misspelled words, yet produced far fewer typos than the human baseline.The authors conjecture that subword-based generation creates fewer typo opportunities than humans’ character-by-character generation.Source: §O.2 · p. 30

O.3 Tech

• Technology-related terms, including URLs and computer-code words, were GPT-2’s most common novel-word category and occurred about twice as frequently as in the baseline.The category likely involves affixation, concatenation, and case manipulation, although the paper does not analyze its internal structure systematically.Source: §O.3 · p. 30

O.4 Number

• GPT-2 generated 672 real numbers, 49 possible dates, 75 number-unit combinations, and 190 mostly North American-format phone numbers, though four letter-coded phones had incorrect correspondences.Real numbers were generally well-formed except for one improperly leading zero; all novel dates were possible dates.Source: §O.4 · pp. 30–31

O.5 Acronym

• The generated text contained 195 novel acronyms, including 75 accompanied by the full expressions they represented.Examples included invented organizational names such as CDSRG, NCBSB, and NIDRS.Source: §O.5 · p. 31

O.6 Non-English

• GPT-2 produced words intended as many specified non-English languages, but the study did not assess their validity or attributed meanings; six pronunciation hints were only approximately correct.The listed languages ranged from Arabic and French to Japanese, Mandarin, Russian, and Spanish.Source: §O.6 · p. 31

O.7 Name

• The generated text contained 1,499 novel names spanning languages or dialects, usernames, personal names, groups, places, uncertain names, corporate names, and scientific species names.The study classified names by type but did not analyze their internal structure.Source: §O.7 · pp. 31–32

O.8.1 Inflectional morphology

• Novel inflection included regular noun plurals and possessives, phrasal adjective comparison with more or most, and -ed verbs used only as passives or past participles.Plural forms used mainly -s, with some -es; no other pluralization processes or suffixal adjective comparatives were observed.Source: §O.8.1 · pp. 33–34
• GPT-2 generates novel -ing forms, but no novel verbs with the third-person present-tense -s suffix.Source: §O.8.1 · p. 34

O.8.2 Lexical word-formation

• GPT-2 produces many compound-word types, including dephrasal compounds formed by joining entire phrases with hyphens.Many dephrasal compounds have a noun-preposition-noun structure, alongside compounds with other structures.Source: §O.8.2 · p. 34
• GPT-2 also generates coordinative compounds whose elements have equal status, including list-like compounds and compounds without an obvious ordering motivation.Source: §O.8.2 · p. 36
• The analysis also identifies neoclassical compounds, compound verbs, and two Digimon-franchise-specific morphemes, digi- and -mon.Neoclassical compounds can stack Greek- or Latin-based affixes, while the franchise-specific forms coin words related to Digimon.Source: §O.8.2 · pp. 42, 44
• The generated text contains derivational affixes expressing diminutive, augmentative, spatial or temporal, negative or reversative, positive, repetitive, and resident meanings.Observed examples include -ling, -lite, mini-, super-, ultra-, cross-, post-, pre-, proto-, sub-, trans-, anti-, de-, dis-, in-, no-, non-, un-, pro-, re-, -an, and -ite.Source: §O.8.2 · pp. 42–43
• Other derivational affixes form nouns, adjectives, verbs, and adverbs from stems in different parts of speech.The categorization includes verb- or adjective-to-noun, noun-to-noun, adjective-forming, verb-forming, and adjective-to-adverb affixes.Source: §O.8.2 · p. 43
• Character-level word manipulation includes repeated letters for emphasis or nervousness and onomatopoeias representing sounds.Examples include “Youuuuuuuuu,” interrupted speech such as “W-what are y-you m-meant-,” and sound sequences such as “tchtch,” “ka-a-la,” and “hwa-hwa.”Source: §O.8.2 · p. 44
• GPT-2 generated three possible portmanteau words, but these may be compounds formed from morphemes meaning “etiquette,” “apocalypse,” and “popsicle.”This interpretation is consistent with findings that language models perform poorly on portmanteau words.Source: §O.8.2 · p. 44

P.1 Plurals

• In an analyzed sample of 74 English plurals, the model correctly chose -s or -es for 72, while all but one generated possessives had the correct ’s or ’ form.Source: §P.1 · p. 44, §P.2 · p. 44

P.3 Acronyms

• 54 of 75 acronym expansions were unsuitable abbreviations, often because of extra, missing, reordered, or incorrect letters.Among 195 novel acronyms, 75 had an accompanying expansion, but only 21 were suitable abbreviations.Source: §P.3 · p. 45

P.4 Examples of incorrect morphology

• GPT-2 produced morphological errors through unwarranted stem changes and inconsistent suffixation, including implausible Pentagon adjectives and competing forms such as Hamiltonan and Hamiltonian.These inconsistencies concern how morphology is applied, rather than factual inconsistency; several variable forms were not counted as ill-formed.Source: §P.4 · p. 45
• Other errors involved missing phonological changes, such as genoshaans and “a-ounce,” while plural nouns in compounds were ambiguous because standard usage has exceptions.The compound examples include “guns-control,” “headswaters,” and “mushrooms-related”; the analysis says it is unclear whether such forms should count as errors.Source: §P.4 · p. 46
• GPT-2 also overregularized morphology by forming syllogist from syllogism and applying -th to 752 instead of the expected -nd.These errors reflect applying a linguistic process too broadly.Source: §P.4 · p. 46

P.5 Syntactic errors

• Most novel words fit their syntactic contexts, but GPT-2 produces errors involving category assignment, compounds, agreement, and malformed constructions.Examples include using bat-washer as a verb, plural cyber-missiles in a compound, look anti-Tunisian, and singular provides with plural load-samples.Source: §P.5 · p. 46

P.6 Agreement

• GPT-2 generally maintains plural agreement and related syntax, including plural verbs, possessives, coreferential pronouns, and determiners, even across three singular distractors.One exception is the singular verb provides used with plural load-samples.Source: §P.6 · p. 46, §P.5 · p. 46, §P.7 · p. 47

P.8 Incrementing/ordering

• GPT-2 successfully increments ordinal forms, variable-name letters, and mostly alphabetical company lists while generating novel words.The model extends Firstly through Fourteenthly, multiplyx through multiplyz, and a company list that is largely alphabetized.Source: §P.8 · p. 47

P.9 Quotation marks

• Novel words are more likely to appear in quotation marks in GPT-2 text, as in human-written text.Examples include “meganiverse,” “co-workplace,” “proto-poetry,” and “un-competition.”Source: §P.9 · p. 48

P.10 Novel words with meanings that are suitable for their context

• GPT-2 sometimes uses novel words in semantically well-suited contexts, including painter-scientist, re-nitrification, co-causation, dataaggregator, and co-workplace.These examples show novel forms whose inferred meanings align with their surrounding contexts.Source: §P.10 · p. 48

P.11 Novel words with meanings that are not suitable for their context

• GPT-2 also produces semantically unsuitable novel words involving derivational misinterpretation, category mismatches, contradictions, impossible spatial relations, and incorrect technical meanings.Examples include judgmentalizing, Brazilianism, disorderlessness, anti-catatonia, front-floor, nitrification, markdown output, IE10-based browsers, and no-knockout.Source: §P.11 · p. 48

P.12 Numbers

• GPT-2 generally fails to understand generated numbers, producing incorrect arithmetic, impossible or implausible quantities, inaccurate conversions, and inconsistent exchange rates.Examples include 1065mhz − 1030mhz = 90.5MHz, a 4ml tank holding 10.4ml, and four different KES-per-pound rates in one text.Source: §P.12 · p. 49

P.13 Phone numbers

• 54 of 77 phone-number cases (70%) match contextually specified North American locations, indicating valid learned associations between area codes and places.The examples include Seattle, New York City, Baltimore, and Pittsburgh area codes paired with matching locations.Source: §P.13 · p. 50
• 70% of generated phone-number cases matched their locations, compared with 98% in the baseline, with errors ranging from nearby to geographically distant area codes.Philadelphia examples used Pittsburgh’s 412 area code, while Colorado and Lubbock examples used New Hampshire’s 603 and Westchester County’s 914 codes.Source: §P.13 · p. 50

P.14 Generalization by composition

• GPT-2 generated 5 plurals and 3 other affixed forms of contextually introduced stems absent from training, best explained as composition rather than analogy.The account involves adding prefixes or suffixes directly to a novel stem from context without reference to another word sharing that affix.Source: §P.14 · pp. 50–51

P.15 Generalization by analogy

• GPT-2’s forms such as quackdom and tearro are better explained by analogy when training data supplies similar word families or token substitutions unavailable to ordinary morphology.For tearro, tornro, tearingros, and tearsros, GPT-2 appears to replace torero’s first four letters with forms from tear’s inflectional paradigm.Source: §P.15 · pp. 51–52

P.16 Full generation about toreros

• The full torero generation repeatedly produces malformed variants such as tearro and tornro, then shifts into incoherent descriptions of Tornros as a dog breed.The passage mixes bullfighting terminology with questions and descriptions of costumes, weapons, fighting, and dog ownership.Source: §P.16 · pp. 52–53

Q Overregularization and nonce CVC forms

• All 92 regular past-tense forms of selected irregular verbs appeared in WebText, and among 1339 nonce CVC words, 765 (57%) had both singular and plural forms in training.No overregularized forms appeared in the top-40 sample, but other decoding methods produced forms such as builded and drawed; dictionary absence therefore does not establish novelty.Source: Q Overregularization and nonce CVC forms · pp. 53–54