PaperlayerView exact revision on arXiv

Source-linked AI summary

Why is AI-generated language so hard to understand?

Thomas Hikaru Clark, Matias B. Martin, Kyle Mahowald

arXiv:2610.29162v1cs.CLcs.AI

TL;DR

The paper investigates whether recent LLM text contains features associated with human comprehension difficulty and whether those features are systematically overrepresented. It compares generated text with human writing and round-trip paraphrases, and tests an information-density account; it finds support for both linguistic overrepresentation and a shift away from human-like output.

  • Problem

    Whether AI-generated text is systematically harder for humans to process because it overuses linguistic features associated with comprehension difficulty remains underexplored.

    Source: §1 · p. 2

  • Method

    The study analyzes a diverse corpus of LLM text against human references and round-trip paraphrases, then tests a compression account through length-penalty fine-tuning.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    The authors report support for both hypotheses: LLMs overuse features associated with comprehension difficulty, and information-density pressure shifts output away from human-like language.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Task objectives are not guaranteed to align with human processing pressures, which may help explain technically grammatical but out-of-distribution constructions associated with comprehension difficulty.

    Source: §6 · p. 9

    The study covers a subset of models, English only, and a limited number of conversation seeds, and relies on proxies because large-scale causal study of post-training is infeasible.

    Source: §7 · p. 9

Abstract

from arXiv · show

There is a widespread sense that AI-generated text has recently become harder to understand. In this study, we investigate a set of psycholinguistic features that are known to correlate with human processing difficulty (e.g., object relative clauses, long syntactic dependencies, low-frequency n-grams). Using a novel corpus of AI-generated text across diverse domains, we find that AI models use these difficult features more often than paraphrases generated by round-trip translation. Compared with human writing, all nine frontier and near-frontier models use more inanimate subjects with animate verbs and more low-frequency n-grams, and most also use longer dependencies, more stacked nouns, and more reduced object relatives in conversation; we also demonstrate substantial variation across model families. We propose and test a hypothesis as to why: while human language is optimized for successful communication, AI-generated text is increasingly optimized for other goals like math, code, and reasoning, in which there is a pressure for \textit{information density} rather than successful communication. We provide a proof-of-concept for this hypothesis by fine-tuning a language model under a length penalty, which leads to a divergence from human-like linguistic behavior. Our findings highlight the axes of misalignment between linguistic processing demands in humans and LLMs, and point toward possible remedies.

1 Introduction

The paper asks why AI-generated language may be hard to understand and whether difficult linguistic features are systematically overrepresented. It compares model text with human writing and round-trip paraphrases, then tests whether pressure for information density may contribute to divergence from human-like language.

1 Introduction

  • A diverse corpus shows difficult linguistic features are overrepresented in recent LLM text, often exceeding human references and appearing less often after round-trip paraphrase.The round-trip comparison holds semantic content fixed while changing wording, helping distinguish models’ characteristic syntactic profiles from semantics.Source: §1 · p. 2
  • The paper frames its question against the view that human language balances ease of production with ease of understanding, a profile that may not persist when communication is not the primary objective.The motivating example illustrates object relatives, denominal verbs, inanimate subjects with animate verbs, stacked nouns, and low-frequency n-grams as features associated with processing difficulty.Source: §1 · pp. 1–2
  • A length-penalty fine-tuning test suggests that compression pressure can favor shorter syntactic alternations, including alternations not seen during training.The proposed account links optimization for tasks such as math and coding, alongside context-window limits, with information compression that competes with intelligibility.Source: §1 · p. 2
  • The authors report support for both hypotheses: LLMs overuse features associated with comprehension difficulty, and information-density pressure shifts output away from human-like language.They identify dimensions of misalignment between human language processing and frontier LLMs and point to possible interventions.Source: §1 · p. 2

2 Background and Related Work

Related work documents distinctive LLM writing patterns and psycholinguistic sources of comprehension difficulty. It also contrasts human processing pressures with LLM capabilities and distinguishes uniform information density from a preference for denser language.

2 Background and Related Work

  • Prior work has examined LLM word overuse and noun-heavy style, and has considered whether post-training contributes to such idiosyncrasies (Juzek and Ward, 2025; Reinhart et al., 2025; Ouyang et al., 2022; Singhal et al., 2024; Lambert et al., 2024; Guo et al., 2025).LLM writing has also motivated text detectors and questions about linguistic content, diversity, and representation (Bao et al., 2024; Agarwal et al., 2025; Ryan et al., 2024).Source: §2 · p. 2
  • Psycholinguistic research links comprehension difficulty to hard-to-predict words, long-distance dependencies, center embedding, legalese, and temporary ambiguity (Hale, 2001; Levy, 2008; Gibson, 1998, 2000; Gibson and Thomas, 1999; Lewis, 1996; Martínez et al., 2022; Christianson et al., 2001).Source: §2 · p. 2
  • LLMs’ larger training experience and ability to attend to long contexts contrast with human memory constraints; prior findings associate artificial memory constraints with better fit to human behavior (Vaswani et al., 2017; Oh and Linzen, 2026; Oh et al., 2024; Kuribayashi et al., 2022; Clark et al., 2025).The cited work suggests that full attention to long contexts may come at a cost to human-likeness.Source: §2 · pp. 2–3
  • Uniform information density proposes spreading information evenly to avoid overloading comprehenders, whereas maximizing information density would categorically favor shorter alternations (Levy and Jaeger, 2007; Meister et al., 2021; Li and Xu, 2026).The background also notes cognitive availability during incremental production as a proposed pressure on language generation.Source: §2 · p. 3

3 Experiment 1: Linguistic features in AI-generated conversations

Experiment 1 tests whether AI-generated text uses processing-difficulty-linked features across chat, coding, and essays. It compares model outputs with genre-matched human references and meaning-preserving round-trip paraphrases, finding broad overuse with variation across models and settings.

3.1 Linguistic features of interest

  • Experiment 1 tests whether models disproportionately use processing-difficulty-linked features, including rare n-grams, long dependencies, object relatives, stacked noun compounds, and denominal verbs.The measures include unattested bigrams and trigram perplexity, standardized dependency length, relative-clause rates, stacked noun compounds, and denominal-verb rates.Source: §3.1 · pp. 3–4
  • The study also counts inanimate subjects paired with active verbs that typically take animate subjects, a construction that may express meaning more compactly.The authors note that avoiding such subjects can require a longer paraphrase.Source: §3.1 · p. 4

3.2 Data collection

  • The corpus covers 14 models, 15 chat and coding seeds, and essays responding to seven PERSUADE 2.0 assignments.Chat sessions were capped at 12 turns and coding sessions at 11 turns plus a closing message.Source: §3.2 · p. 4

3.3 Comparison to human baselines

  • Human baselines use approximately genre-matched OpenSubtitles for chat, StackExchange for coding, and PERSUADE 2.0 student essays for essays; Figure 3 also contrasts references.Wikipedia is included as an additional comparison.Source: §3.3 · p. 4, Figure 3 · p. 5

3.4 Round-trip translation

  • Round-trip translation sends each sentence from English to Japanese and back with NLLB-200, holding semantic content fixed while changing its wording.The authors use the paired texts to identify features the model chose over available alternatives expressing the same content.Source: §3.4 · p. 5

3.5 Results: AI models overuse constructions with known processing costs

  • Across most frontier and near-frontier models, round-trip comparisons show overuse of rare n-grams, denominal verbs, inanimate subjects, and reduced object relatives; Llama and Olmo base/instruct models show weaker effects.Relative to human text, most models also over-represent inanimate subjects and low-frequency n-grams; changes are smaller in PERSUADE essays, where the authors speculate that fewer iterative turns may matter.Source: §3.5 · pp. 5, 7, Figure 4 · p. 6

4 Experiment 2: Effect of length penalties on language preferences

The experiment tests whether reinforcement-learning length penalties shift language preferences toward shorter syntactic variants, including constructions not seen during training. The penalty increased preference for shorter variants, with transfer depending on the setting and KL regularization.

4 Experiment 2: Effect of length penalties on language preferences

  • Experiment 2 uses a length-penalty intervention to test whether pressure for information density shifts output toward difficult, less humanlike linguistic features.It examines a possible mechanism for information-density pressure in language models.Source: §4 · p. 7

4.1 Setup

  • The setup tests whether a length penalty induces categorical preference for shorter alternatives across trained, near-transfer, and far-transfer constructions.Near transfer uses unseen optional-that constructions; far transfer uses unrelated alternations.Source: §4.1 · pp. 7–8

4.2 Results: A length penalty shifts models toward shorter variants, including in untrained constructions

  • With any λ > 0, all seeds without KL regularization converged on the short variant; this preference also transferred to near-transfer constructions.Far transfer appeared only without KL, in the genitive alternation among four tested.Source: §4.2 · p. 8

4.3 Discussion

  • The model was not instructed to compress or told about the alternations, yet the penalty produced an emergent preference for shorter variants.Without KL regularization, short variants exceeded 98% for both trained “that” alternations at every λ > 0; human speakers choose by context, while the models still succeeded because both variants were valid solutions.Source: §4.3 · pp. 8–9

5 General Discussion

The discussion links AI language preferences to training objectives that may prioritize verifiable solutions over clarity, while human language is shaped by comprehension constraints. These differing pressures may yield language that is useful for model tasks but less intelligible to people.

Model Development

  • Verifiable-reward training may favor compression more plausibly than human-feedback training, which has been found to reward longer outputs.The authors speculate that removing human feedback may permit language optimized for dense information compression and complex problem solving.Source: §5 · p. 9
  • The authors note that effective communication can also be rewarded, and that a newer model variant quantifiably improves over an older one.They contrast Opus 5.5 with Opus 5.Source: §5 · p. 9

Implications for psycholinguistics

  • Human language use is shaped by comprehension limits, while AI deployment in coding and mathematical problem-solving creates different pressures on language.The authors identify potential opportunities to study how communication systems emerge under these different pressures.Source: §5 · p. 9

6 Conclusion

The conclusion argues that frontier-model training objectives are not guaranteed to align with the pressures shaping human language. It suggests that outputs may be grammatical yet difficult for people to process, and that future work could better align them with human comprehenders.

6 Conclusion

  • Frontier-model objectives for coding and mathematical reasoning may not align with human language pressures, potentially contributing to grammatical but comprehension-difficult constructions.The authors suggest future work could align outputs better with human comprehenders for user experience and monitorability.Source: §6 · p. 9

7 Limitations

The study is limited to a subset of models, English analyses, and few conversation seeds, and relies on proxies to investigate causal sources of linguistic difficulty.

7 Limitations

  • The study examines only some commonly used models and English, uses few chat-topic and code-repository seeds, and relies on proxies because direct study of post-training at scale is infeasible.The authors recommend replication with more seeds and model families; limited compute also constrained post-training study.Source: §7 · p. 9

A Data collection protocol details

The study collects model text across chat, coding, and essay settings using a shared protocol, then processes the outputs and compares model and human corpora. The protocol specifies each setting’s prompts, limits, and sample construction.

A Data collection protocol details

  • The corpus spans chat, coding, and essays, with shared collection procedures and setting-specific details.The protocol covers the common harness, each setting, text processing, and collection costs.Source: A Data collection protocol details · p. 13

A.1 Common infrastructure

  • Fourteen models were run through a common agent harness where possible, with provider-default generation settings for non-local models.The two base models used text-completion endpoints; local open models ran on an Apple M4 Max, and failed runs were retried.Source: §A.1 · pp. 13–14

A.2 Chat

  • Chat collection covered ten topics, with conversations ending after 12 replies or 20,000 output tokens, whichever came first.Olmo Think used its built-in default system prompt because the harness’s prompt was discarded by its chat template.Source: §A.2 · p. 14

A.3 Code

  • Nine frontier models each completed five repository-based coding sessions using Pi’s default tools and system prompt.Sessions ended after 11 episodes or when episode-final text reached an estimated 8,000 tokens; analysis used turn-final replies.Source: §A.3 · pp. 14–15

A.4 Essay

  • The human essay comparison uses 2,934 high-scoring independent essays from seven PERSUADE 2.0 assignments, while models produced 980 essays.Model essays were not presented with student essays or instructed to write as students.Source: §A.4 · p. 15

A.5 Processing

  • Processing removes markup and code-related material, replaces selected spans with placeholders, and applies shared cleaning steps to student and model essays.Sentence and n-gram measures do not cross paragraph or structural boundaries; essay spelling is not corrected.Source: §A.5 · pp. 15–16

A.6 Cost and timeline

  • The dataset runs cost $211.78, and the full collection, discarded attempts, and pilot and verification runs totaled $251.19.Frontier-model collection took place in September 2026; smaller open-weight models were collected in October.Source: §A.6 · p. 16

B.1 Feature detection

The study operationalizes linguistic features using dependency parses and classifier judgments, compares outputs with genre-matched human references, and standardizes dependency length across sentence-length distributions. It also specifies how figures and round-trip translation checks are interpreted.

B.1 Feature detection

  • Dependency parses identify object relatives, stacked nouns, and dependency lengths; an external classifier judges candidate denominal verbs and inanimate subjects.Candidates are counted when the classifier assigns probability above 0.5.Source: §B.1 · p. 16

B.2 Human baselines and n-gram references

  • N-gram statistics use trigram models built from roughly 500,000-word samples of OpenSubtitles, StackExchange, and Wikipedia.Chat, coding, and essay outputs are scored against the corresponding reference corpora, with unattested pairs reported per 1,000 pairs.Source: §B.2 · p. 16

B.3 Dependency length standardization

  • Dependency length is standardized by sentence-length bins, weighting bin means according to Wikipedia’s distribution of dependency arcs.Sentences over 30 words are excluded, and punctuation arcs and the root are not counted.Source: §B.3 · pp. 16, 18

B.4 Details of Figures 2, 5 and 6

  • Figure 5 diamonds show pooled model values with 95% bootstrap intervals; Figure 6 distinguishes original text from round trips and marks unadjusted paired tests.Figure 2 averages standardized original-to-round-trip differences across measure pairs and applicable settings.Source: §B.4 · p. 18

B.5 Round-trip translation with other translators and pivots

  • The round-trip robustness check compares Russian and Japanese pivots using NLLB and TranslateGemma for the nine frontier models.Each sentence is translated separately, and returned multi-sentence translations are analyzed as such.Source: §B.5 · p. 18

C Reinforcement learning details

This section specifies a KL-regularized reward objective and a GRPO training procedure for encouraging shorter completions while preserving task reward. It also describes the reward checks, sampling setup, and alternations used to train and evaluate the models.

C Reinforcement learning details

  • The objective maximizes expected reward with a KL regularizer relative to a frozen base model.For tied task rewards, the stated optimum compares short and long variants; with β = 0, that optimum does not exist and the shift is limited by training length.Source: C Reinforcement learning details · p. 18
  • The reward combines correctness and format checks with a penalty proportional to completion length.Correctness requires event argument phrases verbatim in their roles and the event verb’s lemma; the format check requires one sentence with specified punctuation and no preamble, quotation marks, or colon.Source: C Reinforcement learning details · p. 18
  • Each GRPO step samples 16 completions for each of eight prompts, scores them, and uses a group-mean baseline without standard-deviation scaling.Sampling uses temperature 1.0 and a limit of 64 new tokens; the same advantage is assigned to every token in a completion.Source: C Reinforcement learning details · pp. 18–19
  • The evaluation stimuli include four training alternations and six held-out alternations, with one dative prompt item varying only its Event line.The held-out alternations are used only to evaluate trained models.Source: Table 3 · p. 19, Figure 8 · p. 19
  • The DR-GRPO loss sums per-token clipped surrogate terms and divides by a constant batch-size and maximum-length product, without per-sequence length normalization.The batch contains 128 completions and the maximum completion length is 64 tokens; with one optimization pass, the loss reduces to a sequence-level policy gradient plus the KL term.Source: C Reinforcement learning details · p. 19

D Additional Tables and Plots

The additional materials report transfer results and describe training configurations, evaluation stimuli, and two sets of plots. The reported far-transfer shifts are limited to the genitive alternation under a specific regularization setting.

D Additional Tables and Plots

  • The 24-run training grid varied λ, β, and seed while holding the remaining configuration identical.Runs took approximately 7.7–8.6 seconds per training step and the full grid used approximately 31 GPU-hours.Source: Table 4 · p. 20, D Additional Tables and Plots · p. 20
  • Only the genitive among four far-transfer alternations shifted with λ, and only without KL regularization.Passive and purpose-clause alternations did not shift with λ; the quantifier alternation was already near ceiling in the base model.Source: D Additional Tables and Plots · p. 20
  • Open-ended responses to 40 prompts unrelated to training were scored for n-gram statistics against Wikipedia.Figure 9 shows individual seeds for the base model and 24 fine-tuned models, with the base model marked by a dashed line.Source: D Additional Tables and Plots · p. 20, Figure 9 · p. 22
  • The stimulus table documents ten chat topics and five public Python repositories used in the chat and coding settings.It includes repository LOC, stars at selection, offline test status, and pinned commit identifiers.Source: Table 5 · p. 21
  • Figure 10 reports mean words per response for original and round-trip text across the nine frontier models.It excludes cases where round-trip length is below half or above twice the source length and shows 95% cluster-bootstrap intervals.Source: Figure 10 · p. 23
Loading 2610.29162v1…