PaperlayerView exact revision on arXiv

Source-linked AI summary

The Deontic Gap: Large Language Models and the Modal Language of Obligation

Daniel Hart, Sarah Allred, Joseph Abbas, Morenike Alugo

arXiv:2608.18144v1cs.CLcs.AI

TL;DR

Because deontic modals encode obligation and interpersonal stance, whether LLMs reproduce contemporary human patterns of their use remains an open question. The paper compares AI- and human-authored text across corpora and genres, finding that AI consistently uses positive deontic modals less often, with the gap concentrated in interpersonal-obligation constructions.

  • Problem

    Whether LLMs reproduce contemporary human use of deontic modals—which calibrate obligation, authority, and speaker-listener relationships—remains an open question.

    Source: §1 · p. 1

  • Method

    The study compares human and AI text across corpora and genres, including prompt-matched samples, and calibrates modal rates against the Google Books Ngram record.

    Source: §2.1 · p. 5, §3.5 · p. 13

  • Results

    7.6 positive modals per 10,000 words for AI versus 25.7 for humans in WritingPrompts; AI rates were lower in all three primary corpora and the ASAP-AES comparison, with gaps concentrated in interpersonal-obligation constructions.

    Source: §3.1 · p. 9, §3.6 · p. 15

  • Takeaways & Limitations

    The pattern is consistent with LLM modal usage reflecting formal-register exposure while comparatively underusing constructions associated with immediate interpersonal obligation.

    Source: §3.6 · p. 15, §4.5 · p. 24

    The contemporary corpora span ten or more genres but do not cover spoken language, instant messaging, or legal and medical professional writing.

    Source: §4.4 · p. 23

Abstract

from arXiv · show

Modal auxiliaries such as must, should, and have to mark necessity and obligation within the contexts of speaker authority and interpersonal stance. We examine whether large language models (LLMs) reproduce contemporary human patterns of deontic modal usage. Across three primary corpora, an external benchmark, two controlled replications, and a naturalistic eleven-model replication, AI-generated text consistently underuses positive deontic modals (must, should, have to, had to) relative to contemporary humans. Historical comparison with the Google Books Ngram corpus (1920-2022), used as a heuristic calibration against the published-prose record, shows that AI modal frequencies fall within the range of formal published English, whereas contemporary human modal rates in informal digital contexts often exceed twentieth-century book baselines. Phrase-level decomposition shows that the AI-human modal gap is concentrated in constructions central to interpersonal stance (should, have to, had to), while AI matches or exceeds humans on need to in instructional and question-answering contexts but not in persuasive student writing, indicating that the modal profile is genre-conditional. The findings suggest that LLM modal usage reflects the formal written resources on which these models were trained, while underusing the modal constructions through which contemporary human writers mark immediate, interpersonal obligation.

1. Introduction

The study finds that AI-generated text consistently underuses positive deontic modals relative to contemporary human writing, with the gap concentrated in interpersonal constructions rather than spread evenly across the modal system. It asks how these differences relate to pragmatic stance and the formal written registers used to calibrate AI modal frequencies.

1. Introduction

  • Deontic modals encode obligation while signaling speaker authority and interpersonal footing, so their variation changes the normative signal readers receive (Knobe & Szabo, 2013; Phillips & Kratzer, 2024).Experimental work found that injunctive formulations using should produce stronger shifts in norm perception and compliance than descriptive formulations of the same content (Kuang and Bicchieri, 2024).Source: §1 · pp. 1–2
  • AI-generated text underuses positive deontic modals in every comparison, with gaps concentrated in should, have to, had to, and conversational can’t, while AI matches or exceeds humans on must, need to, and cannot.The authors characterize this as a register-and-stance asymmetry, not an overall poverty of normative language.Source: §1 · pp. 2–3
  • Prior work distinguishes LLM and human styles (Reinhart et al., 2025; Liang et al., 2024), but modal verbs remain underexamined as discourse-pragmatic markers (Georgiou, 2026).The study extends research on metadiscourse and pragmatic cues by focusing on modal constructions as resources for marking obligation and interpersonal force.Source: §1 · p. 3, §1.2 · p. 4
  • The study examines positive and negative deontic modals and impossibility expressions across three primary corpora, an external benchmark, controlled replications, and a naturalistic eleven-model replication.Google Books Ngram provides a century-plus published-prose baseline for calibrating contemporary AI and human frequencies.Source: §1 · p. 3

1.1. Deontic Modality as Pragmatic Stance

  • The forms differ pragmatically: should commonly expresses recommendation, have to and had to often frame practical constraints, and must and cannot tend toward formal or institutional registers.Need to is frequent in task-oriented instructional contexts, whereas can’t often sounds conversational and immediate; modal counts therefore reflect register and stance, not interchangeable grammatical items.Source: §1.1 · pp. 3–4

1.2. Large Language Models, Register, and Pragmatic Style

  • LLM training mixtures draw heavily on books, web text, and other structured material while underrepresenting everyday informal registers, motivating a prediction of closer alignment with formal prose than contemporary informal obligation patterns.The register-asymmetry account predicts that models may approximate formal modals such as must and cannot while underusing colloquial forms such as have to, had to, and can’t.Source: §1.2 · p. 4

2. Method

The method compares AI- and human-authored text across contemporary genres, historical book corpora, and controlled and naturalistic replications. It operationalizes targeted modal phrase sets and combines pooled-corpus comparisons with document-level robustness and specificity analyses.

2.1. Datasets

  • The study pairs AI and human text across narrative, question-answer, fiction, and student-writing corpora, alongside a multi-model advice benchmark and an external detection benchmark.WritingPrompts and ASAP-AES use 150 prompt-matched examples each; ELEPHANT provides responses from eleven models, while the Pangram sample contains 1,925 documents. The ELEPHANT eligibility threshold retained 2,692 human responses and 55,070 model responses, but excluded many shorter human responses.Source: §2.1 · pp. 5–6

2.2. Historical Comparison: Google Books Ngram Corpus

  • AI rates fall within or below the century-long Google Books range, while contemporary human rates meet or exceed its upper edge across the genre-matched comparisons.The study retrieved 1920–2022 frequencies from general American English and published-fiction corpora, converting them to rates per 10,000 words for era matching.Source: §2.2 · p. 6, Figure 3 · p. 7

2.3. Modal Phrase Lists

  • Modal phrase lists define positive and negative deontic constructions and impossibility terms, with counts treated as modal-resource distributions rather than token-by-token deontic interpretations.Rates use whole-word, case-insensitive matching per 10,000 words; impossibility terms are exploratory, and the phrase-level analysis distinguishes constructions with different pragmatic uses.Source: §2.3 · pp. 7–8

2.4. Analytic Approach

  • Analyses pool text by corpus, compare phrase-level AI–human gaps with historical change, and test prompt-matched and out-of-sample differences at the document level.Paired prompt bootstraps, Wilcoxon tests, and mixed models assess WritingPrompts and ASAP-AES; Pangram uses rank-sum tests, and positive modals fall at the 3.7th percentile of common-word frequency ratios. The small-construction historical correlations are descriptive, not inferential.Source: §2.4 · pp. 8–9

3. Results · 3.1. Overall Differences in the Pragmatic Marking of Obligation

Across the four corpus comparisons, AI-generated text used positive and negative deontic modals less often than human writing, with the largest positive-modal gap in student essays. The positive-modal shortfall was concentrated in constructions of obligation, while impossibility terms showed no consistent pattern.

3.1. Overall Differences in the Pragmatic Marking of Obligation

  • AI used positive deontic modals at lower rates than humans in all four corpus comparisons.Rates per 10,000 words were 9.9 vs. 13.5 for gsingh, 21.8 vs. 29.2 for HC3, 7.6 vs. 25.7 for WritingPrompts, and 25.3 vs. 52.9 for ASAP-AES.Source: §3.1 · p. 9, Table 1 · p. 9
  • 27.5 per 10,000 words was the largest positive-modal gap, between high school student writers and GPT-4o in ASAP-AES.The WritingPrompts fiction gap was 18.1 per 10,000 words, the second largest.Source: §3.1 · p. 9
  • Humans used negative deontic forms more often than AI in all four corpora.Human vs. AI rates per 10,000 words were 4.7 vs. 2.0 in gsingh, 10.0 vs. 3.5 in HC3, 13.4 vs. 8.0 in WritingPrompts, and 18.7 vs. 7.3 in ASAP-AES.Source: §3.1 · p. 9
  • The negative-modal gap was concentrated in contracted forms such as can’t and shouldn’t.The formal uncontracted variants cannot and should not modestly favored AI.Source: §3.1 · p. 9
  • Impossibility terms occurred at low rates for both AI and humans and showed no consistent pattern across datasets.Source: §3.1 · p. 9
  • The reported positive-modal comparisons span three primary corpora and the ASAP-AES controlled replication.The table presents modal rates per 10,000 words across primary corpora and controlled replication samples.Source: §3.1 · p. 9, Table 1 · p. 9

3.2. Robustness of the Paired-Design Gaps Below the Corpus Level

Prompt-matched analyses show that positive deontic-modal gaps persist within documents across both corpora, though the gaps are concentrated in a right tail rather than uniform across pairs.

3.2. Robustness of the Paired-Design Gaps Below the Corpus Level

  • In both corpora, right-skewed rates show that the gap is carried by modal-dense documents rather than a uniform per-document shift.Many short documents on both sides contain no positive modal at all, while human rates exceed AI rates substantially in the upper quartile.Source: §3.2 · p. 10, Figure 1 · p. 11
  • Across paired documents, mean positive-modal rates were 65.0 (human) versus 30.7 (GPT-4o) per 10,000 words in ASAP-AES, and 26.8 versus 7.8 in WritingPrompts.The mean paired differences were 34.1 and 19.0 per 10,000 words, respectively; mixed models estimated GPT-4o rates at 0.48 and 0.30 times human rates.Source: §3.2 · p. 10, Figure 1 · p. 11
  • Human rates exceeded AI rates in 57.0% of ASAP-AES pairs and 54.7% of WritingPrompts pairs; many remaining pairs were tied, often at zero.In WritingPrompts, most GPT-4o stories contained no positive deontic modal, while their human counterparts varied widely and fell above the equal-use line in most pairs.Source: §3.2 · p. 10, Figure 1 · p. 11

3.3. Out-of-Sample Replication Across Domains

On the independent Pangram benchmark, human-authored documents used more positive deontic modals than AI-authored documents, with the pattern holding across domains. However, modal frequency separated individual human and AI documents only weakly, indicating an aggregate difference rather than a reliable marker of authorship.

3.3. Out-of-Sample Replication Across Domains

  • 23.6 human versus 14.1 AI positive modals per 10,000 words on Pangram.The difference was significant (Wilcoxon rank-sum test: W = 393,842, p < .001).Source: §3.3 · p. 12
  • Human mean rates exceeded AI rates in all ten Pangram domains, and human document-level rates ranked higher in nine of ten.The exception was encyclopedic text, where the human and AI distributions overlapped; the pattern also held in creative writing, question-and-answer text, and news.Source: Figure 2 · p. 12, §3.3 · p. 12
  • Positive-modal rate separated individual human and AI documents only weakly, so it cannot reliably classify authorship from a single document.The replication establishes an aggregate difference in obligation marking, not a document-level marker of machine authorship.Source: Figure 2 · p. 12

3.4. Controlled Replication on Student Essays

In a controlled, prompt-matched essay replication, GPT-4o used positive modals at about half the rate of high school students. The gap extended to negative modals and varied by genre for “need to.”

3.4. Controlled Replication on Student Essays

  • High school students used 52.9 positive modals per 10,000 words, versus 25.3 for GPT-4o in paired essay responses.The corpus-level gap had a 95% CI of [19.0, 36.8] per 10,000 words; the Poisson GLMM rate ratio was 0.48 [0.40, 0.57].Source: §3.4 · p. 13
  • Negative modals showed the same direction: students used 18.7 per 10,000 words, versus 7.3 for AI.The gap was concentrated in contracted forms such as can’t, couldn’t, and shouldn’t; AI matched or slightly exceeded human rates on formal uncontracted equivalents.Source: §3.4 · p. 13
  • In persuasive essays, AI substantially underused “need to”: 0.3 per 10,000 words, versus 7.7 for students.This contrasts with AI’s overuse of “need to” in instructional and question-answering text and near-human rates in narrative nonfiction; the passage characterizes its use as register-specific.Source: §3.4 · p. 13

3.5. Historical Calibration Against Published-Prose Register

AI positive-deontic-modal rates fell within the historical published-prose range, while contemporary human writers often exceeded book baselines. The gap varied by genre, narrowing in question-answering text.

3.5. Historical Calibration Against Published-Prose Register

  • In WritingPrompts, humans used 25.7 positive modals per 10,000 words versus 7.6 for GPT-4o, an 18.1-per-10,000-word gap.The rates fell near 2011 and 1979, respectively, on the Google Books scale; the passage attributes the contrast to register rather than era.Source: §3.5 · p. 13
  • In the general corpus, gsingh AI text averaged 9.9 positive modals per 10,000 words, while ChatGPT in HC3 averaged 21.8.The narrower HC3 gap likely reflects its informational, question-answering genre, where LLM instructional style is more congruent with expert writing.Source: §3.5 · p. 13
  • Historical year matches are heuristic coordinates, not claims that AI writes like a specific decade; Google Books records edited published text, unlike informal human writing.Contemporary human modal frequencies exceed historical book baselines, whereas AI modal density falls within the published-book range; the gap tracks genre distance from LLM training's formal register.Source: §3.5 · pp. 13–14

3.6. Phrase-Level Differences in Deontic Force and Stance

Phrase-level comparisons show that the AI–human positive-modal gap centers on interpersonal-obligation forms, while need to and must favor AI in the gsingh and HC3 datasets. Negative-modal differences likewise center on contractions characteristic of informal registers.

3.6. Phrase-Level Differences in Deontic Force and Stance

  • The AI advantage on need to did not generalize to persuasive student writing, where AI used it at 0.3 versus 7.7 per 10,000 words for humans.This ASAP-AES replication contrasts with the need to pattern in gsingh and HC3.Source: §3.6 · p. 14
  • Humans used have to, should, had to, and has to more than AI, while AI matched or exceeded humans on need to and must.Averaged across gsingh and HC3, human vs. AI rates per 10,000 words were: have to, 5.3 vs. 1.7 (gap: -3.6); should, 6.1 vs. 4.6 (gap: -1.5); had to, 1.4 vs. 0.3 (gap: -1.1); has to, 1.5 vs. 0.5 (gap: -1.0); need to, 4.4 vs. 5.3 (gap: +0.9); must, 2.5 vs. 3.4 (gap: +0.9).Source: Figure 4 · p. 14, §3.6 · p. 14
  • Contracted negative modals drove the human advantage, whereas formal uncontracted equivalents modestly favored AI.Averaged across gsingh and HC3, human vs. AI rates per 10,000 words were: can’t, 4.9 vs. 0.9 (gap: -4.0); couldn’t, 0.9 vs. 0.1 (gap: -0.8); shouldn’t, 0.5 vs. 0.04 (gap: -0.5); cannot, 0.7 vs. 1.2 (gap: +0.5); should not, 0.3 vs. 0.5 (gap: +0.2).Source: §3.6 · p. 15
  • The authors interpret the positive-modal gap as concentrated in forms tied to immediate interpersonal obligation, not modal language as such.They associate human use with recommendation, personal constraint, and narrative compulsion, and AI use with formal or procedural necessity; contractions drive nearly the entire negative-modal gap, while uncontracted forms are typical of formal and institutional prose.Source: §3.6 · p. 15

3.7. Illustrative Contrasts in Modal Stance

A prompt-matched story pair illustrates that both texts express necessity but frame it differently: the human writer conveys urgent interpersonal compulsion, while GPT-4o uses more procedural, measured modal language.

3.7. Illustrative Contrasts in Modal Stance

  • In the matched story, the human writer uses “I have to” and “can’t” to frame obligation as personally compelled, while GPT-4o’s “need to” frames it as procedural management.GPT-4o’s “couldn’t help but smile” marks involuntary affect rather than obligation.Source: §3.7 · p. 15

3.8. Historical Change and the AI Modal Deficit

Historical change in published-prose modal frequency does not explain most positive-modal AI–human gaps, although it aligns with negative-modal gaps and with the patterns for need to and must.

Historical Change and the AI Modal Deficit

  • The correlations are descriptive, convergent evidence rather than confirmatory tests, based on few constructions and a coarse 1960–2019 comparison.The analysis covers k = 7 positive and k = 9 negative constructions without inferential corrections; several positive modals rose only in recent decades rather than monotonically.Source: §3.8 · p. 16
  • For negative modals, historical change correlated positively with AI–human gaps (r = .26), with rising constructions showing near-zero gaps and declining or stable ones larger AI shortfalls.This qualitative ordering is consistent with register fixation, though the correlation is modest.Source: Figure 5 · p. 16
  • Positive-modal historical change correlated weakly negatively with AI–human gaps (r = -.31), while AI underused have to, had to, should, has to, and ought to despite varied book trajectories.Need to fits the register prediction after a 620% rise in Google Books since 1960, and must fits weakly with a roughly flat trajectory and modest AI advantage.Source: §3.8 · p. 17

3.9. Generality Across Model Families

Independently developed models reproduce the human–AI deontic gap: Claude and all eleven ELEPHANT models use positive deontic modals less often than humans. The gap is concentrated in deontic constructions rather than reflecting a generic difference in common-word frequencies.

3.9. Generality Across Model Families

  • Claude used 9.7 positive deontic modals per 10,000 words versus humans’ 25.7; all eleven ELEPHANT models also fell below humans (13.0–33.5; rate ratios 0.23–0.66).In the controlled WritingPrompts comparison, Claude’s rate was close to GPT-4o’s 7.6; the model rate-ratio confidence intervals overlapped substantially. ELEPHANT responses came from their original authors and covered advice and moral-judgment settings.Source: §3.9 · p. 17, Figure 6 · p. 17, Figure 7 · p. 19
  • Claude and GPT-4o underused the same constructions—must, had to, should, have to, and need to—relative to human writers.The paired human–Claude difference was 17.1 per 10,000 words (95% CI [10.9, 21.3]); the Poisson mixed model estimated Claude at 0.38 times the human rate (95% CI [0.28, 0.51]).Source: §3.9 · p. 17
  • The repeated gap across independently developed models is consistent with models carrying formal written-text modal norms into creative writing, rather than with a single developer’s pipeline artifact.The ELEPHANT findings likewise make a developer-specific explanation difficult to reconcile with the pattern.Source: §3.9 · p. 17, Figure 6 · p. 17
  • The seven positive deontic modals fell at the 3.7th percentile of GPT-4o-to-human frequency ratios for 300 common words, marking unusually strong underuse.This comparison indicates that the gap is specific to deontic constructions rather than a generic common-vocabulary frequency difference.Source: Figure 7 · p. 19, §3.9 · p. 20

4. Discussion

Across corpora and models, LLMs underuse interpersonal forms of obligation common in contemporary informal writing, a pattern consistent with fixation on formal and procedural registers. Whether this modal gap affects readers or future human language remains unresolved.

4. Discussion

  • The gap recurs across corpora and models; prompt-matched comparisons rule out document length and sampled genre as explanations, while the gap itself varies by genre.The findings fit a register account, but alignment training may also contribute; the relative roles of training-register composition and alignment remain a question for future research.Source: §4 · pp. 20–21, §4.1 · p. 22

4.1. Register Asymmetry and Pragmatic Force

  • In WritingPrompts, humans used positive modals at 25.7 per 10,000 words versus GPT-4o’s 7.6, an 18.1-per-10,000-word gap interpreted as register difference, not temporal displacement.AI modal density falls within the range of formal published-book prose, while contemporary informal fiction exceeds the twentieth-century book record; matched years are only approximate coordinates.Source: §4.1 · p. 21
  • 8.66 vs. 6.35 per 10,000 words (HC3), AI uses need to more than humans; in persuasive essays, it uses less (ASAP-AES: 0.3 vs. 7.7).AI also uses must more than humans across primary corpora (3.4 vs. 2.5 per 10,000 words, averaged across gsingh and HC3), consistent with its prevalence in formal, regulatory, and definitional registers.Source: §4.1 · p. 22

4.3. Implications for Pragmatic Interpretation and Future Research

  • Avoidance of should, have to, and had to may make AI obligations seem less commanding or urgent, but the study does not test readers’ normative uptake.Prior work links modal marking to normative judgments and uptake (Park et al., 2025; Kuang and Bicchieri, 2024; Brady et al., 2017; Capraro & Vanzo, 2019); whether those effects generalize to the observed AI–human frequency differences needs experimental study.Source: §4.3 · p. 23

4.4. Limitations

  • The evidence spans multiple contemporary genres but does not establish that the pattern generalizes to all contexts or future model generations.Limitations include targeted modal lists and unresolved usage ambiguities, Google Books era matching as a rough calibration, and robustness analyses restricted to prompt-matched datasets. The large essay gap comes from timed 8th- and 10th-grade student writing; the Pangram benchmark’s labels were produced by its developers, and its detector accuracy is not independently validated.Source: §4.4 · pp. 23–24

4.5. Conclusion

  • The study establishes a patterned deontic gap and opens questions about whether it affects readers’ interpretation of AI guidance or enters future human speech.The authors identify the possible spread of LLM-favored lexical patterns into human speech as an open question for longitudinal pragmatics.Source: §4.5 · pp. 24–25

Supplementary Material

The supplementary material documents additional analyses of deontic-modal rates, classification performance, historical trends, and human–AI distributions. It also specifies how the ELEPHANT advice-corpus rates and adjusted ratios were calculated.

Supplementary Material

  • Table S1 reports positive deontic-modal rates and length-adjusted model-to-human rate ratios for human writers and eleven language models in the ELEPHANT advice corpus.Rates count occurrences per 10,000 words among responses of at least 50 words; ratios come from model-specific Poisson mixed models with word-count offsets and prompt-level random intercepts. Ratios below 1.00 indicate less frequent positive-obligation marking than among humans; the AI responses were generated by the dataset’s original authors without matched decoding conditions.Source: Table S1 · p. 28, Supplementary Material · p. 28
  • Table S2 presents single-feature classification performance using positive-modal rate on the Pangram benchmark, broken down by domain.Source: Supplementary Material · p. 29, Table S2 · p. 29
  • Google Books trajectories show need to rising steadily, have to, had to, and has to rising only in recent decades, and should and must remaining essentially flat.Because these paths are nonmonotonic, the 1960-to-2019 difference used in Figure 5 is a descriptive summary rather than a precise estimate.Source: Figure S1 · p. 29
  • In WritingPrompts and ASAP-AES, the human–AI modal-rate gap comes from the human distribution’s right tail, while AI texts more often contain no positive deontic modal.Figure S2 shows documents of at least 50 words; the displayed horizontal axis is truncated at 150 per 10,000 words, though higher-rate documents remain in the statistics.Source: Figure S2 · p. 30

Related papers

Loading 2608.18144v1…