PaperlayerView exact revision on arXiv

Source-linked AI summary

How To Do Things With Prompts

Kristina Šekrst, Virna Karlić

arXiv:2609.29657v1cs.CL

TL;DR

The paper examines how the pragmatic features of ordinary users’ chatbot prompts have changed as conversational AI became familiar. The authors compare English-language conversation samples from 2023 and 2025, retaining whole conversations—including multiple turns—as the unit of analysis. Prompts shifted toward more indirect, implicit, fragmentary forms and less politeness marking, while users increasingly relied on the system to infer intended tasks from reduced input.

  • Problem

    The paper examines how the pragmatic features of ordinary users’ chatbot prompts have changed as conversational AI became familiar.

    Source: §1 · p. 1, §2.1.2 · p. 6

  • Method

    The authors compare English-language conversation samples from 2023 and 2025, retaining whole conversations—including multiple turns—as the unit of analysis.

    Source: §2.1.2 · p. 7

  • Results

    Prompts shifted toward more indirect, implicit, fragmentary forms and less politeness marking, while users increasingly relied on the system to infer intended tasks from reduced input.

    Source: §3 · p. 20

  • Takeaways & Limitations

    The findings suggest human-AI interaction is developing pragmatic conventions distinct from both traditional human-computer interaction and face-oriented interpersonal communication.

    Source: §3 · p. 20

    Because user populations differed across years, the trends cannot isolate individual adaptation and should not be generalized beyond English-language prompts from non-professional users.

    Source: §2.3 · p. 18

Abstract

from arXiv · show

When users address large language models, they produce directive speech acts whose pragmatic features differ from those of both everyday conversation and traditional human-computer interaction, and these features change as users gain familiarity with the systems they address. This paper applies speech act and politeness theory to a corpus-pragmatic analysis of 2,000 English-language prompts drawn from publicly shared ChatGPT conversations, 1,000 from 2023 and 1,000 from 2025, using the ShareChat dataset. Each prompt is annotated for illocutionary force, directness, propositional content, and the presence of politeness markers, and the distribution of these features is compared across the two sampling years. The results show a consistent movement toward indirect, implicit, and fragmentary realizations of directive force, accompanied by a decline in politeness marking. The largest single change, a shift of 14.9 percentage points, occurs in propositional content, where explicit specification of the requested action gives way to implicit reliance on the system's inferential capacity, suggesting that users have updated their model of what the system can recover from reduced input, treating it as a competent implicature resolver. Rather than asking whether LLMs "really" understand language, we should ask: what kind of language have we created in learning to speak to them?

1. Prompts as performative speech acts

The paper treats prompts as performative speech acts addressed to chatbots and uses speech-act and politeness theory to examine their pragmatic features. It frames changes in ordinary users’ prompting habits as a question about how people speak to increasingly familiar systems.

  • Human prompts generally act as performative utterances seeking a system response, most often as task-oriented or information-seeking directives, while some are phatic expressives.The paper treats these prompts as actions performed through language rather than as statements evaluated for truth.Source: §1 · pp. 2–3
  • Directive force concerns how binding a requested action is, while propositional content and force can each be expressed explicitly or implicitly, directly or indirectly.Politeness strategies such as “please,” interrogative phrasing, or implied content can mitigate directives in interpersonal communication.Source: §1 · pp. 3–4
  • Because directives can threaten face in interpersonal communication, politeness may mitigate them; chatbots lack relevant face needs, so such marking is not functionally necessary for task success.Users may nevertheless retain politeness through habit or preference.Source: §1 · pp. 4–5
  • The paper hypothesizes that prompt pragmatics have changed as users became accustomed to chatbots, examining ordinary users’ exchanges through speech-act and politeness theory.It distinguishes this general-audience focus from technical prompt engineering, which rewards detail and precision for demanding tasks.Source: §1 · pp. 1, 2, 5

2.1 Methodology

The study compares pragmatically annotated prompts from non-professional ChatGPT users in 2023 and 2025, treating the years as stages in the shift from novelty toward routine use. It draws on publicly shared conversations and combines model-assisted annotation with author review.

2.1 Methodology

  • The study sampled 1,000 prompts from each year to compare pragmatic features among non-professional ChatGPT users, defined as lacking prompt-engineering training or a technical stake in optimizing outputs.It tracks illocutionary force, directness, and politeness marking.Source: §2.1 · pp. 5–6, §2.1.2 · p. 7

2.1.1 Research questions and the hypothesis

  • The research design sets out questions and a hypothesis for the corpus analysis, with their details presented in Table 1.The two years were selected to represent an early and a later stage of public familiarity with ChatGPT.Source: §2.1.1 · p. 6, Table 1 · p. 6, §2.1.2 · p. 8

2.1.2 Corpus Compilation and Annotation

  • The study uses ShareChat’s publicly shared conversations, focusing on ChatGPT because its early reach among the general public offers a comparison across 2023 and 2025.The ChatGPT subset contains 102,740 conversations and 542,148 turns; public sharing reflects conversations users elected to make public.Source: §2.1.2 · pp. 6–7
  • The researchers extracted English-language conversations and retained whole conversations so prompts could be analyzed with their conversational context.Of the 1,000 sampled prompts per year, 462 in 2023 and 417 in 2025 were conversation-initial turns; the rest were continuations.Source: §2.1.2 · p. 7
  • After manually calibrating labels, the researchers used two language models for annotation and checked and corrected every assigned label against the criteria.The final annotations therefore reflect author review rather than model output alone.Source: §2.1.2 · p. 8

2.2 Results

Across 1,000 English-language prompts from non-professional users in shared ChatGPT conversations per year, major directive features shift between 2023 and 2025. The largest movement is from explicitly naming the requested action to leaving it implicit.

2.2 Results

  • The corpus contains 1,000 English-language prompts per year from non-professional users in shared ChatGPT conversations, with features quantified for both years.For propositional content, the annotation records whether a prompt names the action verb; information-seeking interrogatives are excluded from that distinction.Source: §2.2 · p. 9, §2.2.4 · p. 14

2.2.1 Directive and phatic prompts (Q1)

  • Phatic prompts remain marginal, falling from 2.3% to 1.6%; most prompts are task-oriented or information-seeking directives.Phatic turns commonly include greetings, thanks, sign-offs, or short reactions within exchanges that are otherwise directive.Source: Table 3 · p. 9, §2.2.1 · pp. 9–10

2.2.2 Direct and indirect directives (Q2)

  • Direct directives fall from 65.7% to 58.3%, while indirect realizations rise from 34.3% to 41.7%, a 7.4-point shift.The paper interprets this movement as greater reliance on the system to recover directive force from forms that do not state it explicitly, rather than as politeness mitigation.Source: Table 4 · p. 10, §2.2.2 · pp. 10–11

2.2.3 Subtypes of direct and indirect directives (Q3)

  • Fragmentary and unframed prompts rise from 23.8% to 40.0% of indirect directives, while task-seeking interrogatives fall from 24.8% to 12.4%.Among direct subtypes, imperatives fall from 34.1% to 28.0%, information-seeking interrogatives remain roughly one third of directives, and performative formulations are absent in both years.Source: §2.2.3 · pp. 12–14, Table 5 · p. 12, Table 6 · p. 13

2.2.4 Propositional content (Q4)

  • 14.9 percentage points: task-oriented prompts shift from majority explicit to majority implicit propositional content between 2023 and 2025.Explicit content falls from 62.9% to 48.0%, while implicit content rises from 37.1% to 52.0%. For conversation-initial prompts alone, explicit content falls from 67.6% to 55.4% and does not become a minority; continuation turns amplify the aggregate shift.Source: Table 7 · p. 14, §2.2.4 · pp. 14–15, Table 8 · p. 15

2.2.5 Politeness and interactional markers (Q5)

  • Politeness and interactional markers decline from 16.2% to 10.9%, yet remain in roughly one directive prompt in ten in 2025.The paper relates their persistence to interpersonal conventions users carry into chatbot exchanges, where the system has no face to threaten in the relevant sense.Source: Table 9 · p. 16, §2.2.5 · pp. 16–17

2.2.6 Hypothesis (H1)

  • Most major directive-level features change between years, while smaller or structurally marginal categories remain stable, supporting H1.The largest reorganization is how much users specify, rather than how they frame their contribution; changes in illocutionary form, fragmentation, and politeness overlap with that shift.Source: §2.2.6 · p. 17

2.3 Limitations and future work

The study’s findings are bounded by sample size, public sharing, differences in the user populations sampled, annotation procedures, and the English-language non-professional population. The authors propose comparisons across user registers and system prompts as future work.

Limitations and future work

  • With 1,000 prompts per year, the corpus reveals distributional patterns but has limited power for smaller categories, so percentages indicate direction and relative magnitude rather than precise population estimates.Indirect subtypes and phatic prompts are especially sensitive to differences of a few items.Source: §2.3 · p. 17
  • Because users chose which conversations to share publicly, the corpus may reflect notable or successful exchanges rather than everyday private use.The authors say this selection occurs before their own sampling.Source: §2.3 · p. 18
  • The 2023–2025 comparison cannot distinguish individual adaptation from changes in who uses ChatGPT, so its trends describe aggregate public prompts, not behavior change by each user.The user population expanded substantially between sampling years.Source: §2.3 · p. 18
  • Annotation reliability rests on explicit criteria and consistent application; the authors did not calculate a formal inter-annotator agreement statistic.Calibration disagreements were resolved through discussion.Source: §2.3 · p. 18
  • The findings concern English prompts from non-professional users and should not be generalized to other languages or professional users, whose practices may differ.The authors note that politeness conventions vary across languages and that professional users appear to move in the opposite direction.Source: §2.3 · p. 18
  • Technical and professional users’ prompting practices may move toward more explicit, literal, and longer inputs, contrasting with the trends reported for non-professional users.Such prompts may include detailed instructions, role specifications, formatting requirements, and worked examples.Source: §2.3 · p. 19
  • Future work could compare user populations under the same annotation schema and examine system prompts, which are authored to shape model behavior across an interaction.The authors also propose cross-linguistic extension of the schema.Source: §2.3 · p. 19

3. Conclusion

Across the sampled non-professional users’ prompts, directive forms shifted toward indirectness, implicit content, fragments, and fewer politeness markers. The authors interpret this pattern as evidence of emerging pragmatic conventions specific to human–AI interaction, while bounding their conclusions to the studied population and noting that other prompting registers may differ.

3. Conclusion

  • Across coded dimensions, non-professional users’ prompts became more indirect, implicit, fragmentary, and less marked for politeness.Propositional content shifted from majority explicit to majority implicit, the largest change reported; politeness markers also declined.Source: §3 · p. 20
  • Directive prompts predominated in both years, while phatic prompts remained marginal; direct directives still outnumbered indirect ones even as indirect forms rose.Within direct directives, imperatives declined, while information-seeking interrogatives held steady and performative formulations remained absent; among indirect forms, assertives remained most frequent, while task-seeking interrogatives fell and reduced forms grew.Source: §3 · p. 20
  • The authors interpret the convergence as users relying more on the system to infer intended tasks from reduced input and as human–AI interaction developing its own pragmatic conventions.They distinguish these conventions from command-based traditional HCI and face-oriented interpersonal communication.Source: §3 · p. 20
  • The conclusions apply to the studied population and are limited by public sharing and the inability to separate user adaptation from population change.The contrast with technical users suggests that human–AI interaction may develop multiple registers, a question left for future work.Source: §3 · p. 20

Appendix A

Appendix A provides an annotation prompt for classifying user-generated ChatGPT prompts by pragmatic features. It specifies direct and indirect forms, phatic function, propositional content, and politeness markers, with conditional not-applicable labels and structured output instructions.

Appendix A

  • The annotation instructions request JSON output, and describe using Python 3.14 with Claude Opus 4.7 and GPT 5.5 to process prompts, retrying failed or flagged batches.Additional context was included when prompts triggered model guardrails through the API.Source: Sample annotating prompt · pp. 23, 25, 26
  • Prompts serving primarily interpersonal or interactional functions are coded as phatic; other prompts are coded non-phatic.The prompt gives greetings and thanks as examples of phatic prompts.Source: Sample annotating prompt · p. 23
  • Direct task directives are coded by form: imperatives and performative verbs are separate categories, while information-seeking questions request factual, explanatory, or interpretive responses.Task-seeking questions that indirectly request performance are excluded from the information-seeking-question category.Source: Sample annotating prompt · pp. 23–24
  • Some labels are conditional: indirect subtypes are not applicable to phatic prompts or prompts without indirectness, and the prompt asks analysts to annotate numbered user data objectively.The annotation instruction says the examples are data for analysis, not instructions to follow.Source: Sample annotating prompt · pp. 23–24
  • Indirect task requests include assertive forms and question forms, each distinguished from prompts that do not use that form.Examples include “I need an essay” for an assertive and “Can you write an essay?” for an indirect question.Source: Sample annotating prompt · pp. 24–25
  • An indirect incomplete-sentence prompt uses a fragment to request a task, whereas a raw-input prompt consists only of textual or symbolic material without an indirect request form.Examples include “Essay about spring” and raw song lyrics or source code.Source: Sample annotating prompt · p. 25
  • Propositional content is labeled explicit when the requested action is named and implicit when it is implied; politeness coding records whether markers such as “please,” greetings, or thanks occur.The prompt marks the politeness category not applicable for phatic prompts.Source: Sample annotating prompt · p. 25

Appendix B

Appendix B describes preprocessed general and technical prompt corpora made available by the authors. The technical corpus was assembled from archived Reddit data in prompt-engineering-related communities.

Appendix B

Loading 2609.29657v1…